Surround view system

Abstract

A surround view system (20) for a vehicle (10) includes: one or more cameras (220) mounted on the vehicle (10) and configured to capture one or more images of an ambient environment of the vehicle (10); a driver monitoring unit (240) configured to determine a viewing direction of a driver (30) of the vehicle (10); and a plurality of holographic projection units (260) arranged at different positions inside the vehicle (10), where each of the plurality of holographic projection units (260) is configured to generate a holographic image on a different one of a plurality of surfaces inside the vehicle (10), and the surround view system (20) is configured to cause a holographic image to be generated on at least one surface inside the vehicle (10) within the viewing direction determined by the driver monitoring unit (240), wherein the holographic image is generated based on the one or more images captured by the one or more cameras (220).

Description

Technical Field

[0001] The present disclosure relates to a surround view system, and more particularly to a surround view system for a vehicle. Background Art

[0002] A surround view system in a vehicle captures images of the vehicle's surrounding environment and provides a surround view of the surrounding environment on a display of the vehicle (e.g., a display arranged in the center of the vehicle's dashboard). Such surround view systems may not always present a perspective of the surrounding environment that is suitable for a particular situation to assist a driver in optimally evaluating a situation. It may be necessary for the driver to interact with the display (e.g., via a touch panel) to change the perspective to a suitable one. However, such interaction may distract the driver and may cause the driver to not fully focus on the current situation. As a result, driver safety may be reduced and the risk of accidents may increase. There is a need for a surround view system and related method that can present images of the surrounding environment to the driver of a vehicle without causing unnecessary driver distraction to improve road safety. Summary of the Invention

[0003] A surround view system of the present disclosure for use in a vehicle includes: one or more cameras mounted on the vehicle and configured to capture one or more images of the vehicle's surrounding environment; a driver monitoring unit configured to determine the viewing direction of the driver of the vehicle; and a plurality of holographic projection units arranged at different positions inside the vehicle, wherein each holographic projection unit of the plurality of holographic projection units is configured to generate a holographic image on a different one of a plurality of surfaces inside the vehicle, and the surround view system is configured to cause the holographic image to be generated on at least one surface inside the vehicle in the viewing direction determined by the driver monitoring unit, wherein the holographic image is generated based on the one or more images captured by the one or more cameras.

[0004] A method of the present disclosure includes: capturing one or more images of the vehicle's surrounding environment by one or more cameras mounted on the vehicle; determining the viewing direction of the driver of the vehicle by a driver monitoring unit; and generating a holographic image on a different one of a plurality of surfaces inside the vehicle by a plurality of holographic projection units arranged at different positions inside the vehicle, wherein the holographic image is generated on at least one surface inside the vehicle in the viewing direction determined by the driver monitoring unit, wherein the holographic image is generated based on the one or more images captured by the one or more cameras.

[0005] After reviewing the following detailed description and the accompanying drawings, other systems, methods, features, and advantages of the present disclosure will be apparent or will become apparent to those skilled in the art. It is intended that all such additional systems, methods, features, and advantages be included within the scope of this specification, within the scope of the present invention, and be protected by the appended claims. Description of the Drawings

[0006] The apparatus can be better understood with reference to the following description and the accompanying drawings. The components in the drawings are not necessarily drawn to scale, but rather emphasis is placed on illustrating the principles of the present invention. Additionally, in the drawings, the same reference numerals refer to corresponding parts in different views.

[0007] Figure 1 Schematically shows a vehicle having a surround view system.

[0008] Figure 2 Schematically shows a vehicle having a surround view system according to an embodiment of the present disclosure.

[0009] Figure 3 Schematically shows a vehicle having a surround view system according to another embodiment of the present disclosure.

[0010] Figure 4 Schematically shows a holographic projection unit disposed in a vehicle.

[0011] Figure 5 Schematically shows a holographic display unit and an image projected on the windshield of a vehicle.

[0012] Figure 6 Schematically further shows in detail a holographic display unit according to an embodiment of the present disclosure.

[0013] Figure 7 Schematically shows a method according to an embodiment of the present disclosure.

[0014] Figure 8 Schematically shows a flowchart of a method according to an embodiment of the present disclosure. Detailed Description

[0015] As required, detailed embodiments of the present invention are disclosed herein; however, it should be understood that the disclosed embodiments are merely examples of the present invention that can be embodied in various alternative forms. The drawings are not necessarily drawn to scale; some features may be enlarged or minimized to show details of particular parts. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but rather as a representative basis for teaching those skilled in the art to practice the present invention in different ways.

[0016] It will be appreciated that directional terms (e.g., "up", "down", "in", "out", "top", "bottom", etc.) that may be referred to herein merely refer to the orientation of the various components of the device as shown in the accompanying drawings. These terms are provided for context and understanding of the disclosed embodiments.

[0017] A surround view system and related methods according to various embodiments described herein are capable of providing a 360° surround view of the surrounding environment to a driver of a vehicle. The surround view system provides a perspective of the surrounding environment suitable for most situations to assist the driver in evaluating what is going on outside the vehicle. There is no driver intervention to provide the best perspective for any particular situation. At the same time, the surround view system does not consume excessive energy.

[0018] A surround view system and related methods according to various embodiments described herein capture images of the surrounding environment of the vehicle. The captured images are processed and then presented to the driver of the vehicle by projecting the images onto corresponding surfaces of the vehicle. The viewing direction of the driver is determined, and the images are (only) projected onto such surfaces that are within the determined viewing direction. If the viewing direction of the driver changes, images of the surrounding environment regions within the changed field of view of the driver are presented on other surfaces of the vehicle, i.e., on the surfaces that are within the changed field of view. That is, it is not necessary to always present the entire surrounding environment to the driver. Only those surrounding environment regions that are determined to be relevant to the driver may be presented to the driver, where the surrounding environment regions are determined to be relevant to the driver when they are within the driver's current field of view.

[0019] A surround view system according to an embodiment of the present disclosure is configured to cause a holographic image to be generated on one or more surfaces inside the vehicle in the viewing direction (field of view) of the driver determined by the driver monitoring unit, where the holographic image is generated based on one or more images captured by one or more cameras. For example, if it is determined that the driver is looking out of the left front side window, a holographic image representing the surrounding environment in this viewing direction is projected on the left front side window. If it is detected that the driver turns their head to look out of the right front side window, a holographic image representing the surrounding environment in this viewing direction is projected on the right front side window. That is, the driver is always presented with a 3D holographic image of a part of the surrounding environment in the direction they are looking. In this way, any object or obstacle in the driver's viewing direction will be presented to the driver. The driver does not need to view a separate dedicated display, for example, in the vehicle's dashboard. Therefore, the driver does not need to mentally transform the image displayed on the central display unit according to the specific situation. Any object or obstacle originally hidden behind vehicle components (such as doors, pillars, or hoods) can be visualized to the driver. The image presented to the user can represent a larger portion of the surrounding environment, which would originally be visible to the driver through the corresponding window.

[0020] Reference Figure 1 , schematically shows a vehicle 10 having a surround view system 20. The surround view system 20 includes one or more cameras mounted on the vehicle and configured to capture one or more images of the surrounding environment of the vehicle; a driver monitoring unit configured to determine the viewing direction of the driver of the vehicle; and a plurality of holographic projection units arranged at different positions within the vehicle 10. Figure 2 The different elements of the surround view system 20 are not specifically shown in. According to an embodiment of the present disclosure, the surround view system 20 may include a single 360° surround view camera, which is, for example, arranged in the center of the car roof and configured to capture a 360° image of the surrounding environment of the vehicle 10.

[0021] According to other embodiments of the present disclosure and as Figure 2 and Figure 3 schematically shows, for example, the surround view system 20 may include a plurality of cameras 220, where each camera 220 is arranged at a different position on the vehicle 10, for example, on the outside of the vehicle. One or more of the cameras 220 are outward-facing cameras. In the embodiment shown in Figure 2 , for example, a camera 220 is arranged at each corner of the vehicle 10. In Figure 3In the illustrated embodiment, one camera 220 is arranged at the center of the front of the vehicle 10, one camera 220 is arranged on or in the first side view mirror on the right side of the vehicle 10, one camera 220 is arranged on or in the second side view mirror on the left side of the vehicle 10, one camera 220 is arranged at the left rear corner of the vehicle 220, and one camera 220 is arranged at the right rear corner. Each camera 220 is configured to capture an image within a defined viewing angle. This viewing angle can be, for example, between 30° and 180°. However, angles even greater than 180° are also possible. One or more of the plurality of cameras 220 can be, for example, a spherical camera configured to capture a 360° image. In this way, images of the entire surrounding environment can be captured by one or more cameras 220. However, Figure 2 and Figure 3 the positions of the cameras 220 schematically shown in

[0022] are merely examples. In general, one or more cameras 220 can be arranged at any suitable position.

[0023] One or more of the cameras 220 can be static. That is, each camera 220 can be configured to capture a defined portion of the surrounding environment. However, it is also possible that one or more of the one or more cameras 220 can be pivotable cameras such that different portions of the surrounding environment can be captured depending on the orientation of the camera 220.

[0024] The driver monitoring unit 240 is configured to determine the viewing direction of the driver of the vehicle 10. That is, the driver monitoring unit 240 determines whether the driver is looking out through the windshield of the vehicle 10, through the side window, through the rear window, or in any other direction looking forward. The driver monitoring unit 240 can include one or more cameras (e.g., inward-facing cameras), and can determine the viewing direction of the driver, for example, by face recognition or eye tracking technology. The driver monitoring unit 240 can be arranged, for example, at or near the steering wheel of the vehicle 10, as Figure 2 and Figure 3Shown schematically. However, the driver monitoring unit 240 can be arranged in any other suitable location in the vehicle 10. It is also possible that the driver monitoring unit 240 includes a plurality of inward-facing cameras arranged at different locations inside the vehicle 10. In this way, there is always at least one camera capturing the driver's face so that the driver's viewing direction can be clearly determined. However, the driver's viewing direction can also be determined in any other suitable way.

[0025] The surround view system 20 also includes a plurality of holographic projection units 260 arranged at different locations within the vehicle 10. Each of the plurality of holographic projection units 260 is configured to generate a holographic image on a different one of a plurality of surfaces inside the vehicle 10. In Figure 2 the illustrated embodiment, for example, for each of the windshield, the rear window, and each of the front side windows and the rear side windows of the vehicle 10, the surround view system 20 includes a different holographic projection unit 260. That is, each of the holographic projection units 260 is arranged at a different location and projects a holographic image on a different one of the windows of the vehicle 10. However, as Figure 2 shown, the location of the holographic projection unit 260 is merely an example.

[0026] Figure 3 Shown schematically are the holographic projection units 260 arranged at different locations. In Figure 3In the illustrated embodiment, one holographic projection unit 260 projects a holographic image on the windshield, another holographic projection unit 260 projects a holographic image on the rear window, one holographic projection unit 260 projects a holographic image on both the left front side window and the left rear side window, and even additional holographic projection units project holographic images on the right front side window and the right rear side window. Even additional holographic projection units 260 are arranged to project holographic images on the left and right side windows located behind the rear seats in the trunk area of the vehicle 10. Any other number of holographic projection units 260 and any other suitable locations are generally possible. The vehicle windows are not completely transparent and are thus suitable to act as projection screens for holographic images. However, the windows may also be coated with, for example, a corresponding reflective foil. The light emitted by the holographic projection unit 260 is partially reflected by the corresponding window (or surface). The reflected light is perceived by the driver of the vehicle. Similar to a conventional mirror, due to the reflection, the image is perceived as a three-dimensional image located behind the reflective surface. The holographic image may also be projected at least partially on other surfaces outside the windows, such as, for example, on the pillars of the vehicle located between different windows. Such surfaces may be covered with a reflective material, such as a reflective foil, on which the image may be projected. In this way, a 360° panoramic view of the surrounding environment can be presented to the driver without any interruption.

[0027] The driver of the vehicle can observe the holographic image projected on the rear window by looking directly in the direction of the rear window. However, it is also possible that the image is projected on the rear window while the driver is looking at the rearview mirror. The image projected on the rear window is reflected in the rearview mirror and is thus also visible to the driver in the rearview mirror.

[0028] A 360° panoramic surround view can generally be presented to the driver. That is, images of the surrounding environment can be projected simultaneously on all the vehicle windows and optionally on any additional surfaces, such as between the windows. However, this requires a significant amount of power. To keep the power consumption to a minimum, the images may only be presented on those windows and optionally on other surfaces within the viewing direction of the driver determined by the driver monitoring unit 240.

[0029] Similarly, all the cameras in one or more cameras 220 can be kept active simultaneously to capture images of the entire surrounding environment. However, it is also possible to only activate such cameras that are positioned to capture images of the part of the surrounding environment located within the field of view of the driver determined by the driver monitoring unit 240. This can further reduce the power consumption of the surround view system.

[0030] Figure 4Schematically shows a holographic projection unit 260 according to an embodiment of the present disclosure. The holographic projection unit 260 is arranged below the windshield 120 and is configured to project a holographic image onto the windshield 120. The windshield acts as a reflective hologram surface RHS which, when illuminated by a light source, reflects a 3D hologram. The hologram reflected by the RHS is visible within a virtual free programmable distance outside the vehicle. Figure 4 This is schematically shown with respect to two objects 40, 42. The distance of the RHS perceived by the driver of the vehicle 10 from the objects 40, 42 can be adjusted, for example, by suitable software. Different objects can be perceived at different distances relative to the RHS, as Figure 4 is schematically shown with respect to the objects 40, 42. The objects 40, 42 are perceived by the driver 30 as three-dimensional. That is, if the driver 30 moves his head and views the objects 40, 42 from different perspectives, they are perceived by him at different angles, just like in the case of real objects. The objects 40, 42 can be perceived as flat objects, or the objects 40, 42 can have a defined virtual thickness. The eyes of the driver 30 can record two different images, each of the two images representing one of the two objects 40, 42. However, the brain fuses these two images into a single image. Thus, the driver 30 perceives a single image in which any object existing outside the vehicle 10 is presented at a corresponding distance relative to the vehicle 10 and relative to each other.

[0031] Figure 5 Schematically shows the image projected onto the windshield 120 of the vehicle 10 and perceived by the driver 30. The image is a three-dimensional image representing the surroundings of the vehicle in the viewing direction of the driver 30 (outside the windshield in the Figure 5 example). In this way, any object within the viewing direction of the driver 30 is visible to the driver. This includes objects that the driver can see through the corresponding window without projection (e.g., houses and trees), and any object that might otherwise be hidden behind elements of the vehicle 10 (e.g., behind a door, pillar, or hood) (e.g., a tricycle).

[0032] Now refer to Figure 6, more specifically showing the holographic projection unit 260 according to an embodiment of the present disclosure. The holographic projection unit 260 may include, for example, a light source 262, an adjustable diffraction unit 264, a control unit 266, and a data processing unit 268. The light source 262 may be a brightness controllable light source. The brightness of the projection may be adjusted based on the environmental conditions. For example, there may be two different designs, such as a night design and a day design. During the day when the inside and outside of the vehicle are bright, the day design may be used. The holographic projection unit 260 may be configured to change from the day design to the night design when it is detected that the brightness of the outside and / or inside of the vehicle is lower than a defined threshold. However, one or more intermediate stages in which the projection is dimmed to different degrees may also be implemented.

[0033] The diffraction unit 264 deflects the light from the light source 262 onto a reflective hologram surface RHS (e.g., the window of the vehicle 10, or any other suitable surface of the vehicle 10) and into the visual range of the driver 30. Specifically, when illuminated by the light source 262 via the diffraction unit 264, the RHS reflects a three-dimensional hologram in the desired direction (e.g., in the direction of the driver 30 of the vehicle 10) within the visual range. The driver 30 perceives a fringe pattern caused by the pattern of the diffraction unit 264. The fringe pattern corresponds to the reconstructed hologram and the projected elements 40, 42.

[0034] According to an embodiment of the present disclosure, the diffraction unit 264 includes a plurality of phase delay elements, where each phase delay element in the plurality of phase delay elements delays the phase of the reflected or transmitted light by a defined amount. This defined amount of phase delay may be controlled individually for each phase delay element by the control unit 266. Any hologram that imprints a specific phase pattern onto the light of the light source 262 for generating a reconstructed hologram may be implemented in this way or in a similar way. The control unit 266 is configured to provide different control patterns via the diffraction unit 264 for generating different holograms. A plurality of different control patterns may be generated or stored in the data processing unit 268. Based on the image to be displayed by the holographic projection unit 260, the data processing unit 268 may select one or more pre-computed control patterns for the corresponding hologram. However, as Figure 6 shown and as already described above, the holographic projection unit 260 is merely an example. The holographic projection unit 260 may be implemented in any other suitable way so as to generate a three-dimensional scene beyond the two-dimensional RHS.

[0035] The surround view system 20 may remain active after the ignition switch has been turned on until the ignition switch is turned off again. However, it is also possible that the surround view system 20 remains active only under defined circumstances. For example, the surround view system 20 may be activated when the driver 30 performs a parking maneuver, when it is determined that the vehicle 10 is driving in a narrow and / or complex surrounding, or when the vehicle 10 is driving at a speed below or above a defined threshold speed. It is also possible that the surround view system 20 is activated in any other situation in which the projection of an image of the surrounding is considered to increase driver safety.

[0036] Now referring to Figure 7 , a method according to an embodiment of the present disclosure is schematically illustrated. In a first stage, one or more images may be captured by one or more cameras 220, wherein at least a portion of the surroundings of the vehicle 10 is captured in each of the one or more images (step 70). The one or more cameras 220 may be spherical cameras, and the one or more images captured by the one or more cameras may represent a stereoscopic view of the surroundings of the vehicle 10. In a subsequent step, a depth map may be generated based on the stereoscopic view represented by the one or more images (step 72). Based on this depth map, a computer-generated hologram CGH may be generated (step 74), and the CGH may be projected onto a corresponding surface within the vehicle 10 (step 76). Generally, a three-dimensional depth map may be generated from two-dimensional input images, for example by stereo image rectification. This allows the estimation of the distances of different objects and / or road users relative to each other and relative to the vehicle 10. Such techniques are generally known and will not be discussed further herein. Then, the determined three-dimensional depth map may be visualized by the computer-generated hologram CGH. For example, a binary metasurface computer-generated hologram CGH may be used to separate multicolor holographic images in the far field without using any lenses. This technique enables lensless, ultra-miniature augmented and virtual reality displays.

[0037] Now referring to Figure 8 , a method according to an embodiment of the present disclosure is schematically illustrated. The method includes: capturing one or more images of the surroundings of the vehicle 10 by one or more cameras 220 mounted on the vehicle 10 (step 801); determining the viewing direction of the driver 30 of the vehicle 10 by a driver monitoring unit 240 (step 802); and generating a holographic image on a different one of a plurality of surfaces within the vehicle 10 by a plurality of holographic projection units 260 arranged at different positions within the vehicle 10 (step 803), wherein the holographic image is generated on at least one surface within the vehicle 10 in the viewing direction determined by the driver monitoring unit 240, and wherein the holographic image is generated based on the one or more images captured by the one or more cameras 220.

[0038] The description of the embodiments has been presented for purposes of illustration and description. Appropriate modifications and variations can be effected in light of the above description or can be acquired by practice of the methods. The described apparatus is exemplary in nature and can include additional elements and / or omit elements. As used in this application, an element recited in the singular and preceded by the word "a" or "an" should not be construed as excluding a plurality of such elements unless such exclusion is stated. Additionally, a reference to "one embodiment" or "an example" of the present disclosure is not intended to be construed as excluding the existence of additional embodiments that also incorporate the recited features. The terms "first," "second," and "third," etc. are used merely as labels and are not intended to impose numerical requirements or a particular positional order on their objects. The described system is exemplary in nature and can include additional elements and / or omit elements. The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various systems and configurations and other features, functions, and / or properties disclosed. The appended claims particularly disclose the subject matter regarded as novel and nonobvious from the foregoing description.

Claims

1. A surround view system (20) for a vehicle (10), comprising: One or more cameras (220) mounted on the vehicle (10) and configured to capture one or more images of the surroundings of the vehicle (10); A driver monitoring unit (240) configured to determine the viewing direction of the driver (30) of the vehicle (10); And A plurality of holographic projection units (260) arranged at different positions inside the vehicle (10), wherein Each holographic projection unit of the plurality of holographic projection units (260) is configured to generate a holographic image on a different one of a plurality of surfaces inside the vehicle (10), and The surround view system (20) is configured to cause the holographic image to be generated on at least one surface inside the vehicle (10) in the viewing direction determined by the driver monitoring unit (240), wherein the holographic image is generated based on the one or more images captured by the one or more cameras (220).

2. The surround view system (20) according to claim 1, wherein the holographic image generated on at least one surface inside the vehicle (10) represents a part of the surrounding environment in the viewing direction of the driver (30).

3. The surround view system (20) according to claim 1 or 2, wherein the at least one surface includes one or more of a windshield, a left front side window, a right front side window, a left rear side window, a right rear side window, and a rear window.

4. The surround view system (20) according to claim 3, wherein the at least one surface further includes one or more pillars of the vehicle.

5. The surround view system (20) according to claim 4, wherein the pillars are covered with a reflective material.

6. The surround view system (20) according to any one of the preceding claims, wherein each of the one or more cameras (220) has a viewing angle between 30° and 180° or greater than 180°.

7. The surround view system (20) according to any one of the preceding claims, wherein at least one of the one or more cameras (220) is a spherical camera configured to capture 360° images.

8. The surround view system (20) according to any one of the preceding claims, wherein the driver monitoring unit (240) includes one or more inward-facing cameras.

9. The surround view system (20) according to any one of the preceding claims, wherein the driver monitoring unit (240) is configured to determine the viewing direction of the driver (30) by face recognition or eye tracking technology.

10. The surround view system (20) according to any one of the preceding claims, wherein each of the plurality of holographic projection units (260) includes a light source (262), an adjustable diffraction unit (264), a control unit (266), and a data processing unit (268), wherein: The light source (262) is configured to irradiate the adjustable diffraction unit (264), The adjustable diffraction unit (264) deflects light from the light source (262) onto the at least one surface and into the visual range of the driver (30). The control unit (266) is configured to provide different control patterns via the diffraction unit (264) for generating holograms, and The data processing unit (268) is configured to generate or store a plurality of different control patterns, and to select one or more pre - calculated control patterns for a corresponding hologram based on the image to be projected by the holographic projection unit (260).

11. The surround - view system (20) according to claim 10, wherein the light source (262) is a brightness - controllable light source.

12. The surround - view system (20) according to claim 10 or 11, wherein the diffraction unit (264) includes a plurality of phase - delay elements, and each phase - delay element of the plurality of phase - delay elements delays the phase of the reflected or transmitted light by a defined amount.

13. The surround - view system (20) according to claim 12, wherein the defined amount of phase delay is controlled individually for each phase - delay element by the control unit (266).

14. A method, comprising capturing one or more images of the surroundings of the vehicle (10) by one or more cameras (220) mounted on the vehicle (10); determining the viewing direction of the driver (30) of the vehicle (10) by a driver monitoring unit (240); and generating a holographic image on a different one of a plurality of surfaces inside the vehicle (10) by a plurality of holographic projection units (260) arranged at different positions inside the vehicle (10), wherein the holographic image is generated on at least one surface inside the vehicle (10) in the viewing direction determined by the driver monitoring unit (240), and the holographic image is generated based on the one or more images captured by the one or more cameras (220).

Images