Ground line monitoring system

Abstract

A ground line monitoring system includes a camera mounted in a first motor vehicle. A waveguide directs light to the camera, the waveguide having a first in-coupling grating that receives first light imaging data and passes the first light imaging data as light of a first frequency and a second in-coupling grating that receives second light imaging data and passes the second light imaging data as light of a second frequency. A color filter wheel receives the light of the first frequency and the light of the second frequency. Due to rotation of the color filter wheel, an image sensor of the camera receives the light of the first frequency and the light of the second frequency at different times. A controller performs calculations using directions and angles of the light of the first frequency and the light of the second frequency to correct the camera image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to motor vehicle head-up display systems and vehicle ground line affecting image display on a head-up display. BACKGROUND

[0002] In a motor vehicle having a head-up display, the vehicle can have multiple ground lines. Each ground line results in a different eye height and vehicle pitch. In order to align a virtual image with reality on an augmented reality head-up display (ARHUD), information related to the eye height and vehicle pitch is needed. However, the vehicle pitch can vary, especially in trucks, including pickup trucks, where the ground clearance can vary. In addition, the vehicle pitch can also vary depending on the load condition and load distribution of the truck. The difference in vehicle pitch in turn creates some problems, for example, image clarity and accuracy of the virtual image produced on the ARHUD.

[0003] Therefore, while the current vehicle head-up display systems achieve their intended purpose, there is still a need for a new and improved ground line monitoring system that presents data to a head-up display of a motor vehicle. SUMMARY

[0004] According to aspects, a ground line monitoring system includes a camera installed in a first motor vehicle. A waveguide directs light to the camera, the waveguide having a first in-coupling grating that receives first light imaging data and passes the first light imaging data as light of a first frequency and a second in-coupling grating that receives second light imaging data and passes the second light imaging data as light of a second frequency. A color filter wheel receives the light of the first frequency and the light of the second frequency. Due to rotation of the color filter wheel, an image sensor of the camera receives the light of the first frequency and the light of the second frequency at different times. A controller performs a calculation using a direction and an angle of the light of the first frequency and the light of the second frequency to correct the camera image.

[0005] In another aspect of the disclosure, an out-coupling grating of the waveguide receives the first light imaging data directed from the first in-coupling grating via a first refractive path and receives the second light imaging data directed from the second in-coupling grating via a second refractive path and directs the first light imaging data and the second light imaging data to an image sensor.

[0006] In another aspect of the disclosure, a look-up table is saved in a memory of the controller, wherein the controller performs the calculation by applying data of the predefined data saved in the look-up table of the controller.

[0007] In another aspect of the disclosure, ground line and pitch data of the first motor vehicle are saved in a memory of the controller and are accessible for use by the controller.

[0008] In another aspect of the disclosure, a heads-up display in the first motor vehicle presents the camera image.

[0009] In another aspect of the disclosure, the camera image defines a second motor vehicle.

[0010] In another aspect of the disclosure, light entering the camera from a first direction is redirected at a first different angle, and light entering from a second direction is redirected at a second different angle.

[0011] In another aspect of the disclosure, wherein the controller applies the light from the first and second directions and the first and second different angles to center the camera image on the heads-up display.

[0012] In another aspect of the disclosure, the camera is mounted at a predetermined camera height and camera azimuth.

[0013] In another aspect of the disclosure, the camera height and camera azimuth are used to convert an operator eye height and a vehicle ground line.

[0014] According to aspects, a ground line monitoring system includes a camera mounted in a first motor vehicle having a camera height and a camera azimuth. A waveguide produces a time-sequential capture of image data of the camera. Calibration and training steps are performed in which a correlation between appearance differences of features defining a second motor vehicle at different views and the camera height and camera azimuth are evaluated. A lookup table is generated using data collected in the calibration and training steps. The camera captures video including at least features defining the second motor vehicle. Using data in the lookup table, a vehicle ground line and an operator eye height are converted from the camera height and camera azimuth. A centered image of at least the features is generated using the vehicle ground line and the operator eye height presented on a heads-up display of the first motor vehicle.

[0015] In another aspect of the disclosure, if a viewable image received by the camera for display on the heads-up display including the second motor vehicle is different from a preprogrammed "centered" image saved in memory, a graphic displayed on the heads-up display is re-centered.

[0016] In another aspect of the disclosure, image data captured by the camera is compared to a difference between a high position and a low position of a test camera image.

[0017] In another aspect of the disclosure, a particular height and a particular pitch are linked to the image data using data in the lookup table.

[0018] In another aspect of the disclosure, a controller performs a calculation using a direction and an angle of the image data.

[0019] In another aspect of the disclosure, light rays entering the camera from a first direction are redirected at a first different angle; and light entering the camera from a second direction is redirected at a second different angle.

[0020] In another aspect of the disclosure, the controller calculates the first and second directions and the first and second different angles by applying data saved in a lookup table to display the image data as a centered image.

[0021] According to several aspects, a method for performing ground line monitoring of a motor vehicle includes installing a camera in a first motor vehicle; directing light to the camera using a waveguide having a first in-coupling grating that receives first light imaging data and passes the first light imaging data as light of a first frequency; the waveguide having a second in-coupling grating that receives second light imaging data and passes the second light imaging data as light of a second frequency; passing the light of the first frequency and the light of the second frequency through a color filter wheel; receiving the first light frequency and the second light frequency at different times in a timing sequence via an image sensor of the camera due to rotation of the color filter wheel; and performing calculations in a controller using directions and angles of the first light frequency and the second light frequency to correct the camera image.

[0022] In another aspect of the disclosure, the method further includes receiving the first light imaging data directed from the first in-coupling grating via a first refractive path of an out-coupling grating of the waveguide; receiving the second light imaging data directed from the second in-coupling grating via a second refractive path of the out-coupling grating of the waveguide; and directing the first light imaging data and the second light imaging data to the image sensor.

[0023] In another aspect of the disclosure, the method further includes redirecting light entering the camera from a first direction at a first different angle; redirecting light entering the camera from a second direction at a second different angle; and using the controller to apply the first and second directions and the first and second different angles of light to center the camera image on a head-up display of the first motor vehicle.

[0024] Further areas of applicability will become apparent from the description provided herein. It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0025] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way.

[0026] Figure 1 is a front view of a head-up display of a ground line monitoring system in accordance with one exemplary aspect;

[0027] Figure 2 is directing light to Figure 1side view of a waveguide of a camera of the system of

[0028] Figure 3 is Figure 1 front view of a first calibration system configuration of the system of

[0029] Figure 4 is Figure 1 front view of a second calibration system configuration of the system of

[0030] Figure 5 is Figure 1 front view of a third calibration system configuration of the system of

[0031] Figure 6 is Figure 1 front view of a fourth calibration system configuration of the system of DETAILED DESCRIPTION

[0032] The following description is merely exemplary in nature and is not intended to limit the disclosure, application or uses of the present disclosure.

[0033] With reference to Figure 1 A ground line monitoring system 10 is provided for a first motor vehicle 12 having a camera 14 that receives an image of a second motor vehicle 16 ahead of the first motor vehicle 12 on a roadway 18. The camera 14 can be a single camera or can define multiple cameras according to aspects. The ground line monitoring system 10 applies algorithms to camera imaging data defining the second motor vehicle 16 and ground line and pitch data of the first motor vehicle 12 saved in a memory 19 of a controller 20 located within the first motor vehicle 12 and accessible for use by the controller 20. The controller 20 is a non- general purpose electronic control device having a preprogrammed digital computer or processor, a memory or non-transitory computer readable medium for storing data such as control logic, software applications, instructions, computer code, data, lookup tables, etc., and a transceiver or input / output port according to aspects. The computer readable medium includes any type of medium capable of being accessed by a computer, such as read only memory (ROM), random access memory (RAM), a hard disk drive, a compact disc (CD), a digital video disc (DVD), or any other type of memory. The non-transitory computer readable medium does not include a transitory, propagating signal or other signal. The non-transitory computer readable medium includes media that can store data permanently and media that can store data and be subsequently overwritten, such as a rewritable compact disc or an erasable memory device. The computer code includes any type of program code, including source code, object code, and executable code.

[0034] For certain vehicle designs of the first motor vehicle 12, including, for example, pickup trucks, large sport utility vehicles (SUVs), and vans, the first motor vehicle 12 can have multiple ground lines. Each ground line can result in a different eye height and different vehicle pitch for the operator of the first motor vehicle 12. To align the virtual images with real-world objects, the controller 20 computes ground line and pitch data for the first motor vehicle 12 based on features in the images captured by the camera 14 (e.g., the second motor vehicle 16). The computed ground line and pitch data is then communicated to a heads-up display (HUD) 22 of the first motor vehicle 12. The HUD 22 presents information to the operator of the first motor vehicle 12 and is visible through a windshield 24. The ground line and pitch data is applied to adjust the image alignment visible in the HUD 22 to optimize the alignment of virtual images on the HUD 22 with real-world center.

[0035] Reference is made to Figure 2 and again to Figure 1 By introducing different views to the camera sensor of the camera 14 (as described below), the camera 14 can be used as a stereo camera. The ground line and pitch data can be obtained by comparing the same features captured from the different views. Depending on the height pitch angle of the views of the camera 14, the same features will appear to have different sizes and orientations. To generate different views of the same image, i.e., different views of the second motor vehicle 16, a waveguide 26 is provided. The waveguide 26 includes a first in-coupling grating 28 that receives first light imaging data 30 as a first frequency of light, and includes a second in-coupling grating 32 that receives second light imaging data 34 as a second frequency of light. The first light imaging data 30 is directed from the first in-coupling grating 28 via a first refractive path 36 to an out-coupling grating 38 of the waveguide 26. Similarly, the second light imaging data 34 is directed from the second in-coupling grating 32 via a second refractive path 40 to the out-coupling grating 38. The out-coupling grating 38 directs the light of both frequencies to the camera 14, which passes through a rotating color filter wheel 42 and is directed onto an image sensor 44 of the camera 14.

[0036] The first light imaging data 30, which is light as a first frequency, appears to capture an image of the second motor vehicle 16A that is located below a center line 50A of an example HUD image 46. In contrast, the second light imaging data 34, which is light as a second frequency, appears to capture an image of the second motor vehicle 16B that is located above a center line 50B of an example HUD image 52.

[0037] The grating, including the first in-coupling grating 28 and the second in-coupling grating 32, is made using a holographic fabrication method that redirects light in a particular direction. Using holographic interference, the system selects light coming in from a first direction and redirects the light at a different angle. The system also selects light coming in from a second direction and redirects the light at a different second angle. The controller 20 then uses the direction and angle of the light to perform a parallax calculation using the data of the predefined data held in the lookup table 58 shown to correct the camera image 56 so that the item (e.g., the second motor vehicle 16) presented on the HUD 22 is centered in the camera image 56 of the HUD 22. Figure 1 The direction and angle of the light are used to perform a parallax calculation using the data of the predefined data held in the lookup table 58 shown to correct the camera image 56 so that the item (e.g., the second motor vehicle 16) presented on the HUD 22 is centered in the camera image 56 of the HUD 22.

[0038] Referring generally Figures 3 to 6 and referring again to Figure 1 and Figure 2 The ground line monitoring system 10 introduces a stereoscopic-like feature in the camera with waveguide and timing data capture. The same feature in the real world can appear different at different viewing angles and perspectives. During a calibration and training step, the appearance difference of the feature (e.g., the second motor vehicle 16) at different views is evaluated in relation to the camera height and azimuth. The lookup table 58 is then generated. On the road, the camera 14 captures video that includes at least one captured feature (e.g., the second motor vehicle 16) and uses the captured feature to understand the height and azimuth of the camera 14. The vehicle ground line and the operator eye height are converted from the camera height and the camera azimuth using the data in the lookup table 58.

[0039] During the calibration and training step, the camera 14 is set to different heights and pitch. The feature is captured at different distances from the camera 14, e.g., at a high position and a low position, populating the lookup table 58 with feature differences at different vehicle camera heights and different vehicle pitch. The camera 14 is mounted at a known height in the first motor vehicle 12. During a subsequent driving operation, the camera 14 captures image data that is compared to the feature difference between the high position and the low position and linked to a particular height and a particular pitch using the data in the lookup table 58. The operator eye height and the vehicle ground line are then calculated using the controller 20 by applying the known camera height.

[0040] The waveguide-based camera design and corresponding algorithm calculates the ground line and pitch data for the first motor vehicle 12. The ground line and pitch data are then communicated to the HUD 22 for image adjustment to align with the environment. If the viewable image received by the camera 14 for display on the HUD image 46 (e.g., the second motor vehicle 16) is different than the preprogrammed "centered" image held in memory 19, the ground line monitoring system 10 changes the graphics being displayed on the HUD 22 to center the image.

[0041] With more particular reference to Figure 3 , the first calibration system configuration 60 positions the test camera at a high position and zero vehicle pitch.

[0042] With more particular reference to Figure 4 , the second calibration system configuration 62 positions the test camera at a low position and zero vehicle pitch.

[0043] With more particular reference to Figure 5 , the third calibration system configuration 64 positions the test camera at a high position and an exemplary downward -5 degrees of vehicle pitch.

[0044] With more particular reference to Figure 6 , the fourth calibration system configuration 66 positions the test camera at a low position and an exemplary downward -5 degrees of vehicle pitch.

[0045] The ground line monitoring system 10 of the present disclosure has a number of advantages. These include the use of waveguides and a time-sequential capture method to introduce different views to the camera sensor. The camera can then be used as a stereo camera. By comparing the same features captured from different perspectives, ground line and pitch data for the motor vehicle can be obtained. Depending on the height pitch angle of the view, the same feature or features will appear in different sizes and orientations. The system enables correct alignment of augmented reality graphics based on, for example, vehicle load which affects vehicle height and vehicle ground line.

[0046] With regard to the above for Figure 1The controller 20, memory 19 can include computer-readable media (also referred to as processor-readable media) including any non-transitory (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media can take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media can include, for example, optical or magnetic disks and other persistent memory. Volatile media can include, for example, dynamic random access memory (DRAM), which typically constitutes a main memory. Such instructions can be transmitted by one or more transmission media including coaxial cables; copper wire and fiber optics, including the wires that comprise a system bus coupled to a processor of an electronic control unit (ECU). Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, any other magnetic medium, a CD ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, or any other computer-readable medium.

[0047] The databases, data repositories or other data stores described herein can include various kinds of mechanisms for storing, accessing, and retrieving various data, including a hierarchical database, a set of files in a file system, an application database in a proprietary format, a relational database management system (RDBMS), etc. Each such data store is generally included within a computing device employing a computer operating system such as one of those mentioned above, and is accessed via a network in any one or more of a variety of manners as will be appreciated by those skilled in the art. The file system can be accessed from the computer operating system, and can include files stored in various formats. In addition to the language for creating, storing, editing, and executing stored procedures, RDBMSs typically use a structured query language (SQL), such as the procedural language extension to the structured query language (PL / SQL) mentioned above.

[0048] In some examples, system elements can be implemented as computer- readable instructions (e.g., software) stored on computer-readable media (e.g., disks, memories, etc.) associated with the system elements, on one or more computing devices (e.g., servers, personal computers, etc.). Computer program products can include such instructions stored on computer-readable media for performing the functions described herein.

[0049] In this application, including the definitions below, the term "module" or the term "controller" can be replaced with the term "circuit." The term "module" can refer to, be part of, or include: an Application Specific Integrated Circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combination of

[0050] The description of the present disclosure is merely exemplary in nature and variations that do not depart from the spirit of the present disclosure are within the scope of the present disclosure. Such variations are not to be regarded as a departure from the spirit and scope of the present disclosure.

Claims

1. A ground line monitoring system comprising: at least one camera mounted in a first motor vehicle; at least one waveguide directing light to the at least one camera, the waveguide having a first in-coupling grating that receives first light imaging data and passes the first light imaging data as light of a first frequency and a second in-coupling grating that receives second light imaging data and passes the second light imaging data as light of a second frequency; at least one color filter wheel that receives the light of the first frequency and the light of the second frequency; at least one image sensor of the camera that receives the light of the first frequency and the light of the second frequency at different times due to rotation of the at least one color filter wheel; and at least one controller that performs a calculation using a direction and an angle of the light of the first frequency and the light of the second frequency to correct at least one camera image.

2. The ground line monitoring system of claim 1, further comprising at least one out-coupling grating of the waveguide that receives the first light imaging data directed from the first in-coupling grating via a first refractive path and receives the second light imaging data directed from the second in-coupling grating via a second refractive path and directs the first light imaging data and the second light imaging data to the at least one image sensor. the at least one controller performs the calculation by applying pre-defined data saved in the look-up table of the at least one controller.

3. The ground line monitoring system of claim 2, further comprising a lookup table, the lookup table being stored in a memory of the at least one controller, wherein, 4. The ground line monitoring system of claim 3, further comprising ground line and pitch data of the first motor vehicle saved in a memory of the at least one controller and accessible by the at least one controller.

5. The ground line monitoring system of claim 4, further comprising a head-up display in the first motor vehicle that presents a virtual image of the at least one camera image. the at least one camera defines a first camera and a second camera.

6. The ground line monitoring system of claim 5, wherein, light entering the first camera and the second camera from a first direction is redirected at a first different angle using holographic interference, light entering the first camera and the second camera from a second direction is redirected at a second different angle using holographic interference.

7. The ground line monitoring system of claim 6, wherein, the at least one controller applies the light of the first direction and the second direction and the first different angle and the second different angle to obtain a current heading and a current height of the at least one camera.

8. The ground line monitoring system of claim 7, wherein, the at least one camera is mounted at a predetermined camera height and a camera heading.

9. The ground line monitoring system of claim 1, wherein, 10. The ground line monitoring system of claim 9, further comprising an operator’s eye height and a vehicle ground line converted using the current camera height and the camera heading. ​

Images