Baseline monitoring system

Abstract

Baseline monitoring system (10), comprising: at least one camera (14) mounted in a first motor vehicle (12); at least one waveguide (26) guiding light into the at least one camera (14), with a first coupling grating (28) receiving first light imaging data (30) and transmitting the first light imaging data (30) as a first frequency of light, and a second coupling grating (32) receiving second light imaging data (34) and transmitting the second light imaging data (34) as a second frequency of light; at least one color filter wheel (42) receiving the first frequency of light and the second frequency of light; at least one image sensor (44) of the camera (14) receiving the first frequency of light and the second frequency of light at different times due to a rotation of the at least one color filter wheel (42);and at least a controller (20) that performs a calculation using directions and angles of the first frequency of the light and the second frequency of the light to correct at least one camera image (56).

Description

The present invention relates to a baseline monitoring system, in particular to head-up display systems for motor vehicles and to vehicle baselines that influence the image display on the head-up display. In vehicles equipped with a head-up display, a vehicle can have multiple baselines. Each baseline results in a different eye level and vehicle tilt. To align a virtual image with reality on an augmented reality head-up display (ARHUD), information regarding eye level and vehicle tilt is required. However, vehicle tilt can vary, especially in trucks, including pickups, whose ground clearance can differ. Furthermore, vehicle tilt can also vary depending on the truck's load and weight distribution. These differences in vehicle tilt can, in turn, cause problems such as issues with image sharpness and accuracy in the virtual images generated on the ARHUD. While current head-up displays fulfill their intended purpose, there is a need for a new and improved baseline monitoring system to provide data to a motor vehicle's head-up display. DE 10 2017 217 193 A1 discloses a display device for a vehicle with at least one disc, with at least one volume hologram arranged at least within a transparent partial area of ​​the disc, and with at least one light source by means of which light can be coupled into the volume hologram, from which at least one spatially appearing image for a human observer can be generated by means of the volume hologram, wherein at least one camera device having at least one light-sensitive image sensor, the optics of which, via which images can be detected by means of the image sensor, are at least partially formed by the partial area. Summary From several perspectives, a baseline monitoring system comprises a camera mounted in a first vehicle. A waveguide directs light into the camera, with a first coupling grating that receives and transmits the first light imaging data as a first frequency of light, and a second coupling grating that receives and transmits the second light imaging data as a second frequency of light. A color filter wheel receives the first and second light frequencies. An image sensor in the camera receives the first and second light frequencies at different times due to the rotation of the color wheel. A controller performs a calculation using the directions and angles of the first and second light frequencies to correct the camera image. In another aspect of the present invention, a waveguide output receives the first light imaging data transmitted from the first coupling grating via a first refraction path and receives the second light imaging data transmitted from the second coupling grating via a second refraction path and transmits the first light imaging data and the second light imaging data to the image sensor. In another aspect of the present invention, a lookup table is stored in a memory of the controller, wherein the controller performs the calculation by applying data from predefined data stored in the lookup table of the controller. In another aspect of the present invention, a baseline and inclination data of the first motor vehicle are stored in a memory of the controller and are available for use by it. In another aspect of the present invention, a head-up display in the first motor vehicle presents the camera image. In another aspect of the present invention, the camera image defines a second motor vehicle. In another aspect of the present invention, light entering the camera from a first direction is redirected at a first different angle, and light entering from a second direction is redirected at a second different angle. In another aspect of the present invention, the first direction and the second direction and the first different angle and the second different angle of the light are used by the controller to center the camera image on the head-up display. In another aspect of the present invention, the camera is mounted at a predefined camera height and camera orientation. In another aspect of the present invention, the driver's eye level and the vehicle's baseline are converted or reconverted using the camera height and camera orientation. In several respects, a baseline monitoring system comprises a camera mounted in a first vehicle with a defined camera height and orientation. A waveguide generates a temporally sequential acquisition of image data from the camera. A calibration and training step evaluates the correlation between the appearance of a feature defining a second vehicle in various views and the camera height and orientation. A lookup table is generated using the data collected in the calibration and training step. A video is recorded by the camera that includes at least the feature defining the second vehicle. A vehicle baseline and the driver's eye level are derived from the camera height and orientation using data from the lookup table.A centered image of at least the feature is generated using the vehicle's baseline and the driver's eye level and displayed or presented on a head-up display of the first vehicle. In another aspect of the present invention, a graphic displayed on the head-up display is recentered if a visible image containing the second motor vehicle, received from the camera for display on the head-up display, differs from a pre-programmed “centered” image stored in a memory. In another aspect of the present invention, image data captured by the camera are compared with differences between a high position and a low position of a test camera image. In another aspect of the present invention, a specific height and a specific inclination are linked to the image data using data in the lookup table. In another aspect of the present invention, a controller performs a calculation using directions and angles of the image data. In another aspect of the present invention, light entering the camera from a first direction is redirected at a different angle, and light entering the camera from a second direction is redirected at a second different angle. In another aspect of the present invention, the controller calculates the first direction and the second direction and calculates the different angle and the second different angle by applying data stored in the lookup table to display the image data as the centered image. According to several aspects, a method for performing baseline monitoring of motor vehicles comprises: mounting a camera in a first motor vehicle; guiding light into the camera using a waveguide with a first coupling grating that receives first light imaging data and transmits the first light imaging data as a first frequency of light; wherein the waveguide has a second coupling grating that receives second light imaging data and transmits the second light imaging data as a second frequency of light; transmitting the first frequency of light and the second frequency of light through a color filter wheel; receiving the first light frequency and the second light frequency via an image sensor of the camera at different times in temporal sequence due to a rotation of the color wheel;and performing a calculation in a controller using directions and angles of the first light frequency and the second light frequency to correct a camera image. In another aspect of the present invention, the method further comprises: receiving the first light imaging data, which is guided from the first coupling grating via a first refractive path of an output coupling of the waveguide; receiving the second light imaging data, which is guided from the second coupling grating via a second refractive path of the output coupling of the waveguide; and guiding the first light imaging data and the second light imaging data to the image sensor. In another aspect of the present invention, the method further comprises: redirecting light entering the camera from a first direction at a first different angle; redirecting light entering the camera from a second direction at a second different angle; and applying the first direction and the second direction and the first different angle and the second different angle of the light using the controller to center the camera image on a head-up display of the first motor vehicle. Further areas of application will become apparent from the description provided here. Brief description of the drawings The drawings described here are for illustrative purposes only. Fig. 1 is a front cutaway view of a head-up display for a baseline monitoring system according to an exemplary aspect; Fig. 2 is a side cutaway view of a waveguide directing light into a camera of the system of Fig. 1; Fig. 3 is a front cutaway view of a first calibration system configuration for the system of Fig. 1; Fig. 4 is a front cutaway view of a first calibration system configuration for the system of Fig. 1; Fig. 5 is a front cutaway view of a first calibration system configuration for the system of Fig. 1; and Fig. 6 is a front cutaway view of a first calibration system configuration for the system of Fig. 1. Detailed description The following description is merely exemplary and is not intended to limit the present invention, application or uses. Referring to Fig. 1, a baseline monitoring system 10 is provided for a first motor vehicle 12 with a camera 14 that receives images of a second motor vehicle 16 located in front of the first motor vehicle 12 on a roadway 18. Depending on several aspects, the camera 14 can be a single camera or it can define multiple cameras. The baseline monitoring system 10 applies an algorithm to the camera image data defining the second motor vehicle 16, as well as to baseline and inclination data of the first motor vehicle 12, which are stored in a memory 19 of a controller 20 located in the first motor vehicle 12 and are available for use by it.From several perspectives, the Controller 20 is a non-generalized electronic control device comprising a pre-programmed digital computer or processor, memory or non-transient computer-readable medium used to store data such as control logic, software applications, instructions, computer code, data, lookup tables, etc., and a transceiver or input / output ports. The computer-readable medium includes any type of medium accessible by a computer, such as read-only memory (ROM), random-access memory (RAM), a hard disk drive, a compact disc (CD), a digital video disc (DVD), or any other type of storage. The non-transient computer-readable medium excludes wired, wireless, optical, or other communication links carrying transient electrical or other signals.Non-transient computer-readable media includes media on which data can be permanently stored and media on which data can be stored and later overwritten, such as a rewritable optical disc or disk, or an erasable storage device. Computer code includes any type of program code, including source code, object code, and executable code. For certain vehicle designs of the first motor vehicle 12, including vans or light commercial vehicles, large off-road vehicles (SUVs), and light commercial vehicles, the first motor vehicle 12 can have multiple baselines. Each baseline can result in a different eye level for the driver of the first motor vehicle 12 and a different vehicle tilt. To align a virtual image with a real object, the controller 20 calculates the baseline and tilt of the first motor vehicle 12 based on the feature, such as that of the second motor vehicle 16, in the images captured by the camera 14. The calculated baseline and tilt data are then transmitted to a head-up display (HUD) 22 for augmented reality in the first motor vehicle 12. The HUD 22 presents information to a driver of the first motor vehicle 12 and is visible through a windshield 24.The baseline and tilt data are used to adjust or set an image orientation visible in the HUD 22 in order to optimize a central alignment of the virtual image on the HUD 22 to the real world. Referring to Fig. 2 and again to Fig. 1, the camera 14 can operate as a stereoscopic camera by presenting different views to a camera sensor of the camera 14 (discussed below). The baseline and inclination can be obtained by comparing the same feature captured from different views. The same feature appears in different sizes and orientations depending on the vertical tilt angle of the camera 14's view. To generate different views of the same image, i.e., different views of the second vehicle 16, a waveguide 26 is provided. The waveguide 26 includes a first coupling grating 28, which receives first light imaging data 30 as a first frequency of the light, and a second coupling grating 32, which receives second light imaging data 34 as a second frequency of the light.The first light imaging data 30 are guided from the first coupling grating 28 via a first refraction path 36 to an output 38 of the waveguide 26. Similarly, the second light imaging data 34 are guided from the second coupling grating 32 via a second refraction path 40 to the output 38. The output 38 directs the two frequencies of the light into the camera 14, where it passes through a rotating color filter wheel 42 and is directed onto an image sensor 44 of the camera 14. The first light imaging data 30, representing the first frequency of the light, appears to capture an image of the second vehicle 16A, positioned below a centerline 50A of an exemplary HUD image 46. In contrast, the second light imaging data 34, representing the second frequency of the light, appears to capture an image of the second vehicle 16B, positioned above a centerline 50B of an exemplary HUD image 52. The gratings, comprising the first coupling grating 28 and the second coupling grating 32, are created using a holographic fabrication process that redirects the light in a specific direction. Utilizing holographic interference, the system selects the light incident from a first direction and redirects it at a different angle. The system also selects light incident from a second direction and redirects it at a second different angle. The controller 20 then performs a disparity calculation using the directions and angles of the light, applying predefined data stored in a lookup table 58 of the controller 20 (shown in Fig. 1) to correct a camera image 56 so that the elements presented on the HUD 22, such as the second vehicle 16, are centered in the camera image 56 of the HUD 22. With general reference to Figs. 3, 4, 5 to 6, and again to Figs. 1 and 2, the baseline monitoring system 10 introduces stereoscopic-like characteristics in a camera with a waveguide and temporally sequential data acquisition. The same feature in the real world can appear differently in different perspective views and angles. During a calibration and training step, a correlation between appearance differences of the feature, such as the second vehicle 16, in different views and the camera height and orientation is evaluated. Subsequently, the lookup table 58 is created. On the road, the camera 14 records a video that includes at least one captured feature, such as the second vehicle 16, and uses the captured feature to understand the height and orientation of the camera 14.A vehicle baseline and a driver's eye level are converted from a camera height and a camera orientation using data in reference table 58. During the calibration and training step, the camera 14 is oriented at various heights and tilts. Features are captured at different distances from the camera 14, for example, at a high position and a low position, with the stored lookup table 58 containing feature differences at different camera heights and vehicle tilts. The camera 14 is installed in the first vehicle 12 at a known height. During subsequent driving, the camera 14 captures image data, which is compared with the feature differences between the high and low positions and linked to a specific height and distance using the data in the lookup table 58. The driver's eye level and the vehicle's baseline are then calculated using the controller 20 and the known camera height. The waveguide-based camera design and a corresponding algorithm calculate the baseline and tilt angle of the first vehicle 12. The baseline and tilt information is then forwarded to the HUD 22 to adjust the image setting to the surroundings. If the visible image, such as that of the second vehicle 16, received from the camera 14 for display on the HUD image 46, deviates from a pre-programmed "centered" image stored in memory 19, the baseline monitoring system 10 modifies the graphic displayed on the HUD 22 to center the image. More specifically, with reference to Fig. 3, a first calibration system configuration 60 positions a test camera in a high position and with a vehicle inclination of zero. More specifically, with reference to Fig. 4, a second calibration system configuration 62 positions a test camera in a low position and with a vehicle inclination of zero. More specifically, with reference to Fig. 5, a third calibration system configuration 64 positions a test camera in a high position and at an exemplary vehicle inclination of -5 degrees downwards (Fig. 5). More specifically, with reference to Fig. 6, a fourth calibration system configuration 66 positions a test camera in a low position and at an exemplary vehicle inclination of -5 degrees downwards. The baseline monitoring system 10 of the present invention offers several advantages. These include the fact that, using a waveguide approach and temporally sequential acquisition, different views are presented to a camera sensor. The camera can then operate as a stereoscopic camera. A baseline and the inclination of a motor vehicle can be obtained by comparing the same features acquired from different views. The same feature or features appear in different sizes and orientations depending on the vertical tilt angle of the view. The present system enables the correct alignment of an augmented reality graphic based, for example, on the vehicle load, which affects the vehicle height and the vehicle baseline. With reference to the controller 20 described above with respect to Fig. 1, the memory 19 can comprise a computer-readable medium (also referred to as a processor-readable medium) that includes any non-transitory (e.g., physical) medium involved in the provision of data (e.g., Instructions) are involved that can be read by a computer (e.g., by a computer's processor). Such a medium can take many forms, including, but not limited to, non-volatile and volatile media. Non-volatile media can include, for example, optical or magnetic disks and other persistent storage. Volatile media can include, for example, dynamic random-access memory (DRAM), which typically forms main memory. Such instructions can be transmitted by means of one or more transmission media, including coaxial cable, copper wire, and fiber optics or optical waveguides, which includes the wires that form a system bus connected to a processor of an electronic control unit (ECU).Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape, any other magnetic medium, a CD-ROM, a DVD, any other optical medium, punched cards, a paper tape, any other physical medium with hole patterns, random access memory (RAM), a PROM, an erasable programmable read-only memory (EPROM), a FLASH, an electrically erasable programmable read-only memory (EEPROM), any other memory chip or cartridge, or any other medium from which a computer can read. Databases, data archives, or other data storage devices described here can include various types of mechanisms for storing, accessing, and retrieving different types of data, including a hierarchical database, a set of files in a file system, an application database in a proprietary format, a relational database management system (RDBMS), etc. Each such data storage device is generally contained within a computer device or computing unit that uses a computer operating system such as one of those mentioned above, and is accessed over a network in one or more of a variety of ways. A file system can be accessed by a computer operating system and can contain files stored in various formats. An RDBMS generally uses the Structured Query Language (SQL) in addition to a language for creating and retrieving data.Creating, saving, editing and executing stored procedures such as the above-mentioned extensions of a procedural language to a structured query language (PL / SQL). In some examples, system elements can be implemented as computer-readable instructions (e.g., software) on one or more computing devices (e.g., servers, personal computers, etc.) that are stored on associated computer-readable media (e.g., floppy disks or hard drives, memory, etc.). A computer program product can include such instructions stored on computer-readable media to perform the functions described herein. In this application, including the definitions below, the term "module" or the term "controller" may be replaced by the term "circuit". The term "module" may refer to, be part of, or include an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field-programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above components, such as in a system-on-a-chip.

Claims

Baseline monitoring system (10), comprising: at least one camera (14) mounted in a first motor vehicle (12); at least one waveguide (26) guiding light into the at least one camera (14), with a first coupling grating (28) receiving first light imaging data (30) and transmitting the first light imaging data (30) as a first frequency of light, and a second coupling grating (32) receiving second light imaging data (34) and transmitting the second light imaging data (34) as a second frequency of light; at least one color filter wheel (42) receiving the first frequency of light and the second frequency of light; at least one image sensor (44) of the camera (14) receiving the first frequency of light and the second frequency of light at different times due to a rotation of the at least one color filter wheel (42);and at least a controller (20) that performs a calculation using directions and angles of the first frequency of the light and the second frequency of the light to correct at least one camera image (56). Baseline monitoring system (10) according to claim 1, comprising at least one output (38) of the waveguide (26) which receives the first light imaging data (30) transmitted from the first coupling grid (28) via a first refraction path (38) and receives the second light imaging data (34) transmitted from the second coupling grid (32) via a second refraction path (40) and transmits the first light imaging data (30) and the second light imaging data (34) to the at least one image sensor (44). Baseline monitoring system (10) according to claim 2, further comprising a lookup table (58) which is stored in a memory (19) of the at least one controller (20), wherein the at least one controller (20) performs the calculation by applying data from predefined data which is stored in the lookup table (58) of the at least one controller (20). Baseline monitoring system (10) according to claim 3, further comprising a baseline and inclination data of the first motor vehicle (12) which are stored in a memory (19) of the at least one controller (20) and which the at least one controller (20) can access. Baseline monitoring system (10) according to claim 4, further comprising a head-up display (22) in the first motor vehicle (12) which presents a virtual image of the at least one camera image (56). Baseline monitoring system (10) according to claim 5, wherein the at least one camera (14) defines a first camera (14) and a second camera (14). Baseline monitoring system (10) according to claim 6, wherein using holographic interference the light entering the first camera (14) and the second camera (14) from a first direction is redirected at a first different angle and the light entering the first camera (14) and the second camera (14) from a second direction is redirected at a second different angle. Baseline monitoring system (10) according to claim 7, wherein the at least one controller (20) uses the first direction and the second direction and the first different angle and the second different angle of the light to obtain a current orientation and a current height of the at least one camera (14). Baseline monitoring system (10) according to claim 1, wherein the at least one camera (14) is mounted at a predefined camera height and camera orientation. Baseline monitoring system (10) according to claim 9, further comprising a driver's eye level and a vehicle baseline which are converted using a current camera height and camera orientation.

Images