Hybrid projector system with self-checking function

Abstract

A projector system for a head-up display in a vehicle includes: a vehicle vector graphic projector adapted to project a second graphic image onto an interior surface of a vehicle windshield, the second graphic image having a higher brightness than the first graphic image and including at least one vehicle indicator light; and a calibration system adapted to test and calibrate the indicator light displayed within the second graphic image.

Description

Technical Field

[0001] The present disclosure relates to systems and methods for displaying graphics in a vehicle, and more particularly to a graphics projector for use in a vehicle. Background Art

[0002] To improve occupant awareness and convenience, a vehicle may be equipped with a display system that is configured to provide information to the occupant about the vehicle's surroundings. The display system may use a human-machine interface device (e.g., a touch screen) to provide information to the occupant. The display system may also use a head-up display (HUD) to provide information to the occupant.

[0003] Vehicles typically include multiple independently illuminated tell-tale lights. Tell-tale lights, sometimes called warning lights, are indicators of system malfunctions or operation and are indicated by binary (on / off) illumination, symbols, or text legends. Tell-tale lights are not typically part of a reconfigurable display or instrument cluster display due to replacement cost rules and bulb inspection and reliability requirements.

[0004] Therefore, while current vehicle systems achieve their intended purposes, a need remains for a new and improved system for graphic projection that provides a low-cost, highly reliable solution for displaying such indicator lights and replaces separate, dedicated indicator lighting. Summary of the invention

[0005] According to several aspects of the present disclosure, a system for a head-up display in a vehicle includes: a vehicle vector graphics projector, adapted to project a second graphic image onto an inner surface of a vehicle windshield, the second graphic image having a higher brightness than a first graphic image and including at least one vehicle indicator light; and a calibration system, adapted to test and calibrate the indicator light displayed in the second graphic image.

[0006] According to another aspect, a calibration system includes a reference calibration system, which includes a reference windshield; a reference vector graphics projector, wherein the reference vector graphics projector includes a reference vector graphics projector camera; a reference camera; and a reference controller electrically connected to the reference vector graphics projector and the reference camera, wherein the reference controller is programmed to: project a reference graphic onto the reference windshield using the reference vector graphics projector, wherein the reference graphic is a vehicle indicator light and includes a plurality of projection points arranged in a two-dimensional array, wherein an expected x-coordinate and an expected y-coordinate of each of the plurality of projection points are defined by an expected reference graphic matrix; capture a first reference image of the reference graphic using the reference camera; and calculate at least one reference calibration matrix based at least in part on the first reference image of the reference graphic.

[0007] According to another aspect, when calculating at least one reference calibration matrix, the reference controller is also programmed to: determine a reference camera deviation matrix, wherein the reference camera deviation matrix encodes deviations in the reference graphics resulting from characteristics of the reference windshield; and determine a reference vector graphics projector camera transformation matrix based at least in part on the reference camera deviation matrix.

[0008] According to another aspect, when determining the reference camera deviation matrix, the reference controller is also programmed to: identify multiple perceived points of the reference pattern in the first reference image; determine the actual x-coordinate and the actual y-coordinate of each of the multiple perceived points of the reference pattern in the first reference image; generate an actual reference pattern matrix, wherein each element of the actual reference pattern matrix contains the actual x-coordinate and the actual y-coordinate of one of the multiple perceived points of the reference pattern in the first reference image; and calculate the reference camera deviation matrix by subtracting the actual reference pattern matrix from the expected reference pattern matrix.

[0009] According to another aspect, when determining the reference vector graphics projector camera transformation matrix, the reference controller is also programmed to: generate a deviation-compensated reference graphics matrix by summing the expected reference graphics matrix with the reference camera deviation matrix; project the deviation-compensated reference graphics matrix onto the reference windshield using the reference vector graphics projector, wherein the position of each of the multiple projected points of the deviation-compensated reference graphics is defined by the deviation-compensated reference graphics matrix; capture a second reference image of the deviation-compensated reference graphics using the reference vector graphics projector camera; identify multiple perceived points of the deviation-compensated reference graphics in the second reference image; determine the actual x-coordinate and actual y-coordinate of each of the multiple perceived points of the deviation-compensated reference graphics in the second reference image; generate a reference vector graphics projection camera matrix, wherein each element of the reference vector graphics projector camera matrix contains the actual x-coordinate and actual y-coordinate of one of the multiple perceived points of the deviation-compensated reference graphics in the second reference image; and determine the vector graphics projector camera transformation matrix by subtracting the reference vector graphics projector camera matrix from the reference camera deviation matrix.

[0010] According to another aspect, a vehicle vector graphics projector includes a vehicle vector graphics projector camera, and a hybrid projector system includes a vehicle controller electrically connected to the vehicle vector graphics projector, wherein the vehicle controller is programmed to: project a reference graphic onto a vehicle windshield using the vehicle vector graphics projector; capture a calibration image of the reference graphic using the vehicle vector graphics projector camera; calculate a vehicle vector graphics projector calibration matrix based at least in part on at least one reference calibration matrix and the calibration image of the reference graphic; and apply the vehicle vector graphics projector calibration matrix to each frame projected using the vehicle vector graphics projector.

[0011] According to another aspect, when calculating the vehicle vector graphics projector calibration matrix, the vehicle controller is also programmed to: determine the vehicle vector graphics projector deviation matrix based at least in part on at least one reference calibration matrix and a calibration image of a reference graphic; and calculate the vehicle vector graphics projector calibration matrix by subtracting the vehicle vector graphics projector deviation matrix from the reference TWD camera transformation matrix.

[0012] According to another aspect, when determining the vehicle vector graphics projector deviation matrix, the vehicle controller is also programmed to: identify multiple perception points of a reference graphic in a calibration image; determine an actual x-coordinate and an actual y-coordinate of each of the multiple perception points of the reference graphic in the calibration image; generate an actual vehicle vector graphics projection matrix, wherein each element of the actual vehicle vector graphics projection matrix contains the actual x-coordinate and the actual y-coordinate of one of the multiple perception points of the reference graphic in the calibration image; and calculate the vehicle vector graphics projector deviation matrix by subtracting the actual vehicle vector graphics projection matrix from the reference vector graphics projection camera matrix.

[0013] According to another aspect, a reference camera is fixedly mounted at a reference occupant eyellipse position, wherein the reference occupant eyellipse position is a predetermined position in space relative to a reference windshield.

[0014] According to another aspect, a vehicle vector graphics projector includes: a light source module, which includes: a light emitting diode (LED) array configured to generate source light; a filter optically connected to the LED array, wherein the filter is configured to receive the source light and transmit the filtered light; a focusing lens optically connected to the filter, wherein the focusing lens is configured to receive the filtered light and transmit a narrow light beam; an optical collimator optically connected to the light source module, wherein the optical collimator is configured to receive the narrow light beam and transmit the collimated light; and a photoelectric mirror optically connected to the optical collimator, wherein the photoelectric mirror is configured to guide the collimated light to form a graphic.

[0015] According to another aspect, the filter is an optical bandpass filter configured to filter source light and transmit filtered light, wherein the filtered light includes only light having a wavelength within a passband of the optical bandpass filter.

[0016] According to another aspect, the system also includes a plurality of light source modules in optical communication with the optical collimator.

[0017] According to another aspect, the system further includes an optical waveguide in optical communication with the plurality of light source modules and the optical collimator, wherein the optical waveguide is configured to transmit a narrow light beam from each of the plurality of light source modules to the optical collimator.

[0018] According to another aspect, the optical waveguide is a fiber optic cable in optical communication with a focusing lens of each of the plurality of light source modules using a fiber optic coupler.

[0019] According to another aspect, the system further includes a plurality of dichroic mirrors, wherein each of the plurality of dichroic mirrors is in optical communication with one of the plurality of light source modules, wherein each of the plurality of dichroic mirrors is configured to transmit a narrow light beam from one of the plurality of light source modules to the optical collimator.

[0020] According to another aspect, each of the plurality of dichroic mirrors is configured to reflect narrow light beams of predetermined wavelengths and transmit narrow light beams of all other wavelengths.

[0021] According to several aspects of the present disclosure, a method for displaying graphics using a projector system for a head-up display in a vehicle includes: projecting a second graphic image onto an inner surface of a vehicle windshield using a vehicle vector graphics projector, the second graphic image having a higher brightness than the first graphic image and including at least one vehicle indicator light; and testing and calibrating the vehicle indicator light displayed in the second graphic image using a calibration system.

[0022] According to another aspect, testing and calibrating a vehicle signal displayed in a second graphical image using a calibration system further includes projecting a reference graphic onto a reference windshield using a reference vector graphics projector, wherein the reference graphic is a vehicle indicator light, capturing a first reference image of the reference graphic using a reference camera, determining a reference camera deviation matrix, wherein the reference camera deviation matrix encodes deviations in the reference graphic due to characteristics of the reference windshield by identifying a plurality of perception points of the reference graphic in the first reference image, determining an actual x-coordinate and an actual y-coordinate of each of the plurality of perception points of the reference graphic in the first reference image, and generating an actual reference graphic matrix, wherein each element of the actual reference graphic matrix contains a first reference image of the reference graphic. an actual x-coordinate and an actual y-coordinate of one of a plurality of perceived points of a reference graphic, and calculating a reference camera deviation matrix by subtracting the actual reference graphic matrix from an expected reference graphic matrix, wherein the expected reference graphic matrix is ​​a two-dimensional array including expected x-coordinates and expected y-coordinates for each of a plurality of projected points, determining a reference vector graphics projector camera transformation matrix based at least in part on the reference camera deviation matrix, generating a deviation-compensated reference graphic matrix by summing the expected reference graphic matrix with the reference camera deviation matrix, projecting the deviation-compensated reference graphic onto a reference windshield using a reference vector graphics projector, wherein the position of each of the plurality of projected points of the deviation-compensated reference graphic is defined by the deviation-compensated reference graphic matrix, and determining a reference camera deviation matrix based at least in part on the reference camera deviation matrix. Capturing a second reference image of a deviation-compensated reference pattern using a vector graphics projector camera, identifying a plurality of perception points of the deviation-compensated reference pattern in the second reference image, determining an actual x-coordinate and an actual y-coordinate of each of the plurality of perception points of the deviation-compensated reference pattern in the second reference image, generating a reference vector graphics projector camera matrix, wherein each element of the reference vector graphics projector camera matrix comprises an actual x-coordinate and an actual y-coordinate of one of the plurality of perception points of the deviation-compensated reference pattern in the second reference image, and determining a reference vector graphics projector camera transformation matrix by subtracting the reference vector graphics projector camera matrix from the reference camera deviation matrix, and projecting the reference pattern onto a vehicle windshield using a vehicle vector graphics projector. A method of capturing a calibration image of a reference graphic using a vehicle vector graphics projector camera, identifying a plurality of perception points of the reference graphic in the calibration image, determining a vehicle vector graphics projector bias matrix based at least in part on a reference vector graphics camera matrix and the calibration image of the reference graphic, calculating a vehicle vector graphics projector calibration matrix based at least in part on the reference camera bias matrix, a reference TWD camera transformation matrix, and the calibration image, determining an actual x-coordinate and an actual y-coordinate of each of the plurality of perception points of the reference graphic in the calibration image, and generating an actual vehicle vector graphics projector matrix, wherein each element of the actual vehicle vector graphics projection matrix comprises an actual x-coordinate and an actual y-coordinate of one of the plurality of perception points of the reference graphic in the calibration image,and calculating a vehicle vector graphics projector bias matrix by subtracting an actual vehicle vector graphics projector matrix from a reference vector graphics projector camera matrix, and calculating a vehicle vector graphics projector calibration matrix based at least in part on the vehicle vector graphics projector bias matrix by subtracting the vector graphics projector bias matrix from the vector graphics projector camera transformation matrix, and applying the vehicle vector graphics projector calibration matrix to each frame projected using the vehicle vector graphics projector.,

[0023] According to another aspect, projecting a second graphic image onto the inner surface of a vehicle windshield using a vector graphics projector also includes generating source light using a light emitting diode (LED) array of a light source module, receiving the source light using an optical bandpass filter of the light source module optically connected to the LED array, filtering the source light through the optical bandpass filter, transmitting wavelengths within a passband of the optical bandpass filter through the optical bandpass filter, receiving filtered light through a focusing lens of the light source module optically connected to the optical filter, transmitting a narrow light beam through the focusing lens, receiving the narrow light beam through an optical collimator optically connected to the light source module, transmitting collimated light through the optical collimator, and guiding the collimated light to form a graphic through a photoelectric mirror optically connected to the optical collimator.

[0024] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0026] Figure 1 is a schematic diagram of a vehicle having a system according to an exemplary embodiment of the present disclosure;

[0027] Figure 2 is a schematic diagram of a system according to an exemplary embodiment, wherein a second graphic image from a vector graphics projector is displayed adjacent to a first graphic image from a digital optical projector;

[0028] Figure 3 yes Figure 1 a schematic side view of the vehicle shown;

[0029] Figure 4 is a schematic diagram of a substrate for a vehicle windshield according to an exemplary embodiment;

[0030] Figure 5 yes Figure 4 The enlarged part, such as Figure 4 The middle mark is " Figure 5 " is shown in the circle;

[0031] Figure 6 is a schematic diagram of a vector graphics projector including a waveguide including a fiber optic cable;

[0032] Figure 7 is a schematic diagram of a vector graphics projector including a dichroic mirror;

[0033] Figure 8 is a schematic diagram of a system according to an exemplary embodiment, wherein a second graphic image from a vector graphics projector is displayed on top of a first graphic image from a digital optical projector;

[0034] Fig. 9 is a schematic diagram of a reference calibration system according to an exemplary embodiment;

[0035] Fig.10 is a flow chart illustrating a method according to an exemplary embodiment of the present disclosure;

[0036] Fig.11 is a flow chart of an exemplary embodiment of a method for calculating at least one reference calibration matrix according to an exemplary embodiment;

[0037] Fig. 12A is a first exemplary view of a projection field of a reference calibration system according to an exemplary embodiment;

[0038] Fig. 12B is a second exemplary view of a projection field of a reference calibration system according to an exemplary embodiment; and

[0039] Fig.13 FIG. 4 is a flow chart of an exemplary embodiment of a method for calculating a vehicle TWD calibration matrix according to an exemplary embodiment.

[0040] The drawings are not necessarily drawn to scale, and some features may be exaggerated or minimized, for example, to show details of a particular component. In some cases, well-known components, systems, materials, or methods are not described in detail in order to avoid obscuring the present disclosure. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching those skilled in the art to adopt the present disclosure in different ways. DETAILED DESCRIPTION

[0041] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application or use. In addition, it is not intended to be constrained by any express or implied theory presented in the aforementioned technical field, background technology, brief overview or the following detailed description. It should be understood that throughout the drawings, the corresponding reference numerals represent similar or corresponding components and features. As used herein, the term module refers to any hardware, software, firmware, electronic control component, processing logic and / or processor device, individually or in any combination, including but not limited to: application specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated or groups) and memories that execute one or more software or firmware programs, combinational logic circuits and / or other suitable components that provide the described functions. Although the drawings shown herein depict examples with certain element arrangements, additional intermediate elements, devices, features or components may be present in actual embodiments. It should also be understood that the drawings are merely illustrative and may not be drawn to scale.

[0042] As used herein, the term "vehicle" is not limited to automobiles. Although this article mainly describes the present technology in conjunction with automobiles, the present technology is not limited to automobiles. These concepts can be used in a variety of applications, such as aircraft, ships, other vehicles, and consumer electronic components.

[0043] According to an exemplary embodiment, Figure 1 A vehicle 10 is shown according to various embodiments, having an associated hybrid projector system for a head-up display 11 within the vehicle 10. Generally speaking, the hybrid projector system 11 works in conjunction with other systems within the vehicle 10 to display various information and infotainment content to passengers. The vehicle 10 generally includes a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is disposed on the chassis 12 and substantially surrounds the components of the vehicle 10. The body 14 and the chassis 12 may together form a vehicle frame. The front wheels 16 and the rear wheels 18 are each rotatably connected to the chassis 12 near respective corners of the body 14.

[0044] In various embodiments, the vehicle 10 is an autonomous vehicle and the system 11 is incorporated into the autonomous vehicle 10. The autonomous vehicle 10 is, for example, a vehicle 10 that is automatically controlled to transport passengers from one location to another. The vehicle 10 is depicted as a passenger car in the illustrated embodiment, but it should be understood that any other vehicle may also be used, including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), etc. In an exemplary embodiment, the vehicle 10 is equipped with a so-called level 4 or level 5 automation system. The level 4 system represents "high automation" and refers to the driving mode-specific performance of the autonomous driving system in all aspects of dynamic driving tasks, even if the human driver does not respond appropriately to the intervention request. The level 5 system represents "full automation" and refers to the full-time performance of the autonomous driving system in all aspects of dynamic driving tasks under all road and environmental conditions that the human driver can manage.

[0045] As shown, the vehicle 10 generally includes a propulsion system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, a vehicle controller 34, and a communication system 36. In an embodiment where the vehicle 10 is an electric vehicle, the transmission system 22 may not be present. In various embodiments, the propulsion system 20 may include an internal combustion engine, an electric motor such as a traction motor, and / or a fuel cell propulsion system. The transmission system 22 is configured to transmit power from the propulsion system 20 to the front wheels 16 and rear wheels 18 of the vehicle according to a selectable speed ratio. According to various embodiments, the transmission system 22 may include a stepped automatic transmission, a continuously variable transmission, or other suitable transmissions. The braking system 26 is configured to provide braking torque to the front wheels 16 and rear wheels 18 of the vehicle. In various embodiments, the braking system 26 may include friction brakes, wire control brakes, regenerative braking systems (e.g., motors), and / or other suitable braking systems. The steering system 24 affects the position of the front wheels 16 and rear wheels 18. Although depicted as including a steering wheel for illustrative purposes, in some embodiments contemplated within the scope of the present disclosure, the steering system 24 may not include a steering wheel.

[0046] The sensor system 28 includes one or more sensing devices 40a-40n that sense observable conditions of the external environment and / or the internal environment of the autonomous vehicle 10. The sensing devices 40a-40n may include, but are not limited to, radar, lidar, global positioning system, optical camera, thermal camera, ultrasonic sensor, and / or other sensors. The camera may include two or more digital cameras spaced at a selected distance from each other, wherein the two or more digital cameras are used to obtain stereo images of the surrounding environment so as to obtain a three-dimensional image. The plurality of sensing devices 40a-40n are used to determine information about the environment surrounding the vehicle 10. In an exemplary embodiment, the plurality of sensing devices 40a-40n include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor. In another exemplary embodiment, the plurality of sensing devices 40a-40n also include sensors for determining information about the environment surrounding the vehicle 10, such as an ambient air temperature sensor, an atmospheric pressure sensor, and / or a camera and / or video camera, which is positioned to observe the environment in front of the vehicle 10. In another exemplary embodiment, at least one of the plurality of sensing devices 40a-40n is capable of measuring distances in the environment surrounding the vehicle 10. In a non-limiting example in which the plurality of sensing devices 40a-40n include cameras, the plurality of sensing devices 40a-40n measure distances using an image processing algorithm configured to process images from the cameras and determine distances between objects. In another non-limiting example, the plurality of vehicle sensors 40a-40n include a stereo camera with distance measurement capabilities. In one example, at least one of the plurality of sensing devices 40a-40n is fixed to the interior of the vehicle 10, for example, fixed in the roof of the vehicle 10, and can be observed through the windshield 56 of the vehicle 10. In another example, at least one of the plurality of sensing devices 40a-40n is fixed to the exterior of the vehicle 10, for example, fixed on the roof of the vehicle 10, and can observe the environment surrounding the vehicle 10. It should be understood that various additional types of sensing devices, such as LiDAR sensors, ultrasonic ranging sensors, radar sensors, and / or time-of-flight sensors are within the scope of the present disclosure. The actuator system 30 includes one or more actuator devices 42 a - 42 n that control one or more features of the vehicle 10 , such as, but not limited to, the propulsion system 20 , the transmission system 22 , the steering system 24 , and the braking system 26 .

[0047] The vehicle controller 34 includes at least one processor 44 and a computer readable storage device or medium 46. The at least one data processor 44 can be any custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor in a plurality of processors associated with the vehicle controller 34, a semiconductor based microprocessor (in the form of a microchip or chipset), a macroprocessor, any combination thereof, or any device generally used to execute instructions. The computer readable storage device or medium 46 can include volatile and non-volatile storage devices such as read-only memory (ROM), random access memory (RAM), and keep alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the at least one data processor 44 is powered off. The computer readable storage device or medium 46 can be implemented using any of a variety of known storage devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or any other electrical, magnetic, optical, or combination storage device capable of storing data used by the controller 34 in controlling the vehicle 10, some of which represents executable instructions.

[0048] The instructions may include one or more separate programs, each of which includes an ordered list of executable instructions for implementing logical functions. When executed by at least one processor 44, the instructions receive and process signals from the sensor system 28, execute logic, calculations, methods and / or algorithms for automatically controlling components of the vehicle 10, and generate control signals to the actuator system 30 to automatically control components of the vehicle 10 based on the logic, calculations, methods and / or algorithms. Although Figure 1 Only one controller 34 is shown, but embodiments of the vehicle 10 may include any number of controllers 34 that communicate via any suitable communication medium or combination of communication mediums and collaborate to process sensor signals, execute logic, calculations, methods and / or algorithms, and generate control signals that automatically control the functions of the autonomous vehicle 10.

[0049] In various embodiments, one or more instructions of the vehicle controller 34 are embodied in the trajectory planning system and, when executed by at least one data processor 44, generate trajectory outputs that address the kinematic and dynamic constraints of the environment. For example, the instructions receive process sensor and map data as input. The instructions implement a graph-based approach and customized cost functions to handle different road scenarios of cities and highways.

[0050] The communication system 36 is configured to wirelessly communicate information with other remote entities 48, such as, but not limited to, other vehicles ("V2V" communication), infrastructure ("V2I" communication), remote systems, remote servers, cloud computers, and / or personal devices. In an exemplary embodiment, the communication system 36 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communications. However, additional or alternative communication methods such as dedicated short-range communication (DSRC) channels are also considered to be within the scope of the present disclosure. A DSRC channel refers to a one-way or two-way short-range to medium-range wireless communication channel designed specifically for automotive use and a set of corresponding protocols and standards.

[0051] The vehicle controller 34 is a non-universal electronic control device having a pre-programmed digital computer or processor, a memory or non-transitory computer readable medium for storing data such as control logic, software applications, instructions, computer code, data, look-up tables, etc., and a transceiver [or input / output port]. Computer readable media include any type of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard drive, compact disk (CD), digital video disk (DVD), or any other type of memory. "Non-transitory" computer readable media excludes wired, wireless, optical or other communication links that transmit temporary electrical or other signals. Non-transitory computer readable media include media that can permanently store data and media that can store data and then be rewritten, such as rewritable optical disks or erasable storage devices. Computer code includes any type of program code, including source code, object code, and executable code.

[0052] refer to Figure 2 In an exemplary embodiment, the system 11 is a hybrid projector system 11 that includes a digital optical projector (DLP) 50 adapted to project a first graphic image 52 onto an interior surface 54 of a windshield 56 of the vehicle 10 and a vector graphics projector (VGP) 58 adapted to project a second graphic image 60 onto an interior surface 54 of the windshield 56 of the vehicle 10. It should be understood that the system 11 may be used without the DLP 50 without departing from the novel features of the present disclosure.

[0053] The DLP 50 projects a first graphic image 52 that includes various information / infotainment applications. The DLP 50 is capable of projecting two-dimensional and three-dimensional images depicting objects such as pedestrians, buildings, and other vehicles in the environment surrounding the vehicle 10. In an exemplary embodiment, the DLP 50 includes a light source 62 adapted to project an excitation light 64, a focusing lens 66 adapted to focus the excitation light 64 from the light source 62, a color filter 68 (color wheel) adapted to separate the focused excitation light 64 into red, green, and blue light, a shaping lens 70 adapted to focus the excitation light 64 passing through the color filter 68, a digital micromirror device (DMD) 72 adapted to redirect the excitation light 64, and a projection lens 74 adapted to receive the excitation light 64 from the digital micromirror device (DMD) 72 and project the excitation light 64 onto the inner surface 54 of the windshield 56.

[0054] refer to Figure 3 , the vehicle controller 34 includes a DLP engine 76. The DLP engine 76 includes a programmable processor that includes programming for monitoring various inputs and determining which information is suitable for display. The DLP engine 76 can communicate directly with various systems and components, or the DLP engine 76 can communicate via a LAN / CAN system instead or in addition. The DLP engine 76 utilizes information about the operating environment of the vehicle 10 derived from multiple inputs. Various sensing devices 40a-40n collect and provide information. The sensing devices 40a-40n include cameras or image capture devices that capture periodic or sequential images representing the view of the vehicle 10. The windshield 56 is equipped with features that can display images projected thereon while remaining transparent or substantially transparent, so that the occupants of the vehicle 10 can clearly observe the exterior of the vehicle 10 through the windshield 56. It should be understood that, as shown in the figure, the windshield 56 is located at the front of the vehicle 10, and other surfaces within the vehicle 10 can also be used for projection, including side windows and rear windows. Additionally, the view on the front windshield 56 may continue as a continuous image onto the front vehicle "A-pillars" and onto the side windows.

[0055] The DLP engine 76 includes display software or programming to convert requests to display information from the DLP engine 76 in a graphical representation describing the information. The DLP engine 76 includes programming to compensate for the curved and angled surface of the windshield 56 and any other surface onto which the first graphic image 52 is to be projected. The DLP engine 76 controls the light source 62, which includes a laser or projector device that produces the excitation light 64 to project the first graphic image 52.

[0056] The occupant monitoring system 78 includes sensors known in the art to simulate the position of the occupant's head and further simulate the orientation or gaze position of the occupant's eyes. Based on the output of the occupant monitoring system 78 and the input data of the tracked position information about the environment surrounding the vehicle 10, the DLP engine 76 can accurately position the first graphic image 52 so that the occupant sees the first graphic image 52 overlaid with the visual image through the windshield 56.

[0057] The above-described system 11 includes eye sensing and head sensing devices that can estimate eye position and can register an image on the windshield 56 so that the image corresponds to the operator's field of view. However, it should be understood that the estimation of head and eye position can be achieved by a variety of methods. For example, in a process similar to adjusting a rearview mirror, the operator can use a calibration routine to align the graphics with the detected object when entering the vehicle. In another embodiment, the longitudinal seat position in the vehicle 10 can be used to estimate the position of the driver's head. In another embodiment, manual adjustment of one or more rearview mirrors can be used to estimate the position of the operator's eyes. It should be understood that a combination of methods such as seat position and mirror adjustment angle can be used to estimate the operator's head position with improved accuracy. Many methods for achieving accurate registration of the graphics on the windshield 56 are contemplated, and the present disclosure is not intended to be limited to the specific embodiments described herein.

[0058] The windshield 56 serves as a medium through which the relevant features can be observed, and also serves as a display device on which the first graphic image 52 can be displayed. The windshield 56 is transparent and is capable of displaying the image projected by the excitation light 64. Figure 4 and Figure 5 , the occupant 80 can see any object (e.g., cube 82) through the substrate 84 located on the windshield 56. The substrate 84 can be transparent or substantially transparent. When the occupant 80 sees the arbitrary object 82 through the substrate 84, the occupant 80 can also see the image (e.g., circle 86 and triangle 88) created at the substrate 84. The substrate 84 can be a portion of the windshield 56, a glass substrate, a plastic substrate, a polymer substrate, or other transparent (or substantially transparent) medium that will be understood by those of ordinary skill in the art. Other substrates can supplement the substrate 84 to provide coloring, substrate protection, light filtering (e.g., filtering external ultraviolet light), and other functions.

[0059] Figure 5The illumination of the transparent display is depicted with excitation light 64 (e.g., ultraviolet light or infrared light) from a light source 62. The substrate 84 receives the excitation light 64 from the light source 62. The received excitation light 64 can be absorbed by the luminescent material 90 at the substrate 84. When the luminescent material receives the excitation light 64, the luminescent material 90 emits visible light. Therefore, by selectively illuminating the substrate 84 with the excitation light 64, an image (e.g., a circle 86 and a triangle 88) can be produced on the substrate 84.

[0060] In an exemplary embodiment, the luminescent material 90 includes a transparent phosphor embedded in the substrate 84. The transparent phosphor is a luminescent particle that emits fluorescence in response to excitation by the excitation light 64. In an exemplary embodiment, the transparent phosphor is a red, green and blue (RGB) phosphor, allowing full color. The use of monochromatic and / or dual-color phosphors is also within the scope of the present disclosure. When the excitation light 64 is absorbed by the transparent phosphor, the transparent phosphor emits visible light. The excitation light 64 is provided by the light source 62. The use of the substrate 84 and the luminescent material 90 to display graphics is discussed in more detail in U.S. application Ser. No. 17 / 749,464, entitled “Mixed Augmented Reality Head-Up Display for Creating Edge-to-Edge Augmented Reality Views,” filed on May 20, 2022, the entire contents of which are incorporated herein by reference.

[0061] According to an embodiment of the present disclosure, the excitation light 64 may be ultraviolet light. If the excitation light 64 is ultraviolet light, when the luminescent material 90 emits visible light in response to the ultraviolet light, a down-conversion physical phenomenon occurs. Specifically, ultraviolet light has a shorter wavelength and higher energy than visible light. Therefore, when the luminescent material 90 absorbs ultraviolet light and emits lower energy visible light, the ultraviolet light is down-converted into visible light because its energy level decreases when the ultraviolet light is converted into visible light.

[0062] It should be understood that the excitation wavelengths described herein are specific for a given phosphor sheet.Other technologies may also be used, such as holographic grating (Bragg grating) based technologies using visible light projectors, where RGB LEDs are used to make vector graphics projectors and RGB-DLP projectors.

[0063] like Figure 2 and Figure 3As shown, the excitation light 64 is output by the light source 62 of the DLP 50. In an exemplary embodiment, the light source 62 is a micromirror array (MMA) projector (e.g., a digital light processing (DLP) projector 50). In the DLP 50, the first graphic image 52 is created by microscopic small mirrors arranged in a matrix on a semiconductor chip in the DMD 72. The MMA projector that outputs ultraviolet light can be similar to the MMA projector that outputs visible light, except that the color filter 68 has a filter customized for the ultraviolet spectrum. In other embodiments, the DLP 50 is a liquid crystal display (LCD) projector. In an embodiment, the DLP 50 can be a liquid crystal on silicon (LCOS) projector. In an embodiment, the DLP 50 can be an analog projector (e.g., a slide projector or a movie film projector). Those of ordinary skill in the art will understand other types of projectors that can be used to project ultraviolet light onto the substrate 84.

[0064] refer to Figure 6 , which is a schematic diagram of an exemplary embodiment of a VGP 58. The VGP 58 generally includes a first light source module 92a, a second light source module 92b, a third light source module 92c, a fiber coupler 94, and an output module 96.

[0065] The components of VGP 58 (i.e., first light source module 92a, second light source module 92b, third light source module 92c, fiber coupler 94, and output module 96) are optically connected to each other. Within the scope of the present disclosure, optical connection means that light can propagate through the space between components and interact with the optical properties of each component. Optical connection can be promoted by positioning the first light source module 92a, the second light source module 92b, the third light source module 92c, the fiber coupler 94, and the output module 96 so that light is transmitted between components due to the following reasons: the inherent reflection and / or refraction properties of the components and / or the properties of the medium (e.g., air) between the components. Optical connection can also be promoted by using optical waveguides. Within the scope of the present disclosure, an optical waveguide is a physical structure designed to limit and guide light along a specified path (i.e., between the first light source module 92a, the second light source module 92b, the third light source module 92c, the fiber coupler 94, and the output module 96). In a non-limiting example, multiple optical fiber cables 98a, 98b, 98c, 98d are used to establish optical connection. In an exemplary embodiment, the optical fiber cables 98a, 98b, 98c, 98d are optoelectronic cables for transmitting light, which act as optical waveguides. In a non-limiting example, the optical fiber cables 98a, 98b, 98c, 98d contain multiple strands of glass or plastic fibers. The fibers are bundled together and surrounded by a protective coating. The first optical fiber cable 98a is used to establish optical communication between the first light source module 92a and the optical fiber coupler 94. The second optical fiber cable 98b is used to establish optical communication between the second light source module 92b and the optical fiber coupler 94. The third optical fiber cable 98c is used to establish optical communication between the third light source module 92c and the optical fiber coupler 94. The fourth optical fiber cable 98d is used to establish optical communication between the optical fiber coupler 94 and the output module 96. It should also be understood that various additional methods for establishing optical communication between optical components, including, for example, planar waveguides, rib waveguides, channel waveguides, hollow waveguides, etc., are all within the scope of the present disclosure.

[0066] The first light source module 92a includes a light emitting diode (LED) array 100, a filter 102, and a focusing lens 104. The LED array 100 is used to provide a light source for the first light source module 92a. In an exemplary embodiment, the LED array 100 includes a plurality of light emitting diodes (LEDs) (not shown) electrically connected in a series or parallel configuration. The LED array 100 also includes at least two electrical terminals (not shown) electrically connected to the plurality of LEDs. The plurality of LEDs are arranged to achieve a high density of LEDs in a relatively small area. In a non-limiting example, the plurality of LEDs are arranged in a grid pattern. In another non-limiting example, the plurality of LEDs are arranged in a radial pattern. The plurality of LEDs are mechanically fixed to the substrate to provide structural stability for the arrangement of the plurality of LEDs. In a non-limiting example, the substrate is a printed circuit board (PCB). In another non-limiting example, the substrate is a metal core printed circuit board (MCPCB) including a metal layer, which contributes to the thermal management of the plurality of LEDs by heat dissipation.

[0067] When current is passed through the plurality of LEDs using electrical terminals, the plurality of LEDs emit light. The light emitted by the plurality of LEDs of the LED array 100 is referred to as source light 106. In an exemplary embodiment, the source light 106 is polychromatic, meaning that the source light 106 contains light of multiple wavelengths (e.g., light having a wavelength between 380 nanometers and 460 nanometers). In another exemplary embodiment, the source light 106 is monochromatic, meaning that the source light 106 contains only light of a single wavelength (e.g., 380 nanometers, 405 nanometers, or 460 nanometers). It should be understood that the aforementioned wavelengths and wavelength ranges LED array 100 is configured so that the source light 106 is incident on the filter 102.

[0068] The optical filter 102 is used to filter the source light 106. In an exemplary embodiment, the optical filter 102 is configured to receive the source light 106 and transmit the filtered light 108. Within the scope of the present disclosure, the filtered light 108 is light having only wavelengths within the passband of the optical filter 102. In a non-limiting example, the optical filter 102 includes a substrate material (e.g., glass or quartz) on which one or more thin films are deposited. The thin film layers are designed to absorb or refract light of specific wavelengths. In a first exemplary embodiment, the optical filter 102 is an interferometric optical bandpass filter. In a non-limiting example, the interferometric optical bandpass filter includes alternating dielectric layers of materials having different refractive indices. The alternating layers are configured to create an interference pattern that enhances the transmission of wavelengths within the passband and suppresses other wavelengths (similar to a Fabry-Perot interferometer).

[0069] In another exemplary embodiment, the optical filter 102 is an absorptive optical bandpass filter. In a non-limiting example, the absorptive optical bandpass filter includes one or more layers of absorptive dielectric material deposited on a substrate. The absorptive layer dissipates unwanted wavelengths by absorption while transmitting wavelengths within the passband. In another exemplary embodiment, the passband of the filter 102 includes only monochromatic light having a single wavelength (e.g., 380 nanometers, 405 nanometers, 460 nanometers). Therefore, the filtered light 108 only includes light having a single wavelength. It should be understood that the aforementioned wavelengths are merely exemplary in nature, and the passband of the filter 102 may include other wavelengths without departing from the scope of the present disclosure. The optical filter 102 is configured such that the filtered light 108 is incident on the focusing lens 104.

[0070] The focusing lens 104 is used to converge the filtered light 108 to produce a narrow light beam 110. In an exemplary embodiment, the focusing lens 104 includes a curved transparent substrate (e.g., glass or other transparent material). When the filtered light 108 passes through the focusing lens 104, the curvature of the focusing lens 104 causes the filtered light 108 to be refracted toward a focal point 112. The position of the focal point 112 is determined by the curvature and refractive index of the focusing lens 104. The focal point 112 of the focusing lens 104 is the endpoint of the first fiber optic cable 98a, so that the narrow light beam 110 is transmitted from the focusing lens 104 to the fiber optic coupler 94 via the first fiber optic cable 98a.

[0071] It should be understood that the second light source module 92b and the third light source module 92c each include the same components as the first light source module 92a (i.e., the LED array 100, the filter 102, and the focusing lens 104). Therefore, the description provided above for the first light source module 92a also applies to the second light source module 92b and the third light source module 92c. It should be understood by those skilled in the art that the VGP 58 may include any suitable number of light source modules. Although Figure 2 A first light source module 92a and a second light source module 92b are shown. Figure 6 and Figure 7 A first light source module 92a, a second light source module 92b, and a third light source module 92c are shown, but other numbers of light source modules may be used without departing from the scope of the present invention.

[0072] In an exemplary embodiment, each of the first light source module 92a, the second light source module 92b, and the third light source module 92c is configured to generate a narrow light beam 110 having a different wavelength. In a non-limiting example, the first light source module 92a generates a narrow light beam 110 having a wavelength of 380 nanometers. The second light source module 92b generates a narrow light beam 110 having a wavelength of 405 nanometers. The third light source module 92c generates a narrow light beam 110 having a wavelength of 460 nanometers. It should be understood that the first light source module 92a, the second light source module 92b, and the third light source module 92c can be configured to generate narrow light beams 110 having different wavelengths without departing from the scope of the present disclosure.

[0073] The fiber coupler 94 is used to combine the narrow light beams 110 from each of the first light source module 92a, the second light source module 92b, and the third light source module 92c. In an exemplary embodiment, the fiber coupler 94 includes a first input port 114a optically connected to the first light source module 92a via a first optical fiber cable 98a, a second input port 114b optically connected to the second light source module 92b via a second optical fiber cable 98b, and a third input port 114c optically connected to the third light source module 92c via a third optical fiber cable 98c. The fiber coupler 94 also includes an output port 116. Each of the input ports 114a, 114b, 114c and the output port 116 includes an optical fiber (not shown) that is aligned and fused with the optical fibers of the other ports. When light is introduced into one of the input ports 114a, 114b, 114c, the light propagates along the optical fiber and is transmitted to the output port 116. Thus, the intensity of the narrow beam 110 from each of the first, second, and third light source modules 92a, 92b, 92c is summed at the output port 116 of the fiber coupler 94. The output port 116 is in optical communication with the output module 96 via the fourth fiber optic cable 98d.

[0074] The output module 96 is used to direct the narrow light beam 110 from the output port 116 of the fiber coupler 94 so as to project the second graphical image 60. The output module 96 includes an optical collimator 118 and an optoelectronic mirror 120.

[0075] The optical collimator 118 is used to produce collimated light 122 from the narrow light beam 110. In an exemplary embodiment, the optical collimator 118 includes a collimating lens, which is used to focus the narrow light beam 110 into a parallel (i.e., collimated) light beam. In a non-limiting example, the collimating lens is a plano-convex lens or a bi-convex lens, which is configured to produce a desired beam divergence and collimation distance. The optical collimator 118 works by refracting the narrow light beam 110 from the fiber coupler 94 using the collimating lens to produce the collimated light 122. The optical collimator 118 is optically connected to the photoelectric mirror 120.

[0076] The photoelectric mirror 120 is used to guide the collimated light 122 based on the electrical signal. Within the scope of the present disclosure, the photoelectric mirror 120 is an electromechanical device (e.g., a micro-electromechanical system) that is configured to guide light with high precision, accuracy, and speed in response to the electrical signal. In an exemplary embodiment, the photoelectric mirror 120 includes a digital micromirror device (DMD). A DMD is an optical micro-electromechanical system that includes a series of mirrors that can be individually tilted to reflect light in a desired direction. In a non-limiting example, the DMD is used to create a high-resolution image by reflecting the collimated light 122 onto a projection surface. The tilted mirrors on the DMD are controlled by an electronic driver that quickly adjusts the mirrors to project the second graphic image 60 onto the windshield 56.

[0077] In another exemplary embodiment, the electro-optical mirror 120 is a micro-electro-mechanical scanning mirror. In an exemplary embodiment, the micro-electro-mechanical scanning mirror is an electro-optical device that uses at least one mirror to create a high-resolution image. The at least one mirror can be tilted to reflect the collimated light 122 in a desired direction. The tilting action of the at least one mirror is controlled by an electronic driver, which quickly adjusts the mirror to project the second graphic image 60 onto the windshield 56.

[0078] In yet another exemplary embodiment, the electro-optical mirror 120 is a galvanometer mirror. In an exemplary embodiment, the galvanometer mirror is an electro-optical device that utilizes at least one mirror that rotates about an axis to change the direction of the collimated light 122. At least one of the mirrors in the galvanometer mirror is controlled by a galvanometer, which is a motor that converts an electrical current signal into mechanical motion. The galvanometer causes the at least one mirror to rotate rapidly on an axis, allowing the collimated light 122 to be redirected to scan the collimated light 122 across the windshield 56 and display the second graphic image 60. The light transmitted to the windshield 56 by the electro-optical mirror 120 is referred to as projected light 124.

[0079] It should be understood that in order to adjust the overall light intensity of the projected light 124, the VGP 58 may include more or less than three light source modules without departing from the scope of the present disclosure.

[0080] refer to Figure 7 Another exemplary embodiment of the VGP 58 includes a first light source module 92a, a second light source module 92b, a third light source module 92c, a first dichroic mirror 126a, a second dichroic mirror 126b, a third dichroic mirror 126c, and an output module 96. Figure 6 The description of the first light source module 92a, the second light source module 92b, the third light source module 92c and the output module 96 is also applicable to Figure 7 In place of the optical fiber cables 98a, 98b, 98c, 98d and the optical fiber coupler 94, Figure 7The VGP 58 of the illustrated exemplary embodiment utilizes a first dichroic mirror 126a, a second dichroic mirror 126b, and a third dichroic mirror 126c to combine the narrow light beams 110 from each of the first light source module 92a, the second light source module 92b, and the third light source module 92c.

[0081] Within the scope of the present disclosure, a dichroic mirror is a mirror that reflects light of a predetermined wavelength and transmits light of other wavelengths. In an exemplary embodiment, the first dichroic mirror 126a, the second dichroic mirror 126b, and the third dichroic mirror 126c include a base material (e.g., glass, quartz) coated with a plurality of dielectric layers. Each of the plurality of dielectric layers is selected to have a specific refractive index and thickness so that the first dichroic mirror 126a, the second dichroic mirror 126b, and the third dichroic mirror 126c reflect light of a specific wavelength and transmit light of other wavelengths.

[0082] In a non-limiting example, the first light source module 92a generates a narrow light beam 110 having a wavelength of 380 nanometers. The second light source module 92b generates a narrow light beam 110 having a wavelength of 405 nanometers. The third light source module 92c generates a narrow light beam 110 having a wavelength of 460 nanometers. Therefore, the first dichroic mirror 126a is configured to reflect the narrow light beam 110 having a wavelength of 380 nanometers and transmit light of all other wavelengths. The second dichroic mirror 126b is configured to reflect the narrow light beam 110 having a wavelength of 405 nanometers and transmit light of all other wavelengths. The third dichroic mirror 126c is configured to reflect the narrow light beam 110 having a wavelength of 460 nanometers and transmit light of all other wavelengths. It should be understood that the aforementioned wavelengths are merely exemplary in nature.

[0083] The first dichroic mirror 126a is positioned so that the narrow light beam 110 from the first light source module 92a reflected by the first dichroic mirror 126a is incident on the second dichroic mirror 126b. The second dichroic mirror 126b is positioned so that the narrow light beam 110 from the second light source module 92b reflected by the second dichroic mirror 126b is incident on the third dichroic mirror 126c. The third dichroic mirror 126c is positioned so that the narrow light beam 110 from the third light source module 92c reflected by the third dichroic mirror 126c is incident on the output module 96. In an exemplary embodiment, each of the first dichroic mirror 126a, the second dichroic mirror 126b, and the third dichroic mirror 126c is positioned to form a 45 degree angle with the incident angle of the narrow light beam 110 and is aligned along the central axis, as shown in FIG. Figure 7 shown.

[0084] Therefore, the narrow light beam 110 with a wavelength of 380 nanometers from the first light source module 92a is reflected by the first dichroic mirror 126a and incident on the second dichroic mirror 126b, as shown in the first line 128a. The narrow light beam 110 with a wavelength of 380 nanometers from the first light source module 92a is transmitted through the second dichroic mirror 126b and incident on the third dichroic mirror 126c, as shown in the second line 128b. The narrow light beam 110 with a wavelength of 405 nanometers from the second light source module 92b is reflected by the second dichroic mirror 126b and incident on the third dichroic mirror 126c, as shown in the second line 128b. Therefore, as shown in the figure, the narrow light beam 110 with a wavelength of 380 nanometers from the first light source module 92a and the narrow light beam 110 with a wavelength of 405 nanometers from the second light source module 92b are both incident on the third dichroic mirror 126c, as shown in the second line 128b. The narrow light beam 110 with a wavelength of 380 nanometers from the first light source module 92a and the narrow light beam 110 with a wavelength of 405 nanometers from the second light source module 92b are transmitted by the third dichroic mirror 126c and incident on the output module 96, as shown by the third line 128c. The narrow light beam 110 with a wavelength of 460 nanometers from the third light source module 92c is reflected by the third dichroic mirror 126c and incident on the output module 96, as shown by the third line 128c. Therefore, the narrow light beam 110 with a wavelength of 380 nanometers from the first light source module 92a, the narrow light beam 110 with a wavelength of 405 nanometers from the second light source module 92b, and the narrow light beam 110 with a wavelength of 460 nanometers from the third light source module 92c are incident on the output module 96, as shown by the third line 128c.

[0085] The total narrow light beam (i.e., the narrow light beam 110 with a wavelength of 380 nanometers from the first light source module 92a, the narrow light beam 110 with a wavelength of 405 nanometers from the second light source module 92b, and the narrow light beam 110 with a wavelength of 460 nanometers from the third light source module 92c) is incident on the output module 96, collimated by the optical collimator 118 and guided by the photoelectric mirror 120, as described above.

[0086] Reference again Figure 3, the vehicle controller 34 includes a VGP engine 130. The VGP engine 130 includes a programmable processor that includes programming for monitoring various inputs and determining which information is suitable for display. The VGP engine 130 can communicate directly with various systems and components, or the VGP engine 130 can communicate alternatively or additionally through a LAN / CAN system. The VGP engine 130 utilizes information about the operating environment of the vehicle 10 derived from multiple inputs. Various sensing devices 40a-40n collect and provide information. The sensing devices 40a-40n include cameras or image capture devices that capture periodic or sequential images representing views of the vehicle 10. The windshield 56 is equipped with features that can display images projected thereon while remaining transparent or substantially transparent, so that the occupants of the vehicle 10 can clearly observe the exterior of the vehicle 10 through the windshield 56.

[0087] The processor of the VGP engine 130 includes a non-temporary computer-readable storage device or medium. The processor can be a customized or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor in a plurality of processors associated with the VGP engine 130, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or a device generally used to execute instructions. The computer-readable storage device or medium may include, for example, volatile and non-volatile storage in a read-only memory (ROM), a random access memory (RAM), and a keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor is powered off. The computer-readable storage device or medium can be implemented using a variety of storage devices, such as PROM (programmable read-only memory), EPROM (electric PROM), EEPROM (electrically erasable PROM), flash memory, or other electrical, magnetic, optical or combined storage devices capable of storing data, some of which represent executable instructions, used by the VGP engine 130 to control various systems of the vehicle 10. The VGP engine 130 can also be composed of a plurality of controllers electrically connected to each other. The VGP engine 130 may be interconnected with additional systems and / or controllers of the vehicle 10 , allowing the VGP engine 130 to access data such as the speed, acceleration, braking, and steering angle of the vehicle 10 .

[0088] The VGP engine 130 is in electrical communication with the plurality of sensing devices 40a-40n via the vehicle controller 34. In an exemplary embodiment, the electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a serial peripheral interface (SPI) network, or a serial peripheral interface (SPI) network, etc. It should be understood that various additional wired and wireless technologies and communication protocols for communicating with the VGP engine 130 are within the scope of the present disclosure.

[0089] Similar to the DLP 50, the VGP 58 projects light in a predetermined pattern to excite the light-emitting material 90 (transparent phosphor) within the substrate 84 on the windshield 56 to produce the second graphic image 60 on the windshield 56. In an exemplary embodiment, both the DLP 50 and the VGP 58 are disposed near the roof of the vehicle 10. In a non-limiting example, the 380 nanometer light of the projected light 124 excites the red transparent phosphor. The 405 nanometer light of the projected light 124 excites the blue transparent phosphor. The 460 nanometer light of the projected light 124 excites the green transparent phosphor. It should be understood that in some embodiments, the transparent phosphors can be excited by excitation light of other wavelengths. The DLP 50 or the VGP 58 can be configured to generate excitation light of any wavelength required to excite the transparent phosphor.

[0090] In an exemplary embodiment, the vehicle controller 34 uses the system 11 to display graphics that indicate conditions of the vehicle 10 and / or the environment surrounding the vehicle 10 as detected by the plurality of sensing devices 40a-40n. In an exemplary embodiment, the vehicle controller 34 uses the DLP 50 to display a first graphical image 52, which may include two-dimensional and three-dimensional depictions of the environment surrounding the vehicle 10, or images with vehicle information or infotainment content that are overlaid "on top" of the occupant's view of the environment outside the vehicle 10. The DLP is capable of providing a first graphical image that spans across the entire windshield 56. Additionally, the vehicle controller 34 uses the VGP 58 to display a second graphical image 60, which may include quick response alerts to critical safety conditions detected by the sensor system 28 of the vehicle 10. The VGP 58 is not capable of displaying three-dimensional graphical images, however, the VGP 58 is capable of providing immediate and brighter alerts by providing enhanced graphics, such as highlighting pedestrians or points of interest. Reference Figure 1 , a second graphic image 60 may be displayed adjacent to the first graphic image 52, such as an arrow intended to draw the occupant's attention to the left. Figure 8 , the second graphic image 60 may be overlaid on the first graphic image 52, for example Figure 8 A highlighted box around the cyclist.

[0091] Based on the output of the occupant monitoring system 78 and the input data of the tracked position information about the environment surrounding the vehicle 10, the DLP engine 76 can accurately position the first graphic image 52 so that the occupant sees the first graphic image 52 overlaid with the visual image through the windshield 56. In addition, the VGP engine 130 can accurately position the second graphic image 60 relative to the first graphic image 52 so that the occupant sees the second graphic image 60 at an appropriate location proximate to or overlaid on the first graphic image 52.

[0092] In another exemplary embodiment, the second graphic image includes at least one vehicle indicator light. The vehicle indicator lights can convey different types of information. One type of vehicle indicator light or flashes to indicate a fault (such as oil pressure, engine temperature control, charging current, etc.), where the lighting and flashing indicates the progression from a warning to a fault indication. Another type of light is used to remind that a specific service is needed (e.g., an oil change) after a certain time or distance has passed. The hybrid projector system 11 includes a calibration system that is suitable for testing and calibrating the displayed indicator lights projected within the second graphic image. Reference Figure 2 , when the vehicle indicator lights are displayed, the vector graphics projector displays a second graphic image including the vehicle indicator lights at a low position on the windshield.

[0093] refer to Figure 1 , a reference calibration system 140 is shown, which generally includes a reference controller 142 , a reference windshield 144 , a reference vector graphics projector 146 , and a reference camera 148 .

[0094] The reference controller 142 is used to implement a method 200 for calibrating a display of a vehicle, as described below. The reference controller 142 includes at least one processor 150 and a non-transitory computer-readable storage device or medium 152. The processor 150 can be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor in a plurality of processors associated with the reference controller 142, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally a device for executing instructions. The computer-readable storage device or medium 152 can include volatile and non-volatile storage devices such as read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor 150 is powered off. The computer readable storage device or medium 152 may be implemented using a plurality of storage devices, such as a PROM (Programmable Read Only Memory), an EPROM (Electrical PROM), an EEPROM (Electrically Erasable PROM), a flash memory, or other electrical, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable instructions, used by the reference controller 142 to control the system 140. The reference controller 142 may also be comprised of a plurality of controllers in electrical communication with each other. In a non-limiting example, the reference controller 142 is included or integrated in a computer system, such as a server computer, a workstation computer, a personal computer, etc.

[0095] The reference controller 142 is in electrical communication with the reference vector graphics projector 146 and the reference camera 148. In an exemplary embodiment, the electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a serial peripheral interface (SPI) network, or a serial peripheral interface (SPI) network, etc. It should be understood that various additional wired and wireless technologies and communication protocols for communicating with the reference controller 12 are within the scope of the present disclosure.

[0096] The reference windshield 144 is used to provide a reference projection surface for the system 140. In an exemplary embodiment, the reference windshield 144 is selected such that a plurality of variable characteristics of the reference windshield 144 are within a predetermined range for each of the plurality of variable characteristics. Within the scope of the present disclosure, the plurality of variable characteristics include characteristics of the reference windshield 144 that may be affected by manufacturing variations, such as curvature, glass thickness uniformity, glass surface quality, glass coating uniformity, and / or the like. In an exemplary embodiment, the predetermined range for each of the plurality of variable characteristics is determined at least in part based on a statistical analysis of the plurality of windshields. In a non-limiting example, the predetermined range for each of the plurality of variable characteristics is plus or minus one standard deviation from the mean of each of the plurality of variable characteristics. Thus, the reference windshield 144 is selected such that the reference windshield 144 is a statistically accurate approximation of a typical windshield.

[0097] The reference windshield 144 includes a transparent phosphor (not shown) embedded in the reference windshield 144. As described above, with reference to the windshield 56 of the vehicle 10, the transparent phosphor is a luminescent particle that fluoresces in response to being excited by the reference vector graphics projector 146. In an exemplary embodiment, the transparent phosphor is a red, green, and blue (RGB) phosphor. The use of monochromatic and / or bichromatic phosphors is also within the scope of the present disclosure. When the excitation light is absorbed by the transparent phosphor, the transparent phosphor emits visible light. The excitation light is provided by the reference vector graphics projector 146.

[0098] The reference vector graphics projector 146 is used to excite the transparent phosphor in a predetermined pattern to produce a vector graphic on the reference windshield 144. In a first exemplary embodiment, the reference vector graphics projector 146 is a violet / ultraviolet laser projector disposed near the roof of the vehicle 10. In another exemplary embodiment, the reference vector graphics projector 146 is an LED-based projector, such as U.S. application 18 / 186,343, entitled "Graphic Projector for Vehicles," filed on March 20, 2023, the entire contents of which are incorporated herein by reference. The reference vector graphics projector 146 is configured to selectively excite the phosphor of the red, green, or blue transparent light reference windshield 144. The reference vector graphics projector 146 has a projection field 154. In the scope of the present disclosure, the projection field 154 defines the area on which the reference vector graphics projector 146 can project. In a non-limiting example, the projection field 154 is defined by a pixel width and a pixel height (e.g., 1920x1920 pixels).

[0099] The reference vector graphics projector 146 also includes a reference vector graphics projector camera 156. The reference vector graphics projector camera 156 is a camera and / or video camera that views the reference windshield 144 from the perspective of the reference vector graphics projector 146. In an exemplary embodiment, the reference vector graphics projector camera 156 is integrated into the reference vector graphics projector 146, for example, with a lens adjacent to the projection lens of the reference vector graphics projector 146. The reference vector graphics projector 146 is fixedly mounted at a predetermined position and angle relative to the reference windshield 144, thereby eliminating variations caused by the installation or mounting of the reference windshield 144 and the reference vector graphics projector 146. The reference vector graphics projector 146 and the reference vector graphics projector camera 156 are in electrical communication with the reference controller 142, as further described above.

[0100] The reference camera 148 is used to capture images and / or videos of the reference windshield 144. In an exemplary embodiment, the reference camera 148 is a camera and / or video camera positioned to view the reference windshield 144. The reference camera 148 is fixedly mounted at a predetermined position and angle relative to the reference windshield 144. In an exemplary embodiment, the predetermined position of the reference camera 148 is a reference occupant eyellipse position. Within the scope of the present disclosure, the occupant eyellipse position represents the position and orientation of the eyes of a typical occupant when sitting in a vehicle seat. Therefore, the reference occupant eyellipse position is a position relative to the reference windshield 144 at which the occupant eyellipse would be located in the vehicle if the occupant was sitting in the vehicle seat at the same height, distance, orientation, and angle as the reference windshield 144. Therefore, if the reference windshield 144 is installed in a vehicle, the reference camera 148 is configured to view the reference windshield 144 from the perspective of the vehicle occupant. It should be understood that cameras with various sensor types, including, for example, a charge coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor, and / or a high dynamic range (HDR) sensor are within the scope of the present disclosure. In addition, cameras with various lens types, including, for example, wide angle lenses and / or narrow angle lenses, are also within the scope of the present disclosure. Reference camera 148 is in electrical communication with reference controller 142, as further described above.

[0101] The vehicle vector graphics projector 58 also includes a vehicle vector graphics projector camera 158. The vehicle vector graphics projector camera 158 is a camera and / or video camera that views the vehicle windshield 56 from the perspective of the vehicle vector graphics projector 58. In an exemplary embodiment, the vehicle vector graphics projector camera 158 is integrated into the vehicle vector graphics projector 58, such as having a lens adjacent to a projection lens of the vehicle vector graphics projector 58. The vehicle vector graphics projector 58 and the vehicle vector graphics projector camera 158 are in electrical communication with the vehicle controller 34, as further described above.

[0102] In an exemplary embodiment, the vehicle controller 34 uses a vehicle vector graphics projector 58 to display a second graphic image 60 including vehicle indicator lights. In a non-limiting example, the vehicle vector graphics projector 58 is used as a primary instrument to display information such as vehicle speed, coolant temperature, fuel level, charging status, etc., as well as vehicle indicator lights. In an exemplary embodiment, the second graphic image 60 is calibrated prior to display so that distortion of the second graphic image 60 caused by various characteristics is mitigated, as further described below with reference to method 200. Using the vector graphics projector 58 to display graphics is further described in U.S. application Ser. No. 17 / 749,464, filed May 20, 2022, entitled “Mixed Augmented Reality Head-Up Display for Creating Edge-to-Edge Augmented Reality Views,” the entire contents of which are incorporated herein by reference.

[0103] refer to Fig. 9 The method 200 for displaying graphics using the hybrid projector system 11 for a head-up display in a vehicle includes, starting at box 202, projecting a first graphic image 52 onto the inner surface 54 of a windshield 56 of the vehicle 10 using the digital optical projector 50, and moving to box 204, projecting a second graphic image 60 onto the inner surface 54 of the windshield 56 of the vehicle 10 using the vector graphics projector 58. Moving to box 250, the method 200 includes testing and calibrating the vehicle indicator lights displayed in the second graphic image 60 using the calibration system.

[0104] In an exemplary embodiment, projecting the first graphic image 52 onto the inner surface 54 of the windshield 56 of the vehicle 10 using the digital optical projector 50 at box 202 also includes: moving to box 206, projecting excitation light 64 using the light source 62, moving to box 208, focusing the excitation light 64 from the light source 62 with a condenser lens 66, moving to box 210, separating the focused excitation light 64 into red, green and blue light with a color filter 68, moving to box 212, focusing the excitation light 64 passing through the color filter 68 with a shaping lens 70, moving to box 214, redirecting the excitation light 64 with a digital micromirror device (DMD) 72, moving to box 216, receiving the excitation light 64 from the digital micromirror device 72 with a projection lens 74, and moving to box 218, projecting the excitation light 64 onto the inner surface 54 of the windshield 56 with the projection lens 74.

[0105] In an exemplary embodiment, projecting the second graphic image 60 onto the interior surface 54 of the windshield 56 of the vehicle 10 using the vector graphics projector 58 at block 204 further includes: moving to block 220, generating source light 106 using the light emitting diode (LED) array 100 of the light source module 92a, moving to block 222, receiving the source light 106 using the optical bandpass filter 102 of the light source module 92a in optical communication with the LED array 100, moving to block 224, filtering the source light 106 using the bandpass filter 102, and moving to block 226, transmitting the source light 106 having the bandpass filtered light through the bandpass filter 102. The optical filter 102 is used to filter light 108 of a wavelength within the passband of the optical filter 102, moves to frame 228, receives the filtered light 108 using the focusing lens 104 of the light source module 92a optically connected to the filter 102, moves to frame 230, transmits the narrow light beam 110 using the focusing lens 104, moves to frame 232, receives the narrow light beam 110 using the optical collimator 118 optically connected to the light source module 92a, moves to frame 234, transmits collimated light 122 using the optical collimator 118, and moves to frame 236, guides the collimated light 122 using the photoelectric mirror 120 optically connected to the optical collimator 118 to form a pattern.

[0106] In another exemplary embodiment, at box 202, projecting the second graphic image 60 onto the inner surface 54 of the windshield 56 of the vehicle 10 using the vector graphics projector 58 also includes: projecting the second graphic image 60 onto the inner surface 54 of the windshield 56 of the vehicle 10 using a plurality of light source modules 92a, 92b and 92c optically connected to the optical collimator 118 of the vector graphics projector 58.

[0107] In another exemplary embodiment, transmitting the narrow light beam using a focusing lens at box 230 also includes: using fiber optic cable waveguides 98a, 98b, 98c and 98d optically connected to the multiple light source modules 92a, 92b and 92c and the optical collimator 118, transmitting the narrow light beam 110 from each of the multiple light source modules 92a, 92b, 92c to the optical collimator 118.

[0108] In another exemplary embodiment, transmitting the narrow light beam 110 with the focusing lens 104 at box 230 also includes: using a plurality of dichroic mirrors 126a, 126b, 126c (wherein each of the plurality of dichroic mirrors 126a, 126b, 126c is optically connected to one of the plurality of light source modules 92a, 92b, 92c), transmitting the narrow light beam 110 from each of the plurality of light source modules 92a, 92b, 92c to the optical collimator 118, wherein each of the plurality of dichroic mirrors 126a, 126b, 126c is configured to reflect a predetermined wavelength of the narrow light beam 110 and transmit all other wavelengths of the narrow light beam 110.

[0109] In another exemplary embodiment, transmitting the narrow light beam 110 using the focusing lens 104 at box 230 also includes: transmitting the narrow light beam 110 with a wavelength of 380 nanometers through the focusing lens 104 of the first one of the multiple light source modules 92a, transmitting the narrow light beam 110 with a wavelength of 405 nanometers through the focusing lens 104 of the second one of the multiple light source modules 92b, and transmitting the narrow light beam 110 with a wavelength of 460 nanometers through the focusing lens 104 of the third one of the multiple light source modules 92c.

[0110] In an exemplary embodiment, testing and calibrating the vehicle indicator lights displayed in the second graphical image 60 using the calibration system 140 at box 250 also includes: moving to box 252, the calibration system 140 is used to calculate at least one reference value calibration matrix, as further described below. After box 252, the method 200 proceeds to box 254. At box 254, the system 11 is used to calculate the vehicle vector graphics calibration matrix, as further described below. After box 254, the method 200 proceeds to box 256.

[0111] At box 256, the system 11 is used to display graphics based at least in part on the vehicle vector graphics calibration matrix calculated at box 254. In an exemplary embodiment, the vehicle controller 34 identifies an object of interest as described above and determines a graphic to be displayed using the vector graphics projector 58. In a non-limiting example, the object of interest is an obstacle on a road in the environment surrounding the vehicle 10, and the graphic is an animated warning indication including a plurality of video frames displayed in succession. To display each frame of the animated warning indication, the vehicle controller 34 applies the vehicle vector graphics calibration matrix calculated at box 254 to each frame of the animated warning indication. In a non-limiting example, to apply the vehicle TWD calibration matrix, the vehicle controller 34 adds the vehicle vector graphics calibration matrix to the matrix defining each frame of the animated warning indication before projecting each frame of the animated warning indication using the vehicle vector graphics projector 58. Therefore, each frame of the animated warning indication is distortion compensated. After box 256, the method 200 continues to enter the standby state at box 258. It should be appreciated that the method 200 may be repeated for additional areas of the reference windshield 144 and / or the vehicle windshield 56 (ie, additional projection fields 154 ) to compensate and display graphics across the entire windshield surface.

[0112] It should be understood that the method 200 can periodically exit the standby state 258 and repeat some or all of the method steps. In an exemplary embodiment, the method 200 includes a first re-entry point 260 and a second re-entry point 262, from which the method 200 can start under certain conditions. In an exemplary embodiment, if a portion of the system 11 is replaced, the first re-entry point 260 is used. In a non-limiting example, if the vehicle windshield 56 and / or the vehicle vector graphics projector 58 is replaced, the vehicle vector graphics calibration matrix needs to be recalculated at box 254. Therefore, the first re-entry point 260 of the method 200 is used. For example, the method 200 can be started at the first re-entry point 260 by a maintenance technician after replacing the vehicle windshield 56 and / or the vehicle vector graphics projector 58. In an exemplary embodiment, the second re-entry point 262 is used to display the compensation graphics in the normal operation of the system 11. In a non-limiting example, the method 200 exits the standby state 258 and re-enters the method 200 at the second re-entry point 262 on the timer, for example, every three hundred milliseconds.

[0113] refer to Fig.11 , which shows a flow chart of an exemplary embodiment of block 252. Also refer to Fig. 12A , a first exemplary view of the projection field 154 of the system 11 is shown. The exemplary embodiment of box 252 begins at box 300. At box 300, the reference controller 142 projects a reference graphic 160 onto the reference windshield 144 using the reference vector graphics projector 146. Within the scope of the present disclosure, the reference graphic 160 includes a plurality of projection points 162 arranged in a uniform two-dimensional array or grid (i.e., the distance between each of the plurality of perception points and the surrounding points is consistent). Each of the plurality of projection points 162 has an expected x-coordinate and an expected y-coordinate defined relative to the projection field 154. The reference graphic 160 is represented as an expected reference graphic matrix. Within the scope of the present disclosure, the expected reference graphic matrix is ​​a two-dimensional matrix having a plurality of elements. Each of the plurality of elements of the expected reference graphic matrix defines the position of one of the plurality of projection points 162 by storing the expected x-coordinate and the expected y-coordinate of one of the plurality of projection points 162. For Fig. 12A For the exemplary reference pattern 160 shown, the expected reference pattern matrix is:

[0114]

[0115] Where I is the expected reference pattern matrix, x n is the expected x coordinate of one of the plurality of projection points 162 located in column n of the expected reference pattern matrix, and y mis the expected y coordinate of one of the plurality of projection points 162 located in row m of the expected reference pattern matrix. In the row of the expected reference pattern matrix. It should be understood that the reference pattern 160 may have any number of points and the expected reference pattern matrix may have any number of rows and / or columns without departing from the scope of the present disclosure. After block 300, the exemplary embodiment of block 252 proceeds to block 302.

[0116] At block 302, the reference controller 142 uses the reference camera 148 to capture a first reference image 164 of the reference graphic 160 on the reference windshield 144. Due to manufacturing variations of the reference windshield 144 (e.g., variations in curvature, variations in glass thickness uniformity, variations in glass surface quality, variations in glass coating uniformity, etc.), the reference graphic 160 appears distorted in the first reference image 164. In other words, the location of each of the plurality of projected points 162 of the reference graphic 160 does not correspond to the location of each of the plurality of perceived points 166 in the first reference image 164, as shown in FIG. Fig. 12A Within the scope of the present disclosure, the perceived point 166 refers to a point perceived by one of the reference camera 148 , the reference vector graphics camera 156 , or the vehicle vector graphics camera 158 . After block 302 , the exemplary embodiment of block 252 proceeds to block 304 .

[0117] At block 304, the reference controller 142 identifies a plurality of perception points 166 in the first reference image 164 captured at block 302. In an exemplary embodiment, the reference controller 142 uses a computer vision algorithm to identify the plurality of perception points 166 in the first reference image 164. In the scope of the present disclosure, identifying the plurality of perception points 166 means that the reference controller 142 isolates the plurality of perception points 166 from other elements (e.g., background objects, dust particles, camera artifacts, and / or the like) in the first reference image 164 so that the location of each of the plurality of perception points 166 can be determined, as described below. In an exemplary embodiment, the computer vision algorithm is trained by providing it with a plurality of images including points at different locations and angles. After sufficient training of the computer vision algorithm, the reference controller 142 can identify the plurality of perception points 166 in the first reference image 164 with high accuracy and precision. After identifying the plurality of perception points 166 in the first reference image 164, the reference controller 142 determines the actual x-coordinate and the actual y-coordinate of each of the plurality of perception points 166 in the first reference image 164. In an exemplary embodiment, actual coordinates of each of the plurality of perceived points 166 in first reference image 164 are identified by performing pixel measurements for each of the plurality of perceived points 166 relative to projected field 154 in first reference image 164. After block 304, the exemplary embodiment of block 252 proceeds to block 306.

[0118] At block 306, the reference controller 142 generates an actual reference graphic matrix. Within the scope of the present disclosure, the actual reference graphic matrix contains the actual x-coordinates and the actual y-coordinates of each of the plurality of perception points 66 in the first reference image 164, as determined at block 304. After block 306, the exemplary embodiment of block 252 proceeds to block 308.

[0119] At block 308, the reference controller 142 calculates a reference camera deviation matrix. Within the scope of the present disclosure, the reference camera deviation matrix encodes the difference between the reference pattern 160 and the first reference image 164. In an exemplary embodiment, to calculate the reference camera deviation matrix, the actual reference pattern matrix generated at block 306 is subtracted from the expected reference pattern matrix. After block 308, the exemplary embodiment of block 252 proceeds to block 310.

[0120] At block 310 , the reference controller 142 generates a bias-compensated reference pattern matrix by summing the expected reference pattern matrix with the reference camera bias matrix determined at block 308 . After block 310 , the exemplary embodiment of block 252 proceeds to block 312 .

[0121] refer to Fig. 12B , a second exemplary view of the projection field 154 of the system 11 is shown. Fig.11 , Fig. 12A and Fig. 12B At block 312, the reference controller 142 projects the aberration-compensated reference graphic 168 onto the reference windshield 144 using the reference vector graphics projector 146. The location of each of the plurality of projected points 162 of the aberration-compensated reference graphic is defined by the aberration-compensated reference graphic matrix generated at block 310. After block 312, the exemplary embodiment of block 252 proceeds to block 314.

[0122] At block 314, the reference controller 14 uses the reference vector graphics projector camera 156 to capture a second reference image 170 of the deviation-compensated reference graphic 168 on the reference windshield 144. Due to the predetermined position and angle of the reference vector graphics projector camera 156 relative to the reference windshield 144, the deviation-compensated reference graphic 168 appears distorted in the second reference image 170. In other words, the position of each of the plurality of projected points 162 of the deviation-compensated reference graphic 168 does not correspond to the position of each of the plurality of perceived points 166 in the second reference image 170, as shown in FIG. Fig. 12B After block 314 , the exemplary embodiment of block 252 proceeds to block 316 .

[0123] At block 316, the reference controller 142 identifies a plurality of perception points 166 in the second reference image 170 captured at block 314. In an exemplary embodiment, the reference controller 142 uses a computer vision algorithm to identify the plurality of perception points 166 in the second reference image 170. Within the scope of the present disclosure, identifying the plurality of perception points 166 means that the reference controller 142 isolates the plurality of perception points 166 from other elements (e.g., background objects, dust particles, camera artifacts, and / or the like) in the second reference image 170 so that the location of each of the plurality of perception points 166 can be determined, as described below. In an exemplary embodiment, the computer vision algorithm is trained by providing it with a plurality of images including points at different locations and angles. After sufficient training of the computer vision algorithm, the reference controller 142 can identify the plurality of perception points 166 in the second reference image 170 with high accuracy and precision. After identifying the plurality of perception points 166 in the second reference image 170, the reference controller 142 determines the actual x-coordinate and the actual y-coordinate of each of the plurality of perception points 66 in the second reference image 170. In an exemplary embodiment, actual coordinates of each of the plurality of perceived points 166 in the second reference image 170 are identified by performing pixel measurements for each of the plurality of perceived points 166 relative to the projected field 154 in the second reference image 170. After block 316, the exemplary embodiment of block 252 proceeds to block 318.

[0124] At block 318, the reference controller 142 generates a reference vector graphics projector camera matrix. Within the scope of the present disclosure, the reference vector graphics projector camera matrix includes the actual x-coordinate and the actual y-coordinate of each of the plurality of perception points 166 in the second reference image 170, as determined at block 316. The reference vector graphics projector camera matrix is ​​also referred to as a type of reference calibration matrix. After block 318, the exemplary embodiment of block 252 proceeds to block 320.

[0125] At block 320, the reference controller 142 determines a reference vector graphics projector camera transformation matrix. Within the scope of the present disclosure, the reference vector graphics projector camera transformation matrix defines a perspective transformation between the perspective of the reference camera 148 and the perspective of the reference vector graphics projector camera 156. The reference vector graphics projector camera transformation matrix is ​​also referred to as a type of reference calibration matrix. In an exemplary embodiment, to determine the reference vector graphics projector camera transformation matrix, the reference vector graphics projector camera matrix determined at block 318 is subtracted from the reference camera bias matrix determined at block 308. After block 320, the exemplary embodiment of block 252 ends and the method 200 continues as described above.

[0126] refer to Fig.13, a flow chart of an exemplary embodiment of block 254 is shown. The exemplary embodiment of block 254 begins at block 402. At block 402, the vehicle controller 34 projects the reference graphic 160 onto the vehicle windshield 56 using the vehicle vector graphics projector 58. After block 402, the exemplary embodiment of block 254 proceeds to block 404.

[0127] At block 404, the vehicle controller 34 uses the vehicle vector graphics projector camera 158 to capture a calibration image of the reference graphic 160 on the vehicle windshield 56. Due to manufacturing variations of the vehicle windshield 56 (e.g., curvature variations, glass thickness uniformity variations, glass surface quality variations, glass coating uniformity variations, etc.) and the position and angle of the vehicle vector graphics projector 58 relative to the vehicle windshield 56, the reference graphic 160 appears distorted in the calibration image. The calibration image is substantially similar to the reference graphic 160 above. Fig. 12B However, due to different manufacturing variations between the reference windshield 144 and the vehicle windshield 56, the distortion of the reference graphic 160 in the calibration image may be different from the distortion of the reference graphic 160 in the second reference image 170. In addition, installation variations (i.e., position and angle) of the vehicle windshield 56 and the vehicle vector graphics projector 58 relative to the reference windshield 144 and the reference vector graphics projector 146 may also introduce distortion that is not present in the second reference image 170. After block 404, the exemplary embodiment of block 254 proceeds to block 406.

[0128] At box 406, the vehicle controller 34 identifies a plurality of perception points in the calibration image captured at box 404. In the scope of the present disclosure, identifying a plurality of perception points means that the vehicle controller 34 isolates the plurality of perception points from other elements in the calibration image (e.g., background objects, dust particles, camera artifacts, etc.) so that the location of each of the plurality of perception points can be determined, as described below. In an exemplary embodiment, the vehicle controller 34 uses a computer vision algorithm to identify the plurality of perception points in the calibration image. In an exemplary embodiment, the computer vision algorithm is trained by providing it with a plurality of images including points at different positions and angles. After sufficient training of the computer vision algorithm, the vehicle controller 34 can identify the plurality of perception points in the calibration image with high accuracy and precision. After identifying the plurality of perception points in the calibration image, the vehicle controller 34 determines the actual x-coordinate and the actual y-coordinate of each of the plurality of perception points in the calibration image. In an exemplary embodiment, the actual coordinates of each of the plurality of perception points in the calibration image are identified by performing pixel measurements on each of the plurality of perception points relative to the projection field 26 in the calibration image. After block 406 , the exemplary embodiment of block 254 proceeds to block 408 .

[0129] At block 408, the vehicle controller 34 generates an actual vehicle vector graphics matrix. Within the scope of the present disclosure, the actual vehicle vector graphics matrix contains the actual x-coordinates and the actual y-coordinates of each of the plurality of perception points in the calibration image, as determined at block 406. After block 408, the exemplary embodiment of block 254 proceeds to block 410.

[0130] At block 410, the vehicle controller 34 calculates a vehicle vector graphics deviation matrix. Within the scope of the present disclosure, the vehicle vector graphics deviation matrix encodes the difference between the reference graphic 160 and the calibration image from the perspective of the vehicle vector graphics projector camera 158. In an exemplary embodiment, to calculate the vehicle vector graphics deviation matrix, the actual vehicle vector graphics matrix generated at block 408 is subtracted from the reference vector graphics camera matrix generated at block 318. After block 410, the exemplary embodiment of block 254 proceeds to block 412.

[0131] At box 412, the vehicle controller 34 calculates a vehicle vector graphics calibration matrix. Within the scope of the present disclosure, the vehicle vector graphics calibration matrix is ​​a two-dimensional matrix that, when added to the matrix defining the graphics, compensates for distortion introduced by manufacturing variations and installation variations of the vehicle windshield 56 and the vehicle vector graphics projector 58. Therefore, from the perspective of the occupant's eye ellipses of the vehicle 10, the compensated graphics will not appear distorted to the occupant. In an exemplary embodiment, to calculate the vehicle vector graphics calibration matrix, the vehicle vector graphics deviation matrix calculated at box 410 is subtracted from the reference vector graphics camera transformation matrix calculated at box 320. After box 412, the exemplary embodiment of box 254 ends and the method 200 continues as described above.

[0132] It should be understood that the vehicle TWD calibration matrix can be expanded (e.g., using mathematical interpolation and / or extrapolation) to include values ​​corresponding to each pixel of the projection field 154 without departing from the scope of the present disclosure. In addition, it should be understood that the system 11 and the calibration system 140 can be used together, and a system for calibrating a vehicle display can be understood to include both.

[0133] The system 11 and method 200 of the present disclosure have several advantages. Using the reference calibration system 140 and method 200, a reference calibration matrix can be generated in a test environment and can be applied to the system 11 during vehicle production. Therefore, the use of method 200 alleviates the need for complex and / or time-consuming calibration of the system 11 during vehicle production and allows automatic self-calibration of the system 11. In addition, the use of method 200 allows the system 11 to be recalibrated by a maintenance technician without the need for specialized tools or equipment. In addition, the system 11 provides self-testing and calibration of a second graphic image (e.g., a vehicle indicator light) projected by the vector graphics projector 58, thereby allowing the system 11 to ensure the correct display of such a vehicle indicator light. Using a vector graphics projector 58 having a plurality of light source modules 92a, 92b, 92c, each of the plurality of light source modules 92a, 92b, 92c can be individually configured to achieve desired output characteristics (e.g., intensity, wavelength, etc.). In addition, each of the plurality of light source modules 92a, 92b, 92c can be adjusted by the VGP engine 130 to adjust the intensity and wavelength composition of the transmitted light 124 during projection, thereby enabling precise control of the excitation of the transparent phosphor. In addition, the excitation of the transparent phosphor is achieved using a diffuse light source (i.e., an LED array), reducing cost and complexity and avoiding the possible disadvantages of high-power, concentrated light sources (such as the light source 62 of the DLP 50). These characteristics of the vector graphics projector 58 provide an instant response to the rapid display of vehicle indicator lights, and display such vehicle indicator lights in a brighter manner than is possible with a digital light processing projector, thereby ensuring that the attention of the occupants in the vehicle will be drawn to the displayed vehicle indicator lights.

[0134] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A system for a head-up display in a vehicle, comprising: a vehicle vector graphics projector adapted to project a second graphic image onto an interior surface of a vehicle windshield, the second graphic image having a higher brightness than the first graphic image and including at least one vehicle indicator light; as well as A calibration system is adapted to test and calibrate the indicator lights displayed within the second graphical image.

2. The system according to claim 1, wherein: The calibration system comprises a reference calibration system, wherein the reference calibration system comprises: Reference windshield; A reference vector graphics projector, wherein the reference vector graphics projector comprises a reference vector graphics projector camera; reference camera; and a reference controller in electrical communication with the reference vector graphics projector and the reference camera, wherein the reference controller is programmed to: projecting a reference pattern onto the reference windshield using the reference vector graphics projector, wherein the reference pattern is a vehicle indicator light and includes a plurality of projection points arranged in a two-dimensional array, wherein an expected x-coordinate and an expected y-coordinate of each of the plurality of projection points are defined by an expected reference pattern matrix; capturing a first reference image of the reference pattern using the reference camera; and At least one reference calibration matrix is ​​calculated based at least in part on the first reference image of the reference pattern.

3. The system according to claim 2, wherein: In calculating the at least one reference calibration matrix, the reference controller is further programmed to: determining a reference camera deviation matrix, wherein the reference camera deviation matrix encodes deviations in the reference pattern due to characteristics of the reference windshield; and A reference vector graphics projector camera transformation matrix is ​​determined based at least in part on the reference camera bias matrix.

4. The system according to claim 3, wherein: When determining the reference camera bias matrix, the reference controller is further programmed to: identifying a plurality of perception points of a reference pattern in the first reference image; determining an actual x-coordinate and an actual y-coordinate of each of a plurality of perception points of a reference pattern in the first reference image; generating an actual reference pattern matrix, wherein each element of the actual reference pattern matrix comprises an actual x-coordinate and an actual y-coordinate of one of a plurality of perception points of a reference pattern in the first reference image; as well as A reference camera bias matrix is ​​calculated by subtracting the actual reference pattern matrix from the expected reference pattern matrix.

5. The system according to claim 4, wherein: In determining the reference vector graphics projector camera transformation matrix, the reference controller is further programmed to: generating a deviation-compensated reference pattern matrix by summing the expected reference pattern matrix and the reference camera deviation matrix; projecting the deviation-compensating reference pattern onto the reference windshield using the reference vector graphics projector, wherein the position of each of a plurality of projection points of the deviation-compensating reference pattern is defined by the deviation-compensating reference pattern matrix; capturing a second reference image of the deviation-compensated reference graphic using the reference vector graphics projector camera; identifying a plurality of perception points of the deviation compensation reference pattern in the second reference image; determining an actual x-coordinate and an actual y-coordinate of each of a plurality of perception points of the deviation-compensated reference pattern in the second reference image; generating a reference vector graphics projector camera matrix, wherein each element of the reference vector graphics projector camera matrix comprises an actual x-coordinate and an actual y-coordinate of one of a plurality of perception points of the deviation-compensated reference pattern in the second reference image; and The reference vector graphics projector camera transformation matrix is ​​determined by subtracting the reference vector graphics projector camera matrix from the reference camera bias matrix.

6. The system according to claim 5, wherein: The vehicle vector graphics projector includes a vehicle vector graphics projector camera; as well as The hybrid projector system includes a vehicle controller in electrical communication with the vehicle vector graphics projector, wherein the vehicle controller is programmed to: projecting a reference graphic onto a windshield of the vehicle using the vehicle vector graphics projector; capturing a calibrated image of the reference graphic using the vehicle vector graphics projector camera; calculating a vehicle vector graphics projector calibration matrix based at least in part on the at least one reference calibration matrix and the calibration image of the reference graphic; and The vehicle vector graphics projector calibration matrix is ​​applied to each frame projected using the vehicle vector graphics projector.

7. The system according to claim 6, wherein: In calculating the vehicle vector graphics projector calibration matrix, the vehicle controller is also programmed to: determining a vehicle vector graphics projector bias matrix based at least in part on the at least one reference calibration matrix and the calibration image of the reference graphic; and The vehicle vector graphics projector calibration matrix is ​​calculated by subtracting the vehicle vector graphics projector bias matrix from the reference TWD camera transformation matrix.

8. The system according to claim 7, wherein: In determining the vehicle vector graphics projector bias matrix, the vehicle controller is further programmed to: identifying a plurality of perception points of the reference pattern in the calibration image; determining an actual x-coordinate and an actual y-coordinate of each of a plurality of perception points of the reference pattern in the calibration image; generating an actual vehicle vector graphics projection matrix, wherein each element of the actual vehicle vector graphics projection matrix comprises an actual x-coordinate and an actual y-coordinate of one of a plurality of perception points of the reference graphic in the calibration image; as well as A vehicle vector graphics projector bias matrix is ​​calculated by subtracting the actual vehicle vector graphics projector matrix from the reference vector graphics projector camera matrix.

9. The system according to claim 1, wherein: The reference camera is fixedly mounted at a reference occupant eyellipse position, wherein the reference occupant eyellipse position is a predetermined position in space relative to the reference windshield.

10. The system according to claim 1, wherein: The vehicle vector graphics projector comprises: Light source module, including: an array of light emitting diodes (LEDs) configured to generate source light; a light filter in optical communication with the LED array, wherein the light filter is configured to receive source light and transmit filtered light; a focusing lens in optical communication with the filter, wherein the focusing lens is configured to receive the filtered light and transmit a narrow beam of light; an optical collimator in optical communication with the light source module, wherein the optical collimator is configured to receive the narrow light beam and transmit collimated light; and An optoelectronic mirror is in optical communication with the optical collimator, wherein the optoelectronic mirror is configured to direct the collimated light to form a pattern.

Images