Luminosity projection compensation for transparent head-up display (HUD)

Through the cooperation of the processor and the display panel, a reference compensated image is generated and projected compensation is performed, which solves the brightness aberration problem caused by windshield notch or epoxy resin treatment in the vehicle head-up display, and achieves uniform brightness and color of the image Sex improvement.

CN119065135APending Publication Date: 2024-12-03FORD GLOBAL TECH LLC
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410639299.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-31
Filing Date
2024-05-22
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The transparent display in the vehicle head-up display is difficult to effectively solve the brightness aberration of optical images due to the notch or pit on the surface of the windshield, especially the brightness artifacts caused by the change in the surface albedo after the treatment with epoxy resin.

Method used

Through the cooperation of the processor and the display panel, the pixel intensity adjustment value is received, a reference compensated image is generated, and the projector is compensated through the projection compensation lookup table to offset the influence of the spatially changing albedo of the transparent projection surface on the image.

Benefits of technology

It effectively offsets the brightness aberration of the image on the transparent display, improves the brightness uniformity and color uniformity of the image, and solves the photometric projection compensation problem caused by the changes in the surface albedo after epoxy resin treatment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065135A_ABST
    Figure CN119065135A_ABST
Patent Text Reader

Abstract

The present disclosure provides luminosity projection compensation for transparent head-up displays (HUDs). A damaged portion of a vehicle windshield in an area near a transparent display (TD) of a head-up display (HUD) may be treated with a resin. The processed portion may introduce a luminance artifact into the image projected onto the TD. To compensate for an image projected onto the TD for luminance artifacts, a first pixel intensity adjustment value is received via the touch screen. Adjusting pixels of a reference image according to the first pixel intensity adjustment value to provide a reference compensated image, the reference compensated image, when projected onto the TD by a projector of a head-up display (HUD) system, counteracts effects of spatially varying reflective properties of a processed portion of the windshield that would otherwise occur in the reference image displayed on the TD without compensation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a heads-up display system in a vehicle. Background Art

[0002] A vehicle head-up display (HUD) may include a transparent display (TD) including a transparent film on which a holographic optical element (HOE) is recorded. The transparent film is disposed in a portion of a vehicle windshield. The surface of the windshield covers and cushions the TD. An optical image is projected onto the TD through the surface of the windshield by a HUD projector. A notch or pit in the surface of the windshield may cause a spatial variation in the surface albedo of the windshield. The surface albedo is defined as the ratio of the radiant flux leaving the surface to the radiant flux received by the surface. Spatial variations in the surface albedo of the windshield close to the TD may result in brightness aberrations in the portion of the optical image displayed on the TD. In some cases, the windshield with notches or pits must be replaced and the problem solved. In other cases, the notched or pitted area of ​​the windshield may be treated with epoxy to prevent further damage. However, the resin treatment itself may introduce a change in the surface albedo of the windshield in the TD region, resulting in brightness aberrations in the displayed optical image. Summary of the invention

[0003] According to the present invention, a device is provided, which has: a processor; a display panel, the display panel including a touch screen; a memory, the memory storing processor-executable instructions, the processor-executable instructions configuring the processor to: receive a first pixel intensity adjustment value via the touch screen; and adjust pixels of a reference image according to the first pixel intensity adjustment value to provide a reference compensated image, wherein the reference compensated image, when projected onto a transparent projection surface by a projector of a head-up display (HUD) system, offsets the effect of the spatially varying albedo of the transparent projection surface on the reference image when the reference image is displayed on the transparent projection surface.

[0004] According to an embodiment, the processor is also configured to: map pixel values ​​of the reference image to pixel values ​​of the reference compensated image to construct a projection compensation lookup table (LUT); and provide the projection compensation LUT to the projector to compensate the input image before projecting the input image onto the transparent projection surface.

[0005] According to an embodiment, the transparent projection surface includes a portion of a vehicle windshield having a gap, and the processor is further configured to: display a graphic image of a reference windshield on the display panel, the graphic image indicating a reference windshield area that is eligible for repair of the gap; receive an indication of a reference windshield area corresponding to the portion of the vehicle windshield having the gap via the touch screen; and determine whether the vehicle windshield is eligible for photometric projection compensation based on the indication of the reference windshield area.

[0006] According to an embodiment, the processor is further configured to, before receiving the first pixel intensity adjustment value: send the reference image to the projector to be projected onto the transparent projection surface; and display the reference image on the display panel.

[0007] According to an embodiment, the processor is further configured to: after receiving the first pixel intensity adjustment value: display the reference compensated image on the display panel; and send the reference compensated image to the projector to be projected onto the transparent projection surface.

[0008] According to an embodiment, the processor is further configured to: receive a second pixel intensity adjustment value via the touch screen; and adjust the intensity values ​​of the pixels of the reference compensated image to provide a modified reference compensated image.

[0009] According to an embodiment, the processor is also configured to: generate the reference image in a first color channel; display the reference image in the first color channel on the display panel; and provide the reference image in the first color channel to the projector for projection onto the transparent projection surface.

[0010] According to an embodiment, the processor is also configured to: receive the first pixel intensity adjustment value as a sub-pixel intensity adjustment value, and modify the sub-pixels of the reference image in the first color channel according to the sub-pixel intensity adjustment value to generate the reference compensated image in the first color channel.

[0011] According to an embodiment, the processor is further configured to: provide the reference compensated image in the first color channel to the projector for projection onto the transparent projection surface; and display the reference compensated image in the first color channel on the display panel.

[0012] According to an embodiment, the processor is also configured to: map pixel values ​​of the reference image in the first color channel to pixel values ​​of the reference compensated image in the first color channel to provide a mapping; construct a first color channel LUT based on the mapping; and provide the first color channel LUT to the projector.

[0013] According to the present invention, a method includes: receiving a first pixel intensity adjustment value via a touch screen of a graphics device including a display panel; and adjusting pixels of a reference image according to the first pixel intensity adjustment value to provide a reference compensated image, wherein the reference compensated image, when projected onto a transparent projection surface by a projector of a head-up display (HUD) system, offsets the effect of a spatially varying albedo of the transparent projection surface on the reference image when the reference image is displayed on the transparent projection surface.

[0014] In one aspect of the present invention, the method includes: mapping pixel values ​​of the reference image to pixel values ​​of the reference compensated image to construct a projection compensation lookup table (LUT); and providing the projection compensation LUT to the projector to compensate the input image before projecting the input image onto the transparent projection surface.

[0015] In one aspect of the present invention, the transparent projection surface includes a chipped portion of a vehicle windshield, and the method includes: displaying a graphic image of a reference windshield on the display panel, the graphic image indicating a reference windshield area that is eligible for repair of the chip; receiving an indication of a reference windshield area corresponding to the chipped portion of the vehicle windshield via the touch screen; and determining whether the vehicle windshield is eligible for photometric projection compensation based on the indication of the reference windshield area.

[0016] In one aspect of the present invention, the method includes: before receiving the first pixel intensity adjustment value: sending the reference image to the projector to be projected onto the transparent projection surface; and displaying the reference image on the display panel.

[0017] In one aspect of the present invention, the method includes: after receiving the first pixel intensity adjustment value: displaying the reference compensated image on the display panel; and sending the reference compensated image to the projector to be projected onto the transparent projection surface.

[0018] In one aspect of the invention, the method includes: receiving a second pixel intensity adjustment value via the touch screen; and adjusting intensity values ​​of pixels of the reference compensated image to provide a modified reference compensated image.

[0019] In one aspect of the present invention, the method includes: generating the reference image in a first color channel; displaying the reference image in the first color channel on the display panel of the graphics device; and providing the reference image in the first color channel to the projector for projection onto the transparent projection surface.

[0020] In one aspect of the present invention, the method includes: receiving the first pixel intensity adjustment value as a sub-pixel intensity adjustment value, and modifying the sub-pixels of the reference image in the first color channel according to the sub-pixel intensity adjustment value to generate the reference compensated image in the first color channel.

[0021] In one aspect of the present invention, the method includes: providing the reference compensated image in the first color channel to the projector for projection onto the transparent projection surface; and displaying the reference compensated image in the first color channel on the display panel of the graphics device.

[0022] In one aspect of the present invention, the method includes: mapping pixel values ​​of the reference image in the first color channel to pixel values ​​of the reference compensated image in the first color channel to provide a mapping; constructing a first color channel LUT based on the mapping; and providing the first color channel LUT to the projector. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a block diagram of an exemplary vehicle system;

[0024] Figure 2 is a block diagram of a device for photometric projection compensation;

[0025] Figure 3 yes Figure 2 an exploded view of a display panel of the device shown;

[0026] Figure 4 It is shown Figure 2 and Figure 3 A block diagram of the device with further details;

[0027] Figure 5 is a flow chart of a method for photometric projection compensation;

[0028] Figure 6 is a flow chart of a method for photometric projection compensation;

[0029] Figure 7 is a flow chart of a method for screening vehicles to qualify for windshield repair;

[0030] Figure 8is a flow chart of a method for photometric projection compensation; and

[0031] Fig. 9 is an example display depicting a windshield repair area map. DETAILED DESCRIPTION

[0032] A notch in a portion of a vehicle windshield providing a transparent display (TD) may be treated with resin. The treated portion may cause spatial variations in the surface albedo of the windshield in which the TD is disposed, resulting in brightness artifacts in the image displayed on the TD. Projector photometric compensation aims to modify a projector input image to compensate the input image for disturbances introduced by the spatially varying albedo of the display surface. The present disclosure provides an apparatus and method for photometrically compensating a TD for variations in surface albedo.

[0033] Figure 1 1 is a block diagram of an example vehicle 102 including a vehicle system 100 suitable for implementing the apparatus and methods disclosed herein. The vehicle system 100 may include sensors 108, including one or more imaging sensors and one or more ranging sensors. The vehicle 102 includes a vehicle computer 104, which may include one or more electronic control units (ECUs) disposed within the vehicle 102 to control the operation of various devices and components 110 of the vehicle 102. The computer 104 may generally be programmed to send and / or receive messages via a vehicle network 114 to communicate with other devices disposed within the vehicle (e.g., sensors 108, actuators, components 110, communication modules, human machine interfaces (HMIs) 112, etc.).

[0034] The vehicle system 100 may include one or more vehicle networks 114, through which various electronic control units (ECUs) including the computer 104 can communicate with each other to send commands and / or exchange, send or receive data. The vehicle network 114 may include, but is not limited to, a controller area network (CAN), a local interconnect network (LIN), Ethernet, In some implementations, the vehicle computer 104 and / or other components of the vehicle system 100 may be configured to communicate with a remote server 118 via a wide area network (WAN) 116 .

[0035] The computing device 104 includes a processor and memory such as are known. In addition, the memory includes one or more forms of computer-readable media and stores instructions that can be executed by the processor to perform various operations including as disclosed herein. For example, the computing device 104 may include programming to operate one or more of vehicle braking, propulsion (i.e., controlling the acceleration of the vehicle 102 by controlling one or more of an internal combustion engine, an electric motor, a hybrid engine, etc.), steering, climate control, interior lights and / or exterior lights, etc., and determine whether and when the computing device 104 (rather than a human operator) controls such operations.

[0036] The computing device 104 may include more than one computing device (i.e., controllers included in the vehicle 102 for monitoring and / or controlling various vehicle components (i.e., propulsion, braking, steering, etc.), or be communicatively coupled to the more than one computing device via a vehicle communication bus as further described below. The computing device 104 is typically arranged to communicate over a vehicle communication network 114 (i.e., including a bus in the vehicle 102, such as a controller area network (CAN), etc.); the vehicle 102 network 114 may additionally or alternatively include, for example, known wired or wireless communication mechanisms, i.e., Ethernet or other communication protocols.

[0037] The computing device 104 may transmit and / or receive messages to and from various devices in the vehicle (i.e., controllers, actuators, sensors (including sensor 108), etc.) via the vehicle network 114. Alternatively or in addition, where the computing device 104 actually includes multiple devices, the vehicle communication network 114 may be used to communicate between devices represented as the computing device 104 in this disclosure. Furthermore, as mentioned below, various controllers or sensing elements (such as sensors 108) may provide data to the computing device 104 via the vehicle communication network 114.

[0038] Additionally, computing device 104 may be configured to communicate with remote server computer 118 (i.e., cloud server) via network 116 through a vehicle-to-infrastructure (V2X) interface, as described below, which includes hardware, firmware, and software that permit computing device 104 to communicate with remote server computer 118 (i.e., cloud server) via a vehicle-to-infrastructure (V2X) interface, such as a wireless Internet connection. The V2X interface may include a computer configured to communicate with a remote server computer 118 using a network 116 of a wireless or cellular network. The computing device 104 may be configured to communicate with other vehicles 102 through a V2X (vehicle-to-everything) interface using a vehicle-to-vehicle (V2V) network formed between neighboring vehicles 102 on the basis of a mobile ad hoc network or through an infrastructure-based network (i.e., based on cellular communication (C-V2X) wireless communication cellular, dedicated short-range communication (DSRC) and / or similar communications). The computing device 104 also includes a non-volatile memory such as known. The computing device 104 may record data by storing the data in a non-volatile memory for later retrieval and transmission to a server computer 118 or a user mobile device via the vehicle communication network 114 and the vehicle-to-infrastructure (V2X) interface.

[0039] As already mentioned, programming for operating one or more vehicle 102 components (i.e., braking, steering, propulsion, etc.) without human operator intervention is typically included in instructions stored in memory and executable by a processor of the computing device 104. Using data received in the computing device 104 (i.e., sensor data from sensors 108, data from server computers 118, etc.), the computing device 104 can make various determinations and / or control various vehicle 102 components and / or operations. For example, the computing device 104 may include programming to adjust vehicle 102 operating behavior (i.e., the physical manifestation of vehicle 102 operation), such as speed, acceleration, deceleration, steering, etc., as well as strategic behavior (i.e., controlling operating behavior in a manner that is generally intended to achieve efficient traversal of a route), such as the distance between vehicles and / or the amount of time between vehicles, lane changes, minimum gaps between vehicles, left turn crossing path minimums, arrival times at specific locations, and the minimum time from arrival at an intersection to crossing an intersection (without a signal light).

[0040] Sensors 108 may include a variety of imaging devices (such as cameras) and / or other devices known to provide data via a vehicle communication bus. For example, a radar affixed to a front bumper (not shown) of vehicle 102 may provide the distance from vehicle 102 to the next vehicle in front of vehicle 102, or a global positioning system (GPS) sensor disposed in vehicle 102 may provide the geographic coordinates of vehicle 102.

[0041] Figure 2 is a block diagram of an apparatus 300 for photometric projection compensation of a transparent display (TD) 230 for surface albedo variations due to gaps, pits 232 and / or epoxy fillers in a vehicle windshield 250. Figure 2In the example of FIG. 2 , the HUD system 200 includes one or more transparent displays, such as TD 230. TD 230 includes a holographic film on which a hologram of an optical element is recorded as an interference pattern. The film carrying the hologram implements a holographic optical element (HOE) (in Figure 2 250 (not visible in the windshield). The film is disposed in a portion of the windshield 250 (or other glass of the vehicle), between the outer surface and the inner surface of the glass. In the example implementation disclosed herein, the TD 230 is a reflection hologram. The reflection hologram is made by recording light waves arriving on the opposite side of the recording medium to produce an interference pattern. In this example, the HOE recording is "played back" by projecting optical image light onto the film. The interference pattern responds to the projected image light according to the optical transfer function of the optical element whose hologram produced the interference pattern. The reflection HOE implementing a diffractive optical element (DOE) diffracts selected incident light back through the medium (through which the reflection HOE arrived) to form an image in the viewing area or "eye box".

[0042] Essentially, TD 230 is a diffraction grating set in glass that acts as a projection surface or "screen". Therefore, TD230 has wavelength and angle selectivity, and has a diffraction efficiency that varies according to the angle of incidence. For a TD without surface defects, about 60% of the light projected onto the TD passes through the "screen", that is, through the HOE and the windshield. This allows vehicle occupants to "see through" the HOE to perceive the driving environment outside the vehicle.

[0043] A notch or pit in a windshield changes the surface albedo in the notch area. The notch or pit can be repaired by treating the glass with epoxy, for example. However, the epoxy treatment also causes a change in the surface albedo. Under typical ambient lighting conditions, the effect may not be noticeable. However, when a light beam is projected from a DLP projector onto a TD, significant brightness and color aberrations may be present in the area of ​​the optical image displayed on the TD in the treated area.

[0044] Photometric projection compensation is a technique for neutralizing the effects of variations in the albedo of the projection screen surface on the displayed image by compensating for the projected image pixel intensities in areas of the screen that exhibit aberrations. Compensation values ​​for the projected image are typically calculated using a projector-camera system, where the camera serves as a proxy for a human observer. The camera senses and measures the reflective properties of each pixel of the image displayed on the screen surface. A series of transformations determines the pixel intensity, which, when converted to an optical image by the projector and projected onto the screen surface, will be adjusted to the intensity that would be expected if the display surface had a uniform albedo.

[0045] Camera-based techniques are not always a practical solution for photometric projection compensation of vehicle HUD projectors. Vehicle cabin size and space constraints, off-axis projection arrangements, and viewing area (eyebox area) constraints can make reliable camera measurements challenging.

[0046] Thus, the present disclosure provides a photometric compensation device 300 that can be used without a camera to compensate a projected optical image for the effects of variations in surface albedo of a windshield surface. In the examples described herein, the projection surface comprises a TD 230 of a HUD system 200, including a HOE. The TD is disposed within the windshield.

[0047] It should be understood that the present disclosure is not limited to TDs disposed in vehicle windshields or within glass surfaces. The present disclosure encompasses other vehicle glass, such as vehicle windows, including vision glass roofs and rear windows in which TDs may be disposed. A person of ordinary skill will appreciate the wide range of practical applications of the disclosed apparatus and methods after reading the present disclosure. Any glass having a spatially varying surface albedo that would introduce observable brightness aberrations in a displayed image may benefit from the photometric projection compensation disclosed herein.

[0048] The device 300 enables a user 2 as a human observer to interact with a HUD projector (e.g., projector 235) via a touch screen 355 to empirically establish pixel intensity compensation values ​​using the human eye as a light sensor rather than a camera. The user 2 manipulates the touch screen 355 by touching an area (e.g., 334) to adjust the intensity of light projected by the projector 235 onto the TD 230 in a corresponding area of ​​the pit or notch 232 of the windshield 250, which the user 2 observes exhibits brightness aberrations.

[0049] exist Figure 2 In the example of , the device 300 includes a video image output port 344. The processor 310 of the device 300 is configured to establish a communication link with the projector 235 via the corresponding projector communication port 244 via the video output port 344. The processor 310 can be implemented as a single processor structure or a multiple processor structure. The device 300 sends a frame 372 of a digital image to the projector 235 via the video output port 344. The digital image frame is generated by the user 2 interacting with the touch screen 355. The projector 235 receives the frame 372, converts the digital image into an optical image and projects the optical image onto the TD 230. The TD 230 displays the optical image. The user 2 can directly observe the TD 230 to perceive the effect of the spatially varying surface albedo on the displayed optical image.

[0050] Figure 33 is a block diagram of device 300, including an exploded view of display panel 350. Display panel 350 includes emitter array 358, common electrode layer 356, touch screen layer 355, and transparent cover layer 352. Figure 3 In the example of , emitter array 358 is an array of light emitting elements 359 (also referred to as display pixels 359). Each display pixel 359 includes a red sub-pixel (red light emitter 60), a green sub-pixel (green light emitter 61), and a blue sub-pixel (blue light emitter 62). Figure 3 In the example of FIG. 3 , the red light emitter 60, the green light emitter 61, and the blue light emitter 62 are light emitting diodes (LEDs). Thus, each display pixel of the emitter array 358 includes at least one red LED 60, at least one green LED 61, and at least one blue LED 62. The present disclosure is not limited to Figure 3 The examples of FIG. 350 are not limited to the specific display technology shown. A person of ordinary skill will appreciate that a wide range of display technologies will be suitable for implementing display panel 350, emitter array 358, and pixels 359.

[0051] The device 300 may also include a frame buffer 349 and a frame grabber 348. The frame buffer 349 is comprised of a portion of a random access memory (RAM) that includes the memory 320 and is configured to buffer or temporarily store a digital image, such as the reference image 270 to be displayed on the display panel 350. The frame buffer 349 is configured to drive the pixels 359 of the emitter array 358 at pixel intensity values ​​specified by the digital image stored in the frame buffer 349. When the LEDs comprising the pixels are illuminated, an optical image is formed on the display panel 350.

[0052] exist Figure 3 In the example of , reference image 270 is a red channel image. Therefore, reference image 270 includes an array of intensity values ​​for red sub-pixels used to drive emitter array 358. In the methods disclosed herein, the reference image and the reference compensated image may include a red channel image, a green channel image, a blue channel image, and / or an RGB image including all three color channel images. In the example reference images described herein, the reference intensity values ​​of the sub-pixels in the sub-pixel array are the same, for example, 128.

[0053] When the intensity values ​​of the reference image are adjusted (e.g., according to reference values ​​128) by user interaction with the touch screen 355, the result is a compensated reference image, i.e., the intensity adjustment to the reference image compensates the reference image for brightness aberrations that would otherwise be caused by the notch or restoration if the reference image were projected without compensation. The compensated reference image can be produced by one "round" or more rounds of user adjustment that produce a modified reference compensated image. For example, the reference image can be displayed on the display panel 350 of the device 300 and compensated in a first round of user manipulation of pixel intensity values ​​via the touch screen 355. This first round can produce an intermediate compensated reference image, i.e., a modified compensated reference image. For example, the user can make further adjustments in another round, where the intermediate compensated reference replaces the reference image on the display panel 350 while being sent to the projector 235 for projection onto the TD 230 of the HUD system 200 (as described below). As the user manipulates the touch screen 355 to make further adjustments, the user can observe the displayed intermediate compensated image.

[0054] After the user makes the last user adjustment, the processor 310 is configured to map the pixel values ​​of the compensated reference image to the pixel values ​​of the reference image. The mapping may be encoded in a lookup table (LUT) and stored in the projector. Thereafter, when the projector receives the HUD digital image, it may use the LUT to adjust the pixels of the HUD digital image according to the same mapping applied to the reference image to produce a compensated HUD image. The compensated HUD image may then be projected onto a transparent display through a notched or repaired vehicle windshield surface by a projector of a head-up display (HUD) system. The intensity-adjusted light including the compensated HUD optical image offsets the brightness aberration effects that would otherwise be introduced into the HUD optical image displayed on the TD without compensation in the notch or repair area 334.

[0055] As the user manipulates the touch screen 355, the frame grabber 348 captures (i.e., "grabs") frames (images) from the frame buffer 349 and sends the frames to the projector 235 via the video output port 344. The frame grabber 348 allows the image displayed on the display panel 350 to be simultaneously projected onto any one or more of the TDs 220, 230, 240 of the HUD system 200 ( Figure 4 ).

[0056] The processor 310 may be configured by the graphics image manipulation program 327 to generate, modify, and otherwise manipulate the reference images and reference compensated images described herein. The processor 310 may be configured by the multimedia platform 323 to implement the frame grabber 348 and perform decoding, encoding, transcoding, multiplexing, demultiplexing, streaming, and filtering of any type of images, including graphics images in any format described herein.

[0057] In the examples described herein, images are described according to an "RGB" (red, green, blue) color model. An RGB image includes three image channels: red, green, and blue. The red image channel includes a frame of intensity values ​​for red sub-pixels of the RGB image, where the red "sub-pixels" of the RGB image correspond to an array of red light emitters of a display device (the example shown at 60) and / or an array of red light sensors of an imaging device (such as a camera) (not used in the example shown). Similarly, the green image channel includes a frame of intensity values ​​for green sub-pixels (61), and so on for the blue image channel. However, the present disclosure is not limited to application to RGB images. Other suitable color models include YUV, where Y specifies brightness, U specifies blue chromaticity, and V specifies red chromaticity; CMYK (cyan, magenta, yellow, and register color (black)); and HSV (hue saturation value).

[0058] In the example implementations described herein, the RGB image is a 24-bit image, with each channel providing 8 bits to represent the light intensity level of each sub-pixel between 0 and 255. In some implementations, the RGB image can be 48 bits, with each channel having 16 bits to represent the intensity level of the pixel color.

[0059] The red channel image comprises an array in which the red sub-pixel intensity values ​​of the RGB image maintain their values ​​in the RGB image, i.e., the value range is between 0 and 255. The green and blue sub-pixel intensity values ​​of the RGB image are set to 0. The green channel image comprises an array in which the green sub-pixel intensity values ​​of the RGB image maintain their values ​​in the RGB image, i.e., the value range is between 0 and 255. The red and blue sub-pixel intensity values ​​are set to 0. The blue channel image comprises an array in which the blue sub-pixel intensity values ​​in the RGB image are maintained, i.e., the value range is between 0 and 255. The red and green sub-pixel intensity values ​​are set to 0. In other words, to display a color representation of a given color channel of the RGB image, the given color channel maintains its original pixel intensity value and the pixel intensity values ​​in the other two color channels are set to 0.

[0060] In this specification, "digital image" refers to digital values, while the term "optical image" refers to light. The digital values ​​specify the relative intensity of light that will be emitted when the values ​​are applied to drive the light emitters. The relative values ​​specify the mix of red, green, and blue light that defines the color and brightness of each pixel.

[0061] The displayed image is an optical image comprising image modulated light propagating away from the display screen. The projected image is an optical image comprising light propagating from the projector toward the display screen. The display screen is any surface illuminated by light from the projector so as to display an optical image on it.

[0062] The red, green, and blue (RGB) color image includes a red image subchannel comprised of an array of red light intensity values ​​for the red subpixels of the red subarray of emitter array 358, a green image subchannel comprised of an array of green light intensity values ​​for the green subarray of emitter array 358, and a blue image subchannel comprised of an array of blue light intensity values ​​for the blue subarray of emitter array 358. The red subchannel intensity values, the green subchannel intensity values, and the blue subchannel intensity values ​​collectively specify a mix of red, green, and blue light for display pixel 359 of emitter array 358.

[0063] Figure 3 An example red channel reference digital image 270 is shown loaded into frame buffer 349 for display as an optical image on display panel 350. User 2 adjusts the red sub-pixel intensity value by manipulating a touch sensitive element (e.g., 354 of touch screen 355). Processor 310 is configured by graphics image manipulation application 327 to interpret the sensed touch as an adjustment to the pixel intensity value of the displayed image in the area corresponding to the touch area, so that the user manipulation of touch screen 355 produces an array 374 of pixel intensity adjustment values.

[0064] The processor 310 is configured by the graphics image manipulation application 327 to provide color channel projected reference compensated images by adjusting the sub-pixel intensity values ​​of the reference image 270 in the frame buffer 349 by an amount given by the intensity adjustment values ​​of the intensity adjustment array 374 (e.g., increasing or decreasing the sub-pixel intensity values ​​from their reference values ​​of 128 in accordance with a user touch sensed by the touch screen 355). In an implementation using 8 binary digits (bits) to represent pixel intensity values, the value 128 is half of the maximum intensity value 256. The processor 310 is configured to provide reference compensated images for the red, green, and blue image channels.

[0065] The processor 310 loads each successive corresponding reference compensated image for the red image channel, the green image channel, and the blue image channel into the frame buffer 349, where the image is displayed on the display panel 350 and is also grabbed by the frame grabber 348. The processor 310 is configured by the multimedia platform 323 to format the reference compensated image including the grabbed frame to be compatible with the projector 235, and send the properly formatted reference compensated image to the projector 235 for projection onto the transparent display 230. Thus, the device 300 enables the user 2 to observe the reference compensated image displayed on the transparent display 230 while the user 2 manipulates the reference compensated image displayed on the display panel 350 of the device 300.

[0066] In such an implementation, the frame grabber 348 may be configured to convert frames grabbed from the frame buffer 349 to a format compatible with the projector 235 based on, for example, a device profile 375 of the projector 235, which is stored in the memory 320 of the apparatus 300. The device profile 375 may provide a geometric mapping between pixels of the display panel 350 and pixels of the projector 235.

[0067] The projector 235 converts the received array into an optical image 271 according to the pixel intensity values, and projects the optical image 271 onto the TD 230. The TD 230 displays the optical image by diffracting the optical image light to the eye frame area, where the displayed optical image can be observed by the user 2. The user 2 can use the device 300 to adjust the pixel intensity value based on the observed brightness aberration in the optical image displayed on the transparent display 230.

[0068] In this way, the device 300 enables the user 2 to define photometric projection compensation for any image to be displayed on the TD 230. When the compensated projected optical image is displayed on the TD 230, the projected optical image light will be modulated to increase the brightness uniformity of the displayed optical image on the TD 230.

[0069] The device 300 maps the pixel values ​​including the reference array 270 to the pixel values ​​including the compensation array, thereby providing a pixel map. The processor 310 constructs one or more lookup tables (LUTs) based on the pixel value mapping. The device 300 provides one or more LUTs to the projector 235. The projector 235 stores one or more LUTs. Subsequently, in the normal operation of the vehicle, any image to be projected on the TD 230 can be provided to the LUT. The LUT adjusts the red image pixels, the blue image pixels, and the green image pixels according to the mapping stored in the one or more LUTs, thereby providing photometric projection compensation to the image to be projected on the TD. When the TD 230 displays the compensated optical image, the compensated optical image has increased brightness and color uniformity in the area of ​​the notch or epoxy resin treatment 232 compared to its uncompensated corresponding reference image.

[0070] Figure 4 It is shown Figure 2 and Figure 3 The device 300 and the block diagram of the head-up display system 200 are shown. The head-up display (HUD) system 200 includes corresponding projectors 225, 235, 245 and corresponding corresponding projector controllers 226, 236, 246, and the projectors and projector controllers are arranged in the vehicle 102 (at Figure 1) and is arranged to project the HUD optical input image 272 onto corresponding respective transparent projection surfaces including transparent displays 220, 230, 240. Figure 4 In the example of , TD 230 has surface indentations or pits 232. Media interface platform 275 may store HUD digital images used to generate HUD optical image 272, and may include other interfaces for receiving other media from sources external to HUD system 200 for projection onto one or more of TDs 220, 230, 240.

[0071] The device 300 includes a memory 320, a processor 310, a display panel 350 (in Figure 3 ), frame buffer 349, and frame grabber 348 (also shown in Figure 3 ). In some implementations, device 300 may include a graphics tablet configured with a dedicated touch screen drawing application. An example of a suitable touch screen drawing application is the "PatistPro 16TP Drawing Display" available from XPPen Technologies, Inc., 15350 Fairfield Farm Road, Unit G1, Chino Hills, CA 91709 (https: / / www.xp-pen.com / store / ).

[0072] The memory 320 may store reference images 326, pixel maps 328 generated by the methods 500 and 600 described below, libraries, routines and other components implementing the multimedia platform 323, device configuration files 375, processor-executable instructions 324, and processor-executable instructions including a graphics image manipulation program 327. The processor-executable instructions 324 configure the processor 310 to perform the processes, functions and methods disclosed and described herein.

[0073] The graphical image manipulation application 327 includes processor executable instructions that configure the processor 310 to perform the image generation and manipulation functions described herein. The image manipulation functions may be provided by any suitable image manipulation program having touch screen image manipulation capabilities. Examples of suitable touch screen image manipulation programs for Linux, Windows, and macOS devices include GIMP 2.99 (GNU Image Manipulation Program), which is freely distributable and available from www.gimp.org.

[0074] In an example implementation, the multimedia platform 323 implements the frame grabber 348 as part of a cross-platform multimedia framework such as the Fast Moving Picture Experts Group (FFmpeg) platform available from FFmpeg.org. FFmpeg is released under the GNU Lesser General Public License (LGPL) 2.1 or later or 2.0 or later, depending on which options are enabled.

[0075] Figure 5 is a flow chart of a method 500 for photometric projection compensation.

[0076] At block 501 , processor 310 receives an indication of a user selection of a transparent projection screen for photometric projection compensation.

[0077] At block 502 , the processor 310 generates or acquires a first color channel reference image of a selected transparent projection screen based on a device profile of a projector corresponding to the transparent projection screen.

[0078] At block 504 , the processor 310 displays the reference image on a display panel of the user graphics device.

[0079] At block 506 , the processor 310 sends the reference image to the projector for projection onto the projection screen selected at block 501 .

[0080] At block 508 , the processor 310 receives, via the touch screen of the user graphics device, sub-pixel intensity adjustment values ​​for the reference image.

[0081] At block 510 , processor 310 applies the sub-pixel intensity adjustment values ​​to a reference image to provide a first color channel reference compensated image.

[0082] At block 512, processor 310 sends the first color channel reference compensated image to the projector for projection onto the transparent projection screen. When the projector projects the reference compensated image onto the transparent projection screen, user 2 may observe the displayed image including any brightness aberrations in the gap or repaired area.

[0083] At block 514, processor 310 determines whether the user will make more adjustments to the first color channel reference image. If so, processor 310 proceeds to block 508, where processor 310 receives further sub-pixel intensity adjustments via the touch screen. Processor 310 repeats blocks 510, 512, and 514 until processor 310 determines that no further adjustments are to be made to the first color channel of the reference image.

[0084] If no further adjustments are to be made to the first color channel reference image, processor 310 proceeds to block 516. At block 516, processor 310 maps pixel values ​​of the reference image (before adjustments) to pixels of the reference compensated image.

[0085] At block 518 , the processor 310 constructs a look-up table (LUT) based on the map provided at block 516 .

[0086] At block 520 , processor 310 provides the LUT to the projector.

[0087] At block 522, processor 310 determines whether a reference compensated image has been provided for each of the three image color channels. If not, processor 310 returns to block 502. At block 502, processor 310 generates another reference image for the second color channel and repeats blocks 508, 510, 512, 514, 516, 518, and 520 until the last of the three color channel LUTs has been provided to the projector at block 522 and the method ends.

[0088] Figure 6 is a flow chart of a method 600 for providing photometric projection compensation for a projector of a heads-up display (HUD).

[0089] At block 602, processor 310 will Figure 5 The red, green, and blue channel reference compensated images generated at block 510 of method 500 shown in the flowchart of FIG. 5 are combined to provide an RGB reference compensated image.

[0090] At block 604, processor 310 will Figure 5 The red, green, and blue channel reference images generated at block 502 of method 500 shown in the flowchart of FIG. 5 are combined to provide an RGB reference image.

[0091] At block 606 , the processor 310 maps pixel values ​​of the RGB reference image to pixel values ​​of the RGB reference compensated image.

[0092] At block 608 , the processor 310 constructs an RGB LUT based on the map.

[0093] At block 610 , processor 310 supplies the RGB LUT to the projector, and the method ends.

[0094] Figure 72 is a flow chart of a method for performing a preliminary eligibility assessment to determine whether a windshield 250 qualifies for repair and / or photometric projection compensation. Some types of windshield damage require replacement of the windshield. For example, if the breach is within the driver's primary field of view (FoV), some insurance companies, states, or countries mandate replacement of the windshield.

[0095] At block 702 , the processor 310 enters a photometric projection compensation qualification mode.

[0096] At block 704, processor 310 determines the geographic location of vehicle 102. In an example implementation, memory 320 stores graphical representations of reference windshields for a plurality of geographic regions. To determine the geographic region of the vehicle, device 300 may query the vehicle GPS system. Device 300 may retrieve graphical representations of reference windshields indicating reference windshield regions where repair is allowed or not allowed based on the determined geographic region.

[0097] At block 706, processor 310 displays screen 900 showing a repair area map 950 (in Fig. 9 ), the repair zone map indicates which windshield zones A, B, C, D can be repaired based on the geographic location determined at box 704. Table 1 shows the criteria for repairing windshield gaps according to the British Standard Code of Practice BS AU 242 1998. In this case, for example, if the gap is in windshield zone A, damage up to 10 mm can be repaired.

[0098] Table I

[0099] WS Zone Restoration Standards A Repairs damage up to 10mm B Repairs damage up to 15mm C Repairs damage up to 25mm D Repairs damage up to 45mm

[0100] At block 708 , processor 310 receives touch input indicating which of the displayed windshield regions has a chip or repair.

[0101] At block 709 , processor 310 receives a touch input indicating a size of the damage.

[0102] At block 710 , processor 310 determines whether repair is permitted based on the indicated windshield area and the size of the damage.

[0103] If the processor 310 determines that repair is not allowed, the processor 310 proceeds to block 712 and terminates the method.

[0104] If the processor determines that repair is allowed, the processor 310 proceeds to block 714. At block 714, the processor 310 determines that repair is allowed. Figure 8 The flowchart shown implements method 800 .

[0105] At block 716, processor 310 determines whether method 800 has completed. If method 800 has not completed, processor 310 continues to perform the process at block 714. If method 800 has completed, processor 310 ends method 700.

[0106] Figure 8 is a flow chart of a method for photometric projection compensation. At block 802, the processor 310 displays a graphical image representing the windshield 250 on the display panel 350. The graphical representation includes graphical objects representing the TDs 220, 230, 240.

[0107] At block 804 , the processor 310 receives a user selection of one of the TDs 220 , 230 , 240 (eg, the TD 230 with the notch 232 ).

[0108] At block 806, the processor 310 constructs one or more photometric projection compensation LUTs 266 for the selected TD 230, for example, by executing the method 600 or 700. Figure 4 ).

[0109] At block 808 , processor 310 provides one or more photometric projection compensation LUTs 266 to projector 235 , and the method ends.

[0110] Once provided with the projection compensation LUT 266, the controller 236 of the projector 235 can project the projector input image (e.g., Figure 4 272) is provided to LUT 266. For a pixel of the input image, LUT 266 may look up a corresponding pixel intensity adjustment value in the red image channel, the green image channel, and / or the blue image channel to compensate the input image for the change in surface albedo around the notch or processed portion 232, i.e., look up a pixel of the reference image in the same array position as a pixel in the input image to provide a pixel intensity adjustment value for the reference compensated image.

[0111] Computing devices such as those discussed herein typically each include instructions that can be executed by one or more computing devices (such as those identified above) and used to implement the blocks or steps of the processes described above. For example, the process blocks discussed above may be embodied as computer-executable instructions.

[0112] The computer executable instructions may be compiled or interpreted by a computer program created using a variety of programming languages ​​and / or technologies, including, but not limited to, Java, TM, C, C++, Visual Basic, Java Script, Perl, HTML, etc. In general, a processor (e.g., a microprocessor) receives instructions, such as from a memory, a computer-readable medium, etc., and executes these instructions to perform one or more processes, including one or more of the processes described herein. Such instructions and other data can be stored and transmitted using a variety of computer-readable media. Files in a computing device are typically a collection of data stored on a computer-readable medium such as a storage medium, a random access memory, etc.

[0113] Computer-readable media include any media that participate in providing data (e.g., instructions) that can be read by a computer. Such media can take many forms, including but not limited to non-volatile media, volatile media, etc. Non-volatile media include, for example, optical disks or disks and other persistent memories. Volatile media include dynamic random access memory (DRAM), which usually constitutes main memory. Common forms of computer-readable media include, for example, floppy disks, floppy disks, hard disks, tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tapes, any other physical media with hole patterns, RAM, PROMs, EPROMs, flash EEPROMs, any other memory chips or cassettes, or any other non-temporary storage media from which a computer can read stored data or instructions.

[0114] In the accompanying drawings, the same reference numerals indicate the same elements. In addition, some or all of these elements may be changed. With respect to the media, processes, systems, methods, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring in a certain ordered sequence, such processes may be practiced by performing the steps described in an order different from the order described herein. It should also be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the description of the process herein is provided for the purpose of illustrating certain embodiments and should never be interpreted as limiting the claimed invention.

[0115] Therefore, it should be understood that the above description is intended to be illustrative rather than restrictive. After reading the above description, many embodiments and applications other than the examples provided will be apparent to those skilled in the art. The scope of the present invention should not be determined with reference to the above description, but with reference to the appended claims together with the full scope of equivalents to which these claims are assigned. It is contemplated and expected that future developments will occur in the technology discussed herein, and the disclosed systems and methods will be incorporated into such future embodiments. In short, it should be understood that the present invention is capable of modification and variation and is limited only by the appended claims.

[0116] All terms used in the claims are intended to be given their broadest reasonable interpretations and their ordinary meanings as understood by those skilled in the art, unless an explicit indication to the contrary is made herein. Specifically, unless a claim recites an explicit limitation to the contrary, the use of singular articles such as "a / kind", "the", "said", etc. should be interpreted as reciting one or more of the indicated elements.

Claims

1. A method, the method include: rendering a graphical windshield representation of a vehicle windshield on an interactive display screen of a display device; detecting an adjustment to the graphical windshield representation; as well as A projector of the HUD is controlled to adjust an intensity of light projected onto a transparent display (TD) comprising the processed portion of the vehicle windshield in accordance with the adjustment of the graphic windshield representation.

2. The method according to claim 1, wherein include: rendering a first sub-pixel graphical windshield representation of the vehicle windshield on the interactive display; controlling a projector of the HUD to project a first sub-pixel optical image onto a windshield of the vehicle; detecting a user interaction with the first sub-pixel graphical windshield representation; as well as An intensity of the first sub-pixel optical image is controlled based on the user interaction.

3. The method according to claim 2, wherein include: rendering a second sub-pixel graphical windshield representation of the vehicle windshield on the interactive display; controlling the projector of the HUD to project a second sub-pixel optical image onto the vehicle windshield; detecting a user interaction with the second sub-pixel graphical windshield representation; as well as An intensity of the second sub-pixel optical image is controlled based on the user interaction.

4. The method according to claim 3, wherein include: rendering a third sub-pixel graphical windshield representation of the vehicle windshield on the interactive display; controlling the projector of the HUD to project a third sub-pixel image onto the vehicle windshield; detecting a user interaction with the third sub-pixel graphical windshield representation; as well as An intensity of the third sub-pixel image is controlled based on the user interaction.

5. The method according to claim 4, wherein include: receiving a first array of first sub-pixel intensity adjustment values ​​corresponding to user interactions with a first sub-pixel graphical representation of a vehicle windshield; as well as A first sub-pixel digital calibration image of a projector of the HUD is constructed based on the first sub-pixel intensity adjustment value.

6. The method according to claim 1, wherein include: transmitting a graphical user interface to the display device to render the graphical windshield representation on the interactive display screen; identifying a processed portion of the vehicle windshield based on a user interaction with the graphical windshield representation; determining that the processed portion of the vehicle windshield is not eligible for a brightness compensation method; as well as Deactivate the graphical user interface.

7. The method according to claim 1, wherein include: storing the graphical windshield representation in a memory; detecting a user interaction indicating a location on the graphical windshield representation corresponding to a location of the processed portion on the vehicle windshield; including in the graphical windshield representation in the memory a graphical representation of the processed portion at the location on the graphical windshield representation corresponding to the location of the processed portion on the vehicle windshield; as well as The graphical windshield representation including the graphical representation of the processed portion is sent to the display device for rendering on the interactive display screen.

8. The method according to claim 1, wherein include: receiving a first array of first sub-pixel intensity adjustment values ​​corresponding to user interactions with a first sub-pixel graphical representation of a vehicle windshield; as well as The first sub-pixel intensity adjustment value is transformed from a value within a color gamut of the display device to a value within a color gamut of the projector.

9. The method according to claim 1, wherein include: projecting an optical image onto the transparent display at a first intensity defined by the digital calibration image; adjusting the digital calibration image based on the adjustment to the graphical windshield representation to provide an adjusted digital calibration image; as well as The optical image is projected onto the transparent display at a second intensity defined by the adjusted digital calibration image.

10. The method according to claim 9, wherein include: measuring a first intensity distribution of light reflected by the transparent display in response to the optical image being projected at the first intensity defined by the digital calibration image; measuring a second intensity distribution of light reflected by the transparent display in response to the optical image at the second intensity defined by the adjusted digital calibration image; and The digital calibration image is further adjusted based on a difference between the first intensity distribution and the second intensity distribution.

11. A computing device programmed to perform the method of any one of claims 1 to 10.

12. A vehicle comprising the computing device of claim 11.

Citation Information

Cited By

  • Brightness compensation method, head-up display, storage medium and program product

    CN120735587A

  • Brightness compensation method, head-up display, storage medium and program product

    CN120735587B