Multi-mode display

Abstract

The invention discloses a multi-mode display comprising: a mode selector for selecting one of a plurality of modes, each of the modes having a different light configuration, one of the modes comprising a reduced color space mode; and one or more light sources, the one or more light sources being controlled by the mode selector, the one or more light sources being configured to display content to a user using the multi-mode display.

Description

[0001] Related Applications

[0002] The present invention claims priority to U.S. Provisional Application No. 63 / 362,519 filed on April 5, 2022, and the entire contents of that application are incorporated by reference. Technical Field

[0003] The present invention relates to displays and, more particularly, to an architecture for a multi-mode display in which the type of illumination varies. Background Art

[0004] Color sequential displays (typically spatial light modulators) have many advantages over traditional emissive displays. The main advantages include high fill factors per pixel and the ability to tightly control the illumination / emission angle of light from the display. However, a major disadvantage of color sequential displays is the loss of brightness and optical efficiency because each color in the system is only on for a small fraction of the total frame time. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

[0006] Figure 1 One embodiment of one embodiment of a multi-mode LCOS based display system is illustrated.

[0007] Figure 2A and Figure 2B Exemplary light timings for two exemplary modes of a three-light source system are illustrated, showing a color sequential mode and a non-color sequential mode.

[0008] Figure 3A is a diagram of an embodiment of an optoelectronic driving configuration for a common anode three-light source multi-mode display system.

[0009] Figure 3B is a diagram of one embodiment of an optoelectronic driving configuration for a discrete anode three-light source multi-mode display system using a single controller IC.

[0010] Figure 3C is a diagram of one embodiment of an optoelectronic driving configuration for a discrete anode three-light source multi-mode display system using a separate controller IC for each light source.

[0011] Figure 3D is a diagram of one embodiment of an electro-optical drive configuration for a light array based display system.

[0012] Figure 4A and Figure 4Bis a diagram of one embodiment of an optical device configuration for a four-light source multi-mode display system.

[0013] Figure 4C and Figure 4D Exemplary configurations of multi-LED packages are illustrated.

[0014] Figure 4E and Figure 4F An exemplary configuration of a polychromatic light array is illustrated.

[0015] Figure 5 is a perspective view of another embodiment of an optical device configuration for a four-light source multi-mode display system.

[0016] Fig. 6A and Figure 6B Exemplary light timings for two exemplary modes for a four-light source system are illustrated, showing a color sequential mode and a non-color sequential mode.

[0017] 7A to 7D Some exemplary light timings for implementation of various restricted light options are illustrated.

[0018] FIG. 7E to FIG. 7F Some exemplary light timings for a light display having multiple red, blue, and green light sources are illustrated.

[0019] Figure 8 is a block diagram of one embodiment of a multi-mode display system.

[0020] Fig. 9 is a flow chart of one implementation of an automatic mode change trigger.

[0021] Fig.10 It is a flow chart of an implementation scheme of dynamic display optimization.

[0022] Fig.11 is a block diagram of one embodiment of a computer system that may be used with the present application. DETAILED DESCRIPTION

[0023] A display operable in multiple modes is described. Each of these modes has a different light configuration, wherein the light source, their duty cycle, their power level and / or their combination is variable. In one embodiment, these modes include a first mode with full-color RGB and a second mode with a reduced color space (RCS). In one embodiment, these modes include a color sequential operation mode and a non-color sequential (NCS) operation mode. In another embodiment, these modes include a full-color RGB mode (which may or may not be color sequential) and a reduced color space mode (which may or may not be color sequential). The reduced color space mode sacrifices color (the user perceives less color space, or only perceives a single color), but obtains the advantages of increased brightness and / or lower system power consumption. For many use cases included in an augmented reality (AR) head-mounted display (HMD), RCS mode and NCS mode may be advantageous. The augmented reality HMD can be worn outdoors, where the display brightness is required to be visible even in the presence of sunlight. By providing a multi-mode display, a full color space can be provided when operating in a color sequential mode, and additional brightness and reduced power can be provided in an RCS mode. Therefore, the system can be optimized for both use cases without the need for separate displays. The multi-mode display also provides optimized night-time display.

[0024] Reduced Color Space (RCS) mode is a mode in which the display changes its light source so that the color space that can be represented in a single image frame is reduced compared to full color RGB mode. In full color RGB mode, the color space can generally be represented by a triangle, where each of the red, green, and blue light sources represents a vertex of the triangle. In one embodiment, the RCS mode displays only a single color per image frame, in which case each image frame represents only a single point in the color space. In another embodiment, the RCS mode displays 2 color frames per image frame, in which case the color space of the system becomes a 1-dimensional line.

[0025] In one embodiment, the RCS mode may change the color selected in a subsequent frame or series of frames. In one frame, the color space of the system is a one-dimensional line, and in a subsequent frame or series of frames, the color space of the system will be a different one-dimensional line. Drive current tuning can be used to provide such changes. This will result in an experience where the user will perceive one image containing two colors and then a subsequent image containing two different colors. This provides the advantages described above, namely higher brightness and reduced power. In one embodiment, these different color selections are made based on the content being displayed. In another embodiment, the color selection is based on the environment.

[0026] The multi-mode display system can dynamically change the color sequence. In one embodiment, the system can be set to a pre-configured sequence in a plurality of pre-configured sequences, which are designed to address various internal and external factors. In one embodiment, a machine learning / artificial intelligence system is used to analyze the possible color sequences for display and generate settings for optimizing color display. In one embodiment, the color sequence can be optimized based on various factors, including one or more of the following: battery use (efficiency), background, image frame content, ambient light level, time of day, waveguide spectral response, temperature impact on LED or other light sources, etc. In one embodiment, the system can prioritize these customized factors based on user preferences, system settings and / or correlation. In another embodiment, the system can select a pre-configured sequence or optimize the sequence based on sensor input on a device (such as, but not limited to, a light sensor, a camera, an inertial measurement unit, a motion sensor, an eye tracking sensor, a touch sensor, an audio sensor) or via a button or other mechanism of the user input. Therefore, the present system can provide intelligent adjustment to provide a customized color mode display, wherein the mode can be changed based on a variety of factors.

[0027] In one embodiment, the system utilizes an illumination array. The illumination array can be an N×M array of LEDs. Using an array can enable segmented illumination of a spatial light modulator (SLM). In one embodiment, this enables different portions of the display to have different color sequences. In one embodiment, a separate array is used for each color. In another embodiment, a single array can include all available colors. In one embodiment, the colors are RGB LEDs. In one embodiment, the colors include RGB and a fourth color (amber or white), as will be discussed below.

[0028] The following detailed description of the embodiments of the present invention refers to the accompanying drawings, in which the same reference numerals indicate similar elements, thereby illustrating the specific embodiments of practicing the present invention by way of example. The description of these embodiments is detailed enough to enable those skilled in the art to practice the present invention. It will be appreciated by those skilled in the art that other embodiments may be utilized, and logical, mechanical, electrical, functional and other changes may be made without departing from the scope of the present invention. Therefore, the following detailed description should not be construed as restrictive, and the scope of the present invention is limited only by the appended claims.

[0029] Figure 1An embodiment of a multi-mode LCOS-based display system for displaying images (including still images and / or moving images) to a user is illustrated. This is merely an exemplary embodiment, and the system can be implemented in various display systems that utilize sequential color displays. For example, the system can be implemented in a head-up display, a projection display, a head-mounted display (HMD), a virtual reality display, an augmented reality display, a head-up display, etc.

[0030] The display system 100 uses red, green, and blue light sources to display subframes. In one embodiment, the light source is an LED 110. The use of red, green, and blue light sources 110 enables reconstruction of a sufficiently wide color gamut, thereby providing a full-color display experience for the user. In one embodiment, the light source can be a light emitting diode (LED), a laser, a micro-LED, an organic LED (OLED), a superluminescent diode (SLED), a phosphor, or a quantum dot. In one embodiment, the light source can be a segmented LED or an LED array. In one embodiment, the spatial light modulator (SLM) can be a liquid crystal on silicon (LCoS), a digital light processing (DLP), a digital micromirror device (DMD), or a liquid crystal display (LCD).

[0031] The light from these light sources passes through illumination optics 115A to 115C and is combined in one embodiment using a combiner such as an X-cube 120 or other optical element. The combiner 120 used can be different and can be a dichroic plate, a holographic optical element, or any other type of combiner to ensure that the light from the different light sources 110 is correctly positioned. In one embodiment, the light passes through a microlens array 130 and intermediate optics 135. The light is then reflected to a spatial light modulator (SLM), such as a liquid crystal on silicon (LCoS) 140. The modulated light then passes through final optics 150 before being displayed to the user. In one embodiment, the output of the final optics 150 is coupled into a combiner waveguide 160. In one embodiment, the combiner waveguide 160 is used to display an image via glasses, goggles, or another mechanism.

[0032] Typically, such LCoS and DLP systems use color sequential displays. In a color sequential display, the total brightness perceived by the user is a fractional sum of the duty cycle of each color. The duty cycle is the percentage of the frame time that a particular color is on. If the system is run so that each color subframe is on for one-third of the total frame time, the total system brightness will be one-third compared to a system where all three colors are on for the entire frame. Figure 2A An exemplary color sequential display is illustrated where the duty cycle of each of the red, green and blue LEDs is 1 / 3 of the frame.

[0033] The present multi-mode system provides a solution for applications where the ability to adjust color sequence and timing would be beneficial. One such application is where color width (full color) is not as important as brightness, power efficiency, and / or visual contrast. The multi-mode display can operate with different color sequences and timings, which is referred to as reduced color space (ROS) mode in some embodiments. In non-color sequential mode, each image frame displays only a single color. In one embodiment, in non-color sequential mode, one or more light sources can be enabled simultaneously to create a single combined color. This enables the color displayed to the user to be adjusted by changing the relative drive strength of each light source. In one embodiment, red, green, and blue light sources are enabled simultaneously so that the user perceives white. Figure 2B An exemplary non-color sequential mode display is illustrated, where all three colors are turned on simultaneously. In another embodiment, green and red are enabled simultaneously, so that the user perceives amber or yellow. Any other color combination may be used.

[0034] In one embodiment, only a single color is displayed to the user for the full frame. In one embodiment, the single color is green, which provides maximum photopic efficiency to the user's eyes. In one embodiment, the single color is red, which can allow the user to maintain dark / night vision capabilities. In one embodiment, the user can select the color. In one embodiment, the color can be automatically selected based on various factors including environmental and usage conditions.

[0035] In another embodiment, the color timing may vary based on a multi-mode preference. For example, a frame may consist of an RGBRGB sequence, where each on segment is 1 / 6 of the frame. The frame may also be RG / BG, where the red light and the green light are turned on at the same time, and then the blue light and the green light are turned on in another segment. In addition, the duty cycle of the combined light may be less than the length of the frame. For example, the time when the red light, the green light, and the blue light are all turned on may be less than 1 / 3 of the full frame, so that no light is turned on during a portion of the frame. The length of each light source turned on can vary. As described in the RG / BG example, in some cases, one or more lights may be turned on for the full frame, while other lights are turned on for a portion of the frame.

[0036] In one embodiment, multi-mode is enabled by adjusting the driving architecture of the light source. In a typical color sequential display system, such as Figure 3A As illustrated in FIG, the driving electronics for the light source uses a common anode drive architecture. Multi-mode display embodiments that can enable two or more colors simultaneously utilize an architecture that can control the light sources individually. Since light sources of different colors typically have different forward voltages, the drive architecture may need to support multiple forward voltages simultaneously. In one embodiment, this is achieved by using a single anode power supply and varying the cathode voltage for each color. Figure 3Acan be operated in a mode that enables this use case. While this approach can save space, it generally lacks efficiency because the low-side transistors of each color are operated in the linear region to maintain the forward voltage of each color, resulting in power losses.

[0037] In another embodiment, the system supports separate anode voltages for each electro-optic light source, but uses a single controller IC. Figure 3B An exemplary illustration of this configuration is shown in . This architecture may require more PCB space because regulating multiple anode voltages requires filtering of the electronics for each anode voltage line. However, having separate anode voltages generally enables more efficient operation per color when operating in RCS mode compared to a system where the low-side drive transistors operate in the linear region. In another embodiment, the system supports separate anode voltages for each electro-optical light source and uses a separate control IC for each light source. Figure 3C An exemplary configuration for this situation is shown in . In another embodiment, the light sources are designed to have sufficiently similar forward voltages so that a single anode voltage can be used for all light sources, while also enabling each low-side transistor to operate in the saturation region, or in a minimal linear region where the linear operation of the transistor consumes little power. Figure 3A Such a configuration is exemplified in . In one embodiment, sufficiently similar forward voltages for each light source are achieved by using the same diode design to drive each color and varying the emission color of some diodes. In one embodiment, phosphor conversion can be used to vary the diode color.

[0038] Figure 3D An embodiment is illustrated in which a multi-mode system is supported by an LED array 355 comprising N×M LEDs. In one embodiment, the LED arrays 355 are all single color, and there are three separate arrays, one for each color. In one embodiment, when each LED array 355 is single color, there may be three separate controller ICs 350 and associated LED arrays 355. In another embodiment, a single controller IC 350 may control two or more of the LED arrays in the LED array 355. FIG. 3A to FIG. 3C Any of the configurations illustrated in can be used with an LED array.

[0039] In one embodiment, LED array 355 includes multiple colors, rather than a single color array. In one embodiment, there are the same number of LEDs of each color in array 355. In another embodiment, the number of LEDs of each color can vary. For example, there may be more green LEDs than blue or red LEDs. In one embodiment, the LEDs of various colors are distributed so that a portion of the display can be illuminated with all colors. In one embodiment, LED array 355 is controlled by controller IC 350. In one embodiment, a multiplexer is used to address each LED in the array. In another embodiment, there may be a separate horizontal address and a vertical address controlled by a separate portion of controller IC 350. Other ways of controlling each LED in array 355 can be used.

[0040] In one embodiment, the LED array may have a passive matrix backplane. In another embodiment, the LED array may have an active matrix backplane.

[0041] Figure 4A and Figure 4B is a diagram of an embodiment of an optical device configuration for a four-light source multi-mode display system. The system illustrates Figure 4A An exemplary RGB light path and an exemplary NCS lighting path in FIG.

[0042] like Figure 4A As shown, in one embodiment, the system includes four LEDs in two LED packages 410, 460. The first LED package 410 includes red, green, and amber LEDs. The second LED package 460 includes a blue LED. The LED packages used can vary, for example, red / blue can be located in one package and green can be located in another package. Alternatively, one package can include red / green and the other package can include blue / green. Other configurations can also be used. In one embodiment, the color combination used depends on the use case. In one embodiment, the LED package can be an LED array, as will be discussed in more detail below.

[0043] The illumination optics 430, 470 include red / green / amber illumination optics 430 through which light from the red / green / amber LED package 410 passes and blue illumination optics 470 through which light from the blue LED 460 passes. In one embodiment, the illumination optics may include a compound parabolic concentrator. Other types of optics may also be used.

[0044] These lights are combined by combiner 480. Combiner 480 includes two dichroic plates 490, 495. Blue light passes through two plates toward projection optics (not shown), while green light is reflected by one dichroic plate 490 toward projection optics, and red light is reflected by another dichroic plate 495 toward projection optics. Although illustrated as being located at a certain distance, in actual configuration, the distance between the back side of green reflective plate 480 and the front side of red reflective plate 495 is minimized. In one embodiment, the light reflected off the plate closer to red and green LEDs 410 reflects the light from the back side, while the other plate reflects the light from the front side, to minimize displacement. In one embodiment, the dichroic plates 490, 495 of combiner 480 can be manipulable to change the angle when it reflects red / green LEDs compared to the LEDs converted by phosphors. In one embodiment, an actuator is used, and the dichroic plates 490, 495 are shifted between two positions. This has the advantage of better aligning the illumination pupil between full color mode and NCS mode.

[0045] Combiner 480 including a dichroic plate may be replaced by another type of combiner, such as an X-cube (also dichroic), or a combiner using refractive, dichroic, diffractive, freeform, catadioptric, metalens elements, or holographic optical elements.

[0046] Figure 4B The path is illustrated when the first LED package 410 uses a fourth light source (in this example, an amber LED). The first LED package 410 includes an amber (or white) LED that travels through the red / green / amber illumination optics 430 and is reflected by the green and red reflective dichroic plates 490, 495.

[0047] This is merely an exemplary configuration of these elements. In another configuration, the fourth color light source may be separate. In another embodiment, the light sources may come from three separate locations, for example, the fourth light source may have separate optics. Other optical configurations may also be used. Although this illustration relates to "LEDs", each of these light sources may be an LED array. In another embodiment, a single LED array may provide a light source of multiple colors.

[0048] Figure 4C and Figure 4D An exemplary configuration of a multi-LED package is illustrated. In this example layout, a green LED (G) and a red LED (R) are arranged side by side, with a phosphor-converted fourth color LED (P) positioned in the center above the red and green. Figure 4DAs shown, the phosphor-converted fourth color LED (P) can be smaller than the green LED (G) and the red LED (R). This configuration will have an RCS mode with a smaller field of view than the RGB mode. This may be useful in some cases, such as in an AR display, when information is only displayed on a portion of the screen.

[0049] In one embodiment, the fourth color is combined with the other colors using one or more polarizing elements, such as, but not limited to, liquid crystal rotators, liquid crystal elements, polarizing reflectors, geometric phase lenses, polarizing beam splitters, and wave plates. In one embodiment, the light can be polarized such that the RGB light has a first polarization and the fourth light has an opposite polarization.

[0050] In one embodiment, the fourth color is combined with dichroic elements such as, but not limited to, dichroic filters, dichroic combiners, and dichroic reflectors.

[0051] In one embodiment, colors are combined and selected using mechanical elements such as, but not limited to, movable mirrors, movable optics, movable filters, shutters, MEMS devices, deformable mirrors, tunable lenses, or movable prisms.

[0052] In one embodiment, the system has multiple inputs and multiple outputs for illumination, and the inputs and outputs correspond to one or more colors. In another embodiment, the multiple inputs / outputs are angle selective.

[0053] In one embodiment, the illumination color combination uses one or more elements which may include any of the listed polarizing elements, dichroic elements, mechanical elements, and / or multiple elements, or other elements useful for combining illumination.

[0054] Figure 4E An embodiment of a 4×4 LED array is illustrated. The LED array 430 includes multiple LEDs of each color. In this example, there are four clusters 435 of four LEDs, one of which is red (R), one is blue (B), and two are green (G). However, this is merely an exemplary arrangement in which the colors are clustered using two greens. In another embodiment, the LED cluster 435 may alternatively include a fourth color, which may be white, yellow, or amber. Other ways of gathering light may be used. In some embodiments, the shape of each light source may vary. In addition, not all colors need to have equal areas or shapes. For example, since the human eye is most sensitive to green, in some embodiments, the green light source in the array 430 may be larger. Additionally, the layout of the array may vary.

[0055] Figure 4F An N×M LED array is illustrated. The LED array 440 includes M rows of LEDs, each row having N LEDs. As described above, the color arrangement may include one, two, three, or four colors in the array. In one embodiment, each LED represents an array element 445 having all three colors RGB. Although the illustration shows the colors in each array element 445 as the same size, the actual size of the color element may vary by color and / or across the array.

[0056] In addition to arrays, the present application may also use segmented LEDs. A segmented LED is a single color LED with two or more segments that can be controlled individually. For example, this can be used to illuminate a portion of a display area with a specific color. This may be particularly useful for use cases where there are areas where limited color data is continuously displayed. For example, for a head-up display (where the bottom third of the display shows speed, temperature, and similar digital data), it may be efficient to have a segmented LED that can provide a full-color high-color resolution image in the top 2 / 3 of the display while providing a reduced color display in the bottom 1 / 3. Arrays can also be used for this purpose. In one embodiment, the segmented LED can be considered a 1×N array of a single color.

[0057] Figure 5 5 is a perspective view of another embodiment of an optical device configuration for a four-light source multi-mode display system. In this configuration, the full-color mode illumination block 510 is separated from the RCS mode illumination block. The full-color mode illumination block 510 has light sources combined by a combiner 520 and reflected by a reflector 530 toward the projection optics 570. The RCS mode light source 550 passes through a diffractive / holographic combiner 560 on its way to the projection optics 570, which in one embodiment is a waveguide. The projection optics 570 is shared by the RCS mode light 550 and the full-color mode light.

[0058] Fig. 6A and Figure 6B Exemplary light timing for two exemplary modes for a four-light system is illustrated, showing a color sequential mode and a non-color sequential mode. Fig. 6A , the color sequential mode shows each color during 1 / 3 of the full frame. In contrast, for the color sequential mode, in one embodiment, the fourth LED (RCS_0) is turned on during the entire frame. In another embodiment, the fourth LED is turned on during a portion of the frame. In one embodiment, there is some blanking time between frames. In another embodiment, all four LEDs are turned on at the same time to maximize the brightness of the system.

[0059] 7A to 7D Light timing for various embodiments of confined light configurations is illustrated. Fig. 7AAn exemplary timing for green-only display is illustrated. In one embodiment, the green light source is turned on for the entire frame. As shown in Green_0, full brightness. In another embodiment, as shown in Green_0*, the green light source is turned on for 50% of the frame. This is called enhanced contrast mode. Since LCOS takes some time to turn on / off, leaving blanking time between frames can reconstruct the image more accurately and provide higher contrast. Although a lower duty cycle can be used, a fully turned-on system is more efficient because it generally allows a lower drive current, which enables the LED to operate in a more efficient operating region. Alternatively, Green_0** illustrates a distributed green LED turn-on time, where the green light is turned on on two pulses during a single frame, rather than turning the green light source on half a frame and off the other half a frame. This can reduce flicker. Other pulse schemes can be used. Although this is discussed for green light, similar pulsed lighting can be used for any color. Additionally, this type of pulsed lighting can be used with displays that include more than a single color.

[0060] Figure 7B An exemplary timing for red-only display is illustrated. Although this is illustrated as the red light being on for the entire frame, those skilled in the art will appreciate that the red light may be on only part of the time.

[0061] Figure 7C An exemplary timing sequence for a red / green display is illustrated. In one embodiment, the red and green LEDs are enabled simultaneously, and the drive current to each LED is changed to achieve the desired amber hue. In one embodiment, the red and green LEDs are driven sequentially, and a combination of drive current and duty cycle is used to achieve the desired amber hue. In this embodiment, the displayed content can achieve any color between red and green. In another embodiment, the red and green LEDs can be enabled simultaneously for only a portion of the frame, and one of the colors is allowed to be turned on for a different duration. Fig.7D An exemplary timing for a display is illustrated where a first color sequence that displays red and green simultaneously is combined with a second color sequence that displays blue and green simultaneously.

[0062] Although various duty cycles for different examples are illustrated, it should be understood that any of these configurations may have any illustrated duty cycle. To maximize brightness, the duty cycle may be 100%. However, for example, to save battery, or when full brightness is not required, a reduced duty cycle may be used. Additionally, the frame rate may vary for different light configurations. In one embodiment, the frame rate for the full spectrum display is 60 Hz (60 frames per second). In one embodiment, the frame rate may be reduced or increased for reduced color space options. In one embodiment, for example, for a head-up display, the frame rate may vary between 30 Hz and 360 Hz (or the highest frame rate available for the spatial light modulator). In one embodiment, a variable frame rate may be used to display persistent information at a frame rate lower than 60 Hz. For example, for a clock displaying hours and minutes, the frame rate may be reduced to one frame per minute, such as 0.016 Hz.

[0063] Fig. 7E An embodiment of the timing of an LED array is illustrated, in which there are two LEDs for each color. In this example, for the first frame Frame0, the two groups of LEDs are illuminated sequentially, but for the second frame Frame0, only the second group of LEDs is illuminated. In one embodiment, this can enable the second frame to illuminate only a portion of the display.

[0064] For example, if the system is outputting an augmented reality image and there is content on only one portion of the screen, this enables the system to save power and still provide a high quality image in the portion of the frame that has data. This may also be useful for displaying content such as closed captions, road signs, or a continuous clock on a portion of the display. In one embodiment, the second half of the frame may include only a single color or subset of colors being displayed.

[0065] Figure 7FAn embodiment of the timing of an LED array is illustrated, wherein there are two LEDs for each color. In this example, during a single frame, a first group of RGB LEDs (Redo, Green0, and Blue0) are illuminated sequentially during the entire frame. In this example, each of these LEDs illuminates approximately one-third of the frame in a color sequential manner. Another group of LEDs (Red1, Green1, and Blue1) are also illuminated sequentially throughout the frame, but some of these LEDs illuminate for a longer period of time than other LEDs. In this example, the red LED is turned on for one-sixth of the frame, the green LED is turned on for half of the frame, and the blue LED is illuminated for the remaining two-sixths of the frame. Therefore, the duty cycle can vary according to the color and the illuminated portion of the display. In this case, the display panel is synchronized with each illumination sub-segment. This is of course only an example, and those skilled in the art will understand that the relative length of each LED on time can vary, and there may be a period of time when no light is illuminated.

[0066] Figure 8 8 is a block diagram of one embodiment of a multi-mode display system. Mode selection system 810 provides display settings to display system 870, which utilizes color sequences identified by mode selection system 810 when displaying content. In one embodiment, mode selection system 810 utilizes data from sensor 830. In one embodiment, display settings may be specified on a frame-by-frame basis. In another embodiment, display settings may be sent only when the display settings change from the current settings. In one embodiment, mode selection system 810 receives pre-calculated color sequences from AI / ML system 890.

[0067] In one embodiment, the mode selection system 810 is implemented by a computer system including a processor and a memory. In one embodiment, the mode selection system 810 is a computer system that provides instructions to the display system 870. The processor that provides the functionality of the mode selection system 810 can be divided across multiple devices. For example, some or all of the processing can be completed on the glasses or other display system itself. Other parts of the processing can be completed on a remote device. In one embodiment, the pre-calculated sequence 857, the mode selector 855, the environmental background analyzer 840, and the content analysis 850 can be part of the display system 870. In one embodiment, one or more sensors in the sensor 830 can be located in the display system 870. The mode selection system 810 may not be a device separate from the display system 870.

[0068] The AI / ML system 890 includes one or more processors that receive data from and provide data to the mode selection system 810. In one embodiment, the AI / ML system 890 may be a remote server. In one embodiment, the AI / ML system 890 may be implemented in the cloud using distributed processing and storage.

[0069] The display settings from the mode selection system 810 are sent by the display controller 860 based on a determination of one or more factors. As described above, in one embodiment, one of these factors is a user preference received via the user interface 815.

[0070] A particular mode may be automatically enabled in the display system based on a use case or environmental conditions by the mode selection system 810. In one embodiment, the ambient light sensor 832 is turned on when the system is in an AR format or a heads-up format, or another format in which image data is displayed over other external data (as identified by the display format identifier 820).

[0071] In one embodiment, the mode selection system 810 selects the RCS mode when the ambient light sensor 832 detects a high ambient light level. In one embodiment, the mode selection system 810 enables the RCS mode when a thermal sensor 835 in the system detects that the system is getting too hot and the system should be operated in a more power efficient mode. In one embodiment, the thermal sensor 835 indicates when the temperature of the display system 870 or the processing system is above a threshold. In one embodiment, when the temperature is above a threshold, a color display may be selected to reduce power consumption. In one embodiment, when the temperature of the LED changes, the color space may be adjusted to correct for color drift due to temperature. In one embodiment, this may be detected based on data from the thermal sensor 835. In one embodiment, a display sensor may be used to monitor the output of the lighting system 880 and adjust for color drift based on the detected color spectrum compared to the expected color spectrum.

[0072] In one embodiment, when the environment background analyzer 840 detects visual content that can be more advantageously shown in monochrome or reduced color space format, the RCS mode is enabled. In one embodiment, the visual content is, for example, a text-based or reading application, or a simple image. In one embodiment, when the ambient light sensor 832 detects low ambient light and the clock 825 indicates that it is evening, the RCS mode is enabled. In one embodiment, when the clock indicates that it is evening, the system can operate in night mode to better maintain the user's dark vision / night vision ability. In one embodiment, this may include utilizing only a red display. In one embodiment, when the battery sensor 845 detects low power, the RCS mode is enabled in a battery-powered system as a power saving mechanism. In one embodiment, a preference or target can be set for a battery threshold, and when it is lower than the battery threshold, the system switches to a power saving mode. In one embodiment, if the display system is displaying a movie or other content of known length, the system can switch to a reduced color space mode to ensure that the entire movie or other content can be displayed in the case of remaining battery power.

[0073] In one embodiment, the sensor 830 also includes an eye tracker 867 to determine where the user's gaze is looking. In one embodiment, the system can utilize different color spectra for the focal area and peripheral areas of the user's gaze.

[0074] In one embodiment of the AR HMD, the system includes an outward-facing AR background sensor 837 that images the same field of view as the display displayed to the user. The environmental background analyzer 840 analyzes the background data, and the mode selection system 810 selectively enables a specific display mode to optimize the display for the background of the user's environment. In one embodiment, the optimization is designed to provide as high a contrast as possible between the display and the ambient lighting and color conditions. For example, if the user's field of view aligned with the AR display includes light that is primarily red in color, the system can switch to an operating mode that uses more blue and green light to provide a high-contrast virtual image compared to the red light seen by the user.

[0075] In one embodiment, the display field of view is selected by the field of view selector 865. In some embodiments, there may be an RCS mode with a smaller field of view than the RGB mode.

[0076] The user can manually put the display system into a specific mode using the user interface 815. The user interface 815 can be a button, a dial, a screen, a voice interface, or another device that enables the user to select a display mode. In one embodiment, the user can put the system into a dark vision / night vision mode to maintain night vision. In one embodiment, the user can put the system into a night mode to reduce or completely eliminate blue light emissions from the system. In one embodiment, the user can put the system into an RCS mode because they are watching content that is better represented in a monochrome format. In one embodiment, the user can select the RCS mode because they will use the system in a high-brightness environment. In one embodiment, the user can put a battery-powered system into an RCS mode as a power saving mechanism to maintain or extend battery life. The user can also set general preferences via the user interface 815. Such general preferences can be saved as user preferences 817. Preferences can be global, for example, the user prefers high color fidelity rather than lower power consumption; they can also be time-based, for example, at night, the user prefers lower brightness and warmer tones with less blue light. Other preferences can be set by the user. Target selection 852 can be based on a combination of user preferences 817, data from sensors, and image data from content analysis 850.

[0077] The color or color combination for the selected display mode may be selected by the mode selector 855 based on the use case, user preferences, content display, and / or environmental conditions. In one embodiment, the system may select a preconfigured sequence or optimize the sequence based on sensor data from the sensor 830. In one embodiment, the sensor 830 may be located on the device. The sensor may include, but is not limited to, a light sensor 832, an eye tracking sensor 857, a thermal sensor 835, a user interface element 815 such as a touch sensor, an audio sensor, or other type of user input sensor, and other sensors 869 such as a camera, an inertial measurement unit, a motion sensor. The mode selector 855 selects the color, color sequence, and timing. In one embodiment, the mode selector 855 utilizes one of a plurality of pre-computed sequences 857. In one embodiment, the pre-computed sequence 857 is received from the AI / ML system 890.

[0078] In one embodiment, the AI / ML system 890 is a remote server system that calculates potential sequences used by the mode selection system 810. In one embodiment, the AI / ML system 890 is an offline system that pre-calculates such patterns for various lighting, environmental conditions, display content, and other factors. In one embodiment, the AI / ML system 890 utilizes a machine learning system 896 that receives data from a data collection device 894, with various potential sequences and available sensor data and calculated data. In one embodiment, the machine learning system 896 is initially trained with data from expert users that classifies color quality and visibility in various scenes (e.g., scenes with various AR backgrounds and real-world scene content). Then, the trained machine learning system 896 can be used to create pre-calculated sequences optimized for various scenes. The data is passed to the mode selection system 810, where the mode selector 855 can select a pre-calculated sequence 857 based on data from sensors 830, environmental background analyzers 840, and other data sources.

[0079] In one embodiment, a system with an ambient light sensor 832 detects a high brightness environment and displays red, green, and blue simultaneously to provide a user with high brightness white visual content. In one embodiment, a system with an ambient light sensor 832 detects a low brightness environment and displays only red in RCS mode to preserve the user's dark / night vision capabilities. In one embodiment, a battery-powered display detects low battery 845 and displays only green to maximize photopic efficiency and minimize power consumption so that battery life can be extended.

[0080] In one embodiment, a thermal sensor 835 in the system detects that the system is becoming overheated and enables an RCS mode that displays only green in order to maximize photopic efficiency and minimize further heat generated by the system. In one embodiment, green and red light sources are enabled in RCS mode to provide high photopic efficiency to the user while providing a more visually pleasing color than pure green.

[0081] In one embodiment, content analysis 850 analyzes the content of the image being displayed. In one embodiment, the system can optimize the color sequence based on the content being displayed. For example, if the content is best displayed in monochrome, the system can adjust the color sequence to monochrome. In one embodiment, for a display with an array of light sources, a portion of the display can have a different color sequence than another portion of the display. For example, in a display with a continuous clock in one area, that area can utilize a different color scheme than the portion of the display with the image. Similarly, closed captions can be displayed in monochrome while the image above it is displayed in a full spectrum color image.

[0082] In one embodiment, the alternative operating mode has two color sequences selected by the mode selector 855. In one embodiment, the two color sequences are red and green color sequences. In one embodiment, the first color sequence displays red and green simultaneously, and the second color sequence displays blue and green simultaneously. In one embodiment, the two colors selected can change on a per-frame basis. In this case, the user will see a first frame with two colors, followed by subsequent frames with two different colors. For this color switching, in one embodiment, the color selection is made based on the content.

[0083] These embodiments are for illustration and are not intended to be limiting of two color sequences.Operating with two color sequences enables non-monochromatic displays while also achieving some of the advantages of a single color mode (increased brightness and / or power efficiency).

[0084] The display system 870 includes a buffer / memory 872 to store display settings from the mode selection system 810. The display system also includes optics 876, a spatial light modulator 874, an optional waveguide 878, and an illumination system 880. The illumination system 880 can have separate light sources 882, 884 for full color mode versus reduced color space mode. In one embodiment, the full color mode uses red, green, and blue light sources 882, while the RCS mode uses a white light source 884. In one embodiment, the white light source is a white LED.

[0085] In one embodiment, the full color mode uses red, green, and blue light sources 882, while the RCS mode uses phosphor-converted green / amber LEDs 884 placed adjacent to the red and green LEDs in the illumination subsystem. In one embodiment, the RCS light sources are combined using a combiner 888. The combiner 888 may include reflective, refractive, dichroic, diffractive, or holographic optical elements. In one embodiment, the combiner 888 enables the use of a shared optical element 876 between the RGB color sources 882 and the fourth color source 884.

[0086] In an illumination system 880 that uses a dichroic plate as a combiner 892 to combine colors, this arrangement has the benefit of being able to effectively use the same dichroic film combiner for both full color mode and RCS mode. In one embodiment, the spatial light modulator 874 can be an LCoS, DLP, or LCD.

[0087] In one embodiment, the RCS may use a different light source than the light source 882 of the full color mode. In one embodiment, the different light source 884 may have a different illumination angle than the full color mode light source. In one embodiment, the RCS mode has a different field of view than the full color mode. In one embodiment, the light source may be an LED, a laser, a micro LED, an OLED, a SLED, a phosphor, or a quantum dot.

[0088] In one embodiment, the display system 870 has an LCoS spatial light modulator 874 and a polarizing beam splitter as part of the combiner 888. In another embodiment, the display system 870 has an LCoS spatial light modulator and a non-polarizing beam splitter as part of the combiner 888. In one embodiment, the display system has a DLP spatial light modulator with a prism. In another embodiment, the display system has a DLP spatial light modulator without a prism.

[0089] In one embodiment, the display system uses different gamma tables for different types of sequences, including RCS sequences. In one embodiment, the system has separate gamma tables for red, green, and blue, respectively, and another gamma table for a mode where red, green, and blue are on simultaneously. These gamma tables can be custom tuned for optimal contrast, optimal brightness, to compensate for thermal variations in LEDs and panels, or based on some other system performance metric.

[0090] Fig. 9 9 is a flow chart of one embodiment of automatic mode change triggering. The process begins at block 910. In one embodiment, the mode change may be dynamic, and the mode may change on a per-frame basis.

[0091] At block 915, the process determines whether the user has selected a mode. In one embodiment, the user can manually select any mode and override the automatic mode selection. If the user has selected the mode, then at block 920, the display mode is set according to the user's selection. In one embodiment, the display modes may include full color RGB display and various reduced color space modes, which may be non-sequential or sequential.

[0092] If the user has not selected a mode, then at box 925, the system determines whether the system is being used in AR / heads up mode. If the system is not in AR / heads up mode, then at box 930, the process determines whether low battery or high heat is a problem. When battery is low or heat is high, at box 940, a reduced power display mode is selected. In one embodiment, this can be a green-only display mode. Alternatively, another monochrome or reduced color space display mode may be selected. If power or heat is not an issue, then at box 935, a full color space mode is selected. If AR mode is identified at box 925, the process continues to box 945.

[0093] At block 945, environmental factors are identified. These environmental factors may include, for example, brightness, time of day, color spectrum of the real world scene being viewed.

[0094] At block 950, an appropriate mode is selected based on the determination. The process then ends. As described above, in one embodiment, the user can override the automatic mode determination. Additionally, in one embodiment, the user can trigger a re-evaluation of the selected mode.

[0095] Fig.10 is a flow chart of one embodiment of adjusting the color sequence based on various factors. The process begins at block 1010. At block 1015, a default display mode is selected. In one embodiment, the default display mode is full color display.

[0096] At block 1020, the process determines if waveguide tuning should be done. In one embodiment, waveguide tuning is done on the initial configuration of the system. If waveguide tuning has not been done, then at block 1025, the system adjusts the color display based on the spectral characteristics of the waveguides in the actual device. This enables the colors to remain consistent across different waveguides. At block 1027, the default mode is updated based on the optimization. The process then continues to block 1030 to continue customization.

[0097] At block 1030, the process determines whether the user has set a fixed preference. If so, at block 1035, the user's preferred color settings are applied. Then, at block 1037, the process updates the default mode. The process then ends at block 1075 because the user-set preferences override other configuration changes. If the user has not set a fixed preference, the process continues to block 1040 to continue adjustments.

[0098] At block 1040, the process determines whether the display is an augmented reality display. If so, at block 1045, the real world scene on which the display is being displayed is evaluated and the color mode is set based on environmental factors. The process then returns to block 1050. If the display is not an AR display, the process continues directly to block 1050.

[0099] At block 1050, the process determines whether temperature affects color. If so, then at block 1055, the environment in which the display is being displayed is evaluated and the color mode is set to adjust for color drift due to temperature. The process then returns to block 1060. If temperature does not affect the display, the process continues directly to block 1060.

[0100] At block 1060, the process determines whether smart adjustment is enabled. Smart adjustment is adjusting the display color mode based on the content being displayed. In one embodiment, the color mode can be enabled by the user. If smart adjustment is enabled, the process continues to block 1065. At block 1065, the content of the image frame being displayed is evaluated and an optimized color mode is selected for the frame. In one embodiment, as described above, an optimized color frame is selected to enhance display quality. In one embodiment, other factors, such as battery level, may be considered when selecting a display mode.

[0101] At block 1070, the process determines whether the system is still displaying. If so, the process returns to block 1050 to continue monitoring temperature and providing intelligent adjustments. In one embodiment, although only intelligent adjustments and temperature are shown here, the other display factors discussed above will also apply similarly. Once the display ends, the process ends at block 1075.

[0102] Of course, although the above figures are illustrated as flow charts, in one embodiment, the order of operations is not limited to the illustrated order unless the processes are dependent on each other. In addition, in one embodiment, the system can be implemented as an interrupt-driven system, and therefore the system does not check the event, but the event sends a notification to trigger an action.

[0103] Fig.11 is a block diagram of one embodiment of a dedicated computer system. However, it will be apparent to those of ordinary skill in the art that other alternative systems of various system architectures may also be used. In addition, the present system may have certain features implemented using a distributed cloud system.

[0104] Fig.11 The computer system illustrated in the example includes a bus or other internal communication means 1140 for transmitting information, and a processing unit 1110 coupled to the bus 1140 for processing information. The processing unit 1110 can be a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), or any combination of the above elements or another type of processing unit 1110.

[0105] In one embodiment, the system also includes a memory 1120, which may be a random access memory (RAM) or other storage device 1120, coupled to the bus 1140 for storing information and instructions to be executed by the processor 1110. The memory 1120 may also be used to store temporary variables or other intermediate information during execution of instructions by the processing unit 1110. In one embodiment, the system also includes a read-only memory (ROM) 1150 and / or other static storage device 1150 coupled to the bus 1140 for storing static information and instructions for the processor 1110. In one embodiment, the system also includes a data storage device 1130 coupled to the bus 1140 for storing information and instructions.

[0106] In some embodiments, the system may be further coupled to an output device 1170, such as a computer screen, speakers, or other output mechanisms. An input device 1175 may be coupled to the bus 1160. The input device 1175 may be an alphanumeric input device, such as a keyboard, cursor control device 1180, touch screen, or other input mechanism.

[0107] Another device that may be optionally coupled to computer system 1100 is a network device 1185 for accessing other nodes of the distributed system via a network.

[0108] Please note that Fig.11 Any or all of the components of the system shown in and associated hardware may be used in various embodiments of the present invention.

[0109] Those skilled in the art will appreciate that a particular machine embodying the invention may be configured in various ways depending on the particular implementation. The control logic or software implementing the invention may be stored in main memory 1120, mass storage device 1130, or other storage media accessible locally or remotely by processor 1110.

[0110] It will be apparent to one of ordinary skill in the art that the systems, methods, and processes described herein may be implemented as software stored in main memory 1120 or read-only memory 1150 and executed by processor 1110. The control logic or software may also reside on an article of manufacture comprising a computer-readable medium having computer-readable program code embodied therein and which may be read by mass storage device 1130 and used to cause processor 1110 to operate in accordance with the methods and teachings herein.

[0111] The present invention may also be embodied in a dedicated device (such as a head mounted display (HMD) or a head up display in a vehicle or other environment) that includes a subset of the above-mentioned computer hardware components. For example, an appliance may include a processing unit 1110, a data storage device 1130, a bus 1140, and a memory 1120, and may not include an input / output mechanism, or may include only basic communication mechanisms, such as a small touch screen that allows a user to communicate with the device in a basic manner. Generally speaking, the more dedicated the device, the fewer elements are required for the device to operate. In some devices, communication with the user may be performed through a touch screen or similar mechanism. In one embodiment, the device may not provide any direct input / output signals, but may be configured and accessed through a website or other network-based connection through a network device 1185.

[0112] Those of ordinary skill in the art will appreciate that, according to a specific implementation, any configuration of a specific machine implemented as a computer system may be used. Control logic components or software implementing the present invention may be stored on a machine-readable medium that is locally or remotely accessible by processor 1110. Machine-readable media include any mechanism for storing information in a form that is readable by a machine (e.g., a computer). In one embodiment, control logic components may be implemented as transmittable data, such as propagation signals (e.g., carrier waves, infrared signals, digital signals, etc.) in electrical, optical, acoustic or other forms.

[0113] In addition, in one embodiment, the system can be implemented on a distributed computing system. In a distributed computing system, processing can be performed on one or more computer systems. The system can use computer system 1100 to provide local processing, and also utilize one or more remote systems for storage and / or processing. In one embodiment, the system can also utilize distributed computers. In one embodiment, computer system 1100 can represent a client and / or server computer on which software is executed. Without departing from the scope of the present disclosure, other configurations of the processing system that executes the process described herein can be utilized.

[0114] In the foregoing description, the mode selection system has been described with reference to specific exemplary embodiments of the present invention. However, it will be apparent that various modifications and changes may be made to the present disclosure without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Accordingly, the description and drawings are to be regarded as illustrative rather than restrictive.

Claims

1. A multi-mode display, the multi-mode display comprising: a mode selector for selecting one of a plurality of modes, each of the modes having a different light configuration, wherein one of the modes comprises a reduced color space mode; and One or more light sources, the one or more light sources being controlled by the mode selector, the one or more light sources being used to display content to a user using the multi-mode display.

2. The multi-mode display of claim 1, wherein the mode selector comprises a user interface.

3. The multi-mode display of claim 1 , wherein the mode selector comprises: an ambient light sensor for determining an ambient light level; and An ambient context analyzer is configured to determine the light configuration based on the ambient light level.

4. The multi-mode display of claim 1 , wherein the mode selector comprises: a thermal sensor for determining a temperature of the display system; and A mode selector is configured to select the reduced color space mode when the temperature is above a threshold.

5. The multi-mode display of claim 1 , wherein the mode selector comprises: a battery sensor for determining a power level of a battery powered display; and A mode selector is provided for selecting a reduced color space when the power level is below a threshold.

6. The multi-mode display of claim 1 , wherein the mode selector comprises: a clock for determining the time of day; and A mode selector for selecting a color space to preserve scotopic vision capabilities.

7. The multi-mode display of claim 1 , wherein the mode selector comprises: A display data analyzer is provided for determining a color of a real world scene on which an image is to be displayed and selecting the mode based on the color of the real world scene.

8. The multi-mode display of claim 1, wherein the first mode is a color sequential mode, whereby red, green, and blue light sources are sequentially enabled during a frame, and the second mode is an alternative mode.

9. The multi-mode display of claim 8, wherein the alternative mode is a non-color sequential mode comprising all three of the red, green and blue light sources being turned on simultaneously to produce a white image.

10. The multi-mode display of claim 8, wherein the alternative mode is a non-color sequential mode including use of a fourth light source.

11. The multi-mode display of claim 8, wherein the alternative mode is a non-color sequential mode comprising using one of the red, green, and blue light sources.

12. The multi-mode display of claim 8, wherein the alternative mode comprises a color sequential mode using a reduced set of colors.

13. The multi-mode display of claim 1, wherein the frame rates between the modes are different.

14. The multi-mode display of claim 1, wherein the mode is selected based on content being presented.

15. The multi-mode display of claim 1, wherein the mode is selected on a per-frame basis.

16. The multi-mode display of claim 1, further comprising: One or more light sources, the one or more light sources comprising a light array, the light array comprising a plurality of LEDs, wherein the light array is configured to selectively illuminate a portion of a display such that a first portion of the display has a different color spectrum than another portion of the display.

17. A multi-mode display for augmented reality, the multi-mode display comprising: a mode selection system for selecting a color sequence based on data from one or more sensors, wherein the color sequence includes a full spectrum display sequence and a reduced color space display sequence; an illumination source, the illumination source being used to provide a plurality of light sources, the illumination source being controlled by an output end of the mode selection system; a combiner for combining light from the plurality of light sources; a spatial light modulator, the spatial light modulator being used to receive the combined light from the combiner, the spatial light modulator being used to modulate the light; and An output end, wherein the output end is used to output the modulated light to a user.

18. The multi-mode display of claim 17, wherein the one or more sensors include one or more of: an ambient light sensor, a thermal sensor, an augmented reality background sensor, and a battery sensor.

19. The multi-mode display of claim 17, wherein the mode is selected on a per frame basis.

20. A method of providing a multi-mode display, the method comprising: selecting one of a plurality of display modes, each of the display modes having a different light configuration, wherein one of the modes comprises a reduced color space mode; as well as The output of one or more light sources is controlled based on the selected display mode to display content to a user using the multi-mode display.

Images