Variable refresh rate

Abstract

The display controller (200) includes a variable refresh rate circuit (216). The variable refresh rate circuit (216) is configured to receive a first video frame and provide a first dithered segment of the first video frame, the first dithered segment including instructions to display the first dithered segment during receiving a second video frame. The variable refresh rate circuit (216) is further configured to, in response to receiving the second video frame and completing display of the first dithered segment, provide a second dithered segment of the second video frame, the second dithered segment including an instruction to display the second dithered segment.

Description

BACKGROUND OF THE INVENTION

[0001] Many image display systems utilize a spatial light modulator (SLM). An SLM includes an array of individually addressable and controllable pixel elements that modulate light according to an input data stream corresponding to the pixel data of an image frame. A digital micromirror device (DMD) is a type of SLM. A DMD has an array of micromachined pixel elements, each of which has a tiny mirror that can be individually addressed by an electrical signal. Depending on the state of its addressing signal, each mirror element tilts so that it reflects light to or does not reflect light to the image plane.

[0002] Other examples of SLMs include liquid crystal displays (LCDs) or liquid crystal on silicon (LCOS) displays having individually driven pixel elements. An LCOS device includes a liquid crystal display shutter disposed between a mirror and a glass layer. The liquid crystal is modulated to allow light to pass through, and the light passing through the liquid crystal is reflected by the mirror through a color filter. In various types of SLMs, each frame of display pixel data is implemented by loading a storage unit so that the pixel elements can be addressed simultaneously. SUMMARY OF THE INVENTION

[0003] In one example, a display controller includes a variable refresh rate circuit. The variable refresh rate circuit is configured to receive a first video frame and provide a first dither segment of the first video frame, the first dither segment including an instruction to display the first dither segment during the reception of a second video frame. The variable refresh rate circuit is further configured to, in response to receiving the second video frame and completing the display of the first dither segment, provide a second dither segment of the second video frame, the second dither segment including an instruction to display the second dither segment.

[0004] In another example, a method includes: receiving, by a display controller, a first video frame; and providing, by the display controller, the first video frame as first, second, and third color segments to a spatial light modulator (SLM). The color segments include a plurality of dither segments. The method further includes: receiving, by the display controller, a second video frame while the first video frame is being displayed by the SLM; and in response to receiving the second video frame and completing the display of the dither segments of the first video frame, providing, by the display controller, the dither segments of the second video frame to the SLM.

[0005] In another example, a system includes an SLM and a display controller. The display controller includes a video input, an SLM control output, and a variable refresh rate circuit. The SLM control output is coupled to the input of the SLM. The variable refresh rate circuit is configured to receive a first video frame via the video input and to indicate via the SLM control output that the SLM display a first dither segment of the first video frame during reception of a second video frame. The variable refresh rate circuit is further configured to, in response to receiving the second video frame and completing display of the first dither segment, indicate via the SLM control output that the SLM display a second dither segment of the second video frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is an example timing diagram showing display of video frames at variable refresh rates.

[0007] Figure 2 and 3 is a block diagram of an example system for displaying video at variable refresh rates.

[0008] Figure 4 is a timing diagram illustrating example source and display frame timings in a system with variable refresh rates.

[0009] Figure 5 and 6 is a flow diagram of a method for displaying video at variable refresh rates.

[0010] Figure 7 is a flow diagram of an example method for triggering generation of dither segments of a video frame in response to storage of the video frame.

[0011] Figure 8 is a timing diagram illustrating example reception of a video frame stored in a buffer that is subdivided into a plurality of sub-buffers.

[0012] Figure 9 is a flow diagram of an example method for triggering generation of dither segments of a video frame in response to storage of the video frame.

[0013] Figure 10 is a timing diagram illustrating example display frame rate adjustment based on a variable refresh rate source video.

[0014] Figure 11 is a flow diagram of a method for providing a display frame rate in a system with variable refresh rates.

[0015] Figure 12 is a timing diagram illustrating example source and display frame timings in a system with variable refresh rates and a modulated spatial light modulator and / or color wheel. DETAILED DESCRIPTION

[0016] In systems that include both image processing and a display, such as a computer or an augmented reality headset, a host processor that renders image frames can render images at different rates based on, for example, the image complexity in an application such as a game. Sudden complex changes to the image frame require a slower rendering rate, while simple changes can be rendered quickly. When the content does not change, the host processor can delay the generation of new frames. As a result, the display receives image frames at varying rates. To accommodate the host processor rendering frames at different rates, the display controllers and systems described herein support variable refresh rate (VRR) while not exhibiting brightness fluctuations or undesirable artifacts.

[0017] The display controllers described herein can adaptively stretch or shrink each spatial light modulator (SLM) display frame period to match the duration of the current source frame period from the host processor. When the frame rate changes faster than possible sequence changes, the display controller can provide a sequence asynchronous to the incoming frame rate. No dark time is introduced on the SLM because the SLM display frame period is adaptively adjusted to match the current source frame period. As a result, there is no average brightness change per second due to source frame rate variations. No image tearing occurs because a complete image frame is presented in each frame before starting to display the next image frame.

[0018] The display controller can cascade a series of many dither frames to cover the duration of the source frame time, adaptively adding or removing dither frames as needed. For example, the display controller can only keep adding dither frames until the next frame arrives. Each full-color dither frame, although short in duration, displays the entire frame at full bit depth.

[0019] This can be achieved by dithering done by bitplane dithering. Each color segment within a dithered frame displays the entire frame of that color at full bit depth. Additionally, the dither segments performed within a color segment also display the entire frame of that color at full bit depth. Using bitplane dithering, the source frame rate is upconverted to the source frame rate multiplied by the number of dithered frames. For example, 60Hz source rate × 20 dithered frames = 1200Hz display frame rate. Since new source video frames can arrive at any random time, they arrive asynchronously with respect to the dithered frame currently being displayed. Thus, when the source frame rate varies frame by frame, the display frame rate is not an exact multiple of the input frame rate in any given frame. To achieve low latency, the display controller can use a scroll buffer to achieve <4 milliseconds latency. A double buffer can also be used. When a new source video frame is received, to prevent buffer swap artifacts and image tearing in the scroll buffer, the display of the current dither segment is completed while buffering the data from the new source video frame. The display controller can swap buffers at the end of a dither segment (stepping the buffer as a scroll buffer). Alternatively, the buffer swap can occur after the display of a single color segment or after the display of a full dithered frame. In some embodiments, the scroll buffer that captures the new source video data may not be large enough to allow the display of a full color segment or a full dithered frame.

[0020] Figure 1 is an example timing diagram showing the display of video frames at variable refresh rates. Figure 1 Shows video frames 102 and 104 and a synchronization signal 106. The synchronization signal 106 includes a pulse 108 that defines the start of the video frames 102 and 104. The video frames 102 and 104 include intervals corresponding to the pulse 108: a front porch interval, an active interval, and a back porch interval. The front porch interval defines the time between the end of the pulse 108 and the active interval, and the back porch interval defines the time between the end of the active interval and the start of the pulse 108. The active interval is the time during which image data is transmitted.

[0021] The back porch interval and the active interval have the same duration in the video frames 102 and 104. However, the front porch interval of video frame 104 is longer than the front porch interval of video frame 102. Thus, the refresh rate of the video can be changed by varying the duration of the front porch interval. A longer front porch interval results in a slower refresh rate, and a shorter front porch interval results in a faster refresh rate. In a system that includes a variable refresh rate, the last received video frame should continue to be displayed regardless of the duration of the front porch interval of successive video frames.

[0022] Figure 2is a block diagram of an example system 200 that implements video display with variable refresh rates. System 200 includes a display controller 202, a light source driver 204, a light source 206, and a spatial light modulator (SLM) 208. Some examples of system 200 may also include an optical actuator 210. The display controller 202 is coupled to a video source, such as an application processor 212. The application processor 212 may be implemented as any circuitry that provides a source video frame and control signals to the display controller 202.

[0023] The light source 206 generates light 218 that is provided to the SLM 208. The light source 206 may include, for example, a light-emitting diode (LED) or a laser diode that generates the light 218. In some examples, the light 218 generated by the light source 206 may include light of multiple colors (e.g., time-division multiplexed red, green, and blue light).

[0024] The light source driver 204 provides the voltage and current for operating the light source 206. The output of the light source driver 204 is coupled to the input of the light source 206, and the input of the light source driver 204 is coupled to the output of the display controller 202. The display controller 202 generates a light control signal 220 that is provided to the light source driver 204. The light control signal 220 may control the light source 206 to generate the light 218 as a color sequence and / or with varying brightness. The light source driver 204 may change the voltage and / or current of the light control signal 220 (e.g., increase the voltage and / or current) to generate a drive signal provided to the light source 206.

[0025] The SLM 208 is optically coupled to the light source 206 and electrically coupled to the display controller 202. The SLM 208 may be a digital micromirror device, a liquid crystal on silicon device, a micro light-emitting diode (micro-LED) device, or other spatial light modulation device. The SLM 208 receives the light 218 and manipulates the light 218 based on the SLM data and control signal 222 received from the display controller 202 to generate a light output 226. For example, the SLM 208 may include a pixel array (e.g., mirrors, shutters, etc.), and the SLM data and control signal 222 may define the pattern of the pixels of the SLM 208. The pixels of the SLM 208 may reflect or pass the light 218 based on the pattern of the SLM data and control signal 222 to display a video frame.

[0026] In an embodiment of system 200 that includes optical actuator 210, optical actuator 210 may be electrically coupled to display controller 202 and mechanically coupled to an optical plate that is optically coupled to SLM 208. Optical actuator 210 changes the physical position of the optical plate based on actuator control signal 224. The optical plate may be positioned between SLM 208 and light source 206, or may be positioned to receive light output 226 from SLM 208. For example, optical actuator 210 may change the physical position of the optical plate by a distance selected to adjust a portion of the pixels in one or more directions of the position of the display produced by SLM 208 to increase display resolution.

[0027] Display controller 202 includes buffer memory 214 and variable refresh rate circuitry 216. Buffer memory 214 stores video frames received from application processor 212. Buffer memory 214 may be partitioned into two sub-buffers to provide double buffering. In double buffering, a first sub-buffer stores incoming video frames, and a second sub-buffer provides a previously received video frame for controlling light source 206 and SLM 208. In other examples, buffer memory 214 may be further divided into more than two sub-buffers, where each sub-buffer stores a portion of an incoming video frame, and the video data stored in the sub-buffers may be provided for controlling light source 206 and SLM 208 during receipt of subsequent video data. An embodiment of buffer memory 214 that is further divided into more than two sub-buffers may operate as a rolling buffer.

[0028] In some embodiments of system 200, application processor 212 may provide source video frames to display controller 202 at a constant rate. In other embodiments of system 200, application processor 212 may provide source video frames to display controller 202 at a variable rate. For example, application processor 212 may provide a video frame to display controller 202 only when the displayed image has changed relative to a previously provided video frame. In such a system, the rate at which application processor 212 provides video frames to display controller 202 may vary, and variable refresh rate circuitry 216 may adaptively adjust the duration of video frame display to accommodate changes in the source frame rate. For example, variable refresh rate circuitry 216 may stretch or shrink the display frame duration to match the current source frame period of application processor 212. Variable refresh rate circuitry 216 may provide a display frame rate that is asynchronous with the source frame rate. Variable refresh rate circuitry 216 avoids dark times by adaptively adjusting the display frame period to match the current source frame period, which avoids changes in average brightness caused by variations in the source frame rate. Variable refresh rate circuitry 216 also avoids image tearing by completing the display of a frame before starting to display a subsequent frame.

[0029] The variable refresh rate circuit 216 can be implemented using the processor of the display controller 202, which executes instructions stored in the memory of the display controller 202 to provide the functions described herein. In some examples, the variable refresh rate circuit 216 can be implemented using hardware circuits that provide the functionality described herein. The variable refresh rate circuit 216 provides write data 215 to be stored in the buffer memory 214. The write data 215 can be source video received from the application processor 212. The variable refresh rate circuit 216 retrieves read data 217 from the buffer memory 214. The read data can be video data provided to the SLM 208 as part of the SLM data and control signals 222. Refer to Figures 4 to 12 The functionality of the variable refresh rate circuit 216 is further described.

[0030] Figure 3 is a block diagram of an example system 300 that implements a video with a variable refresh rate. System 300 is similar to system 200, but uses a color wheel to generate different light colors provided to the SLM 208. System 300 includes a display controller 302, a light source driver 304, a light source 306, an SLM 208, a motor driver 320, and a color wheel 322. The display controller 302 is similar to the display controller 202 and provides color wheel control. Some examples of system 300 may also include an optical actuator 210. The display controller 302 is coupled to the application processor 212.

[0031] The light source 306 generates light 318 that is provided to the color wheel 322. The light source 206 can include, for example, an LED or a laser diode that generates the light 318. In some examples, the light 318 generated by the light source 206 can include light of a single color (e.g., blue).

[0032] The light source driver 304 provides the voltage and current for operating the light source 306. The output of the light source driver 304 is coupled to the input of the light source 306, and the input of the light source driver 304 is coupled to the output of the display controller 302. The display controller 302 generates a light control signal 308 that is provided to the light source driver 304. The light control signal 308 can control the light source 306 to generate light 318 with varying brightness. The light source driver 304 can change the voltage and / or current of the light control signal 308 (e.g., increase the voltage and / or current) to generate a drive signal provided to the light source 306.

[0033] The color wheel 322 receives the light 318 and can change the color of the light 318 to provide multi - colored light 310 to the SLM 208. Multi - colored light refers to light whose color changes over time (e.g., red at a first time, blue at a second time, and green at a third time). In some embodiments, the color wheel 322 can be a phosphor wheel, which is a rotating wheel having at least some segments that convert the wavelength of the light 318. For example, the color wheel 322 can have a yellow phosphor segment that converts blue light to yellow light (red and green combine to form yellow) or a green phosphor segment that produces green light from blue light. The phosphor wheel can also optionally have segments that reflect or transmit blue light. The phosphor wheel can be used with a blue laser. An additional element that can be used is a rotating color filter, which can have, for example, red segments, green segments, and blue segments. The color filter can be used in combination with the phosphor color wheel or alone without the phosphor color wheel. The rotating color wheel can also be combined with a static phosphor, which can be a yellow phosphor or a white phosphor. The color wheel 322 includes a motor coupled to the color filter and / or the phosphor wheel. The motor rotates the color filter and / or the phosphor wheel. The light 318 passes through the color filter and / or the phosphor wheel to produce the multi - colored light 310.

[0034] The motor driver 320 provides the voltage and current for operating the color wheel 322 (e.g., for operating the motor of the color wheel 322). The output of the motor driver 320 is coupled to the input of the color wheel 322, and the input of the motor driver 320 is coupled to the output of the display controller 302. The display controller 302 generates a motor control signal 324 that is provided to the motor driver 320. The motor driver 320 can change the voltage and / or current of the motor control signal 324 (e.g., increase the voltage and / or current) to produce a drive signal that is provided to the color wheel 322.

[0035] The SLM 208 is optically coupled to the color wheel 322 and electrically coupled to the display controller 302. The SLM 208 can be a digital micromirror device, a liquid crystal on silicon device, or other spatial light modulation device. The SLM 208 receives the multi - colored light 310 and manipulates the multi - colored light 310 based on the SLM data and control signal 222 received from the display controller 302. For example, the SLM 208 can include a pixel array (e.g., mirrors, shutters, etc.), and the SLM data and control signal 222 can define the pattern of the pixels of the SLM 208. The pixels of the SLM 208 can reflect or pass the multi - colored light 310 based on the pattern of the SLM data and control signal 222 to display a video frame.

[0036] The buffer memory 214 and the variable refresh rate circuit 216 operate as described with respect to the display controller 202. Refer to Figures 4 to 12 The functionality of the variable refresh rate circuit 216 is further described.

[0037] Figure 4 is a timing diagram illustrating example sources and display frame timings in system 200 or system 300. Figure 4 Shows source video frames 402, 404, and 405 provided by application processor 212 and received by variable refresh rate circuit 216. In Figure 4 , signal DATEN can be a control signal provided by application processor 212 indicating the validity of source frame data. For example, DATAEN can have a first logic state indicating valid source frame data and a second logic state indicating invalid source frame data. Source video frame 402 has a period of 12.5 milliseconds (ms), and source video frame 404 has a period of 16.67 ms. Thus, the rate of source frames received from application processor 212 varies. Variable refresh rate circuit 216 begins receiving source video frame 404 at time 912 after activating DATEN during source video frame 404. During the reception of source video frame 404, variable refresh rate circuit 216 can provide a dithered segment of source video frame 402 (e.g., a dithered segment of color segment 426, which is an example of color segment 408), the dithered segment containing instructions to display the dithered segment.

[0038] Figure 4 Shows variable refresh rate circuit 216 providing a display frame as a dithered frame (e.g., dithered frame 406) to SLM 208. A dithered frame is a set (e.g., a minimal set) of color segments (e.g., red, green, and blue segments) representing a video frame at full bit depth. Figure 4 Shows multiple dithered frames 406. Different color segments are represented by different hatchings in dithered frame 406. For example, color segment 408 is shown with a unique hatching that can represent red. Color segment 418 is shown with a unique hatching that can represent green. Color segment 420 is shown with a unique hatching that can represent blue. In Figure 4 , each set of color segments 408, 418, and 420 (as represented by the provided hatchings) represents dithered frame 406. Each color segment shows a complete frame at full bit depth. Each color segment contains multiple dithered segments. For example, color segment 420 contains dithered segment 410, color segment 408 contains dithered segment 422, and color segment 418 contains dithered segment 424. A dithered segment is a part of a color segment of a color frame represented at full bit depth.

[0039] Variable refresh rate circuit 216 begins receiving source video frame 404 at time 412 after activating DATEN during source video frame 404. During the reception of source video frame 404, variable refresh rate circuit 216 can continue to provide the data of source video frame 402 to SLM 208 until a portion of source video frame 404 has been received, and thereafter provide the data of source video frame 404 to SLM 208.

[0040] In Figure 4 it, after starting to receive source video frame 404 at time 412 and after a selected portion of source video frame 404 has been received, the display of dither segment 422A of source video frame 402 is completed at time 414. The selected portion of source video frame 404 can be the entire source video frame 404, a color segment of source video frame 404, a dither frame of source video frame 404, a dither segment of source video frame 404, or other portions of source video frame 404. In response to receiving the selected portion of source video frame 404 and completing the display of dither segment 422A of source video frame 402, variable refresh rate circuit 216 provides dither segment 422B of source video frame 404 to SLM 208, which dither segment contains instructions for displaying a dither segment of a second video frame.

[0041] An example of variable refresh rate circuit 216 can also delay providing the data of source video frame 404 to SLM 208 until the display of the color segment of source video frame 402 is completed at time 916. After variable refresh rate circuit 216 starts providing the data of source video frame 404 to SLM 208, variable refresh rate circuit 216 continues to provide the data of source video frame 404 to SLM 208 until a subsequent source video frame 405 is received, and the operations described with respect to times 412, 414, and 416 are applied to transition from source video frame 404 to source video frame 405.

[0042] Figure 5 is a flowchart of a method 500 for displaying video with a variable refresh rate. Although depicted sequentially for convenience, at least some of the illustrated operations can be performed in a different order and / or in parallel. Additionally, some embodiments may perform only some of the illustrated operations. The operations of method 500 can be performed by system 200 or 300, and more specifically by display controller 202 or 302 and SLM 208.

[0043] In block 502, variable refresh rate circuit 216 receives a first video frame (e.g., source video frame 402) from application processor 212. As Figure 4 shown, source video frame 402 has a period of 12.5 milliseconds (ms).

[0044] In block 504, variable refresh rate circuit 216 generates first, second, and third color segments with dither segments from the first video frame, and provides the first, second, and third color segments with dither segments to SLM 208.

[0045] In block 506, SLM 208 displays the first video frame. Displaying the first video frame includes displaying the dither segments generated from the first video frame in block 504.

[0046] In block 508, when the SLM 208 is displaying the first video frame, the variable refresh rate circuit 216 receives a second video frame (e.g., the source video frame 404) from the application processor 212. Figure 4 The source video frame 404 is presented, having a period of 16.67 ms, provided by the application processor 212 and received by the variable refresh rate circuit 216.

[0047] In blocks 510 and 512, the display of the second video frame is delayed until a selected portion of the second video frame has been received, and thereafter the display of the dither segment of the first video frame is completed. In various embodiments of the method 500, the selected portion of the second video frame may be the entire second video frame (e.g., the entire source video frame 404), a color segment of the second video frame, a dither frame of the second video frame, a dither segment of the second video frame, or other portions of the second video frame.

[0048] In block 510, the variable refresh rate circuit 216 receives the second video frame. If the variable refresh rate circuit 216 has received the selected portion of the second video frame, then the method 500 continues in block 512. If the variable refresh rate circuit 216 has not received the selected portion of the second video frame, then the second video frame continues to be received until the selected portion has been received. Refer to Figure 7 and 9 for further explanation of the receipt of the second video frame.

[0049] In block 512, the SLM 208 is displaying the first video frame. If the display of the dither segment of the first video frame (e.g., Figure 4 the dither segment 422A shown in Figure 4 ) is completed, then the method 500 continues in block 514. In

[0050] the display of the dither segment 422A of the source video frame 402 is completed at time 414 after the receipt of the source video frame 404 begins at time 412 and the selected portion of the source video frame 404 has been received. If the display of the dither segment of the first video frame is not completed, then the display of the first video frame continues until the display of the dither segment is completed. Figure 4

[0051] ​In block 516, the SLM 208 displays the dithered segments of the second video frame, thereby initiating the display of the second video frame. After the variable refresh rate circuit 216 begins providing data for the second video frame to the SLM 208, the variable refresh rate circuit 216 continues to provide data for the second video frame to the SLM 208 until a third video frame is received and the operations of method 500 are performed to transition from the display of the second video frame to the display of the third video frame.

[0052] Figure 6 FIG. is a flow chart of a method 600 for displaying video at variable refresh rates. Although depicted sequentially for convenience, at least some of the illustrated operations may be performed in a different order and / or in parallel. Additionally, some embodiments may perform only some of the illustrated operations. The operations of method 600 may be performed by system 200 or 300, and more specifically by display controller 202 or 302 and SLM 208. Method 600 is similar to method 500, having transitions between source video frames at color segment boundaries rather than dither segment boundaries.

[0053] In block 602, the variable refresh rate circuit 216 receives a first video frame (e.g., source video frame 402) from the application processor 212. As Figure 4 shown, the source video frame 402 has a period of 12.5 milliseconds (ms).

[0054] In block 604, the variable refresh rate circuit 216 generates first, second, and third color segments with dithered segments from the first video frame and provides the first, second, and third color segments with dithered segments to the SLM 208.

[0055] In block 606, the SLM 208 displays the first video frame. Displaying the first video frame includes displaying the dithered segments generated from the first video frame in block 504.

[0056] In block 608, while the SLM 208 is displaying the first video frame, the variable refresh rate circuit 216 receives a second video frame (e.g., source video frame 404) from the application processor 212. Figure 4 Shown is the source video frame 404, which has a period of 16.67 ms, provided by the application processor 212 and received by the variable refresh rate circuit 216.

[0057] In blocks 610 and 612, the display of the second video frame is delayed until a selected portion of the second video frame has been received, and thereafter the display of the color segment of the first video frame is completed. In various embodiments of method 500, the selected portion of the second video frame can be the entire second video frame (e.g., the entire source video frame 404), a color segment of the second video frame, a dither frame of the second video frame, a dither segment of the second video frame, or other portions of the second video frame.

[0058] In block 610, the variable refresh rate circuit 216 receives the second video frame. If the variable refresh rate circuit 216 has received the selected portion of the second video frame, then method 600 continues in block 612. If the variable refresh rate circuit 216 has not received the selected portion of the second video frame, then the second video frame continues to be received until the selected portion has been received. Refer to Figure 7 and 9 for further explanation of the reception of the second video frame.

[0059] In block 612, the SLM 208 is displaying the first video frame. If the display of the color segment of the first video frame (e.g., Figure 4 the color segment 426 shown in Figure 4 ) is completed, then method 600 continues in block 614. In

[0060] Figure 4

[0061]

[0062] Figure 7

[0062] Figure 7FIG. 700 is a flow chart of a method for triggering the provision of a dither segment of a video frame in response to the storage of the video frame. Although depicted in sequence for convenience, at least some of the illustrated operations may be performed in a different order and / or in parallel. Additionally, some embodiments may perform only some of the illustrated operations. The operations of method 700 may be performed in conjunction with the operations of method 500 or 600, and may be performed by system 200 or 300, and more specifically by display controller 202 or 302. For example, variable refresh rate circuit 216 may receive a first video frame from application processor 212 in block 502 of method 500.

[0063] In block 702, variable refresh rate circuit 216 receives a second video frame from application processor 212. The operation of block 702 may be performed as part of the operation of block 508 of method 500 or block 608 of method 600. Variable refresh rate circuit 216 stores the second video frame in buffer memory 214. Buffer memory 214 may be further divided into two sub-buffers (double buffer), where each of the sub-buffers may store an entire video frame. Variable refresh rate circuit 216 may store the complete second video frame in one of the sub-buffers of buffer memory 214.

[0064] In block 704, variable refresh rate circuit 216 determines whether the storage of the second video frame or a selected portion thereof is complete. For example, a selected portion of the second video frame may be a color segment or a dither segment of the second video frame.

[0065] If the storage of the selected portion of the second video frame is completed in block 704, then in block 706 in response to completing the storage of the second video frame in the buffer memory, variable refresh rate circuit 216 provides the dither segment of the second video frame to SLM 208.

[0066] In block 708, SLM 208 displays the dither segment of the second video frame received in block 706. The operations of method 700 may be performed any number of times as part of method 500 or method 600.

[0067] Figure 8 FIG. is a timing diagram illustrating an example reception of a video frame stored in a buffer that is further divided into a plurality of sub-buffers to form a scrolling buffer. Figure 8 Illustrates the storage and display of source video frames 802 and 804. The signal VSYNC indicates the start of a new source video frame. Variable refresh rate circuit 216 may Figure 8The synchronization multiple of the source frame rate in [the relevant context] provides SLM display frames to control SLM 208. For example, if the application processor 212 provides source video frames 802 and 804 at a frame rate N, then the variable refresh rate circuit 216 can control SLM 208 by providing SLM display frames at a rate of M*N, where M is an integer. The buffer update signal illustrates an example of the timing for updating sub-buffers of the buffer memory 214. When the source video frame 802 is received, the sub-buffers of the buffer memory 214 are updated sequentially, with each sub-buffer storing a selected portion (e.g., a slice containing multiple rows of the source video frame). When each sub-buffer is updated, the stored data of the current source video frame increases, and the stored data of the previous source video frame decreases. In sequence 806, the stored data of the source video frame 802 increases with each buffer update until the entire source video frame 802 has been stored at buffer update 808. Similarly, in sequence 810, the stored data of the source video frame 804 increases and the stored data of the source video frame 802 decreases until the entire source video frame 804 has been stored at buffer update 812.

[0068] Figure 9 is a flowchart of a method 900 that triggers the provision of a dither segment of a video frame in response to the storage of a portion of the video frame. Although depicted sequentially for convenience, at least some of the illustrated operations may be performed in a different order and / or in parallel. Additionally, some embodiments may only perform some of the illustrated operations. The operations of method 900 may be performed in conjunction with the operations of method 500 or 600, and may be performed by system 200 or 300, and more specifically by display controller 202 or 302. For example, the variable refresh rate circuit 216 may receive a first video frame from the application processor 212 in block 502 of method 500.

[0069] In block 902, the variable refresh rate circuit 216 receives a second video frame from the application processor 212. The operation of block 902 may be performed as part of the operation of block 508 of method 500 or block 608 of method 600. The variable refresh rate circuit 216 stores the second video frame in the buffer memory 214. The buffer memory 214 may be further divided into multiple sub-buffers (rolling buffers), where each of the sub-buffers may store a sub-portion of the video frame (less than the entire video frame). When the variable refresh rate circuit 216 receives the second video frame, the variable refresh rate circuit 216 stores the second video frame in a sub-buffer of the buffer memory 214, as Figure 8 described therein.

[0070] In block 904, the variable refresh rate circuit 216 determines whether the storage of the second video frame or a selected portion thereof is complete. For example, the selected portion of the second video frame may be a color segment or a dither segment of the second video frame.

[0071] If the storage of the selected portion of the second video frame is completed in block 904, then in block 906 the variable refresh rate circuit 216 provides the dithered segments of the second video frame to the SLM 208.

[0072] In block 908, the SLM 208 displays the dithered segments of the second video frame received in block 906. The operations of method 900 can be performed any number of times as part of method 500 or method 600.

[0073] Figure 10 is a timing diagram illustrating an example display frame rate adjustment based on a variable refresh rate source video. Figure 10 Illustrates source video frames and SLM display frames. The application processor 212 provides source video frames at a variable rate, frame 1002 having a period of 6.95 ms and all other frames having a period of 8.33 ms. During interval 1004, the variable refresh rate circuit 216 controls the SLM 208 to provide SLM display frames asynchronous to the source video frames. The variable refresh rate circuit 216 can relatively slowly adjust the timing of the SLM display frames (e.g., small adjustments per source video frame) within an adjustment interval (e.g., 1 second) to synchronize the SLM display frames with the source video frames. For example, the variable refresh rate circuit 216 can perform a background task that continuously adjusts the clock fall frame by frame to slowly adjust the display frame rate to an exact multiple of the source frame rate. In one example, a ~1.0 second time constant (programmable time constant in some examples) is used to provide the slow adjustment. If the source frame rate is stable, then after approximately 1.0 second, the display frame rate will be an exact multiple of the source frame rate and the display frame rate will become fully synchronized with the source frame rate. For cases where the display frame rate has been set to a fixed rate while still supporting variable refresh rate (e.g., for using the optical actuator 210 and / or color wheel 322), the per-frame clock fall adjustment can be disabled.

[0074] During interval 1008 (at the end of the adjustment interval), the variable refresh rate circuit 216 has adjusted the rate of the SLM display frames to be synchronized with the source video frames.

[0075] Figure 10 The dithered frame 1006 shown in represents one of an integer number of dithered frames displayed for the source video frames during interval 1008. A dithered frame is a set (e.g., a minimal set) of color segments (e.g., red, green, and blue segments) that represent a video frame at full bit depth. Different color segments are represented by different hatchings in the dithered frame 1006. For example, color segment 1010 is shown with the unique hatching that can represent red. Color segment 1012 is shown with the unique hatching that can represent green. Color segment 1014 is shown with the unique hatching that can represent blue. InFigure 10 In this case, each set of color segments 1010, 1012, and 1014 (as represented by the provided hatching) represents the dither frame 1006. Each color segment displays the complete frame at full bit depth. Each color segment contains multiple dither segments. For example, color segment 1010 contains dither segments. A dither segment is a part of a color segment of a color frame represented at full bit depth.

[0076] Figure 11 FIG. 1100 is a flowchart of a method 1100 for providing a display frame rate in a system with a variable refresh rate. Although depicted in sequence for convenience, at least some of the illustrated operations may be performed in a different order and / or in parallel. Additionally, some embodiments may perform only some of the illustrated operations. The operations of method 1100 may be performed by system 200 or 300, and more specifically by display controller 202 or 302. Method 1100 may be applied in combination with methods 500, 600, 700, and / or 900.

[0077] In block 1102, the variable refresh rate circuit 216 receives a first video frame (e.g., source video frame 402) from the application processor 212.

[0078] In block 1104, the variable refresh rate circuit 216 receives a second video frame (e.g., source video frame 404) from the application processor 212. The period of the second video frame may be different from the period of the first video frame. The period is the duration of the video frame. Figure 4 FIG. 13 shows that source video frame 402 has a period of 12.5 ms and source video frame 404 has a period of 16.67 ms.

[0079] Some examples of method 1100 may adjust the display frame rate based on the source frame rate or provide a display frame rate different from the source frame rate. In block 1106, the variable refresh rate circuit 216 provides video data for display at a display frame rate different from the frame rates of the first and second video frames. For example, the display frame rate provided by the variable refresh rate circuit 216 may be asynchronous with the source frame rate. In examples of system 200 or system 300 employing the optical actuator 210 and / or the color wheel 322, the variable refresh rate circuit 216 may provide a fixed display frame rate asynchronous with the source frame rate. In such a system, the display frame rate may be synchronized with the optical actuator 210 and / or the color wheel 322. For example, the vibration rate of the optical actuator 210 may be fixed, and / or the rotation rate of the color wheel 322 may be fixed. The display frame rate may be fixed and synchronized with the fixed vibration rate of the optical actuator 210 and / or the fixed rotation rate of the color wheel 322. The application processor 212 may provide the source video frames at a rate different from the display frame rate. For example, the application processor 212 may provide the source video frames at a variable rate or at a rate different from the display frame rate.

[0080] In block 1108, the variable refresh rate circuit 216 adjusts the display frame rate based on the periods of the first and second video frames. For example, the variable refresh rate circuit 216 can adjust the display frame rate so that the display frame rate is synchronized with the source frame rate. During a selected time interval (e.g., 1 second), the variable refresh rate circuit 216 can continuously adjust the timing of display frame generation to produce a display frame rate that is an integer multiple of the source frame rate, as shown Figure 10 therein.

[0081] Because the variable refresh rate circuit 216 allows the source video frame rate to vary, the variable refresh rate circuit 216 works with a wide range of display system configurations. For a system using the color wheel 322, with or without the optical actuator 210, if the source frame rate is fixed, the color wheel 322 can be locked to a multiple of the source frame rate, and the display frame rate can be locked to the color wheel 322. For example, if the source frame rate is 60 Hz, the CW can run at 120 Hz and the display frame rate can be 120 Hz. If the source frame rate is stable, the color wheel 322 can be locked to the source frame rate, meaning that it meets these conditions: 1) at power-up, for 10 frames, the source frame rate is within + / -.2 Hz; and 2) during runtime, for more than 300 frames, the source frame rate remains within + / -.2 Hz. For both conditions 1) and 2), the frame rate tolerance and the number of frames can be programmable.

[0082] If the source frame rate is varying, the color wheel 322 speed is set to a fixed frequency so that it runs asynchronously with the source frame rate. This is to avoid attempting to adjust the color wheel 322 speed to track the source frame rate. The display frame rate is locked to the color wheel 322. The color wheel 322 speed can be set to the system's nominal rated value, such as 120 Hz, 240 Hz, etc. The scroll buffer timing in the frame can be the same as in a system that does not use the color wheel 322. When the scroll buffer starts stepping to display new data after a new source video frame arrives, the color at the first buffer swap is random. If the optical actuator 210 is being used, the sub-frame at which the first buffer swap occurs is also random.

[0083] For systems that do not use the color wheel 322, with or without the optical actuator 210, if the optical actuator 210 is not used, or if the optical actuator 210 can operate at the source frame rate, then the display frame rate can be locked to the source frame rate. In systems that use the optical actuator 210, if the source frame rate changes (for >N frames, not within + / -.2 Hz) and is not within the range supported by the optical actuator 210, then the display frame rate can be fixed to a rate asynchronous to the source frame rate. The scroll buffer timing is the same with or without the optical actuator 210. When the scroll buffer starts stepping to display newly arrived source video frame data, the color at the buffer swap is random. If the optical actuator 210 is being used, then the sub-frame at which the first buffer swap occurs is also random. This allows the source frame rate to be higher than the frame rate that the optical actuator 210 can keep up with. For example, the source frame rate can be 144 Hz, but the actuator and display frame rates can be set to 60 Hz (240 Hz sub-frame rate).

[0084] Figure 12 is a timing diagram illustrating example source and display frame timings in an example of systems 200 or 300 that use the optical actuator 210 and / or the color wheel 322 with variable refresh rates. The optical actuator 210 can be mechanically coupled to an optical plate that is optically coupled to the SLM 208 and electrically coupled to the display controller 202. The optical actuator 210 changes the physical position of the optical plate based on the actuator control signal 224. For example, the optical actuator 210 can change the physical position of the optical plate by a distance that is chosen to adjust a portion of a pixel in one or more directions of the position of the display to be produced by the SLM 208 to increase the display resolution. The display controller 202 provides display data for each physical position of the optical plate to provide an increased display resolution (e.g., 4 times the resolution of the SLM 208). The optical actuator 210 and / or the color wheel 322 can operate at a fixed rate that affects the display rate provided by the display controller 202. The variable refresh rate circuit 216 allows the display controller 202 to receive source video frames from the application processor 212 at the source video frame rate and provide display data to the SLM 208 at a display frame rate different from the source video frame rate.

[0085] Figure 12 Illustrates source video frames and SLM display frames. The source video frames provided by the application processor 212 include source video frames 1202, 1204, 1206, and 1208 each having a different period. At Figure 12In [the context], the signal DATEN can be a control signal provided by the application processor 212 indicating the validity of the source frame data. For example, DATAEN can have a first logic state indicating the validity of the source frame data and a second logic state indicating the invalidity of the source frame data. The variable refresh rate circuit 216 controls the SLM 208, the optical actuator 210, and the color wheel 322 to provide SLM display frames at a fixed display frame rate that is not derived from the rate of the source video frames (independent of the rate of the source video frames). In Figure 12 it, the SLM display frames have a constant period of 8.33 ms, independent of the period of the source data frames. Sub-frames (e.g., dither frames) of the SLM display frames are also shown.

[0086] As described with reference to methods 400 and 500, the variable refresh rate circuit 216 initiates the display of the source video frame after receiving a selected portion of the source video frame and after completing the display of the dither segment of the current SLM display frame. When initiating the display of the source video frame, the data being displayed can be a combination of data from a previous source video frame and data from the current source video frame, as described with respect to Figure 8 it. In Figure 12 it, the display of the source video frame can start from the completion of the display of the color segment. In Figure 12 it, the display of the source video frame 1202 can start at time 1210. The display of the source video frame 1204 can start at time 1212. The display of the source video frame 1206 can start at time 1214. The display of the source video frame 1208 can start at time 1216.

[0087] At times 1210, 1212, 1214, and 1216, while buffering data from the current source video frame, the display of the dither segment of the previous source video frame is completed. If a rolling buffer is used, the buffer can be restarted by providing the data stored in the first buffer segment for display. The initiation of the display of the current source video frame can occur with a random color and a random sub-frame. In some examples, dark times can be included in the sub-frames.

[0088] In this specification, the term "coupled" can encompass connections, communications, or signal paths that enable a functional relationship consistent with this specification. For example, if device A generates a signal to control device B to perform an action, then: (a) in a first example, device A is coupled to device B by a direct connection; or (b) in a second example, device A is coupled to device B through an intermediate component C, provided that the intermediate component C does not change the functional relationship between device A and device B such that device B is controlled by device A via the control signal generated by device A.

[0089] Additionally, in this specification, the recitation "based on" means "at least partially based on". Thus, if X is based on Y, then X may depend on Y and any number of other factors.

[0090] A device "configured to" perform a task or function may be configured (e.g., programmed and / or hardwired) by a manufacturer at the time of manufacture to perform the function, and / or may be configured (or reconfigured) by a user after manufacture to perform the function and / or other additional or alternative functions. The configuration may be performed by firmware and / or software programming of the device, by the construction and / or layout of the hardware components and interconnections of the device, or a combination thereof.

[0091] As used herein, the terms "terminal", "node", "interconnection", "pin", and "lead" may be used interchangeably. Unless specifically stated to the contrary, these terms are generally used to mean the interconnection between device elements, circuit elements, integrated circuits, devices, or other electronic devices or semiconductor components, or their ends.

[0092] A circuit or device described herein as including certain components may in fact be adapted to be coupled to those components for forming the described circuitry or device. For example, a structure described as including one or more semiconductor elements (e.g., transistors), one or more passive elements (e.g., resistors, capacitors, and / or inductors), and / or one or more sources (e.g., voltage and / or current sources) may in fact include only semiconductor elements within a single physical device (e.g., a semiconductor die and / or an integrated circuit (IC) package), and may be adapted to be coupled to at least some of the passive elements and / or sources to form the described structure, for example, by a final user and / or a third party, at the time of manufacture or after manufacture.

[0093] The circuits described herein are reconfigurable to include additional or different components to provide functionality that is at least partially similar to the functionality available prior to the component replacement.

[0094] Although some elements of the described examples are included within an integrated circuit and other elements are external to the integrated circuit, in other examples, additional or fewer features may be incorporated into the integrated circuit. Additionally, some or all of the features described as external to the integrated circuit may be included within the integrated circuit, and / or some of the features described as internal to the integrated circuit may be incorporated external to the integrated circuit. As used herein, the term "integrated circuit" means one or more circuits that: (i) are incorporated in / above a semiconductor substrate; (ii) are incorporated in a single semiconductor package; (iii) are incorporated into the same module; and / or (iv) are incorporated in / on the same printed circuit board.

[0095] In this specification, unless stated otherwise, "about", "substantially", or "essentially" before a parameter means within + / - 10% of the stated parameter, or if the parameter is zero, within a reasonable value about zero.

[0096] Within the scope of the claims, modifications are possible in the described examples, and other examples are possible.

Claims

1. A display controller, comprising: A variable refresh rate circuit configured to: Receive a first video frame; Provide a first dither segment of the first video frame, the first dither segment including an instruction to display the first dither segment during reception of a second video frame; and In response to receiving the second video frame and completing display of the first dither segment, provide a second dither segment of the second video frame, the second dither segment including an instruction to display the second dither segment.

2. The display controller according to claim 1, wherein the display controller includes a buffer configured to store the second video frame, and the variable refresh rate circuit is configured to provide the second dither segment in response to completing storage of the second video frame in the buffer.

3. The display controller according to claim 1, wherein the display controller includes a buffer subdivided into a plurality of sub-buffers configured to store the second video frame, and the variable refresh rate circuit is configured to provide the second dither segment in response to completing storage of a portion of the second video frame in one of the sub-buffers.

4. The display controller according to claim 1, wherein the variable refresh rate circuit is configured to adjust a display frame rate based on a time between receipt of the first video frame and receipt of the second video frame.

5. The display controller according to claim 1, wherein: The display controller is configured to provide the first video frame as a first color segment, a second color segment, and a third color segment; and The variable refresh rate circuit is configured to provide the color segments until receipt of the second video frame.

6. The display controller according to claim 5, wherein: The color segments include a plurality of dither segments; and The variable refresh rate circuit is configured to provide the second dither segment of the second video frame in response to completion of display of one of the color segments or completion of display of the first video frame.

7. The display controller according to claim 1, wherein the variable refresh rate circuit is configured to provide a display frame rate different from source frame rates of the first video frame and the second video frame.

8. A method, comprising: Receiving, by a display controller, a first video frame; Providing, by the display controller, the first video frame as a first color segment, a second color segment, and a third color segment to a spatial light modulator (SLM), wherein the color segments include a plurality of dither segments; Receiving, by the display controller, a second video frame while the first video frame is being displayed by the SLM; and In response to receiving the second video frame and completing display of the dither segments of the first video frame, providing, by the display controller, dither segments of the second video frame to the SLM.

9. The method according to claim 8, further comprising: Storing the second video frame in a buffer memory of the display controller; And In response to completing storage of the second video frame in the buffer memory, providing, by the display controller, the dither segments of the second video frame to the SLM.

10. The method according to claim 8, further comprising: Storing the second video frame in a buffer memory of the display controller, the buffer memory being further divided into a plurality of sub-buffers; And In response to completing storing a portion of the second video frame in one of the sub-buffers, providing, by the display controller, the dithered segment of the second video frame.

11. The method according to claim 8, further comprising adjusting, by the display controller, a display frame rate based on a time between receiving the first video frame and receiving the second video frame.

12. The method according to claim 8, further comprising providing the color segment of the first video frame until the second video frame is received.

13. The method according to claim 8, further comprising displaying the dithered segment of the second video frame in response to completing display of one of the color segments of the first video frame.

14. The method according to claim 8, further comprising providing, by the display controller, a display frame rate different from source frame rates of the first video frame and the second video frame.

15. A system, comprising: A spatial light modulator (SLM); and A display controller, comprising: A video input, An SLM control output coupled to an input of the SLM, and A variable refresh rate circuit configured to: Receive a first video frame via the video input; Indicate, via the SLM control output, that the SLM displays a first dithered segment of the first video frame during reception of a second video frame; and In response to receiving the second video frame and completing display of the first dithered segment, indicate, via the SLM control output, that the SLM displays a second dithered segment of the second video frame.

16. The system according to claim 15, wherein: The display controller includes a buffer configured to store the second video frame; and The variable refresh rate circuit is configured to indicate that the SLM displays the second dithered segment in response to completing storing the second video frame in the buffer.

17. The system according to claim 15, wherein: The display controller includes a buffer further divided into a plurality of sub-buffers configured to store the second video frame; and The variable refresh rate circuit is configured to indicate that the SLM displays the second dithered segment in response to completing storing a portion of the second video frame in one of the sub-buffers.

18. The system according to claim 15, wherein the variable refresh rate circuit is configured to adjust a display frame rate based on a time between receiving the first video frame and receiving the second video frame, or provide a display frame rate different from source frame rates of the first video frame and the second video frame.

19. The system according to claim 15, wherein: The display controller is configured to indicate that the SLM displays the first video frame as a first color segment, a second color segment, and a third color segment; and The variable refresh rate circuit is configured to instruct the SLM to display the color segment until the second video frame is received.

20. The system according to claim 19, wherein: the color segment includes a plurality of dither segments; and the variable refresh rate circuit is configured to instruct the SLM to display the second dither segment of the second video frame in response to completion of display of one of the color segments or completion of display of the first video frame.

Images