Low-latency variable backlight LCD display system

Abstract

In various examples, a low-latency variable-backlight liquid crystal display (LCD) system is disclosed. The LCD system can reduce latency and video lag by using a rendering device to perform an analysis of peak pixel values ​​within a subset of pixels before transmitting a frame to a display device for display. As a result, the display device can receive peak pixel value data before or simultaneously with the frame data and can begin updating the backlight settings of the display without having to wait for a large portion of the frame to be received. In this way, the LCD system can avoid the full-frame latency of conventional systems, allowing the LCD system to more reliably support high-performance applications, such as gaming.

Description

Background Art

[0001] Liquid crystal displays (LCDs), such as high dynamic range (HDR) LCDs, typically use a backlight comprised of a matrix of light emitting diodes (LEDs). To account for differences in brightness across portions of the display, some LEDs can be illuminated more brightly while others can be illuminated more dimly—commonly referred to as local dimming. As a result, higher contrast levels can be achieved and HDR capabilities can be implemented, as opposed to fixed-backlit display devices where a consistent illumination value is applied to each backlight. To determine the illumination values ​​for the various LEDs, a current can be determined for each LED based on the current frame or image to be displayed, the contribution of each LED to each pixel can then be determined to calculate the illumination field for that pixel, and the pixel value for the pixel can be set based on a combination of the associated illumination field and the desired pixel value (e.g., red (R), green (G), and blue (B) values ​​for an RGB display).

[0002] However, conventional algorithms for making backlighting decisions for the current frame must either introduce delays when waiting to receive the frame on the LCD device side before analyzing it (in addition to the delay in performing the analysis), or must predict new pixel values ​​from a previous frame—e.g., leading to potentially non-optimal decisions for dynamic frame content, such as where pixel values ​​change rapidly between consecutive frames. As such, conventional algorithms result in display lag (e.g., the time interval from the first line of a rendered frame transmitted at the rendering device to the first line of the frame visible on the display device) due to delays in receiving and analyzing the current frame data and / or image artifacts caused by reliance on under- or over-bright frames of previous frames in determining associated pixel values. The lag and / or image artifacts introduced from these conventional approaches degrade the overall performance of the system and impact the user experience—particularly in high-performance applications, such as competitive online or cloud gaming, where user response time has a greater impact on user experience and performance. Summary of the Invention

[0003] Embodiments of the present disclosure relate to a low-latency variable-backlight liquid crystal display (LCD) system. Disclosed are systems and methods for performing frame analysis on a rendering device, rather than listening to incoming frames on the LCD device. For example, the rendering device can analyze a rendered frame to determine peak pixel or pixel cell values ​​within a subset of the frame's pixels and transmit the peak determination to the variable-backlight LCD device to more quickly process lighting decisions—e.g., with minimal latency. Thus, and in contrast to conventional systems that wait for most or all of a frame to be received in the LCD device's frame buffer before performing frame analysis, the present systems and methods perform frame analysis on the rendering device—e.g., after rendering, when the frame has been fully buffered—before or simultaneously with transmitting the rendered frame to the LCD device. The LCD device can receive analyzed frame data before receiving the entire frame, enabling lighting decisions and lighting model determination to be performed at the LCD device with minimal latency or delay. For example, when applying a pixel cell value to a pixel of an LCD display, updates to the pixel cell value may already have been determined based on the cumulative lighting contribution of surrounding LEDs, allowing the pixel to be read out to the LCD without waiting for calculated lighting contribution information. Thus, instead of introducing nearly a full frame delay (e.g., at 80% of the frame at a 72 Hz frame refresh rate, with a common line frequency of 160.8 kHz, at 4K resolution, the delay may be approximately 10.7 milliseconds), the only delay may be the transmission delay associated with transmitting the analyzed frame data and the frame data (e.g., when a single line of data containing lighting information from the analyzed frame data is transmitted before or simultaneously with the full frame data, the delay may be approximately 6.7 microseconds), thereby increasing system performance and user experience—particularly for high-performance applications such as gaming. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The present system and method for a low-latency variable-backlight liquid crystal display (LCD) system are described in detail below with reference to the accompanying drawings, wherein:

[0005] Figure 1 is a block diagram of a low-latency variable-backlight LCD system according to some embodiments of the present disclosure;

[0006] Figure 2A shows the look-ahead delay and display lag of a conventional system according to some embodiments of the present disclosure;

[0007] Figure 2B Some embodiments of the present disclosure are shown Figure 1 Low-latency variable backlight LCD system look-ahead delay and display lag;

[0008] Figure 3 shows an example display device and corresponding pixels, backlight, and backlight pixel blocks according to some embodiments of the present disclosure;

[0009] Figure 4 is a flow chart illustrating a method for determining illumination settings and pixel values ​​for a low-latency variable backlight display according to some embodiments of the present disclosure;

[0010] Figure 5 is a block diagram of an example content streaming system suitable for implementing some embodiments of the present disclosure;

[0011] Figure 6 is a block diagram of an example computing device suitable for implementing some embodiments of the present disclosure; and

[0012] Figure 7 is a block diagram of an example data center suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0013] Disclosed are systems and methods relating to low-latency variable backlight liquid crystal display (LCD) systems. Although primarily described with respect to backlit LCDs, this is not intended to be limiting. For example, embodiments of the present disclosure may be implemented using any backlit display, such as a single-layer LCD, a dual-layer LCD, a multi-layer LCD, a macro-area LCD, and / or other display types that include any number of layers using backlighting. Where an LCD is used, the LCD panel may be of any type, such as twisted nematic (TN), in-plane switching (IPS), vertical alignment (VA), advanced fringe field switching (AFFS), and / or other types. Furthermore, although primarily white light emitting diode (LED) backlights are described herein, this is not intended to be limiting, and any type of LED backlight, such as red (R), green (G), blue (B) (RGB) LED backlights, may be used without departing from the scope of the present disclosure. In some embodiments, other types of backlights, such as but not limited to cold cathode fluorescent lamp (CCFL) backlights, may be used in addition to or in place of LED backlights. The backlight described herein can correspond to any backlight arrangement, such as edge-lit LEDs (ELEDs), direct-lit LEDs (DLEDs), local dimming LEDs (e.g., DLED clusters), full-array local dimming (FALD) (e.g., directly controlling individual DLEDs), and / or other arrangements.

[0014] refer to Figure 1 , Figure 1is an example low latency variable backlight LCD system 100 (alternatively referred to herein as "system 100") according to some embodiments of the present disclosure. It should be understood that this and other arrangements described herein are set forth by way of example only. Other arrangements and elements (e.g., machines, interfaces, functions, orders, functional groupings, etc.) may be used in addition to or in place of the arrangements and elements shown, and some elements may be omitted entirely. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in combination with other components, and in any suitable combination and location. The various functions described herein as being performed by an entity may be performed by hardware, firmware, and / or software. For example, the various functions may be implemented by a processor executing instructions stored in a memory. In some embodiments, system 100 may include Figure 5 An exemplary content streaming system 500 of Figure 6 The exemplary computing device 600 and / or Figure 7 The system 100 may include similar components, features, and / or functionality to the exemplary data center 700 of the system 100. For example, the system 100 may be implemented locally—e.g., where frames are rendered using a laptop computer, a desktop computer coupled to a display, a tablet computer, a virtual reality, augmented reality, or mixed reality system, etc.—or may be implemented in a cloud-based environment—e.g., cloud gaming, virtual teleconferencing, streaming, and / or other cloud-based environments—where frames may be rendered remotely, e.g., using one or more servers, and streamed to a local client device.

[0015] System 100 may include one or more computing devices 102 and / or one or more display devices 104. Figure 1 104 may be separate components of system 100, such as a system including a streaming device and a display, a cloud computing device (e.g., a computer system including a streaming device and a display), a cloud computing device (e.g., a computer system including a streaming device and a display), a cloud computing device (e.g., a computer system including a streaming device and a display), a virtual, augmented, or mixed reality headset, a smart TV, a smart phone, etc. In such an example, computing device(s) 102 (alternatively referred to herein as "rendering device 102") may include components that render frames of data prior to transmission to display device 104 for display, such as one or more graphics processing units (GPUs) 106. In some embodiments, computing device 102 and display device 104 may include separate devices or components of system 100, such as a system including a streaming device and a display, a cloud computing device (e.g., similar to a system including a streaming device and a display), a cloud computing device (e.g., a computer ... Figure 5 Example content streaming system 500 and / or Figure 7Examples include a data center 700) and a local display (e.g., using an intermediary device such as a computer, streaming device, game console, etc.), a desktop computer and a coupled display, a game console and a coupled display, another type of computing device 102 (which may or may not include an integral display) and a coupled display (e.g., a second display), and / or the like.

[0016] Thus, in some embodiments, system 100 may correspond to a single device (e.g., an LCD television), or a local device (e.g., a desktop computer, a laptop computer, a tablet computer, etc.), and components of system 100 may be executed locally by system 100. In the case where computing device(s) 102 include local devices (e.g., a game console, an optical disc player, a smartphone, a computer, a tablet computer, etc.), image data may be transmitted over a network (e.g., a LAN) via a wired and / or wireless connection (e.g., via a display port, HDMI, etc.). For example, computing device(s) 102 may render an image (which may include reconstructing an image from encoded image data), store the rendered image in frame buffer 110, scan out the rendered image according to a scan order (e.g., using a video controller) to generate display data, and transmit the display data to display device 104, e.g., via display interface 114, for display.

[0017] In other embodiments, some or all of the components of system 100 may exist separately from display device 104 (e.g., an LCD display device). For example, computing device(s) 102—including GPU(s) 106, 3D engine 108, frame buffer 110, display interface 114A, frame analyzer 112A, and / or lighting decision maker 118 (in Figure 1104 ). The system 100 may be a component of another system separate from the display device 104 . For example, the system 100 may be a component or node of a distributed computing system (such as a cloud-based system) for streaming images, videos, video game instances, and the like. In such embodiments, the system 100 may communicate with one or more computing devices 102 (e.g., a server) over a network (e.g., a wide area network (WAN), a local area network (LAN), or a combination thereof, via wired and / or wireless communication protocols). For example, the computing device(s) 102 may generate and / or render images, encode images, and transmit the encoded image data over the network to another computing device 102 (e.g., a streaming device, a television, a computer, a smartphone, a tablet computer, etc.). The receiving device (which may include another computing device 102 and / or the display device 104 itself) may decode the encoded image data, reconstruct the image (e.g., assign a color value to each pixel), store the reconstructed image data in a frame buffer 116, scan the reconstructed image data from the frame buffer 110 according to a scan order (e.g., using a video controller) to generate display data, and then transmit the display data for display by the display device 104 (e.g., an LCD) of the system 100. Where the image data is encoded, the encoding may correspond to a video compression technique such as, but not limited to, H.264, H.265, M-JPEG, MPEG-4, or the like.

[0018] Thus, whether the process of generating a rendered image for storage in the frame buffer 116 occurs internally (e.g., within the display device 104 (e.g., a television)), locally (e.g., via a locally connected computing device 114), remotely (e.g., via one or more servers in a cloud-based system), or a combination thereof, image data representing the value (e.g., color value, etc.) of each pixel of the display is scanned out from the frame buffer 116 (or other memory device) to generate display data (e.g., representing voltage values, capacitance values, etc.) configured for use by the display device, for example, in digital and / or analog format. Furthermore, the display device 104 (e.g., the LCD layer of the display panel 126) can be configured to receive the display data according to a scan order for proper refreshing.

[0019] The computing device 102 may include a GPU 106 and / or other processor types, such as one or more central processing units (CPUs), one or more data processing units (DPUs), etc., configured to render image data representing still images, video images, and / or other image types. Once rendered or otherwise suitable for display by the display device 104 of the system 100, the image data may be stored in a memory, such as a frame buffer 110. In some embodiments, the image data may represent a sub-image for each panel of the display device 104, for example, in embodiments where the LCD includes two or more panels. Thus, a single image may be divided into two or more separate images to correspond to the number of rows and / or columns of pixels included in each display panel 126.

[0020] The GPU 106 and / or other processing unit types may execute a 3D engine 108, a frame buffer 110, a display interface 114A, and / or a frame analyzer 112A. The 3D engine 108 may include a rendering engine for rendering 3D graphics. Frames (e.g., images, videos, etc.) may be generated using the 3D engine 108, and once rendered, the frames may be stored in the frame buffer 110. In conventional systems, once a frame is stored in the frame buffer 110 of the computing device 102, it may be sent to the display device 104 via the display interface 114A of the computing device 102 and the display interface 114B of the display device 104. The display interface 114 may correspond to a DisplayPort, a mini DisplayPort, HDMI, mini HDMI, micro HDMI, VGA, mini VGA, DVI-D, DVI-I, mini DVI, micro DVI, USB, and / or another display interface type. Once received at the display device 104 and stored in the frame buffer 116, analysis of the frame for making lighting decisions and controlling the backlight of the display device 104 can be performed. However, a noticeable lag or delay may be introduced when waiting to perform analysis on the frame until at least a portion of the frame is stored in the frame buffer 116 on the display side. In addition to the lag or delay from the frame analysis, the lighting decision may require buffering more frames in the frame buffer 116, and the lighting model may further require even more frames—e.g., 80% to 100% of the frames. Thus, these conventional systems may require up to a full frame delay before making a lighting decision for setting the backlight and adjusting pixel values ​​to account for the backlight setting.

[0021] As an example, where the frame resolution is reduced to the backlight (e.g., LED) matrix resolution, pixel subsets may be assigned to corresponding backlights (e.g., backlight or LED pixel blocks). While conceptually pixel subsets may be assigned to corresponding backlights, the illumination contribution for any single pixel may come from the corresponding backlight as well as any number (e.g., all) of the other backlights. In an embodiment, the frame resolution may be divided by the backlight resolution (or a multiple thereof) to determine the size of the backlight pixel block (or pixel subset). In an example where the frame resolution is 3840×2160 and the backlight resolution is 24×16, each backlight pixel block may include a resolution of 160 (e.g., 3840 / 24)×135 (e.g., 2160 / 16). As a non-limiting visual example, and for Figure 3 , where the frame resolution is 16×8 and the LED resolution is 4×4, each backlight pixel block may include a resolution of 4×2. Thus, backlight pixel block 304A may correspond to backlight 306A, backlight pixel block 304B may correspond to backlight 306B, and so on. Therefore, in order to perform frame analysis according to conventional methods, image data corresponding to the top one to one and a half backlight pixel blocks (e.g., using a top-down scan order) may have to be buffered in the frame buffer 116. To determine the lighting decision, an additional three backlight pixel blocks may have to be buffered in the frame buffer 116, and to determine the lighting model, an additional ten backlight pixel blocks (e.g., equal to 14 or 14.5 backlight pixel blocks). Therefore, in the example of a 3840×2160 frame resolution and a 24×16 backlight resolution, 14 to 14.5 of the 16 rows of backlight pixel blocks may need to be stored in the frame buffer 116 before the frame is actually displayed. This lag or delay in conventional systems is Figure 2A , where frame A is rendered using GPU 106A, transmitted to display device 104A, analyzed using frame analysis, analyzed for lighting decisions, analyzed for lighting models, and then displayed. Figure 2A As shown in , the display lag includes almost a full frame of delay.

[0022] To address these shortcomings of conventional systems, the system 100 may include a frame analyzer 112A on the computing device 102 to utilize the entire frame stored in the frame buffer 110. Thus, rather than snooping through the incoming frame data in the display-side frame buffer 116, the frame analysis may be performed at the computing device 102, and data representing the output of the frame analysis (e.g., peak pixel values) may be transmitted before or concurrently with the frame or image data across the display interface 114. In this example, the peak pixel value information may be used (e.g., immediately upon receipt) by the lighting decider 118 and lighting modeler 120 of the display device 104 to update pixel values ​​and set lighting settings for the backlight 128 of the display device 104. In this example, and as Figure 2B As shown in , the only delays may be transmission delays and / or any additional processing delays of the lighting decision maker 118 and / or the lighting modeler 120. During the experiments, an example with a frame resolution of 3840×2160, a backlight resolution of 24×16, a frame refresh rate of 72 Hz and a common line frequency of 160.8 kHz was used. Figure 2A The look-ahead delay of system 100 may be approximately 6.2 microseconds (e.g., the delay from transmitting data (in an embodiment, a single line) representing the analyzed lighting information (peak pixel values) corresponding to a full frame) compared to approximately 10.7 milliseconds in an example conventional implementation (e.g., 0.8 (80% of the frame) × 2160 rows of pixels × 6.2 microseconds per row).

[0023] Thus, within system 100, once a frame is buffered in frame buffer 110, frame analyzer 112A may analyze the frame to determine one or more peak pixel values. For example, frame analyzer 112A may determine one or more peak pixel values ​​for each backlight pixel block (e.g., for each 160×135 pixel block using a 3840×2160 frame resolution and a 24×16 backlight resolution). In the case where display panel 126 includes an RGB display panel (such as an LCD RGB panel), the peak pixel values ​​may include a peak red cell value (e.g., corresponding to a cell including an LC valve and an embedded red filter), a peak green cell value, and / or a peak blue cell value. In some embodiments, the peak values ​​may correspond to actual peak values ​​of one or more of the cells of the backlight pixel block, or may correspond to filtered or analyzed peak values. For example, where filtered peak pixel values ​​are used, outlier pixel values ​​(e.g., which may indicate noise) may be filtered out to prevent an outlier or single high-value pixel from negatively impacting other pixels of a block—e.g., where most of the other pixels have lower pixel values ​​(e.g., closer to 0 on a scale of 0-255) that would require less illumination from the backlight. To filter out outliers, in an embodiment, one or more histograms (e.g., a red cell histogram, a green cell histogram, and / or a blue cell histogram) may be generated for each backlight pixel block to determine the filtered peak red, green, and / or blue cell values ​​for the backlight pixel block. In other embodiments, a single histogram may be generated that includes all pixel values—e.g., including red, green, blue, and / or other color cell values. In any example, the histogram may be used to determine a filtered peak value that may be determined to be associated with the corresponding backlight pixel block. The selected filtered peak(s) may not correspond to the bin(s) with the most data, but rather may be used to select the bin(s) with the highest value for a threshold number (e.g., 3 or more entries) or percentage (e.g., 1% or more) of data. In some examples, the peak(s) selected from the histogram may correspond to a peak that still allows a threshold number or percentage (e.g., greater than 90%) of other pixels in the same backlit pixel block to reach their correct pixel value. For example, where the actual peak pixel value may not allow other pixels of lower value (e.g., closer to 0 on a scale of 0-255) to reach their value (e.g., due to excessive illumination), the histogram may be used to determine a peak pixel value that will still allow at least a threshold number or percentage of other pixels of lower value (and / or other pixels of higher value) to achieve their associated pixel value. Thus, a single high-value outlier pixel may not be relied upon as the peak pixel value for the backlit pixel block.

[0024] In some embodiments, temporal and / or spatial smoothing can be used to determine the peak value(s) for a given backlight pixel block. For example, one or more previous frames of data can be used to determine the peak value(s) - for example, where one or more previous frames include low peak pixel values ​​for a given backlight pixel block and the current frame includes a very high peak pixel value, the current peak pixel value can be weighted by the low peak pixel value(s) of the one or more previous frames. Similarly, where a peak pixel value is determined for a given pixel cell, one or more adjacent pixel cells can be analyzed to determine whether there is a sharp contrast in the pixel values. In this example, where there is a sharp contrast in adjacent pixel values ​​(e.g., a peak pixel value cell is adjacent to a pixel cell having a much lower value), the peak pixel value can be weighted to reduce the value in view of the adjacent pixel cells.

[0025] The output or determination of the frame analyzer 112A - for example, one or more peak pixel values ​​corresponding to each backlight pixel block - can then be transmitted to the display device 104 via the display interface 114. This data can be transmitted before and / or concurrently with the image data being stored in the frame buffer 110, and the data can correspond to an additional line of image or video data. In some embodiments, this peak data can include a plurality of values ​​corresponding to the backlight resolution multiplied by the number of peak values ​​for each backlight pixel block. As such, in the 24×16 backlight resolution example, where peak red, green, and blue cell values ​​are determined for each frame, the peak pixel data can include 24×16×3 (or 1152) values. As another example, where a single peak pixel value is determined for each backlight pixel block, the peak pixel data can include 24×16 (or 384) values.

[0026] In some embodiments, GPU 106 and / or other processing unit types can be used to perform frame analysis in parallel with existing post-processing steps (e.g., tone mapping) such that frame analysis introduces no or minimal additional latency (other than the transmission latency of transmitting peak pixel value data) compared to existing systems. For example, one or more parallel processing units—e.g., threads of GPU(s) 106—can be used to perform existing post-processing techniques and frame analysis in parallel.

[0027] Frame analyzer 112B is shown on display device 104 to capture implementations where display device 104 and / or computing device 102 are not configured for computing device-side frame analysis. As such, in these embodiments, frame analysis may occur on display device 104.

[0028] Once the peak data is received, the lighting decision maker 118 has all the information necessary to begin processing lighting decisions for each backlight 128. For example, the lighting decision maker 118 can use the peak pixel values ​​of the backlight pixel blocks to determine the lighting value for each backlight 128, and thereby determine the current value to drive to each backlight 128. For example, the lighting decision maker 118 can look at one or more backlights (e.g., LEDs) that are near the backlight pixel block currently being evaluated. In such an example, the backlights evaluated for any given backlight pixel block can include the backlight that is most closely positioned relative to the center pixel of the backlight pixel block, and in embodiments, one or more neighboring backlights (e.g., backlights within five or fewer rows and / or five or fewer columns of backlights closest to the center pixel). Although five or fewer rows and / or five or fewer columns are described, this is for example purposes only and is not intended to be limiting. In some examples, any number (e.g., three, four, five, six, etc.) of backlights can be factored into the evaluation. The determination of the number of rows and / or columns of backlights to consider can be based on the shape of the light distribution of a single backlight (e.g., the more concentrated the shape of the light distribution is to the area covered by a single backlight, the less amount of surrounding backlights need to be considered). Similarly, the selection of the number of columns and / or rows can be based on a tradeoff or balance between latency, computation (e.g., the wider the range, the more computation is required), and / or over-illumination (e.g., too small a range may result in over-illumination). Thus, lighting decisions for the closest backlights can allow for consideration of the cumulative light from one or more backlights that have the greatest impact on the illumination field of the backlight pixel block. In considering additional backlights in addition to the backlight that is closest to the center pixel of the backlight pixel block, these additional backlights can be used to add additional illumination to achieve higher pixel values ​​(e.g., values ​​closer to 255 on a scale of 0-255). Additionally, by considering only the backlights closest to the backlight pixel block, processing requirements can be reduced, thereby reducing the runtime of the system 100 without sacrificing too much accuracy in lighting decisions (e.g., because closely positioned backlights can contribute the majority of light to the backlight pixel block).

[0029] As an example, and with reference to Figure 3 To determine the lighting decision for backlight 306 associated with backlight pixel block 304A, backlight 306A and one or more adjacent backlights 306C, 306D, and / or 306E may be taken into account to achieve the appropriate lighting level for the pixels of backlight pixel block 304A. This process may be repeated for each backlight pixel block and / or each backlight until a final lighting decision is determined for a particular frame. The current value for the backlight may then be set for backlight illumination 128, and the associated current may be driven to backlight 306 when the frame data is read out of the frame buffer and processed by valve controller 122.

[0030] See again Figure 1, the illumination modeler 120 can use the illumination decisions for the backlights 128 to drive current values ​​to the backlight illumination 128 and, in conjunction with the valve controller 122, help update pixel values. For example, once the illumination values ​​or settings are determined for each backlight, the cumulative illumination field for each pixel cell or valve (e.g., in an RGB LCD display, a pixel may have a red cell, a green cell, and / or a blue cell) may be known. Using the cumulative illumination value for each pixel cell (which may take into account the illumination contributions from any number (e.g., all) of the backlights 128 of the display device 104), updates to the pixel values ​​buffered in the frame buffer 116 may be made for one or more pixel cells to account for the cumulative illumination at that cell. For example, at a particular pixel (e.g., Figure 3 255 scale, and the backlights 306A, 306B, 306D, 306C, and 306E (and / or other backlights contributing to the cumulative illumination field at pixel 302A) are dimly illuminated, the pixel value can be updated to a pixel value of 255 to account for the lower brightness achievable using the current backlight illumination settings. In such an example, if the original pixel value is to be maintained, a voltage can be applied to the cell(s) of pixel 302A corresponding to the value 240, and due to the dim illumination field, the resulting pixel value can more closely represent the value 225. Thus, by increasing the pixel value—and therefore the voltage applied to the cell(s) of pixel 302A—the resulting pixel value displayed on display device 104 can be closer to the original value 240. A similar process can occur to reduce the brightness of the pixel cell when the cumulative illumination field is very bright. For example, the voltage applied to the pixel cell may be reduced to account for the additional accumulated light at the pixel cell so that the original pixel value may be more accurately represented during display.

[0031] In addition to the illumination settings for each backlight 128, updated pixel values ​​as determined using the illumination modeler 120 and valve controller 122 can be used to display a frame on the display panel 126. For example, the valve controller 122 can read out pixel values ​​from a frame buffer according to a scan order (e.g., top to bottom, center to outside, etc.), can update one or more of the pixel values ​​based on the illumination model, and can transmit the updated pixel values ​​(e.g., can cause voltages to be driven to the pixel cells based on the updated pixel values) to the display panel 126 via the panel interface 124. At substantially the same time (e.g., when a row of pixels is scanned out for display, the associated backlights can be illuminated at the same time), the illumination modeler 120 can drive current values ​​to the backlights 128 to illuminate the backlights according to their respective illumination settings.

[0032] The display panel 126 may include any number of layers, which may include any number of units (or valves), which may correspond to pixels or sub-pixels of pixels, respectively. For example, the layers may include red, green, and blue (RGB) layers, each of which may correspond to a sub-pixel having an associated color (e.g., red, green, or blue) associated therewith via one or more color filter layers of the system 100. Thus, a first unit may correspond to a first sub-pixel having a red color filter connected in series therewith, a second unit may correspond to a second sub-pixel having a blue color filter connected in series therewith, and so on. Although RGB layers are described herein, this is not intended to be limiting, and any different individual colors or combinations of colors may be used according to the embodiments. For example, in some embodiments, the layers may include a monochrome or grayscale (Y) layer, which may correspond to some grayscale range of colors from black to white. As such, the units of the Y layer may be adjusted to correspond to colors on the grayscale color spectrum.

[0033] In the case where the display device 104 corresponds to an LCD, once an update value (e.g., color value, voltage value, capacitance value, etc.) is determined for each cell of each layer of the display panel 126 - for example, using the frame buffer 116, the video controller, the valve controller 122, the panel interface 124, etc. - a signal corresponding to the update value can be applied to each cell via a row driver and a column driver controlled according to a shift register and a clock. For example, for a given cell, the row driver corresponding to the row of cells can be activated according to the shift register (e.g., activated to a value of 1 via a corresponding flip-flop), and the column driver corresponding to the column of cells can be activated to drive a signal (e.g., carrying a voltage) to the transistor / capacitor pair of the cell. As a result, the capacitor of the cell can be charged to a capacitance value corresponding to the color value of the current frame of image data. This process can be repeated for each cell of each layer of the LCD according to a scan order (e.g., from top left to bottom right, center outward, etc.).

[0034] As a non-limiting example of an embodiment of the system 100, the GPU(s) 106 of the computing device(s) 102 (which may include a cloud-based device, a local device, a device integrated with the display device 104, and / or another device type) may query the display device 104 (e.g., via HDMI, DisplayPort, and / or another display interface type) to detect or determine whether the monitor has low-latency HDR capabilities (e.g., whether the monitor is configured to receive peak pixel value data from the computing device(s) 102). For example, the computing device 102 may perform an Extended Display Identification Data (EDID) discovery, and the EDID field may include data indicating that the display device 102 is configured for low-latency HDR mode. If the display device 104 is configured for low-latency HDR according to the systems and methods described herein, the GPU(s) 106 may receive a preferred or associated frame analysis geometry, which may include a backlight resolution, a multiple thereof, and / or another frame analysis geometry. Upon request by the GPU 106, the display device 104 may enter low-latency HDR mode. After rendering a frame and storing it in the frame buffer 110, the GPU can perform an analysis pass on the entire frame using known frame analysis geometry to generate reduced geometry frame analysis data. As an example, this can include determining the maximum component values ​​of the R, G, and B values ​​for each backlight pixel block individually, and / or can include performing some filtering, generating histograms, and / or other analysis to determine the peak pixel values. For example, where the frame resolution is 3840×2160 and the backlight resolution is 24×16, the frame analysis process can include determining the peak pixel value data for each 160×135 rectangle of pixels (e.g., each backlight pixel block). The GPU 106 can perform other operations on the full frame data—such as tone mapping, in which floating-point data is converted to integer data—and the frame analysis operations can be performed in parallel with the tone mapping and / or other post-processing operations, thereby not consuming additional memory bandwidth for the GPU system and not incurring additional latency beyond the amount of time required to transfer one additional line of video or image data corresponding to the peak pixel value. As such, the GPU(s) 106 may transmit peak pixel values ​​(e.g., 1152 analysis values—24×16×3 values) before (e.g., during vertical blanking) or simultaneously with sending frame data to the display device 104. The display device 104 may receive the peak pixel value data, and because the display device 104 is in low-latency HDR mode, the display device 104 may strip the incoming peak pixel value data from the data packet directly to the lighting decider 118—e.g., bypassing the frame analysis stage at the display side.

[0035] Now refer to Figure 4, each block of the method 400 described herein comprises a computing process that can be performed using any combination of hardware, firmware, and / or software. For example, different functions can be implemented by a processor executing instructions stored in a memory. The method 400 can also be embodied as computer-usable instructions stored on a computer storage medium. The method 400 can be provided by a standalone application, a service, or a hosted service (standalone or in combination with other hosted services), or a plug-in to another product, to name a few. Furthermore, by way of example, with respect to Figure 1 The method 400 is described with reference to the system 100 of FIG. However, the method 400 may additionally or alternatively be performed by any one system or any combination of systems, including but not limited to those described herein.

[0036] Figure 4 4 is a flow chart illustrating a method 400 for determining illumination settings and pixel values ​​for a low-latency variable backlight display according to some embodiments of the present disclosure. At block B402, method 400 includes determining one or more peak pixel values ​​corresponding to a subset of pixels of an image. For example, a frame analyzer of computing device 102 may analyze frame data corresponding to the image to determine one or more peak pixel values ​​corresponding to each backlight pixel block.

[0037] At block B404, method 400 includes sending image data representing pixel values ​​corresponding to at least the subset of pixels and data representing the one or more peak pixel values. For example, computing device 102 (e.g., via display interface 114A) may send the pixel value data before or simultaneously with the image or frame data.

[0038] At block B406, method 400 includes determining lighting settings for one or more backlights based at least in part on the one or more peak pixel values. For example, lighting decision maker 118 may analyze the peak pixel values ​​to determine lighting settings for one or more backlights 128. In some non-limiting embodiments, the one or more backlights may include a backlight that is positioned closest to a center pixel of a backlight pixel block and / or one or more adjacent backlights 128 (e.g., backlights in four or fewer rows of backlights and / or four or fewer columns of backlights).

[0039] At block B408, method 400 includes updating one or more of the pixel values ​​corresponding to the subset of pixels based at least in part on the illumination setting to generate updated pixel values. For example, valve controller 122 may use the illumination model from illumination modeler 120 to update one or more pixel values ​​of the backlight pixel block—e.g., to drive different voltages and / or capacitances—to account for the accumulated illumination field of the pixel cell or valve. The updated pixel values ​​may include original pixel values, updated pixel values, or a combination thereof, depending on the illumination field associated with a given frame.

[0040] At block B410, method 400 includes displaying an image based at least in part on the illumination settings for one or more backlights and the updated pixel values ​​for the subset of pixels. For example, illumination modeler 120 may cause currents to be driven to one or more backlights 128, the currents corresponding to the illumination settings for the one or more backlights, and valve controller 122, via panel interface 124, may cause voltages to be driven to cells of display panel 126, the voltages corresponding to the updated pixel values.

[0041] Example content streaming system

[0042] Now refer to Figure 5 , Figure 5 is an example system diagram for a content streaming system 500 according to some embodiments of the present disclosure. Figure 5 Includes (one or more) application servers 502 (which may include Figure 6 ), client device(s) 504 (which may include components, features, and / or functionality similar to the example computing device 600 of Figure 6 ) and network(s) 506 (which may be similar to the network(s) described herein). In some embodiments of the present disclosure, system 500 may be implemented. Application sessions may correspond to game streaming applications (e.g., NVIDIA GeFORCE NOW), remote desktop applications, simulation applications (e.g., autonomous or semi-autonomous vehicle simulation), computer-aided design (CAD) applications, virtual reality (VR) and / or augmented reality (AR) streaming applications, deep learning applications, and / or other application types.

[0043] In system 500, for an application session, client device(s) 504 may receive input data only in response to input to input device(s), transmit the input data to application server(s) 502, receive encoded display data from application server(s) 502, and display the display data on display 524. Thus, more computationally intensive calculations and processing are offloaded to application server(s) 502 (e.g., rendering (specifically ray or path tracing) for graphics output of the application session performed by the GPU(s) of the game server(s) 502). In other words, the application session is streamed from application server(s) 502 to client device(s) 504, thereby reducing the graphics processing and rendering requirements of client device(s) 504.

[0044] For example, with respect to instantiation of an application session, client device 504 can display a frame of the application session on display 524 based on receiving display data from application server(s) 502. Client device 504 can receive input from one of the input device(s) and generate input data in response. Client device 504 can transmit the input data to application server 502 via communication interface 520 and via network 506 (e.g., the Internet), and application server 502 can receive the input data via communication interface 518. The CPU can receive the input data, process the input data, and transmit the data to the GPU, which causes the GPU to generate a rendering of the application session. For example, the input data can represent movement of a user's character in a game session of a gaming application, firing a weapon, reloading, passing a ball, turning a vehicle, etc. Rendering component 512 can render the application session (e.g., representing the results of the input data), and rendering capture component 514 can capture the rendering of the application session as display data (e.g., as image data capturing a frame of the rendering of the application session). Rendering of the application session may include ray or path-traced lighting and / or shading effects computed using one or more parallel processing units (such as GPUs), which may further utilize one or more specialized hardware accelerators or processing cores to execute the ray or path tracing techniques of one or more application servers 502. In some embodiments, one or more virtual machines (VMs)—e.g., including one or more virtual components such as vGPUs, vCPUs, etc.—may be used by application servers 502 to support the application session. Encoder 516 may then encode the display data to generate encoded display data, and the encoded display data may be transmitted to client device 504 via communication interface 518 over network 506. Client device 504 may receive the encoded display data via communication interface 520, and decoder 522 may decode the encoded display data to generate display data. Client device 504 may then display the display data via display 524.

[0045] Example computing device

[0046] Figure 6FIG6 is a block diagram of an example computing device 600 suitable for implementing some embodiments of the present disclosure. Computing device 600 may include an interconnect system 602 that directly or indirectly couples the following devices: memory 604, one or more central processing units (CPUs) 606, one or more graphics processing units (GPUs) 608, a communication interface 610, input / output (I / O) ports 612, I / O components 614, a power supply 616, one or more presentation components 618 (e.g., a display), and one or more logic units 620. In at least one embodiment, computing device 600 may include one or more virtual machines (VMs), and / or any of its components may include virtual components (e.g., virtual hardware components). For non-limiting examples, one or more GPUs 608 may include one or more vGPUs, one or more CPUs 606 may include one or more vCPUs, and / or one or more logic units 620 may include one or more virtual logic units. Thus, computing device 600 may include discrete components (eg, a complete GPU dedicated to computing device 600 ), virtual components (eg, a portion of a GPU dedicated to computing device 600 ), or a combination thereof.

[0047] although Figure 6 The various blocks of are shown as being connected via an interconnect system 602 having wires, but this is not intended to be limiting and is provided for clarity only. For example, in some embodiments, a presentation component 618 such as a display device may be considered an I / O component 614 (e.g., if the display is a touch screen). As another example, the CPU 606 and / or the GPU 608 may include memory (e.g., the memory 604 may represent a storage device in addition to the memory of the GPU 608, the CPU 606, and / or the other components). In other words, Figure 6 The term computing device is illustrative only. No distinction is made between categories such as "workstation," "server," "laptop," "desktop," "tablet," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types, as all are considered within the Figure 6 within the range of computing devices.

[0048] Interconnect system 602 can represent one or more links or buses, such as an address bus, a data bus, a control bus, or a combination thereof. Interconnect system 602 can include one or more links or bus types, such as an industry standard architecture (ISA) bus, an extended industry standard architecture (EISA) bus, a video electronics standard association (VESA) bus, a peripheral component interconnect (PCI) bus, a peripheral component interconnect express (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 606 can be directly connected to memory 604. In addition, CPU 606 can be directly connected to GPU 608. In the case where there is a direct or point-to-point connection between components, interconnect system 602 can include a PCIe link to perform the connection. In these examples, it is not necessary to include a PCI bus in computing device 600.

[0049] Memory 604 may include any of a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 600. Computer-readable media can include volatile and non-volatile media and removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0050] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 604 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computing device 600. As used herein, computer storage media does not include signals themselves.

[0051] Computer storage media may embody computer-readable instructions, data structures, program modules, and / or other data types in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information delivery media. The term "modulated data signal" may refer to a signal that has one or more of its characteristics set or changed in such a manner as to encode information into the signal. By way of example and not limitation, computer storage media may include wired media such as a wired network or a direct wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0052] The CPU 606 can be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 600 to perform one or more of the methods and / or processes described herein. Each of the CPUs 606 can include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a large number of software threads simultaneously. The CPU 606 can include any type of processor and can include different types of processors, depending on the type of computing device 600 implemented (e.g., a processor with fewer cores for mobile devices and a processor with more cores for servers). For example, depending on the type of computing device 600, the processor can be an Advanced RISC (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). The computing device 600 can also include one or more CPUs 606 in addition to one or more microprocessors or supplementary coprocessors such as math coprocessors.

[0053] In addition to or in place of the CPU 606, the GPU 608 may also be configured to execute at least some computer-readable instructions to control one or more components of the computing device 600 to perform one or more of the methods and / or processes described herein. One or more GPUs 608 may be integrated GPUs (e.g., with one or more CPUs 606) and / or one or more GPUs 608 may be discrete GPUs. In embodiments, one or more GPUs 608 may be coprocessors for one or more CPUs 606. The computing device 600 may use the GPU 608 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, the GPU 608 may be used for general-purpose computing on a GPU (GPGPU). The GPU 608 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. The GPU 608 may generate pixel data for outputting an image in response to a rendering command (e.g., a rendering command received from the CPU 606 via a host interface). The GPU 608 may include graphics memory, such as display memory, for storing pixel data or any other suitable data (e.g., GPGPU data). Display memory can be included as part of memory 604. GPU 608 can include two or more GPUs operating in parallel (e.g., via a link). The link can connect the GPUs directly (e.g., using NVLINK) or through a switch (e.g., using NVSwitch). When combined, each GPU 608 can generate pixel data or GPGPU data for different portions or different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU can include its own memory or can share memory with other GPUs.

[0054] In addition to or in lieu of the CPU 606 and / or GPU 608, the logic unit 620 may be configured to execute at least some computer-readable instructions to control one or more components of the computing device 600 to perform one or more methods and / or processes described herein. In embodiments, the CPU 606, GPU 608, and / or logic unit 620 may perform any combination of methods, processes, and / or portions thereof, either separately or in conjunction. The one or more logic units 620 may be part of and / or integrated within the one or more CPUs 606 and / or the one or more GPUs 608, and / or the one or more logic units 620 may be discrete components of or otherwise external to the CPU 606 and / or GPU 608. In embodiments, the one or more logic units 620 may be processors of the one or more CPUs 606 and / or the one or more GPUs 608.

[0055] Examples of logic unit 620 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), a tensor core (TC), a tensor processing unit (TPU), a pixel vision core (PVC), a vision processing unit (VPU), a graphics processing cluster (GPC), a texture processing cluster (TPC), a streaming multiprocessor (SM), a tree traversal unit (TTU), an artificial intelligence accelerator (AIA), a deep learning accelerator (DLA), an arithmetic logic unit (ALU), an application-specific integrated circuit (ASIC), a floating point unit (FPU), an input / output (I / O) element, a peripheral component interconnect (PCI) or a peripheral component interconnect express (PCIe) element, etc.

[0056] The communication interface 610 may include one or more receivers, transmitters, and / or transceivers that enable the computing device 600 to communicate with other computing devices via an electronic communication network, including wired and / or wireless communications. The communication interface 610 may include components and functionality that enable communication over any of a number of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., via Ethernet or InfiniBand communications), low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, the one or more logic units 620 and / or the communication interface 610 may include one or more data processing units (DPUs) for transmitting data received over the network and / or through the interconnect system 602 directly to the one or more GPUs 608 (e.g., their memories).

[0057] The I / O ports 612 can enable the computing device 600 to be logically coupled to other devices including I / O components 614, presentation components 618, and / or other components, some of which can be built into (e.g., integrated into) the computing device 600. Illustrative I / O components 614 include a microphone, a mouse, a keyboard, a joystick, a game pad, a game controller, a satellite dish, a browser, a printer, a wireless device, and the like. The I / O components 614 can provide a natural user interface (NUI) that processes user-generated air gestures, voice, or other physiological input. In some instances, the input can be transmitted to an appropriate network element for further processing. The NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, gesture recognition on and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with the display of the computing device 600 (as described in more detail below). The computing device 600 can include a depth camera such as a stereo camera system, an infrared camera system, an RGB camera system, touch screen technology, and combinations thereof for gesture detection and recognition. Additionally, computing device 600 may include an accelerometer or gyroscope to enable motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by computing device 600 to render immersive augmented or virtual reality.

[0058] The power supply 616 may include a hardwired power supply, a battery power supply, or a combination thereof. The power supply 616 may provide power to the computing device 600 to enable the components of the computing device 600 to operate.

[0059] The presentation component 618 may include a display (e.g., a monitor, a touch screen, a television screen, a head-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The presentation component 618 may receive data from other components (e.g., the GPU 608, the CPU 606, the DPU, etc.) and output the data (e.g., as images, video, sound, etc.).

[0060] Sample Data Center

[0061] Figure 7 An example data center 700 is shown, which may be used in at least one embodiment of the present disclosure. The data center 700 may include a data center infrastructure layer 710, a framework layer 720, a software layer 730, and an application layer 740.

[0062] like Figure 7As shown, the data center infrastructure layer 710 may include a resource coordinator 712, grouped computing resources 714, and node computing resources ("node CRs") 716(1)-716(N), where "N" represents any complete positive integer. In at least one embodiment, the node CRs 716(1)-716(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power modules and cooling modules, etc. In some embodiments, one or more of the node CRs 716(1)-716(N) may correspond to a server having one or more of the above-mentioned computing resources. Furthermore, in some embodiments, the node CRs 716(1)-716(N) may include one or more virtual components, such as vGPUs, vCPUs, etc., and / or one or more of the node CRs 716(1)-716(N) may correspond to a virtual machine (VM).

[0063] In at least one embodiment, the grouped computing resources 714 may include separate groups of node CRs 716 housed in one or more racks (not shown), or many racks (also not shown) housed in data centers at various geographic locations. The separate groups of node CRs 716 within the grouped computing resources 714 may include computing, network, memory, or storage resources that can be configured or allocated to support groupings of one or more workloads. In at least one embodiment, several node CRs 716 including CPUs, GPUs, DPUs, and / or other processors may be grouped in one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0064] The resource coordinator 712 may configure or otherwise control one or more node CRs 716(1)-716(N) and / or grouped computing resources 714. In at least one embodiment, the resource coordinator 712 may comprise a software design infrastructure (SDI) management entity for the data center 700. The resource coordinator 712 may comprise hardware, software, or some combination thereof.

[0065] In at least one embodiment, Figure 7As shown, the framework layer 720 may include a job scheduler 750, a configuration manager 734, a resource manager 736, and a distributed file system 738. The framework layer 720 may include a framework that supports the software 732 of the software layer 730 and / or one or more applications 742 of the application layer 740. The software 732 or the application 742 may include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. The framework layer 720 may be, but is not limited to, a free and open source software network application framework, such as Apache Spark, which can utilize the distributed file system 738 for large-scale data processing (e.g., "big data"). TM (hereinafter referred to as "Spark"). In at least one embodiment, the job scheduler 750 may include a Spark driver to facilitate scheduling of workloads supported by the various layers of the data center 700. In at least one embodiment, the configuration manager 734 may be capable of configuring the different layers, such as the software layer 730 and the framework layer 720 including Spark and a distributed file system 738 for supporting large-scale data processing. The resource manager 736 may be capable of managing the mapping or allocation of clustered or grouped computing resources to support the distributed file system 738 and the job scheduler 750. In at least one embodiment, the clustered or grouped computing resources may include the grouped computing resources 714 at the data center infrastructure layer 710. The resource manager 736 may coordinate with the resource coordinator 712 to manage these mapped or allocated computing resources.

[0066] In at least one embodiment, the software 732 included in the software layer 730 may include software used by at least a portion of the node CRs 716(1)-716(N), the grouped computing resources 714, and / or the distributed file system 738 of the framework layer 720. The one or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0067] In at least one embodiment, the one or more applications 742 included in the application layer 740 may include one or more types of applications used by at least a portion of the node CRs 716(1)-716(N), the grouped computing resources 714, and / or the distributed file system 738 of the framework layer 720. The one or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0068] In at least one embodiment, any of the configuration manager 734, resource manager 736, and resource coordinator 712 can implement any number and type of self-modification actions based on any number and type of data acquired in any technically feasible manner. The self-modification actions can relieve the data center operator of the data center 700 from making potentially poor configuration decisions and can avoid underutilized and / or poorly performing portions of the data center.

[0069] The data center 700 may include tools, services, software, or other resources for training one or more machine learning models or using one or more machine learning models to predict or infer information according to one or more embodiments described herein. For example, a machine learning model may be trained by calculating weight parameters according to a neural network architecture using the software and computing resources described above with respect to the data center 700. In at least one embodiment, using the weight parameters calculated by one or more training techniques, the resources described above with respect to the data center 700 may be used to infer or predict information using a trained machine learning model corresponding to one or more neural networks, such as but not limited to those described herein.

[0070] In at least one embodiment, the data center 700 may use a CPU, an application-specific integrated circuit (ASIC), a GPU, an FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or reasoning using the aforementioned resources. In addition, one or more of the aforementioned software and / or hardware resources may be configured as a service to allow users to train or perform information reasoning, such as image recognition, speech recognition, or other artificial intelligence services.

[0071] Sample network environment

[0072] A network environment suitable for implementing embodiments of the present disclosure may include one or more client devices, servers, network attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may be configured to: Figure 6 The backend device 700 may be implemented on one or more instances of the computing device 600 of the embodiment of the present invention—for example, each device may include similar components, features and / or functions of the computing device 600. In addition, in the case of implementing a backend device (e.g., a server, NAS, etc.), the backend device may be included as part of the data center 700, examples of which are described herein with respect to FIG. Figure 7 Describe in more detail.

[0073] The components of the network environment can communicate with each other through the network, which can be wired, wireless, or both. The network can include multiple networks, or a network of networks. For example, the network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (e.g., the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. In the case where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connections.

[0074] Compatible network environments may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment), and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein with respect to the server may be implemented on any number of client devices.

[0075] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, and the like. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework for supporting software at the software layer and / or one or more applications at the application layer. The software or application may include network-based service software or application programs, respectively. In an embodiment, one or more client devices may use network-based service software or application programs (e.g., by accessing the service software and / or application programs via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a type of free and open source software network application framework that may, for example, use a distributed file system for large-scale data processing (e.g., "big data").

[0076] A cloud-based network environment can provide cloud computing and / or cloud storage that performs any combination of the computing and / or data storage functions described herein (or one or more portions thereof). Any of these various functions can be distributed across multiple locations from a central or core server (e.g., one or more data centers that can be distributed across a state, region, country, global, etc.). If the connection to the user (e.g., client device) is relatively close to an edge server, the core server can assign at least a portion of the functionality to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0077] Client devices may include Figure 6 The client device 600 may be embodied as a personal computer (PC), a laptop computer, a mobile device, a smartphone, a tablet computer, a smartwatch, a wearable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality head-mounted display, a global positioning system (GPS) or device, a video player, a camera, a surveillance device or system, a vehicle, a watercraft, an aircraft, a virtual machine, a drone, a robot, a handheld communication device, a hospital device, a gaming device or system, an entertainment system, an in-vehicle computer system, an embedded system controller, a remote control, an appliance, a consumer electronic device, a workstation, an edge device, any combination of these described devices, or any other suitable device.

[0078] The present disclosure can be described in the general context of machine-usable instructions or computer code executed by a computer or other machine such as a personal digital assistant or other handheld device, including computer-executable instructions such as program modules. Generally, program modules including routines, programs, objects, components, data structures, etc. refer to code that performs a specific task or implements a specific abstract data type. The present disclosure can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The present disclosure can also be practiced in a distributed computing environment where tasks are performed by remote processing devices linked through a communication network.

[0079] As used herein, the statement "and / or" with respect to two or more elements should be interpreted as referring to only one element or combination of elements. For example, "element A, element B and / or element C" may include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or elements A, B and C. In addition, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0080] The subject matter of the present disclosure is described in detail herein to meet statutory requirements. However, the description itself is not intended to limit the scope of the present disclosure. On the contrary, the inventors have contemplated that the claimed subject matter may also be embodied in other ways to include steps that are different from the steps described herein in conjunction with other current or future technologies, or combinations of similar steps. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of the method employed, these terms should not be interpreted as implying any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. A system comprising: Computing equipment for: receiving, from a display device, first data representing a frame analysis geometry associated with the display device; determining one or more peak pixel values ​​corresponding to at least a subset of pixels of a frame based at least in part on analyzing image data corresponding to the subset of pixels using the frame analysis geometry; sending pixel data representing the one or more peak pixel values; as well as After sending the pixel data, sending the image data representing pixel values ​​corresponding to at least the subset of pixels; as well as a display device communicatively coupled to the computing device, the display device configured to: determining, based at least in part on the pixel data, lighting settings for one or more backlights of the display device located proximate the subset of pixels on the display device; updating one or more of the pixel values ​​corresponding to the subset of pixels based at least in part on the lighting setting to generate one or more updated pixel values; as well as The frame is caused to be displayed using the illumination settings of the one or more backlights and the one or more updated pixel values ​​for the subset of pixels.

2. The system of claim 1 , wherein updating one or more of the pixel values ​​is further based at least in part on one or more other lighting settings, the one or more other lighting settings corresponding to one or more other backlights of the display device that are different from the one or more backlights of the display device.

3. The system of claim 1 , wherein the pixel subsets correspond to a backlight resolution of the display device, such that each pixel subset comprising the pixel subsets includes a horizontal resolution equal to the number of columns of pixels divided by the number of columns of the backlight, and a vertical resolution equal to the number of rows of pixels divided by the number of rows of the backlight.

4. The system of claim 3 , wherein the one or more backlights located near the pixel subset include backlights within five or fewer rows of backlights closest to a center pixel of the pixel subset and backlights within five or fewer columns of backlights closest to the center pixel of the pixel subset.

5. The system of claim 1, wherein the display device comprises a liquid crystal display (LCD) device and the one or more backlights comprise light emitting diodes (LEDs).

6. The system of claim 1, wherein the pixel data representing the one or more peak pixel values ​​comprises at least one of a peak red value, a peak green value, or a peak blue value corresponding to the subset of pixels.

7. The system of claim 1 , wherein determining the one or more peak pixel values ​​comprises: generating one or more histograms of pixel values ​​corresponding to the subset of pixels; as well as The one or more peak pixel values ​​are determined using the one or more histograms.

8. The system of claim 1 , wherein updating one or more of the pixel values ​​corresponding to the subset of pixels is based at least in part on a cumulative illumination contribution of backlights of the display device, the backlights comprising the one or more backlights and one or more other backlights in addition to the one or more backlights.

9. The system of claim 1 , wherein the system is included in at least one of: Systems implemented using edge devices; A system comprising one or more virtual machines VM; A system implemented at least in part in a data center; or A system implemented at least in part using cloud computing resources.

10. A processor comprising: Processing circuitry for: receiving, from a computing device, first pixel data representing one or more first peak pixel values ​​corresponding to a first subset of pixels of a first image, the first pixel data generated using the computing device and based at least in part on an analysis of the first image; determining one or more illumination values ​​corresponding to one or more backlights of the display panel based at least in part on the one or more first peak pixel values; receiving, after receiving the first pixel data, first image data representing first pixel values ​​of at least the first subset of pixels of the first image from the computing device; updating at least one of the first pixel values ​​based at least in part on the one or more illumination values ​​to generate one or more updated pixel values; causing the first image to be displayed using the one or more illumination values ​​of the one or more backlights and the one or more updated pixel values ​​of the first subset of pixels on the display panel; receiving, from the computing device, second pixel data representing one or more second peak pixel values ​​corresponding to a second subset of pixels of a second image while the first image is displayed on the display panel, the second pixel data generated using the computing device and based at least in part on an analysis of the second image; as well as Second image data representing second pixel values ​​of at least the second subset of pixels of the second image is received from the computing device after receiving the second pixel data.

11. The processor of claim 10 , wherein displaying the first image is caused by: causing current to be driven to the one or more backlights based at least in part on the one or more illumination values; and Based at least in part on the one or more updated pixel values, voltages are caused to be driven to pixel values ​​of the display panel corresponding to the first subset of pixels.

12. The processor of claim 10, wherein the first pixel data and the second pixel data are generated using one or more graphics processing units (GPUs) of the computing device, the one or more backlights comprise light emitting diodes (LEDs), and the display panel comprises a liquid crystal display (LCD).

13. The processor of claim 10, wherein at least one of the first pixel values ​​is updated based at least in part on one or more other illumination values ​​corresponding to one or more other backlights located a distance from the first subset of pixels.

14. The processor of claim 10, wherein the one or more backlights include at least a backlight positioned closest to a center pixel of the first subset of pixels on the display panel.

15. The processor of claim 14, wherein the one or more backlights include a backlight adjacent to a backlight positioned closest to a location of a center pixel of the first subset of pixels on the display panel.

16. The processor of claim 15 , wherein backlights adjacent to a backlight positioned closest to a center pixel of the first subset of pixels on the display panel are within a plurality of rows and columns of backlights positioned closest to a center pixel of the first subset of pixels on the display panel.

17. The processor of claim 11 , wherein the first pixel data representing the one or more first peak pixel values ​​is generated by: generating one or more histograms corresponding to the first pixel values ​​of the first subset of pixels; and The one or more first peak pixel values ​​are determined using the one or more histograms.

18. A method comprising: determining, using a computing device communicatively coupled to the display device, one or more first peak pixel values ​​corresponding to a first subset of pixels of the first image; sending, using the computing device, first pixel data representing the one or more first peak pixel values ​​to the display device; After sending the first pixel data, sending, using the computing device, first image data representing first pixel values ​​corresponding to at least the first subset of pixels to the display device; determining, using the computing device, one or more second peak pixel values ​​corresponding to a second subset of pixels of a second image; sending, using the computing device, second pixel data representing the one or more second peak pixel values ​​to the display device while the display device displays the first image; as well as After sending the second pixel data, second image data representing second pixel values ​​corresponding to at least the second subset of pixels is sent to the display device using the computing device.

19. The method of claim 18, wherein the method further comprises selecting the one or more backlights based at least in part on a shape of a backlight light distribution of the one or more backlights.

20. The method of claim 18, wherein the determining one or more second peak pixel values ​​corresponding to a second subset of pixels of a second image occurs at least in part during the transmission of the first image data.

Images