Frame rate conversion apparatus and method based on directional modulation and dithering

Abstract

The present application provides a frame rate conversion device based on directional modulation and dithering, which comprises a directional differential modulation generator configured to receive a plurality of input color data representing color components of input pixel colors, a plurality of synchronization signals and a control signal and generate a plurality of modulation data for the plurality of input color data respectively; and a plurality of dithering modules configured to perform K-bit dithering for each input color data to convert the corresponding input color to a corresponding output color with K-bit color depth for each component, where K is an integer equal to or greater than 1. The present application enables a display device to support a frame rate higher than its standard configuration without observable color depth degradation.

Description

Technical Field

[0001] This invention generally relates to active matrix display devices. More specifically, this invention relates to active matrix display devices with frame rate convertible based on digital drive signals. Background Technology

[0002] Display devices often need to process various types of video content and image sources to display smooth and realistic video. Generally, active matrix display devices contain pixels, and each pixel contains driver circuitry. The driver circuitry includes switching elements such as transistors and storage elements such as capacitors for actively addressing the pixel and maintaining its state. Typically, pixels are selected row by row by a gate driver via multiple scan lines, and then the source driver controls the illumination of each pixel in the selected row via corresponding data lines to display the image.

[0003] Active matrix display devices can be driven using either analog or digital drive signals. In analog methods, pixel brightness is controlled by analog signals, such as the voltage or current level of the drive signal. In digital methods, pixel brightness is controlled by the pulse width of the drive signal. Digital methods are more prevalent than analog methods because they can directly use digital video signals for pixel driving, thus requiring relatively simpler driver circuits and consuming less power. Digital methods also offer better brightness uniformity because display quality is less sensitive to changes in the current-voltage characteristics of the transistors in the pixel driver circuit.

[0004] In digital modulation methods, each pixel's image frame is divided into several subframes, and each subframe corresponds to a bit in the digital image data to be displayed. Subframes can have different durations, and these different durations are weighted according to the position of the bit to be represented and based on the following rule: the higher the number of valid bits represented by the subframe, the longer the subframe duration.

[0005] For each subframe, each row of pixels is scanned for a certain scan time. The pixels in the scanned row are then controlled to emit light at a fixed brightness (on) or zero brightness (off) to represent the logic values ​​"1" or "0" respectively, and this state is maintained for the duration of the subframe. Thus, 2-1 can be achieved by summing the duration for which the pixels are on within each frame. K Each level of grayscale.

[0006] Normally, scan lines are scanned sequentially within each subframe, with subframes arranged in ascending / descending order and repeated periodically. However, to achieve high resolution or dynamic range, the scan speed may not be high enough to complete the scan before the next frame begins. If the scan time of the current frame is longer than the period of the last subframe and overflows into the first subframe of the next frame, then two scan lines are operating simultaneously during the first subframe of the next frame.

[0007] Therefore, with limited display capabilities, display devices need to achieve a good balance between color depth and frame rate to achieve optimal display quality. For example, a standard configuration with 24-bit color depth at a 60Hz frame rate is sufficient for most general-purpose display devices. Higher frame rates (e.g., 120Hz) can be used to display fast-moving objects to avoid motion blur, but this results in a lower color depth (e.g., 12-bit). Reduced color depth can lead to inaccurate color rendering. For example, when an image initially displayed with 24-bit color depth (such as...)... Figure 1A When displayed with a 9-bit color depth, as shown, noticeable color banding will occur in certain areas (e.g., Figure 1B (As shown). Therefore, it is desirable for display devices to support frame rates higher than their standard configuration without an observable decrease in color depth. Summary of the Invention

[0008] According to one aspect of the invention, a frame rate conversion apparatus based on directional modulation and dithering is provided, the frame rate conversion apparatus comprising a directional differential modulation generator configured to receive a plurality of input color data representing color components of an input pixel color, a plurality of synchronization signals and control signals, and to generate a plurality of modulation data for each of the plurality of input color data; and a plurality of dithering modules configured to perform K-bit dithering for each input color data to convert the corresponding input color into a corresponding output color having a K-bit color depth for each component, wherein K is an integer equal to or greater than 1.

[0009] According to one aspect of the present invention, a dynamic motion detection method for display is provided, comprising detecting motion content in a video and generating a motion detection signal, and generating a control signal for controlling the display color depth of the video based on the motion detection signal. If the motion detection signal indicates that the video contains significant motion content, the display device displays the video at a higher frame rate and a lower color depth than a standard configuration; and if the motion detection signal indicates that the video is relatively static, the display device displays the video at a lower frame rate and a higher color depth than a standard configuration.

[0010] By applying directional modulation before performing K-bit dithering on each input color data to generate output color data with K-bit color depth per component, display devices can support frame rates higher than their standard configuration without an observable drop in color depth. For example... Figure 1C As shown, by implementing the frame rate conversion method provided by this invention, even if the color depth of an image is reduced from 24 bits to 9 bits, color banding will not occur due to the reduction in color depth. Furthermore, by enabling the display device to dynamically convert its display output format according to the motion content of the video, display quality can be further optimized. Attached Figure Description

[0011] The embodiments of the present invention are described in more detail below with reference to the accompanying drawings, in which:

[0012] Figure 1A The image was originally displayed in 24-bit color depth; Figure 1B Displayed a subtractive color image with a 9-bit color depth; and Figure 1C A 9-bit color depth subtractive image improved by the driving method provided by the present invention is shown;

[0013] Figure 2 A simplified system block diagram of a frame rate-convertible active matrix display device according to an embodiment of the present invention is shown;

[0014] Figure 3 A block diagram of a frame rate conversion apparatus based on directional modulation and jitter according to an embodiment of the present invention is depicted:

[0015] Figure 4A This demonstrates how to convert an input color with 8 bits of color depth per component to an output color with 1 bit of color depth per component. Figure 4B This shows how to convert an input color with 8 bits of color depth per component to an output color with 3 bits of color depth per component;

[0016] Figures 5A-5C It describes how to divide color spaces based on different color depths;

[0017] Figure 6A-6G The illustration shows some exemplary differential modulation directions determined by setting the modulation threshold to half the maximum value of the pixel color;

[0018] Figures 7A-7C The diagram illustrates how modulation is applied and how to determine the color levels for the pixel's color data within the modulation period;

[0019] Figures 8A-8D This demonstrates how input image sources with different display formats can be converted into a mixed output image source with different display formats;

[0020] Figure 9 A simplified block diagram of a dynamic motion detection device according to an embodiment of the present invention is shown;

[0021] Figure 10 This demonstrates how exemplary video clips can be segmented into different video segments to perform motion detection; and

[0022] Figure 11A-11C This demonstrates how to determine different display output formats based on motion detection of different video segments in a sample video clip. Detailed Implementation

[0023] In the following description, a method and system for driving an active matrix display to perform frame rate conversion are illustrated as preferred examples. Those skilled in the art will appreciate that modifications, including additions and / or substitutions, can be made without departing from the scope and spirit of the invention. Specific details may be omitted to avoid obscuring the invention; however, this disclosure is intended to enable those skilled in the art to practice the teachings herein without undue experimentation.

[0024] Figure 2 A simplified system block diagram of a frame rate-convertible active matrix display device 1 according to an embodiment of the present invention is shown. In this embodiment, each pixel color may include red, green, and blue (RGB) pixel color components. Accordingly, each pixel color data may include color component data representing the red, green, and blue (RGB) pixel color components, respectively.

[0025] like Figure 2 As shown, the display device 1 may include a main processor 11; a timing controller 12 connected to the main processor 11; a gate driver 13 connected between the timing controller 12 and an active matrix display panel (not shown); and a source driver 14 connected between the timing controller 12 and the active matrix display panel. The main processor 11 may be configured to generate multiple input color data (R_In, G_In, and B_In) and a synchronization signal (V_Sync) representing the RGB color components of the input pixel colors. The timing controller 12 may be configured to receive the multiple input display data and synchronization data and generate multiple output display data (R_Out / G_Out / B_Out) to the source driver 14 and multiple row selection signals (V_row) to the gate driver 13.

[0026] The timing controller 12 may include a motion detection device 121 for detecting motion content in the video and generating a motion detection signal (V_MD); a frame rate controller 122 is configured to receive the motion detection signal and input display data, and generate a control signal (V_Ctrl) for controlling the display color depth of the video; a frame rate conversion device 123 is configured to receive the control signal and convert the input display data into output display data based on directional modulation and dithering, thereby enabling the display device to display video segments without observable color depth drop when the display frame rate is higher than its standard configuration. The timing controller 12 may also include a frame buffer 124 connected to the frame rate controller 122 and configured to store color data.

[0027] In particular, if the motion detection results indicate that the video contains significant moving content, the display device will display the video at a higher frame rate (e.g., 120Hz) and a lower color depth (e.g., 4-bit color depth per color component) than the standard configuration. If the motion detection results indicate that the video is relatively static, the display device will display the video at a lower frame rate (e.g., 60Hz) and a higher color depth (e.g., 8-bit color depth per color component) than the standard configuration.

[0028] Figure 3 A block diagram of a frame rate conversion apparatus 123 based on directional modulation and jitter according to an embodiment of the present invention is depicted. (Refer to...) Figure 3 The frame rate conversion device 123 based on directional modulation and jitter may include a directional differential modulation generator 310 and multiple jitter modules 320.

[0029] The directional differential modulation generator 310 can be configured to receive multiple input color data (R_In, G_In, and B_In), multiple synchronization signals (V_Sync), and control signals (V_Ctrl); and generate multiple modulation data (R_Mod, G_Mod, and B_Mod) for the multiple input color data respectively.

[0030] Each jitter module 320 may include a residual line buffer 322 configured to track residuals in jitter conversion and generate residual data (R_Res / G_Res / B_Res); and an adapter (or adder) 321 configured to receive corresponding input color data (R_In / G_In / B_In), corresponding modulation data (R_Mod / G_Mod / B_Mod) from the directional differential modulation generator 310, and corresponding residual data (R_Res / G_Res / B_Res) from the corresponding residual line buffer 322, to adjust the corresponding input color data by adding the corresponding modulation data and the corresponding residual data to generate corresponding adapted color data (R_AD / G_AD / B_AD).

[0031] Each dithering module 320 may also include a dithering engine 323, which is configured to receive corresponding adaptive color data (R_AD / G_AD / B_AD) from the corresponding adapter 321 and generate corresponding output color data (R_Out / G_Out / B_Out).

[0032] Depending on the frame rate conversion target, each dithering engine 323 can be configured to perform K-bit dithering to convert the input color into an output color with K-bit color depth per component, where K is an integer equal to or greater than 1, which can be selected by a control signal (V_ctrl) from the frame rate controller 122; and to combine the corresponding input color data with (2 K-1) Dithering thresholds are compared to generate the corresponding output color data. K One possible output color level.

[0033] like Figure 4A As shown, to convert 8-bit color depth input color data (Data_in) for each component into 1-bit color depth output color data (Data_Out), the dithering engine 323 can be configured to perform 1-bit dithering to output two possible output color levels, L0 and L1, which can be set to color values ​​of 0 and 255, respectively. The input color data, which may have 256 possible color levels (0, 1, ..., 255), is compared with a dithering threshold (e.g., 128) to determine the output color level of the output color data. For example, when the value of the input color data is 87 (less than the dithering threshold 128), the dithering engine 323 outputs L0 (i.e., a color value of "0") for the output color data and inputs the residual data (Data_Res) equal to 87 - 0 = 87 into the residual line buffer 322 for dithering the input color of adjacent pixels.

[0034] like Figure 4B As shown, to convert 8-bit color depth input color data (Data_in) for each component into 3-bit color depth output color data (Data_Out), the dithering engine 323 can be configured to perform 3-bit dithering to output eight possible output color levels, L0 to L7, which can be set to color values ​​of 0, 36, 73, ..., 255, respectively. The input color data, which may have 256 possible color levels (0, 1, ..., 255), is compared with seven dithering thresholds (e.g., 18, 55, 91, ..., 236) to determine the output color level of the output color data. For example, when the input color data value is 173 (between 164 and 199), the dithering engine 323 outputs L5 (i.e., a color value of "182") for the output color data and inputs a residual data (Data_Res) equal to 173 - 182 = -9 into the residual line buffer 322 to dither the input color of adjacent pixels.

[0035] The color space cube used to represent pixel colors can be divided into multiple sub-color space cubes based on the desired color depth. For example, in an RGB color space with a K-bit color depth in each RGB direction, a color space cube can be divided into 8 sub-color space cubes. K Each sub-color space cube has K quantization levels in each RGB direction. Correspondingly, each sub-color space cube corresponds to a set of RGB levels.

[0036] Figures 5A-5C It describes how to divide color spaces based on different color depths. (Reference) Figure 5AFor each component with a 1-bit color depth, the color space cube is divided into 8 sub-color space cubes, and there are two color levels (L0 and L1) to represent the pixel color in each RGB component. (See reference) Figure 5B For each component with a 2-bit color depth, the color space cube is divided into 64 sub-color space cubes, and there are four color levels (L0 to L3) used to represent the pixel color in each RGB component. (See reference) Figure 5C For a color depth of 3 bits per component, the color space cube is divided into 512 sub-color space cubes, and there are eight color levels (L0 to L7) used to represent the pixel color in each RGB component. It can be seen that the fewer bits the color depth has, the fewer color levels are available to represent the pixel color in each RGB component, and the greater the loss of resolution due to high quantization error.

[0037] The directional differential modulation generator 310 can also be configured to determine the modulation direction by comparing each color component of the input pixel with a modulation threshold and obtaining a modulation flag value for each color component. For example, if the value of a color component of the input pixel is equal to or greater than the modulation threshold, the modulation flag value of the color component is set to "1"; otherwise, the modulation flag value of the color component is set to "0".

[0038] Accordingly, the modulation flag value of each color component can be used to construct a modulation direction unit vector U_m(x_m, y_m, z_m) in the RGB color space to represent the modulation direction, where x_m, y_m, and z_m are the RGB components of the unit vector U_m, respectively. Each of the RGB components x_m, y_m, and z_m of the modulation direction unit vector U_m can have a binary value ("1" or "0"), which can be determined by comparing each input color data with a modulation threshold. For example, if the R component value of the input pixel color is equal to or greater than the modulation threshold, x_m is set to "1"; otherwise, x_m is set to "0". In other words, whether differential modulation is applied to a color component (direction) in the color space depends on whether the component value on that color component (direction) is equal to or greater than the modulation threshold.

[0039] Figure 6A-6G Some exemplary differential modulation directions are shown, determined by setting the modulation threshold to half the maximum value (M) of the RGB components within the input pixel.

[0040] Reference Figure 6A When all R, G, and B component values ​​of a pixel are equal to or greater than M / 2, the RGB components x_m, y_m, and z_m of the modulation direction unit vector U_m are all equal to "1". Therefore, the differential modulation direction is in the white (ΔW) direction.

[0041] Reference Figure 6B When the R component value of a pixel is equal to or greater than M / 2 and the G and B component values ​​of the pixel are both less than M / 2, the RGB components x_m, y_m, and z_m of the modulation direction unit vector Um are equal to "1", "0", and "0", respectively. Therefore, the differential modulation direction is the red (ΔR) direction.

[0042] Reference Figure 6C When the G component value of a pixel is equal to or greater than M / 2 and the R and B component values ​​of the pixel are both less than M / 2, the RGB components x_m, y_m, and z_m of the modulation direction unit vector U_m are equal to "0", "1", and "0", respectively. Therefore, the differential modulation direction is the green (ΔG) direction.

[0043] Reference Figure 6D When the B component value of a pixel is equal to or greater than M / 2 and the R and G component values ​​of the pixel are both less than M / 2, the RGB components x_m, y_m, and z_m of the modulation direction unit vector U_m are equal to "0", "0", and "1", respectively. Therefore, the differential modulation direction is in the blue (ΔB) direction.

[0044] Reference Figure 6E When the R and G component values ​​are both greater than or equal to M / 2 and the B component value is less than M / 2, the RGB components x_m, y_m, and z_m of the modulation direction unit vector U_m are equal to "1", "1", and "0", respectively. Therefore, the differential modulation direction is the yellow (ΔY) direction.

[0045] Reference Figure 6F When the G and B component values ​​are both greater than or equal to M / 2 and the R component value is less than M / 2, the RGB components x_m, y_m, and z_m of the modulation direction unit vector U_m are equal to "0", "1", and "1", respectively. Therefore, the differential modulation direction is in the cyan (ΔC) direction.

[0046] refer to Figure 6G When the B and R component values ​​are both greater than or equal to M / 2 and the G component value is less than M / 2, the RGB components x_m, y_m, and z_m of the modulation direction unit vector U_m are equal to "1", "0", and "1", respectively. Therefore, the differential modulation direction is in the magenta (ΔM) direction.

[0047] The directional differential modulation generator 310 can also be configured to apply directional differential modulation to each color component of a pixel based on a determined modulation direction to obtain modulation data for that color component.

[0048] Differential modulation can be performed across a sequence of image frames using a differentially modulated data sequence over a modulation period. Within the modulation period, the modulated data for the color components obtained in the i-th frame can be given by the following formula:

[0049] Xmi =X oi +d i for i = 1, 2, ..., N,

[0050] Where X mi X is the modulation data of the color components obtained in the i-th frame. oi d is the raw input data of the color components in the i-th frame. i is the differential modulation value used in the i-th frame, which may be positive or negative, and N is the total number of frames in one modulation period.

[0051] Preferably, a differential modulation value sequence d with a sum equal to 0 can be selected. i ,Right now In order to apply differential modulation in a balanced manner across image frames.

[0052] Within the modulation period, the jitter engine 323 can also be configured to be based on the modulation data {X} obtained across N frames. mi The N color levels of the output pixel color components are determined by the expression {i = 1, 2, ..., N}.

[0053] Based on the modulation data X obtained in the i-th frame mi The color levels of the output pixel color components can be determined using the algorithm given by the following formula:

[0054]

[0055] Among them, C i L is the color level of the input color data obtained in the i-th image frame. k It is the k-th color level defined in the color space to be displayed, which has a color depth of K bits per component.

[0056] Dithering engine 323 can also be configured to average the color levels of the output pixel color components determined across a frame sequence during the modulation period to obtain the average display color C. avg It is given by the following formula: Set the average display color value to the output color value.

[0057] Figures 7A-7C The illustration shows how modulation is applied under three different conditions and how color levels are determined for the output pixel color components within a 6-frame (F1 to F6) modulation period. For simplicity, the three-dimensional (3D) sub-color space cube is simplified to two-dimensional (2D) sub-color space blocks arranged along the color component directions, each sub-color space block corresponding to a color level in a certain component direction. Furthermore, only three sub-color space blocks are displayed per frame, i.e., L... k-1 L k and L k+1Because modulation will only cause the color components of the output pixel to switch between adjacent color levels, and it is also assumed that the color component value X0 of the output pixel is between and Between, represented as L k A point within the corresponding sub-color space square. For example, setting the differential modulation value (d) used in the modulation period. i The values ​​are: d1 = 0, d2 = -δ, d3 = δ, d4 = 0, d5 = -2δ and d6 = 2δ, where δ is a predefined differential value.

[0058] refer to Figure 7A In this case, the color components of the output pixel have a greater than 100%. And less than L k The value of . That is, The color level of the pixel color components in the above 6 frames is determined as follows: C1 = L k C2 = L k-1 C3 = L k C4 = L k C5 = L k-1 , and C6 = L k Display color average C avg equals (2L) k-1 +4L k ) / 6, that is, having a value between L k-1 and L k The color values ​​between.

[0059] refer to Figure 7B In this case, the color components of the output pixel have a value equal to L. k The value of . That is, X0 = L k The color level of the pixel color components in the above 6 frames is determined as follows: C1 = L k C2 = L k C3 = L k C4 = L k C5 = L k-1 and C6=L k+1 Display color average C avg equals (L) k-1 +4L k +L k+1 ) / 6, which means having equal to L k. The color value.

[0060] refer to Figure 7C In this case, the color components of the output pixel have a value greater than L. k and less than The value of . That is, The color level of the pixel color components in the above 6 frames is determined as follows: C1 = L kC2 = L k C3 = L k+1 C4 = L k C5 = L k , and C6 = L k+1 Display color average C avg equals (4L) k +2L k+1 ) / 6, that is, having a value between L k and L k+1. The color values ​​between.

[0061] from Figures 7A-7C It can be seen that if dithering is performed without modulation, the color level of the output pixel's color component will be determined to be L for each frame. k By applying modulation, in Figure 7A The middle has greater than And less than L k The pixel of color component value can have a value between L during the modulation period. k-1 and L k The average display color value between; Figure 7B It has equal to L k The pixel of color component value has an equal to L during the modulation period. k The average display color value; in Figure 7C The middle has greater than L k And less than The pixel of color component value has a value between L during the modulation period. k and L k+1 The average displayed color value between [the two values]. In other words, by applying directional modulation, the observable color depth drop caused by frame rate conversion can be eliminated.

[0062] Return to reference Figure 3 An input image source for a display device with a 60Hz frame rate and 8-bit color depth per component can be converted into an output image at a 240Hz frame rate with 2-bit color depth per component, at a 180Hz frame rate with 3-bit color depth per component, or at a 120Hz frame rate with 4-bit color depth per component by a frame rate conversion device based on dithering and directional modulation.

[0063] The frame rate conversion device based on dithering and directional modulation can also be configured to support the conversion of input image sources (e.g., from a computer graphics card) displayed at other frame rates (including, but not limited to, 240Hz, 200Hz, 180Hz, 150Hz, 120Hz, 100Hz, and 80Hz) to allow the display device to display the image source at a lower frame rate (e.g., 60Hz).

[0064] In some embodiments, the output image source may be a mixture of different display formats. Figures 8A-8D It shows how different input image sources are converted into different output image sources with different display formats.

[0065] refer to Figure 8A An input image source with 8-bit color depth per component at a frame rate of 200Hz can be converted into a mixed output image source with 3-bit color depth per component at a frame rate of 180Hz and 2-bit color depth per component at a frame rate of 240Hz.

[0066] refer to Figure 8B An input image source with 8-bit color depth per component at a frame rate of 150Hz can be converted into a mixed output image source with 4-bit color depth per component at a frame rate of 120Hz and 3-bit color depth per component at a frame rate of 180Hz.

[0067] refer to Figure 8C An input image source with 8-bit color depth per component at a frame rate of 100Hz can be converted into a mixed output image source with 4-bit color depth per component at a frame rate of 120Hz and 8-bit color depth per component at a frame rate of 60Hz.

[0068] refer to Figure 8D An input image source with 8-bit color depth per component at 80Hz frame rate can be converted into a mixed output image source with 4-bit color depth per component at 120Hz frame rate and 8-bit color depth per component at 60Hz frame rate.

[0069] Figure 9 A simplified block diagram of a dynamic motion detection device 121 according to an embodiment of the present invention is shown. (Refer to...) Figure 9 The dynamic motion detection process may include: a) dividing the display screen of the display device into multiple regions by a brightness accumulator 910; b) calculating the brightness values ​​of multiple regions in the first frame by the brightness accumulator 910; c) storing the brightness values ​​of multiple regions in the first frame into a first brightness data array A1 by a storage unit 920; d) calculating the brightness values ​​of multiple regions in the second frame by the brightness accumulator 910, the second frame being ΔF frames after the first frame, where ΔF is an integer greater than 1, preferably 15; e) storing the brightness values ​​of the second region into a second brightness data array A2 by the storage unit 920; f) comparing the first and second brightness data arrays A1 and A2 by a brightness change detector 930 to obtain a brightness difference array; g) detecting the brightness change of each region by the brightness change detector 930 by comparing each element of the brightness difference array with one or more voting thresholds; h) generating a vote by the brightness change detector 930 based on the comparison results of each element of the brightness difference array.

[0070] In some embodiments, if the comparison result is that the element is equal to or lower than a first voting threshold, the vote for that element may have a first voting value; if the comparison result is that the element is higher than the first voting threshold and lower than a second voting threshold, the vote for that element may have a second voting value higher than the first voting value; if the comparison result is that the element is equal to or higher than the second voting threshold, the vote for that element may have a third voting value higher than the second voting value.

[0071] The dynamic motion detection process may further include i) the majority voting logic unit 940 calculating the sum of votes generated by all elements of the luminance difference array; j) the majority voting logic unit 940 comparing the calculated sum of votes with one or more motion detection thresholds to determine the motion detection result; and k) the majority voting logic unit 940 generating a motion detection signal (V_MD) to the frame rate controller 122.

[0072] In some embodiments, if the sum of the calculated votes is equal to or greater than a motion detection threshold, the motion detection result can be determined to indicate that the video contains significant moving content. Based on the determined motion detection result, the frame rate controller 122 can determine to display the video at a higher frame rate and lower color depth than a standard configuration (e.g., a frame rate of 120 Hz and a color depth of 4 bits per color component). If the sum of the calculated votes is less than the motion detection threshold, the motion detection result can be determined to indicate that the video is relatively static. Based on the determined motion detection result, the frame rate controller 122 can determine to display the video at a lower frame rate and higher color depth than a standard configuration (e.g., a frame rate of 60 Hz and a color depth of 8 bits per color component).

[0073] A new round of motion detection can be performed as follows: The second frame from the previous round of motion detection is used as the first frame for the new round; the region brightness value of the new second frame is calculated, with the second frame following the first frame by ΔF frames; the calculated region brightness value of the next frame is used to overwrite the brightness data array (e.g., brightness data array A1) storing the region brightness values ​​of the first frame from the previous round of motion detection; and steps f) to k) are repeated. Since it is not necessary to calculate the region brightness value of the first frame, the computation time for the new round of motion detection can be greatly reduced.

[0074] Figure 10 The diagram illustrates how an exemplary video clip (showing "a person jumping from left to right") is segmented into different video segments to perform motion detection. Figure 11A-11C This demonstrates how to determine different display output formats based on motion detection of different video segments in a sample video clip.

[0075] Reference Figure 10The display screen used to show this exemplary video clip is divided into 14x8 = 112 areas. The example video clip, originally with a frame rate of 120Hz and 8-bit color depth per color component, is divided into three video segments, VS1, VS2, and VS3, for motion detection. In each video segment, the second frame is 15 frames after the first frame.

[0076] refer to Figure 11A The first and second frames of video clip VS1 are represented as F1 and F2 respectively. 16 Calculate the luminance values ​​of 112 regions in frame F1 and store them in the first 14x8 luminance data array A1. Calculate frame F 16 The 112 area brightness values ​​are then stored in a second 14x8 brightness data array A2. The first and second brightness data arrays A1 and A2 are compared to obtain a 14×8 brightness difference array, with each brightness difference corresponding to one area. A corresponding vote is generated based on the brightness difference for each area. If the brightness difference is equal to or lower than a first voting threshold (e.g., 5%), the corresponding vote has a first voting value of "0"; if the brightness difference is higher than the first voting threshold by 5% but lower than a second voting threshold (e.g., 20%), the corresponding vote has a second voting value of "1"; and if the brightness difference is equal to or higher than the second voting threshold by 20%, the corresponding vote has a third voting value of "2". The sum of all generated votes is calculated to determine a first motion detection result. Based on the first motion detection result, a corresponding motion detection signal is generated and transmitted to the frame rate controller 122. For example, if the sum of the calculated votes is less than the motion detection threshold (e.g., 100), based on the motion detection signal, the frame rate controller 122 can determine a display output format with 8 bits of color depth per color component at a frame rate of 60Hz.

[0077] refer to Figure 11B The second frame of the previous video segment VS1 is taken as the first frame of video segment VS2. Therefore, the first and second frames of video segment VS2 are represented as F. 16 and F 31 (that is, in F) 16 (After 15 frames). Frame F is stored in the second 14x8 luminance data array A2. 16 Simultaneously, calculate the frame F of the 112 region brightness values. 31The 112 area brightness values ​​are then stored in the first 14x8 brightness data array A1. The first and second brightness data arrays A1 and A2 are then compared to obtain a 14×8 brightness difference array, with each brightness difference corresponding to one area. A corresponding vote is generated based on the brightness difference for each area. If the brightness difference is equal to or lower than a first voting threshold (e.g., 5%), the corresponding vote has a first voting value of "0"; if the brightness difference is higher than the first voting threshold by 5% but lower than a second voting threshold (e.g., 20%), the corresponding vote has a second voting value of "1"; and if the brightness difference is equal to or higher than the second voting threshold by 20%, the corresponding vote has a third voting value of "2". The sum of all generated votes is calculated to determine a second motion detection result. Based on the second motion detection result, a corresponding motion detection signal is generated and transmitted to the frame rate controller 122. For example, if the calculated sum of votes is equal to or greater than a motion detection threshold (e.g., 100), based on this motion detection signal, the frame rate controller 122 can determine a display output format with 4 bits of color depth per color component at a frame rate of 120Hz.

[0078] refer to Figure 11C The second frame of the previous video segment VS2 is taken as the first frame of video segment VS3. Therefore, the first and second frames of video segment VS3 are represented as F. 31 and F 46 (that is, in F) 31 (After 15 frames). Frame F is stored in the first 14x8 luminance data array A1. 31 Simultaneously, calculate the frame F of the 112 region brightness values. 46 The 112 area brightness values ​​are then stored in a second 14x8 brightness data array A2. The first and second brightness data arrays A1 and A2 are then compared to obtain a 14×8 brightness difference array, with each brightness difference corresponding to one area. A corresponding vote is generated based on the brightness difference for each area. If the brightness difference is equal to or lower than a first voting threshold (e.g., 5%), the corresponding vote has a first voting value of "0"; if the brightness difference is higher than the first voting threshold by 5% but lower than a second voting threshold (e.g., 20%), the corresponding vote has a second voting value of "1"; and if the brightness difference is equal to or higher than the second voting threshold by 20%, the corresponding vote has a third voting value of "2". The sum of all generated votes is calculated to determine a third motion detection result. Based on the third motion detection result, a corresponding motion detection signal is generated and transmitted to the frame rate controller 122. For example, if the sum of the calculated votes is equal to or greater than a motion detection threshold (e.g., 100), based on this motion detection signal, the frame rate controller 122 can determine a display output format with 4-bit color depth per color component at a frame rate of 120Hz.

[0079] The embodiments disclosed herein may be implemented using general-purpose or special-purpose computing devices, computer processors, or electronic circuit systems, including but not limited to digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and other programmable logic devices configured or programmed according to the teachings of this disclosure. Computer instructions or software code that run in general-purpose or special-purpose computing devices, computer processors, or programmable logic devices may be made by those skilled in the art based on the teachings of this disclosure. This invention also includes computer storage media in which computer instructions or software code are stored, which can be used to program a computer or microprocessor to perform any of the processes of this invention. The storage medium may include, but is not limited to, ROM, RAM, flash memory devices, or any type of medium or device suitable for storing instructions, code, and / or data.

[0080] Those skilled in the art will understand that the above embodiments are merely illustrative of the working principle and practical application of the present invention, enabling other skilled in the art to understand various embodiments of the invention and various modifications suitable for specific intended uses, and are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations will be readily apparent to those skilled in the art.

Claims

1. A frame rate conversion device based on directional modulation and jitter, comprising: A directional differential modulation generator is configured to receive multiple input color data representing color components of an input pixel color, multiple synchronization signals, and control signals, and generate multiple modulation data for each of the multiple input color data; as well as Multiple dithering modules are configured to perform K-bit dithering for each input color data to convert the corresponding input color into a corresponding output color with K-bit color depth for each component, where K is an integer equal to or greater than 1. Each dithering module includes: The residual line buffer is configured to track the residual in the corresponding jitter conversion and generate the corresponding residual data. An adapter is configured to receive corresponding input color data, corresponding modulation data from the directional differential modulation generator, and corresponding residual data from a corresponding residual line buffer, to adjust the corresponding input color data by adding the corresponding modulation data and the corresponding residual data to generate corresponding adapted color data; and The dithering engine is configured to receive the corresponding adapted color data from the corresponding adapter and to match the corresponding adapted color data with (2) K -1) Dithering thresholds are compared to generate the corresponding output color data. K It calculates the possible output color levels and generates the corresponding output color data; The directional differential modulation generator is further configured as follows: The modulation direction of differential modulation is determined by comparing each input color data with a modulation threshold to obtain a modulation flag value for each color component; and Based on the determined modulation direction, directional differential modulation is applied to each color component of the input pixel color to obtain modulation data for the color component.

2. The frame rate conversion apparatus according to claim 1, wherein the modulation threshold is set to half of the maximum value of the input pixel color component.

3. The frame rate conversion apparatus according to claim 2, wherein the differential modulation is performed using a differential modulation data sequence across the image frame sequence in the modulation period.

4. The frame rate conversion apparatus according to claim 3, wherein the modulation data of the color components obtained in the i-th frame within the modulation period is given by the following formula: for , in It is the modulation data of the input pixel color components obtained in the i-th frame. It is the raw input data of the input pixel color components in the i-th frame. is the differential modulation value used in the i-th frame, and N is the total number of frames within the modulation period.

5. The frame rate conversion apparatus according to claim 4, wherein the differential modulation value sequence The sum of is equal to 0.

6. The frame rate conversion apparatus according to claim 5, wherein the dithering engine is further configured to determine N color levels of the output pixel color components based on modulation data obtained across N frames.

7. The frame rate conversion apparatus according to claim 6, wherein the modulation data is based on the modulation data obtained in the i-th frame. The color levels of the output pixel color components are determined using the algorithm given by the following formula: in, The color level of the output color data obtained in the i-th image frame. It is the k-th color level defined in the color space to be displayed, which has a color depth of K bits per component.

8. The frame rate conversion apparatus according to claim 7, wherein the jitter engine is further configured to The average color level of the output pixel color components determined across the frame sequence within the modulation period is taken to obtain the average display color; and Set the average color value to the output color value.

9. A frame rate convertible active matrix display device, comprising the frame rate conversion device based on directional modulation and dithering as described in claim 1.

10. A frame rate conversion method based on directional modulation and jitter, comprising: The directional differential modulation generator receives multiple input color data representing the color components of the input pixel color, multiple synchronization signals, and control signals. A directional differential modulation generator generates multiple modulation data for each of the multiple input color data; Multiple dithering modules perform K-bit dithering on each input color data to convert the corresponding input color into a corresponding output color with a K-bit color depth for each component, where K is an integer equal to or greater than 1; The K-bit jitter mentioned above includes: The corresponding residual line buffer tracks the corresponding residual in the jitter conversion and generates the corresponding residual data. The corresponding adapter receives the corresponding input color data, the corresponding modulation data from the directional differential modulation generator, and the corresponding residual data from the corresponding residual line buffer, so as to adjust the corresponding input color data by adding the corresponding modulation data and the corresponding residual data to generate the corresponding adapted color data. The corresponding dithering engine receives the appropriate color data from the corresponding adapter. The corresponding dithering engine will match the appropriate color data with (2) K -1) Dithering thresholds are compared to generate the corresponding output color data. K It identifies possible output color levels and generates corresponding output color data; The step of generating the multiple modulation data for the multiple input color data includes: The directional differential modulation generator determines the modulation direction of the differential modulation by comparing each input color data with a modulation threshold and obtaining a modulation flag value for each color component; and The directional differential modulation generator applies directional differential modulation to each color component of the input pixel color based on a determined modulation direction to obtain modulation data for the color components.

11. The frame rate conversion method according to claim 10, wherein the modulation threshold is set to half of the maximum value of the input pixel color component.

12. The frame rate conversion method according to claim 11, wherein the differential modulation is performed using a differential modulation data sequence across the image frame sequence in the modulation period.

13. The frame rate conversion method according to claim 12, wherein the modulation data of the color components obtained in the i-th frame within the modulation period is given by the following formula: for , in It is the modulation data of the input pixel color components obtained in the i-th frame. It is the raw input data of the input pixel color components in the i-th frame. is the differential modulation value used in the i-th frame, and N is the total number of frames within the modulation period.

14. The frame rate conversion method according to claim 13, wherein the differential modulation value sequence The sum of is equal to 0.

15. The frame rate conversion method according to claim 14 further includes determining N color levels of the output pixel color components by the dithering engine based on the modulation data obtained across N frames.

16. The frame rate conversion method according to claim 15, wherein the modulation data is based on the modulation data obtained in the i-th frame. The color levels of the output pixel color components are determined using the algorithm given by the following formula: in, The color level of the output color data obtained in the i-th image frame. It is the k-th color level defined in the color space to be displayed, which has a color depth of K bits per component.

17. The frame rate conversion method according to claim 16, further comprising: The dithering engine averages the color levels of the output pixel color components determined across the frame sequence within the modulation period to obtain the average display color. and The dithering engine sets the average color value to the output color value.

Images