Image compensation method for display apparatus, display apparatus, electronic device, and computer-readable storage medium

Abstract

The present invention relates to an image compensation method for a display apparatus, a display apparatus, an electronic device, and a computer-readable storage medium. The method comprises: acquiring a grayscale image of an image of a display apparatus; performing first compensation calculation on the grayscale image in a time domain to determine first compensation data; performing second compensation calculation on the grayscale image in a spatial domain to determine second compensation data; and adjusting the image on the basis of the first compensation data and the second compensation data.

Description

Image compensation method of display device, display device, electronic equipment and computer readable storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of display. More particularly, it relates to an image compensation method of display device, display device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] MLED (Mini Light Emitting Diode or Micro Light Emitting Diode) devices integrate a large number of small-sized LED chips together through micro-matrix packaging technology to form a backlight source. This technology can achieve fine control of the brightness of each LED, thereby achieving more delicate color representation and higher contrast. At the same time, due to the larger number of LEDs, the backlight uniformity of MiniLED is better, and the visual effect is more uniform and consistent. MLED has many advantages such as outstanding brightness, wide color gamut coverage, and clear resolution.

[0003] There is a challenge of image sticking in the display field. In the field of OLED displays, solutions to the problem of image sticking include adjusting the voltage according to the OLED aging curve to maintain consistent brightness, and using an external compensation method in the column driver IC using an analog-to-digital converter (ADC) to achieve uniform pixel brightness. However, MLED is a new type of backlight technology, and it is complex to solve the problem of image sticking of MLED devices.

[0004] SUMMARY

[0005] Embodiments of the present application provide an image compensation method. The method comprises: obtaining a grayscale image of an image of the display device; performing first compensation calculation on the grayscale image in the time domain to determine first compensation data; performing second compensation calculation on the grayscale image in the spatial domain to determine second compensation data; and adjusting the image according to the first compensation data and the second compensation data.

[0006] In some embodiments, the compensation calculation on the grayscale image in the time domain to determine the first compensation data comprises obtaining historical frame image data of a first number of frames before the current frame image data; for each frame in the historical frames, assigning a respective weight coefficient according to a lookup table comprising a time domain weight coefficient related to the spatial position of the frame; and performing weighted summation on the historical frame image data according to the respective weight coefficients to obtain the first compensation data.

[0007] In some embodiments, the closer the receiving time of the historical frame to the receiving time of the current frame, the larger the corresponding weight coefficient of the historical frame.

[0008] In some embodiments, the weight coefficient corresponding to the edge region of the display device is greater than the weight coefficient corresponding to the center region of the display device.

[0009] In some embodiments, the method further comprises employing a memory to store data in real time in the weighted summation process and to store the first compensation data.

[0010] In some embodiments, the compensation calculation in the time domain on the gray scale image to determine the first compensation data further comprises: obtaining initialization frame image data with a second frame number, wherein the second frame number is less than the first frame number; writing supplementary frame data with a third frame number and the initialization frame image data into the memory, wherein the sum of the third frame number and the second frame number is equal to the first frame number.

[0011] In some embodiments, the compensation calculation in the spatial domain on the gray scale image to determine the second compensation data comprises: performing compensation calculation by a bilinear interpolation method to determine the second compensation data.

[0012] In some embodiments, the compensation calculation by the bilinear interpolation method to determine the second compensation data comprises: writing data after the first compensation calculation into a first memory; in a first time period, the first memory outputs first row data in the first data; in a second time period, the first row data is written into a second memory; in a third time period, the first memory outputs second row data and the second memory outputs first row data, wherein the first row and the second row are adjacent two rows.

[0013] In some embodiments, the capacity of the first memory is greater than the capacity of the second memory.

[0014] In some embodiments, the addresses of the first memory and the second memory are column-aligned.

[0015] In some embodiments, the method further comprises mean value processing on the gray scale image, wherein the mean value processing comprises: dividing a pixel array of the display device into a plurality of partitions; and calculating a mean value of the gray scale value of each partition in the plurality of partitions.

[0016] In some embodiments, calculating the mean value of the gray scale value of each partition comprises: for each row of each partition, calculating the sum of the gray scale values of the pixels in the row; accumulating the sum of each row to obtain an accumulated value; and dividing the accumulated value by the total number of pixels in the each partition to obtain the mean value of the gray scale value of the partition.

[0017] In some embodiments, the method further comprises: writing the sum of the gray scale values of the partition in the current row into a memory; reading the sum of the gray scale values of a row before the current row from the memory, wherein the memory performs synchronous read and write operations.

[0018] Embodiments of the present application also provide a display device. The display device comprises: a gray scale image acquisition unit configured to acquire a gray scale image of an image of the display device; a first compensation data generation unit configured to perform first compensation calculation on the gray scale image in a time domain to generate first compensation data; a second compensation data generation unit configured to perform second compensation calculation on the gray scale image in a spatial domain to generate second compensation data; and a compensation unit configured to adjust the image according to the first compensation data and the second compensation data.

[0019] In some embodiments, the display device further comprises a memory configured to perform at least one of the following: store data in the first compensation calculation process and the first compensation data; store data in the second compensation calculation process and the second compensation data; and store data in the process of acquiring the gray scale image.

[0020] In some embodiments, the first compensation data generation unit comprises a look-up table for determining a time domain weight coefficient of each frame image data in a plurality of historical frame image data before the current frame image data.

[0021] Embodiments of the present application provide an electronic device. The electronic device comprises the display device as described above.

[0022] Embodiments of the present application also provide a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments will be briefly described below, and it should be known that the drawings described below only relate to some embodiments of the present application, but not limit the present application, wherein:

[0024] FIG. 1(a) is a schematic diagram of a display device;

[0025] FIG. 1(b) is a schematic diagram of a display effect of the display device of FIG. 1(a);

[0026] FIG. 2 is a flowchart of an image compensation method of a display panel according to an embodiment of the present application;

[0027] FIG. 3 is a flowchart of a method of determining first compensation data, according to some embodiments of the present application;

[0028] FIG. 4 is a diagram of weight coefficients of different regions of a display device, according to an embodiment of the present application;

[0029] FIG. 5 is a flowchart of a time dimension processing state machine (FSM), according to an embodiment of the present application;

[0030] FIG. 6 is a further flowchart of a method of determining first compensation data, according to an embodiment of the present application;

[0031] FIG. 7 is a logic diagram of history frame data processing, according to an embodiment of the present application;

[0032] FIG. 8 is a flowchart of a method of determining second compensation data, according to an embodiment of the present application;

[0033] FIG. 9 is a logic flowchart of a state machine of spatial dimension processing, according to an embodiment of the present application;

[0034] FIG. 10 is a diagram of region decomposition of a display device, according to an embodiment of the present application;

[0035] FIG. 11 is a diagram of logic timing involving mean value calculation of gray scale values, according to an embodiment of the present application;

[0036] FIG. 12 is a logic flowchart of a state machine (FSM) of a mean value unit, according to an embodiment of the present application;

[0037] FIG. 13 is a flowchart of an image compensation method, according to an embodiment of the present application;

[0038] FIG. 14 is a diagram of a display device, according to an embodiment of the present application;

[0039] FIG. 15 is a diagram of an electronic device, according to an embodiment of the present application;

[0040] FIG. 16 is a diagram of display effects of a display device, according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the described embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of the present application.

[0042] When introducing elements of the present application and embodiments thereof, the articles "a", "an", and "the" are intended to mean that there are one or more of the elements. The terms "comprising", "including", "containing" and "having" are intended to be inclusive and allow for additional elements.

[0043] For the purposes of the present description, the terms "upper", "lower", "left", "right", "vertical", "horizontal", "top", "bottom", and derivatives thereof shall relate to the application as they are oriented in the drawings. The terms "overlying", "on top of", "positioned on" or "positioned on top of" mean that a first element is present on a second element, wherein intervening elements can be present between the first element and the second element. The term "contacting" means connecting one element, such as a first structure, to another, such as a second structure, with or without additional elements at the interface between the two elements.

[0044] Display technology presents the challenge of image sticking. Existing research and compensation methods have not fully solved the problem of image sticking in MLED displays.

[0045] Fig. 1(a) is a schematic diagram of a display device, and Fig. 1(b) is a schematic diagram of the display effect of the display device of Fig. 1(a). In venues such as performance venues, conference rooms, etc., large screens are indispensable equipment. Large screens are usually composed of several smaller screens spliced together. In order to facilitate splicing, each single screen of the spliced screen is fixed by structural members such as metal frames in Fig. 1(a). The structural members for fixation are in contact with the single display screen, which will affect heat dissipation, thereby causing the temperature of different areas of the display screen close to the structural members and far from the structural members to be inconsistent. As shown in Fig. 1(b), the light-emitting efficiency of LED lamps will be affected by temperature, which leads to the "Tic-Tac-Toe" phenomenon on the MLED spliced screen, causing uneven display. The image sticking problem of the "Tic-Tac-Toe" phenomenon on the MLED has affected the overall user experience.

[0046] Embodiments of the present application propose a scheme for real-time compensation of display unevenness (especially the "Tic-Tac-Toe" phenomenon) based on time and space dimensions, provide innovative FPGA implementation of compensation algorithm, combine the influencing factors of time and space dimensions, effectively reduce the "Tic-Tac-Toe" phenomenon, and fill the technical gap.

[0047] Fig. 2 is a flowchart of an image compensation method of a display panel according to an embodiment of the present application. As shown in Fig. 2, the image compensation method of the display panel according to the embodiment of the present application can include the following steps:

[0048] S101, acquire a gray image of an image of a display device;

[0049] S103, perform a first compensation calculation on the gray image in a time domain to determine first compensation data;

[0050] S105, perform a second compensation calculation on the gray image in a spatial domain to determine second compensation data;

[0051] S107, adjust the image according to the first compensation data and the second compensation data.

[0052] Through such a scheme, a scheme for compensating display non-uniformity (especially, the "cross" phenomenon) in real time based on time and space dimensions can be implemented.

[0053] FIG. 3 is a flowchart of a method for determining first compensation data according to some embodiments of the present application. As shown in FIG. 3, the compensation calculation on the gray image in the time domain to determine the first compensation data can include:

[0054] S201, acquire historical frame image data of a first number of frames before current frame image data. Here, "historical frame" refers to a certain number (i.e., the first number of frames) of frames before the current frame in terms of time dimension. In some embodiments, "image data" can include RGB values of a specific frame in a color space. In other embodiments, "image data" can include gray scale values (i.e., brightness) of a specific frame in a gray scale space.

[0055] S203, for each frame in the historical frames, assign a respective weight coefficient according to a lookup table including time domain weight coefficients related to the spatial position of the frame. Considering the display non-uniformity of different display regions of the display device, the lookup table can include multiple sets of weight coefficients, wherein different sets of weight coefficients can be applied to frames in different spatial positions.

[0056] S205, weight-sum the historical frame image data according to the respective weight coefficients to obtain the first compensation data. In some embodiments, specifically, the gray scale data of each frame of the historical frames can be multiplied by the respective weight coefficient, respectively, and then summed up.

[0057] In some embodiments, the closer the receiving time of the historical frames to the receiving time of the current frame, the greater the corresponding weight of the historical frames. In some embodiments, the weight coefficient corresponding to the edge region of the display device is greater than the weight coefficient corresponding to the center region of the display device.

[0058] In some embodiments, the compensation method of the image further comprises employing a memory to store data in the weighted summation process and store the first compensation data in real time. In some embodiments, the memory can be a RAM memory. In other embodiments, the memory can be a FLASH memory. The following is only exemplary description with the memory as a RAM memory.

[0059] Specifically, in the embodiment, when calculating the history data influence value in the time dimension, the 1800 frames of image data sampled at equal intervals are superimposed according to certain weights to obtain the total influence of the history data, and the superposition of the time window is fixed-pointed, that is, the 1800 frames of image data sampled at equal intervals are multiplied with the time window weight vector at corresponding positions and accumulated, so that the fixed-pointed total image of the history data can be obtained.

[0060] In the above embodiment, the selection of the number of history frames is based on the following reasons: the inventors found that the screen after working for 30 minutes tends to be stable, the image of the display device is set to one frame per second, and 30 minutes corresponds to 1800 frames, so 1800 frames of history data frames are selected for operation. The inventors calculated the coefficients of 1800 time dimension calculations through testing and calculation, established a lookup table (LUT) to save the 1800 frame time window weight vector in the ROM, and stored the intermediate process data of the time domain calculation in the RAM. The gray scale data is multiplied with the corresponding time window weight vector, and the result is written into the RAM. It should be noted that the first number of history frames is not limited to 1800 frames. For example, 1600 frames, 2100 frames, etc. can also be taken as needed.

[0061] FIG. 4 is a schematic diagram of weight coefficients of different regions of a display device according to an embodiment of the present application. Since the heating speed of different regions of a unit screen is different, the accumulated weight of different regions is also different. As shown in FIG. 4, the screen is divided into two different regions, a center region of a unit screen and an edge region. Different weights are used for different regions, and the temperature rise curves corresponding to different region types are processed to obtain corresponding 1800 frame weight coefficients. The weight coefficients of the edge region of the screen and the weight coefficients of the center region are recorded in a lookup table LUT, respectively. According to the weight coefficients, all frames in the cache pool are weighted and accumulated. During the accumulation, different weight coefficients are used for different regions for weighting, and the final history influence map can be obtained.

[0062] FIG. 5 is a flowchart of a time dimension processing state machine (FSM) according to an embodiment of the present application. As shown in FIG. 5, a one-stage time domain calculation state machine can be used to realize the read and write operations of the RAM, and control the intermediate product and accumulation of the time domain calculation.

[0063] The embodiment of the present application uses pipeline operation, simplifies logic, reduces the number of RAMs, and uses multiple LUTs to replace the FPGA logic implementation method of multiplication and division, thereby solving the technical problem of insufficient FPGA resources.

[0064] FIG. 6 is a further flowchart of a method for determining first compensation data according to an embodiment of the present application. As shown in FIG. 6, the compensation calculation in the time domain on the gray-scale image to determine the first compensation data further includes the following steps:

[0065] S601, obtain initialization frame image data with a second frame number, wherein the second frame number is less than the first frame number. The "crosshatch" phenomenon is more obvious at the initial screen lighting, so compensation is performed at the beginning. Through such design of the initialization frame, better compensation effect can be obtained. For the calculation of the initialization frame image data, a similar method to the historical frame image data can be used to perform compensation calculation in the time domain and the spatial domain. Here, the details are not repeated.

[0066] S603, write the supplementary frame data with a third frame number and the initialization frame image data into a memory (for example, a RAM memory or a FLASH memory), wherein the sum of the third frame number and the second frame number is equal to the first frame number. Although the frame number (i.e., the second frame number) of the initialization frame image data is less than the frame number (i.e., the first frame number) of the historical frame image data, in order to make them correspond in time, the supplementary frame data with the third frame number is referenced. Specifically, the supplementary frame can be "0" data, thereby only occupying a place and not participating in actual calculation.

[0067] In some embodiments, specifically, the frame number (i.e., the first frame number) of the historical frame image data can be 1800, the frame number (i.e., the second frame number) of the initialization frame image data can be 60, and the frame number (i.e., the third frame number) of the supplementary frame can be 1740.

[0068] The selection of 1800 frame data is based on the internal framework, and based on the consideration of screen residual stability, a longer time sequence of data is used to obtain more stable and accurate image compensation values. The 60 frame iterative update calculation is mainly based on the external framework, and this part of processing is mainly aimed at the crosshatch residual image phenomenon obvious in the starting process. Through the fast update of a shorter time sequence of 60 frames, effective compensation of the crosshatch residual image in the initial starting process is realized. This method reflects the depth optimization of time sequence data processing in FPGA implementation, so that the MLED display technology obtains significant improvement in residual image compensation.

[0069] Figure 7 is a logic diagram of history frame data processing according to an embodiment of the present application. In some embodiments, 1800 frames of history frame data and 60 frames of initialization frame data are not stored using two RAMs, but are stored using one RAM, thereby saving logic resources of the FPGA chip. As shown in Figure 7, 1800 frames of history frame data, 1740 frames of "0" padding data, and 60 frames of initialization frame data can be stored in one RAM. In another embodiment, 1740 frames of "0" padding data, 60 frames of initialization frame data, and 1800 frames of history frame data can be stored in one RAM.

[0070] In some embodiments, performing compensation calculation on the gray scale image in the spatial domain to determine the second compensation data can include performing compensation calculation to determine the second compensation data by using a bilinear interpolation method. For example, in an embodiment, the single panel is divided into 8x9 partitions (which will be described in more detail later), and data of the partitions can be interpolated into 16x18 and then copied into 160x180. Both steps use a bilinear interpolation formula to perform calculation operations.

[0071] A(1-X)(1-Y)+BX(1-Y)+C(1-X)Y+DXY

[0072] wherein A, B, C, and D are values of four nearest pixel points in a grid of original data before interpolation processing, and X and Y are interpolation weights. A, B, C, and D are located at the upper left corner, the upper right corner, the lower left corner, and the lower right corner of a new pixel point after interpolation processing.

[0073] Figure 8 is a flowchart of determining the second compensation data according to an embodiment of the present application. In some embodiments in which the bilinear interpolation method is used to perform compensation calculation to determine the second compensation data, as shown in Figure 8, determining the second compensation data can include the following steps:

[0074] S801, writing data after first compensation calculation into a first memory;

[0075] S803, outputting first row data in the first data from the first memory in a first time period;

[0076] S805, writing the first row data into a second memory in a second time period;

[0077] S807, outputting second row data from the first memory and outputting the first row data from the second memory in a third time period, wherein the first row and the second row are two adjacent rows.

[0078] In some embodiments, the capacity of the first memory is greater than the capacity of the second memory. In some embodiments, the addresses of the first memory and the addresses of the second memory are column-aligned. In embodiments, the first memory can be one of a RAM memory and a FLASH memory. In some embodiments, the second memory can be one of a RAM memory and a FLASH memory. The following is an exemplary description with the first memory and the second memory being both RAM memories.

[0079] When performing bilinear interpolation, two rows of data need to be accessed simultaneously: the current row and the previous row. This is because the bilinear interpolation algorithm needs to use the pixels in these two rows of data to calculate the value of the new pixel. In order to effectively manage these two rows of data and prevent errors, a ping-pong operation as described above is used. This ensures that when performing bilinear interpolation, we can always access the data of the current row and the previous row. This operation can be implemented by two RAMs. Specifically, a larger-capacity RAM (e.g., the first RAM) and a smaller-capacity RAM (e.g., the second RAM) can be used.

[0080] The first RAM and the second RAM are described in more detail below.

[0081] First RAM (larger-capacity RAM):

[0082] Write logic: When new data arrives, the large RAM stores the entire data at once.

[0083] Read logic: When bilinear interpolation is needed, the large RAM reads out one row of data at a time for the algorithm to use.

[0084] Second RAM (smaller-capacity RAM):

[0085] Write logic: When the large RAM reads out a row of data, the small RAM delays for one beat, i.e., writes this row of data in the next clock cycle. In this way, the small RAM always saves the data read by the large RAM in the previous cycle, i.e., the previous row of data.

[0086] Read logic: When the large RAM reads out a new row of data, the small RAM reads out the old row of data it previously stored, i.e., the previous row of data. In this way, the bilinear interpolation algorithm can obtain the data of the current row and the previous row simultaneously.

[0087] Figure 9 is a state machine logic flow diagram of spatial dimension processing according to an embodiment of the present application. As shown in Figure 9, the reading of data is implemented using a state machine: when reading the first row of data, we process this row of data alone and do not output, because there is no previous row of data to compare at this time. Starting from the second row, we read the data one by one and leave time for the calculation of the bilinear interpolation algorithm until the end of the row is reached. Under certain conditions, if we find that it is a special processing row when the tenth clock cycle is reached, then the interpolation calculation will end directly. If four rows of data have been read, the state machine will wait for the feedback of the following read operation. Only when a high level signal is received, the state machine will continue to the next one-by-one reading cycle; if no high level signal is received, the state machine remains waiting. When reading and writing data, we need to ensure that the read addresses of the large RAM and the small RAM are column-aligned, so that the bilinear interpolation algorithm can accurately obtain the corresponding current row and previous row of data.

[0088] In some embodiments, the compensation method further comprises mean processing the grayscale image, wherein the mean processing comprises the following steps: dividing the pixel array of the display device into a plurality of partitions; and calculating a mean value of the grayscale values of each of the plurality of partitions. For the case of a tiled screen composed of a plurality of unit screens, the unit screens can be divided into a plurality of partitions for better mean processing.

[0089] In some embodiments, calculating the mean value of the grayscale values of each partition comprises: for each row of each partition, calculating the sum of the grayscale values of the pixels of the row; accumulating the sum of each row to obtain an accumulated value; and dividing the accumulated value by the total number of pixels in the each partition to obtain the mean value of the grayscale values of the partition.

[0090] In some embodiments, the sum of the grayscale values of the current row of the partition is written to the RAM, and the sum of the grayscale values of the row before the current row is read from the RAM, wherein the RAM performs synchronous read and write operations. In this way, data reading and writing can be pipelined, reducing the data storage requirements.

[0091] Figure 10 is a schematic diagram of area decomposition of a display device according to an embodiment of the present application. In the embodiment shown in Figure 10, the display device is taken as an example comprising 8 unit screens. The actual number of unit screens is not limited to 8 and can be set as needed. In the embodiment shown in Figure 10, the unit screens comprise 160x180 pixels. In order to reduce the amount of data processing, we divide the unit screens into 72 partitions of 8x9. The number of pixels in each row of each partition is 20, and the number of pixels in each column of each partition is 20. Of course, the partitions can be set to comprise other numbers of pixel arrays as needed. The following is described taking a partition comprising a 20x20 pixel array (i.e., 400 pixels) as an example.

[0092] The mean unit can be used to calculate the average value of the gray scale of each partition, i.e. the average gray scale of each partition area of a frame of gray scale image. For each row of each partition, the sum of the gray scale values of the pixels in the row is calculated, and then the sum of each row is accumulated to obtain an accumulated value, and the accumulated value is divided by the total number of pixels in the partition (for example, 400) to obtain the average value of the gray scale of the partition.

[0093] Fig. 11 is a logic timing diagram related to the calculation of the average value of the gray scale according to an embodiment of the present application. As shown in Fig. 11, the accumulation operation is started when the image signal arrives, and the sum of the gray scale values of the pixels in a row in the partition is obtained. At this time, the sum of each row is accumulated: the calculation starts from the first partition, and the accumulation operation is implemented in the last partition. The sum of the gray scale values of the pixels in the current row of each partition is stored in the 0-7 addresses of the RAM. The address is cleared each time the row is changed, and the address is incremented by 1 when the enable signal is valid. Then, according to the address of the RAM, the operation is performed in a pipeline manner until the last partition is completed. The sum of the gray scale values of the pixels in the current row of each partition is written into the RAM, and the accumulated gray scale value of the row before the current row is read from the RAM. The RAM performs synchronous read and write operations, and the read and write operations are synchronized, i.e. the read and write operations are performed simultaneously. In this way, the data read and write operations can be performed in a pipeline manner, and the values of each row are continuously accumulated. Multiple RAMs can be used, and each RAM is independent and multiplexed.

[0094] Finally, the average value of the gray scale of each partition is obtained: the total number of pixels in a 20x20 size area is 400. 1 / 400 is fixed-pointed, multiplied by 2 14 , and the data after the fixed-pointing is 41820. The fixed-pointed average value is quantized to 8 bits by rounding, and the data is super-range corrected. Since 1 / 400 is a decimal number, it needs to be expanded to a non-decimal number. The inventor found that 2 14 can make the decimal point after the fixed-pointing basically zero. Thus, the division operation can be converted to a multiplication operation: a / b=a x (1 / b). The fixed-pointing algorithm value is not limited, and other suitable values can be selected during the fixed-pointing operation.

[0095] Through the above scheme, the division operation can be replaced by the multiplication operation, the algorithm of the average value is changed to multiplication, the FPGA can be used for special algorithm hardware design, real-time processing can be achieved, and the defects of weak computing power and low computing efficiency of general-purpose processors can be compensated.

[0096] Figure 12 is a finite state machine (FSM) logic flow chart of the mean unit according to an embodiment of the present application. As shown in the finite state machine (FSM) logic flow chart of Figure 12, in an embodiment, a one-segment state machine can be used to implement the read and write operations of the RAM, reading the accumulated values of the first n-1 rows, adding the data of the nth row, and writing the accumulated values of the first n rows.

[0097] Figure 13 is a flow chart of an image compensation method according to an embodiment of the present application. In the embodiment shown in Figure 13, the image compensation method includes the following steps:

[0098] S1301, obtaining an image of a display device;

[0099] S1303, converting the obtained image into a grayscale image. In the embodiment, the RGB data is converted into grayscale value processing. In this step, data format conversion and storage optimization are implemented, effectively reducing the complexity and resource requirements of FPGA processing. This is because the image of the display device is usually an RGB color image. The RGB color image contains three color channels, but the volume of the grayscale image is much smaller. Converting the color image to a grayscale image can effectively reduce the storage space occupied by the image. In addition, grayscale image processing can greatly reduce the amount of calculation, reduce the processing difficulty, and improve the processing speed. Therefore, the RGB data is converted into YCbCr (abbreviated as YCC) grayscale data.

[0100] To convert the RGB color image to a grayscale image, the grayscale value of a pixel point can be obtained according to the RGB value of the pixel point, so as to convert an RGB color image to a grayscale image. Each component in RGB is 8 bits, a total of 3x8=24 bits, and each component of RGB is in the value range [0, 255]. Let the original RGB image data be I RGB , and the calculation formula of the fixed-point grayscale data I Gray is as follows:

[0101] I Gray = A x I R + B x I G + C x I B

[0102] Wherein, the proportional coefficients A, B, and C of R, G, and B are obtained through specific screen field testing.

[0103] The obtained data is restored by rounding and truncation to obtain an 8-bit gray scale image. In addition, if the highest bit is high level 1, it means that the value of the color channel exceeds the range of 8-bit unsigned integer, so it is set to 255. If the highest bit is low level, it is set to the value of the low 8 bits, preserving the original value. The video image data is processed to ensure that the value of the color channel is within the range of 8-bit unsigned integer, avoiding out-of-range situations.

[0104] S1305, mean value processing. The process of mean value processing can be referred to the foregoing, which will not be repeated here.

[0105] S1307, writing / degree gray scale information through DDR.

[0106] S1309, performing first compensation calculation in the time domain on the gray scale image by the time domain calculation unit to generate first compensation data. The calculation and generation process of the first compensation data can be referred to the foregoing, which will not be repeated here.

[0107] S1311, performing cross-clock domain processing on the data output from the time domain calculation unit. Since the DDR has a fixed 200M clock, but the clock of the HDMI is not the same. Through cross-clock domain processing, it can be ensured that the data is crossed from one clock domain (DDR clock domain) to another clock domain (clock domain of the video image signal), and the cross-clock operation is realized by BRAM (Block RAM).

[0108] S1313, performing second compensation calculation in the spatial domain on the gray scale image by the spatial domain calculation unit to generate second compensation data. The calculation and generation process of the second compensation data can be referred to the foregoing, which will not be repeated here. The image compensation is performed by combining the time dimension (such as the historical data of the pixel) and the spatial dimension (such as the spatial information around the pixel). The time series data is used to analyze the pixel change trend, and the compensation strategy of each pixel point is optimized by combining the spatial data processing. This combination allows the system to more accurately analyze and perform dynamic compensation of the image.

[0109] S1315, using the pixel compensation unit, compensating the image to obtain the compensated target frame image. According to the gray scale chart, the maximum compensation value is found. The maximum compensation value corresponding to each gray scale is different. In order to measure the maximum compensation value range of different gray scales, the brightness of different gray scales at the lowest screen temperature and the brightness of different gray scales at the highest screen temperature need to be tested, and the difference between the gray scale value at the lowest temperature and the gray scale value at the highest temperature at the same brightness is taken as the maximum compensation value of the gray scale value at the highest temperature. Then the compensation value is calculated. As shown in the figure, the compensation value can be obtained by proportionally accumulating the two branches of the temperature difference compensation coefficient and the cross-shaped compensation coefficient (see the time domain calculation unit and the spatial domain calculation unit), and then multiplying the maximum compensation value chart and the maximum brightness coefficient. After obtaining the compensation value chart, the original RGB image can be compensated. The compensation chart is obtained by subtracting the compensation value chart from the original RGB image.

[0110] S1317, outputting the compensated image to the display device for display.

[0111] FIG. 14 is a schematic diagram of a display device according to an embodiment of the present application. As shown in FIG. 14, the display device 1400 according to an embodiment of the present application can include a gray scale image acquisition unit 1401, a first compensation data generation unit 1402, a second compensation data generation unit 1403, and a compensation unit 1404. The gray scale image acquisition unit 1401 is configured to acquire a gray scale image of an image of the display device, the first compensation data generation unit 1402 is configured to perform a first compensation calculation in a time domain on the gray scale image to generate first compensation data, the second compensation data generation unit 1403 is configured to perform a second compensation calculation in a spatial domain on the gray scale image to generate second compensation data, and the compensation unit 1404 is configured to adjust the image according to the first compensation data and the second compensation data.

[0112] In some embodiments, the display device further includes a RAM memory. The RAM memory is configured to at least one of: store data and the first compensation data in the first compensation calculation process; store data and the second compensation data in the second compensation calculation process; and store data in the process of acquiring the gray scale image.

[0113] In some embodiments, the first compensation data generation unit includes a lookup table. The lookup table is used to determine the spatial-related time domain weight coefficient of each frame image data in the plurality of historical frame image data before the current frame image data.

[0114] FIG. 15 is a schematic diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 15, the electronic device 1500 according to an embodiment of the present application can include a display device 1501. The display device 1501 can be the display device shown in FIG. 14.

[0115] Fig. 16 is a schematic diagram of display effects of a display device according to an embodiment of the present application. The left part of Fig. 16 is a display effect when the display device is just powered on, and the right part of Fig. 16 is a display effect after the display device is powered on for 30 minutes. As shown in Fig. 16, for the display device according to the embodiment of the present application, the obvious "crosshatch" image retention phenomenon at the start has been eliminated, and the entire screen display shows consistent brightness and chromaticity. The continuous lighting for 30 minutes also shows no obvious crosshatch image retention, significantly improving the visual experience of the user.

[0116] The present application introduces in detail a compensation method of a "crosshatch" from the perspective of FPGA implementation, including FPGA implementation methods in the time dimension (i.e., time domain) and the space dimension (i.e., spatial domain). The biggest difference between FPGA implementation and algorithm implementation is that the hardware resources of FPGA are limited. Compared with algorithm implementation, FPGA implementation faces more difficulties. The embodiment of the present application adopts pipelining operation, LUT and reduction of RAM usage to optimize logic resources. By pipelining the data, the storage requirement of the data is reduced, the LUT lookup table is used to replace the multiplication and division operation, and the logic space of the FPGA is saved by using "0 supplement" and shared RAM. The successful integration of FPGA in MLED display can support the mass production of MLED display, improve the overall picture quality and stability, and improve the picture consistency of the screen of MLED product.

[0117] The compensation method of the embodiment of the present application solves the crosshatch image retention phenomenon of the MLED display spliced screen, and does not need to add additional hardware cost. In addition to this, the deployment of FPGA has wide application in large-scale production of MLED display and related expansion products, and belongs to the industry-leading innovation. In addition to the rectangular frame proposed in this paper, other related structures such as circular arc, X-shaped and triangular structures can use related schemes for algorithm deployment. As for the algorithm deployment of FPGA, some logical innovation schemes are adopted, such as using a RAM and multiple LUT tables to process different frames to solve the problem of excessive FPGA resource consumption. In summary, this method realizes the elimination of the "crosshatch" phenomenon of the MLED COG display screen from the FPGA level, improves the performance of the screen, and promotes the innovation of display technology.

[0118] The embodiment of the present application can realize the innovation of image input data processing: in the embodiment, the RGB data is converted into gray value processing. This step realizes data format conversion and storage optimization, effectively reducing the complexity and resource demand of FPGA processing.

[0119] Embodiments of the present application can realize the combination of time and space dimensions: this scheme combines the time dimension (such as historical data of pixels) and the spatial dimension (such as spatial information around the pixel) to compensate for residual images. The time series data is used to analyze the pixel change trend, and the spatial data is processed to optimize the compensation strategy of each pixel point. This combination allows the system to more accurately analyze and dynamically compensate the image.

[0120] Embodiments of the present application can realize the innovation of time frame processing method: the calculation of 1800 frame weighted average involves the weighted average calculation of 1800 frame image data of the internal frame. The selection of 1800 frame data is based on the consideration of screen residual image stability, and the use of this long time series data to obtain more stable and accurate image compensation value. The 60 frame iterative update calculation is mainly based on the external frame, and this part of the processing is mainly aimed at the obvious cross-shaped residual image phenomenon in the startup process. Through the fast update of the short time series of 60 frames, the effective compensation of the cross-shaped residual image in the early startup is realized. This method reflects the deep optimization of time series data processing in FPGA implementation, which significantly improves the MLED display technology in the aspect of residual image compensation.

[0121] Embodiments of the present application can realize the optimization of FPGA weighted average and iterative update calculation. When processing the frame data of the internal and external frames, RAM is used to realize data control input and processing output, and cross-clock processing is performed. Efficient use of RAM resources is realized, and the data processing speed and accuracy are optimized through cross-clock processing technology.

[0122] In order to obtain pixel compensation values, the scheme adopts heat diffusion filtering technology to compensate in the spatial domain. The application of this technology is an important innovation of FPGA in image processing, which not only improves the accuracy of image compensation, but also improves the processing speed.

[0123] Embodiments of the present application also realize dynamic memory allocation and optimization of RAM usage: in order to optimize the logic resources of FPGA, embodiments of the present application adopt flow operation, LUT lookup table instead of traditional algorithm operation, and shared RAM and zero padding operation to reduce RAM usage. These strategies greatly improve the resource utilization and operation efficiency of FPGA.

[0124] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0125] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit), and the apparatus can be implemented as special purpose logic circuitry.

[0126] Computers suitable for the execution of a computer program include, by way of example, general and / or special purpose microprocessors, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The essential elements of a computer are a central processing unit for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, a portable storage device (e.g., a universal serial bus (USB) flash drive), or a portable computing device, name a few. Computers suitable for the execution of a computer program include, by way of example, general and / or special purpose microprocessors, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The essential elements of a computer are a central processing unit for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, a portable storage device (e.g., a universal serial bus (USB) flash drive), or a portable computing device, name a few.

[0127] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0128] While the specification contains many specifics, these should not be construed as limiting the scope of any invention or of what can be claimed, but as merely providing illustrations of some of the embodiments of the inventions. Certain features that are, for clarity, described above in the context of separate embodiments can also be provided in combination in a single embodiment. Conversely, various features that are, for brevity, described above in the context of a single embodiment can also be provided separately or in any suitable subcombination. In addition, while features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a subcombination or variation of a subcombination.

[0129] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring or implying that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated in a single software product or packaged into multiple software products.

[0130] Accordingly, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0131] The following points need to be explained:

[0132] (1) The drawings of the embodiments of the present application only relate to the structures involved in the embodiments of the present application, and other structures can refer to the structures commonly involved.

[0133] (2) In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other to obtain new embodiments.

[0134] Having described certain specific embodiments, these embodiments are presented by way of example only and are not intended to limit the scope of the present application. Indeed, the novel embodiments described herein can be embodied in various other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein can be made without departing from the spirit of the application. The appended claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the application.

Claims

1. An image compensation method for a display device, characterized in that, The method includes: Obtain a grayscale image of the image from the display device; Perform a first compensation calculation in the time domain on the grayscale image to determine the first compensation data; A second compensation calculation in the spatial domain is performed on the grayscale image to determine the second compensation data; The image is adjusted based on the first compensation data and the second compensation data.

2. The method according to claim 1, wherein, Performing time-domain compensation calculations on the grayscale image to determine the first compensation data includes: Retrieve historical frame image data for the first number of frames preceding the current frame image data; For each frame in the historical frames, a weight coefficient is assigned according to a lookup table that includes time-domain weight coefficients related to the spatial location of the frame. The historical frame image data are weighted and summed according to their respective weight coefficients to obtain the first compensation data.

3. The method according to claim 2, wherein, The closer the reception time of the historical frame is to the reception time of the current frame, the larger the corresponding weight coefficient of the historical frame.

4. The method according to claim 2, wherein, The weighting coefficient corresponding to the edge region of the display device is greater than the weighting coefficient corresponding to the center region of the display device.

5. The method according to claim 2 further includes using a memory to store the data in the weighted summation process and the first compensation data in real time.

6. The method according to claim 5, further comprising performing time-domain compensation calculation on the grayscale image to determine the first compensation data: Obtain initial frame image data with a second frame number, wherein the second frame number is less than the first frame number; Supplementary frame data with a third frame number and the initial frame image data are written into the memory, wherein the sum of the third frame number and the second frame number is equal to the first frame number.

7. The method according to claim 1, wherein, Performing spatial domain compensation calculations on the grayscale image to determine the second compensation data includes: The second compensation data is determined by performing compensation calculations using bilinear interpolation.

8. The method according to claim 7, wherein the compensation calculation is performed using bilinear interpolation to determine the second compensation data, comprising: The data after the first compensation calculation is written into the first memory; During the first time period, the first memory outputs the first row of data from the first data. In the second time period, the first row of data is written into the second memory; During the third time period, the first memory outputs the second line of data, and the second memory outputs the first line of data. The first row and the second row are two adjacent rows.

9. The method according to claim 8, wherein, The capacity of the first memory is greater than the capacity of the second memory.

10. The method according to claim 9, wherein, The addresses of the first memory and the second memory are aligned in the column direction.

11. The method according to claim 2, wherein, The method further includes averaging the grayscale image, wherein the averaging process includes: The pixel array of the display device is divided into multiple partitions; Calculate the average grayscale value of each of the multiple partitions.

12. The method according to claim 11, wherein, Calculating the mean grayscale value for each partition includes: For each row of each partition, calculate the sum of the grayscale values ​​of the pixels in that row; The sum of each row is accumulated to obtain the cumulative value; Divide the accumulated value by the total number of pixels in each partition to obtain the mean grayscale value of the partition.

13. The method of claim 12, further comprising: Write the sum of the grayscale values ​​of the current row in the partition to memory; Read the sum of the grayscale values ​​of the row preceding the current row from the memory, where, The memory performs synchronous read and write operations.

14. A display device, characterized in that, The display device includes: A grayscale image acquisition unit, configured to acquire a grayscale image of an image from the display device; A first compensation data generation unit is configured to perform a first compensation calculation in the time domain on the grayscale image to generate first compensation data. The second compensation data generation unit is configured to perform a second compensation calculation in the spatial domain on the grayscale image to generate second compensation data. A compensation unit configured to adjust the image based on the first compensation data and the second compensation data.

15. The display device of claim 14, further comprising a memory configured to perform at least one of the following: Store the data from the first compensation calculation process and the first compensation data; Store the data from the second compensation calculation process and the second compensation data; Store the data obtained during the process of acquiring the grayscale image.

16. The display device according to claim 14, wherein, The first compensation data generation unit includes a lookup table, which is used to determine the temporal weight coefficients of each frame image data in a plurality of historical frame image data preceding the current frame image data.

17. An electronic device comprising a display device according to any one of claims 14-16.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1-13.

Images