Regional dimming-oriented artificial intelligence pixel compensation method and system

Through the convolutional neural network combined with multiple pixel compensation algorithms, the images with different characteristics are processed adaptively, which solves the problem of power consumption waste and display distortion of the LCD display when the image tone is dim, and achieves the pixel compensation effect of high contrast and low distortion.

CN118982473AInactive Publication Date: 2024-11-19ANHUI UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411070853.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When the image tone of existing LCD displays is dim, traditional global dimming leads to waste of power consumption, and regional dimming algorithms may cause display distortion, which cannot effectively reduce power consumption and improve contrast.

Method used

Convolutional neural network is used to combine multiple pixel compensation algorithms to adaptively output low-distortion and high-contrast compensation images. By independently creating data sets and building neural network frameworks, the network is trained to process images with different features.

Benefits of technology

The pixel compensation effect that is better than traditional algorithms in contrast, peak signal-to-noise ratio, structural similarity and color difference is achieved, breaking the limitations of specific methods being applicable to specific feature images, and improving the display quality and energy efficiency of the display.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982473A_ABST
    Figure CN118982473A_ABST
Patent Text Reader

Abstract

The invention discloses a regional dimming-oriented artificial intelligence pixel compensation method and system. The method comprises the following steps of S1, acquiring an image data set; s2, constructing an initial pixel compensation network; s3, training the initial pixel compensation network based on the image data set to obtain a pixel compensation network; and S4, performing pixel compensation on a to-be-compensated image by using the pixel compensation network to obtain a compensated image. According to the method, various types of images can be processed, the current situation that a current specific method is suitable for images with specific characteristics is broken through, and the method is superior to a traditional algorithm in indexes such as contrast, peak signal-to-noise ratio, structural similarity and chromatic aberration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of pixel compensation, and in particular relates to an artificial intelligence pixel compensation method and system for regional dimming. Background Art

[0002] With the development of the times, people have higher and higher requirements for the display effect of monitors. Liquid Crystal Display is the current mainstream display device. It is mainly composed of LED backlight, liquid crystal panel and drive circuit, which makes full use of the characteristics of liquid crystal showing different forms under different voltages. Liquid crystal itself does not emit light, so it needs to be illuminated by LED backlight. The liquid crystals that make up LCD have three colors: red, green and blue. They are arranged in a certain order, called three primary colors. These three colors can present a myriad of colors when combined in different proportions. These liquid crystals are changed by voltage, and then under the illumination of LED backlight, different voltage changes can produce three primary colors of different intensities, and different intensities of three primary colors can be mixed to present different colors. The main energy consumption of LCD lies in the backlight module.

[0003] At present, the mainstream of traditional LCD monitors is global dimming, that is, no matter what type of image is displayed, the backlight module of the monitor always keeps each lamp bead fully lit. However, when the tone of the displayed image is relatively dim or the tone of some areas in the displayed image is dim, too high backlight brightness will not only lead to poor display quality, such as making the part that should be black appear gray, but more importantly, it will also lead to waste of power consumption. The regional dimming algorithm can dynamically adjust the brightness of the backlight source according to the content of the image to be displayed, but this will cause distortion of the displayed image, so a pixel compensation algorithm is also needed to compensate for the lost pixel value, so as to achieve the effect of reducing power consumption and improving contrast.

[0004] Currently, common pixel compensation algorithms include linear compensation algorithm, non-linear compensation algorithm, logarithmic function compensation algorithm (LFC), etc. For images with different characteristics, suitable pixel compensation algorithms may be different. When an image has multiple features, only one set of regional dimming algorithm may not achieve good results. The present invention utilizes a convolutional neural network and combines the compensation results of multiple pixel compensation algorithms, so that the network can adaptively output low-distortion, high-contrast compensated images. Summary of the invention

[0005] The present invention aims to address the deficiencies of the prior art and proposes an artificial intelligence pixel compensation method and system for regional dimming. First, a data set is independently created, then a neural network framework is established, the screened data set is put into the network for training, and finally the network is enabled to adaptively output a pixel compensated image.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] An artificial intelligence pixel compensation method for local dimming includes the following steps:

[0008] S1. Obtain image dataset;

[0009] S2. Construct an initial pixel compensation network;

[0010] S3. Training the initial pixel compensation network based on the image data set to obtain a pixel compensation network;

[0011] S4. Perform pixel compensation on the image to be compensated using the pixel compensation network to obtain a compensated image.

[0012] Preferably, the method for obtaining the image data set includes:

[0013] S1.1. Select several sample images;

[0014] S1.2. Perform pixel compensation processing on the same sample image using different compensation algorithms to obtain several compensated images of the sample image;

[0015] S1.3. Obtain the index data of several compensated images, perform weighted summation on the index data, and select the compensated image with the largest sum value as the label image of the sample image, wherein the index data include: contrast, color difference, peak signal-to-noise ratio and structural similarity;

[0016] S1.4. Perform the operations of S1.2-S1.3 on different sample images respectively, and integrate the label images corresponding to the sample images to obtain the image data set.

[0017] Preferably, the initial pixel compensation network comprises: a down-sampling unit and an up-sampling unit;

[0018] The downsampling unit is composed of 6 convolution blocks;

[0019] The upsampling unit is composed of 8 transposed convolution blocks.

[0020] Preferably, the training method comprises:

[0021] S3.1. Performing data enhancement on the image data set, wherein the data enhancement method is horizontal flipping, so that the data volume of the image data set is doubled to obtain image data after data enhancement;

[0022] S3.2. Initialize the parameters of the initial pixel compensation network using Xavier;

[0023] S3.3. Set the hyperparameters of the network, use Adam as the optimizer, set the initial learning rate of the initial pixel compensation network, adjust the learning rate strategy based on cosine annealing, and then set the loss function of the network;

[0024] S3.4. Use the data-enhanced image data to train the initialized initial pixel compensation network to obtain the pixel compensation network.

[0025] Preferably, the pixel compensation method includes:

[0026] S4.1. Acquire the image to be compensated;

[0027] S4.2. Input the image to be compensated into the pixel compensation network, obtain a deep feature map by downsampling, restore the deep feature map to the size of the original image by upsampling, and while upsampling, splice the up- and down-sampled feature maps in the dimension of the number of channels through a jump connection structure to obtain the compensated image.

[0028] The present invention also provides an artificial intelligence pixel compensation system for regional dimming, wherein the compensation system applies any of the compensation methods described above, and comprises: a data set acquisition module, a network construction module, a network training module and a compensation module;

[0029] The data set acquisition module is used to acquire an image data set;

[0030] The network construction module is used to construct an initial pixel compensation network;

[0031] The network training module is used to train the initial pixel compensation network based on the image data set to obtain a pixel compensation network;

[0032] The compensation module uses the pixel compensation network to perform pixel compensation on the image to be compensated to obtain a compensated image.

[0033] Preferably, the data set acquisition module includes: a sample selection unit, a first compensation unit, a label image acquisition unit and an integration unit;

[0034] The sample selection unit is used to select a number of sample images;

[0035] The first compensation unit uses different compensation algorithms to perform pixel compensation processing on the same sample image to obtain a plurality of compensated images of the sample image;

[0036] The label image acquisition unit is used to acquire index data of several compensated images, perform weighted summation on the index data, and select the compensated image with the largest sum value as the label image of the sample image, wherein the index data includes: contrast, color difference, peak signal-to-noise ratio and structural similarity;

[0037] The integration unit is used to integrate the label images corresponding to the sample images to obtain the image data set.

[0038] Preferably, the initial pixel compensation network comprises: a down-sampling unit and an up-sampling unit;

[0039] The downsampling unit is composed of 6 convolution blocks;

[0040] The upsampling unit is composed of 8 transposed convolution blocks.

[0041] Preferably, the network training module includes: a data enhancement unit, an initialization unit, a hyperparameter setting unit and a training unit;

[0042] The data enhancement unit is used to perform data enhancement on the image data set, and the data enhancement method is horizontal flipping to double the data volume of the image data set to obtain data-enhanced image data;

[0043] The initialization unit is used to initialize the parameters of the initial pixel compensation network using Xavier;

[0044] The parameter setting unit is used to set the hyperparameters of the network, using Adam as the optimizer, setting the initial learning rate of the initial pixel compensation network, adjusting the learning rate strategy based on the cosine annealing, and then setting the loss function of the network;

[0045] The training unit trains the initialized initial pixel compensation network using the data-enhanced image data to obtain the pixel compensation network.

[0046] Preferably, the compensation module comprises: an image acquisition unit and a second compensation unit;

[0047] The image acquisition unit is used to acquire the image to be compensated;

[0048] The second compensation unit is used to input the image to be compensated into the pixel compensation network, obtain a deep feature map by downsampling, restore the deep feature map to the size of the original image by upsampling, and while upsampling, splice the up and down sampled feature maps in the dimension of the number of channels through a jump connection structure to obtain the compensated image.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention applies convolutional neural networks in deep learning to pixel compensation. By independently creating data sets and constructing a neural network framework, the network learns what kind of compensated image should be output for input images with different features, thereby realizing an adaptive image pixel compensation that can process various types of images, breaking the current status quo of specific methods being applicable to images with specific features, and outperforming traditional algorithms in terms of indicators such as contrast, peak signal-to-noise ratio, structural similarity and color difference. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0052] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of the initial pixel compensation network structure according to an embodiment of the present invention;

[0054] Figure 3 Schematic diagram of the system structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Embodiment 1

[0058] In this embodiment, if Figure 1As shown, an artificial intelligence pixel compensation method for local dimming includes the following steps:

[0059] S1. Obtain image dataset.

[0060] The method for obtaining an image data set includes: S1.1. selecting a plurality of sample images; S1.2. using different compensation algorithms to perform pixel compensation processing on the same sample image to obtain a plurality of compensated images of the sample image; S1.3. obtaining index data of the plurality of compensated images, and performing weighted summation on the index data, and selecting a compensated image with the largest sum value as a label image of the sample image, wherein the index data include: contrast, color difference, peak signal-to-noise ratio and structural similarity; in this embodiment, the weighted sum formula is as follows:

[0061] score=(CR_cp-CR_ori)÷CR_ori+SSIM+PSNR-20×CD

[0062] Among them, CR_cp represents the contrast of the compensated image, CR_ori represents the contrast of the original image, SSIM represents the structural similarity, PSNR represents the peak signal-to-noise ratio, and CD represents the color difference; S1.4. Perform S1.2-S1.3 operations on different sample images respectively, and integrate the label images corresponding to the sample images to obtain the image data set.

[0063] S2. Construct an initial pixel compensation network.

[0064] The initial pixel compensation network includes: a downsampling unit and an upsampling unit; the downsampling unit is composed of 6 convolution blocks, each of which contains a convolution layer and an activation function layer; the upsampling unit is composed of 8 transposed convolution blocks, each of which contains a transposed convolution layer and an activation function layer.

[0065] In this embodiment, if Figure 2As shown, the deep feature map is obtained by downsampling, so as to learn the deep features of the image. The final result needs to be restored to the size of the original display image, so it is also necessary to combine upsampling. The above adopts the feature map obtained by downsampling while restoring to the display image size, and also combines the feature map obtained by downsampling through jump connection, so as to achieve the effect of accelerating network convergence and avoid the degradation problem caused by the deepening of the network depth. Specifically, among the 6 convolution blocks of the downsampling unit, the first convolution block is composed of a 5×5 convolution kernel and a ReLU activation function. This embodiment uses a three-channel color image for training, so the number of input and output channels of the convolution layer is (3,16), the step size is 2, and the convolution blocks are composed of 3×3 convolution kernels and activation functions. The number of input and output channels of the following 5 convolution blocks are (16,32), (32,64), (64,128), (128,256) and (256,512), respectively, and the step size is 2. The upsampling unit consists of 8 transposed convolution blocks, of which the first 5 transposed convolution blocks are composed of transposed convolution layers and ReLU activation functions, with the input and output channel numbers being (512, 256), (512, 128), (256, 64), (128, 32), and (64, 16) respectively. The convolution kernel sizes are 4×4, 4×4, 3×4, 4×4, and 3×3, with a step size of 2. Except for the padding of the last block, the padding of the remaining blocks is 0. In this way, the upsampling ensures that the image can be enlarged while being able to jump-connect with the downsampled feature map. The last three transposed convolution blocks are composed of convolution kernel sizes of 3×3, 6×6, and 7×7, with step sizes of 2, 1, and 1 respectively. Except for the padding of the first convolution layer, the padding of the other two blocks is 0, so that the image size is restored to 1080×1920.

[0066] S3. Train an initial pixel compensation network based on the image data set to obtain a pixel compensation network.

[0067] The training method includes: S3.1. Performing data enhancement on the image data set by horizontal flipping to double the data volume of the image data set and obtain image data after data enhancement; S3.2. Initializing the parameters of the initial pixel compensation network using Xavier; S3.3. Setting the hyperparameters of the network, using Adam as the optimizer, setting the initial learning rate of the initial pixel compensation network, adjusting the learning rate based on the cosine annealing strategy, and then setting the loss function of the network; S3.4. Using the image data after data enhancement to train the initial pixel compensation network and obtain the pixel compensation network.

[0068] In this embodiment, the Xavier initialization method is used to initialize the parameters of the initial pixel compensation network, which can effectively alleviate the problem of gradient disappearance or explosion. The number of training sets in the image data set of this embodiment is 1100, and after data enhancement, it is 2200. The data enhancement method is horizontal flipping, and the test set is 182. Adam is used as the optimizer, and the initial learning rate is 10e -3 , and combined with the CosineAnnealingLR learning rate adjustment strategy, T_max is set to 10, so that the learning rate will be dynamically adjusted in the form of a cosine function to help the model converge better. The loss function uses MSELoss and perception LPIPS loss, the weight of the MSELoss function is 0.7, and the weight of the perception LPIPSloss function is 0.3. The pixel compensation network is trained using the image data after data enhancement to obtain the pixel compensation network.

[0069] S4. Perform pixel compensation on the image to be compensated using a pixel compensation network to obtain a compensated image.

[0070] The pixel compensation method includes: S4.1. Obtaining an image to be compensated; S4.2. Inputting the image to be compensated into a pixel compensation network, and while upsampling, splicing the up- and down-sampled feature maps in the dimension of the number of channels through a jump connection structure to obtain a compensated image.

[0071] After the image is compensated by the method of this embodiment, the contrast, structural similarity, peak signal-to-noise ratio and color difference are better than those of the traditional algorithm. The results are shown in Table 1:

[0072] Table 1

[0073] Contrast Peak signal-to-noise ratio Structural similarity Chromatic Aberration Nonlinear Compensation Method 188.96 21.24 0.996 0.28 Logarithmic function compensation method 261.74 9.61 0.972 0.32 Linear compensation method 225.20 14.78 0.988 0.75 The present invention 266.6513 23.1294 0.998 0.23

[0074] Embodiment 2

[0075] In this embodiment, if Figure 3 As shown, an artificial intelligence pixel compensation system for regional dimming includes: a data set acquisition module, a network construction module, a network training module and a compensation module.

[0076] The dataset acquisition module is used to acquire image datasets.

[0077] The data set acquisition module includes: a sample selection unit, a first compensation unit, a label image acquisition unit and an integration unit; the sample selection unit is used to select a number of sample images; the first compensation unit uses different compensation algorithms to perform pixel compensation processing on the same sample image to obtain a number of compensated images of the sample image; the label image acquisition unit is used to obtain index data of a number of compensated images, and perform weighted summation on the index data, and select a compensated image with the largest sum value as the label image of the sample image, and the index data include: contrast, color difference, peak signal-to-noise ratio and structural similarity; the integration unit is used to integrate the label images corresponding to the sample images to obtain an image data set.

[0078] The network building module is used to build the initial pixel compensation network.

[0079] The initial pixel compensation network includes: a downsampling unit and an upsampling unit; the downsampling unit is composed of 6 convolution blocks; the upsampling unit is composed of 8 transposed convolution blocks.

[0080] The network training module is used to train an initial pixel compensation network based on an image data set to obtain a pixel compensation network.

[0081] The network training module includes: a data enhancement unit, an initialization unit, a hyperparameter setting unit and a training unit; the data enhancement unit is used to perform data enhancement on the image data set, and the data enhancement method is horizontal flipping, so that the data volume of the image data set is doubled, and the image data after data enhancement is obtained; the initialization unit is used to initialize the parameters of the initial pixel compensation network using Xavier; the parameter setting unit is used to set the hyperparameters of the network, using Adam as the optimizer, setting the initial learning rate of the initial pixel compensation network, adjusting the learning rate strategy based on the cosine annealing, and then setting the loss function of the network; the training unit uses the image data after data enhancement to train the initialized initial pixel compensation network to obtain the pixel compensation network.

[0082] The compensation module uses a pixel compensation network to perform pixel compensation on the image to be compensated to obtain a compensated image.

[0083] The compensation module includes: an image acquisition unit and a second compensation unit; the image acquisition unit is used to acquire the image to be compensated; the second compensation unit is used to input the image to be compensated into the pixel compensation network, obtain a deep feature map by downsampling, and restore the deep feature map to the size of the original image by upsampling to obtain a compensated image.

[0084] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. An artificial intelligence pixel compensation method for regional dimming, It is characterized in that The following steps are involved: S1. Obtain image dataset; S2. Construct an initial pixel compensation network; S3. Training the initial pixel compensation network based on the image data set to obtain a pixel compensation network; S4. Perform pixel compensation on the image to be compensated using the pixel compensation network to obtain a compensated image.

2. According to the artificial intelligence pixel compensation method for regional dimming described in claim 1, It is characterized in that The method for obtaining the image data set includes: S1.

1. Select several sample images; S1.

2. Perform pixel compensation processing on the same sample image using different compensation algorithms to obtain several compensated images of the sample image; S1.

3. Obtain the index data of several compensated images, perform weighted summation on the index data, and select the compensated image with the largest sum value as the label image of the sample image, wherein the index data include: contrast, color difference, peak signal-to-noise ratio and structural similarity; S1.

4. Perform the operations of S1.2-S1.3 on different sample images respectively, and integrate the label images corresponding to the sample images to obtain the image data set.

3. According to the artificial intelligence pixel compensation method for regional dimming described in claim 1, It is characterized in that The initial pixel compensation network includes: a down-sampling unit and an up-sampling unit; The downsampling unit is composed of 6 convolution blocks; The upsampling unit is composed of 8 transposed convolution blocks.

4. According to the artificial intelligence pixel compensation method for local dimming of claim 1, It is characterized in that The training method comprises: S3.

1. Performing data enhancement on the image data set, wherein the data enhancement method is horizontal flipping, so that the data volume of the image data set is doubled to obtain image data after data enhancement; S3.

2. Use Xavier to initialize the parameters of the initial pixel compensation network; S3.

3. Set the hyperparameters of the network, use Adam as the optimizer, set the initial learning rate of the initial pixel compensation network, adjust the learning rate strategy based on cosine annealing, and then set the loss function of the network; S3.

4. Use the data-enhanced image data to train the initialized initial pixel compensation network to obtain the pixel compensation network.

5. According to the artificial intelligence pixel compensation method for regional dimming as claimed in claim 3, It is characterized in that The pixel compensation method comprises: S4.

1. Acquire the image to be compensated; S4.

2. Input the image to be compensated into the pixel compensation network, obtain a deep feature map by downsampling, restore the deep feature map to the size of the original image by upsampling, and while upsampling, splice the up- and down-sampled feature maps in the dimension of the number of channels through a jump connection structure to obtain the compensated image.

6. An artificial intelligence pixel compensation system for regional dimming, the compensation system applying the compensation method according to any one of claims 1 to 5, It is characterized in that include: Dataset acquisition module, network construction module, network training module and compensation module; The data set acquisition module is used to acquire an image data set; The network construction module is used to construct an initial pixel compensation network; The network training module is used to train the initial pixel compensation network based on the image data set to obtain a pixel compensation network; The compensation module uses the pixel compensation network to perform pixel compensation on the image to be compensated to obtain a compensated image.

7. According to claim 6, an artificial intelligence pixel compensation system for local dimming, It is characterized in that The data set acquisition module includes: a sample selection unit, a first compensation unit, a label image acquisition unit and an integration unit; The sample selection unit is used to select a number of sample images; The first compensation unit uses different compensation algorithms to perform pixel compensation processing on the same sample image to obtain a plurality of compensated images of the sample image; The label image acquisition unit is used to acquire index data of several compensated images, perform weighted summation on the index data, and select the compensated image with the largest sum value as the label image of the sample image, wherein the index data includes: contrast, color difference, peak signal-to-noise ratio and structural similarity; The integration unit is used to integrate the label images corresponding to the sample images to obtain the image data set.

8. According to claim 6, an artificial intelligence pixel compensation system for local dimming, It is characterized in that The initial pixel compensation network includes: a down-sampling unit and an up-sampling unit; The downsampling unit is composed of 6 convolution blocks; The upsampling unit is composed of 8 transposed convolution blocks.

9. According to claim 6, an artificial intelligence pixel compensation system for local dimming, It is characterized in that The network training module includes: a data enhancement unit, an initialization unit, a hyperparameter setting unit and a training unit; The data enhancement unit is used to perform data enhancement on the image data set, and the data enhancement method is horizontal flipping to double the data volume of the image data set to obtain data-enhanced image data; The initialization unit is used to initialize the parameters of the initial pixel compensation network using Xavier; The parameter setting unit is used to set the hyperparameters of the network, using Adam as the optimizer, setting the initial learning rate of the initial pixel compensation network, adjusting the learning rate strategy based on the cosine annealing, and then setting the loss function of the network; The training unit trains the initialized initial pixel compensation network using the data-enhanced image data to obtain the pixel compensation network.

10. According to claim 8, an artificial intelligence pixel compensation system for local dimming, It is characterized in that The compensation module comprises: an image acquisition unit and a second compensation unit; The image acquisition unit is used to acquire the image to be compensated; The second compensation unit is used to input the image to be compensated into the pixel compensation network, obtain a deep feature map by downsampling, restore the deep feature map to the size of the original image by upsampling, and while upsampling, splice the up and down sampled feature maps in the dimension of the number of channels through a jump connection structure to obtain the compensated image.

Citation Information

Patent Citations

  • Backlight diffusion method for image processing based on residual network

    CN110838090A

  • Pixel compensation method and device for regional dimming

    CN115312007A

  • Regional backlight dimming method and device based on neural network

    CN115516496A