Field color sequential display driving method, device and related equipment with adaptive refresh rate

Abstract

The present application discloses a field color sequential display driving method, apparatus, and related equipment with an adaptive refresh rate. The method comprises: using a trained image classification model to determine target driving information that matches the characteristics of a target image, wherein the image classification model is a first driving information that uses a training image as a training sample to make the average color difference value of the training image less than a preset threshold, or, when there is no driving algorithm that makes the average color difference value of the training image less than the threshold, a second driving information containing the highest refresh rate is trained as a sample label; determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms, and calculating the driving signal of the target image in each field based on the target driving algorithm. The present application dynamically adjusts the refresh rate according to the image content, and determines a suitable driving algorithm according to the refresh rate, which can ensure reasonable fidelity in image display and effectively reduce the power consumption of the display.

Description

Technical Field

[0001] The present application relates to the technical field of field color sequential display, and more specifically, to a field color sequential display driving method, apparatus, and related equipment with an adaptive refresh rate. Background Art

[0002] In recent years, mini-LED, an emerging submillimeter display device, has been widely used in liquid crystal displays (LCDs). This allows LCDs to achieve thickness and contrast comparable to OLEDs while retaining the high brightness and durability of traditional LCDs. These advantages have led to their widespread use in televisions, tablets, laptops, and other applications. However, like traditional LCDs, mini-LED LCDs (LCDs using mini-LED backlighting) suffer from a significant drawback: low light efficiency. Due to their spatial color mixing, the presence of color filters inherent in the LCD structure results in a loss of approximately two-thirds of the light efficiency.

[0003] To address these shortcomings, the color filter of the mini-LED LCD can be removed and color can be formed using temporal color mixing. This means replacing the traditional white-backlit mini-LED array with a mini-LED array that can rapidly flash RGB light. When the RGB light of the mini-LED flashes quickly enough, the human eye's persistence of vision allows for the synthesis of a color image on the retina. This type of display is called a field sequential color liquid crystal display with mini-LED backlight (mini-LED FSC-LCD). Mini-LED FSC-LCD retains the advantages of mini-LED LCD and, thanks to the use of temporal color mixing to replace color filters, boasts three times the resolution, three times the luminous efficiency, and three times the maximum brightness, making it an ideal display for next-generation VR / AR devices.

[0004] Like traditional field sequential color LCDs (FSC-LCDs), mini-LED FSC-LCDs also have the problem of color breakup (CBU). That is, when the human eye is relatively displaced from the image displayed by the FSC-LCD, each field of the FSC-LCD cannot perfectly overlap on the retina, forming RGB color stripes in the edge area. This phenomenon is called color breakup, and harmful color breakup reduces the display effect of the FSC-LCD.

[0005] To address the CBU phenomenon, the most direct approach is to increase the display's refresh rate to above 540Hz. However, this approach is currently difficult to implement due to the slow response speed of FSC-LCD liquid crystals. Therefore, researchers have proposed a series of driving algorithms that change the way each field of a color sequential display is presented, rather than simply increasing the refresh rate to reduce color separation. The 240Hz driving algorithms mainly include: 240Hz-Stencil algorithm, 240Hz-LPD algorithm, 240Hz-Edge-Stencil algorithm, 240Hz-RGB algorithm, etc.; the 180Hz driving algorithms mainly include: 180Hz-Stencil algorithm, 180Hz-LPD algorithm, 180Hz-Edge-Stencil algorithm, 180Hz-RGB algorithm, etc.; and the 120Hz driving algorithms mainly include: 120Hz-Stencil algorithm, 120Hz-LPD algorithm, 240Hz-Edge-Stencil algorithm, 120Hz-RGB, 120Hz-Stencil-LPD algorithm, etc. The above algorithms can all effectively reduce CBU, but they all have a common disadvantage, that is, they are all based on field color sequential display driving algorithms with a fixed refresh rate.

[0006] Fixed refresh rate display drivers have the following disadvantages: For the 240Hz refresh rate driving algorithm, the processed image is basically undistorted, but the high refresh rate will result in higher power consumption; for the 180Hz refresh rate driving algorithm, depending on the image content, some images will produce significant distortion, but due to the lower refresh rate, power consumption will also be lower; for the 120Hz refresh rate driving algorithm, most image content will produce significant distortion, but there will still be a small amount of image content with low distortion rate, but compared with the first two refresh rates, the power consumption at 120Hz refresh rate is the lowest. FSC algorithms with different refresh rates process the same image content with different distortion levels; FSC algorithms with fixed refresh rates also process different image content with different distortion levels. Among them, the 120Hz, 180Hz, and 240Hz mentioned above are based on a 60 frame rate, which is then decomposed into two, three, and four fields. The distortion value will change accordingly for different frame rates.

[0007] Therefore, how to balance power consumption and image fidelity is a topic worthy of study. Summary of the Invention

[0008] In view of this, the present application provides a field color sequential display driving method, apparatus and related equipment with adaptive refresh rate, so as to effectively reduce the display power consumption of the display while ensuring image fidelity.

[0009] To achieve the above objectives, the present application provides, in a first aspect, a field color sequential display driving method with an adaptive refresh rate, comprising:

[0010] Determining target driving information that matches the features of the target image using a trained image classification model, wherein the image classification model is first driving information trained using the training image as a training sample so that the average color difference value of the training image is less than a preset threshold, or, when there is no driving algorithm that makes the average color difference value of the training image less than the preset threshold, using second driving information having a highest refresh rate as a sample label for training, wherein the first driving information includes a minimum refresh rate that makes the average color difference value of the training image less than the preset threshold, or a driving algorithm having the minimum refresh rate;

[0011] Determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms;

[0012] Calculating the simulated backlight distribution and transmittance of the target image in each field based on the target driving algorithm;

[0013] The driving signal of the target image in each field is calculated based on the simulated backlight distribution and transmittance.

[0014] Preferably, the target driving information is a target refresh rate; and the training method of the image classification model includes:

[0015] Get a training image set;

[0016] For each training image in the training image set:

[0017] Applying the candidate driving algorithm to the training image, and calculating the average color difference value of the training image under each candidate driving algorithm, to obtain the average color difference value of the training image under each candidate driving algorithm;

[0018] Determine whether there is an average color difference value less than the preset threshold;

[0019] If so, marking the minimum refresh rate value among the candidate driving algorithms whose average color difference value is less than the preset threshold as the matching refresh rate of the training image;

[0020] If not, marking the highest refresh rate among the candidate driving algorithms as the matching refresh rate of the training image;

[0021] Inputting the training image into the image classification model to obtain a refresh rate corresponding to the training image output by the image classification model;

[0022] The parameters of the image classification model are updated by taking the refresh rate corresponding to the output training image approaching the matching refresh rate of the training image mark as a training target.

[0023] Preferably, the target driving information is a target driving algorithm; the training method of the image classification model includes:

[0024] Get a training image set;

[0025] For each training image in the training image set:

[0026] Applying the candidate driving algorithm to the training image, and calculating the average color difference value of the training image under each candidate driving algorithm to obtain the average color difference value of the training image under each candidate driving algorithm;

[0027] Determine whether there is an average color difference value less than the preset threshold;

[0028] If so, marking the driving algorithm with the smallest refresh rate among the candidate driving algorithms whose average color difference value is less than the preset threshold as the matching driving algorithm for the training image;

[0029] If not, marking the candidate driving algorithm configured with the highest refresh rate as the matching driving algorithm for the training image;

[0030] Inputting the training image into the image classification model to obtain the driving algorithm corresponding to the training image output by the image classification model;

[0031] The parameters of the image classification model are updated with the driving algorithm corresponding to the output training image being close to the matching driving algorithm marked by the training image as a training goal.

[0032] Preferably, the method for calculating the average color difference value of the training image under the candidate driving algorithm includes:

[0033] The average color difference value DR is calculated using the following equation:

[0034]

[0035] Where x represents the number of rows of display pixels and y represents the number of columns of display pixels; The color difference between each pixel in the first image and the second image is calculated by the following equation:

[0036]

[0037] The first image is the original image of the training image, and the second image is the display image obtained by calculating the training image using the candidate driving algorithm;

[0038] ΔL * represents the brightness difference between the first image and the second image, Δa * represents the red-green color difference between the first image and the second image, Δb * Indicates the blue-yellow color difference between the first image and the second image.

[0039] Preferably, the candidate driving algorithms include driving algorithms of the same type but with different refresh rates; and the process of determining a target driving algorithm matching the target driving information from the preset candidate driving algorithms includes:

[0040] A driving algorithm having a refresh rate consistent with the target refresh rate is selected from preset candidate driving algorithms, and the driving algorithm is determined as the target driving algorithm.

[0041] Preferably, the candidate driving algorithms include driving algorithms with different refresh rates under different types of driving algorithms; the process of determining a target driving algorithm matching the target driving information from the preset candidate driving algorithms includes:

[0042] Inputting the target image into a trained second image classification model to obtain a driving algorithm output by the image classification model that matches the target image;

[0043] The second image classification model is trained using the training image as a training sample and the driving algorithm matched with the training image as a sample label;

[0044] Among the driving algorithms output by the second image classification model, a driving algorithm consistent with the target refresh rate is selected, and the driving algorithm is determined as the target driving algorithm.

[0045] Preferably, the candidate driving algorithms include Stencil method, Edge-Stencil method, LPD method, Stencil-LPD method and / or RGB method.

[0046] A second aspect of the present application provides a field color sequential display driving device with an adaptive refresh rate, comprising:

[0047] a driving information determination unit, configured to determine target driving information that matches features of a target image using a trained image classification model, wherein the image classification model is first driving information trained using a training image as a training sample so that an average color difference value of the training image is less than a preset threshold, or, when no driving algorithm exists that causes an average color difference value of the training image to be less than the preset threshold, using second driving information containing a highest refresh rate as a sample label for training, wherein the first driving information includes a minimum refresh rate that causes an average color difference value of the training image to be less than the preset threshold, or a driving algorithm having the minimum refresh rate;

[0048] a driving algorithm determining unit, configured to determine a target driving algorithm that matches the target driving information from preset candidate driving algorithms;

[0049] a backlight compensation calculation unit, configured to calculate, based on the target driving algorithm, a simulated backlight distribution and transmittance of the target image in each field;

[0050] The driving signal determination unit is configured to calculate the driving signal of the target image in each field according to the simulated backlight distribution and transmittance.

[0051] A third aspect of the present application provides a field color sequential display driving device with an adaptive refresh rate, comprising: a memory and a processor;

[0052] The memory is used to store programs;

[0053] The processor is used to execute the program to implement the various steps of the above-mentioned adaptive refresh rate field color sequential display driving method.

[0054] In a fourth aspect, the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the various steps of the field color sequential display driving method with adaptive refresh rate as described above are implemented.

[0055] As can be seen from the above technical solution, the present application first uses a trained image classification model to determine target drive information that matches the characteristics of the target image. The image classification model is trained using the training image as a training sample, using a first drive information that ensures the average color difference value of the training image is less than a preset threshold. Alternatively, when no drive algorithm exists that ensures the average color difference value of the training image is less than the preset threshold, a second drive information with the highest refresh rate is trained as a sample label. The first drive information includes a minimum refresh rate that ensures the average color difference value of the training image is less than the preset threshold, or a drive algorithm with the minimum refresh rate. Then, a target drive algorithm that matches the target drive information is determined from a set of preset candidate drive algorithms. It can be understood that an average color difference value less than the preset threshold ensures that the training image does not produce distortion outside the tolerance range when displayed, ensuring that the image display achieves reasonable fidelity. The minimum refresh rate ensures that the power consumption of the display is minimized. Therefore, the target drive algorithm can achieve good fidelity and low display power consumption for each target image. Next, based on the target drive algorithm, the simulated backlight distribution and transmittance of the target image in each field are calculated. Finally, based on the simulated backlight distribution and transmittance, the drive signals for the target image in each field are calculated, thereby achieving the target image's driven display on the display. This application uses deep learning to dynamically adjust the refresh rate based on image content and determines an appropriate drive algorithm based on the refresh rate, ensuring reasonable fidelity in image display and effectively reducing the display's power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0057] Figure 1 A schematic diagram of a field color sequential display driving method with adaptive refresh rate disclosed in an embodiment of the present application;

[0058] Figure 2 The schematic diagram of the deep residual network model disclosed in the embodiment of this application is illustrated;

[0059] Figure 3 The residual structure diagram disclosed in the embodiment of the present application is illustrated;

[0060] Figure 4 A schematic diagram illustrating a method for determining a matching refresh rate disclosed in an embodiment of the present application;

[0061] Figure 5 A schematic diagram illustrating a method for determining a matching drive algorithm disclosed in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of a field color sequential display driving device with adaptive refresh rate disclosed in an embodiment of the present application;

[0063] Figure 7 Another schematic diagram of the field color sequential display driving device with adaptive refresh rate disclosed in an embodiment of the present application;

[0064] Figure 8 Schematic diagram of a field color sequential display driver device with adaptive refresh rate disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0066] The following describes the field color sequential display driving method with adaptive refresh rate provided by the embodiment of the present application. Figure 1 The field color sequential display driving method with adaptive refresh rate provided in the embodiment of the present application may include the following steps:

[0067] Step S101 : using the trained image classification model, determining target driving information that matches the features of the target image.

[0068] The target image is an image to be displayed in a field color sequential display, which may be a static image or a dynamic image frame in a video.

[0069] The image classification model is a deep learning model, specifically a first driving information that uses a training image as a training sample to make the average color difference value of the training image less than a preset threshold, or, when there is no driving algorithm that makes the average color difference value of the training image less than the preset threshold, a second driving information containing the highest refresh rate is used as a sample label for training.

[0070] The average color difference value refers to the average color difference between each pixel in the original image and the corresponding pixel in the displayed image calculated by the driving algorithm. Generally, this threshold can be set to 2. That is, when the average color difference value of an image is less than 2, it can be considered that the image is not distorted, or the degree of distortion is within an acceptable range.

[0071] The first driving information may include a minimum refresh rate, or a driving algorithm with a minimum refresh rate. For example, assuming that a certain image is processed using the 240Hz-Stencil algorithm, the 180Hz-Stencil algorithm, and the 120Hz-Stencil algorithm, and the average color difference values ​​are 1, 1.5, and 2.5, respectively, then, since the average color difference values ​​driven by the 240Hz-Stencil algorithm and the 180Hz-Stencil algorithm are both less than the preset threshold value 2, the driving algorithm with the minimum refresh rate is screened out, that is, the 180Hz-Stencil algorithm. At this time, if the first driving information is the minimum refresh rate, the first driving information is determined to be 180Hz; if the first driving information is the driving algorithm with the minimum refresh rate, the first driving information is determined to be the Stencil algorithm below 180Hz.

[0072] Similarly, the second driving information may include the highest refresh rate, or a driving algorithm configured with the highest refresh rate. Assuming that an image is processed using the 240Hz-Stencil algorithm, the 180Hz-Stencil algorithm, and the 120Hz-Stencil algorithm, with average color difference values ​​of 2, 2.5, and 3.5, respectively, then, since the average color difference values ​​obtained under each candidate driving algorithm are not less than the preset threshold of 2, the 240Hz or 240Hz-Stencil algorithm is directly determined as the second driving information.

[0073] Step S102 : determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms.

[0074] Specifically, when the target driving information is the minimum refresh rate, assuming that the preset candidate driving algorithm is the Stencil algorithm at various refresh rates, the Stencil algorithm of the corresponding refresh rate is determined as the target driving algorithm based on the minimum refresh rate.

[0075] If the driving information indicates a driving algorithm with a minimum refresh rate, the driving algorithm with the minimum refresh rate can be directly determined as the target driving algorithm. It is understood that the driving algorithm with the minimum refresh rate is always included in the preset candidate driving algorithms. In other words, the driving algorithm used by the image classification model in step S101 during training is included in the preset candidate algorithms.

[0076] Step S103 , calculating and obtaining the simulated backlight distribution and transmittance of the target image in each field based on the target driving algorithm.

[0077] The simulated backlight distribution can be calculated based on the ideal backlight distribution and the light diffusion characteristics of the field color sequential display. The ideal backlight distribution describes the brightness level of the backlight dimming module in the color sequential display and can be calculated based on the target driving algorithm determined in step S102.

[0078] Because dynamic backlighting reduces the brightness of certain areas of the image, resulting in image distortion, grayscale compensation is required. Therefore, after calculating the simulated backlight distribution, the transmittance must be further calculated. Based on this simulated backlight distribution, the transmittance of each field in the LCD can be calculated.

[0079] Step S104 : calculating and obtaining the driving signal of the target image in each field according to the simulated backlight distribution and transmittance.

[0080] The driving signal of a certain field reflects the image display status of the field. In the field color sequential display, the images of each field flash in sequence at a preset frequency to form the original color image of the target image.

[0081] The present embodiment first uses a trained image classification model to determine target drive information that matches the characteristics of a target image. The image classification model is trained using a training image as a training sample, using a first drive information that ensures the average color difference value of the training image is less than a preset threshold. Alternatively, if no drive algorithm exists that ensures the average color difference value of the training image is less than the preset threshold, a second drive information with the highest refresh rate is used as a sample label. The first drive information includes a minimum refresh rate that ensures the average color difference value of the training image is less than the preset threshold, or a drive algorithm with the minimum refresh rate. A target drive algorithm that matches the target drive information is then determined from a set of preset candidate drive algorithms. It is understood that an average color difference value less than the preset threshold ensures that the training image does not produce distortion beyond the tolerance range when displayed, ensuring reasonable image fidelity. The minimum refresh rate minimizes the power consumption of the display. Therefore, the target drive algorithm can achieve good fidelity reproduction of each target image and low display power consumption. Next, the simulated backlight distribution and transmittance of the target image in each field are calculated based on the target drive algorithm. Finally, based on the simulated backlight distribution and transmittance, the drive signals for the target image in each field are calculated, thereby achieving the target image's display drive. This application uses deep learning to dynamically adjust the refresh rate based on image content and determines an appropriate drive algorithm based on this refresh rate, ensuring reasonable fidelity in image display and effectively reducing the display's power consumption.

[0082] In some embodiments of the present application, the image classification model used in step S101 above may be a deep residual network (ResNet). ResNet is a deep network constructed from multiple residual blocks, which effectively solves the degradation problem of deep networks and can train deeper networks. At the same time, batch normalization is used instead of the dropout function to solve the problem of gradient vanishing or gradient exploding.

[0083] Among them, ResNet networks have several structures, including 34 layers, 50 layers, 101 layers, and 152 layers. The following takes the ResNet_50 model as an example to specifically introduce and analyze the structure of the deep residual network model.

[0084] The deep residual network model can be divided into 8 building layers, where one building layer can contain one or more network layers and one or more building blocks (such as ResNet building blocks). Specifically, Figure 2 As shown, the first construction layer 11 includes 1 ordinary convolution layer and a maximum pooling layer; the second construction layer 12 includes 3 residual modules; the third construction layer 13, the fourth construction layer 14 and the fifth construction layer 15 all start with a downsampling residual module, followed by 3 residual modules, 5 residual modules and 2 residual modules respectively; the sixth construction layer 16, the seventh construction layer 17 and the eighth construction layer 18 are composed of an average pooling layer, a fully connected layer and a Softmax layer respectively.

[0085] See also Figure 3 The main branch of the residual structure of ResNet_50 (including the residual module and the downsampled residual module) has three convolutional layers: the first convolutional layer is a 1*1 convolutional layer, which is used to reduce the dimension of the channel; the second convolutional layer is a 3*3 convolutional layer with a stride of 2, which is used to reduce the height and width of the feature matrix to half of the original; the third convolutional layer is a 1*1 convolutional layer, which is used to restore the dimension of the channel. Among them, the downsampled residual module also includes a 1*1 convolutional layer on the shortcut branch. The number of convolution kernels in this convolutional layer is the same as the number of convolution kernels in the third layer on the main branch, so that the output of this convolutional layer and the main branch can be added. In addition, when the sizes of the input feature matrix and the output feature matrix are the same, the ordinary residual module (ID Block) is used; when the sizes of the input feature matrix and the output feature matrix are inconsistent, the downsampled residual module (Conv Block) is used.

[0086] Before use, the image classification model must be trained to output a driving algorithm that matches a single-frame input image. The following describes training methods for image classification models, respectively, for scenarios where the target driving information is a target refresh rate and for scenarios where the target driving information is a target driving algorithm.

[0087] In an optional embodiment, the target driving information mentioned in step S101 is a target refresh rate, and the training method of the image classification model may include:

[0088] S1, obtain the training image set.

[0089] For example, a certain number (e.g., 10,000) of pictures can be randomly obtained from the COCO2017 dataset as training images, and a training set can be constructed using the selected pictures.

[0090] S2, applying the candidate driving algorithms to the training image one by one, and calculating the average color difference value of the training image under each candidate driving algorithm to obtain the average color difference value of the training image under each candidate driving algorithm.

[0091] S3, determine whether there is an average color difference value less than a preset threshold, if so, execute S4; if not, execute S5.

[0092] S4, marking the minimum refresh rate among the candidate driving algorithms whose average color difference value is less than a preset threshold as the matching refresh rate of the training image.

[0093] S5: Mark the candidate driving algorithm configured with the highest refresh rate as the matching driving algorithm for the training image.

[0094] It can be understood that in extreme cases, if for a certain image, the average color difference value obtained after processing it using any candidate driving algorithm is above the preset threshold, according to the rule that the larger the refresh rate, the lower the average color difference value, the maximum refresh rate in the candidate driving algorithm can be determined as the matching refresh rate.

[0095] For example, see Figure 4 Assuming that the candidate driving algorithms include 120Hz FSC algorithm, 180Hz FSC algorithm and 240Hz FSC algorithm, for each input image, one candidate driving algorithm is used to process it one by one, and then the average color difference value of the image before and after processing is determined. If for this image, there is a candidate driving algorithm with an average color difference value less than 2, then the one with the smallest refresh rate is selected as the matching refresh rate; if for this image, there is no candidate driving algorithm with an average color difference value less than 2, then 240Hz is determined as the matching refresh rate.

[0096] After all images in the training image set are labeled, the images in the image training set can be divided into a training set, a validation set, and a test set in a ratio of 8:1:1, so as to train the image classification model later.

[0097] S6: Input the training image into the image classification model to obtain the refresh rate corresponding to the training image output by the image classification model.

[0098] S7, taking the refresh rate corresponding to the output training image approaching the matching refresh rate of the training image mark as a training target, and updating the parameters of the image classification model.

[0099] In some embodiments of the present application, the target driving information mentioned in step S101 is a target driving algorithm. The training method of the image classification model may include:

[0100] S1, obtain the training image set.

[0101] S2, applying the candidate driving algorithms to the training image one by one, and calculating the average color difference value of the training image under each candidate driving algorithm to obtain the average color difference value of the training image under each candidate driving algorithm.

[0102] S3, determine whether there is an average color difference value less than a preset threshold, if so, execute S4; if not, execute S5.

[0103] S4: Mark the driving algorithm with the smallest refresh rate among the candidate driving algorithms whose average color difference value is less than a preset threshold as a matching driving algorithm for the training image.

[0104] S5: Mark the candidate driving algorithm configured with the highest refresh rate as a matching driving algorithm for the training image.

[0105] Similarly, in extreme cases, if for a certain image, the average color difference value obtained after processing it using any candidate driving algorithm is above the preset threshold, according to the rule that the larger the refresh rate, the lower the average color difference value, the maximum refresh rate among the candidate driving algorithms can be determined as the matching refresh rate, and the candidate driving algorithm corresponding to the matching refresh rate can be determined as the matching driving algorithm.

[0106] For example, see Figure 5Assuming that the candidate driving algorithms include the 120Hz FSC algorithm, the 180Hz FSC algorithm, and the 240Hz FSC algorithm, for each input image, one candidate driving algorithm is used to process it one by one, and then the average color difference value of the image before and after processing is determined. If for this image, there is a candidate driving algorithm with an average color difference value less than 2, then the candidate driving algorithm with the smallest refresh rate is selected as the matching refresh rate, and the candidate driving algorithm corresponding to the matching refresh rate is confirmed as the matching driving algorithm; if for this image, there is no candidate driving algorithm with an average color difference value less than 2, then 240Hz FSC is determined as the matching driving algorithm.

[0107] S6: Input the training image into the image classification model to obtain the driving algorithm corresponding to the training image output by the image classification model.

[0108] S7, taking the driving algorithm corresponding to the output training image and the matching driving algorithm marked by the training image as a training target, updating the parameters of the image classification model.

[0109] For example, the prepared data set can be input into the ResNet network structure in batches, and the data set can be iterated 100 times to finally obtain a new set of weight parameters as the final training result. The final trained weight parameters are loaded into the image classification model to obtain the trained image classification model.

[0110] Through the above training process, the image classification model can eventually output driving information that matches the input image. This matching process relies on evaluating the average color difference value of the image under a specific driving algorithm. The larger the color difference value, the greater the distortion rate.

[0111] The CIE 1976 L*a*b* color space, also known as the CIE LAB color space, is a uniform color space recommended by the Commission Internationale de l'Eclairage (CIE). L* represents the lightness coordinate, indicating the brightness of the color; a* and b* represent different hues: a* represents red and green, and b* represents yellow and blue. In some embodiments of the present application, the method for calculating the average color difference value of a training image under a candidate driving algorithm may include:

[0112] The average color difference value DR is calculated using the following equation:

[0113]

[0114] Where x represents the number of rows of display pixels and y represents the number of columns of display pixels; The color difference between each pixel in the first image and the second image is calculated by the following equation:

[0115]

[0116] The first image is the original image of the training image, and the second image is the display image obtained by processing the training image using the candidate driving algorithm; ΔL * Represents the brightness difference between the first image and the second image, Δa * Indicates the red and green difference between the first image and the second image, Δb * Indicates the blue-yellow color difference between the first and second images.

[0117] On the one hand, for the case where the target drive information is a target refresh rate, in some embodiments of the present application, the candidate drive algorithms may include drive algorithms with different refresh rates under the same type of drive algorithm. For example, the candidate drive algorithms include a 120Hz LPD algorithm, a 180Hz LPD algorithm, and a 240Hz LPD algorithm; or, the candidate drive algorithms include a 120Hz Stencil algorithm, a 180Hz Stencil algorithm, and a 240Hz Stencil algorithm; or, the candidate drive algorithms include a 120Hz Edge-Stencil algorithm, a 180Hz Edge-Stencil algorithm, and a 240Hz Edge-Stencil algorithm. Based on this, the process of determining the target drive algorithm that matches the target drive information from the preset candidate drive algorithms in step S102 may include:

[0118] A driving algorithm having a refresh rate consistent with a target refresh rate is selected from preset candidate driving algorithms, and the driving algorithm is determined as a target driving algorithm.

[0119] For example, in the case where the candidate driving algorithms include a 120 Hz LPD algorithm, a 180 Hz LPD algorithm, and a 240 Hz LPD algorithm, if the target refresh rate is 180 Hz, the 180 Hz LPD is determined as the target driving algorithm.

[0120] On the other hand, for the case where the target drive information is a target refresh rate, in some embodiments of the present application, the candidate drive algorithms may include drive algorithms for different refresh rates under different types of drive algorithms. For example, the candidate drive algorithms include the 240Hz Edge algorithm, 240Hz GPDK algorithm, 240Hz Global Stencil-FSC algorithm, 240HzLPDK algorithm, 240Hz RGBK algorithm and 240Hz Stencil-FSC algorithm under 240Hz, the 180Hz LPD algorithm, 180HzRGB algorithm, 180Hz Stencil algorithm and 180Hz Edge-Stencil algorithm under 180Hz, and the 120Hz LPD algorithm, 120Hz RGB algorithm, 120Hz Stencil algorithm, 120Hz Edge-Stencil algorithm and 120Hz Stencil-LPD algorithm under 120Hz. Based on this, the process of determining the target drive algorithm that matches the target drive information from the preset candidate drive algorithms in step S102 may include:

[0121] S1, inputting the target image into the trained second image classification model to obtain a driving algorithm output by the image classification model that matches the target image.

[0122] The second image classification model is trained using training images as training samples and driving algorithms matched with the training images as sample labels.

[0123] S2. Select a driving algorithm consistent with the target refresh rate from among the driving algorithms output by the second image classification model, and determine the driving algorithm as the target driving algorithm.

[0124] By expanding the range of candidate driving algorithms to color sequential driving algorithms for each field at various refresh rates and using the trained second image classification model to determine the target driving algorithm, image driving with better color separation and lower average color difference value can be achieved.

[0125] In order to reduce computational complexity, in an optional implementation, a discrete Fourier transform (DFT) and a Gaussian low-pass filter (GLPF) may be used to simulate the actual backlight intensity distribution, thereby calculating the simulated backlight distribution.

[0126] Specifically, first, based on the ideal backlight distribution of the entire single-frame image area calculated in step S202, combined with the light diffusion characteristics of the color sequential display, the simulated backlight distribution of the entire single-frame image area in each field is calculated respectively.

[0127] In some embodiments of the present application, the process of calculating the simulated backlight distribution in step S103 may include: calculating the light diffusion function generated by all LEDs on the liquid crystal panel under an ideal backlight distribution; or measuring the light diffusion intensity of all LEDs on the liquid crystal panel under an ideal backlight distribution. The simplified formula of the light diffusion function may be:

[0128]

[0129] Where D(u,v) is the distance from the Fourier transform origin, D0 is the cutoff frequency, and (u,v) represents the position in the frequency domain. D0 is directly related to backlight spread; smaller D0 values ​​allow for lower frequency content and result in a more blurred backlight image. Therefore, by controlling D0, we can simulate the backlight intensity distribution under any point spread function.

[0130] Because dynamic backlight technology reduces the brightness of certain areas of the image, resulting in image distortion, grayscale compensation is required. Therefore, after calculating the simulated backlight distribution, the transmittance needs to be further calculated.

[0131] Specifically, the transmittance of each field in the liquid crystal display is calculated based on the simulated backlight distribution. The transmittance calculation formula can be:

[0132]

[0133]

[0134] in, and I i Indicates image brightness; and BL i Indicates the intensity of the traditional full-on backlight and blurred backlight image when using local color backlight dimming technology. Then, by taking T for each LCD pixel R 、T G and T B The minimum transmittance value is used to calculate T min , to generate the first field of liquid crystal signal. The new liquid crystal signal T' of R, G and B fields R , T' G and T' B Determined by formula (5).

[0135] The following describes an adaptive refresh rate field color sequential display driving device provided in an embodiment of the present application. The adaptive refresh rate field color sequential display driving device described below and the adaptive refresh rate field color sequential display driving method described above can be referenced to each other.

[0136] See Figure 6 The field color sequential display driving device with adaptive refresh rate provided in the embodiment of the present application may include:

[0137] A driving information determination unit 21 is configured to determine target driving information that matches features of a target image using a trained image classification model, wherein the image classification model is first driving information that uses a training image as a training sample so that an average color difference value of the training image is less than a preset threshold, or, when a driving algorithm that causes an average color difference value of the training image to be less than the preset threshold does not exist, is trained using second driving information having a highest refresh rate as a sample label, wherein the first driving information includes a minimum refresh rate that causes an average color difference value of the training image to be less than the preset threshold, or a driving algorithm having the minimum refresh rate;

[0138] a driving algorithm determining unit 22, configured to determine a target driving algorithm that matches the target driving information from preset candidate driving algorithms;

[0139] a backlight compensation calculation unit 23, configured to calculate the simulated backlight distribution and transmittance of the target image in each field based on the target driving algorithm;

[0140] The driving signal determination unit 24 is configured to calculate the driving signal of the target image in each field according to the simulated backlight distribution and transmittance.

[0141] In some embodiments of this application, see Figure 7 The above-mentioned adaptive refresh rate field color sequential display driving device further includes a model training unit 25, the target driving information is a target refresh rate, and the process of the model training unit 25 training the image classification model may include:

[0142] Get a training image set;

[0143] Applying the candidate driving algorithm to a training image, calculating an average color difference value of the training image under each candidate driving algorithm, and marking a minimum refresh rate value among the candidate driving algorithms whose average color difference value is less than a preset threshold as a matching refresh rate of the training image;

[0144] Inputting the training image into the image classification model to obtain a refresh rate corresponding to the training image output by the image classification model;

[0145] The parameters of the image classification model are updated by taking the refresh rate corresponding to the output training image approaching the matching refresh rate of the training image mark as a training target.

[0146] In some embodiments of the present application, the target driving information is a target driving algorithm, and the process of the model training unit 25 training the image classification model may include:

[0147] Get a training image set;

[0148] Applying the candidate driving algorithms to a training image, calculating an average color difference value of the training image under each candidate driving algorithm, and marking a driving algorithm with a minimum refresh rate among the candidate driving algorithms whose average color difference value is less than a preset threshold as a matching driving algorithm for the training image;

[0149] Inputting the training image into the image classification model to obtain the driving algorithm corresponding to the training image output by the image classification model;

[0150] The parameters of the image classification model are updated with the driving algorithm corresponding to the output training image being close to the matching driving algorithm marked by the training image as a training goal.

[0151] In some embodiments of the present application, the process of the model training unit 25 calculating the average color difference value of the training image under the candidate driving algorithm may include:

[0152] The average color difference value DR is calculated using the following equation:

[0153]

[0154] Where x represents the number of rows of display pixels and y represents the number of columns of display pixels; The color difference between each pixel in the first image and the second image is calculated by the following equation:

[0155]

[0156] The first image is the original image of the training image, and the second image is the display image obtained by calculating the training image using the candidate driving algorithm;

[0157] ΔL * represents the brightness difference between the first image and the second image, Δa * represents the red-green color difference between the first image and the second image, Δb * Indicates the blue-yellow color difference between the first image and the second image.

[0158] In some embodiments of the present application, the candidate driving algorithms include driving algorithms of the same type but with different refresh rates. The process of the driving algorithm determination unit 22 determining a target driving algorithm that matches the target driving information from the preset candidate driving algorithms may include:

[0159] A driving algorithm having a refresh rate consistent with the target refresh rate is selected from preset candidate driving algorithms, and the driving algorithm is determined as the target driving algorithm.

[0160] In some embodiments of the present application, the candidate driving algorithms include driving algorithms with different refresh rates under different types of driving algorithms; the process of the driving algorithm determination unit 22 determining the target driving algorithm matching the target driving information from the preset candidate driving algorithms may include:

[0161] Inputting the target image into a trained second image classification model to obtain a driving algorithm output by the image classification model that matches the target image;

[0162] The second image classification model is trained using the training image as a training sample and the driving algorithm matched with the training image as a sample label;

[0163] Among the driving algorithms output by the second image classification model, a driving algorithm consistent with the target refresh rate is selected, and the driving algorithm is determined as the target driving algorithm.

[0164] The adaptive refresh rate field color sequential display driving device provided in the embodiment of the present application can be applied to adaptive refresh rate field color sequential display driving devices, such as computers. Optionally, Figure 8 The hardware structure diagram of the field color sequence display driver device with adaptive refresh rate is shown. Figure 8 The hardware structure of the adaptive refresh rate field color sequential display driving device may include: at least one processor 31, at least one communication interface 32, at least one memory 33 and at least one communication bus 34.

[0165] In the embodiment of the present application, the number of the processor 31, the communication interface 32, the memory 33, and the communication bus 34 is at least one, and the processor 31, the communication interface 32, and the memory 33 communicate with each other through the communication bus 34;

[0166] The processor 31 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0167] The memory 33 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0168] The memory 33 stores a program, and the processor 31 can call the program stored in the memory 33, wherein the program is used to:

[0169] Determining target driving information that matches the features of the target image using a trained image classification model, wherein the image classification model is first driving information trained using the training image as a training sample so that the average color difference value of the training image is less than a preset threshold, or, when there is no driving algorithm that makes the average color difference value of the training image less than the preset threshold, using second driving information having a highest refresh rate as a sample label for training, wherein the first driving information includes a minimum refresh rate that makes the average color difference value of the training image less than the preset threshold, or a driving algorithm having the minimum refresh rate;

[0170] Determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms;

[0171] Calculating the simulated backlight distribution and transmittance of the target image in each field based on the target driving algorithm;

[0172] The driving signal of the target image in each field is calculated based on the simulated backlight distribution and transmittance.

[0173] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0174] An embodiment of the present application further provides a storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0175] Determining target driving information that matches the features of the target image using a trained image classification model, wherein the image classification model is first driving information trained using the training image as a training sample so that the average color difference value of the training image is less than a preset threshold, or, when there is no driving algorithm that makes the average color difference value of the training image less than the preset threshold, using second driving information having a highest refresh rate as a sample label for training, wherein the first driving information includes a minimum refresh rate that makes the average color difference value of the training image less than the preset threshold, or a driving algorithm having the minimum refresh rate;

[0176] Determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms;

[0177] Calculating the simulated backlight distribution and transmittance of the target image in each field based on the target driving algorithm;

[0178] The driving signal of the target image in each field is calculated based on the simulated backlight distribution and transmittance.

[0179] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0180] In summary:

[0181] The present embodiment first uses a trained image classification model to determine target drive information that matches the characteristics of a target image. The image classification model is trained using a training image as a training sample, using a first drive information that ensures the average color difference value of the training image is less than a preset threshold. Alternatively, if no drive algorithm exists that ensures the average color difference value of the training image is less than the preset threshold, a second drive information with the highest refresh rate is used as a sample label. The first drive information includes a minimum refresh rate that ensures the average color difference value of the training image is less than the preset threshold, or a drive algorithm with the minimum refresh rate. A target drive algorithm that matches the target drive information is then determined from a set of preset candidate drive algorithms. It is understood that an average color difference value less than the preset threshold ensures that the training image does not produce distortion beyond the tolerance range when displayed, ensuring reasonable image fidelity. The minimum refresh rate minimizes the power consumption of the display. Therefore, the target drive algorithm can achieve good fidelity reproduction of each target image and low display power consumption. Next, the simulated backlight distribution and transmittance of the target image in each field are calculated based on the target drive algorithm. Finally, based on the simulated backlight distribution and transmittance, the drive signals for the target image in each field are calculated, thereby achieving the target image's display drive. This application uses deep learning to dynamically adjust the refresh rate based on image content and determines an appropriate drive algorithm based on this refresh rate, ensuring reasonable fidelity in image display and effectively reducing the display's power consumption.

[0182] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0183] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0184] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A field color sequential display driving method with adaptive refresh rate, characterized in that: include: Determining target driving information that matches the features of the target image using a trained image classification model, wherein the image classification model is first driving information trained using the training image as a training sample so that the average color difference value of the training image is less than a preset threshold, or, when there is no driving algorithm that makes the average color difference value of the training image less than the preset threshold, using second driving information having a highest refresh rate as a sample label for training, wherein the first driving information includes a minimum refresh rate that makes the average color difference value of the training image less than the preset threshold, or a driving algorithm having the minimum refresh rate; Determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms; Calculating the simulated backlight distribution and transmittance of the target image in each field based on the target driving algorithm; Calculating and obtaining driving signals of the target image in each field according to the simulated backlight distribution and transmittance; If the target driving information is a target refresh rate, the training method of the image classification model includes: Get a training image set; For each training image in the training image set: Applying the candidate driving algorithm to the training image, and calculating the average color difference value of the training image under each candidate driving algorithm to obtain the average color difference value of the training image under each candidate driving algorithm; Determine whether there is an average color difference value less than the preset threshold; If so, marking the minimum refresh rate value among the candidate driving algorithms whose average color difference value is less than the preset threshold as the matching refresh rate of the training image; If not, marking the highest refresh rate among the candidate driving algorithms as the matching refresh rate of the training image; Inputting the training image into the image classification model to obtain a refresh rate corresponding to the training image output by the image classification model; Taking the refresh rate corresponding to the output training image approaching the matching refresh rate of the training image mark as a training target, updating the parameters of the image classification model; If the target driving information is a target driving algorithm; the training method of the image classification model includes: Get a training image set; For each training image in the training image set: Applying the candidate driving algorithm to the training image, and calculating the average color difference value of the training image under each candidate driving algorithm to obtain the average color difference value of the training image under each candidate driving algorithm; Determine whether there is an average color difference value less than the preset threshold; If so, marking the driving algorithm with the smallest refresh rate among the candidate driving algorithms whose average color difference value is less than the preset threshold as the matching driving algorithm for the training image; If not, marking the candidate driving algorithm configured with the highest refresh rate as the matching driving algorithm for the training image; Inputting the training image into the image classification model to obtain the driving algorithm corresponding to the training image output by the image classification model; Taking the driving algorithm corresponding to the output training image as close to the matching driving algorithm marked by the training image as a training goal, updating the parameters of the image classification model; The method for calculating the average color difference value of the training image under the candidate driving algorithm includes: The average color difference value DR is calculated using the following equation: Where x represents the number of rows of display pixels and y represents the number of columns of display pixels; The color difference between each pixel in the first image and the second image is calculated by the following equation: The first image is the original image of the training image, and the second image is the display image obtained by calculating the training image using the candidate driving algorithm; ΔL * represents the brightness difference between the first image and the second image, Δa * represents the red-green color difference between the first image and the second image, Δb * Indicates the blue-yellow color difference between the first image and the second image.

2. The method according to claim 1, characterized in that The candidate driving algorithms include driving algorithms of the same type but with different refresh rates; The process of determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms includes: A driving algorithm having a refresh rate consistent with the target refresh rate is selected from preset candidate driving algorithms, and the driving algorithm is determined as the target driving algorithm.

3. The method according to claim 1, characterized in that The candidate driving algorithms include driving algorithms with different refresh rates under different types of driving algorithms; The process of determining a target driving algorithm that matches the target driving information from preset candidate driving algorithms includes: Inputting the target image into a trained second image classification model to obtain a driving algorithm output by the image classification model that matches the target image; The second image classification model is trained using the training image as a training sample and the driving algorithm matched with the training image as a sample label; Among the driving algorithms output by the second image classification model, a driving algorithm consistent with the target refresh rate is selected, and the driving algorithm is determined as the target driving algorithm.

4. The method according to claim 1, wherein The candidate driving algorithms include a Stencil method, an Edge-Stencil method, an LPD method, a Stencil-LPD method and / or an RGB method.

5. A field color sequential display driving device with adaptive refresh rate, characterized in that: include: a driving information determination unit, configured to determine target driving information that matches features of a target image using a trained image classification model, wherein the image classification model is first driving information trained using a training image as a training sample so that an average color difference value of the training image is less than a preset threshold, or, when no driving algorithm exists that causes an average color difference value of the training image to be less than the preset threshold, using second driving information containing a highest refresh rate as a sample label for training, wherein the first driving information includes a minimum refresh rate that causes an average color difference value of the training image to be less than the preset threshold, or a driving algorithm having the minimum refresh rate; a driving algorithm determining unit, configured to determine a target driving algorithm that matches the target driving information from preset candidate driving algorithms; a backlight compensation calculation unit, configured to calculate, based on the target driving algorithm, a simulated backlight distribution and transmittance of the target image in each field; a driving signal determination unit, configured to calculate a driving signal for the target image in each field based on the simulated backlight distribution and transmittance; The target driving information is a target refresh rate; The training method of the image classification model includes: Get a training image set; For each training image in the training image set: Applying the candidate driving algorithm to the training image, and calculating the average color difference value of the training image under each candidate driving algorithm to obtain the average color difference value of the training image under each candidate driving algorithm; Determine whether there is an average color difference value less than the preset threshold; If so, marking the minimum refresh rate value among the candidate driving algorithms whose average color difference value is less than the preset threshold as the matching refresh rate of the training image; If not, marking the highest refresh rate among the candidate driving algorithms as the matching refresh rate of the training image; Inputting the training image into the image classification model to obtain a refresh rate corresponding to the training image output by the image classification model; Taking the refresh rate corresponding to the output training image approaching the matching refresh rate of the training image mark as a training target, updating the parameters of the image classification model; The target driving information is a target driving algorithm; The training method of the image classification model includes: Get a training image set; For each training image in the training image set: Applying the candidate driving algorithm to the training image, and calculating the average color difference value of the training image under each candidate driving algorithm to obtain the average color difference value of the training image under each candidate driving algorithm; Determine whether there is an average color difference value less than the preset threshold; If so, marking the driving algorithm with the smallest refresh rate among the candidate driving algorithms whose average color difference value is less than the preset threshold as the matching driving algorithm for the training image; If not, marking the candidate driving algorithm configured with the highest refresh rate as the matching driving algorithm for the training image; Inputting the training image into the image classification model to obtain the driving algorithm corresponding to the training image output by the image classification model; Taking the driving algorithm corresponding to the output training image as close to the matching driving algorithm marked by the training image as a training goal, updating the parameters of the image classification model; The method for calculating the average color difference value of the training image under the candidate driving algorithm includes: The average color difference value DR is calculated using the following equation: Where x represents the number of rows of display pixels and y represents the number of columns of display pixels; The color difference between each pixel in the first image and the second image is calculated by the following equation: The first image is the original image of the training image, and the second image is the display image obtained by calculating the training image using the candidate driving algorithm; ΔL * represents the brightness difference between the first image and the second image, Δa * represents the red-green color difference between the first image and the second image, Δb * Indicates the blue-yellow color difference between the first image and the second image.

6. A field color sequential display driver device with adaptive refresh rate, characterized in that: include: memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the steps of the field color sequential display driving method with adaptive refresh rate according to any one of claims 1 to 4.

7. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the field color sequential display driving method with adaptive refresh rate according to any one of claims 1 to 4 is implemented.

Images