Liquid crystal display device, method for controlling liquid crystal display device, and program

Abstract

To provide a technique capable of controlling the emission intensity of a backlight module by using a machine learning model trained so that the luminance of a display image in local dimming approaches a target value.SOLUTION: A liquid crystal display device of the present invention comprises: a liquid crystal panel; input means that inputs data of a first image; a backlight module that can change emission luminance for each of a plurality of division areas; generation means that inputs data generated from the first image to a neural network to generate the emission intensity of every division area of the backlight module; calculation means that calculates a correction value for correcting the first image on the basis of the generated emission intensity; correction means that corrects the first image to a second image on the basis of the calculated correction value; light emission control means that controls the emission intensity of every division area of the backlight module on the basis of the generated emission intensity; and display control means that controls the transmittance of the liquid crystal panel on the basis of data of the second image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a liquid crystal display device capable of changing the light emission brightness of a backlight module using a machine learning model, a method for controlling a liquid crystal display device, and a program. [Background technology]

[0002] There is a demand for higher contrast in display devices that display images with a relatively wide dynamic range, such as HDR (High Dynamic Range) images. Typical display devices include OLED (Organic Light Emitting Diode) display devices and LCD (Liquid Crystal Display) devices. In OLED display devices, an organic EL (Electro Luminescence) element emits light for each pixel, whereas in LCD devices, the liquid crystal panel adjusts the amount of light transmitted from the backlight module for each pixel. Since LCD devices cannot completely block the light emitted from the backlight module, light leakage causes black floating. As a result, LCD devices have lower display contrast than OLED display devices, which are self-emitting display devices.

[0003] To reduce black floating and improve contrast in LCD devices, a technology called local dimming is commonly used. Local dimming is a technology that reduces black floating by controlling the light emission brightness of a backlight module for each divided area. Conventional local dimming typically applies specific rules to the backlight module's control values ​​for each divided area to prevent insufficient brightness in small bright areas. However, the specific rules (e.g., spatial filters) applied in local dimming have different effects depending on the image pattern, resulting in excessive or insufficient brightness of the backlight module. As such, local dimming based on specific rules poses challenges in optimizing the brightness of the backlight module.

[0004] One technique for optimizing the brightness of a backlight module is, for example, a method using a machine learning model. Patent Document 1 discloses a technique in which, when a power limit occurs, a specific texture (e.g., glossiness) is detected using a machine learning model, and the brightness of the backlight module outside the detected area is reduced. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-211581 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the method disclosed in Patent Document 1 reduces the brightness of the backlight module outside of areas with a specific texture (e.g., gloss) when power is limited, which limits the areas and timing in which the brightness of the backlight module is optimized.

[0007] Therefore, an object of the present invention is to provide a technology that can control the light emission intensity of a backlight module using a machine learning model that has been trained to bring the brightness of a displayed image by local dimming closer to a target value. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, a first aspect of the present invention is a liquid crystal display device comprising: a liquid crystal panel; an input means for inputting data of a first image; a backlight module that irradiates light onto the liquid crystal panel and is capable of changing the emission brightness for each of a plurality of divided regions; a generation means that inputs data generated from the first image into a neural network to generate the emission intensity of each divided region of the backlight module; a calculation means that calculates a correction value for correcting the first image based on the emission intensity generated by the generation means; a correction means that corrects the first image to a second image based on the correction value calculated by the calculation means; an emission control means that controls the emission intensity of each divided region of the backlight module based on the emission intensity generated by the generation means; and a display control means that controls the transmittance of the liquid crystal panel based on the data of the second image.

[0009] A second aspect of the present invention is a control method for a liquid crystal display device having a liquid crystal panel, an input means for inputting data of a first image, and a backlight module that irradiates light onto the liquid crystal panel and is capable of changing the emission brightness for each of a plurality of divided areas, the control method comprising: a generation step of inputting data generated from the first image into a neural network to generate the emission intensity of each divided area of ​​the backlight module; a calculation step of calculating a correction value for correcting the first image based on the emission intensity generated by the generation step; a correction step of correcting the first image to a second image based on the correction value calculated by the calculation step; an emission control step of controlling the emission intensity of each divided area of ​​the backlight module based on the emission intensity generated by the generation means; and a display control step of controlling the transmittance of the liquid crystal panel based on the data of the second image.

[0010] A third aspect of the present invention is a program for causing a computer to function as each of the means of the liquid crystal display device described above. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a liquid crystal display device capable of local dimming according to image characteristics by controlling a backlight module using a machine learning model, a method for controlling a liquid crystal display device, and a program. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing functional blocks of a liquid crystal display device according to a first embodiment. [Figure 2] FIG. 3 is a diagram showing an example of divided regions of a backlight according to the first embodiment. [Figure 3] FIG. 4 is a diagram showing an example of luminance estimation points of a backlight according to the first embodiment. [Figure 4] FIG. 2 is a block diagram showing functional blocks of a model learning unit according to the first embodiment. [Figure 5] FIG. 2 is a block diagram illustrating functional blocks of an LD feature quantity generating unit according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a checkered pattern image. [Figure 7] FIG. 10 is a diagram showing an example of a graph relating to local dimming according to a specific rule. [Figure 8] FIG. 10 is a diagram illustrating an example of a graph related to local dimming by the machine learning model of the first embodiment. [Figure 9] FIG. 10 is a block diagram showing functional blocks of a model learning unit according to a second embodiment. [Figure 10] FIG. 10 is a block diagram showing functional blocks of an LD image generating unit according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the technical scope of the present invention is determined by the claims and is not limited to the embodiments exemplified below. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the present invention. The contents of this specification and the drawings are merely examples and should not be considered as limiting the present invention. Various modifications are possible based on the spirit of the present invention, and these are not excluded from the scope of the present invention. In other words, all configurations that combine each embodiment and its modified examples are included in the present invention. [Example]

[0014] A first embodiment of the present invention will now be described. Fig. 1 is a block diagram showing functional blocks of a liquid crystal display device 100 according to the first embodiment. The liquid crystal display device 100 includes an image input / conversion unit 101, a local dimming control unit 102, a liquid crystal panel control unit 103, a liquid crystal panel 104, a backlight control unit 105, and a backlight module 106.

[0015] The image input / conversion unit 101 acquires image data (image data) from the outside. Specifically, the image input / conversion unit 101 has an input interface such as an SDI (Serial Digital Interface), and inputs image data from the outside into the liquid crystal display device 100 via the input interface. The image input / conversion unit 101 then performs conversion processing such as gradation conversion and signal format conversion on the acquired (input) image data, and outputs the converted image data as an input image.

[0016] The gradation conversion is, for example, gradation conversion using a one-dimensional lookup table (1D-LUT) and is gradation conversion according to the gamma value (panel gamma) of the liquid crystal panel 104. Here, consider a case where the gamma characteristic (correspondence relationship between gradation value and luminance; gradation characteristic) of the image data acquired from the outside is a linear characteristic in which luminance increases linearly with an increase in gradation value, and the panel gamma is 2.0. In this case, gradation conversion is performed using the inverse gamma of the panel gamma (i.e., 1 / 2.0). As a result, the acquired image data (image data having a linear characteristic) is converted into image data having a gamma characteristic in which luminance is proportional to the 1 / 2.0 power of the gradation value. Note that the conversion process in the image input / conversion unit 101 is not limited to gradation conversion using a 1D-LUT, but may also include conversion process using a three-dimensional lookup table (3D-LUT), gain adjustment, offset adjustment, matrix conversion, etc.

[0017] Signal format conversion is a process of converting the signal format of image data from, for example, YCbCr, XYZ, or the like to RGB. Note that the signal formats before and after conversion are not limited to YCbCr, XYZ, and RGB.

[0018] The local dimming control unit 102 is composed of a preprocessing unit 10201, a learned model holding unit 10202, a backlight control value generation unit 10203, a backlight luminance estimation unit 10204, a correction coefficient generation unit 10205, and an image correction unit 10206.

[0019] The preprocessing unit 10201 generates features (image features) based on the input image output from the image input / conversion unit 101. In the first embodiment, as shown in FIG. 2, the preprocessing unit 10201 generates the image features by calculating the maximum gradation value and the average gradation value of the input image corresponding to a plurality of divided regions. That is, when the number of horizontal divisions of the backlight module 106 is n and the number of vertical divisions is m, the preprocessing unit 10201 generates n×m two-dimensional data as the image features. Then, the preprocessing unit 10201 outputs the generated image features to the backlight control value generation unit 10203.

[0020] The trained model holding unit 10202 holds a model (trained model) that has been trained to convert image features into backlight control values. The backlight control values ​​are control values ​​for controlling the backlight module 106. The trained model is a group of parameters that are applied to a neural network that converts image features into backlight control values, and includes weights and biases between neurons that make up the neural network. Note that the trained model held in the trained model holding unit 10202 may be one that converts an input image into a backlight control value. The trained model held in the trained model holding unit 10202 is applied to the backlight control value generation unit 10203.

[0021] The backlight control value generation unit 10203 generates a backlight control value based on the image feature amount output from the preprocessing unit 10201. The backlight control value is a control value for controlling the light emission brightness of the backlight module 106 via the backlight control unit 105. As shown in FIG. 2, the backlight module 106 has a plurality of divided regions that make up the display surface set in advance, a plurality of light sources corresponding to each of the divided regions, and the light emission brightness can be changed for each divided region. The light source of the backlight module 106 is not particularly limited, but may be, for example, an LED (Light Emitting Diode). The backlight control value generation unit 10203 generates a backlight control value for each divided region. Then, the backlight control value generation unit 10203 outputs the backlight control value to the backlight brightness estimation unit 10204 and the backlight control unit 105.

[0022] In the first embodiment, the backlight control value generation unit 10203 is at least configured with a neural network that inputs image feature quantities and outputs backlight control values. When the number of horizontal divisions of the backlight module 106 is n and the number of vertical divisions is m, the neural network of the backlight control value generation unit 10203 has a structure that inputs and outputs n×m two-dimensional data. Furthermore, the neural network of the backlight control value generation unit 10203 has a structure that can apply the trained model stored in the trained model storage unit 10202. Note that the network structure of the neural network of the backlight control value generation unit 10203 is not limited as long as it can generate backlight control values. Furthermore, the input to the neural network of the backlight control value generation unit 10203 is not limited to image feature quantities and may be an input image.

[0023] The backlight control value generation unit 10203 may include a configuration for generating backlight control values ​​based on specific rules in addition to a neural network. Furthermore, the backlight control value generation unit 10203 may include multiple neural networks and specific rules for generating backlight control values. In this case, the backlight control value generation unit 10203 can switch between neural networks and specific rules.

[0024] The backlight luminance estimation unit 10204 estimates (calculates) the luminance of light irradiated from the backlight module 106 to the liquid crystal panel 104 (backlight luminance) based on the backlight control value output from the backlight control value generation unit 10203. The backlight luminance estimation unit 10204 estimates (calculates) the backlight luminance based on the backlight control value of each light source (each divided region) and a luminance distribution model of light emitted from the light source (the portion of the backlight module 106 corresponding to the divided region). Here, the backlight luminance estimation unit 10203 estimates (calculates) the backlight luminance for each luminance estimation point discretely arranged on the display surface as shown in FIG. 3(A). Specifically, the backlight luminance L is calculated by performing a product-sum operation on the backlight control value Bij and the weight Wij according to the following equation (1): In the following equation (1), n ​​represents the number of horizontal divisions of the backlight module, and m represents the number of vertical divisions of the backlight module. Furthermore, Bij is the backlight control value for vertical position i and horizontal position j, Wij is the weight applied to the backlight control value Bij, and L is the backlight brightness at the brightness estimation point. The light intensity of the backlight lit in each divided area decreases as the distance increases. Therefore, the weight Wij applied to the backlight control value Bij increases as the distance from the brightness estimation point decreases.

[0025]

number

[0026] In the example of FIG. 3(A), one luminance estimation point is arranged for each divided region of the backlight module 106, but this is not limiting. For example, it may be arranged at the four corners of the divided region as shown in FIG. 3(B), or one for each four divided regions as shown in FIG. 3(C). The backlight luminance estimation unit 10204 interpolates the backlight luminance between the luminance estimation points to scale it to the resolution of the input image. Therefore, when the luminance estimation points are arranged as shown in FIG. 3(B), the density of the luminance estimation points is higher than that of FIG. 3(A), which reduces the interpolation error but increases the amount of calculation. On the other hand, when the luminance estimation points are arranged as shown in FIG. 3(C), the density of the luminance estimation points is lower than that of FIG. 3(A), which reduces the calculation amount but increases the interpolation error. Methods for scaling the backlight luminance include, for example, bicubic interpolation and bilinear interpolation, but are not limited to these methods as long as they can scale to the resolution of the input image. The backlight luminance estimation unit 10204 then outputs the estimated backlight luminance to the correction coefficient generation unit 10205. Note that the backlight luminance estimation unit 10204 may output backlight luminances that are discretely arranged as shown in Fig. 3(A) to the correction coefficient generation unit 10205 without scaling the calculated backlight luminances. In this case, the correction coefficient generation unit 10205 scales the calculated correction coefficients to the resolution of the input image.

[0027] The correction coefficient generation unit 10205 generates a correction coefficient to be applied to the input image output from the image input / conversion unit 101, based on the backlight luminance output from the backlight luminance estimation unit 10203. In the first embodiment, the correction coefficient Gt is calculated as the reciprocal of the backlight luminance L (a value normalized to 0.0 to 1.0) according to the following formula (2). For example, when the backlight luminance is reduced to 1 / 3, the input image can be multiplied by its reciprocal, i.e., by three times, to compensate for the reduction in backlight luminance. Note that the method of calculating the correction coefficient is not limited to the reciprocal of the backlight luminance.

[0028]

number

[0029] The image correction unit 10206 generates (calculates) pixel values ​​of the corrected image by multiplying the correction coefficients output from the correction coefficient generation unit 10205 by pixel values ​​of the input image output from the image input / conversion unit 101. In the first embodiment, RGB values ​​(R value, G value, B value) = (Vr, Vg, Vb), which are pixel values ​​of the input image, are multiplied by the correction coefficient Gt according to the following equation (3), to generate RGB values ​​(Vrc, Vgc, Vbc), which are pixel values ​​of the corrected image. Then, the image correction unit 10206 outputs the generated corrected image.

[0030]

number

[0031] The liquid crystal panel control unit 103 controls the transmittance (transmittance distribution within the display surface) of the liquid crystal panel 104 based on (in response to) the corrected image output from the image correction unit 10206 so that an image based on the corrected image is displayed on the liquid crystal panel 104.

[0032] The liquid crystal panel 104 is controlled by the liquid crystal panel control unit 103 and displays an image on the display surface.

[0033] The backlight control unit 105 controls the light emission brightness of the backlight module 106 (light source of the backlight module 106) according to the backlight control value output from the backlight control value generation unit 10203. For example, the backlight control unit 105 determines the duty ratio of PWM (Pulse Width Modulation) control according to the backlight control value, and controls the light emission brightness of the backlight module 106 by PWM control at the determined duty ratio.

[0034] The backlight module 106 irradiates light onto the back surface of the liquid crystal panel 104. As described above, the light emission luminance of the backlight module 106 is changeable. In the first embodiment, a plurality of divided regions constituting the display surface are set in advance, and the backlight module 106 has a plurality of light sources corresponding to the plurality of divided regions, respectively, and the light emission luminance of each divided region is changeable.

[0035] As described with reference to FIG. 1, in the liquid crystal display device 100 of the first embodiment, the backlight module 106 can be controlled by applying the trained model to the neural network of the backlight control value generation unit 10203.

[0036] 4 is a block diagram showing functional blocks of a model learning unit 200 according to the first embodiment. The model learning unit 200 includes an image input / conversion unit 201, a preprocessing unit 202, a backlight control value generation unit 203, a first setting unit 204, an LD feature generation unit 205, a second setting unit 206, a target feature generation unit 207, an error calculation unit 208, and a learned model output unit 209. The model learning unit 200 is, for example, a program that runs on a computer, but is not limited thereto, and may operate as a functional block that constitutes the liquid crystal display device 100.

[0037] The image input / conversion unit 201 acquires image data (image data) from the outside. Specifically, the image input / conversion unit 101 acquires image data from a group of files stored in the computer's memory, but is not limited to this, and the image data may be input via an input interface such as SDI. The image input / conversion unit 101 can then perform the same conversion processing on the acquired (input) image data as the image input / conversion unit 101 of the liquid crystal display device 100, and outputs the image data after the conversion processing as an input image. However, the image input / conversion unit 201 may acquire (input) image data that has been converted in advance. In this case, the image input / conversion unit 201 does not need to perform the image data conversion processing.

[0038] The preprocessing unit 202 generates features (image features) based on the input image output from the image input / conversion unit 201. Specifically, the preprocessing unit 202 generates image features in the same manner as the preprocessing unit 10201 of the liquid crystal display device 100. That is, when the number of horizontal divisions of the backlight module 106 of the liquid crystal display device 100 is n and the number of vertical divisions is m, the preprocessing unit 202 generates n×m two-dimensional data as image features. Then, the preprocessing unit 202 outputs the generated image features to the backlight control value generation unit 203, the LD feature generation unit 205, the target feature generation unit 207, and the error calculation unit 208, respectively.

[0039] The backlight control value generation unit 203 is configured with a neural network that receives the image feature values ​​output from the preprocessing unit 202 and outputs a backlight control value. The neural network of the backlight control value generation unit 203 has the same structure as the neural network of the backlight control value generation unit 10203 of the liquid crystal display device 100. That is, when the number of horizontal divisions of the backlight module 106 of the liquid crystal display device 100 is n and the number of vertical divisions is m, the neural network of the backlight control value generation unit 203 has a structure that inputs and outputs n×m two-dimensional data. The backlight control value generation unit 203 receives the parameters generated by the error calculation unit 208 and reflects them in the neural network. The backlight control value generation unit 203 then outputs the backlight control value generated by the neural network to the LD feature generation unit 205.

[0040] The first setting unit 204 outputs the parameters to the LD feature generation unit 205. Specifically, the first setting unit 204 outputs the panel contrast and the luminance distribution model to the LD feature generation unit 205. The panel contrast is a contrast value of the liquid crystal panel 104 of the liquid crystal display device 100. The luminance distribution model is a model of the luminance distribution of light emitted from the light source of the liquid crystal display device 100 (a portion of the backlight module 106 corresponding to the divided region). The parameters set by the first setting unit 204 are acquired from a computer memory, but are not limited to this and may be input externally via an OSD (On Screen Display) menu or the like. Note that the parameters set by the first setting unit 204 to the LD feature generation unit 205 are not limited to the panel contrast and the luminance distribution model and may include other parameters required by the LD feature generation unit 205.

[0041] The LD feature generation unit 205 generates LD (Local Dimming) features based on the image features output from the preprocessing unit 202 and the backlight control values ​​output from the backlight control value generation unit 203. In the first embodiment, the LD feature generation unit 205 generates the LD features using parameters output from the first setting unit 204. The parameters output from the first setting unit 204 to the LD feature generation unit 205 are panel contrast and a luminance distribution model, but are not limited to these and may be other parameters, or external parameters may not be used. The LD feature corresponds to a simulation image of local dimming in the liquid crystal display device 100. The LD feature generation unit 205 then outputs the generated LD feature to the error calculation unit 208. Details of the processing by the LD feature generation unit 205 will be described later.

[0042] The second setting unit 206 outputs the parameters to the target feature amount generation unit 207. Specifically, the second setting unit 206 outputs a target contrast to the target feature amount generation unit 207. The target contrast is a contrast value for converting the image feature amount output from the preprocessing unit 202 into a target contrast. The parameters set by the second setting unit 206 are acquired from the computer's memory, but are not limited to this and may be input externally via an OSD (On Screen Display) menu or the like. Note that the parameters set by the second setting unit 206 to the target feature amount generation unit 207 are not limited to the target contrast and may include other parameters required by the target feature amount generation unit 207.

[0043] The target feature amount generation unit 207 generates a target feature amount based on the image feature amount output from the preprocessing unit 202. The target feature amount corresponds to a display image that is a target when local dimming control is performed on the liquid crystal display device 100. In the first embodiment, the target feature amount generation unit 207 generates the target feature amount by converting the image feature amount into the target contrast output from the second setting unit 206. Specifically, the target feature amount generation unit 207 calculates the data value Vt of the target feature amount data by performing a gain / offset calculation on the data value V of the image feature amount based on the target contrast Ct according to the following equation (4). The target contrast is acquired from the first setting unit 204. If the contrast ratio of the target contrast is 1 million to 1, 1 million is entered into the target contrast Ct.

[0044]

number

[0045] In the first embodiment, the target feature generation unit 207 generates a target feature by converting an image feature into a target contrast, but the present invention is not limited to this and may generate a target feature that corresponds to a target display image when local dimming control is performed on the liquid crystal display device 100. For example, the target feature generation unit 207 may acquire a target feature that is prepared in advance from an external source. Alternatively, the target feature generation unit 207 may output the image feature output from the pre-processing unit 202 as the target feature without modification. The target feature generation unit 207 then outputs the generated target feature to the error calculation unit 208.

[0046] The error calculation unit 208 generates parameters of the neural network of the backlight control value generation unit 203 based on the difference between the LD feature output from the LD feature generation unit 205 and the target feature output from the target feature generation unit 207. The difference between the LD feature and the target feature is, for example, L1 (the sum of the absolute values ​​of the differences). In this case, the error calculation unit 208 calculates parameters of the neural network of the backlight control value generation unit 203 so that the difference between the LD feature and the target feature is minimized. Then, the backlight control value generation unit 203 outputs the generated parameters to the backlight control value generation unit 203.

[0047] Note that the method for generating parameters by the error calculation unit 208 is not limited to one based on the difference between the LD feature and the target feature, and other methods may be used. For example, the error calculation unit 208 may generate parameters for the neural network of the backlight control value generation unit 203 based on the determination of the authenticity of the LD feature and the target feature by a classifier. The classifier is configured with a neural network. In this case, the error calculation unit 208 trains the neural network so that when a combination of the image feature and the target feature is input, the output value approaches 1 (authentic), and when a combination of the image feature and the LD feature is input, the output value approaches 0 (fake). Furthermore, the error calculation unit 208 calculates the parameters for the neural network of the backlight control value generation unit 203 so that the output value of the classifier when a combination of the image feature and the LD feature is input approaches 1 (authentic).

[0048] Note that the parameter generation method by the error calculation unit 208 may be a combination of multiple methods. For example, a method based on the difference between the LD feature amount and the target feature amount and a method based on the truth determination of the LD feature amount and the target feature amount by a classifier may be combined.

[0049] The trained model output unit 209 outputs the parameters of the neural network of the backlight control value generation unit 203 as a trained model to the outside. For example, the trained model output unit 209 outputs the trained model to the trained model holding unit 10202 of the liquid crystal display device 100 via an external memory or the like.

[0050] As described with reference to FIG. 4, the model learning unit 200 of the first embodiment can learn a neural network that outputs backlight control values ​​so that the image displayed by local dimming approaches the target value.

[0051] 5 is a block diagram showing functional blocks of the LD feature amount generation unit 205 according to Example 1. The LD feature amount generation unit 205 includes a backlight luminance estimation unit 20501, a correction coefficient generation unit 20502, a correction unit 20503, a contrast conversion unit 20504, and a synthesis unit 20505.

[0052] The backlight luminance estimation unit 20501 estimates and calculates the luminance (backlight luminance) of light irradiated onto the liquid crystal panel 104 when the backlight module 106 of the liquid crystal display device 100 is caused to emit light using the backlight control value output from the backlight control value generation unit 203. Similar to the backlight luminance estimation unit 10204, the backlight luminance estimation unit 20501 estimates (calculates) the backlight luminance for each luminance estimation point discretely arranged on the display surface as shown in FIG. 3(A) based on the backlight control value and a luminance distribution model. Here, the backlight luminance estimation unit 20501 estimates (calculates) one backlight luminance for each divided region of one backlight module. The backlight luminance estimation unit 20501 acquires the luminance distribution model used for the estimation calculation from the first setting unit 204. The backlight luminance estimation unit 20501 then outputs the estimated backlight luminance to the correction coefficient generation unit 20502 and the synthesis unit 20505.

[0053] The correction coefficient generation unit 20502 generates a correction coefficient to be applied to the image feature amount based on the backlight luminance output from the backlight luminance estimation unit 20501. The correction coefficient generation unit 20502 generates the correction coefficient by calculating, for example, the reciprocal of the backlight luminance, similar to the correction coefficient generation unit 10205 of the liquid crystal display device 100. The correction coefficient generation unit 20502 then outputs the generated correction coefficient to the correction unit 20503.

[0054] The correction unit 20503 generates (calculates) a corrected feature by multiplying the image feature output from the preprocessing unit 206 by the correction coefficient output from the correction coefficient generation unit 20502. Specifically, the correction unit 20503 calculates a data value Vc of the corrected feature by multiplying the data value V of the image feature by the correction coefficient Gt according to the following equation (5). Here, the data value V of the feature is a value normalized between 0.0 and 1.0.

[0055]

number

[0056] Then, the correction unit 20503 outputs the generated corrected feature amount to the contrast conversion unit 20504.

[0057] The contrast conversion unit 20504 generates a panel feature by converting the corrected feature output from the correction unit 20503 into a panel contrast. The panel feature corresponds to a simulation image obtained when the backlight module 106 of the liquid crystal display device 100 is caused to emit light uniformly and the corrected feature output from the correction unit 20503 is displayed on the liquid crystal panel 104. Specifically, the data value Vp of the panel feature is calculated by performing a gain / offset calculation on the data value Vc of the corrected feature based on the panel contrast Cp according to the following equation (6). The panel contrast is the contrast value of the liquid crystal panel 104 of the liquid crystal display device 100, and is obtained from the first setting unit 204. If the contrast of the liquid crystal panel 104 of the liquid crystal display device 100 is 1000:1, the panel contrast Cp is set to 1000.

[0058]

number

[0059] Then, the contrast conversion unit 20504 outputs the corrected feature amount (panel feature amount) converted into panel contrast to the synthesis unit 20505.

[0060] The synthesis unit 20505 generates an LD (Local Dimming) feature by multiplying the panel feature output from the contrast conversion unit 20504 by the backlight luminance output from the backlight luminance estimation unit 20501. Specifically, the synthesis unit 20505 calculates a data value Vl of the LD feature by multiplying the data value Vp of the panel feature by the backlight luminance L according to the following equation (7).

[0061]

number

[0062] As described with reference to FIG. 5, the LD feature quantity generating unit 205 of the first embodiment can generate data corresponding to a local dimming simulation image for calculating an error from a target feature quantity.

[0063] The difference between local dimming based on a specific rule and local dimming based on the machine learning model of the first embodiment will be described with reference to FIGS.

[0064] Figure 6(A) is an example of an 8 x 8 checkered pattern image, and Figure 6(B) is an example of a 16 x 16 checkered pattern image. The checkered pattern image in Figure 6 is an image in which white and black rectangles are arranged alternately.

[0065] 7 to 8 are examples of graphs in which data relating to the checkered pattern image in FIG. 6 is sampled horizontally from the arrow portion. The solid lines in FIGS. 7 to 8 are graphs in which the feature quantities of the checkered pattern image in FIG. 6 are normalized to 0 to 100%. The dotted lines in FIGS. 7 to 8 are graphs of backlight control values ​​generated based on the feature quantities indicated by the solid lines. The dashed lines in FIGS. 7 to 8 are graphs of the luminance (backlight luminance) of light irradiated onto the liquid crystal panel when the backlight module is caused to emit light at the backlight control values ​​indicated by the dotted lines. Note that, for the backlight luminance indicated by the dashed lines in FIGS. 7 to 8, the backlight luminance at which an image with the maximum grayscale value input to the liquid crystal panel is displayed at a predetermined white luminance (for example, 1000 nit) is set to 100%.

[0066] Fig. 7(A) is a graph showing the results when local dimming is performed on the checker pattern image of Fig. 6(A) according to a specific rule. Fig. 7(B) is a graph showing the results when local dimming is performed on the checker pattern image of Fig. 6(B) according to a specific rule. Fig. 8(A) is a graph showing the results when local dimming is performed on the checker pattern image of Fig. 6(A) using the machine learning model of Example 1. Fig. 8(B) is a graph showing the results when local dimming is performed on the checker pattern image of Fig. 6(B) using the machine learning model of Example 1.

[0067] First, with reference to FIG. 7, local dimming based on a specific rule will be described. In local dimming based on a specific rule, a backlight control value is generated by applying a predetermined spatial filter to the feature quantities of an image so as not to reduce the brightness of bright areas of the image. The spatial filter may be, for example, a high pass filter (HPF) or a low pass filter (LPF). By applying a predetermined spatial filter, i.e., a predetermined HPF or LPF, to the feature quantities of the image, the value of an area with a difference in brightness is amplified, as shown by the dotted line in FIG. 7. Furthermore, as shown by the dotted line in FIG. 7, when a backlight control value is generated by applying a predetermined spatial filter to a statistical quantity (feature quantity), the degree of increase in backlight brightness varies depending on the image pattern (image characteristics). Therefore, when a backlight control value is generated by applying a predetermined spatial filter to a statistical quantity (feature quantity), the brightness of the backlight irradiated on the liquid crystal panel may increase excessively depending on the image pattern (image characteristics), as shown by the dashed line in FIG. 7. In FIG. 7, among the dark areas (areas where the feature amount shown by the solid line is near 0%), the areas where the backlight brightness shown by the dashed line is excessively increased (for example, areas exceeding 100%) are visually perceived as black floating.

[0068] Next, local dimming using a machine learning model according to a first embodiment will be described with reference to FIG. 8. In local dimming using a machine learning model according to the first embodiment, a machine learning model that generates a backlight control value is trained so that the brightness of a displayed image by local dimming approaches a target value. As a result, the machine learning model according to the first embodiment generates a backlight control value (dotted line in FIG. 8) according to the image pattern (image characteristics) so that the minimum necessary backlight brightness (dashed line in FIG. 8) is ensured in bright areas (white rectangular areas in FIG. 6). Furthermore, the machine learning model according to the first embodiment generates a backlight control value (dotted line in FIG. 8) according to the image pattern (image characteristics) so that the backlight brightness (dashed line in FIG. 8) is reduced as much as possible in dark areas (black rectangular areas in FIG. 6).

[0069] As described above, in the first embodiment, in a liquid crystal display device in which the emission luminance of the backlight module can be changed, a control value for the backlight module can be generated using a machine learning model. By training the machine learning model so that the luminance of the image displayed by local dimming approaches a target value, local dimming according to the characteristics of the image becomes possible. [Example]

[0070] A second embodiment of the present invention will be described below. In the first embodiment, an example was described in which training data for a machine learning model is generated using feature quantities of input image data. In the first embodiment, the feature quantities of the image data are calculated based on the maximum gradation value and the average gradation value of the image data for each divided region. By generating training data using the feature quantities of the image data, the size of the training data is reduced, thereby reducing the amount of calculation required to train the machine learning model. On the other hand, the feature quantities of the image data are rounded off by the maximum or average value of pixel values ​​within a predetermined range. Therefore, in order to train the machine learning model more accurately, it is desirable to calculate the error on a pixel-by-pixel basis. Therefore, in the second embodiment, an example will be described in which training data for a machine learning model is generated using input image data, thereby calculating the error of the machine learning model on a pixel-by-pixel basis. The configuration of a liquid crystal display device 100 according to the second embodiment is the same as that of the liquid crystal display device 100 of FIG. 1.

[0071] 9 is a block diagram showing functional blocks of a model learning unit 200 according to the second embodiment. The model learning unit 200 according to the second embodiment is different from the model learning unit 200 in FIG. 4 in some functional blocks. Specifically, the model learning unit 200 according to the second embodiment has a configuration in which the LD feature generation unit 205 is replaced with an LD image generation unit 211, the target feature generation unit 207 is replaced with a target image generation unit 212, and the error calculation unit 208 is replaced with an error calculation unit 213, compared to the model learning unit 200 in FIG. 4. In the model learning unit 200 according to the second embodiment, the same components as those in the model learning unit 200 according to the first embodiment are designated by the same reference numerals and names as those in the first embodiment, and detailed descriptions thereof will be omitted as appropriate.

[0072] The LD image generation unit 211 generates an LD (Local Dimming) image based on the input image output from the image input / conversion unit 201 and the backlight control value output from the backlight control value generation unit 203. In the second embodiment, the LD image generation unit 211 generates the LD image using parameters output from the first setting unit 204. The parameters output from the first setting unit 204 to the LD image generation unit 211 are the panel contrast and the luminance distribution model, but are not limited to these and may be other parameters, or external parameters may not be used. The LD image is a simulation image of local dimming in the liquid crystal display device 100. Then, the LD image generation unit 211 outputs the generated LD image to the error calculation unit 213. Details of the processing of the LD image generation unit 211 will be described later.

[0073] The target image generation unit 212 generates a target image based on the input image output from the image input / conversion unit 201. The target image is a display image that is a target when local dimming control is performed on the liquid crystal display device 100. In the second embodiment, the target image generation unit 212 generates the target image by converting the input image output from the image input / conversion unit 201 into the target contrast output from the second setting unit 206. Specifically, the target image generation unit 212 calculates the RGB values ​​(Vrt, Vgt, Vbt) that are pixel values ​​of the target image by adjusting the RGB values ​​(R value, G value, B value) = (Vr, Vg, Vb) of the input image based on the target contrast Ct according to the following equation (8). The target contrast is acquired from the first setting unit 204. If the contrast ratio of the target contrast is 1,000,000:1, the target contrast Ct is set to 1,000,000.

[0074]

number

[0075] In the second embodiment, the target image generation unit 211 generates a target image by converting an input image into a target contrast, but the present invention is not limited to this and any display image that serves as a target when local dimming control is performed in the liquid crystal display device 100 may be used. For example, the target image generation unit 211 may acquire a target image prepared in advance from the outside. Alternatively, the target image generation unit 211 may output the input image output from the image input / conversion unit 201 as the target image as is. The target image generation unit 211 then outputs the generated target image to the error calculation unit 213.

[0076] The error calculation unit 213 generates parameters for the neural network of the backlight control value generation unit 203 based on the difference between the LD image output from the LD image generation unit 211 and the target image output from the target image generation unit 212. The difference between the LD image and the target image is, for example, L1 (the sum of the absolute values ​​of the differences). In this case, the error calculation unit 213 calculates parameters for the neural network of the backlight control value generation unit 203 so that the difference between the LD image and the target image is minimized. Then, the backlight control value generation unit 203 outputs the generated parameters to the backlight control value generation unit 203.

[0077] Note that the method for generating parameters by the error calculation unit 213 is not limited to one based on the difference between the LD image and the target image, and other methods may be used. For example, the error calculation unit 213 may generate parameters for the neural network of the backlight control value generation unit 203 based on the authenticity determination of the LD image and the target image by a classifier. The classifier is configured with a neural network. In this case, the error calculation unit 213 is trained so that when a combination of an input image and a target image is input, the output value approaches 1 (authentic), and when a combination of an input image and an LD image is input, the output value approaches 0 (fake). Furthermore, the error calculation unit 213 calculates parameters for the neural network of the backlight control value generation unit 203 so that the output value of the classifier when a combination of an input image and an LD image is input approaches 1 (authentic).

[0078] Note that the parameter generation method by the error calculation unit 213 may be a combination of multiple methods. For example, a method based on the difference between the LD image and the target image and a method based on the authenticity determination of the LD image and the target feature amount by a classifier may be combined.

[0079] Fig. 5 is a block diagram showing functional blocks of an LD image generation unit 211 according to Example 2. The configuration of the LD image generation unit 211 according to Example 2 is the same as that of the LD feature amount generation unit 205 in Fig. 5, but the processing of each functional block is partially different.

[0080] The backlight luminance estimation unit 21101 estimates and calculates the luminance (backlight luminance) of light irradiated onto the liquid crystal panel 104 when the backlight module 106 of the liquid crystal display device 100 is caused to emit light using the backlight control value output from the backlight control value generation unit 203. Similar to the backlight luminance estimation unit 10204 in FIG. 1, the backlight luminance estimation unit 21101 estimates (calculates) the backlight luminance for each discretely arranged luminance estimation point and scales the estimated backlight luminance to the resolution of the input image. The backlight luminance estimation unit 21101 then outputs the estimated backlight luminance to the correction coefficient generation unit 21102 and the synthesis unit 21105.

[0081] The correction coefficient generation unit 21102 generates correction coefficients to be applied to the input image based on the backlight luminance output from the backlight luminance estimation unit 21101. The correction coefficient generation unit 21101 generates correction coefficients by calculating correction coefficients corresponding to pixels of the input image, similar to the correction coefficient generation unit 10205 in FIG. 1. The correction coefficient generation unit 21102 then outputs the generated correction coefficients to the correction unit 21103.

[0082] The correction unit 21103 generates (calculates) pixel values ​​of a corrected image by multiplying pixel values ​​of the input image output from the image input / conversion unit 201 by correction coefficients output from the correction coefficient generation unit 20502. Similar to the image correction unit 10206 in FIG. 1 , the correction process of the correction unit 21103 generates RGB values ​​(Vrc, Vgc, Vbc) that are pixel values ​​of the corrected image by multiplying RGB values ​​(R value, G value, B value) = (Vr, Vg, Vb) that are pixel values ​​of the input image by a correction coefficient Gt. Then, the correction unit 21103 outputs the generated corrected image to the contrast conversion unit 21104.

[0083] The contrast conversion unit 21104 generates a panel image by converting the corrected image output from the correction unit 21103 into a panel contrast. The panel image is a simulation image obtained when the backlight module 106 of the liquid crystal display device 100 is caused to emit light uniformly and the corrected image output from the correction unit 21103 is displayed on the liquid crystal panel 104. Specifically, the contrast conversion unit 21104 calculates the RGB values ​​(Vrp, Vgp, Vbp) of the pixel values ​​of the panel image by adjusting the RGB values ​​(R value, G value, B value) = (Vrc, Vgc, Vbc) of the corrected image based on the panel contrast Cp according to the following equation (9). The panel contrast is the contrast value of the liquid crystal panel 104 of the liquid crystal display device 100 and is obtained from the first setting unit 204. If the contrast of the liquid crystal panel 104 of the liquid crystal display device 100 is 1000:1, the panel contrast Cp is set to 1000.

[0084]

number

[0085] Then, the contrast conversion unit 21104 outputs the corrected image (panel image) converted into the panel contrast to the synthesis unit 21105.

[0086] The synthesis unit 21105 generates an LD (Local Dimming) image by multiplying the panel image output from the contrast conversion unit 21104 by the backlight luminance output from the backlight luminance estimation unit 21101. Specifically, according to the following equation (10), the synthesis unit 21105 multiplies the RGB values ​​(R value, G value, B value) = (Vrp, Vgp, Vbp) that are pixel values ​​of the panel image by the backlight luminance L to calculate the RGB values ​​(Vrl, Vgl, Vbl) that are pixel values ​​of the LD image.

[0087]

number

[0088] As described above, in the second embodiment, input image data is used to generate training data for the machine learning model, and the error of the machine learning model is calculated on a pixel-by-pixel basis. This allows the machine learning model to learn more accurately, even when the input image data has a high spatial frequency, such as a natural image. [Explanation of symbols]

[0089] 100 LCD display device 101 Image input and conversion unit 102 Local dimming control unit 10201 Pretreatment section 10202 Trained model storage unit 10203 Backlight brightness value generation unit 10204 Backlight brightness estimation unit 10205 Correction coefficient generator 10206 Image correction unit 103 LCD panel control unit 104 LCD panel 105 Backlight control unit 106 Backlight Module

Claims

1. An LCD panel, an input means for inputting data of a first image; a backlight module that irradiates the liquid crystal panel with light and that can change the light emission brightness for each of a plurality of divided regions; a generating means for inputting data generated from the first image into a neural network to generate a light emission intensity for each divided region of the backlight module; a calculation means for calculating a correction value for correcting the first image based on the light emission intensity generated by the generation means; a correction means for correcting the first image to a second image based on the correction value calculated by the calculation means; a light emission control unit that controls the light emission intensity of each divided region of the backlight module based on the light emission intensity generated by the generation unit; a display control means for controlling the transmittance of the liquid crystal panel based on the data of the second image; A liquid crystal display device comprising:

2. 2. The liquid crystal display device according to claim 1, wherein the data generated from the first image is a feature amount of the first image.

3. 3. The liquid crystal display device according to claim 1, wherein the generation means is capable of changing the trained model applied to the neural network.

4. An LCD panel, an input means for inputting data of a first image; a backlight module that irradiates the liquid crystal panel with light and that can change the light emission brightness for each of a plurality of divided regions; A method for controlling a liquid crystal display device having a generating step of inputting data generated from the first image into a neural network to generate a light emission intensity for each divided region of the backlight module; a calculation step of calculating a correction value for correcting the first image based on the light emission intensity generated in the generation step; a correction step of correcting the first image to a second image based on the correction value calculated in the calculation step; a light emission control step of controlling the light emission intensity of each divided region of the backlight module based on the light emission intensity generated by the generating means; a display control step of controlling the transmittance of the liquid crystal panel based on data of the second image; A control method comprising:

5. A program for causing a computer to function as each of the means of the liquid crystal display device according to any one of claims 1 to 3.

6. an input step of inputting data of an input image; a first generation step of inputting data generated from the input image into a neural network to generate a light emission intensity of a backlight module; a second generation step of generating an output image based on data generated from the input image and the emission intensity generated in the generation step; a third generation step of generating a ground truth image based on data generated from the input image; a calculation step of calculating a difference between the output image and the correct image; an updating step of updating parameters of the neural network based on the difference; A method for generating a trained model, comprising:

7. The trained model generation method according to claim 6, characterized in that in the first generation step and the second generation step, the data generated from the input image is a feature of the input image.

8. The trained model generation method according to claim 6, characterized in that in the third generation step, the data generated from the input image is a feature of the input image.

9. The trained model generation method according to claim 6, characterized in that in the third generation step, the data generated from the input image is the input image.

10. The trained model generation method according to claim 6, characterized in that in the second generation step, the output image is generated based on data generated from the input image, the luminescence intensity generated by the generation step, and the contrast of the liquid crystal panel.

11. The method for generating a trained model according to claim 10, characterized in that in the second generation step, the contrast of the liquid crystal panel is changeable.

12. The method for generating a trained model according to claim 6, characterized in that in the third generation step, the output image is generated based on data generated from the input image and a target contrast.

13. The method for generating a trained model according to claim 12, characterized in that, in the third generation step, the target contrast is changeable.

Images