Generation device

Abstract

To provide an image processing system that can reduce the display unevenness of an image to be displayed in a display device.SOLUTION: A generation device includes a display device, an image capture device, and a learning device. The leaning device stores a table expressing information related to the correspondence between first image data and second image data that is generated by causing the display device to display an image corresponding to the first image data and capturing the image with the image capture device. The learning device generates teacher data on the basis of the table and generates a machine learning model using the generated teacher data. By performing image processing based on the machine learning model for the image data input to the display device, the display unevenness of the image displayed on the display device can be reduced.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] One aspect of the present invention relates to an image processing system. [Background technology]

[0002] Display devices such as liquid crystal displays and organic EL displays are made by applying resist to a substrate. After that, the film is exposed to light through a mask to perform patterning, thereby manufacturing the film. In order to enlarge the display device, the substrate must also be enlarged, but the mask is In such cases, it may not be possible to enlarge the display to fit the size of the device. As a method, the substrate surface is divided into multiple exposure areas corresponding to the size of the mask, and the exposure A method of exposing each area is disclosed in Patent Document 1.

[0003] Furthermore, due to defects or deterioration of the characteristics of display elements, transistors, etc., which are included in the pixels of the display device, In this case, defective pixels may occur. Defective pixels may be bright or dark spots, for example. When viewing an image displayed on a display device, bright points are more noticeable than dark points, so visibility is affected. Therefore, if there are too many bright spots, it becomes impossible to display a high-quality image on the display device. Patent Document 2 discloses a method for darkening bright spots during the manufacturing process of a display device. is shown. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-198990 [Patent Document 2] Special Publication No. 2018-514801 Summary of the Invention [Problem to be solved by the invention]

[0005] When performing divided exposure as described above, the mask position may be shifted from the exposure area. The exposure amount at the boundary of the exposed area may be different from the exposure amount in other areas. By this, the characteristics of the elements of the pixels provided at the boundary of the exposure area are This may cause the characteristics of the elements in the exposed area to differ even if the gradation is the same. The brightness of the light emitted by the pixels located on the boundary and the brightness of the light emitted by the pixels located in other areas are The difference in brightness may be perceived as unevenness in the display. do.

[0006] As a method to make the display unevenness less noticeable, One possible method is to perform image processing. For example, a method using machine learning can be considered. Specifically, a machine learning model is generated by a generating device, and an image input to a display device is generated. A method is conceivable in which image processing is performed on image data using a machine learning model. When image processing is performed using a machine learning model generated by a generation device, a display device and a generation device are It can be said that the image processing system is composed of the image processing device and the image processing apparatus.

[0007] In addition, even if a pixel is not a bright point at the time of manufacturing the display device, it may become a bright point after using the display device for a long period of time. This can cause deterioration of the display elements and transistors that make up the pixels, resulting in fluctuations in electrical characteristics. Such bright spots may be removed during the manufacturing process of the display device. It's difficult.

[0008] According to one embodiment of the present invention, display unevenness of an image displayed on a display device can be made less noticeable. Another object of the present invention is to provide an image processing system. To provide an image processing system capable of making high-quality images displayed on a screen. Another object of the present invention is to provide an image processing system having a large-sized display device. Another object of the present invention is to provide a high-resolution image display device. Another object of the present invention is to provide an image processing system having a display device capable of One aspect of the present invention is to provide an image processing system that can perform image processing in a short time. Another object of one embodiment of the present invention is to provide an image processing system having a highly reliable display device. One of the objectives is to provide a system.

[0009] Another aspect of the present invention is a novel image processing system, a novel image processing method, a novel generation device, A new machine learning model generation method, a new image processing device, a new display device, etc. One of our goals is to provide

[0010] The description of these problems does not preclude the existence of other problems. It is not necessary for the present invention to solve all of these problems. The above will be made clear from the description, drawings, claims, etc. It is possible to extract other issues from the descriptions in the patent, claims, etc. [Means for solving the problem]

[0011] One aspect of the present invention is an image processing system including a display device, an imaging device, and a learning device; The present invention also relates to a method for generating a machine learning model using the image processing system. The pixels are arranged in a matrix of m rows and n columns (m and n are integers of 2 or greater). The database includes first image data and a database for the first image data. The image is acquired by displaying the corresponding image on a display device and capturing the image with an imaging device. The table is stored in the memory 100. The table is generated based on the second image data. The first image data has first gradation values ​​in m rows and n columns, and the second image data has second gradation values ​​in m rows and n columns. Specifically, the table includes a first gradation value and a coordinate corresponding to the coordinate of the first gradation value. and the second gradation value.

[0012] When generating a machine learning model, first, an image corresponding to the first learning image data is displayed on a display device. The image displayed on the display device is captured by the imaging device to produce a second learning image. Next, the learning device performs a second learning process on the first learning image data. By performing image processing based on the image data for use, a third image having third gradation values ​​in m rows and n columns is obtained. Specifically, the learning device generates the first learning image data. Then, image processing is performed to approximate the second learning image data, and the first image in the mth row and nth column is obtained. For example, the third learning image data having a gradation value of 3 is generated. The sum of the third gradation values ​​in the first column is the sum of the third gradation values ​​in the first row and first column to the mth row and nth row of the second learning image data. Image processing is performed on the first learning image data so that the sum of the gradation values ​​of the first column is equal to the sum of the gradation values ​​of the second column. Carry out the process.

[0013] Then, the learning device calculates the third gradation values ​​of the first row, the first column, through the mth row, and the nth column. The second gradation values ​​in the first column to the mth row and nth column are selected. For example, The value corresponding to the second gradation value or the closest value is set for each of the first row, first column to the mth row, nth column. Next, image data including a first gradation value corresponding to the selected second gradation value is selected. Then, the learning device generates a fourth learning image data set. When the input data is used, the image data output matches the fourth learning image data. Generate a machine learning model.

[0014] The machine learning model generated by the learning device is supplied to the display device. Image processing using a machine learning model is performed on image data input to the display device. For example, it is possible to reduce display unevenness for image data input to a display device. Image processing such as this can be performed using machine learning models.

[0015] One aspect of the present invention is a display device, an imaging device, and a learning device, wherein the display device includes an input unit and a machine learning processing unit, and a matrix of pixels with m rows and n columns (m and n are integers greater than or equal to 2). and a display unit configured to display the image data. The learning device includes a database, an image processing unit, an image generation unit, and The database includes a learning unit, and the database includes a first image data input to the input unit and a first An image corresponding to the image data is displayed on the display unit, and the image capturing device captures the image displayed on the display unit. and a table generated based on the second image data acquired by capturing the image so as to include the second image data. The first image data has first gradation values ​​arranged in m rows and n columns, and the second image data has m rows and n columns of second gradation values, and the table has first gradation values ​​and coordinates of the first gradation values. and a second gradation value of the coordinates corresponding to the first learning input to the input unit. By performing image processing on the training image data based on the second training image data, The second learning image data is generated based on the first learning image data. An image corresponding to the image data for use is displayed on the display unit, and the image capturing device displays the image on the display unit. The third learning image data is image data acquired by capturing an image including has third gradation values ​​arranged in m rows and n columns, and the image generating unit generates a second gradation value selected based on the third gradation value. and generating fourth learning image data, which is image data including a first gradation value corresponding to the gradation value of The learning unit has a function of generating an image to be output when the first learning image data is input. A machine learning model is generated so that the data matches the fourth training image data, and machine learning is performed. The machine learning processing unit has a function of outputting a machine learning model to the processing unit, and the machine learning processing unit outputs the Image processing with the function of processing content image data using machine learning models It is a system.

[0016] Alternatively, in the above aspect, the first learning image data has fourth gradation values ​​arranged in m rows and n columns, The second learning image data has a fifth gradation value of m rows and n columns, and the image processing unit The difference between the sum of the fourth gradation values ​​and the sum of the fifth gradation values ​​is The image processing function may be implemented so that the difference between the total and the total is smaller than the difference between the total and the total.

[0017] Alternatively, in the above aspect, the machine learning model may be a neural network model. good.

[0018] Alternatively, in one aspect of the present invention, pixels are arranged in a matrix of m rows and n columns (m and n are integers of 2 or more). A method for generating a machine learning model using an image processing system having a display unit with a plurality of rows, comprising: An image corresponding to the first image data having the first gradation value in the nth row and column is extracted from the pixels to obtain the first gradation value. The first value is displayed on the display unit by emitting light of a brightness corresponding to the value. By capturing an image so as to include an image corresponding to the image data of m rows and n columns, and obtaining second image data having a value, and determining a first gradation value and a coordinate corresponding to the first gradation value. and a table representing the second gradation value of the coordinates, and an image corresponding to the first learning image data is generated. The image displayed on the display unit includes an image corresponding to the first learning image data. By capturing an image in this manner, the second learning image data is obtained, and the first learning image data is By performing image processing on the data based on the second learning image data, the first image in the mth row and nth column is obtained. A third learning image data having a gradation value of 3 is generated, and the selected Fourth learning image data, which is image data including a first gradation value corresponding to the second gradation value. The image data output when the first learning image data is input is the fourth learning image data. A method for generating a machine learning model that matches training image data is.

[0019] Alternatively, in the above aspect, the first learning image data has fourth gradation values ​​arranged in m rows and n columns, The second learning image data has a fifth gradation value of m rows and n columns, and the image processing is The difference between the sum of the fourth gradation values ​​and the sum of the fifth gradation values ​​is The above may be performed so that the difference between

[0020] Alternatively, in the above aspect, the machine learning model may be a neural network model. good.

[0021] Another aspect of the present invention is a display device including an imaging device and a generating device, wherein the display device includes: An input unit, a bright spot correction unit, and a matrix of m rows and n columns (m and n are integers of 2 or more) of pixels are arranged. The generating device has a database and an image generating unit, and the data The database includes a first database image data input to the input unit and a first database image data. an image corresponding to the image data is displayed on the display unit, and the image capturing device and second database image data obtained by capturing an image including the The created table is stored, and the first database image data is a first gradation image data of m rows and n columns. the second database image data has second gradation values ​​in m rows and n columns, and the table represents a first gradation value and a second gradation value at a coordinate corresponding to the coordinate of the first gradation value, The imaging device displays on the display unit an image corresponding to the first bright spot correction image data input to the input unit. When the image displayed on the display unit is captured, the second bright spot correction image data is generated. The second image data for bright spot correction has a function of acquiring third gradation values ​​of m rows and n columns. The image generating unit generates a first gradation corresponding to the second gradation value selected based on the third gradation value. The third image data for bright spot correction is image data including a value, and the third image data for bright spot correction is generated. The positive part is the threshold value of the first gradation value of m rows and n columns contained in the third bright spot correction image data. The bright spot correction unit has a function of detecting the coordinates of the first gradation value that is equal to or less than the value m When content image data having a fourth gradation value in row n column is input to the input unit, This is an image processing system that has the function of reducing the fourth gradation value of the same coordinates as the coordinates.

[0022] Another aspect of the present invention is a display device including an imaging device and a generating device, wherein the display device includes: An input unit, a bright spot correction unit, and a matrix of m rows and n columns (m and n are integers of 2 or more) of pixels are arranged. The generating device has a database and an image generating unit, and the data The database includes a first database image data input to the input unit and a first database image data. an image corresponding to the image data is displayed on the display unit, and the image capturing device and second database image data obtained by capturing an image including the The created table is stored, and the first database image data is a first gradation image data of m rows and n columns. the second database image data has second gradation values ​​in m rows and n columns, and the table represents a first gradation value and a second gradation value at a coordinate corresponding to the coordinate of the first gradation value, The imaging device displays on the display unit an image corresponding to the first bright spot correction image data input to the input unit. When the image displayed on the display unit is captured, the second bright spot correction image data is generated. The second image data for bright spot correction has a function of acquiring third gradation values ​​of m rows and n columns. The image generating unit generates a first gradation corresponding to the second gradation value selected based on the third gradation value. The third image data for bright spot correction is image data including a value, and the third image data for bright spot correction is generated. The positive part is the first gradation value of the m rows and n columns of the third bright spot correction image data. The coordinates of the first gradation value that is equal to or less than the threshold value are detected as the first bright point coordinates. The correction unit corrects the second gradation value of the m rows and n columns of the second bright spot correction image data. and detecting the coordinates of the third gradation value equal to or greater than the threshold value as the second bright point coordinates, The point correction unit receives content image data having a fourth gradation value of m rows and n columns from the input unit. When the fourth gradation value of the coordinates that are the same as the first or second bright point coordinates is decreased, It is an image processing system.

[0023] Another aspect of the present invention is a display device including an imaging device and a generating device, wherein the display device includes: An input unit, a bright spot correction unit, and a matrix of m rows and n columns (m and n are integers of 2 or more) of pixels are arranged. The generating device has a database and an image generating unit, and the data The database includes a first database image data input to the input unit and a first database image data. an image corresponding to the image data is displayed on the display unit, and the image capturing device and second database image data obtained by capturing an image including the The created table is stored, and the first database image data is a first gradation image data of m rows and n columns. the second database image data has second gradation values ​​in m rows and n columns, and the table represents a first gradation value and a second gradation value at a coordinate corresponding to the coordinate of the first gradation value, The imaging device displays on the display unit an image corresponding to the first bright spot correction image data input to the input unit. When the image displayed on the display unit is captured, the second bright spot correction image data is generated. The second image data for bright spot correction has a function of acquiring third gradation values ​​of m rows and n columns. The image generating unit generates a first gradation corresponding to the second gradation value selected based on the third gradation value. The third image data for bright spot correction is image data including a value, and the third image data for bright spot correction is generated. The positive part is the first gradation value of the m rows and n columns of the third bright spot correction image data. The coordinates of a first gradation value that is equal to or less than the threshold value and equal to or greater than the second threshold value are defined as first bright point coordinates, The coordinates of the first gradation value less than the second threshold value are detected as second bright point coordinates. The bright spot correction unit has a function of correcting a third gradation of m rows and n columns contained in the second bright spot correction image data. Among the values, coordinates of a third gradation value equal to or greater than a third threshold value are detected as third bright point coordinates. The bright spot correction unit receives content image data having a fourth gradation value of m rows and n columns. When input to the input unit, the coordinates of the first bright spot and the coordinates of the third bright spot are the same as each other. The fourth gradation value has a function of decreasing the fourth gradation value, and the fourth gradation value has the same coordinates as the second bright point coordinates. It is an image processing system that has the function of reducing

[0024] Alternatively, in the above aspect, the display device includes a machine learning processing unit, and the generation device includes an image processing unit. and a learning unit, wherein the image processing unit performs a first learning image data input to the input unit. Then, image processing is performed based on the second learning image data to generate a third learning image data. The second learning image data corresponds to the first learning image data. The image to be captured is displayed on the display unit, and the image capturing device captures an image including the image displayed on the display unit. The third learning image data is the image data acquired by The image generating unit has a gradation value of a function of generating fourth learning image data, which is image data including the first gradation value; The learning unit determines whether the image data output when the first learning image data is input is the fourth learning image data. A machine learning model that matches the training image data is generated, and the machine learning model is input to the machine learning processing unit. The machine learning processing unit has a function of outputting the content image data input to the input unit. The device may have a function to process the data using a machine learning model.

[0025] Alternatively, in the above aspect, the first learning image data has a sixth gradation value in m rows and n columns, The second learning image data has a seventh gradation value of m rows and n columns, and the image processing unit The difference between the sum of the 6th gradation values ​​and the sum of the 7th gradation values ​​is The image processing function may be implemented so that the difference between the total and the total is smaller than the difference between the total and the total.

[0026] Alternatively, in the above aspect, the machine learning model may be a neural network model. good. [Effects of the Invention]

[0027] According to one aspect of the present invention, it is possible to make display unevenness of an image displayed on a display device less noticeable. According to one embodiment of the present invention, a display device can be provided. It is an object of the present invention to provide an image processing system capable of making high-quality images displayed on a display device. Furthermore, one aspect of the present invention provides an image processing system having a large display device. Furthermore, according to one embodiment of the present invention, a high-resolution image can be displayed. An image processing system including a display device can be provided. This makes it possible to provide an image processing system that can perform image processing in a short time. In addition, one aspect of the present invention provides an image processing system having a highly reliable display device. It is possible.

[0028] Furthermore, according to one aspect of the present invention, a novel image processing system, a novel image processing method, a novel generating method, generation device, a new machine learning model generation method, a new image processing device, a new display device, etc. can be provided.

[0029] The effects of one embodiment of the present invention are not limited to the effects listed above. This does not preclude the existence of other effects. The effects not mentioned in this section are effects that a person skilled in the art would be able to understand by noticing the description. It can be derived from the descriptions in documents, drawings, etc., and can be extracted appropriately from these descriptions. One aspect of the present invention has at least one of the above-listed effects and / or other effects. Therefore, one aspect of the present invention is that, in some cases, It may not have the effects listed above. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image processing system. [Figure 2] Fig. 2A is a block diagram showing an example of the configuration of a display unit, Fig. 2B1 and Fig. 2B2 are circuit diagrams showing an example of the configuration of a pixel. [Figure 3] 3A and 3B are schematic diagrams showing an example of an image processing method. [Figure 4] FIG. 4 is a flowchart showing an example of a method for generating a table. [Figure 5] 5A and 5B are schematic diagrams showing an example of a method for generating a table. [Figure 6] 6A and 6B are schematic diagrams showing an example of a method for generating a table. [Figure 7] FIG. 7 is a flowchart illustrating an example of a method for generating a machine learning model. [Figure 8] 8A and 8B are schematic diagrams showing an example of a method for generating a machine learning model. [Figure 9] 9A and 9B are schematic diagrams showing an example of a method for generating a machine learning model. [Figure 10] FIG. 10 is a schematic diagram illustrating an example of a method for generating a machine learning model. [Figure 11] 11A and 11B are schematic diagrams showing an example of an image processing method. [Figure 12] 12A and 12B are graphs showing an example of an image processing method. [Figure 13] Figures 13A1 and 13A2 are graphs showing an example of an image processing method, and Figure 13B is a schematic diagram showing an example of an image processing method. [Figure 14] FIG. 14 is a graph showing an example of an image processing method. [Figure 15] Fig. 15A is a diagram showing an example of the configuration of a machine learning model, and Fig. 15B is a schematic diagram showing an example of a learning method. [Figure 16] 16A and 16B are schematic diagrams showing an example of calculations by a machine learning model. [Figure 17] FIG. 17 is a graph showing the learning results according to the example. DETAILED DESCRIPTION OF THE INVENTION

[0031] The embodiments will be described in detail with reference to the drawings. However, the present invention is not limited to the following description. The present invention may be modified in various forms and details without departing from the spirit and scope of the present invention. It will be readily understood by those skilled in the art that the present invention can be carried out in the following embodiments. It should not be construed as being limited to the description of the form.

[0032] In the configuration of the invention described below, the same parts or parts having similar functions are The same reference numerals are used in common between different drawings, and repeated explanations thereof will be omitted. When referring to a function, the hatch pattern may be the same and no particular symbol may be assigned.

[0033] In addition, the position, size, range, etc. of each component shown in the drawings are not necessarily the same as in reality for ease of understanding. Therefore, the disclosed invention may not necessarily represent the position, size, range, etc. Furthermore, the present invention is not limited to the position, size, range, etc. disclosed in the drawings.

[0034] In addition, the ordinal numbers "first," "second," and "third" used in this specification are intended to be used to indicate a mixture of elements. The numbers are added to avoid confusion and are not intended to limit the number.

[0035] (Embodiment) In this embodiment, an image processing system or the like according to one embodiment of the present invention will be described with reference to the drawings. do.

[0036] <Image processing system> FIG. 1 is a block diagram showing an example of the configuration of an image processing system 10. The system includes a display device 20, an imaging device 30, and a generating device 40. Here, the generating device 40 is It is preferable to provide it in a device with high computing power, such as a server.

[0037] The display device 20 includes an input unit 21, a display unit 22, a machine learning processing unit 23, and a bright spot correction unit 50. The generating device 40 includes a database 42, an image extracting unit 43, an image processing unit 44, and an image generating unit. The image forming unit 45 includes a generating unit 46 and a learning unit 46 .

[0038] 2A is a block diagram showing an example of the configuration of the display unit 22. As shown in FIG. 2, pixels 24 are arranged in a matrix of m rows and n columns (m and n are integers of 2 or greater). The pixels 24 in the same row are electrically connected to each other via the same wiring 134, and the pixels in the same column are electrically connected to each other via the same wiring 134. The pixels 24 are electrically connected to each other via the same wiring 126. and an image can be displayed on the display unit 22 using the display element.

[0039] In this specification, the pixels 24 in m rows and n columns are referred to as pixels 24(1,1) to 24(2,3), respectively. (m,n) to distinguish them. Other elements may also be described in the same way. For example, (1,1) to (m,n) are sometimes called coordinates.

[0040] In this specification and the like, the term "display element" can be rephrased as "display device." Light-emitting elements can be called light-emitting devices, and liquid crystal elements can be called liquid crystal devices. For other elements, the word "element" can be replaced with "device." There are cases where this happens.

[0041] In FIG. 1, arrows indicate data exchange between components of the image processing system 10. Note that the data exchange shown in Figure 1 is an example, and if the data is not connected by arrows, In some cases, data can be exchanged between different components. Even when components are coupled, there are cases where data is not exchanged between them.

[0042] Image data is input to the input unit 21. The image data input to the input unit 21 is displayed on the the image processing unit 22, the machine learning processing unit 23, the database 42, the image processing unit 44, or the learning unit 46. You can exert your power.

[0043] The image data input to the input unit 21 includes database image data DG IN , learning images Image data LG IN , bright spot correction image data BCGIN , and content image data CG I N Database image data DG IN The input unit 21 is connected to the display unit 22, and and can be supplied to the database 42. IN is the input unit 21 The bright spot correction image data B can be supplied to the display unit 22 and the image processing unit 44. CG IN can be supplied from the input unit 21 to the display unit 22 and the image processing unit 44. Content image data CG IN can be supplied from the input unit 21 to the machine learning processing unit 23. can.

[0044] The display unit 22 has a function of displaying an image corresponding to the image data. can be a set of gradation values. For example, the image data supplied to the display unit 22 can be In this case, the pixel 24 corresponds to the grayscale value. By emitting light of a corresponding brightness, an image can be displayed on the display unit 22. The gradation value can be a digital value. For example, the gradation value is an 8-bit digital value. In this case, the gradation value can be an integer between 0 and 255.

[0045] The machine learning processing unit 23 generates the image data based on the machine learning model generated by the generating device 40. Specifically, the learning unit 46 has a function of performing image processing on the machine learning data. Based on the model MLM, content image data CG input from the input unit 21 IN whereas The image data that has undergone image processing by the machine learning processing unit 23 is The data is content image data CGML to the bright spot correction unit 50.

[0046] Machine learning models such as multilayer perceptrons and neural network models are used. In particular, applying a neural network model can improve image processing. This is preferable because it allows efficient processing and allows high-quality images to be displayed on the display unit 22. Here, as a neural network model, for example, an autoencoder, a U- NET, pix2pix, and other generative models can be used. Even if it is a learning model, if it can be used for Bayesian estimation, it can be used as a machine learning model (MLM). In addition, the machine learning model MLM performs learning and inference by independently calculating the input and output values. It is preferable that this can be done in a stand-alone manner.

[0047] The bright spot correction unit 50 corrects the content image data CG ML Bright spot correction unit 50 is specifically content image data CG ML It has a function to correct the gradation value of As will be described in detail later, the bright spot correction unit 50 corrects the bright spot image data generated by the image generation unit 45. The image extraction unit 43 generates the image data BCG_1 for bright spot correction, or the image data BCG_2 for bright spot correction. Content image data CG ML The image after correction has a function to correct the gradation value of the Image data is content image data CG COR is supplied to the display unit 22 as

[0048] In this specification, the bright spot correction image data BCG_1 and the bright spot correction image data BC G_2 etc. may be collectively referred to as bright spot correction image data BCG. Specifically, If you write "bright spot correction image data BCG", for example, bright spot correction image data BCG_1 , or the bright spot correction image data BCG_2. Similar statements may be made.

[0049] Here, the bright spot correction unit 50 adjusts the content so that bright spots on the display unit 22 are less noticeable. Tsu image data CG ML The bright spot correction unit 50 has a function of correcting the bright spots, for example, by converting the bright spots into dark spots. Content image data CG ML As a result, the display device By providing a bright spot correction unit 50 in the display unit 20, the quality of the image displayed on the display unit 22 can be improved. can be done.

[0050] In this specification, the terms "darkening" and "making it a dark point" refer to the process of emitting light from a pixel 24 that is a bright point. Therefore, even if the pixel 24 is darkened, the brightness of the light is reduced. The brightness of the light emitted from the pixel 24 does not have to be zero.

[0051] The display device 20 may not have the bright spot correction unit 50. In this case, the bright spot correction image Data BCG IN is not input to the input unit 21. Also, the machine learning processing unit 23 outputs Content image data CG ML can be supplied to the display unit 22.

[0052] The imaging device 30 has a function of capturing an image and acquiring imaging data. Specifically, the camera 30 can capture an image including the image displayed on the display unit 22. The acquired image data is supplied to the image extraction unit 43. Data DG INAn image corresponding to the image is displayed on the display unit 22, and the image is captured so as to include the image. The imaging data acquired by the imaging device 30 is converted into imaging data IMG DG In addition, Learning image data LG IN The image corresponding to the image is displayed on the display unit 22, and the image is displayed so as to include the image. The imaging data acquired by the imaging device 30 through imaging is referred to as imaging data IMG LG Let's say Furthermore, the bright spot correction image data BCG IN An image corresponding to the image is displayed on the display unit 22. The imaging data acquired by the imaging device 30 by capturing an image including the image is called imaging data. TaIMG BCG The imaging data can be a set of gradation values.

[0053] The image extraction unit 43 extracts the image data IMG DG , image data IMG LG , and imaging data IM G BCG and the like, and has a function of extracting data representing the image displayed on the display unit 22. When the imaging device 30 captures an image including the image displayed on the display unit 22, In some cases, an area other than the display unit 22 may also be imaged. For example, in addition to the display unit 22, the housing of the display device 20 may also be imaged. The image extraction unit 43 may capture images other than the image displayed on the display unit 22 in this way. When the image data includes the other part, the data of the part representing the image displayed on the display unit 22 is Data extraction is performed using pattern matching and template matching. For example, the image displayed on the display unit 22 and the image displayed on the display device 20 and extracting data representing the image displayed on the display unit 22 from the imaging data including the housing. When displaying the pattern, a pattern representing the housing of the display device 20 is specified, and the part that does not include the pattern is displayed. The image data can be used as part of the image displayed on the display unit 22. Data IMG DG , image data IMG LG , and image data IMG BCG etc. By detecting the image, data representing the image displayed on the display unit 22 can be extracted. .

[0054] The image extraction unit 43 extracts the image data IMG DG The data extracted from the database image data DG DP In addition, the image extraction unit 43 extracts the captured image data IMG LG The data extracted from Learning image data LG DP Furthermore, the image extraction unit 43 extracts the captured image data IMG BCG mosquito The data extracted from the image data for bright spot correction BCG DP Database image data DG DP is supplied to the database 42, and the learning image data LG DP , and bright spot correction image Image data BCG DP is supplied to the image processing unit 44. The bright spot correction image data BCG DP may be supplied to the image generating unit 45 instead of to the image processing unit 44.

[0055] The image extraction unit 43 also extracts the bright spot correction image data BCG DP is supplied to the bright spot correction unit 50. The bright spot correction image data BCG supplied to the bright spot correction unit 50 DP The bright spot correction The correct image data is designated as BCG_2.

[0056] As will be described in detail later, the image processing system 10 stores database image data DG IN and database image data DG DP Obtain a table showing the correspondence between The image processing system 10 also has the function of IN And learning images Image data LG DP Furthermore, the image processing system 10 has a function of comparing the bright spot correction. Image data for BCG IN and bright spot correction image data BCG DP It has the function to compare.

[0057] Therefore, the database image data DG IN The resolution of the image represented by the TaDG DP It is preferable that the resolution of the image represented by is equal to that of the database. Image data DG IN The number of rows and columns of the gradation values ​​included in the database image data DG D P It is preferable that the number of rows and columns of the gradation values ​​included in the database are equal. Image data DG IN contains m rows and n columns of gradation values, the database image data DG D P It is also preferable that the gradation values ​​included in the learning image data LG IN The resolution of the image represented by and the learning image data LG DP The resolution of the image represented by is equal to Specifically, the learning image data LG IN The number of rows and columns of gradation values ​​included in , learning image data LG DP It is preferable that the number of rows and columns of the gradation values ​​included in For example, the learning image data LG IN contains m rows and n columns of grayscale values, Data LG DPIt is also preferable that the gradation values ​​included in the bright spot correction image are arranged in m rows and n columns. Image data BCG IN The image resolution represented by and the bright spot correction image data BCG DP Image represented by It is preferable that the resolution of the bright spot correction image data BCG is equal to that of the bright spot correction image data BCG. IN to The number of rows and columns of the included gradation values ​​and the image data for bright spot correction BCG DP of the tone values ​​contained in It is preferable that the number of rows and the number of columns are equal. IN to m If the gradation value of row n column is included, the image data for bright spot correction BCG DP The tone values ​​included in m It is preferable to have n rows and n columns.

[0058] In this specification, database image data DG IN The gradation value of In addition, the database image data DG DP The gradation value of the second gradation This is sometimes called adjustment value.

[0059] The image extraction unit 43 performs up-conversion or For example, the image extraction unit 43 can perform down-conversion on the image data IM G DG The number of rows or columns of the gradation values ​​contained in the data extracted from the database image data D G IN If the number of rows or columns is less than the number of rows or columns of the image data IMG DG from The extracted data can be up-converted. 3 is the image data IMG DG The number of rows or columns of the gradation values ​​contained in the data extracted from the data Base image data DG INIf the number of rows or columns is greater than the number of rows or columns, the image extraction unit 43 extracts the captured image data. TaIMG DG Down-conversion can be performed on the data extracted from the From the above, the database image data DG DP The number of rows and columns of the gradation values ​​included in the data Base image data DG IN The number of rows and columns of the grayscale values ​​contained in the matrix can be made equal to the number of rows and columns of the grayscale values ​​contained in the matrix. Image data IMG LG , and image data IMG BCG The same applies to Up-conversion and down-conversion are performed using the nearest neighbor method, bilinear method, This can be done by the bicubic method or the like.

[0060] The database 42 contains database image data DG IN and database image data D G DP A table T representing the correspondence between and can be stored. Table T is Specifically, the database image data DG IN and a first gradation value that the database Image data DG DP The table T represents information about the correspondence between the second gradation value of and the For example, a first gradation value, a second gradation value at a coordinate corresponding to the coordinate of the first gradation value, The table T represents, for example, a first gradation value and a coordinate that is the same as the coordinate of the first gradation value. and the second gradation value.

[0061] The image processing unit 44 receives the learning image data LG IN On the other hand, the learning image data LG DP Based on By performing image processing based on the IP It has the function of generating Mari, learning image data LG INand learning image data LG DP Based on the comparison results, Learning image data LG IN By performing image processing on the learning image data LG IP For example, the learning image data LG IN For the training image DataLG DP By performing image processing to make the learning image data LG IP of The image processing unit 44 also has a function of generating the bright spot correction image data BCG IN Against Even if the image is processed in the same way, the bright spot correction image data BCG IP It has the function of generating .

[0062] The image processing unit 44 receives, for example, learning image data LG IP The sum of the gradation values ​​of the learning image Image data LG DP The difference between the sum of the gradation values ​​of and is the learning image data LG IP has The sum of the gradation values ​​and the learning image data LG IN The sum of the gradation values ​​of and is smaller than the difference So, the learning image data LG IN It has the function of converting the gradation values ​​of the The image processing unit 44 receives, for example, learning image data LG IP The sum of the gradation values ​​of Learning image data LG DP The learning image data L G IN It has the function of converting the gradation values ​​of the "learning image data" by image processing. TaLG IP The sum of the gradation values ​​of the learning image data LG IP All the gradations that It may be the sum of the values, or the sum of some of the gradation values. LG IN The sum of the gradation values ​​of the learning image data LG IN All the tonal values ​​that It may be the sum of all the gradation values, or the sum of some of the gradation values. LG DP The sum of the gradation values ​​of the learning image data LG DP All the tonal values ​​that It may be the sum of all the gradation values, or the sum of some of the gradation values.

[0063] Furthermore, the image processing unit 44 may, for example, IP Learning image data LG D P Peak Signal-to-Noise Ratio (PSNR) Ratio) or Structural SIMilar (SSIM) ity) is the learning image data LG IN The PSNR or SSIM for In this way, the learning image data LG IN It has the function of converting the gradation values ​​of the The image processing unit 44 receives, for example, learning image data LG DP PSNR or SSI for To maximize M, we use the learning image data LG IN Convert the gradation values ​​of By doing so, the learning image data LG IP It has the function of generating

[0064] The image processing performed by the image processing unit 44 can be, for example, gamma correction. By setting the comma value to an appropriate value, the above image processing can be performed.

[0065] The image processing unit 44 receives, for example, the bright spot correction image data BCG IP The sum of the gradation values ​​of Point correction image data BCG DP The difference between the sum of the gradation values ​​of and is the image data for bright spot correction. BCG IP The sum of the gradation values ​​of the bright spot correction image data BCG IN The sum of the gradation values ​​of , the difference between the bright spot correction image data BCG IN The gradation values ​​of The image processing etc. is performed using the learning image data LG IP Bright spot correction image data BCG IP and read as learning image data LG DP For bright spot correction Image data BCG DP and read as learning image data LG IN Bright spot correction image data B CG IN The above explanation can be applied by reading it as follows:

[0066] The image generation unit 45 generates learning image data LG IP Based on the gradation value of For example, the database image data DG IN but database image data DG having first gradation values ​​arranged in m rows and n columns; DP is the second level of m rows and n columns The learning image data LG IP has m rows and n columns of gray scale values. In this case, the image generating unit 45 generates the learning image data LG IP The gradation value of the 1st row, 1st column to the mth row, nth column Based on each of the above, the second gradation values ​​in the first row and first column to the m-th row and n-th column can be selected. Specifically, the learning image data LG IP The value that matches or is closest to the gradation value of can be selected for each of the first row, first column to the m-th row, n-th column. For example, if a table T is a database of k (k is an integer equal to or greater than 2) image data DG IN and k database image data DG DP It represents information about the correspondence between and . In this case, for example, the learning image data LG IP The i-th row and j-th column of (i is an integer between 1 and m, j k second gradation values ​​that match or are closest to the gradation value of The image generating unit 45 can select the second gradation value from the i-th row and j-th column of the bright point. Correction image data BCG IP Based on the gradation value of It has a function of selecting the second gradation value included in the table T. The generated image data for bright spot correction BCG DP to the image generating unit 45, Image data BCG DP Based on the gradation values ​​of the table, A second gradation value included in rule T can be selected.

[0067] In this specification, for example, the gradation value at the i-th row and j-th column is referred to as the "gradation value at coordinates (i, j)." There are cases where this happens.

[0068] The image generating unit 45 also generates learning image data LG IP to the second tone value selected based on Learning image data LG, which is image data including the corresponding first gradation value GEN Generate Similarly, the image generating unit 45 generates the bright spot correction image data BCG IP Selected based on a bright spot correction image, which is image data including a first gradation value corresponding to the selected second gradation value; For example, the learning image data LG IP i Table T contains multiple second gradation values ​​in the i-th row and j-th column that match the gradation value in the j-th row. If the second gradation values ​​are selected, one second gradation value can be selected from the plurality of second gradation values. Then, the first gradation value corresponding to the selected second gradation value is added to the learning image data LG G EN Bright spot correction image data BCG IP The same applies to the following:

[0069] Also, for example, learning image data LG IP The value in the i-th row and j-th column of If the second gradation value is not included in table T, the second gradation value in the i-th row and j-th column must not be selected. In this case, the learning image data LG GEN The tone value in the i-th row and j-th column of the training image DataLG IP The value can be the same as the gradation value in the ith row and jth column of the bright spot correction image data. BCG IP The same applies to the following:

[0070] The learning unit 46 receives the learning image data LG IN and learning image data LG GEN and, using For example, the learning unit 46 generates a machine learning model MLM. Data LG IN The image data output when input is the learning image data LG GEN and The learning unit 46 has a function of generating a machine learning model MLM that matches the A machine learning model MLM is used, for example, to IN and learning image data LG GENIn this way, the machine learning model has the function of generating the data through supervised learning. The model MLM can be generated by learning. The learning model MLM can be said to be a trained machine learning model.

[0071] The machine learning model MLM generated by the learning unit 46 is supplied to the machine learning processing unit 23. The learning processing unit 23 performs inference based on the machine learning model MLM to obtain the following information about the image data: Image processing can be performed on the image.

[0072] 2B1 and 2B2 are circuit diagrams showing examples of the configuration of the pixel 24 shown in FIG. 2B1 is a circuit diagram showing an example of the configuration of a sub-pixel included in a pixel 24. , a transistor 161, a transistor 171, a capacitor 173, and a light-emitting element 170. In the pixel 24 shown in Figure 2B1, the light emitting element 170 can be the display element.

[0073] One of the source and drain of the transistor 161 is electrically connected to the gate of the transistor 171. The gate of the transistor 171 is electrically connected to one electrode of the capacitor 173. One of the source and drain of the transistor 171 is connected to one of the light-emitting elements 170. It is electrically connected to the electrode.

[0074] The other of the source and the drain of the transistor 161 is electrically connected to the wiring 126 . The gate of the transistor 161 is electrically connected to the wiring 134. The other of the source or drain and the other electrode of the capacitor 173 are electrically connected to the wiring 174. The other electrode of the light emitting element 170 is electrically connected to a wiring 175 .

[0075] A constant potential can be applied to the wiring 174 and the wiring 175. For example, in FIG. As shown, the anode of the light emitting element 170 is connected to one of the source and drain of the transistor 171. When the cathode of the light emitting element 170 is electrically connected to the wiring 175, In this case, a high potential can be supplied to the wiring 174 and a low potential can be supplied to the wiring 175.

[0076] The light emitting element 170 can be, for example, an organic EL element or an inorganic EL element.

[0077] When the pixel 24 provided in the display unit 22 has the configuration shown in FIG. 2B1, the light emitting element 170 The magnitude of the current flowing through the light emitting element 170 is controlled to control the luminance of the light emitting element 170. The larger the current flowing through the light emitting element 170, the more light can be emitted. The luminance of light emitted by the device can be increased.

[0078] The pixel 24 shown in FIG. 2B2 includes a transistor 162, a capacitor 181, a liquid crystal element 180, and In the pixel 24 shown in FIG. 2B2, the liquid crystal element 180 can be used as the display element. .

[0079] One of the source and drain of the transistor 162 is electrically connected to one electrode of the liquid crystal element 180. One electrode of the liquid crystal element 180 is electrically connected to one electrode of the capacitor 181. To be continued.

[0080] The other of the source and the drain of the transistor 162 is electrically connected to the wiring 126 . The gate of the transistor 162 is electrically connected to the wiring 134. The electrode is electrically connected to a wiring 182. The other electrode of the liquid crystal element 180 is electrically connected to a wiring 183. and electrically connected to each other.

[0081] A constant potential can be supplied to the wiring 182 and the wiring 183. The line 183 can be supplied with a low potential, for example.

[0082] When the pixel 24 provided in the display unit 22 has the configuration shown in FIG. 2B2, the liquid crystal element 180 The liquid crystal molecules contained therein are aligned according to the voltage applied between the two electrodes of the liquid crystal element 180. The molecules direct light, for example from a backlight unit that may be included in the display device 20. As described above, the other electrode of the liquid crystal element 180 is The liquid crystal element 180 is electrically connected to the line 183 and is supplied with a constant potential. By controlling the potential of the electrode, the pixel 24 emits light with a brightness corresponding to the potential. Therefore, an image can be displayed on the display unit 22.

[0083] FIG. 3A shows the content image data CG input to the input unit 21. IN directly to the display unit 22. When the input is made, the content image G_1 displayed on the display unit 22 DP Schematic diagram showing an example of FIG. 3B shows the content image data CG IN The machine learning processing unit 23 and the bright spot correction When input to the display unit 22 via the main unit 50, a content image displayed on the display unit 22 G_2 DP FIG.

[0084] Content image data CG IN If the image is input to the display unit 22 without image processing, As described above, uneven display and bright spots may occur. 2 shows the appearance of display irregularities 25 and bright spots 51 in the displayed image.

[0085] Content image data CG IN For this, image processing is performed using the machine learning model MLM. By doing so, the machine learning processing unit 23 obtains content image data C that cancels out the display unevenness. G ML In FIG. 3B, the machine learning processing unit 23 generates a content image data. Data CG IN Among these, data corresponding to the area 26 is data that cancels out the display unevenness 25 By adding ML This shows how to generate an example. For example, when the brightness of the part where the display unevenness 25 occurs is higher than the brightness of the surrounding area of ​​the part, , the brightness of the region 26 can be lower than the brightness of the periphery of the region 26 .

[0086] Furthermore, the content image data is subjected to bright spot correction image data BCG (for example, bright spot correction Image processing based on either image data for correction BCG_1 or image data for correction BCG_2 By performing this process, the bright spot correction unit 50 corrects the bright spots to make them less noticeable. Tsu image data CG COR For example, a contrast that darkens bright spots can be generated. Content image data CG COR In FIG. 3B, the content image data CG ML Among them, data corresponding to an area 52 where a bright spot 51 occurs is added to make the bright spot 51 less noticeable. By adding data that improves the brightness, the bright spot correction unit 50 corrects the content image data CG COR This shows how to generate

[0087] As shown in FIG. 3B, the content image data CG IN Against Then, the machine learning processing unit 23 and the bright spot correction unit 50 perform image processing, and the display unit 22 displays Therefore, an image can be displayed in which display unevenness and bright spots are not noticeable.

[0088] As described above, as the display device 20 becomes larger and the area of ​​the display section 22 becomes larger, the display unevenness increases. In addition, the pixels 24 provided in the display unit 22 are becoming smaller, and the image quality of the display unit 22 is becoming smaller. When the pixel density is increased, the characteristics of the display element and transistors of the pixel 24 are deteriorated. 4, the variation between the display elements 4 becomes large, and the display unevenness is likely to occur. As described above, the display unevenness displayed on the display unit 22 can be made less noticeable. This prevents the display unevenness from being visible in the image displayed on the display unit 22, and In addition, the display unevenness of the image displayed on the display unit 22 is not visible. The density of the pixels 24 provided in the display section 22 is increased while suppressing the occurrence of a problem, and high-precision display is provided in the display section 22. It is possible to display detailed images.

[0089] Furthermore, as described above, the display element, transistor, etc., included in the pixel 24 may have poor characteristics or may have deteriorated. For these reasons, some pixels 24 may become bright spots or dark spots. When viewing an image displayed on the display unit 22, bright points are more noticeable than dark points, and therefore, there is an effect on visibility. In one aspect of the present invention, the bright spot correction unit 50 or the like corrects the bright spot to, for example, a dark spot. By doing so, it is possible to display a high-quality image on the display unit 22. This can also be performed by the machine learning processing unit 23.

[0090] The image processing system 10 shown in FIG. 1 has a function for generating a machine learning model MLM. The generation device 40 is provided with a learning unit 46 for performing processing using the machine learning model MLM. The display device 20 can be provided with a machine learning processing unit 23 having the above-mentioned functions. Even if the display device 20 does not generate the machine learning model MLM, the display device 20 can display the machine learning model MLM. The machine learning model MLM is generated using a large number of image data sets for learning. Data LG IN , and learning image data LG GEN etc., and high computing power is required. From the above, by providing the learning unit 46 in the generation device 40, The computing power may be less than that of the generating device 40 .

[0091] <How to generate a machine learning model> The method for generating the machine learning model MLM will be explained below with reference to the drawings. As shown in FIG. 2A, the display unit 22 has pixels 24 arranged in a matrix of m rows and n columns. The gradation values ​​of the image data are 8-bit digital values. The smaller the gradation value, the lower the brightness of the light emitted from the pixel 24. If the possible values ​​are integers between 0 and 255, then the gradation value 0 is The brightness of the light is at its lowest.

[0092] [Table generation method] FIG. 4 is a flowchart showing an example of a method for generating a table T stored in the database 42. As shown in FIG. 4, the table T is 5A and 5B, and 6A and 6B are generated by the method of step S01. 10A to 10C are schematic diagrams showing the operations in steps S04 to S05.

[0093] To generate a table T, first, the database image data DG IN Display device 2 The database image data DG input to the input unit 21 is IN is input to the display unit 22, and the database image data DG IN Compatible with Specifically, the image data in the database DG I N The pixel 24 emits light of a luminance corresponding to the first gradation value of the mth row and nth column. The image is then displayed on the display unit 22 .

[0094] In FIG. 5A, the database image data DG IN represents an image with the same brightness across the entire surface. In other words, it is assumed that all pixels 24 emit light of the same brightness. The image displayed on the display unit 22 is formed by the difference in brightness between the light emitted from some pixels 24 and the other pixels. The brightness of the light emitted from the pixel 24 is different. In other words, uneven display occurs. In 5A, the generated display unevenness is shown as display unevenness 27.

[0095] Next, the image displayed on the display unit 22 is captured by the imaging device 30. 30 is the image data IMG DG is acquired (step S02).

[0096] Here, when an image displayed on the display unit 22 of the display device 20 is captured by the imaging device 30, In this case, an object other than the display unit 22 may be captured. For example, the housing of the display device 20 may be captured. In FIG. 5A, the image data IMGDG In this example, the display device 20 shown in FIG. The part enclosed by the dashed line is included in the above.

[0097] Thereafter, the image extracting unit 43 extracts the image data IMG DG From database image data DG DP Specifically, the image data IMG DG Then, the display unit 22 The data representing the displayed image is extracted. For example, as shown in FIG. 5B, In addition to the image displayed on the display device 20, the housing of the display device 20 also stores image data IMG DG If it is included in The data representing the image displayed on the display unit 22 is referred to as image data IMG DG Extracted from the table The data representing the housing of the display device 20 is removed. DG DP Get the image data IMG DG Extracting data from the pattern, as mentioned above, This can be done by pattern matching, template matching, etc.

[0098] As mentioned above, the database image data DG IN The image resolution and the database image Image Data DG DP It is preferable that the resolution of the image represented by is equal to that of the database. Image data DG IN has m rows and n columns of first gradation values, DG DP It is preferable that the second gray scale values ​​are arranged in m rows and n columns. The data representing the image displayed on the display unit 22 is taken as image data IMG DG When extracted from The gradation values ​​of the extracted data may not be in m rows and n columns. For example, There are cases where the number of rows of gradation values ​​is small, or cases where the number of rows of gradation values ​​is greater than m. Also, if there are only grayscale values ​​for fewer than n columns, or if there are grayscale values ​​for more than n columns, It may have a value.

[0099] As mentioned above, the image data IMG DG If the gradation values ​​of the data extracted from are m rows and n columns, If not, up-conversion or down-conversion is performed on the data. The extraction unit 43 performs the process to output the database image data D G DP It is preferable that the image has m rows and n columns of second gradation values. The output 43 is the image data IMG DG The data extracted from has fewer than m rows of gradation values, Or, if the number of columns is less than n, the image extraction unit 43 extracts the image data IMG DG Data extracted from Furthermore, the image extraction unit 43 can perform up-conversion on the captured image data. TaIMG DG If the data extracted from has more than m rows or more than n columns of gradation values The image extraction unit 43 extracts the image data IMG DG Down-converted data extracted from As a result, the database image data DG DP Included in The number of rows and columns of the second gradation values ​​is set to the database image data DG IN The first gradation included in The number of rows and columns of the values ​​can be m rows and n columns. Up-conversion and down-conversion are performed using the nearest neighbor method, bilinear method, and bilinear method. This can be done by the cubic method or the like.

[0100] And the database image data DG IN and database image data DG DP and, versus A table T representing information about the responses is stored in the database 42 (step S04). As mentioned above, the table T is specifically a database of image data DG IN The first The gradation value of 1 and the database image data DG DP The second gradation value of The table T represents, for example, information corresponding to the first gradation value and the coordinate of the first gradation value. The table T represents, for example, the first gradation value and the second gradation value of the corresponding coordinate. The coordinates of the gradation value of the first gradation value and the second gradation value of the same coordinates are represented.

[0101] 6A is a diagram showing an example of a table T. In FIG. 6A, the left side of the arrow indicates a database Image data DG IN The first gradation value of the image data is shown to the right of the arrow. TaDG DP 2 shows the second gradation value that the

[0102] Here, the pixel 24 has the function of emitting red light, green light, and blue light. In this case, the image data is a gradation value that represents the brightness of red light (red gradation value) and a green The gradation value that represents the brightness of colored light (green gradation value) and the gradation value that represents the brightness of blue light (blue gradation value) ) For example, the red gradation value of the i-th row and j-th column is R, the green gradation value is G, and the blue gradation value is is B, and [R,G,B] (i,j) For example, the first row and first column The red gradation value of the mth row and nth column is R, the green gradation value is G, and the blue gradation value is B. All red gradations are R, all green gradations are G, and all blue gradations are B. R, G, B](1,1)~(m,n) The light emitted from the pixel 24 is expressed as follows: The pixel 24 is not limited to red light, green light, and blue light. For example, the pixel 24 may emit white light. Alternatively, the pixel 24 may emit cyan light, magenta light, and yellow light. Also, the pixel 24 does not have to emit red light, green light, or blue light. Furthermore, the number of colors of light emitted by the pixel 24 is not limited to three, but may be, for example, one or two colors of light. Alternatively, light of four or more colors may be emitted.

[0103] When the red, green, and blue gradation values ​​are each 8-bit digital data, In this case, the values ​​that R, G, and B can take can be integers between 0 and 255. If the luminance of light emitted from the pixel 24 is assumed to be lower as the gradation value is smaller, Therefore, for example, in all pixels 24, , [0,0,0] means that no red light, green light, or blue light is emitted. (1,1)~(m,n) It can be shown by writing:

[0104] From the above, the database image data DG IN The first gradation value is [0,0,0] (1 ,1)~(m,n) If so, light is emitted from pixels 24(1,1) to 24(m,n). Therefore, the database image data DG DP has The second tone value is also [0,0,0] (1,1)~(m,n) It can be said that:

[0105] In the method for generating a machine learning model using the image processing system 10, the first gradation value is [1,0 ,0] (1,1)~(m,n)〜[255,0,0] (1,1)~(m,n) It is Database image data DG IN are input to the display device 20, and the database image data TaDG DP Also, if the first gradation value is [0,1,0] (1 ,1)~(m,n) 〜[0,255,0] (1,1)~(m,n) A database that is Image data DG IN are input to the display device 20, and the database image data DG DP Furthermore, if the first gradation value is [0,0,1] (1,1)~ (m,n) 〜[0,0,255] (1,1)~(m,n) The database image data TaDG IN are input to the display device 20, and the database image data DG DP has The second gradation value is obtained. That is, the database image data DG IN The image represented by For example, the entire surface can be a monochromatic image with the same brightness.

[0106] In this specification, the term "monochromatic" means that a pixel emits light of one color to be displayed. For example, a pixel can emit red light, green light, and blue light. , the red image, the green image, and the blue image are called monochrome images.

[0107] In this specification, for example, database image data DG IN in the i-th row and j-th column When the red gradation value of the eye is 1, the corresponding database image data DG DP Red in Gradation value R1 DP (i, j) is written as (i, j). For example, the database image data DG IN In the i-th row and j-th column, if the red gradation value is 255, then the corresponding database image Data DG DP The red gradation value in R255 DP It is written as (i,j). For example, database image data DG IN If the green gradation value of the i-th row and j-th column is 1 in , the corresponding database image data DG DP The green gradation value in G1 DP (i,j) and Also, for example, database image data DG IN In the i-th row and j-th column, If the color gradation value is 255, the corresponding database image data DG DP Green floor in Adjustment value G255 DP (i, j) is written as (i, j). For example, the database image data D G IN If the blue gradation value in the i-th row and j-th column of Data DG DP The blue gradation value in B1 DP (i,j) is written as (i,j). Database image data DG IN If the blue gradation value of the i-th row and j-th column is 255 in , the corresponding database image data DG DP The blue gradation value in B255 DP (i,j ) is written as follows.

[0108] Here, the first gradation value and the second gradation value corresponding to the first gradation value are not necessarily the same. For example, the database image data DG IN The first gradation value of 0] (1,1)~(m,n) Even if it was, R128 DP (1,1) to R128 D P (m,n) is not necessarily always 128. It may be larger than 128 or smaller. In addition, for example, R128 DP (1,1) to R128 DP (m,n)gasu In other words, as mentioned above, the database image data DG DP In some cases, display irregularities may occur in the image represented by the image.

[0109] In addition, for all gradation values, the database image data DG IN is input to the display device 20. , database image data DG DP It is not necessary to acquire the second gradation value of , tone value [1,0,0] (1,1)~(m,n) 〜[255,0,0] (1,1)~( m,n) Among them, some gradation values ​​of the database image data DG IN is input to the display device 20 and database image data DG DP In addition, the second gradation value of the gradation Adjustment value [0,1,0] (1,1)~(m,n) 〜[0,1,0] (1,1)~(m,n) Among them, some gradation values ​​of the database image data DG IN is input to the display device 20, and the data Database image data DG DP Furthermore, the second gradation value [ 0,0,255] (1,1)~(m,n) 〜[0,0,255] (1,1)~(m,n ) Among them, some gradation values ​​of the database image data DG IN is input to the display device 20, and Database image data DG DP may acquire a second gradation value that the

[0110] As mentioned above, the database image data DG of some gradation values IN is input to the display device 20. , database image data DG DP When acquiring the second gradation value of the display device 20 Database image data DG of gradation values ​​that are not input IN The corresponding database image data TaDG DP The second gradation value of the database image data D G IN and a second gradation value corresponding to the first gradation value. For example, it can be calculated by proportional interpolation. It can be calculated using

[0111] For example, the tone value [127,0,0] (1,1)~(m,n) Database image data D G DP and [129,0,0] (1,1)~(m,n) Database image data DG D P is input to the display device 20, but the gradation value [128,0,0] (1,1)~(m,n) No Database image data DG DP is not input to the display device 20. R127 DP The value of (i,j) is 120, R129 DP Set the value of (i,j) to 124, and R1 28 DP (i,j) to R127 DP (i,j) and R129 DP By proportional interpolation of (i,j) In this case, R128 DP The value of (i,j) can be 122. The green gradation value and blue gradation value can also be calculated in the same way. .

[0112] Database image data DG for some gradation values IN is input to the display device 20. This reduces the number of operations required to generate table T. This allows table T to be generated in a short time.

[0113] In FIG. 6A, the database image data DG IN The image represented by is a monochrome image. In other words, database image data DG IN and database image data DG DP Red gradation of The value, green gradation value, and blue gradation value are to be acquired separately. One aspect of the invention is not limited to this. Figure 6B is a modified example of Figure 6A, and shows the database image data. TaDG IN and database image data DG DP Red gradation value, green gradation value, and blue gradation value of This differs from the case shown in FIG. 6A in that the values ​​are acquired all at once.

[0114] In the case shown in FIG. 6B, the first gradation value is [0,0,0] (1,1)~(m,n) ~[2 55,255,255] (1,1)~(m,n) The database image data DG IN are input to the display device 20, and the database image data DG DP The second gradation value. In other words, the database image data DG IN The image represented by the red gradation value, the green The image data is a database image data set with the same color gradation value and blue gradation value (white image). TaDG DP The red, green, and blue gradation values ​​are obtained. , and table T. Specifically, table T is a database of image data DG IN and the database image data of the coordinates corresponding to the red gradation values. TaDG DP The table T represents the database image data. TaDG IN and a database of green gradation values ​​and coordinates corresponding to the coordinates of the green gradation values. Image data DG DP and the green gradation value of the database. Source image data DG IN and the coordinates of the blue gradation value and the coordinates of the corresponding blue gradation value. , database image data DG DP The table T represents the blue gradation value of, for example, Database image data DG IN The red gradation value of the pixel 101 has the same coordinate as the red gradation value of the pixel 101. The coordinates of the database image data DG DP and the red gradation value of the The table T is, for example, a database of image data DG IN and the green gradation value of The database image data DG with the same coordinates as the value coordinates DP and the green gradation value of Furthermore, table T can be, for example, a database of image data DG IN The blue gradation value of and the database image data DG at the same coordinates as the coordinates of the blue gradation value. DP The blue and the color gradation value.

[0115] Database image data DG IN By making the image represented by the Database image data DG to be input IN Therefore, the number of This reduces the number of calculations required to generate the table T. The table T can be generated in a short time. IN of The red gradation value, the green gradation value, and the blue gradation value do not all have to be the same, and Image data DG IN The gradation value of one color among the red gradation value, the green gradation value, and the blue gradation value of may be different from the gradation values ​​of other colors. IN Red gradation of The value, green tone value, and blue tone value may be different from each other.

[0116] [How to generate a machine learning model] FIG. 7 shows the machine learning model MLM generated using table T stored in database 42. 7 is a flowchart showing an example of a method for generating a machine learning model MLM is generated by the method shown in steps S11 to S16. 9A, 9B, and 10 show the operations in steps S11 to S16. FIG.

[0117] To generate a machine learning model MLM, first, we need to create the training image data LG. IN The display The learning image data LG input to the input unit 21 is I N is input to the display unit 22, and the display unit 22 displays the learning image data LG IN The corresponding image is displayed (step S11). Specifically, the learning image data LG IN has, m The pixel 24 emits light with a brightness corresponding to the grayscale value of the row n column, thereby displaying an image on the display unit 22. FIG. 8A shows a state in which display unevenness 28 occurs in the image displayed on the display unit 22. This shows the attitude.

[0118] Next, the image displayed on the display unit 22 is captured by the imaging device 30. 30 is the image data IMG LG is acquired (step S12).

[0119] As described above, the image displayed on the display unit 22 of the display device 20 is captured by the imaging device 30. In this case, an object other than the display unit 22 may be captured. In FIG. 8A, the image data IMG LG In this case, the display device shown in FIG. The part of position 20 enclosed by the dashed line is included.

[0120] Thereafter, the image extracting unit 43 extracts the image data IMG LG Learning image data LG DP Get (Step S13). The operation in step S13 is DG Captured Data IMG LG and database image data DG DP The learning image data LG DP 5B is replaced with FIG. 8B, the operation in step S03 You can refer to the explanation in

[0121] Next, the learning image data LG IN On the other hand, the learning image data LG DP The image is drawn to get closer to the The image processing unit 44 performs image processing. IP (Step S14). FIG. 9A shows the learning image input to the image processing unit 44. DataLG IN and learning image data LG DP and the learning image output from the image processing unit 44. Image data LGIP An example of this is shown below.

[0122] Learning image data LG IP The generation of learning image data LG IP Sum of the tonal values ​​of and learning image data LG DP The difference between the sum of the gradation values ​​of and is the learning image data LG IP The sum of the gradation values ​​of the learning image data LG IN is smaller than the difference between the sum of the gradation values ​​of To make it smaller, the learning image data LG IN The gradation values ​​of the image are converted by image processing. For example, the learning image data LG IP The sum of the gradation values ​​of Image data LG DP The learning image data LG IN The learning image data LG IP of It can be generated.

[0123] In addition, the learning image data LG IP The generation of learning image data LG DP against PSNR or SSIM is IN PSNR or SSIM for To make it larger, the learning image data LG IN Convert the gradation values ​​of For example, the learning image data LG DP PSNR for is set to maximize SSIM by using the training image data LG IN The gradation values ​​of By converting it further, the learning image data LG IP can be generated.

[0124] As mentioned above, the image processing performed by the image processing unit 44 can be, for example, gamma correction. In this case, the above image processing can be performed by setting the gamma value to an appropriate value. do.

[0125] Here, the image processing unit 44 receives the learning image data LG IN Image processing for each color is performed Specifically, for example, the learning image data LG IN Gamma correction for When performing this, it is preferable to calculate the gamma value for each color. For example, IP and the sum of the red gradation values ​​of the learning image data LG DP The sum of the red gradation values ​​of The difference between and is the learning image data LG IP The sum of the red gradation values ​​of the learning image data LG IN The learning image data L is calculated so that the sum of the red gradation values ​​of G IN It is preferable to convert the red gradation value of the learning image by image processing. DataLG IP and the sum of the green gradation values ​​of the learning image data LG DP Green floors owned by The difference between the sum of the adjustment values ​​and the learning image data LG IP The sum of the green gradation values ​​of Image data LG IN The sum of the green gradation values ​​of and is smaller than the difference between Image data LG IN It is preferable to convert the green gradation value of , learning image data LG IP and the sum of the blue gradation values ​​of the learning image data LG DP but The difference between the sum of the blue gradation values ​​and the learning image data LG IP The sum of the blue gradation values ​​of Total and learning image data LG IN The sum of the blue gradation values ​​of and is smaller than the difference between 2, learning image data LG IN It is preferable to convert the blue gradation value of the For example, the learning image data LG IP The sum of the red gradation values ​​of the training image data TaLG DP The learning image data LG IN but It is preferable to convert the red gradation value by image processing. G IP The sum of the green gradation values ​​of the learning image data LG DP The sum of the green gradation values ​​of The learning image data LG IN The green gradation value of It is preferable to convert the learning image data LG IP The sum of the blue gradation values ​​of However, the learning image data LG DP The total blue gradation value of the training image is set to be equal to the total blue gradation value of the training image. Image data LG IN It is preferable to convert the blue gradation value of the color by image processing.

[0126] As mentioned above, "Learning Image Data LG IP The "total of gradation values ​​of the learning image" is, for example, Image data LG IP It may be the sum of all the gradation values ​​of m rows and n columns of Alternatively, it may be the sum of some of the gradation values. IN The gradation that For example, the sum of the learning image data LG IN All of the m rows and n columns of gradation values ​​that It may be the sum of all the gradation values, or the sum of some of the gradation values. Image data LGDP The "total of gradation values ​​of the learning image data LG DP has The sum of all the gradation values ​​in the m rows and n columns may be used, or the sum of some of the gradation values ​​may be used. It may also be possible to use the following.

[0127] In this specification, learning image data LG IP The red gradation value of the i-th row and j-th column of IP (i, j) are written as (i, j). IP The i-th row and j-th column of The green gradation value of IP (i, j) are written as (i, j). Furthermore, the learning image data LG IP but The blue gradation value in the ith row and jth column is B IP This is indicated as (i,j).

[0128] As described above, the learning image data LG output from the image processing unit 44 IP The image represented by Learning image data LG DP On the other hand, when image processing is performed by the image processing unit 44, the image becomes similar to the Learning image data LG IN Since the image shown by does not contain display irregularities28, Learning image data LG IP The image represented by does not include the display irregularity 28.

[0129] Thereafter, based on the table T, the image generating unit 45 generates the learning image data LG IP For learning from Image data LG GEN 9B shows the image generated by the image generating unit 45 (step S15). Bull T and learning image data LG IP is input, and the learning image data LG GEN is output This shows how it works.

[0130] Specifically, first, the learning image data LG IP Based on the gradation value of For example, select the gradation value of 2 for the learning image data LG IP A value that matches the tone value of The second gradation value with the closest value is selected for each of the first row, first column to the mth row, nth column. Specifically, for example, R IP The red gradation value that matches or is closest to (i,j) is R 0 DP (i,j) to R255 DP (i,j). Also, for example, G IP ( i,j) or the green gradation value of the closest value is set to G0 DP (i,j) to G25 5 DP (i,j). Furthermore, for example, B IP The value that matches (i,j), or is the closest blue shade, B0 DP (i,j) to B255 DP Among (i,j) In FIG. 9B, the selected second gradation value in the first row and first column is set as [Ra DP (1 ,1),Gb DP (1,1),Bc DP (1,1)] (1,1) (a, b, c are 0 (an integer greater than or equal to 255). Also, the second gradation value selected in the mth row and nth column is expressed as [Rs DP ( m,n),Gt DP (m,n),Bu DP (m,n)] (m,n) (s, t, u are an integer between 0 and 255 inclusive).

[0131] Next, the image generating unit 45 generates a second gradation value corresponding to the selected second gradation value based on the table T. Learning image data LG, which is image data including a gradation value of 1 GEN Specifically, For example, the image generating unit 45 generates a second gradation value corresponding to the selected first row, first column to m-th row, n-th column. The learning image data is image data including the first gradation values ​​in the first row and the first column to the mth row and the nth column corresponding to the first gradation values. Data LG GEN For example, in the example shown in FIG. 9B, the learning image data LG GEN In the first row and first column, the red gradation value is a, the green gradation value is b, and the blue gradation value is c. The red gradation value of the mth row and nth column is s, the green gradation value is t, and the blue gradation value is u.

[0132] For example, the learning image data LG IP The value in the i-th row and j-th column of If the second gradation value is not included in table T, the second gradation value in the i-th row and j-th column must not be selected. In this case, the learning image data LG GEN The tone value in the i-th row and j-th column of the training image DataLG IP It can be set to the same value as the gradation value in the i-th row and j-th column of BaR IP The red gradation value that matches (i,j) is R0 DP (i,j) to R255 DP (i ,j), if it is not included in the learning image data LG GEN The red gradation value of the i-th row and j-th column of is R IP (i,j). Also, for example, G IP Green matches (i,j) Color gradation value is G0 DP (i,j) to G255 DP If it is not included in (i,j), Learning image data LG GEN The green gradation value in the ith row and jth column of IP Let (i,j) Furthermore, for example, B IP The blue gradation value that matches (i,j) is B0 DP (i,j ) to B255 DP If it is not included in (i,j), the learning image data LG GEN of The blue gradation value of the ith row and jth column is B IP It can be (i,j).

[0133] As mentioned above, the learning image data LG IP The image shown does not include display irregularities. On the other hand, database image data DG having a second gradation value DP includes display irregularities, etc. Also, database image data DG having a first gradation value IN includes display irregularities, etc. Therefore, the learning image data LG IP The second tone value is selected based on the tone value of Learning image data LG, which is image data including the corresponding first gradation value GEN is for learning purposes Image data LG DP This allows the image data to be created in a way that cancels out the unevenness of the display that appears on the screen. In FIG. 9B etc., the learning image data LG GEN Of these, the data corresponding to area 29 , the display unevenness 28 is eliminated. If the brightness of the portion where unevenness 28 occurs is higher than the brightness of the surrounding area of ​​the portion, the area 29 The brightness of the area 29 can be lower than the brightness of the periphery of the area 29.

[0134] After step S15, the learning unit 46 uses the learning image data LG IN and the learning image data L G GEN A machine learning model MLM is generated using the above (step 16). Learning image data LG IN The image data output when input is the learning image data L G GENWe generate a machine learning model MLM that matches the above. MLM is, for example, a learning image data LG IN and learning image data LG GEN and, the teacher The learning image data L G IN and learning image data LG GEN and are input to the learning unit 46 and output from the learning unit 46. The image data to be used is the learning image data LG GEN The same image data as the machine learning This shows how to generate a machine learning model (MLM). For example, a neural network model can be applied as M.

[0135] The above is an example of a method for generating a machine learning model MLM using the image processing system 10. As described above, the machine learning model MLM is used to perform the processing on the image data input to the input unit 21. By performing image processing using the image processing unit 23, the machine learning processing unit 23 can generate image data that cancels out the display unevenness. The image data output from the machine learning processing unit 23 is displayed on the display unit 2. 2, the display unit 22 can display an image in which the display unevenness is made less noticeable. Therefore, as described above, display unevenness is visually recognized in the image displayed on the display unit 22. The display device 20 can be enlarged while suppressing the problem of the image being displayed on the display unit 22. The display unit 22 is provided with a pixel 24, and the pixel 24 is provided with a pixel 24a. The density can be increased, and high-definition images can be displayed on the display unit 22.

[0136] In addition, by using the machine learning model MLM to process images, the display unevenness can be reduced as mentioned above. In addition to this, it can also counteract elements that degrade the quality of the displayed image. For example, lines Therefore, the display unit 22 can display high-quality images. It is possible.

[0137] Here, image data is input to the input unit 21, and the same processing as in steps S11 to S15 is carried out. When image data generated by the image generating unit 45 through processing is input to the display unit 22 Even if the display unit 22 is in a low-luminance state, it is possible to display an image in which display irregularities and the like are not noticeable. However, performing steps S11 to S15 requires high computing power. On the other hand, image processing using the generated machine learning model MLM is performed in steps S11 to S15. This can be done with lower computational power than step S15. By using this, image processing can be performed in a short time. Image processing can be performed within the display device 20 without using a device with high computing power. Cut.

[0138] In addition, the image data input to the input unit 21 may be adjusted to make the display unevenness less noticeable. If the image processing can be performed by a device with high computing power, for example, Image processing is performed in the same manner as steps S11 to S15 without using the machine learning model MLM. Also, if the computing power of the display device 20 is sufficiently high, the input image processing for making display unevenness less noticeable for image data input to unit 21, Without using the machine learning model MLM, the same method as steps S11 to S15 is used. When performing image processing without using the machine learning model MLM, the generating device 40 The learning unit 46 may not be included.

[0139] <Bright spot correction method> The bright spot correction method, which is an image processing method according to one embodiment of the present invention, will be described below with reference to the drawings. explain.

[0140] In the bright spot correction method according to one aspect of the present invention, after detecting the pixel 24 that will be a bright spot, 11A and 11B show a method for detecting a pixel 24 that is a bright spot. 11A is a diagram showing an example of a method M1, and FIG. This method is referred to as method M2. In addition, in FIGS. 11A and 11B, it is assumed that there is no display unevenness. is doing.

[0141] In method M1, first, the bright spot correction image data BCG_1 IN The input of the display device 20 Then, the learning image data LG IN Bright spot correction image data BCG _1 IN And the image data IMG LG Image data IMG BCG and learning image data LG DP Bright spot correction image data BCG DP and learning image data LG IP Bright spot correction for image Data BCG IP and learning image data LG GEN The image data for bright spot correction is BCG_1. These steps are replaced with steps S11 to S15 shown in FIG. 7, etc. (Steps S11' to S15'). For example, steps S11' and S15' In S12', the bright spot correction image data BCG_1 IN The image corresponding to the The image displayed on the display unit 22 is captured by the imaging device 30, and the captured image data is IMG BCG In addition, the bright spot correction image data acquired in step S13' is acquired. BCG DP can have m rows and n columns of grayscale values. In the database image data DG IN and a database image Data DG DP Based on the table T representing information on the correspondence between the second gradation value of The image generating unit 45 generates the bright spot correction image data BCG IP Bright spot correction image data BCG The bright spot correction image data BCG is generated. DP is not supplied to the image processing unit 44, When the image is supplied to the image generating unit 45, the operation shown in step S14' is not performed. In step S15', the image generating unit 45 generates the bright spot correction image data BCG DP Bright spot from Correction image data BCG_1 is generated.

[0142] Here, if the pixel that becomes the bright spot is included in the pixels 24(1,1) to 24(m,n), In this case, the bright spot correction image data BCG DP Among the grayscale values ​​of m rows and n columns of The gradation value of the coordinates corresponding to the coordinates of pixel 24, for example, the gradation value of the coordinates identical to pixel 24 that is the bright point, is For example, the bright spot correction image data BCG DP The gradation values ​​of 255, the coordinates corresponding to the coordinates of the pixel 24 that is the bright spot The gradation value is 255 or a value close to it. DP If the gradation value of a certain coordinate in is higher than the gradation value of the coordinates around the coordinate, In the image data BCG_1, the gradation value can be lowered.

[0143] After step S15', the bright spot correction unit 50 of the display device 20 receives the bright spot correction image data Based on BCG_1, the coordinates of the pixel 24 that becomes the bright spot are detected. Specifically, Among the gradation values ​​of m rows and n columns of the bright spot correction image data BCG_1, the gradation values ​​that are below the threshold value are The coordinates of the adjustment values ​​can be used as the bright spot coordinates. The bright spot coordinates are detected by the generating device 40. The detection of the bright spot coordinates may be performed by, for example, the image generating unit 45 of the generating device 40. You may go.

[0144] Here, the bright spot correction image data BCG DP , and both the bright spot correction image data BCG_1 In this case, if the difference between the gradation value of the bright spot coordinate and the gradation value of the coordinates around the bright spot coordinate is large, the bright spot This is preferable because the point coordinates can be detected with high accuracy. BCG_1 IN It is preferable that the gradation value of is an intermediate gradation. Image data BCG_1 IN All of the m rows and n columns of gradation values ​​of are set to 127 or its vicinity. In FIG. 11A, the bright spot correction image data BCG_1 IN The gradation value of The brightness correction image data BCG DP and the surrounding area as bright spots. 11A shows an example in which a gradation value 53 higher than the gradation value 53 is included. Among the gradation values ​​of the image data BCG_1, the gradation value 54 at the same coordinate as the gradation value 53 is It can be made lower than the surrounding gradation value.

[0145] In method M2, first, the bright spot correction image data BCG_2 IN The input of the display device 20 Then, the learning image data LG IN Bright spot correction image data BCG _2 IN And the image data IMG LG Image data IMG BCG and learning image data LG DP is read as the bright spot correction image data BCG_2, and steps S11 to S14 shown in FIG. The same operations as in step S13 are performed (steps S11'' to S13''). For example, in steps S11'' and S12'', the bright spot correction image data BC G_2 IN An image corresponding to the image is displayed on the display unit 22, and the image displayed on the display unit 22 is captured by the imaging device. By capturing an image with the device 30, image data IMG BCG Also, in step S1 The image data for bright spot correction BCG_2 acquired in 3'' has gradation values ​​of m rows and n columns. It is possible.

[0146] Here, if the pixel that becomes the bright spot is included in the pixels 24(1,1) to 24(m,n), In this case, the pixel that becomes a bright spot among the gradation values ​​of m rows and n columns that the bright spot correction image data BCG_2 has The gradation value of the coordinates corresponding to the coordinates of pixel 24, for example, the gradation value of the coordinates identical to pixel 24 that is the bright point, is For example, if the gradation value of the bright spot correction image data BCG_2 is between 0 and 255, the coordinates corresponding to the coordinates of the pixel 24 that is the bright spot The gradation value is 255 or a value close to it.

[0147] After step S13'', the bright spot correction unit 50 of the display device 20 generates the bright spot correction image data. Based on the data BCG_2, the coordinates of the pixel 24 that becomes the bright spot are detected. is the gradation value of m rows and n columns of the bright spot correction image data BCG_2 that is equal to or greater than the threshold value. The coordinates of the gradation values ​​can be used as the coordinates of the bright spots. The detection of the bright spot coordinates may be performed by, for example, the image extraction unit 43 included in the generating device 40. You may go there.

[0148] Here, in the bright spot correction image data BCG_2, the gradation value of the bright spot coordinates and the periphery of the bright spot coordinates are If the difference between the gradation values ​​of the sides and is large, the coordinates of the bright spots can be detected with high accuracy, which is preferable. On the other hand, if the gradation value of the image data input to the display unit 22 is too small, the gradation value Even if a pixel 24 can become a bright spot, it does not become a bright spot, and the coordinates of the bright spot can be detected with high accuracy. Bright spot correction image data BCG_2 IN The gradation value of For example, it is preferable to determine the brightness correction image data BCG_2 IN m All the gradation values ​​in row n and column n are set to 0 or more and 127 or less, or 31 or more and 127 or less, or 63 or more and 1 In FIG. 11B, the image data for bright spot correction BCG_2 is An example is shown in which a point includes a gradation value of 55, which is higher than the surrounding gradation values.

[0149] By detecting the coordinates of the bright spots, the bright spot correction unit 50 can correct the bright spots. For example, content image data CG ML Among the m rows and n columns of gray scale values ​​of The gradation value of the coordinates corresponding to the coordinates, for example, the same coordinates as the bright spot coordinates, can be reduced. For example, it can be set to 0. The bright spot correction unit 50 reduces the gradation value of the coordinates corresponding to the bright spot coordinates. Kushita content image data CG COR is generated and supplied to the display unit 22, For example, the pixel 24 that is a bright point can be made a dark point. When viewing an image, bright points stand out more than dark points, so they have a large impact on visibility. , content image data CG COR By displaying an image corresponding to the This allows the image displayed on the display unit 22 to be of high quality.

[0150] FIG. 12A shows the actual measured gradation values ​​of the bright spot correction image data BCG_1 and the TaBCG_1 IN A graph showing the relationship between the gradation value and the Here, the bright spot correction image data BCG_1 IN The gradation value of the image is the same across the entire surface. In 12A, the same bright spot correction image data BCG_1 IN Multiple processes for the tone value There are lots of batches, but this is because the actual measurement values ​​of the gradation values ​​of the bright spot correction image data BCG_1 are This is because the coordinates are plotted.

[0151] The line 56 shown in FIG. 12A represents the bright spot correction image data BCG_1 IN At each gradation value, 12A represents the average gradation value of the batched image data for bright spot correction BCG_1. , the average gradation value of the bright spot correction image data BCG_1 and the bright spot correction image data BCG_ 1 IN The relationship between the gradation value of and can be linearly approximated.

[0152] Here, as described above, the gradation value of the bright spot correction image data BCG_1 at the bright spot coordinates is The gradation value is lower than that of the bright spot coordinates. IN of A threshold value is set for each gradation value, and for example, the gradation of the image data for bright spot correction BCG_1 of m rows and n columns is Among the values, the coordinates of the gradation values ​​less than the threshold value can be used as the bright spot coordinates. The threshold value is shown by line 57. Line 57 can be expressed by a linear equation with a positive slope. can.

[0153] In the method M1, a plurality of image data for bright spot correction BCG_1 having different gradation values IN Prepare , and by generating the bright spot correction image data BCG_1 for each, Determining that the coordinates of the pixel 24 are not bright spot coordinates, and determining the coordinates of the pixel 24 that are not bright spots Therefore, the bright spot correction unit 50 and the like can suppress the determination that the coordinates of the bright spot are , the coordinates of the bright spots can be detected with high accuracy.

[0154] FIG. 12B shows the bright spot correction image data BCG generated by the image extraction unit 43. DP Measurement of the gradation value value, and the bright spot correction image data BCG input to the input unit 21 IN The relationship between the gradation value and As shown in FIG. 12B, the bright spot correction image data BCG DP and the tone value of Bright spot correction image data BCG IN The relationship between the gradation values ​​cannot be linearly approximated. It is approximated by a gmoid curve.

[0155] Next, an example of a pixel 24 that can be detected by method M2 will be described. 13A2 shows the gradation values ​​of the bright spot correction image data BCG_2 and the bright spot correction image data BCG_3. BCG_2 IN 10 is a graph showing the relationship between the gradation values.

[0156] The graph 61 shown in FIG. 13A1 is a graph of the brightness correction image at the time of manufacturing the display device 20, for example. The average value of the m rows and n columns of the gradation values ​​of the data BCG_2 and the bright spot correction image data BCG_ 2 IN The relationship between the gradation values ​​can be as follows. IN The gradation value of the bright spot correction image data can be set to the same value over the entire surface. Among the gradation values ​​of data BCG_2, some gradation values ​​exhibit the behavior shown in Graph 63. In other words, when the gradation value of the image data input to the display unit 22 becomes high, The brightness of the light emitted from the pixel 24 in the image sensor decreases. The pixel 24 behaving in this manner is prone to deterioration, and when the pixel 24 is used for a long period of time, i.e., When a voltage is supplied to the display element of the pixel 24 for a long period of time, the behavior changes as shown in the graph of FIG. On the other hand, pixel 24, which behaves as shown in graph 61, is less likely to deteriorate. Even if used for a long period of time, the behavior shown in graph 63A will not occur.

[0157] The pixel 24 that behaves as shown in graph 63A can be said to be a bright spot. The pixel 24 that exhibits the behavior shown in the graph 63 of FIG. 13A1 is, for example, Although they are not bright spots, they are pixels that are likely to become bright spots as the display device 20 is used. As described above, when the pixel 24 becomes a bright spot, it has a large effect on visibility. The pixel 24 behaving as shown in FIG. 63 is darkened by, for example, reducing the voltage supplied to the display element. This makes it possible to prevent the pixel 24 from becoming a bright spot. Therefore, the reliability of the display device 20 can be improved.

[0158] FIG. 13B is a diagram showing an example of a method for detecting a pixel 24 that exhibits the behavior shown in the graph 63. As shown in FIG. 13B, the bright spot correction image data BCG_2 IN The gradation value of For example, bright spot correction image data BCG_2 IN All of the m rows and n columns of gradation values ​​that , 255, or its vicinity. IN Based on Then, by performing steps S11'' to S13'', the bright spot correction image data BCG Among the gradation values ​​of pixel _2, the gradation value corresponding to pixel 24 exhibiting the behavior shown in graph 63 is In FIG. 13B, the brightness correction image data BCG_2 is 2. The pixel 24 corresponding to the gradation value 65 is included in the graph. The pixel 24 behaves as shown in FIG. 63, that is, the pixel 24 becomes a bright spot after long-term use. It is possible.

[0159] Therefore, by the method M2, not only the pixel 24 that is already a bright point but also the pixel 24 that is already a bright point are used. On the other hand, in method M1, the pixel 24 that is likely to become a bright spot due to the use can be detected. , pixel 2 that exhibits the behavior shown in graph 63 among the gradation values ​​of the bright spot correction image data BCG_1 The gradation value corresponding to 4 is higher than the gradation values ​​of the surrounding areas. Correction image data BCG_1 IN When the gradation value is increased, the image behaves as shown in graph 63. On the other hand, the bright spot correction image data input to the input unit 21 is BCG_1 IN If the gradation value of is lowered, graph 63 becomes closer to graph 61, so graph Therefore, in the method M1, it becomes difficult to detect the pixel 24 that exhibits the behavior shown in 63. It is difficult to detect pixel 24 that behaves as shown in graph 63.

[0160] From the above, by performing both method M1 and method M2, for example, pixel 24, which is a bright spot, can be Not only can it be detected with high accuracy, but it can also detect bright spots on the image that are likely to become bright spots as the display device 20 is used. In this way, by performing both method M1 and method M2, For example, it is possible to comprehensively detect pixels 24 that should be dark points.

[0161] FIG. 14 shows the actual measured gradation values ​​of the bright spot correction image data BCG_1 and the BCG_1 IN 5 is a graph showing the relationship between the gradation value and the line 57A instead of the line 57, and The graph differs from that shown in FIG. 12A in that line 57B is included.

[0162] In FIG. 14, the gradation values ​​of the bright spot correction image data BCG_1 are shown by the line 57A. The gradation value indicated by the line 57B is the first threshold value, and the gradation value indicated by the line 57C is the second threshold value. The first threshold is less than the value indicated by line 56, and the second threshold is less than the first threshold. Like line 57, lines 57A and 57B can be expressed by linear equations with positive slopes. do.

[0163] An example of an image processing method using the image processing system 10 will be described with reference to FIG. The graph shown in Fig. 14 is created by the method M1. In the graph shown in Fig. 14, for example, Among the gradation values ​​of the image data for bright spot correction BCG_1 of m rows and n columns, those that are equal to or less than the first threshold value, and The coordinates of the gradation value equal to or greater than the second threshold value are defined as the first bright point coordinates. The coordinates of the gradation value are set as the second bright point coordinates.

[0164] Furthermore, the bright spot coordinates are detected by the method M2, and the detected bright spot coordinates are set as the third bright spot coordinates.

[0165] And content image data CG ML is input to the bright spot correction unit 50, Tsu image data CG ML Among the m rows and n columns of gray scale values ​​that are the same as the first bright point coordinates, The gradation value of the coordinates that are the same as the third bright point coordinates is reduced. Regardless of whether the coordinates of the first bright point are the same as the coordinates of the second bright point, the gradation value of the coordinates that are the same as the coordinates of the first bright point is reduced. This makes it possible to correct bright spots and display high-quality images on the display unit 22. It can be done.

[0166] By performing image processing using the above method, it is possible to improve visibility without creating dark spots, for example. This can prevent the pixels 24 that are not affected from becoming dark points. The quality of the image displayed on the display unit 22 is lowered by making the pixel 24 a dark point. In the example shown in FIG. 14, for example, the first pixel 24 of the m-th row and n-th column The pixel 24 having the same coordinates as the first bright point but different from the third bright point coordinates is not regarded as a dark point. Therefore, even if the content is Tsu image data CG ML Among the m rows and n columns of gray scale values ​​of However, the gradation values ​​at coordinates different from the third bright point coordinates may not be corrected.

[0167] The above is an example of the bright spot correction method, which is an image processing method according to one embodiment of the present invention.

[0168] <Example of machine learning model configuration> FIG. 15A is a diagram showing an example of the configuration of a machine learning model MLM. As shown in FIG. The machine learning model MLM consists of an input layer IL, a hidden layer ML1, a hidden layer ML2, and a hidden layer ML3. The neural network model can be configured as an input layer OL and an output layer OL. The IL, hidden layer ML1, hidden layer ML3, and output layer OL are composed of neurons. The input layer IL has multiple layers, and the neurons in each layer are connected to each other. Image data can be input to the .

[0169] The image data input to the input layer IL is a matrix of m rows and n columns, which is represented by the sub-pixels of the display unit 22. For example, if the pixel 24 emits red (R) light, A pixel has a sub-pixel that emits green (G) light and a sub-pixel that emits blue (B) light. When doing this, the image data is a matrix of m rows and n columns with red gradation values ​​as components and green gradation values ​​as components. and an m row n column matrix with blue gradation values ​​as components. The image data can be structured to have three matrices.

[0170] Assuming that the image data contains matrices as above, the number of neurons in the input layer IL is can be set to the same number as the number of elements in the matrix. For example, if the image data is 12 If there are three matrices with 00 rows and 1920 columns, the number of neurons in the input layer IL is 19 It can be 20 x 1200 x 3. Also, if the image data contains a matrix, The number of neurons in the output layer OL can be set to the same number as the number of elements in the matrix. For example, if the image data has three matrices with 1200 rows and 1920 columns as described above, In this case, the number of neurons in the output layer OL can be 1920 × 1200 × 3. .

[0171] The intermediate layer ML1 has a function of generating data D1 to be supplied to the intermediate layer ML2. 1 can be a matrix having h elements x (h is an integer greater than or equal to 2).

[0172] In this specification, for example, h components x are referred to as components x1 to x h and write Other ingredients will be described in the same way.

[0173] The number of neurons in the hidden layer ML1 is greater than the number of neurons in the input layer IL. As a result, the number of components contained in the data D1 is calculated based on the number of components contained in the image data input to the input layer IL. The number of components that can be processed by the hidden layer ML1 can be increased by one. More on this later.

[0174] The hidden layer ML2 has a function of converting a component x into a component y. For example, the hidden layer ML2 The component x1 to the component x2 are calculated by a nonlinear polynomial function of one variable. h The components y1 to y2 are Minutes h An example of this function is shown below.

[0175]

number

[0176] where i can be an integer between 1 and h. The above formula uses the component x as the independent variable, It is a function with component y as the dependent variable and a as the coefficient. The function is x to the dth power (d is 2 or more). In FIG. 15A, the hidden layer ML2 performs the calculation process shown in the above formula. They plan to do the following.

[0177] Another example of a nonlinear polynomial function of one variable is shown below.

[0178]

number

[0179] The above formula is a function with x as the independent variable, y as the dependent variable, and a and b as coefficients. The number has a term containing the cosine of the component x and a term containing the sine of the component x. Note that the function is The function may not have a term containing the cosine of the component x. It is not necessary for the term to have a value that includes the term.

[0180] As a result, the hidden layer ML2 is composed of components y1 to y h It is possible to generate data with The data is called data D2. Data D2 is a matrix, just like data D1. It is possible.

[0181] The hidden layer ML3 has the function of generating data to be supplied to the output layer OL. The number of neurons in the output layer OL is set to be greater than the number of neurons in the output layer OL. The number of components in the image data output from the output layer OL is set to be greater than the number of components in the data D2. For details of the calculation process that the hidden layer ML3 can perform, see This will be discussed later.

[0182] It should be noted that two or more intermediate layers may be provided between the input layer IL and the intermediate layer ML2. Two or more intermediate layers may be provided between the intermediate layer ML2 and the output layer OL.

[0183] FIG. 15B shows the machine learning model MLM when the machine learning model MLM has the configuration shown in FIG. 15A. 7 is a diagram illustrating an example of a method for generating an MLM. As shown in FIG. 7, the machine learning model MLM is 15A. This can be said to be a diagram illustrating an example of the operation of step S16 in the case of the configuration shown in FIG. .

[0184] As described above, the machine learning model MLM can be generated by the learning unit 46. For example, the learning image data LG IN and learning image data LG GEN Using the and, For example, a machine learning model MLM can be generated using the learning image data LG IN Enter When this is done, the image data output is the learning image data LG GEN The coefficient a is set to match 1,0 or coefficient a n,k By acquiring values ​​of the parameter by learning, the learning unit 46 obtains the values ​​of the parameter by learning. In addition, when the hidden layer ML2 performs the calculation shown in Equation 2, In addition to the value of coefficient a, the value of coefficient b is also obtained through learning.

[0185] 16A and 16B are diagrams showing the results of the machine learning processing unit 23 to which the machine learning model MLM is applied. As shown in FIG. 16A, the hidden layer ML1 calculates the content. Image data CG IN and the filter fa, and the product-sum operation can be performed. In the example shown in 6A, the content image data CG IN is a 1200-by-1920 matrix in 3 In other words, the content image data CG INThe width is 1920 and the height is 12 The data is 00, and the number of channels is 3. The number of channels of the filter fa is 3, and the intermediate layer ML1 allows content image data CG IN and 9 filters fa (filter fa1 To perform such a product-sum operation, From the middle layer ML1, data D1 with a height of 1200, width of 1920, and number of channels of 9 is sent. The data D1 can be output as components x1 to x 1920×1200×9 of This includes:

[0186] The components x1 to x2 of the data D1 1920×1200×9 In the hidden layer ML2, Then, the components y1 to y 1920×1200×9 Convert to Component y1 to component y 1920×1200×9 Data D2 Let's say.

[0187] As shown in FIG. 16B, the hidden layer ML3 performs a product-sum operation on the data D2 and the filter fb. Here, data D2 has a height of 1200 and a width of 1200, just like data D1. The data can be 1920 wide and 9 channels. The number is set to 9, and the hidden layer ML3 generates data D2 and three filters fb (filter fb1 To perform such a product-sum operation, As a result, the middle layer ML3 outputs data with a width of 1920, a height of 1200, and three channels. The data can be input as content image data CG ML It can be said that .

[0188] From the above, using the machine learning model MLM, for example, content image data CG IN Con Content image data CG ML can be converted to

[0189] As described above, in the machine learning model MLM having the configuration shown in FIG. 15A, in the hidden layer ML2, , components x1 to x included in data D1 h Using a nonlinear polynomial function of one variable, and components y1 to y h This allows us to convert, for example, a linear function of one variable, is a function of one variable of a monomial, from component x1 to component x h are the components y1 to y h to Compared to the case of conversion, inference using the machine learning model MLM can be performed with higher accuracy. In addition, the number of filters fa shown in FIG. 16A and the number of channels of the filter fb shown in FIG. 16B Since the number of trained models can be reduced, it is possible to generate machine learning models (MLMs) through training and to The amount of calculation required for inference by the learning model MLM can be reduced. and inference can be performed at high speed. [Example]

[0190] In this example, the results of learning to obtain the machine learning model MLM shown in FIG. This article explains:

[0191] In this example, image data with a width of 1920, a height of 1200, and three channels is used as training data. The machine learning model MLM was generated by supervised learning using the data D and the correct answer data. The component x included in 1 is calculated by the hidden layer ML2 using the formula 1, formula 2, or formula “y=ax+b " was used to convert it into the component y. In Equation 1 and Equation 2, d=5 was used.

[0192] The hidden layer ML1 performs the calculation shown in FIG. 16A, and the hidden layer ML3 performs the calculation shown in FIG. 16B. The component x contained in the data D1 is calculated using Equation 1 or Equation 2. When converting to y, the hidden layer ML1 converts the image data and the filter fa with 3 channels. 1 to filter fa9, and the hidden layer ML3 performs a product-sum operation on data D1 and , and filters fa1 to fa3 having nine channels are used to perform product-sum operations. In addition, when the component x contained in the data D1 is converted into the component y using the formula "y=ax+b", In this case, the hidden layer ML1 generates image data and filters fa1 to f a 162 The hidden layer ML3 performs a multiplication and addition operation on the data D2 and the number of channels The filters fb1 to fb3 of the filter 162 are used to perform a product-sum operation.

[0193] Figure 17 is a graph showing the relationship between SSIM and the number of learning epochs. M was calculated using test data and correct answer data. The larger the SSIM, the better the test accuracy. Since the similarity between the target data and the correct data is high, the machine learning model MLM can make predictions with high accuracy. The test data, like the training data and the correct answer data, has a width of 19 The image data was 20, height 1200, and number of channels 3.

[0194] As mentioned above, the component x included in the data D1 is converted to the component y using Equation 1 or Equation 2. When converted, the filter is filtered more than when component x is converted to component y using the formula “y=ax+b”. The number of filters fa and the number of channels in filter fb are small. As shown, when the number of learning times is 200 or more, the component x is expressed as Equation 1 or Equation 2. If component x is converted to component y using the formula "y=ax+b", then component x is converted to component y using the formula "y=ax+b". The SSIM was larger than when [Explanation of symbols]

[0195] 10: Image processing system, 20: Display device, 21: Input unit, 22: Display unit, 23: Mechanics Learning processing unit, 24: pixel, 26: region, 29: region, 30: imaging device, 33: pixel, 40: Generation device, 42: database, 43: image extraction unit, 44: image processing unit, 45: image generation section, 46: learning section, 50: bright spot correction section, 51: bright spot, 52: area, 53: gradation value, 54: Gradation value, 55: Gradation value, 56: Line, 57: Line, 57A: Line, 57B: Line, 61: Graph, 63: graph, 63A: graph, 65: gradation value, 126: wiring, 134: wiring, 161: Transistor, 162: Transistor, 170: Light-emitting element, 171: Transistor, 17 3: capacitance, 174: wiring, 175: wiring, 180: liquid crystal element, 181: capacitance, 182: wiring Wire, 183: Wiring

Claims

1. The image extracting unit, the database, the image processing unit, the image generating unit, and the learning unit are included. The database stores a table generated based on the first image data and the second image data acquired by the image extraction unit, the first image data has first gradation values ​​arranged in m rows and n columns (m and n are integers of 2 or greater); the second image data has second gradation values ​​arranged in m rows and n columns, the table represents the first gradation value and the second gradation value at a coordinate corresponding to the coordinate of the first gradation value; the image processing unit has a function of generating third learning image data by performing image processing on the first learning image data based on the second learning image data; the second learning image data is image data acquired from the image extraction unit, the third learning image data has third gradation values ​​arranged in m rows and n columns, the image generation unit has a function of generating fourth learning image data, which is image data including the first gradation value corresponding to the second gradation value selected based on the third gradation value; The learning unit is a generating device having a function of generating a machine learning model such that the image data output when the first learning image data is input matches the fourth learning image data.

2. In claim 1, the first learning image data has fourth gradation values ​​arranged in m rows and n columns, the second learning image data has fifth gradation values ​​arranged in m rows and n columns, The image processing unit is a generating device having a function of performing image processing so that the difference between the sum of the third gradation values ​​and the sum of the fifth gradation values ​​is smaller than the difference between the sum of the third gradation values ​​and the sum of the fourth gradation values.

3. In claim 1, A generating device in which the machine learning model is a neural network model.

Images