Image correction method

Abstract

Image correction method. In projection from a projector to a projection surface, side effects due to performing color correction are avoided. The correction method includes: in a case where at least one of a color correction gain of R, a color correction gain of G, or a color correction gain of B is less than a maximum threshold value, performing color correction based on the color correction gain; and in a case where the color correction gain of R, the color correction gain of G, and the color correction gain of B all exceed a minimum threshold value, performing color correction based on the color correction gain.

Description

Technical Field

[0001] This disclosure relates to a method for correcting images projected by a projector. Background Technology

[0002] In the past, projectors almost always used dedicated screens (such as white screens) set up in dark rooms. However, with the miniaturization, lower price, and higher brightness of projectors, the demand for projection onto walls or other projection surfaces in offices or living rooms has increased.

[0003] In such cases, it is rare for the projection surface to have uniform and high reflectivity. In addition, there are also the effects of ambient light color and brightness, which can cause the original image to be projected to deviate significantly from the projected image.

[0004] To address this problem, a color correction technique has been proposed as follows: The relationship between the input image fed to the projector and the image captured by the camera on the projection surface is pre-calculated, and the color of the input image is corrected based on this relationship to mitigate the deviation. Furthermore, this color correction technique is disclosed, for example, in Patent Document 1.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2007-322671

[0006] However, the image of the projected object needs color correction when appropriate conditions are met. This is because if the color of the image of the projected object is corrected for reasons other than when appropriate conditions are met, the color correction result will actually worsen. Summary of the Invention

[0007] One aspect of the image correction method disclosed herein includes: sequentially projecting a plurality of pattern images onto a projection surface using a projector having a light modulation element comprising a plurality of first pixels, wherein the pixel values ​​of the plurality of first pixels in the plurality of pattern images are different from those in each other; sequentially capturing images of the projection surface onto which the plurality of pattern images are sequentially projected using a camera having an imaging element comprising a plurality of second pixels, thereby obtaining a plurality of captured images corresponding one-to-one with the plurality of pattern images; generating a color transformation matrix based on the plurality of captured images, transforming the pixel values ​​of each of the plurality of second pixels to the pixel values ​​of each of the plurality of first pixels; calculating a color correction gain applied to the pixel values ​​of the plurality of first pixels to make the color of the image of the projection surface captured by the camera close to a target color based on the pixel values ​​of each of the plurality of second pixels and the color transformation matrix; and performing color correction based on the color correction gain if at least one of the color correction gain of R, the color correction gain of G, or the color correction gain of B is less than a maximum threshold.

[0008] Another aspect of the image correction method disclosed herein includes: sequentially projecting a plurality of pattern images onto a projection surface using a projector having a light modulation element comprising a plurality of first pixels, wherein the pixel values ​​of the plurality of first pixels in the plurality of pattern images are different from those in each other; sequentially capturing images of the projection surface onto which the plurality of pattern images are sequentially projected using a camera having an imaging element comprising a plurality of second pixels, thereby obtaining a plurality of captured images corresponding one-to-one with the plurality of pattern images; generating a color transformation matrix based on the plurality of captured images, transforming the pixel values ​​of each of the plurality of second pixels into the pixel values ​​of each of the plurality of first pixels; calculating a color correction gain applied to the pixel values ​​of the plurality of first pixels to make the color of the image of the projection surface captured by the camera close to a target color based on the pixel values ​​of each of the plurality of second pixels and the color transformation matrix; and performing color correction based on the color correction gain when the color correction gain of R, the color correction gain of G, and the color correction gain of B all exceed a minimum threshold. Attached Figure Description

[0009] Figure 1 This is a block diagram of a projection-type display device used in a correction method according to one embodiment.

[0010] Figure 2 This is a flowchart illustrating the process of the correction method.

[0011] Figure 3 This is a diagram showing the RGBK pattern used in the correction method.

[0012] Figure 4 This is an example of an averaged image obtained in an ideal projection environment using this correction method.

[0013] Figure 5 This is an example of a de-gamma-processed image obtained in an ideal projection environment using this correction method.

[0014] Figure 6 This is a diagram illustrating the color output characteristics obtained in an ideal projection environment using this correction method.

[0015] Figure 7 This is a diagram illustrating the target white in this correction method.

[0016] Figure 8 This is a diagram illustrating the averaged captured image obtained in a real projection environment using this correction method.

[0017] Figure 9 This is an example of a gamma-degraded image obtained in a real projection environment using this correction method.

[0018] Figure 10This is a diagram illustrating the color output characteristics obtained in a real projection environment using this correction method.

[0019] Figure 11 This diagram illustrates the color correction performed by the color correction unit in this correction method.

[0020] Figure 12 This is a diagram illustrating the calculation method for the color correction gain in this correction method.

[0021] Figure 13 This is a diagram illustrating the calculation method for the color correction gain in this correction method.

[0022] Label Explanation

[0023] 100 Projection-type display device; 2 Projection surface; 30 Camera; 40 Control unit; 50 Projector; 20 Memory; 10 Image processing unit; 11 Color correction gain calculation unit; 12 Color correction unit. Detailed Implementation

[0024] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. Furthermore, the dimensions and scales of the various parts in the drawings differ appropriately from the actual figures, and some parts are shown schematically for ease of understanding. Moreover, the scope of the present disclosure is not limited to these embodiments unless otherwise specified in the following description.

[0025] 1. Implementation Method

[0026] 1-1. Overview of the projection display device used in the calibration method

[0027] Figure 1 This is a block diagram of a projection-type display device 100 used in a correction method according to one embodiment. (See diagram below.) Figure 1 As shown, the projection display device 100 includes an image processing unit 10, a memory 20, a camera 30, a control unit 40, and a projector 50.

[0028] The projector 50 has an optical system including multiple light modulation elements for first pixels, and is a unit that projects images onto the projection surface 2 via this optical system. The projector 50 includes, for example, a light source, a light modulator, and a lens. The light source may include, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, emitting red, green, and blue light respectively. The light modulator may include, for example, three light modulation elements arranged corresponding to R, G, and B. Each light modulation element may include, for example, a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (Digital Micromirror Device), generating image light of each color by modulating light of the corresponding color. The image light of each color generated by the light modulator is synthesized by a color synthesis optical system to become full-color image light. In this embodiment, the projector 50 projects an image provided by the image processing unit 10 onto the projection surface 2.

[0029] The camera 30 has an image sensor containing multiple second pixels and is a unit that captures images projected onto the projection surface 2 through the image sensor.

[0030] The control unit 40 is a unit that controls the camera 30, the projector 50 and the image processing unit 10, and may include a processor such as a CPU (Central Processing Unit).

[0031] Like the control unit 40, the image processing unit 10 is a computer including a processor such as a CPU. The image processing unit 10 can be composed of a single processor or multiple processors. Furthermore, some or all of the functions of the image processing unit 10 can be implemented using hardware such as a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field Programmable Gate Array). Alternatively, the image processing unit 10 can be integrated with the control unit 40.

[0032] The memory 20 includes volatile storage units such as RAM and non-volatile storage units such as ROM and HD. The volatile storage units are used as working areas by the control unit 40 and the image processing unit 10. The non-volatile storage units store various programs executed by the control unit 40 and the image processing unit 10.

[0033] Figure 1 The program PRG stored in memory 20 is shown. The computer's image processing unit 10 executes this program PRG, functioning as the color correction gain calculation unit 11 and the color correction unit 12. Furthermore, Figure 1The data for target white, maximum threshold, minimum threshold, and averaged pixel value used or generated by the image processing unit 10 are shown. Furthermore, the contents of these data will be explained in detail in the operational description of this embodiment.

[0034] The color correction unit 12 is a unit that performs color correction on the input image of the projector 50 by multiplying the pixel values ​​of the image signal input from the outside by a color correction gain. Here, the color correction gain is the gain multiplied by the image signal so that the color observed on the projection surface 2 (wall, etc.) when the image signal is projected by the projector 50 onto any projection surface 2 is the same as the color observed on the projection surface 2 (white screen) when projected under ideal conditions (white screen, dark room).

[0035] The color correction gain calculation unit 11 is a unit that calculates the color correction gain provided to the color correction unit 12 and switches between implementing and not implementing color correction.

[0036] In this embodiment, the color correction gain calculation unit 11 projects multiple pattern images, each with different pixel values, onto the projection surface 2 using a projector 50 equipped with a light modulation element containing multiple first pixels. Furthermore, the color correction gain calculation unit 11 uses a camera 30 equipped with an imaging element containing multiple second pixels to sequentially capture the multiple pattern images projected onto the projection surface 2, thereby obtaining multiple captured images that correspond one-to-one with the multiple pattern images.

[0037] Furthermore, the color correction gain calculation unit 11 calculates a color transformation matrix that transforms the pixel values ​​of multiple second pixels into the pixel values ​​of multiple first pixels based on multiple captured images. Then, the color correction gain calculation unit 11 calculates a color correction gain applied to the pixel values ​​of multiple first pixels based on the pixel values ​​of each of the multiple second pixels and the color transformation matrix, in order to make the color of the image of the projection surface 2 captured by the camera 30 close to the target color.

[0038] In a preferred embodiment, the color correction gain calculation unit 11 at least obtains the color output characteristics in an actual projection environment where projection is made onto an arbitrary projection surface 2, such as a wall, in any room. Here, the color output characteristics are a matrix that transforms the pixel values ​​of a plurality of first pixels into the pixel values ​​of a plurality of second pixels. Then, the color correction gain calculation unit 11 calculates a color transformation matrix based on these color output characteristics. Specifically, the inverse matrix of the color output characteristics is used as the color transformation matrix. Then, the color correction gain calculation unit 11 calculates the color correction gain based on the target white obtained from the color output characteristics in an ideal projection environment where projection is made onto a projection surface 2, which is a white screen, in a dark room, the pixel values ​​for black display obtained from the color output characteristics in the actual projection environment, and the color transformation matrix. Furthermore, it is arbitrary whether the color correction gain calculation unit 11 obtains the color output characteristics in the ideal projection environment by measurement or provides the color output characteristics from an external source.

[0039] Furthermore, if the color correction gain calculation unit 11 causes the color correction unit 12 to perform color correction when at least one of the color correction gain of R, the color correction gain of G, or the color correction gain of B is less than the maximum threshold Th_max, the color correction unit 11 will not perform color correction.

[0040] In addition, if the color correction gain of R, the color correction gain of G, and the color correction gain of B are all greater than the minimum threshold Th_min, the color correction unit 11 causes the color correction unit 12 to perform color correction; otherwise, color correction is not performed.

[0041] When projecting in an actual projection environment, the color correction unit 12 uses the color correction gain calculated by the color correction gain calculation unit 11 to correct the pixel values ​​of the input image signal and provides them to the projector 50.

[0042] 1-2. Image Correction Methods

[0043] Figure 2 This is a flowchart illustrating the process of the image correction method of this embodiment. The correction method is performed by a color correction gain calculation unit 11, which is a function of a computer.

[0044] exist Figure 2In this process, steps S11 to S14 are preprocessing steps performed in an ideal projection environment in a dark room, projecting onto a projection surface 2 that serves as a white screen. This preprocessing is performed before the projection display device 100 is manufactured or shipped. Steps S21 to S25 and S31 to S34 are main processes performed in an actual projection environment, projecting onto an arbitrary projection surface 2 such as a wall from an arbitrary room. Alternatively, the preprocessing may not be performed by the color correction gain calculation unit 11; instead, the color transformation matrix under the ideal projection environment may be pre-calculated by other units and provided to the color correction gain calculation unit 11.

[0045] In step S11, the color correction gain calculation unit 11 projects the R, G, B, and K patterns sequentially onto the projection surface 2 (white screen) via the projector 50, and captures images of each pattern projected onto the projection surface 2 via the camera 30.

[0046] Figure 3 This is an example of the R, G, B, and K patterns projected in step S11. The R pattern is a pattern where the R value is 255, the G value is 0, and the B value is 0 in all the first pixels constituting a pattern. The G pattern is a pattern where the R value is 0, the G value is 255, and the B value is 0 in all the first pixels constituting a pattern. The B pattern is a pattern where the R value is 0, the G value is 0, and the B value is 255 in all the first pixels constituting a pattern. The K pattern is a pattern where the R value is 0, the G value is 0, and the B value is 0 in all the first pixels constituting a pattern.

[0047] Next, in step S12, the color correction gain calculation unit 11 averages the pixel values ​​(RGB values) of a plurality of second pixels within a specified range in each captured image of the R, G, B, K pattern captured in step S11. Figure 4 The averaged pixel values ​​R0ave, G0ave, and B0ave of the R, G, B, and K colors of the captured images obtained in step S12 are shown respectively.

[0048] Next, in step S13, the color correction gain calculation unit 11 performs gamma demapping on the averaged pixel values ​​obtained in step S12. Figure 5The pixel values ​​R0lin, G0lin, and B0lin for each color in the linear space obtained through gamma demapping are shown. Through gamma demapping, the color correction gain calculation unit 11 can, for example, obtain the grayscale values ​​of each of the multiple second pixels when the camera 30 captures the R pattern, that is, the Y value in the XYZ values ​​measured by a spectroradiometer when the input of the camera 30 changes, i.e., the change in the output of the camera 30 is linearly correlated with the first correspondence. This also applies to the G and B patterns. Linearity means that the correlation coefficient between the change in the input of the camera 30 and the change in the output of the camera 30 is 0.99 or higher. The first correspondence is an example of the input-output characteristics of the camera 30.

[0049] Next, in step S14, the color correction gain calculation unit 11 subtracts the pixel value of K (R0lin, G0lin, B0lin)_R, (R0lin, G0lin, B0lin)_G, and (R0lin, G0lin, B0lin)_B of each color (R0lin, G0lin, B0lin)_K from the pixel values ​​of R, G, and B obtained in step S13, respectively, and calculates the color output characteristics (R0out, G0out, B0out).

[0050] Figure 6 This is a diagram illustrating the color output characteristics obtained in step S14. These color output characteristics (R0out, G0out, B0out) are a matrix that transforms the pixel values ​​of multiple first pixels into the pixel values ​​of multiple second pixels in an ideal projection environment.

[0051] Next, in step S15, as Figure 7 As shown, the color correction gain calculation unit 11 calculates the target white values ​​Rtarget, Gtarget, and Btarget based on the color output characteristics obtained in step S14. That is, the color correction gain calculation unit 11 calculates... Figure 6 In the color output characteristics, the diagonal elements R_R0out, G_G0out, and B_B0out are set to target white Rtarget, Gtarget, and Btarget. These target white values ​​Rtarget, Gtarget, and Btarget are the target pixel values ​​used to make the second pixel (captured image) of the projection surface 2 display white in an ideal projection environment.

[0052] Here, in this embodiment, for example, the grayscale values ​​of each of the multiple first pixels when the projector 50 projects the R pattern, i.e., the Y value in the XYZ values ​​measured by the spectroradiometer when the input of the projector 50 changes, i.e., the change in the output of the projector 50, are linear after gamma demapping. Linearity means that the correlation coefficient between the change in the input of the projector 50 and the change in the output of the projector 50 is 0.99 or higher. In this embodiment, the correspondence between the change in the input of the projector 50 and the change in the output of the projector 50 is called the second correspondence. The second correspondence is an example of the input-output characteristics of the projector 50. As described above, the first correspondence is also linear after gamma demapping; therefore, in this embodiment, after gamma demapping, the change in the input of the camera 30 and the change in the input of the projector 50 are linear. Therefore, in steps S11 to S15, the input-output characteristics of the camera 30 and the input-output characteristics of the projector 50 are both linear.

[0053] Thus, the number of patterns can be made to be the four patterns representing the three primary colors of light, namely R, G, and B patterns, and the K pattern used to compensate for the influence of ambient light.

[0054] Next, the color correction gain calculation unit 11 performs steps S21 to S25 and S31 to S34 in the actual projection environment. The actual projection environment is the environment in which the user sets the projection display device 100.

[0055] First, in step S21, the color correction gain calculation unit 11, in the same manner as in step S11, projects the R, G, B, and K patterns onto the projection surface 2 in sequence through the projector 50, and takes pictures of the patterns of each color projected onto the projection surface 2 in sequence through the camera 30.

[0056] Next, in step S22, the color correction gain calculation unit 11, in the same manner as in step S12, averages the pixel values ​​(RGB values) of a plurality of second pixels within a specified range in each captured image of the R, G, B, K pattern captured in step S21. Figure 8 The averaged pixel values ​​Rave, Gave, and Bave of the R, G, B, and K colors of the captured images obtained in step S22 are shown respectively.

[0057] Next, in step S23, the color correction gain calculation unit 11 performs gamma demapping on the averaged pixel value obtained in step S22, similar to step S13. Figure 9 The image shows the pixel values ​​Rlin, Glin, and Blin for each color in the linear space obtained through gamma removal processing.

[0058] Next, in step S24, the color correction gain calculation unit 11, similarly to step S14, subtracts the pixel value of K (Rlin, Glin, Blin)_R, (Rlin, Glin, Blin)_G, and (Rlin, Glin, Blin)_B of each color (Rlin, Glin, Blin) obtained in step S23, respectively, to calculate the color output characteristics (Rout, Gout, Bout) in the actual projection environment.

[0059] Figure 10 This is a diagram illustrating the color output characteristics obtained in step S24. These color output characteristics (Rout, Gout, Bout) are matrices that transform the pixel values ​​of multiple first pixels into the pixel values ​​of multiple second pixels in the actual projection environment.

[0060] Figure 11 This diagram illustrates the color correction performed by the color correction unit 12 in this embodiment. During color correction, by multiplying by an appropriate color correction gain vector Gr, Gg, Gb, the dynamic range of the pixel value (RGB value) of the second pixel of the captured image is converged to the pixel values ​​Rlin, Glin, Blin displayed as black in the actual projection environment (see reference). Figure 9 The range is within the range of the target white pixel values ​​Rtarget, Gtarget, and Btarget.

[0061] Therefore, in step S25, the color correction gain calculation unit 11 as follows Figure 12 As illustrated, calculate the pixel value Rlin (in the actual projection environment when the K pattern is displayed) by subtracting the second pixel from the target white (Rtarget, Gtarget, Btarget). Figure 9 (1.7 in the middle), Glin (in the middle) Figure 9 (1.3 in the middle), Blin (in) Figure 9 The color vector after the value is 0.2 is multiplied by the color output characteristics of the actual projection environment. Figure 10 The inverse matrix of ) is the color transformation matrix, from which the color correction gain Gr is calculated (in Figure 11 The values ​​are 1.66 (Gg) (1.67) and 2.53 (Gb).

[0062] And, as Figure 13As illustrated, the color correction gain calculation unit 11 normalizes the color correction gains Gr, Gg, and Gb calculated in step S25 using their maximum values. In this example, Gr = 1.66, Gg = 1.67, and Gb = 2.53; therefore, Gr, Gg, and Gb are normalized by dividing by the maximum value of 2.53. Furthermore, the color correction gain calculation unit 11 performs a transformation process on the normalized color correction gains Gr, Gg, and Gb from linear space to PC space used in cameras, PCs, etc. This yields color correction gains Gr, Gg, and Gb that can be applied to the color correction unit 12. In this example, Gr = 0.83, Gg = 0.83, and Gb = 1.0.

[0063] Furthermore, the camera 30 and projector 50 used in steps S21 to S25 are the same as those used in steps S11 to S15. Therefore, in steps S21 to S25, the input-output characteristics of the camera 30 and the input-output characteristics of the projector 50 are also linear.

[0064] Next, in step S31, the color correction gain calculation unit 11 performs a first implementation determination related to color correction. That is, the color correction gain calculation unit 11 determines whether at least one of the color correction gains Gr, Gg, and Gb is lower than the maximum threshold Th_max. In this embodiment, the maximum threshold Th_max is 0.95. If the determination result is "yes", the processing of the color correction gain calculation unit 11 proceeds to step S32; if the result is "no", the processing ends.

[0065] Next, when proceeding to step S32, the color correction gain calculation unit 11 performs a second implementation determination related to color correction. That is, the color correction gain calculation unit 11 determines whether the color correction gains Gr, Gg, and Gb all exceed the minimum threshold Th_min. In this embodiment, the minimum threshold Th_min is 0.5. If the determination result is "yes", the processing of the color correction gain calculation unit 11 proceeds to step S34; otherwise, it proceeds to step S33.

[0066] Next, when proceeding to step S33, the color correction gain calculation unit 11 displays a message such as "Please reproject onto a light-colored screen" on the projection surface 2 via the projector 50. Then, the color correction gain calculation unit 11 ends the process.

[0067] On the other hand, when the results of the first implementation judgment in step S31 and the second implementation judgment in step S32 are both "yes" and the process proceeds to step S34, the color correction gain calculation unit 11 sets the calculated color correction gains Gr, Gg, and Gb to the color correction unit 12 and performs color correction. Then, the color correction gain calculation unit 11 ends the process.

[0068] As described above, in this embodiment, color correction is not performed if the result of the first implementation determination (step S31) is "No," that is, if the color correction gains Gr, Gg, and Gb are all not lower than the maximum threshold Th_max. The reason is as follows: The fact that the color correction gains Gr, Gg, and Gb are all not lower than the maximum threshold Th_max means that, because the color of the wall serving as the projection surface 2 is too pale / light, the color of the captured image is too close to the target white. If color correction is performed under such circumstances, the projected image will suffer from color degradation and image quality deterioration compared to before the correction. This is because color differences below the performance value of color correction in the projection display device 100 (the minimum color difference that can be corrected, determined by the sensitivity deviation of the camera 30, the projection light intensity deviation of the projector 50, the color output characteristics of the camera 30 / projector 50, the color correction algorithm, etc.) cannot be corrected. In this embodiment, to prevent such side effects, color correction is not performed if the result of the first implementation determination is "No."

[0069] Furthermore, in this embodiment, color correction is not performed if the result of the second implementation determination (step S32) is "No," meaning that at least one of the color correction gains Gr, Gg, and Gb does not exceed the minimum threshold Th_min. The reason is as follows: At least one of the color correction gains Gr, Gg, and Gb not exceeding the minimum threshold Th_min means that the color of the wall serving as the projection surface 2 is too saturated, and the color of the captured image is too far removed from the target white. If color correction is performed under such circumstances, a side effect occurs, such as a reduction in the upper limit of the displayable brightness and excessive reduction in the contrast of the bright room (e.g., below 7), leading to image quality degradation. In this embodiment, to prevent such side effects, color correction is not performed if the result of the second implementation determination is "No."

[0070] According to this embodiment, color correction is performed when both the results of the first implementation determination (step S31) and the second implementation determination (step S32) are "yes". Therefore, color correction can be performed without side effects such as the color of the captured image being too close to the target white, deterioration of the color of the projected image, deterioration of image quality, or deterioration of image quality caused by a reduction in the upper limit of the displayable brightness and an excessive reduction in the contrast of the bright room (e.g., below 7).

[0071] 2. Variations

[0072] The methods illustrated above can be varied. The following examples illustrate specific variations applicable to the above methods. Two or more methods selected from the following examples can be appropriately combined without contradiction.

[0073] 2-1. Variation Example 1

[0074] In the above embodiment, both the first implementation judgment (step S31) and the second implementation judgment (step S32) are executed, and color correction is performed if the result of these two judgments is "yes". However, it is also possible that, instead of this, the party executing the first implementation judgment (step S31) or the second implementation judgment (step S32) performs color correction if its judgment result is "yes".

[0075] For example, in a certain variant 1-1, the color correction gain calculation unit 11 only performs the first implementation judgment (step S31), and performs color correction if at least one of the color correction gains Gr, Gg, and Gb is less than the maximum threshold Th_max, otherwise it does not perform color correction.

[0076] Furthermore, for example in other variations 1-2, the color correction gain calculation unit 11 only performs the second implementation judgment (step S32), performs color correction when all color correction gains Gr, Gg, and Gb exceed the minimum threshold Th_min, and does not perform color correction when this is not the case, and displays a message on the projection surface 2 urging it to change to a lighter color.

[0077] 2-2. Variation Example 2

[0078] It also allows users to select which judgment to perform in the first implementation judgment (step S31) and the second implementation judgment (step S32).

[0079] 2-3. Variation Example 3

[0080] In the above embodiment, if the first implementation judgment (step S31) and the second implementation judgment (step S32) are both executed, and the result of at least one of them is "no", then color correction is not performed. However, instead, the results of the first implementation judgment (step S31) and the second implementation judgment (step S32) can be displayed by the projector 50, allowing the user to choose whether to perform color correction.

[0081] 2-4. Variation Example 4

[0082] The maximum threshold Th_max and the minimum threshold Th_min can also be set by the user through certain operations.

[0083] 3. Notes

[0084] The following is a summary published in this note.

[0085] (Appendix 1) A correction method includes: sequentially projecting a plurality of pattern images onto a projection surface using a projector having a light modulation element comprising a plurality of first pixels, wherein the pixel values ​​of the plurality of first pixels in the plurality of pattern images are different from each other; sequentially capturing images of the projection surface onto which the plurality of pattern images are sequentially projected using a camera having an imaging element comprising a plurality of second pixels, thereby obtaining a plurality of captured images corresponding one-to-one with the plurality of pattern images; generating a color transformation matrix based on the plurality of captured images, transforming the pixel values ​​of each of the plurality of second pixels into the pixel values ​​of each of the plurality of first pixels; calculating a color correction gain applied to the pixel values ​​of the plurality of first pixels to make the color of the image of the projection surface captured by the camera close to a target color based on the pixel values ​​of each of the plurality of second pixels and the color transformation matrix; and performing color correction based on the color correction gain if at least one of the color correction gain of R, the color correction gain of G, or the color correction gain of B is less than a maximum threshold.

[0086] In this method, color correction is performed when the color correction gain of at least one color is below the maximum threshold. Therefore, when the color of the projection surface is light, the color correction result can be prevented from deteriorating due to factors such as camera performance.

[0087] (Note 2) The correction method according to Note 1 includes: performing color correction based on the color correction gain when the color correction gain of R, the color correction gain of G and the color correction gain of B all exceed a minimum threshold.

[0088] According to this method, color correction is performed when the color correction gain of R, the color correction gain of G, and the color correction gain of B all exceed the minimum threshold. Therefore, when the color of the projection surface is saturated, the deterioration of the bright room contrast due to color correction can be avoided.

[0089] (Note 3) The correction method according to Note 2 includes: displaying a message recommending that the projection surface be changed to a lighter projection surface when at least one of the color correction gain of R, the color correction gain of G and the color correction gain of B does not exceed the minimum threshold.

[0090] According to this method, users can easily determine which projection surface should be changed to avoid degrading the contrast of a bright room.

[0091] (Note 4) The correction method according to any one of Notes 1 to 3, wherein the input-output characteristics of the projector are linear with the input-output characteristics of the camera, and the color transformation matrix is ​​calculated based on the color output characteristics measured by the projection of each pattern of R, G, B, and K.

[0092] According to this method, color correction gain is obtained by projecting four RGBK patterns, thus shortening the time required to obtain color correction gain.

[0093] (Appendix 5) A correction method includes: sequentially projecting a plurality of pattern images onto a projection surface using a projector having a light modulation element comprising a plurality of first pixels, wherein the pixel values ​​of the plurality of first pixels in the plurality of pattern images are different from each other; sequentially capturing images of the projection surface onto which the plurality of pattern images are sequentially projected using a camera having a camera element comprising a plurality of second pixels, thereby obtaining a plurality of captured images corresponding one-to-one with the plurality of pattern images; generating a color transformation matrix based on the plurality of captured images, transforming the pixel values ​​of each of the plurality of second pixels into the pixel values ​​of each of the plurality of first pixels; calculating a color correction gain applied to the pixel values ​​of the plurality of first pixels to make the color of the image of the projection surface captured by the camera close to a target color based on the pixel values ​​of each of the plurality of second pixels and the color transformation matrix; and performing color correction based on the color correction gain when the color correction gain of R, the color correction gain of G, and the color correction gain of B all exceed a minimum threshold.

[0094] According to this method, color correction is performed when the color correction gain of R, the color correction gain of G, and the color correction gain of B all exceed the minimum threshold. Therefore, when the color of the projection surface is saturated, the deterioration of the bright room contrast due to color correction can be avoided.

[0095] (Note 6) The correction method according to Note 5 includes: displaying a message recommending that the projection surface be changed to a lighter projection surface when at least one of the color correction gain of R, the color correction gain of G and the color correction gain of B does not exceed the minimum threshold.

[0096] According to this method, users can easily determine which projection surface should be changed to avoid degrading the contrast of a bright room.

[0097] (Note 7) According to the correction method described in Note 5 or 6, wherein the input-output characteristics of the projector and the input-output characteristics of the camera are linear, and the color transformation matrix is ​​calculated based on the color output characteristics measured by the projection of each pattern of R, G, B, K.

[0098] According to this method, color correction gain is obtained by projecting four RGBK patterns, thus shortening the time required to obtain color correction gain.

Claims

1. An image correction method, comprising: A projector equipped with a light modulation element containing a plurality of first pixels is used to sequentially project a plurality of pattern images onto a projection surface, wherein the pixel values ​​of the plurality of first pixels in the plurality of pattern images are different from those in each other; By sequentially capturing images of the projection surface onto which the multiple pattern images are sequentially projected using a camera with a camera element containing multiple second pixels, multiple captured images corresponding one-to-one with the multiple pattern images are obtained; Based on the multiple captured images, a color transformation matrix is ​​generated that transforms the pixel values ​​of each of the multiple second pixels into the pixel values ​​of each of the multiple first pixels; Based on the pixel values ​​of the plurality of second pixels and the color transformation matrix, calculate the color correction gain applied to the pixel values ​​of the plurality of first pixels to make the color of the image of the projection surface captured by the camera close to the target color; and If at least one of the color correction gain of R, the color correction gain of G, or the color correction gain of B is less than a maximum threshold, color correction based on the color correction gain is performed.

2. The image correction method according to claim 1, wherein, include: If the color correction gain of R, the color correction gain of G, and the color correction gain of B all exceed the minimum threshold, color correction based on the color correction gain is performed.

3. The image correction method according to claim 2, wherein, include: If at least one of the color correction gain of R, the color correction gain of G, and the color correction gain of B does not exceed the minimum threshold, a message recommending changing the projection surface to a lighter color is displayed.

4. The image correction method according to any one of claims 1 to 3, wherein, The input-output characteristics of the projector are linear with those of the camera. The color transformation matrix is ​​calculated based on the color output characteristics measured by projecting patterns of R, G, B, and K.

5. An image correction method, comprising: A projector equipped with a light modulation element containing a plurality of first pixels is used to sequentially project a plurality of pattern images onto a projection surface, wherein the pixel values ​​of the plurality of first pixels in the plurality of pattern images are different from those in each other; By sequentially capturing images of the projection surface onto which the multiple pattern images are sequentially projected using a camera with a camera element containing multiple second pixels, multiple captured images corresponding one-to-one with the multiple pattern images are obtained; Based on the multiple captured images, a color transformation matrix is ​​generated that transforms the pixel values ​​of each of the multiple second pixels into the pixel values ​​of each of the multiple first pixels; Based on the pixel values ​​of the plurality of second pixels and the color transformation matrix, calculate the color correction gain applied to the pixel values ​​of the plurality of first pixels to make the color of the image of the projection surface captured by the camera close to the target color; and If the color correction gain of R, the color correction gain of G, and the color correction gain of B all exceed the minimum threshold, color correction based on the color correction gain is performed.

6. The image correction method according to claim 5, wherein, include: If at least one of the color correction gain of R, the color correction gain of G, and the color correction gain of B does not exceed the minimum threshold, a message recommending changing the projection surface to a lighter color is displayed.

7. The image correction method according to claim 5 or 6, wherein, The input / output characteristics of the projector and the input / output characteristics of the camera are linear. The color transformation matrix is ​​calculated based on the color output characteristics measured by projecting patterns of R, G, B, and K.

Images