Information Processing Method

Abstract

To provide a simple configuration that makes it possible to correct colors of images displayed on monitors so that the colors become uniform among the monitors.SOLUTION: An information processing device (10) comprises: an acquisition unit (14) which acquires a display image that is displayed on a monitor (22) and includes a region with a known color value and a captured image that includes an object being the real thing with a known color value; and a color correction unit (16) which calculates an adjustment parameter for correcting a color of any image displayed on the monitor (22) according to both of the color of the region in the display image included in the captured image and the color of the object being the real thing included in the captured image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention , love Report processing method By law Regarding. [Background technology]

[0002] Color management tools for aligning the colors of digital images across multiple monitors are known as prior art. Color management tools adjust the monitor's surrounding environment or control the monitor's color reproduction by measuring colors using dedicated equipment. Patent Document 1 discloses an image quality adjustment device that can adjust the display settings of a second LCD monitor so that the brightness and color data of the second LCD monitor are at the same level as those of a first LCD monitor. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-124490 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques are cumbersome because the user must use a specific color measurement device to make adjustments while evaluating the color reproduction of the monitor, the ambient light, etc. Furthermore, when adjusting the color of an object displayed on the monitor, there is a problem in that the difference between the color of the object and the actual color of the object is not taken into consideration.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and aims to make it possible to correct an image displayed on a monitor so that the color of the image is uniform between monitors using a simple configuration. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to one aspect of the present invention includes an acquisition unit that acquires a display image displayed on a monitor, the display image including an area whose color value is known, and a captured image that includes an actual object whose color value is known, and a color correction unit that calculates adjustment parameters for correcting the color of any image displayed on the monitor based on both the color of the area in the display image included in the captured image and the color of the actual object included in the captured image.

[0007] In order to solve the above problem, an information processing method according to one aspect of the present invention includes an acquisition step of acquiring a display image displayed on a monitor, the display image including an area whose color value is known, and a captured image including an actual object whose color value is known, and a calculation step of calculating adjustment parameters for correcting the color of any image displayed on the monitor based on both the color of the area in the display image included in the captured image and the color of the actual object included in the captured image. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to correct an image displayed on a monitor so that the colors of the image are uniform across the monitors, using a simple configuration. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a functional block diagram of an information processing system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a captured image captured by an imaging unit. [Figure 3] FIG. 4 is a diagram showing an example of monitor characteristics of a first monitor and a second monitor. [Figure 4] 1 is a flowchart showing a processing flow according to the first embodiment. [Figure 5] 10 is a conceptual diagram showing the process performed by the information processing system up to the point where a color-corrected image is displayed on the screen of a display unit. [Figure 6] 10 is a flowchart showing a processing flow according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Embodiment 1] An embodiment of the present invention will be described in detail below, which relates to an information processing system capable of correcting an image so that the colors of the image displayed on the monitors are uniform across the monitors.

[0011] [1. Example of information processing system configuration] An information processing system according to this embodiment will be described. Fig. 1 is a functional block diagram of an information processing system 1 according to this embodiment. As shown in Fig. 1, the information processing system 1 includes an information processing device 10, an imaging unit (camera) 20, and a display unit (monitor) 22.

[0012] The information processing device 10 is a device realized as a personal computer or the like, and includes a control unit 12 and a storage unit 18.

[0013] The control unit 12 is a control device that controls the entire information processing device 10, and includes an acquisition unit 14 and a color correction unit 16. The control unit 12 also controls the display on the display unit 22.

[0014] The acquisition unit 14 acquires a captured image captured by the imaging unit 20. For example, the acquisition unit 14 acquires a display image displayed on the display unit 22, the display image including a region where color values ​​indicating colors are known, and a captured image including an actual object where color values ​​are known. Here, the object is an object where color values ​​are known as described above, for example, a color chip (color chart) on which multiple color samples are printed. Furthermore, the display image displayed on the display unit 22, the display image including a region where color values ​​indicating colors are known, may be a display image of the object.

[0015] Fig. 2 is a diagram showing an example of the captured image. In Fig. 2, captured image 30 is a captured image captured by imaging unit 20. Display image 32 is a display image of object 34 (color patch) displayed on display unit 22. In the example of display image 32, at least the color values ​​(e.g., RGB values) of the area showing the color patch are known, and these color values ​​are not affected by monitor characteristics, which will be described later. The color patch located in the lower right of captured image 30 is a color patch corresponding to the actual object 34.

[0016] The color correction unit 16 corrects the color of an image displayed on the display unit 22. For example, the color correction unit 16 calculates adjustment parameters, which are information for correcting the color of an arbitrary image displayed on the display unit 22, based on both the color of an area in the captured image 30 whose color value is known in the display image 32 and the color of an actual object 34 included in the captured image 30. The color correction unit 16 then corrects the color of the arbitrary image using the calculated adjustment parameters. Note that the arbitrary image may include an image of an object whose actual color value is unknown. Furthermore, when the display image acquired by the acquisition unit 14 is a display image of the object 34, it is desirable for the color correction unit 16 to calculate adjustment parameters that bring the color of the object 34 in the display image closer to the color of the actual object 34. However, in many cases, the image to which the adjustment parameters are applied is an image other than the object.

[0017] Furthermore, the color correction unit 16 may compare the colors of the captured image, which are colors of areas in the displayed image whose color values ​​are known, with the colors of the actual objects included in the captured image, to calculate monitor characteristics (color development characteristics) related to the color development of the display unit 22. The monitor characteristics may include gamma characteristics that indicate the relationship between input gradation values ​​and output gradation values ​​in the image display on the display unit 22. The monitor characteristics may also change depending on the settings of the display unit 22. The color correction unit 16 may also calculate adjustment parameters for performing color correction corresponding to the monitor characteristics for any image displayed on the display unit 22.

[0018] The color of the display image 32 in the captured image 30 is a color that reflects the camera characteristics (sensitivity characteristics) related to the color of the output image of the imaging unit 20, the ambient light, and the monitor characteristics, in addition to the true color (e.g., RGB value) of the display image 32. In contrast, the color of the object 34 included in the captured image 30 is a color that reflects the camera characteristics and the ambient light, in addition to the true color of the object 34.

[0019] The camera characteristics may also include gamma characteristics that indicate the relationship between input gradation values ​​and output gradation values ​​in the image output of the imaging unit 20. The camera characteristics may also change depending on the settings of the imaging unit 20.

[0020] Because the camera characteristics and the ambient light are common, these image effects can be canceled out and the color correction unit 16 can calculate the monitor characteristics. That is, the color correction unit 16 calculates the monitor characteristics by comparing the color of the object 34 included in the captured image 30 with the color of the display image 32 in the captured image 30.

[0021] Graph 40 in Fig. 3 shows an example of the monitor characteristics of the first monitor, and graph 42 shows an example of the monitor characteristics of the second monitor. In each graph, the horizontal axis corresponds to the input gradation value, and the vertical axis corresponds to the output gradation value. From another perspective, for example, the horizontal axis of each graph corresponds to the color of the actual object included in the captured image, and the vertical axis corresponds to the color of the object in the display image included in the captured image.

[0022] 3, the monitor characteristics can be calculated for each color in the RGB (Red, Green, Blue) color space. For example, for the blue color in graph 40, an input gradation value of 200 roughly corresponds to an output gradation value of 100. Note that the monitor characteristics may also be calculated for each color in a color space other than the RGB color space. Furthermore, when the colors of areas in the display image 32 whose color values ​​are known and the color of the object 34 are widely distributed in the color space, highly accurate color correction becomes possible, which is preferable.

[0023] Furthermore, the color correction corresponding to the monitor characteristics described above refers to color correction such as gamma correction that offsets the monitor characteristics, and means color correction that creates a linear relationship between the input gradation value of the captured image acquired by the acquisition unit 14 and the output gradation value that reflects the monitor characteristics.

[0024] From another perspective, the above-mentioned adjustment parameters are information that specifies how to correct the colors of the target image, and are, for example, information that indicates the correspondence between the input gradation values ​​of the image before correction and the output gradation values ​​of the image after correction for each primary color of the color space.

[0025] The storage unit 18 is a storage device that stores various types of information. For example, the storage unit 18 stores information indicating the correspondence between a target color and the gradation values ​​of each primary color that is the basis of the target color, as well as information indicating known color values ​​of the display image 32 and the target object 34. The information may be referenced when the color correction unit 16 determines monitor characteristics, etc.

[0026] The imaging unit 20 is a camera that captures a captured image. For example, the imaging unit 20 captures a single captured image that includes both the display image 32 of the object 34 displayed on the display unit 22 and the actual object 34.

[0027] The display unit 22 is a monitor that displays moving images, text, etc. on the screen based on the control of the control unit 12. For example, the display unit 22 displays a display image 32 of an object 34.

[0028] When the information processing device 10 is realized as a smartphone, a tablet, or the like, the imaging unit 20 and the display unit 22 are typically realized as an integral part of the information processing device 10. Alternatively, at least some of the functions of the information processing device 10 may be realized on a remote server, and the functions may be provided by a web application or the like. In the above configuration, the imaging unit 20 and the display unit 22, or a device associated with the imaging unit 20 and the display unit 22, have a function of communicating with the server.

[0029] [2. Examples of processing by information processing systems] Next, an example of a process in which the information processing system 1 corrects the color of an image displayed on the display unit 22 will be described. Fig. 4 is a flowchart showing the flow of the process in this example. Fig. 5 is a conceptual diagram showing the process in which the information processing system 1 displays a color-corrected image on the screen of the display unit 22 (first monitor and second monitor). In the following example, the display unit 22 will be described as displaying an image of an object 34.

[0030] In step S101, the display unit 22 displays a display image 32 of the object 34 on the screen under the control of the control unit 12. An image 50 in Fig. 5 is an example of an image to be displayed on the first monitor and the second monitor. Since the first monitor and the second monitor have different monitor characteristics for each color, the image 50 is displayed in different colors on each monitor.

[0031] In step S102, the image capturing unit 20 captures a single captured image 30 including the display image 32 of the object 34 displayed on the display unit 22 and the actual object 34.

[0032] In step S103, the imaging unit 20 outputs the captured image 30 to the information processing device 10. The acquisition unit 14 acquires the captured image. The process of the acquisition unit 14 in step S103 is an example of an acquisition step in the present disclosure.

[0033] In step S104, the color correction unit 16 calculates the monitor characteristics of the display unit 22 by referring to both the color of the object 34 in the display image 32 included in the captured image 30 and the color of the actual object 34 included in the captured image. As described above, Fig. 3 shows the monitor characteristics of the first monitor and the second monitor.

[0034] In step S105, the color correction unit 16 calculates adjustment parameters for performing color correction such as gamma correction corresponding to the calculated monitor characteristics on an arbitrary image. The processing of the color correction unit 16 in steps S104 and S105 is an example of a calculation step in the present disclosure.

[0035] In step S106, the color correction unit 16 uses the calculated adjustment parameters to correct the color of any image displayed on the display unit 22. The processing by the color correction unit 16 in step S106 is an example of a color correction step in the present disclosure.

[0036] Image 52 in Figure 5 shows an image on which color correction unit 16 has performed color correction corresponding to the monitor characteristics of the first monitor, and image 62 shows an image on which color correction unit 16 has performed color correction corresponding to the monitor characteristics of the second monitor. Since the monitors have different monitor characteristics, the colors of image 52 and image 62 are different from each other. As described above, the image on which color correction unit 16 performs color correction using adjustment parameters is not limited to an image including object 34.

[0037] In step S107, the display unit 22 displays the image, color-corrected by the color correction unit 16, on the screen.

[0038] 5 shows the image displayed on the first monitor, and image 64 shows the image displayed on the second monitor. Images 54 and 64 also include a color chart corresponding to object 34 and an actual food sample 60 for comparison. 5, a color chart is used as object 34, but if the color values ​​are known, for example, food sample 60 itself, which is the main display object, may be used as the object. In image 54, the first monitor displays, on the same screen, display image 56 corresponding to the display image displayed on the screen in step S101 and display image 58 corresponding to the display image displayed on the screen in step S107. Similarly, in image 64, the second monitor displays, on the same screen, display image 66 corresponding to the display image displayed on the screen in step S101 and display image 68 corresponding to the display image displayed on the screen in step S107.

[0039] Image 58 displayed on the first monitor and image 68 displayed on the second monitor have each been subjected to color correction by color correction unit 16 to offset the monitor characteristics of each monitor, so that the hues of images 58 and 68 are very close to each other. This is because each monitor can display colors that are based on the true color of object 34, reflect the camera characteristics of imaging unit 20 and ambient light, but are not dependent on the monitor characteristics of display unit 22.

[0040] Furthermore, following the process of step S107, the information processing system 1 may repeat the process from S102 onwards one or more times to further adjust the color of the display image.

[0041] As described above, the information processing method executed by the information processing device 10 includes an acquisition step and a calculation step. In the acquisition step, a captured image including a display image 32, which is a display image displayed on a monitor (display unit 22) and includes an area with known color values, and a real object 34 is acquired. In the calculation step, adjustment parameters for correcting the color of an arbitrary image displayed on the monitor are calculated based on both the color of the area in the display image 32 included in the captured image 30 and the color of the real object 34 included in the captured image 30. According to the configuration of this example, it is possible to correct the color of images displayed on each monitor so that the colors are uniform across the monitors with a simple configuration without changing the display settings on the monitors.

[0042] [Embodiment 2] A second embodiment of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and redundant description will not be repeated.

[0043] In this embodiment, an information processing system is described that can correct the colors of an object or the like in any image displayed on each monitor so that the colors of the displayed image become closer to the actual colors and become uniform across the monitors.

[0044] [1. Example of information processing system configuration] This embodiment also uses the configuration shown in the functional block diagram of Fig. 1. However, the acquisition unit 14 according to this embodiment is configured to acquire information about ambient light incident on the screen of the display unit 22 from a sensor or the like (not shown), and acquire information indicating the camera characteristics of the imaging unit 20 from the imaging unit 20 or the storage unit 18. Here, the information about ambient light is information indicating elements of ambient light that affect how an object present under that ambient light appears, such as information indicating the illuminance or color tone of the ambient light.

[0045] Furthermore, the color correction unit 16 according to this embodiment is configured to calculate adjustment parameters for correcting the color of the image displayed on the display unit 22 in accordance with the monitor characteristics of the display unit 22 as well as at least one of ambient light and camera characteristics, or both image effects that do not distinguish between them.

[0046] In this embodiment, the configuration in which color correction unit 16 calculates adjustment parameters for correcting the colors of a displayed image is not limited to a specific configuration. For example, information indicating the correspondence between each input gradation value of the image and between ambient light and each output gradation value may be stored in storage unit 18, and color correction unit 16 may refer to this information. Typically, for example, color correction unit 16 performs a correction to reduce the luminance of the displayed image if the illuminance of the ambient light is relatively high, and performs a correction to increase the luminance of the displayed image if the illuminance is relatively low.

[0047] In addition, the color correction unit 16 may calculate adjustment parameters for applying color correction to the target image, which is a color correction that offsets camera characteristics, for example, to create a linear relationship between the input gradation value of the display image to be captured and the output gradation value of the captured image.

[0048] [2. Examples of processing by information processing systems] Next, an example of a process in which the information processing system 1 corrects the color of a display image displayed on the display unit 22 will be described. Fig. 6 is a flowchart showing the flow of the process in this example. This example will also be described with reference to Fig. 5. However, in this embodiment, the color hues of some images included in Fig. 5 actually differ from those described above in the first embodiment.

[0049] In steps S101 to S103, the same processes as those in embodiment 1 are executed. Following the process in step S103, the process in step S203 is executed.

[0050] In step S203, the acquisition unit 14 acquires information about the ambient light incident on the screen of the display unit 22.

[0051] In step S204, the acquisition unit 14 acquires information indicating the camera characteristics of the imaging unit 20.

[0052] In step S104, color correction unit 16 calculates the monitor characteristics of display unit 22. In this example, the description will be given assuming that color correction unit 16 calculates the monitor characteristics of the first monitor and the second monitor, which are the monitor characteristics shown in FIG.

[0053] In step S205, the color correction unit 16 calculates adjustment parameters for performing color correction, such as gamma correction, on any image, corresponding to ambient light, the camera characteristics of the imaging unit 20, and the monitor characteristics of the display unit 22. The color correction unit 16 may also calculate adjustment parameters for performing color correction corresponding to ambient light, the camera characteristics, and the monitor characteristics separately, and the order of each correction is not limited to a specific order. The processing of the color correction unit 16 in step S104 and this step S205 is an example of a calculation step in the present disclosure.

[0054] In step S106, the color correction unit 16 uses the calculated adjustment parameters to correct the color of any image displayed on the display unit 22. The processing by the color correction unit 16 in step S106 is an example of a color correction step in the present disclosure.

[0055] 5 corresponding to this example corresponds to an image on which the color correction unit 16 has performed corrections corresponding to the ambient light, the camera characteristics of the image capture unit 20, and the monitor characteristics of the first monitor. Also, image 62 corresponds to an image on which the color correction unit 16 has performed corrections corresponding to the ambient light, the camera characteristics of the image capture unit 20, and the monitor characteristics of the second monitor. The colors of images 52 and 62 are different from each other.

[0056] In step S107, the display unit 22 displays the image corrected by the color correction unit 16 on the screen.

[0057] The color hues of images 58 and 68 in Figure 5 corresponding to this example are both very close to those of the actual food sample 60. This is because the effects of ambient light when the display image was captured, the camera characteristics of imaging unit 20, and the monitor characteristics of each monitor are offset by the correction by color correction unit 16 in step S106, and each monitor displays a color that is close to the true color of food sample 60. In other words, the display image of food sample 60 that is close to the true color displayed on display unit 22 and the actual food sample 60 are placed under the same ambient light, and so their colors appear close to each other.

[0058] Furthermore, following the process of step S107, the information processing system 1 may repeat the process from S102 onwards one or more times to further adjust the color of the image.

[0059] Note that the image color correction according to the ambient light and camera characteristics may be performed by the image capturing unit 20. In this configuration, the image capturing unit 20 functions as part of the acquisition unit 14 and the color correction unit 16.

[0060] Furthermore, the processing shown in the flowchart of FIG. 6 is not necessarily limited to being executed in the order described above in this example, and for example, the order in which the processing in step S203 and the processing in step S204 are executed may be reversed.

[0061] Furthermore, it is desirable for the color correction unit 16 to calculate adjustment parameters for correcting the color of the image in accordance with both the ambient light and the camera characteristics and the monitor characteristics, but this is not a required configuration, and the color correction unit 16 may be configured to calculate adjustment parameters in accordance with either one of the ambient light and the camera characteristics and the monitor characteristics.

[0062] According to the configuration of this example, with a simple configuration and without changing the display settings on the monitor side, it is possible to make the colors of objects, etc. in any image displayed on each monitor closer to the actual colors and display them uniformly across the monitors.

[0063] [Modifications of Embodiments 1 and 2] In the above embodiments 1 and 2, the color correction unit 16 corrects the color of any image displayed on the monitor (display unit 22), but the target of color correction performed by the color correction unit 16 is not limited to the displayed image.

[0064] For example, the information processing system 1 may be configured to include a printing unit instead of the display unit 22, and the color correction unit 16 may perform color correction on the print image printed by the printing unit.

[0065] That is, in a broad sense, the information processing device 10 includes an acquisition unit 14 and a color correction unit 16, and the acquisition unit 14 may be configured to acquire an output image output from an output unit (printing unit, display unit 22), the output image including an area with known color values, and a captured image including a real object 34. Furthermore, the color correction unit 16 may be configured to calculate adjustment parameters for correcting the color of any image output from the output unit, depending on both the color of the area in the output image included in the captured image and the color of the real object 34 included in the captured image. Furthermore, the printed image printed by the printing unit and the display image displayed on the monitor are examples of the output image.

[0066] [Software implementation example] The functions of the information processing device 10 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 12).

[0067] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0068] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0069] Furthermore, for example, the information processing device 10 may be realized as a smartphone or the like, and the program may be realized as an application for the smartphone or the like.

[0070] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0071] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0072] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0073] 1. Information Processing Systems 10. Information processing equipment 12 Control Unit 14 Acquisition Department 16 Color Correction Section 18 Memory section 20 Imaging unit (camera) 22 Display unit (monitor, output unit)

Claims

[Claim 1] an acquisition step of acquiring a plurality of captured images including a display image displayed on a monitor, the display image including an area having a known color value, and an actual object having a known color value, the captured images each including a display image on a different monitor; a calculation step of calculating, for each different monitor, an adjustment parameter for correcting a color of an arbitrary image displayed on the monitor, according to both a color of the area in the display image included in the captured image and a color of the actual object included in the captured image; Including, In the acquiring step, information indicating camera characteristics of a camera that captured the captured image and information regarding ambient light incident on a screen of the monitor are further acquired; In the calculation step, calculating monitor characteristics of the monitor for each different monitor by referring to a color of the area in the display image included in the captured image and a color of the actual object included in the captured image; calculating, for each different monitor, an adjustment parameter for correcting a color of an image displayed on the monitor, based on the information on the monitor characteristics, the camera characteristics, and the ambient light; 23. An information processing method comprising:

Images