Color confirmation system, color confirmation method, and program

Abstract

To cope with an individual difference in color appearance when checking the color of an object via a display.SOLUTION: A system according to the present invention includes: an observer registration unit to register an observer in an observer database; an observer selection unit to select the observer from the observer database via the display, the observer observing an image showing an object for observation; an observation light source selection unit to select an observation light source for observing the object; an image conversion unit to generate an image to be displayed on a display in which the color of an object image is converted into a color to be displayed on the display, by using each of a chart image that captures a chart and is captured with the same capturing condition as the object image that captures the object, the observation light source selected by the observation light source selection unit, a color-matching function according to the observer selected by the observer selection unit, and the color characteristic of the display; and an image display unit to display the image to be displayed on a display on the display.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a color confirmation system, a color confirmation method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, it has been common to capture an image of an object, transmit it to a remote location, and display the image on a display at the remote location to confirm the color of the object (see, for example, Patent Document 1 and Non-Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-176546 A [Non-patent literature]

[0004] [Non-Patent Document 1] Chiba University, National University Corporation, "Achieving high quality remote medical care with a simple color correction method! Remotely obtain vital information from smartphone videos," [online], September 28, 2020, PRTIMES, [Retrieved March 5, 2021], Internet<https: / / prtimes.jp / main / html / rd / p / 000000433.000015177.html> Summary of the Invention [Problem to be solved by the invention]

[0005] When checking the color of an object through a display, it may be difficult to check the color correctly due to individual differences in the appearance of the object color and the display color. In addition, when checking the color through a display, it is considered that actual objects such as color samples and proof copies, which have been used in conventional color checking of printed matter, are not used. Therefore, when checking the color through a display, there is a problem that it is difficult to check later whether the color checking was done correctly or not.

[0006] In view of the above-mentioned problems, an object of the present invention is to provide a color confirmation system, a color confirmation method, and a program that can accommodate individual differences in color appearance when confirming the color of an object via a display. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a color confirmation system according to one aspect of the present invention includes an observer registration unit that registers an observer who observes an image displayed on a display in an observer database, an observer selection unit that selects an observer who observes an image showing an object to be observed via the display from the observer database, an observation light source selection unit that selects an observation light source for observing the object when the image showing the object is observed via the display, and a color matching function according to a chart image in which a chart is captured under the same imaging conditions as an object image in which the object is captured, the observation light source selected by the observation light source selection unit, and the observer selected by the observer selection unit. an image conversion unit that generates an image for display in which the color of the object image is converted into a color to be displayed on the display, using a color conversion number and color characteristics of the display, respectively, and an image display unit that displays the image for display on the display, wherein a chart shown in the chart image has a known spectral reflectance, the spectral reflectance of the chart is stored in a chart spectral reflectance database, the color characteristics of the display are stored in a display characteristic database, the viewing illuminant is stored in a viewing illuminant database, and a color matching function according to the viewer selected by the viewer selection unit is selected from a color matching function database in which a plurality of color matching functions are stored.

[0008] A color confirmation method according to one aspect of the present invention includes an observer registration unit registering an observer who observes an image displayed on a display in an observer database, an observer selection unit selecting an observer who observes an image showing an object to be observed via the display from the observer database, an observation light source selection unit selecting an observation light source for observing the object when the image showing the object is observed via the display, and an image conversion unit selecting a chart image in which a chart is captured under the same imaging conditions as an object image in which the object is captured, the observation light source selected by the observation light source selection unit, a color matching function according to the observer selected by the observer selection unit, and a previous a color characteristic of the display, and a color matching function corresponding to the observer selected by the observer selection unit is selected from a color matching function database in which a plurality of color matching functions are stored.

[0009] A program according to one aspect of the present invention causes a computer to include an observer registration unit that registers an observer who observes an image displayed on a display in an observer database, an observer selection unit that selects from the observer database an observer who observes an image showing an object to be observed via the display, an observation light source selection unit that selects an observation light source for observing the object when the image showing the object is observed via the display, and a chart image in which a chart is captured under the same imaging conditions as an object image in which the object is captured, an observation light source selected by the observation light source selection unit, color matching functions according to the observer selected by the observer selection unit, and color characteristics of the display. and an image display unit that displays the image for display on the display, using each of the color characteristics of the object image and the color conversion unit and the color conversion unit, wherein a chart shown in the chart image has a known spectral reflectance, the spectral reflectance of the chart is stored in a chart spectral reflectance database, color characteristics of the display are stored in a display characteristic database, the viewing illuminant is stored in a viewing illuminant database, and a color matching function according to the viewer selected by the viewer selection unit is selected from a color matching function database in which a plurality of color matching functions are stored. Effect of the Invention

[0010] According to the present invention, it is possible to accommodate individual differences in color appearance when checking the color of an object via a display. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an overview of a color confirmation system 1 according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of the configuration of an image capturer terminal 10 according to a first embodiment. [Diagram 3] 2 is a block diagram showing an example of the configuration of a server device 20 according to the first embodiment. FIG. [Figure 4] 2 is a block diagram showing an example of the configuration of an observer terminal 30 according to the first embodiment. FIG. [Diagram 5] FIG. 4 is a diagram showing an example of a chart spectral reflectance DB 220 according to the first embodiment. [Figure 6] 10 is a diagram showing an example of an observation light source DB 221 according to the first embodiment. FIG. [Figure 7] FIG. 2 is a diagram showing an example of an observer DB 222 according to the first embodiment. [Figure 8] 4 is a diagram showing an example of a display characteristic DB 223 according to the first embodiment. FIG. [Figure 9] FIG. 4 is a diagram showing an example of a color matching function DB 224 according to the first embodiment. [Figure 10] 4 is a flowchart showing a flow of processing performed by the color confirmation system 1 according to the first embodiment. [Figure 11] FIG. 2 is a diagram showing an overview of a color confirmation system 1 according to a modified example of the first embodiment. [Figure 12] 10 is a flowchart showing a flow of processing performed by the color confirmation system 1 according to a modified example of the first embodiment. [Figure 13] 13A to 13C are diagrams illustrating a process in which the color confirmation system 1 according to the modified example of the first embodiment displays a determination image CHD. [Figure 14] 13A to 13C are diagrams illustrating a process in which the color confirmation system 1 according to the modified example of the first embodiment displays a determination image CHD. [Figure 15] 13A to 13C are diagrams illustrating a process in which the color confirmation system 1 according to the modified example of the first embodiment displays a determination image CHD. [Figure 16] 13A to 13C are diagrams illustrating a process in which the color confirmation system 1 according to the modified example of the first embodiment displays a determination image CHD. [Figure 17] FIG. 11 is a block diagram showing an overview of a color confirmation system 1A according to a second embodiment. [Figure 18] FIG. 11 is a block diagram showing an example of the configuration of an observer terminal 30A according to the second embodiment. [Figure 19]13 is a flowchart showing a flow of processing performed by a color confirmation system 1A according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] A color confirmation system 1 according to an embodiment will be described below with reference to the drawings. Fig. 1 is a block diagram showing an overview of the color confirmation system 1 according to the first embodiment. The color confirmation system 1 is a system in which a remote observer KU confirms the color of an object B via a display D in response to a request from an image-taking person SU.

[0013] 1, the color confirmation system 1 includes, for example, an image capturer terminal 10, a server device 20, and an observer terminal 30. The components of the color confirmation system 1 (the image capturer terminal 10, the server device 20, and the observer terminal 30) are communicatively connected via a communication network NW.

[0014] The image-capturer terminal 10 is a terminal device of the image-capturer SU. The image-capturer terminal 10 is, for example, a PC (Personal Computer), a tablet terminal, or a smartphone. The image-capturer SU is a user who requests confirmation of the color of the object B. The object B may be any object. For example, the object B is an object whose color may need to be confirmed, such as a colored printed matter or human skin.

[0015] The image-capturing person SU captures an image of an object B with a camera CA. The camera CA is a device equipped with a general imaging function, such as a digital camera or a camera (imaging unit 15 described later) built into a smartphone. If the camera CA is an imaging device separate from the image-capturing person terminal 10, the image captured by the camera CA (object image BG) is imported into the image-capturing person terminal 10 by connecting the camera CA and the image-capturing person terminal 10 with a USB cable, for example.

[0016] The photographer SU captures the chart CH under the same imaging conditions as those under which the object B was captured. The imaging conditions here refer to conditions under which the image was captured, such as the lighting at the time of capturing the image and the settings of camera parameters such as the exposure of the camera CA. The chart CH may be any chart as long as it is an object showing a color with a known spectral reflectance. When the color range of the object B is limited to a certain degree, a chart having a patch with a spectral reflectance close to the range of colors that the object B can take may be used. When the object B is limited to a certain degree and the spectral characteristics are limited to a certain degree, a chart having a patch with a spectral characteristic close to that of the object B may be used.

[0017] If there are no changes to the imaging conditions, it is sufficient to image the chart CH once. There is no need to image the chart CH every time the object B is imaged.

[0018] The image-taking person terminal 10 transmits to the server device 20 an image of the object B (object image BG) and an image of the chart CH (chart image CHG).

[0019] The server device 20 is a computer device that mediates between a request from the image-capturing person SU and confirmation by the observer KU. The server device 20 is, for example, a server device, a cloud, a PC, etc. The server device 20 generates an image (a display image, which will be described later) in which the color of the object image BG is converted according to the environment of the observer KU and the observer terminal 30. A method in which the server device 20 generates the display image will be described in detail later. The server device 20 transmits the display image to the observer terminal 30.

[0020] The observer terminal 30 is a terminal device of the observer KU. The observer terminal 30 is, for example, a server device, a PC, a tablet terminal, a smartphone, etc. The observer KU is a user who checks the color of the object B.

[0021] The observer KU inputs information for identifying the observer KU. For example, a list of observers registered in a database (an observer DB 222 described later) is displayed on the display screen of the observer terminal 30. The observer KU inputs information for identifying the observer KU by selecting a name or the like corresponding to the observer from the list displayed on the screen.

[0022] In addition, the observer KU selects an observation illuminant. The observation illuminant here is a light source selected by the observer KU as an observation illuminant for observing the object B. For example, when the observer KU wants to observe the object B under a light source of "natural light", the observer KU selects "natural light" as the observation illuminant.

[0023] The observer terminal 30 transmits the information input by the observer KU (information identifying the observer KU) and information indicating the observation light source selected by the observer KU to the server device 20. The observer terminal 30 receives the display image from the server device 20, and displays the received image on the display D. The observer KU checks the color of the image displayed on the display D.

[0024] 2 is a block diagram showing an example of the configuration of the image capturer terminal 10 according to the first embodiment. The image capturer terminal 10 includes, for example, a communication unit 11, a storage unit 12, a control unit 13, a display unit 14, and an image capture unit 15.

[0025] The communication unit 11 communicates with the server device 20 and the observer terminal 30. For example, the communication unit 11 transmits an object image BG and a chart image CHG to the server device 20.

[0026] The storage unit 12 is configured by a storage medium such as a HDD, a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a RAM (Random Access read / write Memory), a ROM (Read Only Memory), or a combination of these. The storage unit 12 stores programs for executing various processes of the image capturer terminal 10, and temporary data used when performing various processes.

[0027] The control unit 13 is realized by causing a CPU (Central Processing Unit) provided as hardware in the image capturer terminal 10 to execute a program. The control unit 13 comprehensively controls the image capturer terminal 10. The control unit 13 controls each of the communication unit 11, the storage unit 12, the display unit 14, and the image capture unit 15.

[0028] The control unit 13 includes, for example, a chart image acquisition unit 130, an object image acquisition unit 131, and a device control unit 132. The chart image acquisition unit 130 acquires, for example, a chart image CHG from the imaging unit 15 (camera CA) and outputs the acquired image to the device control unit 132. The object image acquisition unit 131 acquires an object image BG from the imaging unit 15 (camera CA) and outputs the acquired image to the device control unit 132. The device control unit 132 comprehensively controls the imager terminal 10. The device control unit 132 transmits, for example, the chart image CHG acquired from the chart image acquisition unit 130 and the object image BG acquired from the object image acquisition unit 131 to the server device 20 via the communication unit 11.

[0029] The display unit 14 includes a display device such as a liquid crystal display, and displays the object image BG and the chart image CHG according to the control of the control unit 13 .

[0030] The imaging section 15 includes an imaging device that captures an image, and captures images of the object B, the chart CH, and the like in response to control by the control section 13.

[0031] 3 is a block diagram showing an example of the configuration of the server device 20 according to the first embodiment. The server device 20 includes, for example, a communication unit 21, a storage unit 22, and a control unit .

[0032] The communication unit 21 communicates with the image capturer terminal 10 and the observer terminal 30. For example, the communication unit 21 receives an object image BG and a chart image CHG from the image capturer terminal 10. In addition, the communication unit 21 transmits an image to be displayed on a display to the observer terminal 30.

[0033] The storage unit 22 is configured by a storage medium such as a HDD, a flash memory, an EEPROM, a RAM, a ROM, or a combination of these. The storage unit 22 stores programs for executing various processes of the server device 20 and temporary data used when performing various processes.

[0034] The storage unit 22 stores, for example, a chart spectral reflectance DB 220, a viewing illuminant DB 221, an observer DB 222, a display characteristic DB 223, and a color matching function DB 224.

[0035] The chart spectral reflectance DB 220 is a database in which the spectral reflectance of the chart CH is stored (see FIG. 5).

[0036] The observation light source DB221 is a database that stores the spectral radiation distribution of the observation light source for observing the object B (see FIG. 6). The observation light source here is the observation light source for observing the object B, and is, for example, a light source selected by the observer KU. For example, when the observer KU wants to observe the object B under a light source of "natural light", the observer KU selects "natural light" as the observation light source.

[0037] In the above, the observation illuminant stored in the observation illuminant DB 221 is selected as the observation illuminant for observing the object B, but the present invention is not limited to this. The observation illuminant stored in the observation illuminant DB 221 may be used as information on a light source for illuminating a space in which the display D is provided.

[0038] The observer DB222 is a database in which observers KU are stored (see FIG. 7). The display characteristic DB223 is a database in which characteristics (color characteristics) of a display D are stored (see FIG. 8). The color matching function DB224 is a database in which color matching functions are stored (see FIG. 9).

[0039] 9, the color matching function DB 224 stores color matching functions for each observer KU, but is not limited to this. The color matching function DB 224 may be a database in which a plurality of representative color matching functions are stored.

[0040] Here, the chart spectral reflectance DB220 is an example of a "chart spectral reflectance database." The viewing illuminant DB221 is an example of a "viewing illuminant database." The observer DB222 is an example of an "observer database." The display characteristics DB223 is an example of a "display characteristics database." The color matching function DB224 is an example of a "color matching function database."

[0041] The control unit 23 is realized by causing a CPU, which is provided as hardware in the server device 20, to execute a program. The control unit 23 comprehensively controls the server device 20. The control unit 23 controls each of the communication unit 21 and the storage unit 22.

[0042] The control unit 23 includes, for example, an acquisition unit 230, an image conversion unit 231, an image output unit 232, and a device control unit 233. The acquisition unit 230 acquires images captured by the image capturer SU, i.e., an object image BG and a chart image CHG, and outputs the acquired images to the image conversion unit 231. The acquisition unit 230 also acquires information indicating an observation light source selected by the observer KU, and outputs the acquired information to the image conversion unit 231. The acquisition unit 230 also acquires information about the observer KU, and outputs the acquired information to the image conversion unit 231. Here, the information about the observer KU output by the acquisition unit 230 to the image conversion unit 231 is used, for example, to identify the color matching function of the observer during image conversion.

[0043] The image conversion unit 231 performs image conversion. The image conversion here means converting the color of the object image BG in accordance with the observer KU and the environment of the observer terminal 30. A method for generating an image to be displayed on a display will be described below.

[0044] (Method of generating images for display) Display image generating procedure (1): The image conversion unit 231 calculates the colorimetric value of each patch on the chart CH. The colorimetric value is a coordinate value in a color space. The image conversion unit 231 refers to the chart spectral reflectance DB 220 to acquire the spectral reflectance of the chart CH. The image conversion unit 231 refers to the viewing illuminant DB 221 to acquire the spectral radiation distribution of the viewing illuminant selected by the observer KU. The image conversion unit 231 also refers to the color matching function DB 224 to acquire the color matching function of the observer KU. The image conversion unit 231 calculates the colorimetric value of each patch on the chart CH using the acquired information, i.e., the spectral reflectance of the chart CH, the spectral radiation distribution of the viewing illuminant, and the color matching function of the observer KU.

[0045] Display image generation procedure (2): The image conversion unit 231 uses the pixel values ​​(e.g., RGB values) of the pixels in the portion corresponding to each patch of the chart image CHG and the colorimetric values ​​of each patch calculated in the display image generation procedure (1) to calculate a matrix (conversion matrix) for calculating the colorimetric values ​​from the pixel values ​​(e.g., RGB values) by, for example, multiple regression analysis.

[0046] Display image generating procedure (3): The image conversion unit 231 refers to the display characteristic DB 223 and acquires the display characteristics of the display D. The image conversion unit 231 uses the acquired display characteristics and the color matching function of the observer KU to calculate a color conversion table that describes the relationship between the device values ​​(e.g., RGB values) of the display D and the colorimetric values. The device values ​​are values ​​that are input to the display D when displaying colors on the display D.

[0047] The color conversion table here is, for example, a multidimensional lookup table. In this case, the image conversion unit 231 divides each RGB channel (pixel value) equally at an appropriate interval as the display characteristics, for example. The image conversion unit 231 divides the pixel values ​​0 to 255 into 51 levels for the RGB values, for example, in increments of 5. When the spectral radiation distribution of the display color for each combination of RGB values ​​divided into 51 levels is described in the display characteristics DB 223, the image conversion unit 231 calculates each colorimetric value for the spectral radiation distribution using the color matching function of the observer KU. This allows the image conversion unit 231 to generate a multidimensional lookup table in which the colorimetric values ​​for each area obtained by dividing the RGB values ​​at equal intervals are described. When the display characteristics described in the display characteristics DB 223 are different from the division set by the image conversion unit, the image conversion unit 231 may perform calculations by obtaining the spectral radiation distribution for the RGB values ​​corresponding to the set division by interpolation calculation or the like.

[0048] Alternatively, the color conversion table may be composed of a tone reproduction curve and a conversion matrix. In this case, the image conversion unit 231 acquires a spectral radiation distribution corresponding to the tone of each RGB channel (including the tone 255 of each RGB, i.e., data corresponding to the primary colors of RGB) from the characteristics described in the display characteristic DB 223. The image conversion unit 231 calculates each colorimetric value for the acquired spectral distribution using a color matching function for the observer KU. This allows the image conversion unit 231 to obtain a tone reproduction curve from the colorimetric values ​​corresponding to the tone of each RGB channel, and a conversion matrix from the colorimetric values ​​corresponding to each primary color of RGB.

[0049] The image conversion unit 231 uses the calculated color conversion table to calculate a conversion table for converting colorimetric values ​​into device values ​​(e.g., RGB values) of the display D. When the color conversion table is a multidimensional lookup table, the image conversion unit 231 equally divides the color space of the colorimetric values ​​(e.g., XYZ tristimulus values). The image conversion unit 231 searches for device values ​​(e.g., RGB values) for reproducing the colorimetric values ​​(e.g., XYZ values) via the display D for the colorimetric values ​​(e.g., XYZ values) corresponding to each lattice point obtained by dividing the color space, using the above multidimensional lookup table. For example, an optimization method or the like may be used for the search. The image conversion unit 231 can generate a table for calculating device values ​​(e.g., RGB values) of the display D corresponding to the colorimetric values ​​to be displayed, based on the search results. When the color conversion table is composed of a tone reproduction curve and a conversion matrix, the image conversion unit 231 calculates the inverse matrix of the conversion matrix, thereby making it possible to calculate the device values ​​from the colorimetric values.

[0050] Display image generation procedure (4): The image conversion unit 231 converts the pixel values ​​(e.g., RGB values) of the object image BG into colorimetric values ​​using the matrix calculated in the display image generation procedure (2). Then, the image conversion unit 231 converts the colorimetric values ​​into device values ​​(e.g., RGB values) of the display D using the conversion table or the gradation line and matrix calculated in the display image generation procedure (3).

[0051] The image conversion unit 231 generates an image for display by converting pixel values ​​(e.g., RGB values) of the object image BG into device values ​​(e.g., RGB values) of the display D. The image conversion unit 231 outputs the generated image for display to the image output unit 232.

[0052] The image output unit 232 outputs an image to be displayed on a display.

[0053] The device control unit 233 comprehensively controls the server device 20. For example, the device control unit 233 outputs the object image BG and the chart image CHG, which the communication unit 21 receives from the image capturer terminal 10, to the acquisition unit 230. In addition, the device control unit 233 outputs information for identifying the observer KU and information indicating the observation light source, which the communication unit 21 receives from the observer terminal 30, to the acquisition unit 230. The device control unit 233 transmits the display image output by the image output unit 232 to the observer terminal 30 via the communication unit 21.

[0054] 4 is a block diagram showing an example of the configuration of the observer terminal 30 according to the first embodiment. The observer terminal 30 includes, for example, a communication unit 31, a storage unit 32, a control unit 33, and a display unit 34. The communication unit 31 communicates with the server device 20 and the image capturer terminal 10. For example, the communication unit 31 transmits, to the server device 20, information for identifying the observer KU and information indicating the observation light source selected by the observer KU. The communication unit 31 also receives an image to be displayed on a display from the server device 20.

[0055] The storage unit 32 is configured by a storage medium such as a HDD, a flash memory, an EEPROM, a RAM, a ROM, or a combination of these. The storage unit 32 stores programs for executing various processes of the observer terminal 30 and temporary data used when performing various processes.

[0056] The control unit 33 is realized by causing a CPU provided as hardware in the observer terminal 30 to execute a program. The control unit 33 comprehensively controls the image capturer terminal 10. The control unit 33 controls each of the communication unit 31, the storage unit 32, and the display unit .

[0057] The display unit 34 includes a display device such as a general self-luminous display, such as an LCD monitor or an organic EL, and displays, under the control of the control unit 33, a screen for registering an observer KU, a screen for selecting an observer KU, a screen for selecting an observation light source, and images for display.

[0058] The control unit 33 includes, for example, an observer registration unit 330 , an observer selection unit 331 , an observation light source selection unit 332 , a display observation light source acquisition unit 333 , and an apparatus control unit 334 .

[0059] The observer registration unit 330 registers the observer KU in advance. For example, the observer KU uses an input device such as a keyboard connected to the observer terminal 30 to input information identifying the observer KU, such as information indicating a name, ID, color matching function, face photo, etc. The observer registration unit 330 acquires the information identifying the observer KU input by the observer KU, and transmits the acquired information to the server device 20. The information transmitted to the server device 20 (information identifying the observer KU) is registered by the server device 20, for example, in the observer DB 222.

[0060] The observer selection unit 331 selects an observer KU. For example, the observer KU visually checks a list (a list of observers registered in the observer DB 222) displayed on the display unit 34 of the observer terminal 30. The observer KU selects himself / herself from the list using an input device such as a keyboard. As a result, information indicating the selected observer KU is notified to the observer terminal 30. The observer selection unit 331 acquires information indicating the observer KU selected by the observer KU, and transmits the acquired information to the server device 20.

[0061] Alternatively, the observer selection unit 331 may select the observer KU by acquiring the ID of the observer KU. In this case, the color confirmation system 1 is a system that requires, for example, logging in using the ID of the observer KU when performing color confirmation. The observer terminal 30 acquires the ID of the observer KU that is input by the observer KU when logging in to the color confirmation system 1. The observer selection unit 331 selects the observer KU from the observer DB 222 based on the ID input to the observer terminal 30.

[0062] The observation light source selection unit 332 selects an observation light source. For example, the observer KU visually checks a list (a list of compatible observation light sources, for example, a list of observation light sources registered in the observation light source DB 221) displayed on the display unit 34 of the observer terminal 30. The observer KU selects an observation light source with which the observer KU wishes to observe the object B from the list, using an input device such as a keyboard. As a result, information indicating the observation light source selected by the observer KU is notified to the observer terminal 30. The observer selection unit 331 acquires information indicating the observation light source selected by the observer KU, and transmits the acquired information to the server device 20.

[0063] The process performed by the display observation light source acquisition unit 333 will be described in detail in a modified example of the second embodiment, which will be described later.

[0064] The device control unit 334 comprehensively controls the observer terminal 30. For example, the device control unit 334 transmits information indicating the name, etc. of the observer KU registered by the observer registration unit 330 to the server device 20 via the communication unit 21. The device control unit 334 transmits information specifying the observer KU selected by the observer selection unit 331 to the server device 20 via the communication unit 21. In addition, the device control unit 334 transmits information indicating the observation light source selected by the observation light source selection unit 332 to the server device 20 via the communication unit 21. The device control unit 334 outputs the display image received by the communication unit 21 from the server device 20 to the display unit 34.

[0065] 5 is a diagram showing an example of the chart spectral reflectance DB 220 according to the first embodiment. The chart spectral reflectance DB 220 is generated for each chart CH, for example. The chart spectral reflectance DB 220 includes items such as patch No. and spectral reflectance characteristics. The patch No. is identification information such as a number that uniquely identifies a patch shown on the chart CH. The spectral reflectance characteristics is information indicating the spectral reflectance characteristics of a patch identified by the patch No., for example, information indicating the spectral reflectance.

[0066] 6 is a diagram showing an example of the observation light source DB 221 according to the first embodiment. The observation light source DB 221 includes items such as an observation light source No. and a spectral radiation characteristic. The observation light source No. is identification information such as a number that uniquely identifies the observation light source. The spectral radiation characteristic is information indicating the spectral radiation characteristic of the observation light source identified by the observation light source No., for example, information indicating a spectral radiation distribution.

[0067] 7 is a diagram showing an example of the observer DB2221 according to the first embodiment. The observer DB222 includes items such as observer No., name, ID, color matching function, and face photo. The observer No. is identification information such as a number that uniquely identifies the observer KU. The ID is information indicating the ID (Identification) of the observer KU identified by the observer No. The color matching function is information indicating the color matching function of the observer KU identified by the observer No. The face photo is information indicating the face photo of the observer KU identified by the observer No.

[0068] 8 is a diagram showing an example of a display characteristic DB223 according to the first embodiment. The display characteristic DB223 is generated for each display D, for example. The display characteristic DB223 includes items such as an R value, a G value, a B value, and a spectral radiation characteristic. The R value is a value of a red component among pixel values ​​displayed in each pixel of the display D. The G value is a value of a green component among pixel values ​​displayed in each pixel of the display D. The B value is a value of a blue component among pixel values ​​displayed in each pixel of the display D. The spectral radiation characteristic is information indicating the spectral radiation characteristic of light emitted by the display D corresponding to a pixel value (RGB value) specified by the R value, the G value, and the B value, and is, for example, information indicating a spectral radiation distribution.

[0069] In the example of this figure, it is shown that the spectral radiation characteristic of display D corresponding to pixel value (0, 0, 0) is M1(λ), ​​and the spectral radiation characteristic of display D corresponding to pixel value (10, 0, 0) is M2(λ), where λ represents wavelength.

[0070] 9 is a diagram showing an example of the color matching function DB 224 according to the first embodiment. The color matching function DB 224 includes items such as an observer No. and a color matching function. The observer No. is identification information such as a number that uniquely identifies an observer KU. The color matching function is information indicating a color matching function of an observer KU identified by the observer No., and is, for example, information indicating tristimulus values ​​consisting of an X characteristic, a Y characteristic, and a Z characteristic.

[0071] In the example of this figure, the color matching functions of observer KU identified by observer No. (observer U1) are shown to have an X characteristic of x1(λ), ​​a Y characteristic of y1(λ), ​​and a Z characteristic of z1(λ). Also, the color matching functions of observer KU identified by observer No. (observer U2) are shown to have an X characteristic of x2(λ), a Y characteristic of y2(λ), and a Z characteristic of z2(λ). Here, λ indicates wavelength.

[0072] FIG. 10 is a flowchart showing the flow of the process performed by the color confirmation system 1 according to the first embodiment.

[0073] First, the color confirmation system 1 captures an image of the chart CH (step S10). For example, the image-capturing person SU captures an image of the chart CH by operating the camera CA. As a result, the image-capturing person terminal 10 acquires a chart image CHG and transmits the acquired chart image CHG to the server device 20. Note that unless the imaging conditions, such as the light source that illuminates the imaging target, the image input device (camera CA) used for imaging, and the camera parameter settings of the image input device, are changed, this step S10 only needs to be performed once, and there is no need to repeat step S10 every time the object B is imaged in the subsequent step S11.

[0074] Next, the color confirmation system 1 captures an image of the object B (step S11). For example, the image capturer SU operates the camera CA to capture an image of the object B under the same imaging conditions as those under which the chart CH was captured. As a result, the image capturer terminal 10 acquires an object image BG and transmits the acquired object image BG to the server device 20.

[0075] The color confirmation system 1 registers the observer KU (step S12). This step S12 may be a processing step that is performed in advance, separate from the series of flows shown in FIG. 10. For example, the observer KU inputs information for registering the observer KU, such as a name, an ID, and a color matching function, into the observer terminal 30. As a result, the observer terminal 30 acquires the information for registering the observer KU, and transmits the acquired information to the server device 20.

[0076] The server device 20 registers the observer KU by storing the information for registering the observer KU, acquired from the observer terminal 30 in step S12, in the observer DB 222. Furthermore, when the server device 20 acquires a color matching function for the observer KU in step S12, the server device 20 stores the acquired color matching function in the color matching function DB 224 in association with the observer KU.

[0077] The color confirmation system 1 selects the observer KU (step S13). For example, the observer KU selects himself / herself from a list of observers displayed on the observer terminal 30. As a result, the observer terminal 30 obtains information for identifying the observer KU, and transmits the obtained information to the server device 20.

[0078] The color confirmation system 1 selects an observation illuminant (step S14). For example, the observer KU selects an observation illuminant with which he / she wishes to observe the object B from a list of observation illuminants displayed on the observer terminal 30. As a result, the observer terminal 30 obtains information for identifying the observation illuminant, and transmits the obtained information to the server device 20.

[0079] In addition, the observation light source DB221 pre-stores observation light sources that may be used to observe the object B, and based on the information stored in the observation light source DB221, a list of observation light sources is displayed on the display unit 34 of the observer terminal 30.

[0080] The color confirmation system 1 converts the object image BG (step S15). Specifically, the color confirmation system 1 acquires the spectral reflectance of the chart image CHG captured in step S10 from the chart spectral reflectance DB 220. The color confirmation system 1 acquires the pixel value of each pixel of the object image BG captured in step S11. The color confirmation system 1 acquires the color matching function of the observer KU selected in step S13 from the color matching function DB 224. The color confirmation system 1 acquires the spectral radiation distribution of the observation light source selected in step S14 from the observation light source DB 221. In addition, the color confirmation system 1 acquires the characteristics of the display D from the display characteristic DB 223. Then, the color confirmation system 1 generates the image for display by converting the color of the object image BG into the color of the image for display by the above-mentioned procedure (1) to procedure (4) for generating an image for display.

[0081] The color confirmation system 1 displays the image for display (step S16). The color confirmation system 1 transmits the image for display generated by the server device 20 in step S15 to the observer terminal 30. The observer terminal 30 displays the image for display received from the server device 20 on the display unit 34.

[0082] As described above, the color confirmation system 1 according to the first embodiment includes the observer registration section 330, the observer selection section 331, the observation light source selection section 332, the image conversion section 231, and the display section 34 (one example of an "image display section"). The observer registration section 330 registers the observer KU in the observer DB 222 (observer database). The observer KU is an observer who observes an image displayed on the display D. The observer selection section 331 selects, from the observer DB 222, the observer KU who observes an image showing an object (image for display) via the display D. The observation light source selection section 332 selects an observation light source for observing the object B when the observer KU observes the image showing the object (image for display) via the display D. The image conversion section 231 generates an image for display. The image for display is an image in which the color of the object image BG is converted into a color to be displayed on the display D. The image conversion unit 231 generates an image for display using a chart image CHG captured under the same imaging conditions as the object image BG, an observation illuminant selected by the observation illuminant selection unit 332, a color matching function corresponding to the observer KU selected by the observer selection unit 331, and color characteristics of the display D. The display unit 34 displays the image for display. The chart CH shown in the chart image CHG is a chart whose spectral reflectance is known and whose spectral reflectance is stored in a chart spectral reflectance DB220 (chart spectral reflectance database). The color characteristics of the display D are stored in a display characteristic DB223 (display characteristic database). The observation illuminant is stored in a viewing illuminant DB221 (viewing illuminant database). The color matching function corresponding to the observer KU selected by the observer selection unit 331 is selected from a color matching function DB224 (color matching function database). A plurality of color matching functions are stored in the color matching function DB224.

[0083] As a result, in the color confirmation system 1 according to the first embodiment, the color matching function of the observer KU can be used to generate an image for display on the display, which makes it possible to accommodate individual differences in color appearance when checking the color of an object through the display D.

[0084] Furthermore, in the color confirmation system 1 according to the first embodiment, the color matching function DB 224 may store a color matching function for each observer registered in the observer DB 222. In this case, the image conversion section 231 selects a color matching function for the observer KU (the observer selected by the observer selection section 331) using the color matching function DB 224. This makes it possible for the color confirmation system 1 according to the first embodiment to select a color matching function according to each observer KU, and to generate an image to be displayed on a display according to the appearance of the observer KU.

[0085] Furthermore, in the color confirmation system 1 according to the first embodiment, a plurality of representative color matching functions may be stored in the color matching function DB 224. In this case, the image conversion unit 231 selects a color matching function for the observer KU using the color matching function DB 224. As a result, the color confirmation system 1 according to the first embodiment provides the same effects as the above-mentioned embodiments.

[0086] Furthermore, in the color confirmation system 1 according to the first embodiment, the observer DB 222 may store an ID (an observer ID that uniquely identifies an observer KU registered in the observer DB 222). In this case, the observer selection unit 331 selects an observer KU (an observer who observes the object image via the display D) according to the ID input by the observer KU. As a result, in the color confirmation system 1 according to the first embodiment, it is possible to select an observer KU by the easy method of having the observer KU input an ID.

[0087] (Modification of the first embodiment) Here, a modification of the first embodiment will be described. This modification differs from the above-described embodiment in that the viewer registration section 330 determines color matching functions for viewer KU.

[0088] Fig. 11 is a diagram showing an overview of a color confirmation system 1 according to a modified example of the first embodiment. As shown in Fig. 11, in this modified example, an observer KU carries a chart CH# with a known spectral reflectance when performing registration.

[0089] The chart CH# here may be the same as the chart CH captured by the photographer SU in the first embodiment, or may be a different chart. The chart CH# may also be a color chart made up of patches that reflect the spectral reflectance characteristics of the object B. The chart CH# may be a reflective object or a self-luminous object. The chart CH# may also be a printed matter in which the object B to be observed is spectrally reproduced. In the following, a case in which the chart CH# is a reflective object will be described as an example.

[0090] The observer terminal 30 displays a plurality of judgment images CHD (judgment images CHD1 to CHD3...) for determining the color matching functions of the observer KU on the display D. The judgment image CHD is an image in which the color of each patch on the chart CH# is converted into a color corresponding to each of the plurality of color matching functions. A method for generating the judgment image CHD will be described below.

[0091] (Method of generating judgment image CHD) Judgment image CHD generating procedure (1): The observer registration unit 330 acquires the spectral reflectance of the chart CH#. The observer registration unit 330 may acquire the spectral reflectance of the chart CH# by referring to the chart spectral reflectance DB 220, or may acquire the spectral reflectance of the chart CH# by referring to another database.

[0092] Judgment image CHD generating procedure (2): The display observation light source acquisition unit 333 acquires a display observation light source. The display observation light source is, for example, a light source provided in the environment in which the display D is observed, and is a light source that illuminates a chart CH# held by the observer KU.

[0093] The display observation light source acquisition unit 333 displays, for example, a list (a list of compatible observation light sources, for example, a list of observation light sources registered in the observation light source DB 221) displayed on the display unit 34 of the observer terminal 30. The observer KU acquires the display observation light source from the list displayed on the display unit 34 of the observer terminal 30. For example, if the room in which the display D is installed is illuminated by a fluorescent light, the observer KU selects "fluorescent light" as the display observation light source. The display observation light source acquisition unit 333 acquires information indicating the display observation light source selected by the observer KU, and outputs the acquired information to the observer registration unit 330.

[0094] Alternatively, the display observation light source acquisition unit 333 may acquire information indicating the display observation light source from an image. For example, when a panoramic image of the environment in which the display D is observed is acquired by the display observation environment image acquisition unit 335 described later, the display observation light source acquisition unit 333 estimates the spectral radiation distribution of the display observation light source based on the color of the chart CH# shown in the panoramic image, and outputs the light source having the estimated spectral radiation distribution to the viewer registration unit 330 as the display observation light source.

[0095] Judgment image CHD generation procedure (3): The observer registration unit 330 calculates the colorimetric values ​​(X1, Y1, Z1) of each patch in the chart CH# using the spectral reflectance of the chart CH# obtained in the judgment image CHD generation procedure (1), the display observation illuminant obtained in the judgment image CHD generation procedure (2), and a corresponding color matching function, as shown in the following equation (1).

[0096] X1 = ∫R(λ)P(λ)x1(λ)dλ Y1 = ∫R(λ)P(λ)y1(λ)dλ Z1 = ∫R(λ)P(λ)z1(λ)dλ …(1) however, R(λ) is the spectral reflectance of the patch on chart CH# P(λ) is the spectral irradiance distribution of the display observation light source. (x1(λ), ​​y1(λ), ​​z1(λ)) are the color matching functions for observer U1 λ is the wavelength

[0097] Procedure (4) for generating a judgment image CHD: The observer registration unit 330 calculates a color conversion table that describes the relationship between the display color (e.g., RGB value) of the display D and the colorimetric value. The method for calculating the conversion table is similar to the procedure (3) for generating an image for display described above, and therefore the description thereof will be omitted.

[0098] Judgment image CHD generation procedure (5): The observer registration unit 330 converts the colorimetric values ​​of each patch of the chart CH# calculated in the judgment image CHD generation procedure (3) into a display color (e.g., RGB value) of the display D using the conversion table calculated in the judgment image CHD generation procedure (4). The observer registration unit 330 displays the converted display color of the display D on the display D as the color of each patch of the chart CH corresponding to the color matching function of the observer KU.

[0099] The observer registration unit 330 repeats the above-mentioned judgment image CHD generation procedure (1) to judgment image CHD generation procedure (5) while changing the observer's color matching function, generates multiple judgment images CHD (judgment images CHD1 to CHD3, ...) showing charts CH# corresponding to the respective color matching functions of the multiple observers, and displays the generated images on the display D.

[0100] The observer KU compares the multiple judgment images CHD displayed on the display D with the colors of the chart CH# at hand, and selects the judgment image CHD that the observer KU feels has the closest color appearance to the color of the chart CH# at hand. The observer registration unit 330 acquires information indicating the judgment image CHD selected by the observer KU, and determines the color matching function corresponding to the acquired judgment image CHD as the color matching function for that observer KU. The observer terminal 30 transmits information indicating the color matching function for the observer KU determined by the observer registration unit 330 to the server device 20. The server device 20 stores the information indicating the color matching function for the observer KU acquired from the observer terminal 30 in the observer DB 222.

[0101] 12 is a flowchart showing a flow of processing performed by the color confirmation system 1 according to the modified example of the first embodiment. The color confirmation system 1 acquires the spectral reflectance of the chart CH# by, for example, referring to the chart spectral reflectance DB 220 (step S20). The color confirmation system 1 acquires the spectral radiation distribution of the display viewing light source by, for example, referring to the viewing light source DB 221 based on information indicating the light source selected by the viewer KU (step S21). The color confirmation system 1 calculates the colorimetric values ​​of each patch of the chart CH# for each color matching function stored in the color matching function DB 224 based on the spectral reflectance of the chart CH# and the spectral radiation distribution of the display viewing light source (step S22). The color confirmation system 1 calculates a conversion table for converting the colorimetric values ​​into the display color of the display D based on the color characteristics of the display D stored in the display characteristics DB 223 and the color matching functions stored in the color matching function DB 224, and converts the colorimetric values ​​of each patch in the chart CH# into the display color of the display D corresponding to the color matching function using the calculated conversion table (step S23).

[0102] The color confirmation system 1 displays on the display D an image (judgment image CHD) showing the color of each patch on the chart CH# converted into the display color on the display D (step S24). The color confirmation system 1 acquires information showing the judgment image CHD selected by the observer KU (step S25). Here, the image selected by the observer KU is the judgment image CHD that the observer KU felt to have the closest color appearance to the color of the chart CH# in hand. The color confirmation system 1 determines the color matching function corresponding to the judgment image CHD selected by the observer KU as the color matching function for that observer KU (step S26).

[0103] Here, a variation of displaying the judgment image CHD to the observer KU will be described. In order to accurately determine the color matching function for the observer KU, it is necessary to appropriately select the judgment image CHD that the observer KU feels is similar in color. On the other hand, for example, when a large number of color matching functions are stored in the color matching function DB 224, a large number of judgment images CHD corresponding to each color matching function are generated, so it is necessary to efficiently determine the judgment image CHD that the observer KU feels is similar in color from the large number of judgment images CHD. In this way, it is necessary to accurately or efficiently select the judgment image CHD that the observer KU feels is similar in color depending on the situation. For this reason, when the judgment image CHD is displayed on the display D, the display is devised.

[0104] 13 to 16 are diagrams illustrating a process in which the color confirmation system 1 according to the modified example of the first embodiment displays the determination image CHD.

[0105] 13, for example, the observer registration unit 330 displays a plurality of determination images CHD side by side on a display D. The observer KU compares a chart CH# at hand with each of the plurality of determination images CHD displayed on the display D, and selects the determination image CHD which the observer feels is closest in color. The observer registration unit 330 determines the color matching function used in calculating the determination image CHD selected by the observer KU as the color matching function for that observer.

[0106] As shown in Fig. 14, the observer registration unit 330 may sequentially display one judgment image CHD selected from a plurality of judgment images CHD on the display D. For example, the observer registration unit 330 enables the observer KU to forward or reverse the images displayed on the display D by operating the keyboard or mouse, etc., and to switch and display the judgment image CHD. The observer KU, for example, compares a chart CH# at hand with the judgment image CHD while switching between their displays, and selects the image which the observer feels to be closest in color. The observer registration unit 330 determines the color matching function used in the calculation of the judgment image CHD selected by the observer KU as the color matching function for that observer.

[0107] As shown in FIG. 15, the observer registration unit 330 may display two judgment images CHD selected from a plurality of judgment images CHD on the display D. The observer KU compares the two judgment images CHD on the left and right with a chart CH# at hand, and selects the judgment image CHD that the observer KU feels is closer in color. The observer registration unit 330 leaves the selected judgment image CHD, and switches the unselected image to the next judgment image CHD for display (winner). That is, the observer registration unit 330 performs an operation to sequentially display the remaining judgment images CHD while leaving the judgment image CHD selected by the observer KU, and repeats this operation for all judgment images CHD. In this way, the observer registration unit 330 determines the judgment image CHD finally selected by the observer KU. The observer registration unit 330 determines the color matching function used in the calculation of the determined judgment image CHD as the color matching function for the observer KU.

[0108] Alternatively, the observer registration unit 330 generates in advance a set (pair) of two judgment images CHD selected from a plurality of judgment images CHD. Then, as shown in the example of FIG. 15, one of the generated pairs is displayed on the display D. The observer KU compares the left and right images displayed on the display D with a chart CH# at hand and selects an image that the observer KU feels is similar in color. The observer registration unit 330 allows the observer KU to select an image that the observer feels is similar in color for each of all pairs generated in advance. The observer registration unit 330 extracts an image selected by the observer KU for each of all pairs generated in advance, generates pairs again using the extracted images, displays the regenerated pairs on the display D, and allows the observer KU to select an image that the observer feels is similar in color (tournament method). The observer registration unit 330 determines the color matching function used in the calculation of the judgment image CHD that ultimately won (selected by the observer KU) as the color matching function for that observer KU.

[0109] Alternatively, the observer registration unit 330 may display two judgment images CHD selected from the multiple judgment images CHD on the display D, have the observer KU select the image that the observer KU feels is similar in color, and repeat the above judgment for all image combinations (paired comparison). After all judgments have been completed, the observer registration unit 330 determines the color matching function used in calculating the image that has been selected most frequently by the observer KU as the color matching function for that observer KU.

[0110] As shown in Fig. 16, the observer registration unit 330 may display an image corresponding to one patch selected from the chart CH# on the display D as the judgment image CHD, instead of displaying images of all patches in the chart CH#. This method can be applied to each of the display examples in Figs. 13 to 15 described above. For example, instead of displaying an image of the entire chart CH# in each of the judgment images CHD in Figs. 13 to 15, an image of some patches of the chart CH# is displayed. The observer registration unit 330 evaluates whether observer KU perceives the colors as similar on a patch-by-patch basis, rather than on the entire chart CH#, and determines the color matching function from the result.

[0111] As described above, in the color confirmation system 1 according to the first embodiment, the observer registration unit 330 displays, on the display D, an image (color sample object image) of the chart CH# (sample object serving as a color sample) for determining a color matching function for the observer KU. The observer registration unit 330 determines a color matching function according to the observer KU (an observer registered in the observer DB 222) based on input information input by the observer KU. The input information here is information indicating the degree to which the color (first color) of each patch when the chart CH# is directly observed (visually recognized) and the color (second color) of the judgment image (image of the sample object) displayed on the display D look the same. In this way, in the color confirmation system 1 according to the first embodiment, it is possible to determine a color matching function for the observer KU, and it is possible to determine a color matching function suitable for the observer KU even if the observer KU does not recognize which color matching function is his / her own color matching function.

[0112] Furthermore, in the color confirmation system 1 according to the first embodiment, the chart CH# may be a color chart made up of patches reflecting the spectral reflectance characteristics of the object B to be observed. In this case, the color matching function for the observer KU can be determined based on the color reflecting the spectral reflectance characteristics of the object B, making it possible to determine a color matching function that better reflects the color of the object B, and enabling the observer KU to more accurately confirm the color of the object B.

[0113] Furthermore, in the color confirmation system 1 according to the first embodiment, the chart CH# may be a printed matter on which the object B to be observed is spectrally reproduced. In this case, the color of the object B is spectrally reproduced on the chart CH#. Therefore, the color matching function for the observer KU can be determined based on an image in which the object B is spectrally reflected. This makes it possible to determine a color matching function that better reflects the color of the object B, enabling the observer KU to more accurately confirm the color of the object B.

[0114] Furthermore, in the color confirmation system 1 according to the first embodiment, the chart CH# may be a self-luminous object. In this case, it is sufficient to know at least the spectral radiation distribution of the chart CH#, and there is no need to acquire the display observation light source. This makes it possible to omit the display observation light source acquisition unit 333.

[0115] Second embodiment Here, a second embodiment will be described. This embodiment differs from the above-described embodiments in that an image of the environment in which color confirmation is performed (display observation environment) is captured via a display D.

[0116] Fig. 17 is a block diagram showing an outline of a color confirmation system 1A according to the second embodiment. As shown in Fig. 17, the color confirmation system 1A of the present embodiment includes an observer terminal 30A and an all-sky image capturing device 30B.

[0117] The omnidirectional image capturing device 30B captures a omnidirectional image in a display viewing environment. The image captured by the omnidirectional image capturing device 30B is an image of the display viewing environment. This image shows, for example, a display D, a display display image displayed on the display D, and a viewer KU.

[0118] The observer terminal 30A acquires an image (display observation environment image) captured by the omnidirectional image capturing device 30B, and generates observation evidence based on the acquired image. The observation evidence is information that serves as evidence that color confirmation has been performed. For example, the observation evidence indicates the observation result (result of color confirmation), the observer KU, and an image of the observation environment. The image of the observation environment here is, for example, a omnidirectional image captured by the omnidirectional image capturing device 30B of the observer KU confirming the color.

[0119] Fig. 18 is a block diagram showing an example of the configuration of an observer terminal 30A according to the second embodiment. As shown in Fig. 18, the observer terminal 30A includes, for example, a device control unit 334, a display viewing environment image acquisition unit 335, and a display viewing environment confirmation unit 336.

[0120] The display viewing environment image acquisition section 335 acquires the panoramic image captured by the panoramic image capturing device 30B, and outputs the acquired image to the display viewing environment confirmation section 336.

[0121] Here, a general omnidirectional camera can be used as the omnidirectional image capturing device 30B. The omnidirectional image capturing device 30B captures an image of an environment in which an observer KU who checks the color of the object B observes the display D from a remote location.

[0122] The timing at which the omnidirectional image capturing device 30B captures an image may be determined arbitrarily. For example, the image may be captured at a specific timing, such as when an image is displayed. The specific timing may be, for example, when the observer KU logs in to the color confirmation system 1, when the observer KU is registered, when the observer KU is selected, or when an image for display is displayed on the display D. Alternatively, the image may be captured periodically at a predetermined interval. Note that the spectral sensitivity of the omnidirectional camera in the omnidirectional image capturing device 30B is assumed to be known in advance by performing a measurement or the like.

[0123] In addition, when the observer KU makes some judgment in color confirmation at a remote location, for example, when the remote color proofing is approved, when some diagnosis is made in remote medical treatment, etc., the display observation environment may be captured by the omnidirectional image capturing device 30B. This makes it possible to record the situation at the time of the judgment by the observer KU as evidence.

[0124] A chart with a known spectral reflectance may be placed in the display viewing environment, which makes it possible to estimate the illumination of the display viewing environment (display viewing environment light source) from the image portion of the image (display viewing environment image) captured by the omnidirectional image capturing device 30B where the chart is captured.

[0125] The display viewing environment checking section 336 checks whether color checking at a remote location is being performed correctly, based on the display viewing environment image acquired from the display viewing environment image acquiring section 335 .

[0126] Specifically, the display viewing environment confirmation unit 336 extracts a face image area of ​​the viewer KU from the display viewing environment image. The display viewing environment confirmation unit 336 acquires a preregistered face image of the viewer KU selected by the viewer selection unit 331 from the viewer DB 222. The display viewing environment confirmation unit 336 compares the face image extracted from the display viewing environment image with the face image of the viewer KU acquired from the viewer DB 222, and determines whether the viewer KU matches the viewer KU selected by the viewer selection unit 331. If there is a mismatch in the viewer KU as a result of the comparison, the display viewing environment confirmation unit 336 may display a mismatch on the display D, or may consider the selection by the viewer selection unit 331 to be incorrect and change the viewer KU. In this case, an image for display is generated again using the color matching function of the changed viewer KU, and the regenerated image is displayed on the display unit 34.

[0127] Note that biometric information of the observer KU may be used together with or instead of the face image of the observer KU. The biometric information here is, for example, a feature amount extracted from the face image. In this way, it is possible to determine whether the observer KU matches the observer KU selected by the observer selection unit 331 without using the face image itself.

[0128] Furthermore, the display viewing environment confirmation unit 336 extracts an image area of ​​the display portion of the display D from the display viewing environment image. The display viewing environment confirmation unit 336 calculates colorimetric values ​​of the displayed image (image for display) from pixel values ​​(e.g., RGB values) of the extracted image area. The display viewing environment confirmation unit 336 compares the calculated colorimetric values ​​with the colorimetric values ​​to be displayed on the display D calculated by the image conversion unit 231. If there is a mismatch in the colorimetric values ​​as a result of the comparison, the display viewing environment confirmation unit 336 notifies the observer KU by, for example, displaying a message indicating a mismatch on the display D.

[0129] Fig. 19 is a flowchart showing the flow of processing performed by the color confirmation system 1A according to the second embodiment. Steps S30 to S32 and S34 to S36 in this flowchart are similar to steps S10 to S12 and S14 to S16 in the flowchart of Fig. 10 described above, and therefore their explanation will be omitted.

[0130] In step S32, the observer registration unit 330 registers the observer KU. In addition to the process shown in step S12 described above, the observer registration unit 330 acquires face information of the observer to be registered, and registers the acquired face image in association with the name, ID, etc. of the observer KU. The observer registration unit 330 may acquire the face information of the observer KU by having the observer KU select a face image of the observer KU that has been captured in advance by some means.

[0131] Furthermore, the observer registration section 330 may register the omnidirectional image captured by the omnidirectional image capturing device 30B at the time of registration as a face image of the observer KU.

[0132] In step S33, the observer selection unit 331 selects an observer KU. Instead of the process shown in step S13 described above, the observer selection unit 331 causes the omnidirectional image capturing device 30B to capture an omnidirectional image, and extracts a face image of a user who is currently going to view the display D from the image. The observer selection unit 331 compares the extracted face image with each face image of the observer KU acquired from the observer DB 222, identifies an observer KU that matches the extracted face image, and selects the identified observer KU as the observer KU who will perform color confirmation. Note that the above is optional, and the same process as step S13 may be performed in step S33.

[0133] In step S37, the omnidirectional image capturing device 30B captures an image of the display viewing environment. The display viewing environment image acquiring unit 335 acquires an omnidirectional image (display viewing environment image) captured by the omnidirectional image capturing device 30B. The omnidirectional image capturing device 30B is set so that the display viewing environment image includes at least the display screen of the display D to be observed and the face of the viewer KU.

[0134] In step S38, the display viewing environment checking section 336 acquires the display viewing environment image from the display viewing environment image acquiring section 335, and performs the following checks based on the acquired image.

[0135] Confirmation item (1): The display observation environment confirmation unit 336 performs face confirmation of the observer KU. Specifically, the display observation environment confirmation unit 336 extracts a face image area in which the face portion of the observer KU is captured from the display observation environment image. The display observation environment confirmation unit 336 judges whether or not the extracted face image area matches the registered face image of the observer KU selected by the observer selection unit 331. If it is judged that there is no match as a result of the judgment, the display observation environment confirmation unit 336 issues a warning by, for example, displaying a message indicating that the observer KU has not been correctly selected or that the observer KU does not match.

[0136] In this case, the display viewing environment confirmation section 336 may search for face images registered in the viewer DB 222 based on the face image extracted from the image, and specify a viewer KU that matches the face image extracted from the image. In this case, the display viewing environment confirmation section 336 may automatically switch the viewer KU, or may display a message on the display D asking the user whether to switch the viewer KU, to ask the user whether to switch the viewer KU.

[0137] Confirmation item (2): The display viewing environment confirmation unit 336 confirms the image for display on the display. Specifically, the display viewing environment confirmation unit 336 extracts a screen area in which the display screen of the display D is captured from the display viewing environment image. The display viewing environment confirmation unit 336 determines whether or not the colorimetric values ​​of the image for display on the display displayed in the extracted screen area match the colorimetric values ​​intended by the image conversion unit 231 by the following procedure.

[0138] Step (2-1): The display viewing environment confirmation unit 336 calculates a spectral radiation distribution from pixel values ​​(e.g., RGB values) of an image to be displayed on the display. The display viewing environment confirmation unit 336, for example, refers to the display characteristics DB 223 and acquires a spectral radiation distribution of an image corresponding to a combination of display colors (e.g., RGB values) displayed by the display D. The display viewing environment confirmation unit 336 calculates a multidimensional lookup table that calculates a spectral radiation distribution corresponding to pixel values ​​of an image to be displayed on the display from the correspondence relationship between the display colors and the spectral radiation distribution acquired from the display characteristics DB 223.

[0139] Here, the display characteristics DB 223 stores the spectral radiation distribution of the image displayed for all combinations of the RGB values ​​of the display colors displayed by the display D into equally divided areas. For RGB values ​​other than the display colors corresponding to the grid of the divided areas, it is possible to calculate the spectral radiation distribution of the image displayed when the RGB values ​​of any display color on the display D are input by performing an interpolation calculation from the surrounding grid points.

[0140] Step (2-2): The display viewing environment confirmation unit 336 calculates the camera RGB values ​​from the spectral radiation distribution calculated in step (2-1) and the spectral sensitivity of the omnidirectional image capturing device 30B. The camera RGB values ​​calculated here will match the RGB values ​​displayed in the screen area extracted from the image if the image for display is displayed with correct colorimetric values.

[0141] Step (2-3): The display viewing environment confirmation unit 336 calculates the difference between the RGB values ​​of the image to be displayed on the display and the camera RGB values ​​calculated in step (2-2). The display viewing environment confirmation unit 336 compares the calculated difference with a predetermined threshold. If the difference is smaller than the threshold, the display viewing environment confirmation unit 336 determines that the display is correct. On the other hand, if the difference is larger than the threshold, the display viewing environment confirmation unit 336 determines that the display is not correct.

[0142] The above-mentioned processes shown in steps S37 and S38 may be executed periodically, not only after the display image is displayed. Alternatively, they may be executed at any timing in response to a user's instruction. These processes may be executed when some kind of judgment is made during the color confirmation work, such as when an instruction to correct the color is given or when the color is approved. By executing the processes at such various timings, it is possible to ensure that the color confirmation is performed correctly. The image captured in step S37 may be recorded in combination with various judgments in the process of confirming the color, such as a judgment that the color is OK or NG, or a judgment to instruct a change in the color tone. This allows the situation when the color is confirmed remotely, such as who looked at what, how the judgment was made, etc., to be confirmed at a later date.

[0143] As described above, the color confirmation system 1A according to the second embodiment further includes a display observation environment image acquisition unit 335 and a display observation environment confirmation unit 336. The display observation environment image acquisition unit 335 acquires a panoramic image of the observation environment in which the display is observed. The display observation environment confirmation unit 336 determines whether or not the color of the object B is reproduced in the display display image displayed on the display D, based on the image acquired by the display observation environment image acquisition unit 335. In this way, the color confirmation system 1A according to the second embodiment can leave evidence of whether or not color confirmation has been performed correctly in a remote location, and can confirm that color confirmation has been performed correctly even without meeting face-to-face.

[0144] Moreover, in the color confirmation system 1A according to the second embodiment, the display viewing environment image acquisition section 335 acquires an image including an observer observing the image displayed on the display D. The display viewing environment confirmation section 336 selects the observer observing the image displayed on the display D, based on the image acquired by the display viewing environment image acquisition section 335. As a result, in the color confirmation system 1A according to the second embodiment, it is possible to automatically select the observer KU, even if the observer KU does not perform an operation to input an ID or the like.

[0145] (Modification of the second embodiment) Here, a modified example of the second embodiment will be described. This modified example differs from the above-described embodiment in that a chart with a known spectral reflectance is placed in the display viewing environment.

[0146] The omnidirectional image capturing device 30B captures an image including a chart placed in the display viewing environment.

[0147] The display viewing environment confirmation unit 336 extracts a chart area in which a chart is captured from an image (display viewing environment image) captured by the omnidirectional image capturing device 30B. The display viewing environment confirmation unit 336 estimates the spectral radiation distribution of the light source illuminating the display viewing environment from the RGB values ​​of the chart area extracted from the image, based on the spectral reflectance of the chart and the spectral sensitivity of the omnidirectional image capturing device 30B. An existing technique is used as the estimation method for estimating the spectral radiation distribution.

[0148] When registering an observer KU, the observer registration unit 330 applies the method described in the modified example of the first embodiment described above to determine color matching functions for that observer KU, and uses the spectral radiation distribution estimated by the display viewing environment confirmation unit 336 as the spectral radiation distribution of the light source that illuminates the display viewing environment.

[0149] As described above, in the color confirmation system 1A according to the modified example of the second embodiment, a chart with a known spectral reflectance is placed in the viewing environment in which the display D is viewed. The display viewing environment image acquisition section 335 acquires an image including the chart placed in the viewing environment in which the display D is viewed. The viewer registration section 330 estimates the spectral radiation characteristics of the light source that illuminates the viewing environment in which the image displayed on the display D is viewed, based on the image acquired by the display viewing environment image acquisition section 335. As a result, in the color confirmation system 1A according to the modified example of the second embodiment, it is possible to automatically estimate the display viewing light source even if the viewer KU does not specify the display viewing light source.

[0150] In at least one of the above-described embodiments, a case has been described in which the process related to color confirmation is triggered by a request for color confirmation from the image-capturer SU. However, the present invention is not limited to this. For example, the observer KU may check the colors of a group of images collectively uploaded to the server device 20 by the image-capturer SU at any timing.

[0151] The above describes each embodiment of the present invention. Note that the color confirmation system 1, the imager terminal 10, the server device 20, and the observer terminal 30 in each of the above-mentioned embodiments may be realized in whole or in part by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. Furthermore, the above program may be for realizing part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in a computer system, or may be realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0152] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0153] 1. Color Check System 10. Photographer's terminal 20 Server device 30 Observer terminal 130 Chart image acquisition unit 131 Object image acquisition unit 230 Acquisition Department 231 Image conversion unit 232 Image output unit 330 Observer Registration Department 331 Observer Selection Section 332 Observation light source selection section 333 Display Observation Light Source Acquisition Unit 335 Display Observation Environment Image Acquisition Unit 336 Display Observation Environment Check Unit 34 Display unit (image display unit)

Claims

1. an observer registration unit that registers an observer who observes an image displayed on a display in an observer database; an observer selection unit that selects an observer who is to observe an image showing an object to be observed via the display from the observer database; an observation light source selection unit that selects an observation light source for observing the object when an image showing the object is observed via the display; an image conversion unit that generates an image for display in which colors of the object image are converted into colors to be displayed on the display, using a chart image obtained by capturing an image of a chart captured under the same capturing conditions as the object image obtained by capturing the object, an observation light source selected by the observation light source selection unit, a color matching function according to the observer selected by the observer selection unit, and color characteristics of the display; an image display unit that displays the display image on the display; Equipped with The chart shown in the chart image has a known spectral reflectance, and the spectral reflectance of the chart is stored in a chart spectral reflectance database; The color characteristics of the display are stored in a display characteristics database; The viewing illuminant is stored in a viewing illuminant database; the color matching function according to the observer selected by the observer selection unit is selected from a color matching function database in which a plurality of color matching functions are stored. A color confirmation system.

2. the color matching function database stores color matching functions in association with each observer registered in the observer database; the image conversion unit uses the color matching function database to select a color matching function according to the observer selected by the observer selection unit.

2. The color verification system of claim 1.

3. The color matching function database stores a plurality of representative color matching functions, the image conversion unit uses the color matching function database to select a color matching function according to the observer selected by the observer selection unit.

2. The color verification system of claim 1.

4. The observer database stores an observer ID that uniquely identifies an observer registered in the observer database, the observer selection unit selects, from the observer database, an observer who will observe the image of the object through the display, in response to the input observer ID. The color confirmation system according to any one of claims 1 to 3.

5. the observer registration unit displays, on the display, an image of a sample object serving as a color sample for determining a color matching function, and determines a color matching function corresponding to an observer to be registered in the observer database based on input information input by the observer to be registered in the observer database; The input information is information indicating a degree to which a first color when the sample object is directly observed and a second color in the image of the sample object displayed on the display have the same color appearance. A color confirmation system according to claim 1, claim 3 or claim 4.

6. The sample object is a reflective object; a display observation light source acquisition unit that acquires a display observation light source that illuminates the sample object and is provided in a space in which the display is observed; the observer registration unit generates a color sample object image to be displayed on the display based on the spectral reflectance of the sample object and the display observation light source acquired by the display observation light source acquisition unit, and determines color matching functions of the observer to be registered in the observer database.

6. The color confirmation system of claim 5.

7. The sample object is a color chart consisting of patches reflecting the spectral reflectance characteristics of an object to be observed.

7. The color confirmation system of claim 6.

8. The sample object is a print in which the object to be observed is spectrally reproduced.

7. The color confirmation system of claim 6.

9. The sample object is a self-luminous object.

6. The color confirmation system of claim 5.

10. a display viewing environment image acquisition unit that acquires a panoramic image of a viewing environment in which the display is viewed; a display viewing environment determination unit that determines whether or not the color displayed on the display is the color that should be displayed when the image for display on the display is displayed, based on the image acquired by the display viewing environment image acquisition unit; Further comprising: A color confirmation system according to any one of claims 1 to 9.

11. the display viewing environment image acquisition unit acquires an image including an observer observing an image displayed on the display; the observer selection unit selects an observer who will observe the image displayed on the display based on the image acquired by the display observation environment image acquisition unit. The color verification system of claim 10.

12. A chart having a known spectral reflectance is placed in a viewing environment for viewing the display; the display viewing environment image acquisition unit acquires an image including a chart arranged in a viewing environment in which the display is viewed. the observer registration unit estimates, based on the image acquired by the display observation environment image acquisition unit, a spectral radiation characteristic of a light source that illuminates an observation environment in which an image displayed on the display is observed; A color confirmation system according to claim 10 or 11.

13. an observer registration unit registers an observer who observes the image displayed on the display in an observer database; an observer selection unit selects an observer who observes an image showing an object to be observed via the display from the observer database; an observation light source selection unit selecting an observation light source for observing the object when an image showing the object is observed via the display; an image conversion unit generates an image for display in which colors of the object image are converted into colors to be displayed on the display, using a chart image obtained by capturing an image of a chart captured under the same capturing conditions as those of the object image obtained by capturing the object, an observation light source selected by the observation light source selection unit, a color matching function according to the observer selected by the observer selection unit, and color characteristics of the display; an image display unit displays the display image on the display; A color confirmation method, comprising: The chart shown in the chart image has a known spectral reflectance, and the spectral reflectance of the chart is stored in a chart spectral reflectance database; The color characteristics of the display are stored in a display characteristics database; The viewing illuminant is stored in a viewing illuminant database; the color matching function according to the observer selected by the observer selection unit is selected from a color matching function database in which a plurality of color matching functions are stored. How to check the color.

14. Computer, an observer registration unit that registers an observer who observes an image displayed on a display in an observer database; an observer selection unit that selects an observer who is to observe an image showing an object to be observed via the display from the observer database; an observation light source selection unit that selects an observation light source for observing the object when an image showing the object is observed via the display; an image conversion unit that generates an image for display in which colors of the object image are converted into colors to be displayed on the display, using a chart image obtained by capturing an image of a chart captured under the same capturing conditions as the object image obtained by capturing the object, an observation light source selected by the observation light source selection unit, a color matching function according to the observer selected by the observer selection unit, and color characteristics of the display; an image display unit that displays the display image on the display; A program for causing a computer to function as a The chart shown in the chart image has a known spectral reflectance, and the spectral reflectance of the chart is stored in a chart spectral reflectance database; The color characteristics of the display are stored in a display characteristics database; The viewing illuminant is stored in a viewing illuminant database; the color matching function according to the observer selected by the observer selection unit is selected from a color matching function database in which a plurality of color matching functions are stored. program.

Images