Information processing apparatus, information processing method, and program

Abstract

To provide a technique for reducing a work load for performing color matching between cameras for a color whose spectral reflectance is not acquired in advance.SOLUTION: An information processing device includes an acquisition unit configured to acquire a first image captured by a first imaging device under first illumination, and a first calculation unit configured to calculate spectral reflectances of first one or more colors corresponding to one or more pixels in the first image based on a pixel value of each of the one or more pixels in the first image, spectral reflectances of three primary colors measured in advance under second illumination, spectral sensitivity of the first imaging device, and a spectral distribution of the first illumination.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present embodiment relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, when photographing using multiple cameras, a process for color matching between the cameras is performed before photographing. For example, this process involves photographing a color chart placed in the photographing environment with a reference camera (a camera that serves as a reference for color matching) and a target camera (a camera that is the target of color matching). Then, based on the pixel values ​​of the color chart in each captured image, parameters for matching the colors of the reference camera and the target camera are generated. However, in actual photographing situations, there is a limit to the amount of time available for photographing, and some users may not have the time to photograph a color chart, or may find photographing for color matching to be a hassle.

[0003] Therefore, a technique for performing color matching between cameras without photographing a color target in the shooting environment is known. This technique converts pixel values ​​of a color target image photographed under the illumination of a preliminary environment into pixel values ​​when the image is photographed under illumination of a shooting environment different from the illumination of the preliminary environment, and generates color matching parameters. Patent Document 1 discloses a technique for estimating the spectral reflectance of pixel values ​​of an image based on the spectral reflectance of a prepared color target or the like, the spectral distribution of the illumination, and the spectral sensitivity of the camera, and then converting the estimated spectral reflectance into pixel values ​​when the image is photographed under a different illumination environment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6517766 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology of Patent Document 1 requires that the spectral reflectance of the subject to be photographed be prepared in advance in order to estimate the spectral reflectance of pixel values. For example, if the subject to be photographed is human skin, the spectral reflectances of many colors corresponding to human skin must be prepared in advance. However, there are many different subjects that may be photographed, and it is not easy to prepare a color chart that includes the colors of all possible subjects and measure the spectral reflectances in advance.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a technology that reduces the workload required to perform color matching between cameras for colors whose spectral reflectance has not been acquired in advance. [Means for solving the problem]

[0007] In order to solve the above problem, the present invention provides an information processing device comprising: an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; and a first calculation means for calculating the spectral reflectance of each of one or more first colors corresponding to the one or more pixels in the first image based on the pixel values ​​of each of one or more pixels in the first image, the spectral reflectances of the three primary colors measured in advance under a second illumination, the spectral sensitivity of the first imaging device, and the spectral distribution of the first illumination. [Effects of the Invention]

[0008] According to the present invention, it is possible to reduce the workload of performing color matching between cameras for colors for which the spectral reflectance has not been acquired in advance.

[0009] Other features and advantages of the present invention will become more apparent from the accompanying drawings and the following detailed description of the preferred embodiment of the present invention. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 shows a system configuration. [Figure 2] FIG. 1 is a block diagram showing the arrangement of an information processing apparatus 102 according to a first embodiment. [Figure 3] FIG. 10 is a diagram showing an example of camera spectral sensitivity. [Figure 4] FIG. 10 is a diagram showing an example of the spectral distribution of lighting. [Figure 5] 10A and 10B are diagrams showing examples of spectral reflectance of primary color RGB patches. [Figure 6] FIG. 10 is a diagram showing an example of a primary color RGB patch. [Figure 7] 4 is a flowchart showing the detailed operation of a spectral reflectance calculation unit 202. [Figure 8] FIG. 10 is a block diagram showing the arrangement of an information processing apparatus 102 according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] [First embodiment] <System configuration> 1 is a diagram showing a system configuration. This system includes an image capturing device 100, an image capturing device 101, an information processing device 102, a display device 103, and lighting 104 (first lighting) in the image capturing environment.

[0013] The imaging devices 100 and 101 transmit images obtained by photographing a subject to the information processing device 102 via a video line. The imaging device 100 is an imaging device that serves as a reference for color matching (hereinafter, may be referred to as a "reference camera"), and the imaging device 101 is an imaging device that is the target of color matching (hereinafter, may be referred to as a "target camera"). Below, the reference camera and the target camera may be collectively referred to as a "camera."

[0014] 1, the imaging devices 100 and 101 transmit captured images to the information processing device 102 via video lines, but a configuration without using video lines may also be employed. For example, the captured images may be transferred from the imaging devices 100 and 101 to the information processing device 102 via a recording medium such as a memory card capable of storing captured images. As another example, the imaging devices 100 and 101 and the information processing device 102 may be connected to a local area network (LAN) by wire or wirelessly, and the captured images may be transferred from the imaging devices 100 and 101 to the information processing device 102 via the LAN.

[0015] The information processing device 102 generates color matching parameters based on the images received from the reference camera and the target camera, the spectral distribution of the lighting 104 at the shooting site, and the spectral sensitivities of the reference camera and the target camera, etc. The generated color matching parameters are transmitted from the information processing device 102 to the target camera via a recording medium such as a memory card or a LAN.

[0016] The display device 103 displays a user interface (UI) when generating color matching parameters in the information processing device 102, as well as images and information for confirming the effects of the generated color matching parameters. The display device 103 receives image information and other additional information from the information processing device 102 via a cable conforming to a common image standard such as HDMI or DVI.

[0017] <Configuration of information processing device 102> 2 is a block diagram showing the configuration of an information processing device 102 according to the first embodiment. The information processing device 102 includes a storage unit 200, a pixel value extraction unit 201, a spectral reflectance calculation unit 202, an RGB calculation unit 203, and a parameter generation unit 204. The functions of each unit of the information processing device 102 are realized by, for example, a recording medium (memory) storing a program and one or more processors that execute the program.

[0018] The storage unit 200 stores the spectral sensitivity of each camera (hereinafter, sometimes referred to as "camera spectral sensitivity") estimated in advance in a previous environment and the spectral distribution of the lighting 104 at the shooting site. The storage unit 200 also stores the spectral reflectance of the primary colors RGB (three primary colors) measured in advance under lighting (second lighting) in the previous environment (not shown).

[0019] The storage unit 200 also stores the spectral reflectance of the color chart measured under the illumination of the preliminary environment. This color chart includes one or more color patches corresponding to one or more colors. Therefore, the storage unit 200 stores the spectral reflectance of each of the one or more colors (third one or more colors) of the color chart.

[0020] 3 is a diagram showing an example of camera spectral sensitivities, which include a spectral sensitivity (B spectral sensitivity 110) that recognizes the B component of the RGB components, a spectral sensitivity (G spectral sensitivity 111) that recognizes the G component of the RGB components, and a spectral sensitivity (R spectral sensitivity 112) that recognizes the R component of the RGB components.

[0021] In this embodiment, the reference camera and the target camera have different spectral sensitivities. Therefore, for example, if the reference camera has the spectral sensitivities shown in FIG. 3, the target camera will have a different spectral sensitivities from those shown in FIG. 3.

[0022] Fig. 4 is a diagram showing an example of the spectral distribution of lighting. In the example of Fig. 4, a spectral distribution 113 is an example of the spectral distribution of an LED.

[0023] In this embodiment, the illumination 104 of the shooting environment and the illumination of the previous environment have different spectral distributions. Therefore, for example, if the illumination 104 of the shooting environment has the spectral distribution 113 shown in Fig. 4, the illumination of the previous environment will have a different spectral distribution from that shown in Fig. 4.

[0024] The storage unit 200 may store a plurality of spectral distributions measured in advance and corresponding to a plurality of types of illumination (for example, halogen, tungsten, and ambient light (morning, daytime, and night)). In this case, the user can select a spectral distribution corresponding to the type of illumination 104 in the image capture environment from the plurality of stored spectral distributions as the spectral distribution of the illumination 104 in the image capture environment. Alternatively, the user may attach a spectroscopic sensor or the like to the image capture device 100 or 101 and acquire the spectral distribution of the illumination 104 in the image capture environment in real time at the image capture site.

[0025] Fig. 5 is a diagram showing examples of the spectral reflectance of the primary color RGB patches, including the spectral reflectance 114 of the primary color B patch, the spectral reflectance 115 of the primary color G patch, and the spectral reflectance 116 of the primary color R patch.

[0026] FIG. 6 is a diagram illustrating an example of a primary color RGB patch. The primary color RGB patch 105 is used by a user to measure the spectral reflectance of the reference primary colors RGB as a preliminary preparation. The primary color RGB patch 105 includes a primary color R patch 106, a primary color G patch 107, and a primary color B patch 108. The user faces the primary color RGB patch 105 directly against a spectroradiometer under the lighting of the preliminary environment and measures the spectral reflectance of the primary color RGB patch 105. As an example, the primary color RGB patch 105 may be made of a material that is highly sensitive to the peak of the spectral wavelength of each RGB pixel (a material that is highly sensitive to the spectral wavelength of each RGB pixel). Note that instead of the spectral reflectance measured using the primary color RGB patch 105, a value measured by a spectroradiometer using light generated by a device that generates peaks at specific spectral wavelengths, such as a laser or LED, may be used as the spectral reflectance of the primary colors RGB.

[0027] The pixel value extraction unit 201 extracts pixel values ​​(RGB values) of colors that are important in shooting at the shooting site (hereinafter referred to as "important colors") from an image captured by a camera under the lighting 104 of the shooting environment. The pixel value extraction unit 201 may extract pixel values ​​of multiple important colors. The pixel values ​​of each of the extracted one or more important colors (first one or more colors) are output to the spectral reflectance calculation unit 202. Note that the captured image from which the important colors are extracted may be an image captured by a reference camera (reference image) or an image captured by a target camera (target image). The pixel value extraction unit 201 may also extract pixel values ​​of the important colors from both the reference image and the target image. In other words, the pixel value extraction unit 201 may extract pixel values ​​of one or more important colors (first one or more colors) from one of the reference image and the target image (first image) and also extract pixel values ​​of one or more important colors (second one or more colors) from the other (second image).

[0028] The method of selecting pixels of important colors can be, for example, a method in which a user selects them from the captured image on a UI, or a method in which the pixel value extraction unit 201 automatically selects them based on the frequency of appearance of pixel values ​​in the captured image. Therefore, for example, the pixel value extraction unit 201 selects one or more pixels in the reference image based on the frequency of appearance of pixel values ​​in the reference image, and extracts the pixel values ​​of each of the selected one or more pixels as pixel values ​​of one or more important colors. Alternatively, the pixel value extraction unit 201 may treat all pixels in the captured image as pixels of important colors and extract the pixel values ​​of each pixel in the captured image.

[0029] The spectral reflectance calculation unit 202 calculates the spectral reflectance of an important color based on the pixel values ​​of the important colors extracted by the pixel value extraction unit 201, the spectral reflectance of the primary color RGB patches 105 stored in the storage unit 200, the camera spectral sensitivity, and the spectral distribution of the lighting 104 in the shooting environment. As the camera spectral sensitivity, the spectral sensitivity of the reference camera is used for the pixel values ​​of the important colors extracted from the reference image, and the spectral sensitivity of the target camera is used for the pixel values ​​of the important colors extracted from the target image.

[0030] First, the spectral reflectance calculation unit 202 converts the pixel values ​​(RGB values) of the important colors extracted by the pixel value extraction unit 201 into tristimulus values ​​(hereinafter, also referred to as "XYZ values"). The conversion from RGB values ​​to XYZ values ​​is performed, for example, according to the following equation (1). TIFF2026004029000002.tif2169

[0031] In equation (1), M is a matrix coefficient for converting RGB values ​​into XYZ values. For example, the matrix coefficient M may be calculated using reference values ​​of the RGB primary colors or a reference white. Alternatively, the 3x3 matrix specified in ITU-R BT.709 may be used as the matrix coefficient M.

[0032] Next, the spectral reflectance calculation unit 202 calculates predicted values ​​(XYZ values) when the primary color RGB patch 105 is photographed under the lighting 104 of the shooting environment, based on the spectral reflectance of the primary color RGB patch 105, the camera spectral sensitivity, and the spectral distribution of the lighting 104 of the shooting environment. The predicted values ​​(XYZ values) are calculated according to the following formula (2). TIFF2026004029000003.tif8141

[0033] In equation (2), Ch represents the spectral reflectance of the color chart. Here, since the primary color RGB patch 105 is used as the color chart, Ch represents the spectral reflectance of the primary color RGB patch 105. L represents the spectral distribution of the illumination 104, and C represents the camera spectral sensitivity. c represents the color, which is either R, G, or B (when c = R, an X value is obtained; when c = G, a Y value is obtained; and when c = B, a Z value is obtained). λ represents the spectral wavelength. Considering that the color chart and primary color RGB patch 105 are composed of multiple patches and that the spectral reflectance and spectral intensity are treated as data in 10-nm increments, the spectral data can be expressed as a 41-dimensional vector from 380 nm to 780 nm. A color chart with N patches can be treated as a matrix with 41 × N elements (when the primary color RGB patch 105 is used as the color chart in equation (2), N is 3). The type of illumination is generally expressed as P types. Based on the above, the elements of equation (2) are defined as follows: TIFF2026004029000004.tif2675TIFF2026004029000005.tif2561TIFF2026004029000006.tif2633

[0034] In the above, the spectral reflectance and spectral intensity are treated as data in 10-nm increments, but this is not limiting and they may be treated as data in 1-nm increments, for example. In this case, the spectral data becomes a 401-dimensional vector.

[0035] Next, the spectral reflectance calculation unit 202 calculates the ratio between the XYZ values ​​of the important colors calculated by equations (1) and (2) and the XYZ values ​​of the primary color RGB patch 105. The ratio is found from the XYZ values ​​of the important colors and the XYZ values ​​of the primary colors RGB under the same lighting environment (i.e., the environment of the lighting 104). For example, the ratio is calculated according to the following equation (3). TIFF2026004029000007.tif20132

[0036] Finally, the spectral reflectance calculation unit 202 calculates the spectral reflectance of the important color by multiplying the ratio between the XYZ value of the important color calculated by equation (3) and the XYZ value of the primary color RGB patch 105 by the spectral reflectance of the primary color RGB patch 105. The calculated spectral reflectance is output to the RGB calculation unit 203. The detailed operation of the spectral reflectance calculation unit 202 will be described later.

[0037] The RGB calculation unit 203 calculates the camera shooting values ​​(RGB values) at the shooting location based on the spectral reflectances of the important colors received from the spectral reflectance calculation unit 202, the spectral reflectances of the color chart stored in the storage unit 200, the camera spectral sensitivity, and the spectral distribution of the lighting 104 in the shooting environment. The calculation of the camera shooting values ​​is performed for each of the reference camera and the target camera. When calculating the shooting values ​​of the reference camera, the spectral sensitivity of the reference camera is used as the camera spectral sensitivity, and when calculating the shooting values ​​of the target camera, the spectral sensitivity of the target camera is used as the camera spectral sensitivity. The camera shooting values ​​(RGB values) calculated here are the camera shooting values ​​(RGB values) of each color when multiple colors corresponding to multiple patches in the color chart and multiple colors including one or more important colors are photographed at the shooting location. In other words, the calculation here is performed for each camera and for each color. In the following, the color chart corresponding to the spectral reflectance of the color chart stored in the memory unit 200 may be referred to as the "first color chart," and the virtual color chart that combines multiple colors of the first color chart with one or more important colors may be referred to as the "second color chart."

[0038] First, the RGB calculation unit 203 calculates the camera's shooting prediction values ​​(XYZ values) according to equation (2). If the number of patches in the first color chart is N and the number of important colors is Q, the second color chart contains (N+Q) colors and can be treated as a matrix with 41 × (N+Q) elements. Based on the above, Ch in equation (2) is defined as follows: TIFF2026004029000008.tif2675

[0039] Next, the RGB calculation unit 203 converts the calculated predicted shooting values ​​(XYZ values) into RGB values. The conversion from the XYZ values ​​to the RGB values ​​is performed, for example, according to the following equation (4). TIFF2026004029000009.tif2177

[0040] The matrix coefficient M in equation (4) is the inverse matrix of the matrix coefficient M used in equation (1). The RGB values ​​calculated from equation (4) are output to the parameter generating unit 204.

[0041] The parameter generation unit 204 receives from the RGB calculation unit 203 the camera predicted values ​​(RGB values) of each color of the second color chart for each of the reference camera and the target camera, and generates color matching parameters based on the received camera predicted values. The color matching parameters are parameters that bring the camera predicted values ​​of the target camera closer to the camera predicted values ​​of the reference camera. In other words, the color matching parameters are parameters for performing color matching between images captured by the reference camera and the target camera. If the camera predicted value of the reference camera is Rkot and the camera predicted value of the target camera is Tkot, the color matching parameters can be calculated as Rkot-Tkot. By performing this calculation for each color of the second color chart, color matching parameters for each of multiple colors, including one or more important colors, are calculated.

[0042] The information processing device 102 transmits the generated color matching parameters to the target camera, thereby making it possible to match (bring closer) the color of the target camera to the color of the reference camera.

[0043] In this embodiment, taking into consideration that there will be a difference between the RGB values ​​extracted from the image and the RGB values ​​calculated from the camera spectral sensitivity, in order to eliminate the difference, a configuration is used in which the spectral reflectance is calculated from the RGB values ​​extracted from the image, and the RGB values ​​calculated from the spectral reflectance and the camera spectral sensitivity are used to generate color matching parameters.

[0044] In this way, the information processing device 102 generates color matching parameters based on a plurality of colors including one or more colors of the first color chart and one or more important colors (at least one of one or more important colors of the reference image and one or more important colors of the target image). Therefore, according to this embodiment, even if the subject includes a color not included in the first color chart (a color whose spectral reflectance has not been acquired in advance), it is possible to perform highly accurate color matching for one or more important colors corresponding to one or more pixel values ​​extracted by the pixel value extraction unit 201 without significantly increasing the workload of the user.

[0045] Note that, in this embodiment, it is not essential to use the first color chart. The information processing device 102 may generate color matching parameters based on one or more important colors (at least one of one or more important colors of the reference image and one or more important colors of the target image) without using one or more colors of the first color chart. In this case, the second flowchart described above does not include the colors of the first flowchart, but includes one or more important colors corresponding to one or more pixel values ​​extracted by the pixel value extraction unit 201. Even in this case, it is possible to perform high-precision color matching for one or more important colors without significantly increasing the user's workload.

[0046] <Detailed Operation of Spectral Reflectance Calculation Unit 202> 7 is a flowchart showing the detailed operation of the spectral reflectance calculation unit 202. In S100, the spectral reflectance calculation unit 202 receives pixel values ​​of important colors from the pixel value extraction unit 201. Here, as an example, it is assumed that there is one important color and the pixel values ​​are R=218, G=206, and B=193, but as mentioned above, there may be multiple important colors.

[0047] In S101, the spectral reflectance calculation unit 202 converts the received pixel values ​​(RGB values) of important colors into XYZ values ​​according to equation (1). As the matrix coefficient M in equation (1), for example, a coefficient calculated using the reference values ​​of the three RGB primary colors or a reference white is used. Alternatively, the 3x3 matrix specified in ITU-R BT.709 may be used as the matrix coefficient M. For example, the XYZ values ​​are calculated as follows: X = 0.3397, Y = 0.6070, Z = 0.3245. TIFF2026004029000010.tif20161

[0048] In S102, the spectral reflectance calculation unit 202 acquires the spectral reflectance of the primary color RGB patch 105 stored in the storage unit 200, the camera spectral sensitivity, and the spectral distribution of the lighting 104 in the shooting environment. In this example, it is assumed that the pixel values ​​of the important colors are extracted from the reference image. Therefore, in S102, the spectral sensitivity of the reference camera is acquired as the camera spectral sensitivity.

[0049] In S103, the spectral reflectance calculation unit 202 calculates (estimates) predicted photographing values ​​(XYZ values) when the primary color RGB patches 105 are photographed under the lighting 104 of the photographing environment, based on the spectral reflectance of the primary color RGB patches 105, the camera spectral sensitivity, and the spectral distribution of the lighting 104 of the photographing environment. The calculation here is performed according to formula (2), and the predicted photographing values ​​(XYZ values) of the primary color RGB patches are calculated as follows: X = 2.6838 (the sum of the X value of the primary color R patch, the X value of the primary color G patch, and the X value of the primary color B patch), Y = 5.0113 (the sum of the Y value of the primary color R patch, the Y value of the primary color G patch, and the Y value of the primary color B patch), and Z = 2.8266 (the sum of the Z value of the primary color R patch, the Z value of the primary color G patch, and the Z value of the primary color B patch). TIFF2026004029000011.tif21123

[0050] In S104, the spectral reflectance calculation unit 202 estimates the spectral reflectance of the pixel value of the important color. The spectral reflectance of the patches of the first color chart is assumed to be data in 10 nm increments. Therefore, in order to match the spectral reflectance of the patches of the first color chart, the spectral reflectance of the important color is also treated as data in 10 nm increments, resulting in a 41-dimensional vector from 380 nm to 780 nm. First, the spectral reflectance calculation unit 202 calculates the ratio of the XYZ values ​​of the primary color RGB patch 105 and the important color according to equation (3): X = 0.1266, Y = 0.1211, Z = 0.1148. TIFF2026004029000012.tif21136

[0051] As described above, the ratio of each of XYZ is calculated by dividing the XYZ values ​​of the important color by the XYZ values ​​of the primary color RGB patch.

[0052] Next, the spectral reflectance calculation unit 202 calculates (estimates) the spectral reflectance of the important color by multiplying each of the spectral reflectances of the primary color RGB patches by the ratio of each of X, Y, and Z. As described above, the calculated spectral reflectance of the important color is output to the RGB calculation unit 203.

[0053] Thereafter, the RGB calculation unit 203 calculates the camera capture predicted values ​​(XYZ values) for each camera and for each color, as described above. For example, for the important colors used as examples in the description of FIG. 7, the camera capture predicted values ​​of the target camera are calculated as X=0.3277, Y=0.6027, and Z=0.3333.

[0054] <Summary of the First Embodiment> As described above, according to the first embodiment, the information processing device 102 acquires a first image captured by a first imaging device (reference camera or target camera) under first illumination (illumination 104 of the imaging environment). The information processing device 102 also calculates the spectral reflectance of each of the first one or more colors (one or more important colors) corresponding to one or more pixels in the first image, based on the pixel values ​​of each of the one or more pixels in the first image, the spectral reflectances of the three primary colors measured in advance under second illumination (illumination of the previous environment), the spectral sensitivity of the first imaging device, and the spectral distribution of the first illumination.

[0055] Therefore, according to this embodiment, it is possible to reduce the workload for performing color matching between cameras for colors for which the spectral reflectance has not been acquired in advance (colors not included in the first color chart).

[0056] As described above, it is not essential to use the first color chart in this embodiment. Furthermore, the information processing device 102 may calculate the spectral reflectance of each of one or more important colors (one or more second colors) for one or more pixels of a second image captured by a second imaging device (one of the reference camera and the target camera, which is different from the first imaging device) under the first illumination, in addition to the first image.

[0057] [Second embodiment] In the first embodiment, the processing in a situation where the spectral sensitivities of both the reference camera and the target camera are known (stored in the storage unit 200) has been described. In contrast, in the second embodiment, the processing in a case where the spectral sensitivities of the target camera are unknown will be described. In this embodiment, the basic configuration of the system is the same as in the first embodiment (see FIG. 1). Below, differences from the first embodiment will be mainly described.

[0058] The target camera of this embodiment holds color matching parameters for the reference camera that are generated in advance based on a first color chart photographed under a certain lighting condition, and can output a captured image that matches the color of the reference camera under a certain lighting condition.

[0059] 8 is a block diagram showing the configuration of an information processing device 102 according to the second embodiment. The information processing device 102 includes a storage unit 200, a pixel value extraction unit 301, a spectral reflectance calculation unit 302, and an image generation unit 303. The functions of each unit of the information processing device 102 are realized by, for example, a recording medium (memory) storing a program and one or more processors that execute the program.

[0060] The storage unit 200 of this embodiment is generally the same as the storage unit 200 of the first embodiment. However, in this embodiment, the spectral sensitivity of the target camera does not need to be stored in the storage unit 200.

[0061] The pixel value extraction unit 301 extracts pixel values ​​from the captured image captured by the target camera and outputs them to the spectral reflectance calculation unit 302. While the pixel value extraction unit 201 in the first embodiment extracted pixel values ​​of one or more important colors from one or both of the target image and the reference image, the pixel value extraction unit 301 in the second embodiment extracts pixel values ​​contained in the entire target image. Note that the target image in the second embodiment has been color-matched using the color-matching parameters for the reference camera described above. Therefore, each pixel value extracted by the pixel value extraction unit 301 is a pixel value that has been color-matched to the reference camera.

[0062] The spectral reflectance calculation unit 302 calculates the spectral reflectance of each pixel value (the spectral reflectance of each color corresponding to each pixel in the target image) based on the pixel values ​​extracted by the pixel value extraction unit 301, as well as the spectral reflectance of the primary color RGB patch 105, the spectral sensitivity of the reference camera, and the spectral distribution of the lighting 104 in the shooting environment, all of which are stored in the storage unit 200. The method of calculating the spectral reflectance here is the same as the calculation method used by the spectral reflectance calculation unit 202 in the first embodiment. However, while in the first embodiment the spectral reflectance was calculated for the pixel value of each of one or more important colors, in this embodiment the spectral reflectance is calculated for each pixel value of the entire target image.

[0063] If the target image contains multiple identical pixel values, the spectral reflectance calculation unit 302 may output the spectral reflectance of the pixel value that has already been calculated to the image generation unit 303 without recalculating the identical pixel values.

[0064] The image generation unit 303 generates an image that is color-aligned with respect to the reference camera based on the spectral reflectance of each pixel value received from the spectral reflectance calculation unit 202, as well as the spectral sensitivity of the reference camera and the spectral distribution of the illumination 104 in the shooting environment, which are stored in the storage unit 200. First, the image generation unit 303 calculates predicted shooting values ​​(XYZ values) of the spectral reflectance of each pixel value of the entire target image according to equation (2). Next, the image generation unit 303 converts each calculated predicted shooting value (XYZ value) into an RGB value according to equation (4). As a result, color-aligned pixel values ​​(RGB values) are acquired for each pixel value of the entire target image, and a color-aligned target image is generated.

[0065] <Summary of the second embodiment> As described above, according to the second embodiment, the information processing device 102 acquires a first image (target image) captured by a first imaging device (target camera) under first illumination (illumination 104 of the imaging environment). The information processing device 102 also calculates the spectral reflectance of each color corresponding to each pixel in the first image based on the pixel value of each pixel in the first image, the spectral reflectance of the three primary colors measured in advance under second illumination (illumination of the previous environment), the spectral sensitivity of the second imaging device (reference camera), and the spectral distribution of the first illumination. The information processing device 102 then generates a second image corresponding to the first image, color-matched based on the second imaging device, based on the spectral reflectance of each color corresponding to each pixel in the first image, the spectral sensitivity of the second imaging device, and the spectral distribution of the first illumination.

[0066] Therefore, according to this embodiment, it is possible to reduce the workload of performing color matching between cameras for colors for which the spectral reflectance has not been acquired in advance. Furthermore, in the second embodiment, there is no need to prepare the spectral sensitivities of the target cameras in advance. Therefore, the workload of preparatory work is significantly reduced, especially when a large number of target cameras are used.

[0067] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0068] [summary] The above-described embodiment discloses at least the inventions shown in the following items, but is not limited to these inventions. (Item 1) an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a first calculation means for calculating a spectral reflectance of each of the one or more first colors corresponding to the one or more pixels in the first image based on pixel values ​​of each of the one or more pixels in the first image, the spectral reflectances of the three primary colors measured in advance under a second illumination, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; An information processing device comprising: (Item 2) A second calculation means, calculating first one or more pixel values ​​obtained by capturing the first one or more colors with the first image capture device under the first illumination based on the spectral reflectance of each of the first one or more colors, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; calculating second one or more pixel values ​​obtained by capturing the first one or more colors with the second image capture device under the first illumination, based on the spectral reflectance of each of the first one or more colors, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; A second calculation means; a generating means for generating parameters for performing color matching between images captured by the first image capturing device and the second image capturing device under the first illumination, based on the first one or more pixel values ​​and the second one or more pixel values; 2. The information processing device according to item 1, further comprising: (Item 3) the acquisition means acquires a second image captured by the second imaging device under the first illumination; the first calculation means calculates the spectral reflectance of each of the one or more second colors corresponding to the one or more pixels in the second image based on pixel values ​​of each of the one or more pixels in the second image, the spectral reflectance of the three primary colors measured in advance under the second illumination, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; The second calculation means calculating third one or more pixel values ​​obtained by capturing the second one or more colors with the first image capture device under the first illumination, based on the spectral reflectance of each of the second one or more colors, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; calculating fourth one or more pixel values ​​obtained by capturing the second one or more colors with the second image capture device under the first illumination based on the spectral reflectance of each of the second one or more colors, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; the generating means generates the parameters further based on the third one or more pixel values ​​and the fourth one or more pixel values. 3. The information processing device according to item 2, (Item 4) The second calculation means calculating one or more fifth pixel values ​​obtained by capturing the one or more third colors with the first image capture device under the first illumination, based on the spectral reflectance of each of the one or more third colors previously measured under the second illumination, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; calculating sixth one or more pixel values ​​obtained by capturing the third one or more colors with the second image capture device under the first illumination based on the spectral reflectance of each of the third one or more colors, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; the generating means generates the parameters further based on the fifth one or more pixel values ​​and the sixth one or more pixel values. 4. The information processing device according to item 2 or 3. (Item 5) further comprising a selection means for selecting the one or more pixels in the first image; 5. The information processing device according to any one of items 1 to 4. (Item 6) the selecting means selects the one or more pixels in the first image based on a frequency of occurrence of pixel values ​​in the first image. 6. The information processing device according to item 5, (Item 7) an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a calculation means for calculating the spectral reflectance of each color corresponding to each pixel in the first image based on the pixel value of each pixel in the first image, the spectral reflectance of the three primary colors measured in advance under a second illumination, the spectral sensitivity of a second image capture device, and the spectral distribution of the first illumination; a generation means for generating a second image corresponding to the first image, the second image being color-matched based on the second image capture device, based on the spectral reflectance of each color corresponding to each pixel in the first image, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; An information processing device comprising: (Item 8) An information processing method executed by an information processing device, an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a first calculation means for calculating a spectral reflectance of each of the one or more first colors corresponding to the one or more pixels in the first image based on pixel values ​​of each of the one or more pixels in the first image, the spectral reflectances of the three primary colors measured in advance under a second illumination, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; An information processing method comprising: (Item 9) An information processing method executed by an information processing device, an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a calculation means for calculating the spectral reflectance of each color corresponding to each pixel in the first image based on the pixel value of each pixel in the first image, the spectral reflectance of the three primary colors measured in advance under a second illumination, the spectral sensitivity of a second image capture device, and the spectral distribution of the first illumination; a generation means for generating a second image corresponding to the first image, the second image being color-matched based on the second image capture device, based on the spectral reflectance of each color corresponding to each pixel in the first image, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; An information processing method comprising: (Item 10) A program for causing a computer to function as each of the means of the information processing device described in any one of items 1 to 7.

[0069] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0070] 100...imaging device, 101...imaging device, 102...information processing device, 103...display device, 104...lighting, 200...storage unit, 201...pixel value extraction unit, 202...spectral reflectance calculation unit, 203...RGB calculation unit, 204...parameter generation unit

Claims

1. an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a first calculation means for calculating a spectral reflectance of each of the one or more first colors corresponding to the one or more pixels in the first image based on pixel values ​​of each of the one or more pixels in the first image, the spectral reflectances of the three primary colors measured in advance under a second illumination, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; An information processing device comprising:

2. A second calculation means, calculating first one or more pixel values ​​obtained by capturing the first one or more colors with the first image capture device under the first illumination, based on the spectral reflectance of each of the first one or more colors, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; calculating second one or more pixel values ​​obtained by capturing the first one or more colors with the second image capture device under the first illumination, based on the spectral reflectance of each of the first one or more colors, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; a second calculation means; a generating means for generating parameters for performing color matching between images captured by the first image capturing device and the second image capturing device under the first illumination, based on the first one or more pixel values ​​and the second one or more pixel values; 2. The information processing apparatus according to claim 1, further comprising:

3. the acquisition means acquires a second image captured by the second image capture device under the first illumination; the first calculation means calculates the spectral reflectance of each of the one or more second colors corresponding to the one or more pixels in the second image based on pixel values ​​of each of the one or more pixels in the second image, the spectral reflectance of the three primary colors measured in advance under the second illumination, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; The second calculation means calculating third one or more pixel values ​​obtained by capturing the second one or more colors with the first image capture device under the first illumination, based on the spectral reflectance of each of the second one or more colors, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; calculating fourth one or more pixel values ​​obtained by capturing the second one or more colors with the second image capture device under the first illumination based on the spectral reflectance of each of the second one or more colors, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; the generating means generates the parameters further based on the third one or more pixel values ​​and the fourth one or more pixel values.

3. The information processing apparatus according to claim 2, wherein:

4. The second calculation means calculating one or more fifth pixel values ​​obtained by capturing an image of the one or more third colors with the first image capture device under the first illumination, based on the spectral reflectance of each of the one or more third colors previously measured under the second illumination, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; calculating sixth one or more pixel values ​​obtained by capturing the third one or more colors with the second image capture device under the first illumination based on the spectral reflectance of each of the third one or more colors, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; the generating means generates the parameters further based on the fifth one or more pixel values ​​and the sixth one or more pixel values.

3. The information processing apparatus according to claim 2, wherein:

5. further comprising a selection means for selecting the one or more pixels in the first image; 2. The information processing apparatus according to claim 1, wherein:

6. the selecting means selects the one or more pixels in the first image based on a frequency of occurrence of pixel values ​​in the first image.

6. The information processing apparatus according to claim 5,

7. an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a calculation means for calculating the spectral reflectance of each color corresponding to each pixel in the first image based on the pixel value of each pixel in the first image, the spectral reflectance of the three primary colors measured in advance under a second illumination, the spectral sensitivity of a second image capture device, and the spectral distribution of the first illumination; a generation means for generating a second image corresponding to the first image, the second image being color-matched based on the second image capture device, based on the spectral reflectance of each color corresponding to each pixel in the first image, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; An information processing device comprising:

8. An information processing method executed by an information processing device, an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a first calculation means for calculating a spectral reflectance of each of the one or more first colors corresponding to the one or more pixels in the first image based on pixel values ​​of each of the one or more pixels in the first image, the spectral reflectances of the three primary colors measured in advance under a second illumination, the spectral sensitivity of the first image capture device, and the spectral distribution of the first illumination; An information processing method comprising:

9. An information processing method executed by an information processing device, an acquisition means for acquiring a first image captured by a first imaging device under a first illumination; a calculation means for calculating the spectral reflectance of each color corresponding to each pixel in the first image based on the pixel value of each pixel in the first image, the spectral reflectance of the three primary colors measured in advance under a second illumination, the spectral sensitivity of a second image capture device, and the spectral distribution of the first illumination; a generation means for generating a second image corresponding to the first image, the second image being color-matched based on the second image capture device, based on the spectral reflectance of each color corresponding to each pixel in the first image, the spectral sensitivity of the second image capture device, and the spectral distribution of the first illumination; An information processing method comprising:

10. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 7.

Images