Image processing device, control method, program, and color chart

Abstract

To generate a color chart to estimate the camera's spectral characteristics with high accuracy.SOLUTION: An image processing device includes control means for generating design data for a color chart including a plurality of color patches selected such that an evaluation value E of the color chart calculated on the basis of the spectral reflectance of the plurality of color patches according to the formula (I) is smaller. (Rm, Rn: spectral reflectance of color patches with index numbers m and n, N: number of color patches in the color chart, Σ: sum for all combinations of m and n).SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus, a control method, a program, and a color chart. [Background technology]

[0002] In recent years, various camera manufacturers have released various cameras, but because the hardware specifications and development processes of each company are not standardized, images taken of the same subject may not have the same RGB signals. For example, the RGB signals obtained by capturing an image with a camera depend on the optical system such as the photographic lens, the gamma function of the camera, the spectral transmittance of the color filter, the spectral sensitivity characteristics of the sensor, the filter arrangement, the white balance, etc. Therefore, the RGB signals obtained by capturing an image will be different even if the same subject is captured.

[0003] In recent years, therefore, techniques have been proposed for correcting RGB signals so that the same subject photographed with different types of cameras can be output with the same RGB signals (for example, Patent Documents 1 and 2). These correction techniques are realized by photographing a color chart such as a Macbeth chart for color calibration, and generating a three-dimensional lookup table (3DLUT) that associates the photographed pixel values ​​of each camera with each other.

[0004] When creating a 3DLUT to match colors between different types of cameras using the above technology, it is necessary to photograph a color target under a lighting environment equivalent to that of the actual shooting site. Patent Document 3 proposes a technology to generate a 3DLUT without photographing a color target at the shooting site by estimating the spectral characteristics of the camera from an image of a color target photographed in advance in a different environment. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 4136820 [Patent Document 2] Patent No. 5097927 [Patent Document 3] Patent No. 4182023 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology disclosed in Patent Document 3 has a large difference in the accuracy of prediction of the camera's spectral characteristics depending on the color chart used for shooting. Experiments by the inventors have shown that it is difficult to estimate the camera's spectral characteristics with high accuracy using a commercially available color chart.

[0007] Therefore, an object of the present invention is to generate a color chart for estimating the spectral characteristics of a camera with high accuracy. [Means for solving the problem]

[0008] In order to solve this problem, for example, the present invention has the following configuration: That is, the image processing device is characterized by comprising a control means for generating design data for a color chart including a plurality of color patches selected so that an evaluation value E of the color chart calculated based on the spectral reflectance of the plurality of color patches according to formula (I) becomes smaller.

[0009]

number

[0010] According to the present invention, it is possible to generate a color chart for estimating the spectral characteristics of a camera with high accuracy. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a functional block diagram of a system including an image processing apparatus according to a first embodiment. [Diagram 2] 2 shows an example of a hardware configuration for realizing the image processing apparatus according to the first embodiment. [Diagram 3] 5 is a flowchart showing the flow of processing by the image processing device according to the first embodiment. [Figure 4] 11 is a flowchart showing details of the chart evaluation value calculation process in S502. [Diagram 5] FIG. 13 is a diagram showing the system configuration of a second embodiment. [Figure 6] 11 is a flowchart showing details of the design data creation process in S204. [Figure 7] A diagram showing the overall appearance of the generated color chart. [Figure 8] FIG. 11 is a diagram showing an example of the spectral characteristics of color patch materials that make up a generated color chart. [Figure 9] 11 is a flowchart showing an outline of the operation of an image processing device according to a third embodiment. [Figure 10] FIG. 11 is a view showing a UI for setting conditions for a color chart to be generated by the image processing apparatus according to the third embodiment. [Figure 11] 13 is a diagram showing the relationship between the vertical and horizontal directions of a color patch material and the XY coordinates in the third embodiment. [Figure 12] FIG. 13 is a diagram showing an error display in the third embodiment. [Figure 13] FIG. 13 is a diagram illustrating a color patch material list after a color patch material has been added in the third embodiment. [Figure 14] 11 is a table showing examples of evaluations of a color chart according to an embodiment and a commercially available color chart using an evaluation value E. [Figure 15] 11 is a table showing the peak spectral wavelengths of color patch materials obtained in a color chart using the present invention, which was generated through experiments by the inventors. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe multiple features, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated explanations are omitted. Also, each process (step) in the flow chart is indicated by a reference number beginning with S.

[0013] [Embodiment 1] First, the first embodiment will be described. In this embodiment, a color chart (hereinafter referred to as a color patch material or a color patch) using color materials that can be candidates for constituting a color chart is input to an image processing device disclosed in this embodiment. The image processing device performs a spectroscopic measurement of the input color patch material. The image processing device stores the spectroscopic measurement data of the measurement result together with the material name of the input color patch material. Then, based on the stored spectroscopic measurement data, the image processing device evaluates an optimal combination of color patch materials for generating a color chart for estimating camera spectral characteristics (hereinafter referred to as a color chart), and outputs the evaluation result as design data. Finally, based on the output design data, the image processing device allocates color chart materials with reference to the material names, and generates a color chart. Hereinafter, the details of this embodiment will be described with reference to the drawings. Note that this embodiment will be described assuming that a color chart having 36 color patch materials is generated.

[0014] (Overall composition) FIG. 1 is a functional block diagram of a system including an image processing apparatus according to the first embodiment.

[0015] The image processing device 100 is the main function of this embodiment, and has a role of creating design data based on spectroscopic measurement data of a color patch material.

[0016] The input unit 101 receives input of color patch materials that are used to create the color chart disclosed in this embodiment. In this embodiment, for example, a color chart cut into a 2 cm square and coated or printed with color materials is used as the color patch materials, which are stacked vertically and placed in a predetermined position in the input unit 101 of this system. The input unit 101 removes the vertically stacked color patch materials from the top by vacuum suction or the like, and sends them to the spectral data measurement unit 102. The input unit 101 assigns index numbers to the input color patch materials in the order they were removed, together with the names of the color patch materials. The input unit 101 sends the index numbers to the spectral data measurement unit 102.

[0017] The spectral data measurement unit 102 measures the spectral reflectance of the color patch material sent from the input unit 101. In this embodiment, the spectral data measurement unit 102 has, for example, a spectrometer and a setting stand, and sets the color patch material sent from the input unit 101 on the setting stand. Then, the spectral data measurement unit 102 measures the spectral reflectance of the color patch material set on the setting stand using the spectrometer. The spectral data measurement unit 102 sends to the spectral data storage unit 103 the patch data in which the spectrometer measurement data, which is data on the measured spectral reflectance, is linked to the index number sent from the input unit 101. In addition, the spectral data measurement unit 102 stores the color patch material after the measurement. For example, the spectral data measurement unit 102 stores the color patch materials after the measurement by stacking them vertically together.

[0018] The spectral data storage unit 103 stores the patch data generated by the spectral data measurement unit 102. In addition, the spectral data storage unit 103 transfers the spectroscopic measurement data to the spectral data evaluation unit 104 together with an index number in response to a request from the spectral data evaluation unit 104.

[0019] The spectral data evaluation unit 104 extracts one candidate piece of spectroscopic measurement data from the spectroscopic measurement data in the patch data held in the spectral data holding unit 103. Then, the spectral data evaluation unit 104 refers to the spectroscopic measurement data constituting the provisional design data of the color chart held in the design data holding unit 105, and evaluates whether the spectroscopic measurement data is suitable for the color chart. Note that the provisional design data is data of a color chart in which the number of color patch materials (the number of color patches) does not reach a predetermined target number of charts in the design data, and can be said to be provisional design data in the middle of generating the design data. Details of the evaluation process will be described later. If the spectral data evaluation unit 104 determines that the spectroscopic measurement data of the color patch material is suitable for the color chart as a result of the evaluation, it sends the spectroscopic measurement data of the color patch material to the design data holding unit 105.

[0020] The design data holding unit 105 acquires and holds the spectroscopic measurement data evaluated by the spectroscopic data evaluation unit 104 as spectroscopic measurement data of a color patch material suitable for the color chart as design data or provisional design data for the color chart. At this time, the design data holding unit 105 acquires the name of the color patch material linked to the spectroscopic measurement data from the spectroscopic data holding unit 103 based on the index number of the spectroscopic measurement data.

[0021] The chart generation unit 106 acquires the color patch material held in the spectral data measurement unit 102 based on the design data of the color chart held in the design data holding unit 105, and generates and outputs a color chart. In this embodiment, for example, the chart generation unit 106 generates a color chart using a machine tool 1014 described later. The chart generation unit 106 refers to the index number of the color patch material based on the design data, and extracts the color patch material corresponding to the relevant index number from the spectral data measurement unit 102. Then, the chart generation unit 106 pastes the color patch material on the mount of the color chart. The chart generation unit 106 performs this operation the number of times equal to the number of color patch materials to generate a color chart.

[0022] (Hardware configuration for realizing this embodiment) FIG. 2 shows an example of a hardware configuration for realizing the image processing device 100. As shown in FIG.

[0023] The image processing device 100 includes a CPU 1001, a RAM 1002, a ROM 1003, an auxiliary storage I / F (interface) 1004, a HDD 1005, an input I / F (interface) 1006, an output I / F (interface) 1007, and a network I / F (interface) 1012. The components of the image processing device 100 are connected to each other by a system bus 1008. The image processing device 100 is also connected to an external storage device 1009 and an input device 1011 via the input I / F 1006. The image processing device 100 is also connected to a machine tool 1014 and a monitor 1010 via the output I / F 1007. The image processing device 100 is also connected to a transmission / reception device 1013 via the network I / F 1012.

[0024] The CPU (Central Processing Unit) 1001 is a processor and an example of a control means. The CPU 1001 executes a program stored in the ROM 1003 or the HDD 1005 using the RAM 1002 as a work memory, and controls each component of the image processing device 100 via a system bus 1008. The RAM (Random Access Memory) 1002 is an example of a volatile storage device. The RAM 1002 has an area for storing computer programs and data loaded from the ROM 1003 or the HDD 1005. The RAM 1002 also has an area for storing data received from the transmitting / receiving device 1013 via a network I / F 1012. The ROM (Read Only Memory) 1003 stores setting data of the image processing device 100, computer programs and data related to the startup of the image processing device 100, computer programs and data related to the basic operation of the image processing device 100, and the like. The HDD (Hard Disk Drive) 1005 stores an OS (Operating System), computer programs and data for making the CPU 1001 execute or control various processes described as processes performed by the image processing device 100. The CPU 1001, for example, loads a program stored in the ROM 1003 or the HDD 1005 into the RAM 1002 to realize the functions of the spectral data storage unit 103, the spectral data evaluation unit 104, and the design data storage unit 105. In addition to the CPU 1001 and the RAM 1002, a GPU (Graphics Processing Unit) and a VRAM (Video Random Access Memory) may be provided. The HDD 1005 stores various data and programs handled by the image processing device 100. The CPU 1001 writes data to the HDD 1005 and reads data and programs stored in the HDD 1005 via a system bus 1008. In addition to or instead of the HDD 1005, a non-volatile storage device such as an optical disk drive or a flash memory may be used.

[0025] The input I / F 1006 is, for example, a serial bus interface such as USB or IEEE1394. The image processing device 100 inputs data, commands, and the like from an external device via the input I / F 1006. In this embodiment, the image processing device 100 acquires data from an external storage device 1009 (for example, a storage medium such as a hard disk, a memory card, a CF card, an SD card, or a USB memory) via the input I / F 1006. In this embodiment, the image processing device 100 acquires a user's instruction input to an input device 1011 via the input I / F 1006. The input device 1011 is an input device such as a mouse or a keyboard, and inputs a user's instruction. In addition, data obtained when the spectral data measurement unit 102 performs spectral measurement of a spectral color patch material is input to the image processing device 100 via the input I / F 1006.

[0026] The output I / F 1007 is a serial bus interface such as USB or IEEE1394, similar to the input I / F 1006. The output I / F 1007 may be a video output terminal such as DVI or HDMI (registered trademark). In this embodiment, the image processing device 100 outputs data processed by the CPU 1001 to a monitor 1010 (various image display devices such as a liquid crystal display) via the output I / F 1007. In addition, a spectrometer for transferring a color patch material from the input unit 101 to the spectroscopic data measurement unit 102 and for performing spectroscopic measurement, and a machine tool 1014 for creating a color chart based on design data are controlled via the output I / F 1007.

[0027] The network I / F 1012 receives information sent from an external device via a transmitter / receiver device 1013, such as a connector for connecting to a network such as Ethernet (registered trademark), and stores the information in a RAM 1002 or the like within the image processing device 100 via a system bus 1008.

[0028] The image processing device 100 has other components in addition to those described above, but they are not the main focus of this embodiment and will not be described here. The image processing device 100 can be configured by a device such as a personal computer or a workstation. In this case, each process performed by the spectral data storage unit 103, the spectral data evaluation unit 104, and the design data storage unit 105 of the image processing device 100 is realized as a function of a program executed inside the image processing device 100.

[0029] In this embodiment, the image processing device 100 having the above configuration measures the color patch material using a spectrometer or the like connected to the output I / F 1007 based on a command from the CPU 1001, and acquires spectroscopic measurement data. In addition, the image processing device 100 designs and creates a color chart using a machine tool 1014 or the like connected to the output I / F 1007 based on the spectroscopic measurement data.

[0030] (Overall processing flow of image processing device) Next, the flow of processing in the system according to this embodiment will be described with reference to Fig. 3, which is a flowchart showing the flow of processing in the image processing device 100 according to this embodiment.

[0031] In S201, the input unit 101 removes the vertically stacked color patch materials from the top by vacuum suction or the like, as described above, and sends them to the spectral data measurement unit 102. At the same time, the input unit 101 assigns an index number to the input color patch material and sends the index number to the spectral data measurement unit 102.

[0032] In S202, the spectral data measurement unit 102 generates patch data based on the transmitted color patch material. The patch data includes the index number of the color patch material, the name of the color patch material, and spectral measurement data indicating the spectral reflectance of the color patch material. The spectral measurement data of the color patch material does not yet have a value at this point, and is stored in S203. In addition, the spectral data measurement unit 102 also initializes the design data. The initialized design data may be, for example, null data that does not include data on the color patch material.

[0033] In S203, the spectral data measurement unit 102 measures the spectral reflectance of the color patch material, and stores the spectral measurement data obtained by the measurement in the patch data. The patch data in which the spectral measurement data is stored together with the index number and the color patch material name is sent to the spectral data storage unit 103. The spectral data storage unit 103 stores the received patch data.

[0034] In S204, the spectral data evaluation unit 104 evaluates the spectroscopic measurement data of the patch data stored in the spectral data storage unit 103, and if it determines that the spectroscopic measurement data is suitable for the color chart, it sends the spectroscopic measurement data and the index number to the design data storage unit 105. The design data storage unit 105 creates and stores design data that combines the color patch material name acquired from the spectral data storage unit 103 based on the index number and the spectroscopic measurement data received from the spectral data evaluation unit 104.

[0035] The design data is data that is a design drawing of the color chart to be generated, and stores the number of patches of the color chart, the patch size, the layout data of the color chart material in the patch, and the patch data of each color patch material. When this process starts, blank data is stored as the initial parameters. When the process of S204 is completed, the actual data is stored in the design data.

[0036] In S205, the chart generation unit 106 creates a color chart based on the design data of the color chart held in the design data holding unit 105. At this time, the color patch material used for creating the color chart is the color patch material held in the spectral data measurement unit 102. Details of the color chart will be described later.

[0037] <Details of the design data creation process in S204> Details of the design data creation process in S204 will be described using the flowchart in FIG. 6. In this process, first, the spectral data evaluation unit 104 sets one patch data selected from all the patch data that are candidates for the input color chart as the initial patch data. Then, the spectral data evaluation unit 104 generates provisional design data for the color chart based on the initial patch data. In other words, the spectral data evaluation unit 104 generates provisional design data having only the data of one color patch material.

[0038] Furthermore, the spectral data evaluation unit 104 updates the provisional design data by adding patch data different from the initial patch data selected from all the patch data. The spectral data evaluation unit 104 calculates the evaluation value of the generated provisional design data. If it evaluates that the added color patch material is suitable for the color chart, the provisional design data with the added color patch material is set as the new provisional design data. If it determines that the added color patch material is not suitable for the color chart, the color patch material is discarded from the provisional design data and the provisional design data is not updated. The spectral data evaluation unit 104 repeats this process. When the number of color patch materials included in the provisional design data reaches the target chart number, the spectral data evaluation unit 104 sends the provisional design data to the design data holding unit 105 as the design data. The design data holding unit 105 holds the said design data. Details will be described below.

[0039] In S501, the spectral data evaluation unit 104 selects and sets the initial patch data of the provisional design data for calculating the evaluation value E.

[0040] In S502, the spectral data evaluation unit 104 creates provisional design data based on the initial patch data set in S501 and calculates the evaluation value E. Details will be described later.

[0041] In S503, the spectral data evaluation unit 104 creates provisional design data for all possible patterns for the initial patch data and determines whether the evaluation value E of the color chart has been calculated. If the spectral data evaluation unit 104 has calculated the evaluation value E for the provisional design data of all the initial patch data, it proceeds to S505; otherwise, it proceeds to S504. As a result, the spectral data evaluation unit 104 generates provisional design data including color patch materials of the target number of charts for a plurality of initial patch data. Therefore, when proceeding to S505, a plurality of provisional design data including color patch materials of the target number of charts are generated.

[0042] In S504, the spectral data evaluation unit 104 changes the setting of the initial patch data and acquires the changed patch data from the spectral data holding unit 103. The spectral data evaluation unit 104 changes the initial patch data, for example, to the patch data of a color patch material for which provisional design data has not yet been created.

[0043] In S505, the spectral data evaluation unit 104 selects the provisional design data of the color chart with the smallest evaluation value E from among the plurality of provisional design data including color patch materials of the target number of charts, and outputs the selected provisional design data to the design data holding unit 105 as design data.

[0044] <Details of the color chart evaluation value calculation process in S502> Details of the calculation process of the evaluation value E of the color chart in S502 will be described using the flowchart of FIG. 4.

[0045] In S301, the spectral data evaluation unit 104 performs an initialization operation prior to evaluation. Specifically, the current total selection number N of the provisional design data is set to 1. In addition, the spectral data evaluation unit 104 incorporates the initial patch data selected in advance into the provisional design data, and stores it in the design data storage unit 105.

[0046] In S302, the spectral data evaluation unit 104 initializes the evaluation value E, setting it to E = 1.0. The evaluation value E will be described later.

[0047] In S303, the spectral data evaluation unit 104 acquires one piece of patch data from the multiple pieces of patch data stored in the spectral data storage unit 103. At this time, the spectral data evaluation unit 104 selects and acquires patch data other than the initial patch data and the previously selected patch data.

[0048] In S304, the spectral data evaluation unit 104 extracts spectroscopic measurement data from each of the acquired patch data. Then, the spectral data evaluation unit 104 calculates the evaluation value E' between the color patch materials by calculating the following formula (1).

[0049]

number

[0050] Here, R is the spectroscopic measurement data that is the spectral reflectance of each color patch material, and m and n are the index numbers of the patch data incorporated in the provisional design data of the design data holding unit 105 and the index numbers of the patch data acquired in S303. That is, Rm and Rn are either the spectroscopic measurement data included in the patch data incorporated in the provisional design data of the design data holding unit 105 or the spectroscopic measurement data included in the patch data acquired in S303. The spectroscopic measurement data Rm and Rn that are the spectral reflectance are three-dimensional vectors in which each element in the RGB or Lab color space has a positive value, for example. Therefore, from the definition of the inner product, the evaluation value E' is cosθ when the angle between the vector Rm and the vector Rn is θ. A small evaluation value E' indicates that the angle θ between the vector Rm and the vector Rn is close to 90°, and indicates that the directions of both vectors are significantly different. When the data of the spectral reflectance at the time of measurement is vector data in the RGB color space, it is preferable to convert R of the spectroscopic measurement data to vector data in the Lab color space. For example, if the number of color patch materials included in the provisional design data is two, the index number of the initial patch data is 1, and the index number of the patch data of the set provisional design data is 2, then m=1 and n=2.

[0051] In S305, the spectral data evaluation unit 104 determines whether the calculation in S304 has been performed for all patch data candidates. If the spectral data evaluation unit 104 has performed the calculation, the process proceeds to S307. If not, the process proceeds to S306.

[0052] In S306, when calculating the evaluation value E' in S304, the spectral data evaluation unit 104 changes the index number of the patch data of the provisional design data corresponding to n, thereby changing the color chart candidate colors.

[0053] In S307, the spectral data evaluation unit 104 calculates the sum of all the combinations of the evaluation values ​​E′ calculated in S303 to S306 for m and n as the total number of combinations. N C 2This is divided by and determined as the evaluation value E. Specifically, this is expressed by the following formula (2). Therefore, the evaluation value E can also be said to be the arithmetic mean of the evaluation values ​​E'. As described above, a small evaluation value E' means that the directions of the Rm vectors Rm and Rn are significantly different from each other, so a small evaluation value E means that the directions of all the vectors R included in the color chart vary. In other words, a small evaluation value E means that the colors of the color patch materials of the color chart are not similar to each other.

[0054]

number

[0055] Here, the numerator of formula (2) is the sum of the evaluation values ​​E' of all combinations of m and n calculated in S303 to S306. N is the number of color patch materials in the current color chart listed in the provisional design data, and if two color patch materials are selected in the current provisional design data, N will be 3, including the color patch materials in the initial patch data. When the design data for the color chart is finally completed, N will be 36, which is the target number of charts.

[0056] In S308, the spectral data evaluation unit 104 determines whether to update the provisional design data based on the evaluation value E. Specifically, the spectral data evaluation unit 104 compares the evaluation value E newly calculated in step S307 with the currently held evaluation value E. Here, the currently held evaluation value E is the evaluation value E of the provisional design data (an example of the first provisional design data) before the newly added color patch material is added. The newly calculated evaluation value E is the evaluation value E of the provisional design data (an example of the second provisional design data) including the newly added color patch material. If the evaluation value E calculated in S307 is smaller than the currently held evaluation value E, the spectral data evaluation unit 104 proceeds to S309, otherwise to S311. In other words, if the evaluation value E is not reduced by the newly added color patch material, the spectral data evaluation unit 104 discards the added color patch material and does not update the provisional design data. As a result, the spectral data evaluation unit 104 updates the provisional design data so that the evaluation value E becomes smaller every time a color patch material is added.

[0057] In S309, the spectroscopic data evaluation unit 104 increments N by 1, and the process proceeds to S310.

[0058] In S310, the spectral data evaluation unit 104 changes the currently held evaluation value E to the evaluation value E confirmed in S307. At the same time, the spectral data evaluation unit 104 updates the provisional design data of the color chart corresponding to the confirmed evaluation value E. As a result, the spectral data evaluation unit 104 adds a color patch material to the provisional design data when the evaluation value E becomes smaller, and therefore the evaluation value E of the provisional design data becomes smaller with the addition of the color patch material.

[0059] In S311, the spectral data evaluation unit 104 determines whether N is less than a predetermined target number of charts (36 in this embodiment). If N is less than the target number of charts, the process proceeds to S302; if not, the process ends.

[0060] (Details of the generated color chart) Next, a camera spectral characteristic color chart generated by the image processing device described in this embodiment will be described in detail.

[0061] If the camera spectral characteristics are defined in 10 nm increments for the visible wavelengths of 380 nm to 730 nm, it is preferable that the color chart has at least 36 color patch materials in order to estimate the camera's spectral characteristics. In addition, in order to reduce the above-mentioned evaluation value E, it is assumed that the spectral correlation between the color patch materials is low, that is, the spectral orthogonality is high. Furthermore, it is highly likely that the color patch materials selected have multiple patches each having peaks in the spectral wavelength bands of 440 nm to 470 nm, 500 nm to 540 nm, and 600 nm or more, which are the R, G, and B spectral sensitivity characteristics of a general camera.

[0062] Fig. 7 is a diagram showing the overall appearance of a color chart generated by the inventors' experiments. Fig. 8 is a diagram showing an example of the spectral characteristics of the color patch materials constituting the generated color chart. Fig. 15 is a table showing the peak spectral wavelengths of the color patch materials obtained in the color chart generated by the inventors' experiments using the present invention. The evaluation value E of the created color chart was 0.527.

[0063] The chart base 501 is, for example, a gray color base.

[0064] The color patch material 502 is pasted with a color patch material adopted for this color chart from the input color patch material group. The color patch materials 502 each have a specified spectral reflectance. As shown in Fig. 8, the spectral reflectance of a certain color patch material 502 has a peak wavelength at 520 nm, as shown in graph 601(a). Another color patch material shown in graph 601(b) has a peak wavelength at 460 nm. Moreover, the red (R) color patch material shown in graph 601(c) has a peak wavelength at 650 nm.

[0065] Moreover, the color chart generated in this embodiment has a plurality of color patch materials 502 having a reflectance peak (=peak spectral wavelength band) at a spectral wavelength of 600 nm or more, as shown in Fig. 15. Similarly, the color chart has a color patch material 502 having a reflectance peak at a spectral wavelength of 500 nm to 540 nm, and a plurality of color patch materials 502 having a reflectance peak at a spectral wavelength of 440 nm to 470 nm.

[0066] As shown in FIG. 8 and FIG. 15, the waveform of the spectral reflectance of each patch has a clear peak compared to a general color patch material, and the spectral reflectance other than the peak tends to be low, making it easier to select a color chart. In contrast, when trying to create a color chart containing all hues using general reflective colorants, the spectral reflectance includes not only color patch materials with peaks as described above, but also color patch materials whose spectral reflectance characteristics do not have a clear peak or whose shape is close to a sigmoid function. In such general color charts, the orthogonality between the color patch materials is low, so the evaluation value E of the color chart tends to be large. According to experiments by the inventors, it has been found that when the evaluation value E of the color chart is a specific value or more, the estimation error of the camera characteristics becomes large. The specific value is, for example, 0.7.

[0067] FIG. 14 is a table showing an example of evaluation of the color chart of the embodiment and the color chart of the commercially available comparative example using the evaluation value E. The result of Comparative Example 1 in FIG. 14 is 0.816, and the result of Comparative Example 2 is 0.862. From this result, it can be seen that the evaluation value E does not fall below 0.7 by simply using a commercially available color chart as is. In contrast, when the color chart proposed in this embodiment is evaluated using the evaluation value E, it falls below 0.7. From this, it can be seen that the color chart of the embodiment proposed in this embodiment has a different tendency compared to a general color chart. The color chart of the embodiment created in this proposal cannot be realized by simply using a commercially available color chart as is, and it is necessary to select and combine color patch materials with different spectral characteristics from the color patch materials that make up the commercially available color chart. By selecting and combining color patch materials in this way, it is possible to reduce the evaluation value E to 0.7 or less.

[0068] As described above, according to this embodiment, design data for a color chart including a plurality of color patch materials is generated using color patch materials selected so that the evaluation value E is smaller based on the evaluation value E calculated from the spectral reflectance of the color patch materials. As a result, this embodiment can generate a color chart that is small in evaluation value E, for example, 0.7 or less, by varying the color patch materials in the color space and that can estimate the camera's spectral characteristics with high accuracy. As a result, this embodiment can obtain a color chart that can estimate camera spectral characteristics data with high accuracy, which is necessary when generating 3DLUTs corresponding to various subjects for matching colors between different cameras without photographing a color chart at the shooting site.

[0069] In this embodiment, provisional design data is generated by adding one color patch material to provisional design data that has already been generated, and when the evaluation value E becomes smaller than the evaluation value E before the color patch material was added, the provisional design data to which the color patch material has been added is updated as new provisional design data. As a result, in this embodiment, the amount of calculation can be reduced compared to the case where design data for a target number of charts is generated and then the evaluation values ​​E are compared, and design data for a color chart can be generated efficiently.

[0070] In this embodiment, design data is generated by updating the provisional design data by adding color patch materials until the target number of charts is reached while decreasing the evaluation value E. In this way, this embodiment can efficiently generate design data with a desired number of charts with a small evaluation value E.

[0071] In this embodiment, while changing the initial patch data, multiple pieces of provisional design data for the target number of charts are generated, and the multiple pieces of provisional design data having the smallest evaluation value E are used as design data for the color chart. In this way, this embodiment can generate design data having a smaller evaluation value E.

[0072] In the present embodiment, an example has been shown in which the color chart is created by a machine tool, but in addition to pasting color patch materials by a machine tool, a method of generating the color chart by printing using a printer or the like is also conceivable.

[0073] In this embodiment, an example of making a color chart using a reflective material is shown, but the color patch material is not limited to a reflective material, and a self-luminous material (such as an LED) may be used. In this case, a dedicated device for making the LED emit light and measuring it is required for spectroscopic measurement of the material. In addition, when outputting a color chart based on design data, a dedicated machine tool is required to wire the LED and assemble a circuit for supplying power.

[0074] [Embodiment 2] In the first embodiment, color patch materials that are candidates for constituting a color chart for predicting camera spectral characteristics are input to an image processing device and spectroscopically measured, and design data is generated from the obtained spectroscopic measurement data. In this embodiment, the process flow is the same, but the evaluation value E that is calculated when generating the design data is calculated on the cloud. Details of this are described below. Note that in this embodiment, only the parts that are different from the first embodiment are described.

[0075] FIG. 5 is a diagram showing a system configuration of the second embodiment.

[0076] The network 400 refers to a general network such as the Internet.

[0077] The main functions of the design data generation unit 401 include a spectral data evaluation unit 404 and a design data holding unit 405. The design data generation unit 401 uses a network 400 to input data necessary for calculations and to output the results of the calculations.

[0078] The spectral data storage unit 403 has the same main function as the spectral data storage unit 103. The spectral data storage unit 403 differs from the spectral data storage unit 103 in that the spectral measurement data of the stored patch data is input to a spectral data evaluation unit 404 via a network 400.

[0079] The main function of the spectral data evaluation unit 404 is the same as that of the spectral data evaluation unit 104. The spectral data evaluation unit 404 differs from the spectral data evaluation unit 104 in that the spectral measurement data of the patch data is acquired via the network 400.

[0080] The main function of the design data holding unit 405 is the same as that of the design data holding unit 105. The difference from the design data holding unit 105 is that the generated design data is sent to a chart generating unit 406 via a network 400.

[0081] The chart generating unit 406 receives design data via the network 400. The color patch material is input from the spectral data measuring unit 102. Based on these input data, a color chart for estimating the camera spectral characteristics is generated.

[0082] As described above, according to this embodiment, a color chart can be obtained that can more accurately predict camera spectral characteristic data, which is necessary when generating a 3DLUT that matches colors between cameras without photographing a color chart at the shooting site.

[0083] [Embodiment 3] In the first and second embodiments, an example was described in which a color chart having 36 color patch materials is generated based on input color patch materials. In this embodiment, a case is described in which a UI is provided for specifying the total number of color patch materials to be incorporated in the color chart, the color patch size, and the size of the color chart itself. As in the second embodiment, only the differences between this embodiment and the first and second embodiments will be described.

[0084] Fig. 10 is a diagram showing a UI (User Interface) for setting conditions for a color chart generated by an image processing device in embodiment 3. Fig. 11 is a diagram showing the relationship between the vertical and horizontal directions and the X and Y coordinates of a color patch material in embodiment 3. Fig. 12 is a diagram showing an error display in embodiment 3. Fig. 13 is a diagram showing a color patch material list after a color patch material has been added in embodiment 3.

[0085] The main window 800 contains buttons and dialogues required for setting the conditions of the color chart, a window displaying a drawing of the planned color chart upon completion, and the like.

[0086] Dialog 801 is a field for specifying the number of color patch materials in the horizontal direction. The user specifies how many color patch materials are to be arranged in the horizontal direction of the color chart by inputting a numerical value in dialog 801. In this case, the X and Y directions are X for the horizontal direction (rightward) and Y for the vertical direction (downward) with respect to the UI, as shown in Fig. 11. In the example of Fig. 10, 9 color patch materials are specified in the X direction and 6 in the Y direction.

[0087] The dialogue 802 is a field for specifying the number of color patch materials in the vertical direction. The user inputs a numerical value in the dialogue 802 to specify how many color patch materials are to be arranged in the vertical direction of the color chart.

[0088] At this time, if the user inputs values ​​in the dialogs 801 and 802 that are too large for the patch size of the color chart, an error window such as that shown in Fig. 12 is displayed, and the user is prompted to re-specify the parameters. This error window is displayed not only when the number of patch directions is specified, but also when the number of patches is specified, and the relationship between the patch size and the number of patches results in exceeding the chart size.

[0089] A dialog 803 is a color chart size designation section for designating the size of one color chart. The user selects a paper size such as A4 or B5 from the list in the dialog 803.

[0090] The color patch material list 804 serves as a list display section of color charts including color patch materials, and displays a list of color charts of the color patch materials input from the input section 101.

[0091] The additional color patch loading button 805 serves as a color patch input section, and pressing this button loads an additional color chart of the color patch material. The loaded color chart of the color patch material is registered as a new color chart at the bottom of the color patch material list 804 as shown in Fig. 13. The user then inputs a name for the color chart separately.

[0092] The planned completion diagram display window 806 becomes a design data display section, and displays how the color patch materials will be arranged on the actual color chart based on the specified chart size, patch size, horizontal number of patches X, and vertical number of patches Y. At this time, the color patch materials are displayed as being arranged at equal intervals on the color chart.

[0093] The evaluation value display window 807 is a window that displays the predicted evaluation value E of a color chart when a color chart for estimating camera spectral characteristics is created under the currently set conditions. For example, before a color chart is selected, "----" is displayed, but once a color chart is selected, an evaluation value such as "0.685" is displayed.

[0094] A dialog 808 is a color patch size designation section for designating the size of each color patch material. In Fig. 10, the size of one side of each color patch material is designated in mm.

[0095] A chart selection start button 809 is a button for starting a color chart selection process to determine what evaluation value a color chart will have if it is created under the currently set conditions.

[0096] A pointer 810 is used to select various dialog boxes and buttons, and in this embodiment, is operated by a pointing device such as a mouse connected to the input I / F 1006.

[0097] The chart creation start button 811 is a button for starting the creation of a color chart under the currently set conditions and with a performance evaluation value E. When the user presses the chart creation start button 811, the design data is transmitted to the chart generation unit 106, and the chart generation unit 106 creates a color chart. Note that the chart creation start button 811 cannot be pressed and is grayed out unless the chart selection start button 809 is pressed once to create an evaluation value for the color chart.

[0098] FIG. 9 is a flowchart showing an outline of the operation of the image processing device according to the third embodiment.

[0099] In S701, the input unit 101 inputs color patch materials that are candidates for forming a color chart.

[0100] In S702, the input unit 101 inputs the color patch material name input in S701.

[0101] In S703, the user inputs various settings required for creating a color chart, such as the chart size, the number of patches X, the number of patches Y, the patch size, etc. The order of S701 and S703 may be reversed. In this case, the steps are processed in the order of S703, S701, and S702.

[0102] In S704, when the user presses the chart selection start button 809, the spectral data evaluation unit 104 calculates an evaluation value E based on the currently set information to evaluate whether the spectral measurement data is suitable for the color chart, and the design data holding unit 105 generates provisional design data based on the evaluation result. The processing content at this time is the same as in the first and second embodiments.

[0103] In S705, the evaluation value E calculated in S704 is displayed in an evaluation value display window 807, and the selection result of the color patch material stored in the provisional design data is reflected on a completion plan drawing display window 806.

[0104] In S706, the user determines whether or not to generate the currently selected provisional design data as design data. If the user presses the chart creation start button 811 to input that design data is to be generated, the process proceeds to S707, and if the user wishes to review the settings again, the process proceeds to S701.

[0105] In S707, the design data holding unit 105 outputs the generated design data to the chart generating unit 106. The chart generating unit 106 generates and outputs a color chart proposal based on the input design data.

[0106] As described above, according to this embodiment, a color chart for estimating camera spectral characteristics can be generated in accordance with a user's specification.

[0107] [Other embodiments] When trying to estimate the spectral characteristics of a camera with a narrow color gamut such as Rec.709, depending on the color materials used in the chart, color saturation may occur when shooting with the camera. If there are only a few patches that do not cause color saturation (for example, 10 patches out of 36 patches cause color saturation in the captured image), the spectral characteristics of the camera may not be estimated correctly. To address this, a combination of patches that results in a smaller evaluation value E may be selected from candidate patches (color materials) within a specified color gamut such as Rec.709. For example, in the above case, the total number of patches in the chart is 46.

[0108] In the third embodiment, the user manually inputs the color patch material name when inputting the color patch material, but the appearance of the color patch material may be photographed and the input unit 101 or the spectral data measurement unit 102 may automatically input the color patch material name. In addition, the patch size may be set to a rectangle or other shape instead of a square on the color chart setting UI. Furthermore, after the patch size is specified, the number X of color patch materials and the number Y of color patch materials may be automatically calculated and set so that the largest number of color patch materials can be automatically arranged in the color chart.

[0109] Moreover, the third embodiment can be realized as a computer program that runs on the image processing device 100 disclosed in this embodiment.

[0110] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0111] The disclosure of this specification includes the following image processing device, control method, program, and color chart. (Item 1) An image processing device comprising a control means for generating design data for a color chart including a plurality of color patches selected so that an evaluation value E of the color chart calculated based on the spectral reflectance of the plurality of color patches based on formula (I) is smaller.

number

number

number

[0112] 100: image processing device; 102: spectral data measurement section; 103: spectral data storage section; 104: spectral data evaluation section; 105: design data storage section; 106: chart generation section; 502: color patch material

Claims

1. An acquisition means for acquiring the spectral reflectance of multiple color patches, The system includes a control means for generating design data for a color chart that includes a plurality of color patches selected such that the evaluation value E of the color chart, calculated from the spectral reflectances of the plurality of color patches based on equation (I), becomes smaller. The control means generates first provisional design data having the selected plurality of color patches, and if the evaluation value E of second provisional design data, which has color patches not included in the first provisional design data added, is smaller than the evaluation value E of the first provisional design data, the second provisional design data is set as new provisional design data. [Math 1] (Rm, Rn: spectral reflectances of color patches with index numbers m and n, N: number of color patches on the color chart, an integer greater than or equal to 3, Σ: sum for all combinations of m and n)

2. The image processing apparatus according to claim 1, characterized in that the plurality of color patches for the color chart are selected such that the evaluation value E is 0.7 or less.

3. The image processing apparatus according to claim 1, characterized in that the control means updates the provisional design data by adding color patches until the number of color patches in the provisional design data reaches a predetermined number.

4. The image processing apparatus according to claim 3, characterized in that the control means generates a plurality of provisional design data sets in which the number of color patches is a predetermined number, and the provisional design data set with the smallest evaluation value E is used as the design data.

5. The image processing apparatus according to claim 1, characterized in that the control means generates the design data including a plurality of color patches having the highest reflectance peak at a spectral wavelength of 600 nm or more.

6. The image processing apparatus according to claim 1, characterized in that the control means generates the design data including a plurality of color patch materials having the highest reflectance peak at a spectral wavelength of 500 nm to 540 nm.

7. The image processing apparatus according to claim 1, characterized in that the control means generates the design data including a plurality of color patch materials having the highest reflectance peak at a spectral wavelength of 440 nm to 470 nm.

8. The image processing apparatus according to claim 1, characterized in that the control means generates the design data based on the number of color patches to be placed on the color chart input by the user.

9. The image processing apparatus according to claim 8, characterized in that the control means generates the design data based on the size of the color chart input by the user.

10. The image processing apparatus according to claim 9, characterized in that the control means outputs an error if the size of the color chart cannot correspond to the number of selected color patches.

11. The image processing apparatus according to claim 1, characterized in that the control means displays a color chart including the selected plurality of color patches.

12. The image processing apparatus according to claim 1, characterized in that the control means displays a planned completion diagram of a color chart including the selected plurality of color patches.

13. If the evaluation value E of the second provisional design data is not smaller than the evaluation value E of the first provisional design data, the control means discards the added color patch and does not update the provisional design data. The image processing apparatus according to feature 1.

14. Obtain the spectral reflectance of multiple color patches, A design data for a color chart is generated that includes multiple color patches selected such that the evaluation value E of the color chart, calculated from the spectral reflectance of the multiple color patches based on equation (I), becomes smaller. A control method for an image processing apparatus, characterized in that it generates first provisional design data having the selected multiple color patches, and if the evaluation value E of second provisional design data obtained by adding color patches not included in the first provisional design data is smaller than the evaluation value E of the first provisional design data, the second provisional design data is made into new provisional design data. [Math 2] (Rm, Rn: spectral reflectances of color patches with index numbers m and n, N: number of color patches on the color chart, an integer greater than or equal to 3, Σ: sum for all combinations of m and n)

15. A program that, when read and executed by a computer, causes the computer to perform the method described in claim 14.

16. The system includes multiple color patches, and the evaluation value E calculated from the spectral reflectance of the multiple color patches based on formula (I) is 0.7 or less. A control means for generating design data for a color chart generates a first provisional design data having the selected plurality of color patches, and if the evaluation value E of the second provisional design data, which has color patches not included in the first provisional design data added, is smaller than the evaluation value E of the first provisional design data, the second provisional design data is set as the new provisional design data. [Math 3] (Rm, Rn: spectral reflectances of color patches with index numbers m and n, N: number of color patches on the color chart, an integer greater than or equal to 3, Σ: sum for all combinations of m and n)

Images