Color correction data creation device, color correction data creation device control method, and program

Abstract

To provide a color correction data creation device which can perform color matching independent of a change in an imaging position of a camera and the hue of a light source.SOLUTION: In a color correction data creation device 201, an image inference unit 204 infers an image of a chart region included in an image obtained when a corresponding imaging device images a chart illuminated by a reference light source from a reference imaging position on the basis of a cut-out image acquired from an image cut-out unit 202. An image inference unit 205 infers the image of the chart region included in the image obtained when the corresponding imaging device images the chart illuminated by the reference light source from the reference imaging position on the basis of the cut-out image acquired from the image cut-out unit 203. A color correction data creation unit 206 creates color correction data for performing correction such that the inferred image acquired from the image inference unit 205 has the same hue as the inferred image acquired from the image inference unit 204.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a color correction data creating device, a control method for a color correction data creating device, and a program. [Background technology]

[0002] When photographing the same subject with multiple cameras, color matching is performed before the actual photographing to match the color characteristics of the images captured by each camera. In color matching, a light source and a chart for color matching are prepared. Multiple cameras are arranged in front of the chart and each captures an image of the chart illuminated by the light source. A color correction lookup table (hereinafter referred to as "LUT") is created based on the captured images. However, during the actual photographing, the cameras and light sources may be located in different locations than those used for color matching, and the photographing may be performed using a light source different from that used for color matching. If the color and brightness of the light source illuminating the subject during color matching differ from those during the actual photographing, a discrepancy in the color characteristics of the images captured by each camera will occur even if a pre-created color correction LUT is used. To address this issue, a technology has been proposed for creating simulated images captured under different lighting conditions using a learning model trained by inputting captured images of the surrounding area and lighting conditions, including the position and intensity of the light source (see, for example, Patent Document 1). In addition, a technology has been proposed in which a sensor model is trained using white balance information at the time of shooting, and the sensor-captured image before image processing is inferred from the image captured by the camera to reproduce faithful colors (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2019-144899 [Patent Document 2] Japanese Patent Application Laid-Open No. 2000-311243 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology of Patent Document 1 uses a learning model in which the brightness of the light source is changed as the lighting condition. Therefore, it is not possible to estimate a pseudo-captured image when the color of the light source changes from the time of color matching. Furthermore, the technology of Patent Document 2 uses a learning model in which the camera sensor characteristics are inferred from images captured with a representative light source and the white balance information at that time. Therefore, it is not possible to estimate a pseudo-captured image that takes into account color changes caused by changing the camera's shooting position from the time of color matching. Thus, in the past, when the camera's shooting position or the color of the light source changes from the time of color matching, discrepancies occur in the color characteristics of the images captured by each camera.

[0005] An object of the present invention is to provide a color correction data creation device, a control method for a color correction data creation device, and a program that can perform color matching that is not dependent on changes in the shooting position of a camera or the color tone of a light source. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the color correction data creation device of the present invention is a color correction data creation device that acquires multiple images obtained by photographing a specified subject using multiple imaging devices, and is characterized by comprising: an output means that inputs images of a specified subject area included in the acquired images into a machine-learned learned model and outputs an inferred image obtained by inferring the image of the specified subject area included in an image obtained when the specified subject is photographed under specified shooting conditions; and a creation means that creates color correction data that matches the color characteristics of the multiple inferred images output by the output means. [Effects of the Invention]

[0007] According to the present invention, color matching can be performed that is not dependent on the photographing position of the camera or changes in the color tone of the light source. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of an imaging environment in which an unknown image is captured in a first embodiment of the present invention. [Figure 2] 1 is a block diagram showing a schematic configuration of a color correction data creating device according to an embodiment of the present invention; [Figure 3] FIG. 1 is a diagram illustrating an example of a shooting environment for learning images in the present embodiment. [Figure 4] FIG. 2 is a schematic diagram of learning of a learning model in the present embodiment. [Figure 5] FIG. 10 is a block diagram schematically illustrating a configuration of a color correction data creating device according to a second embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of an imaging environment in which an unknown image is captured in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings. First, a color correction data creation device and a control method thereof according to a first embodiment of the present invention will be described. The color correction data creation device according to the embodiment of the present invention is a device that creates color correction data that aligns the color characteristics of multiple images acquired from different image capture devices. In the first embodiment, as an example, a configuration will be described that creates color correction data that corrects the color characteristics of images acquired from image capture devices 101 and 102 in FIG. 1 so as to approximate each other. Image capture devices 101 and 102 each capture a chart 104 illuminated by a light source 103 from different locations as shown in FIG. 1, and transmit the captured images to the color correction data creation device. Chart 104 is, for example, a Macbeth chart in which multiple squares of different colors are arranged.

[0010] 2 is a block diagram showing a schematic configuration of a color correction data creation device 201 according to an embodiment of the present invention. The color correction data creation device 201 includes two image cropping units 202 and 203, two image inference units 204 and 205, and a color correction data creation unit 206.

[0011] The image cropping unit 202 crops out an area to be used for color matching from an input image, which is an image acquired from the imaging device 101. For example, the image cropping unit 202 crops out an area in which the chart 104 appears from the input image (hereinafter referred to as the "chart area") and outputs the image of this area as a cropped image to the image inference unit 204. As a method for cropping out the chart area, the image cropping unit 202 may recognize the chart area using AI or the like and crop this area from the input image. Alternatively, the user using the color correction data creation device 201 may specify the chart area in the input image, and the image cropping unit 202 may crop the area specified by the user from the input image.

[0012] The image cropping unit 203 uses the same cropping method as the image cropping unit 202 to crop an area to be used for color matching from the input image, which is an image acquired from the imaging device 102, and outputs the image of this area as a cropped image to the image inference unit 205.

[0013] The image inference unit 204 inputs the unknown cropped image acquired from the image cropping unit 202 into the trained model, and outputs an inferred image obtained by inferring an image of a chart region included in an image obtained when photographed under predetermined photographing conditions (to be described later) to the color correction data creation unit 206. The trained model is a model obtained by any machine learning. The trained model is, for example, a trained neural network whose parameters have been adjusted by backpropagation or the like. Note that the trained model may be a model other than a neural network. The image inference unit 205 inputs the unknown cropped image acquired from the image cropping unit 203 into the trained model, and outputs an inferred image obtained by inferring an image of a chart region included in an image obtained when photographed under the above-mentioned predetermined photographing conditions to the color correction data creation unit 206.

[0014] The color correction data creation unit 206 creates color correction data that corrects the color characteristics of the multiple acquired inference images to bring them closer together. For example, color correction data is created so that the hue of each color in the chart area of ​​the inference image corresponding to the image captured by the imaging device 102 matches the hue of the corresponding color in the inference image corresponding to the image captured by the imaging device 101. The color correction data is 3DLUT data. The 3DLUT data is correction LUT data that converts the RGB signal values ​​of the acquired image into corrected RGB signal values. Since it is generally difficult to create correction value table data that corresponds to all input signals, correction LUT data is created by dividing combinations of input signals into a certain range of gradation widths.

[0015] For example, if the input signal is 8-bit data and correction LUT data is created by dividing the gradation range from 0 to 255 into 15 steps, the number of data points in the correction LUT data is 4913 (=17 3 ) pieces. In this case, the image data included in the divided range is corrected by creating color correction data using tetrahedral interpolation or the like. In this embodiment, the color correction data is 3DLUT data, but the color correction data is not limited to 3DLUT data. For example, if color correction is performed using a one-dimensional LUT, the color correction data becomes 1DLUT data. In this embodiment, the color correction data is LUT data that corrects RGB signal values, but the present invention is not limited to this. If a YUV signal and a YCbCr signal are used, the LUT data may be YUV signal or YCbCr signal.

[0016] Next, learning of the learning model in this embodiment will be described.

[0017] Fig. 3 is a diagram showing an example of an environment for capturing images for learning in this embodiment. In Fig. 3, a chart 300 similar to chart 104 is placed on a stand, and an imaging device is placed in front of chart 300. Light sources 306 and 307 are also placed so as to illuminate chart 300.

[0018] In this embodiment, the chart 300 is photographed from different angles using the same imaging device. As an example, images are taken from five locations: imaging position 301, imaging position 302, imaging position 303, imaging position 304, and imaging position 305. Imaging position 301 is in front of the chart 300. Imaging position 302 is a location where the horizontal angle with respect to the chart 300 is 30° based on a reference axis connecting imaging position 301 and chart 300. Imaging position 303 is a location where the horizontal angle with respect to the chart 300 is −30° based on the reference axis. Imaging position 304 is a location where the horizontal angle with respect to the chart 300 is 60° based on the reference axis. Imaging position 305 is a location where the horizontal angle with respect to the chart 300 is −60° based on the reference axis.

[0019] The light source 306 and the light source 307 are light sources with different spectral characteristics. In this embodiment, the light source 306 is used as the reference light source. First, of the light source 306 and the light source 307, only the light source 306 is turned on, and images of the chart 300 are taken at each of the five shooting positions described above using the same imaging device. Next, of the light source 306 and the light source 307, only the light source 307 is turned on, and similarly, images of the chart 300 are taken at each of the five shooting positions described above using the same imaging device. The multiple images obtained by taking images in this manner are used as learning images. Note that, for ease of explanation, this embodiment has been described as using images taken at different horizontal angles as learning images. However, from the perspective of improving the learning accuracy of the learning model, it is preferable to also use images taken at different vertical angles as learning images.

[0020] Fig. 4 is a schematic diagram of learning of the learning model in this embodiment. The learning model performs machine learning using a reference image 400 from among multiple images obtained by capturing images in the capturing environment of Fig. 3 as training data, and using images 401 remaining from the multiple images excluding the reference image as input. The reference image 400 is an image obtained by capturing a chart 300 illuminated by a light source 306, which is a reference light source, from a capturing position 301, which is a reference capturing position, that is, from the front of the chart 300. Note that, in this embodiment, the reference capturing position is taken as the capturing position 301 as an example, but another capturing position may also be used as the reference capturing position.

[0021] The learning model is trained so that an image obtained by shooting at a location other than the reference shooting position or an image obtained by shooting using a light source other than the reference light source becomes an image obtained when the chart 300 illuminated by the reference light source is shot at the reference shooting position. Using such a trained model, the color correction data creation device 201 can infer an image obtained when a chart illuminated by the reference light source is shot from the reference shooting position from an image obtained by shooting from an angle other than the reference shooting position or an image obtained by shooting using a light source other than the reference light source.

[0022] Next, a description will be given of the color correction data creation process performed by the color correction data creation device 201. The color correction data creation process is executed when the color correction data creation device 201 acquires images from the image capture devices 101 and 102. These images are obtained by the image capture devices 101 and 102 capturing images of the chart 104 illuminated by the light source 103 from a position other than the front of the chart 104. It should be noted that the light source 103 has the same spectral characteristics as the light source 307, which is not the reference light source.

[0023] In the color correction data creation process, the image cropping unit 202 outputs a cropped image obtained by cropping a chart region from an image acquired from the image capture device 101 to the image inference unit 204. The image inference unit 204 outputs an inferred image obtained by inputting this cropped image into the trained model to the color correction data creation unit 206. This inferred image is an image obtained by inferring an image of the chart region included in an image obtained when the image capture device 101 captures an image of the chart 104 illuminated by the light source 306, which is the reference light source, from directly in front of the chart 104, which is the reference shooting position.

[0024] Furthermore, in the color correction data creation process, the image cropping unit 203 outputs a cropped image obtained by cropping a chart region from an image acquired from the image capture device 102 to the image inference unit 205. The image inference unit 205 outputs an inferred image obtained by inputting this cropped image into the trained model to the color correction data creation unit 206. This inferred image is an image obtained by inferring an image of the chart region included in an image obtained when the image capture device 102 captures an image of the chart 104 illuminated by the light source 306, which is the reference light source, from directly in front of the chart 104, which is the reference shooting position.

[0025] The color correction data creation unit 206 creates color correction data for correcting the inference image acquired from the image inference unit 205 so that the inference image acquired from the image inference unit 204 has the same color tone.

[0026] According to the above-described embodiment, the image inference unit 204 infers, based on the acquired cropped image, an image of the chart area included in the image obtained when the imaging device 101 captures the chart 104 illuminated by the reference light source from the reference shooting position. Furthermore, the image inference unit 205 infers, based on the acquired cropped image, an image of the chart area included in the image obtained when the imaging device 102 captures the chart 104 illuminated by the reference light source from the reference shooting position. In this way, even if the imaging devices 101 and 102 capture the chart 104 from different angles or capture the chart 104 illuminated by different light sources, it is possible to infer an image of the chart area included in the image obtained when the chart 104 illuminated by the reference light source is captured from the reference shooting position. Furthermore, the color correction data creation unit 206 creates color correction data that corrects the inferred image acquired from the image inference unit 205, i.e., the inferred image corresponding to the image captured by the imaging device 102, so that it has the same color as the inferred image acquired from the image inference unit 204, i.e., the inferred image corresponding to the image captured by the imaging device 101. In this way, it is possible to create color correction data that corrects the colors of inferred images obtained by inferring images of chart regions included in images captured at the same shooting position using the same light source so that the colors are the same between the inferred images. This enables color matching that is not dependent on changes in the shooting position of the imaging device or the color of the light source.

[0027] In the above-described embodiment, the trained model is obtained by machine learning using training images obtained by photographing the chart 300 illuminated by a reference light source from a reference photographing position as training data and training images obtained by photographing from an angle different from the reference photographing position as input. This makes it possible to infer, from images photographed from an angle other than the reference photographing position, the image that would be obtained when the chart illuminated by the reference light source is photographed from the reference photographing position.

[0028] In the above-described embodiment, the trained model is obtained by machine learning using as input training images obtained by photographing the chart 300 illuminated by a light source different from the reference light source. This makes it possible to infer, from images obtained by photographing using a light source other than the reference light source, an image that would be obtained when a chart illuminated by the reference light source is photographed from a reference photographing position.

[0029] In the above-described embodiment, the learning model may further learn using as input the shooting setting information used when capturing the learning image. The shooting setting information includes, for example, gamma information, color gamut information, color temperature information, ISO information, and F-number of the camera settings. Here, the imaging device can capture images by changing shooting settings such as gamma, color gamut, color temperature, ISO, and F-number. Even when capturing images of the same subject under the same environment, different shooting settings will output images with different colors. Therefore, from the perspective of improving inference accuracy, it is preferable to have the learning model learn about the shooting setting information. In contrast, in this embodiment, the learning model learns using as input the shooting setting information used when capturing the learning image. This makes it possible to infer, taking into account the shooting setting information, an image of the chart area included in an image obtained when capturing a chart illuminated by a reference light source from a reference shooting position, thereby reducing inference errors caused by differences in shooting settings.

[0030] Furthermore, in the above-described embodiment, the learning model may be trained using model information of the imaging device used to capture the training images as input. Imaging devices are developed by various manufacturers, and various models exist depending on the application. Even when the same shooting settings, such as the above-described gamma, color gamut, and color temperature, are used, the color characteristics vary from imaging device to imaging device. Therefore, from the perspective of improving inference accuracy, it is preferable to train the learning model using model information of the imaging device. In contrast, in the present embodiment, the learning model is trained using model information of the camera used to capture the training images as input. This allows the model information of the camera to be taken into account in inferring an image of the chart region included in an image obtained when a chart illuminated by a reference light source is captured from a reference shooting position, thereby reducing inference errors caused by differences in the model of the imaging device.

[0031] In the above-described embodiment, the image inference unit 204 and the image inference unit 205 may input an input image, which is an image acquired from the imaging device 101 or 102, to the trained model as an unknown image instead of a cropped image, and perform image inference processing. For example, when a chart is photographed using the entire shooting angle of view, the chart will be visible in the entire input image, and there is no need to perform the above-described cropping processing on such an input image. In such a case, the image inference unit 204 and the image inference unit 205 input the input image as an unknown image, instead of a cropped image, to the trained model, and perform image inference processing. This makes it possible to infer an image of the chart region included in an image obtained when a chart illuminated by a reference light source is photographed from a reference shooting position, without performing processing to crop the chart region from the input image.

[0032] In a configuration in which only an image obtained by photographing the chart at the full angle of view is acquired as the input image, the color correction data creation device 201 does not need to include the image cropping unit 202 and the image cropping unit 203.

[0033] In the above-described embodiment, the combination of the image cropping unit and the image inference unit may be increased depending on the number of input images. For example, a reference image is determined from the acquired multiple input images, and color correction data is created for each of the other images to correct the colors of the reference image. This makes it possible to collectively create multiple color correction data to approximate the colors of the reference image.

[0034] Furthermore, in the above-described embodiment, the learning model may be trained using light source characteristic information of the light source used when capturing the learning images as input. The light source characteristic information is information indicating the type of light source, such as sunlight, incandescent lamp, fluorescent lamp, or LED, and information on the spectral shape of the light source. This makes it possible to infer, by taking into account the light source characteristic information, an image of the chart region included in an image obtained when capturing an image of a chart illuminated by a reference light source from a reference capturing position, thereby reducing inference errors caused by differences in light source characteristics.

[0035] Furthermore, in the above-described embodiment, when capturing images for training, images of the chart 300 displayed on a display may be captured instead of the chart 300 printed on paper. Recently, VFX (short for visual effects) photography has been performed by replacing an entire wall with a display. When an entire wall is used as a display, it is possible to capture images of the chart displayed on the display, rather than capturing images of a chart printed on paper, and perform color matching between the imaging device and the display. In this case, the above-described embodiment can only train a training model using a combination of paint applied to paper and a light source, resulting in inference errors. This is because even if the same color as the chart is displayed on the display, the color of the inference image will differ due to differences in the reflection and spectral characteristics of the paint. Furthermore, the display has viewing angle characteristics that differ from the reflection of the paint, which also leads to errors in the inference image. In contrast, in the present embodiment, images of the chart 300 displayed on a display are captured when capturing images for training. This allows the training model to learn about the spectral and viewing angle characteristics of the display, thereby reducing inference errors caused by differences in the spectral and viewing angle characteristics of the display.

[0036] In addition, by preparing multiple displays with different spectral characteristics and viewing angle characteristics, taking images by changing the type of display and the angle of the imaging device, and using these images as learning images, it is possible to estimate images that take into account differences in display models.

[0037] In the above-described embodiment, if a change in ambient light is detected based on the acquired inference image, the above-described color correction data creation process may be performed to create new color correction data. Generally, when photographing outdoors, the weather changes over time, so even if color correction data is created as described above, the ambient light may change and the color correction data may not be optimal. In contrast, in the present embodiment, if a change in ambient light is detected based on the acquired inference image, the above-described color correction data creation process is performed to create new color correction data. This allows color correction data that corresponds to changes in ambient light to be created.

[0038] Next, a color correction data creation device and a control method thereof according to a second embodiment of the present invention will be described. The second embodiment is basically the same as the first embodiment described above in terms of configuration and operation, but differs from the first embodiment in that color correction data is created without photographing the chart 104. Therefore, a description of the overlapping configuration and operation will be omitted, and only the different configurations and operations will be described below.

[0039] Fig. 5 is a block diagram showing a schematic configuration of a color correction data creation device 501 according to the second embodiment. In Fig. 5, the color correction data creation device 501 includes the image inference units 204 and 205 shown in Fig. 2, and further includes an image clipping unit 502, an image clipping unit 503, and a color correction data creation unit 504.

[0040] Fig. 6 is a diagram showing an example of an imaging environment in which an unknown image is captured in the second embodiment. In the second embodiment, the imaging device 101 and the imaging device 102 capture images of a car 600, a person 601, and a person 602 illuminated by a light source 103 from different locations as shown in Fig. 6, and transmit the captured images to the color correction data creation device 501.

[0041] The image cropping unit 502 crops an area to be used for color matching from an input image obtained from the imaging device 101. For example, the image cropping unit 502 crops an area from the input image that includes all of the car 600, person 601, and person 602 (hereinafter referred to as the "subject area"), and outputs the image of the subject area as a cropped image to the image inference unit 204. As a method for cropping the subject area, the image cropping unit 502 recognizes the subject area using AI or the like and crops the subject area from the input image. In subject recognition, a region of a target object or a region of an object containing many colors in the input image is recognized as the subject area. The image inference unit 204 inputs the unknown cropped image obtained from the image cropping unit 502 into a trained model, and outputs an inferred image to the color correction data creation unit 504, which is an inferred image of the subject area included in an image obtained when the car 600, person 601, and person 602, which are subjects illuminated by a reference light source, are photographed from a reference photographing position.

[0042] The image cropping unit 503 crops out an area to be used for color matching from the input image, which is an image acquired from the imaging device 102, using the same cropping method as the image cropping unit 502, and outputs the image of this area as a cropped image to the image inference unit 205. The image inference unit 205 inputs the unknown cropped image acquired from the image cropping unit 503 into a trained model, and outputs to the color correction data creation unit 504 an inferred image obtained by inferring an image of a subject area included in an image obtained when photographing the subjects, a car 600, a person 601, and a person 602, illuminated by a reference light source from a reference photographing position.

[0043] The color correction data creation unit 504 creates color correction data that corrects the color characteristics of the inference images acquired from the image inference unit 204 and the image inference unit 205 so that they are closer to each other. For example, color correction data is created so that the hue of each color in the subject area of ​​the inference image corresponding to the image captured by the imaging device 102 matches the hue of the corresponding color in the inference image corresponding to the image captured by the imaging device 101. When creating color correction data, the color correction data creation unit 504 first compares the inference image acquired from the image inference unit 204 with the inference image acquired from the image inference unit 205, recognizes the subjects included in both inference images, and extracts the colors of the recognized subjects. Specifically, the color correction data creation unit 504 extracts the colors of the car 600, the person 601, and the person 602 included in both of the acquired inference images. Next, the color correction data creation unit 504 creates color correction data that matches the hue of the extracted colors. In creating color correction data, if the number of extracted colors is less than a predetermined value, or if the number of extracted colors is equal to or greater than a predetermined value but there is a bias in the colors, color correction data is created that performs color correction for the extracted colors and their approximate colors.Also, if the number of extracted colors is equal to or greater than a predetermined value, or if there is no bias in the colors, color correction data is created that performs color correction for all target colors.

[0044] In the second embodiment, the trained model used by the color correction data creation device 501 needs to estimate the colors of the subjects recognized by the image clipping unit 502 and the image clipping unit 503. For this reason, when training the trained model, images of various subjects illuminated by the light source 306, such as a car 600, a person 601, and a person 602, in the shooting environment of Fig. 3 are used as training images.

[0045] In the second embodiment described above, the image cropping unit 502 outputs a cropped image of an object region from an input image obtained by the imaging device 101 capturing an image of the car 600, person 601, and person 602 as objects to the image inference unit 204. The image cropping unit 503 outputs a cropped image of an object region from an input image obtained by the imaging device 102 capturing an image of the car 600, person 601, and person 602 as objects to the image inference unit 205. The image inference units 204 and 205 each infer an image of an object region included in an image obtained when each imaging device captures an object illuminated by a reference light source at a reference shooting position. The color correction data creation unit 504 creates color correction data that corrects the color characteristics of the inferred images obtained from the image inference unit 204 and the image inference unit 205 so that they are closer to each other. This allows color matching to be performed independently of changes in the shooting position of the imaging device or the color of the light source, even when an object other than a chart is captured.

[0046] Although the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0047] The present invention also includes a case where a software program that realizes the functions of the above-described embodiments is supplied to a system or device having a computer capable of executing the program directly from a recording medium or via wired or wireless communication, and the program is executed. Therefore, the program code itself that is supplied to and installed on a computer to realize the functional processing of the present invention also realizes the present invention. In other words, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In this case, the form of the program is not important, as long as it has the program functions, such as object code, a program executed by an interpreter, or script data supplied to an OS.

[0048] The recording medium for supplying the program may be, for example, a hard disk, a magnetic recording medium such as a magnetic tape, an optical / magneto-optical storage medium, or a non-volatile semiconductor memory.

[0049] Another method of supplying the program is to store the computer program forming the present invention in a server on a computer network, and have connected client computers download the computer program. [Explanation of symbols]

[0050] 101 Imaging device 102 Imaging device 104 Charts 201 Color correction data creation device 202 Image Cutout Section 203 Image Cutout 204 Image Inference Department 205 Image Inference Department 206 Color Correction Data Creation Department 400 reference images 401 other images 402 Inference Images 501 Color correction data creation device 502 Image Cutout Section 503 Image Cutout 504 Color Correction Data Creation Department 600 cars 601 People 602 People

Claims

1. A color correction data creation device that acquires a plurality of images obtained by photographing a predetermined subject using a plurality of imaging devices, an output means for inputting an image of a predetermined subject area included in the acquired image into a machine-learned trained model, and outputting an inferred image obtained by inferring an image of the predetermined subject area included in an image obtained when the predetermined subject is photographed under predetermined photographing conditions; A color correction data creation device characterized by comprising a creation means for creating color correction data that matches the color characteristics of multiple inference images output by the output means.

2. The color correction data creation device described in claim 1, characterized in that the multiple inferred images are images obtained by inferring images of a specified subject area included in images obtained when the multiple imaging devices each photograph the specified subject illuminated by a reference light source from a reference shooting position.

3. The color correction data creation device described in claim 2, characterized in that the trained model is obtained by machine learning using training images obtained by photographing the specified subject illuminated by the reference light source from the reference shooting position as training data and training images obtained by photographing the specified subject from an angle different from the reference shooting position as input.

4. The color correction data creation device described in claim 2 or 3, characterized in that the trained model is obtained by machine learning using as input a training image obtained by photographing the specified subject illuminated by a light source other than the reference light source.

5. The color correction data creation device according to claim 3 or 4, characterized in that the trained model is obtained by machine learning using as input shooting setting information used when the training image was taken.

6. The color correction data creation device according to any one of claims 3 to 5, characterized in that the trained model is obtained by machine learning using as input information about the model of the imaging device that captured the training image.

7. 7. The color correction data generating device according to claim 1, wherein the predetermined subject is a Macbeth chart printed on a sheet of paper.

8. 7. The color correction data generating device according to claim 1, wherein the predetermined subject is a Macbeth chart displayed on a display.

9. 9. The color correction data generating device according to claim 1, further comprising image cutting means for cutting out an image of the predetermined subject area from the acquired image.

10. The color correction data creation device described in any one of claims 1 to 9, characterized in that the creation means recognizes subjects contained in all of the multiple inference images, extracts the color of the recognized subjects, and creates color correction data that matches the color characteristics of the extracted color.

11. The color correction data creation device according to claim 10, characterized in that the creation means creates color correction data for performing color correction on the extracted colors and their approximate colors when the number of the extracted colors is less than a predetermined value or when there is a bias in the hue of the extracted colors, and creates color correction data for performing color correction on all target colors when the number of the extracted colors is more than the predetermined value or when there is no bias in the hue of the extracted colors.

12. 12. The color correction data creation device according to claim 1, wherein the creation means creates new color correction data when a change in ambient light is detected based on the plurality of inference images.

13. 1. A control method for a color correction data creation device that acquires a plurality of images obtained by photographing a predetermined subject using a plurality of imaging devices, comprising: an output process in which an image of a predetermined subject area included in the acquired image is input to a machine-learned trained model, and an inferred image is output, inferring an image of the predetermined subject area included in an image obtained when the predetermined subject is photographed under predetermined photographing conditions; A control method for a color correction data creation device, characterized by having a creation process of creating color correction data that matches the color characteristics of the multiple inference images output in the output process.

14. 13. A program for causing a computer to execute each means of the color correction data creating device according to claim 1.

Images