Image processing device and image processing method

Abstract

To efficiently match colors between captured images.SOLUTION: In order to solve the problem, an image processing apparatus according to the present invention includes: first acquisition means that acquires RAW image data and target image data of a target image captured by a second imaging device different from a first imaging device that has captured the RAW image data; estimation means that estimates a development parameter of the first imaging device so that an image after development of the RAW image data approximates image characteristics of the target image, using a learning model that has learned a correspondence relation between the development parameter used to develop the RAW image data and the image developed using the development parameter; and development means that develops the RAW image data using the development parameter estimated by the estimation means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing technique for developing a captured image. [Background technology]

[0002] The RAW data obtained from the image sensor of a digital camera or digital cinema camera during image capture holds the A / D converted output value of the image sensor itself. By performing development processing on the RAW data, image data (hereinafter also referred to as developed image) representing the captured image is generated. The development processing here is composed of one or more image processing such as demosaicing processing that converts pixel data in a Bayer array into pixel values ​​of RGB three channels, exposure correction, white balance correction, and noise removal processing. The parameters used in the development processing are also called development parameters, and if the photographer holds the RAW data, he or she can generate a developed image of his or her choice by adjusting the development parameters after capturing the image. Patent Document 1 describes that the development parameters used to develop the RAW data are obtained based on aesthetics that represent the desirability of the image.

[0003] Here, when trying to match the colors of images captured with different models of cameras, even if one tries to apply the same development parameters, it may not be possible to apply them because the development parameter items differ depending on the camera manufacturer, and even between cameras from the same manufacturer, the sensor output values ​​and development processes differ. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-162421 A Summary of the Invention [Problem to be solved by the invention]

[0005] In this case, the photographer must manually adjust the development parameters within the adjustable range to obtain the desired image while referring to the image taken with the camera to which the colors are to be matched, which requires time and effort. Also, some cameras have a color conversion function using 3DLUT to match colors, but creating a 3DLUT requires specialized measuring equipment and expertise.

[0006] The present invention has been made in consideration of the above-mentioned problems, and has an object to efficiently match colors between captured images. [Means for solving the problem]

[0007] In order to solve the above problems, an image processing device according to the present invention includes a first acquisition means for acquiring RAW image data and captured image data of a target image captured by a second imaging device different from a first imaging device that captured the RAW image data, and a learning model that learns a correspondence relationship between development parameters used to develop the RAW image data and an image developed using the development parameters, and adjusts development parameters of the first imaging device so that an image after development of the RAW image data approximates image characteristics of the captured image. and an estimation means for estimating the development parameters of the first imaging device. A developing means for developing the RAW image data; , and the learning model is a model generated for each item of the development parameters used by the estimation means. It is characterized by: Effect of the Invention

[0008] According to the present invention, by estimating development parameters that will develop an image close to a desired image from the results of learning combinations of development parameters and developed images, it is possible to efficiently match colors between captured images. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a hardware configuration of an image processing device 1. [Diagram 2] FIG. 1 is a block diagram showing a logical configuration of an image processing device 1. [Diagram 3] 4 is a flowchart showing the flow of processing by the image processing device 1. [Figure 4] FIG. 13 is a diagram showing an example of a graphical user interface. [Diagram 5] 4 is a flowchart of a process executed by a learning data generating unit 203. [Figure 6] FIG. 4 is a diagram showing an example of development parameters. [Figure 7] FIG. 1 is a diagram showing an example of a learning dataset. [Figure 8] 6 is a flowchart of a process executed by a development parameter estimation unit 204. [Figure 9] 5A to 5C are views for explaining a learning process executed by a development parameter estimation unit 204. [Figure 10] 4 is a flowchart of a process executed by a learning data generating unit 303. [Figure 11] 4 is a flowchart of a process executed by a learning data generating unit 303. [Figure 12] FIG. 1 is a diagram showing an example of a learning dataset. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the present embodiment will be described with reference to the drawings. Note that the following embodiment does not necessarily limit the present invention. Furthermore, not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present invention. EXAMPLES

[0011] <Hardware configuration of image processing device 1> FIG. 1 is a diagram showing an example of a hardware configuration of an image processing device 1. The image processing device 1 includes a CPU 101, a ROM 102, a RAM 103, a VC (video card) 111, a general-purpose I / F 114, and a SATA (serial ATA) I / F 118. The CPU 101 executes an OS and various programs stored in the ROM 102, an external storage device 110, etc., using the RAM 103 as a work memory. The CPU 101 also controls each component via a system bus 108. An input device 116 such as a mouse and a keyboard and an imaging device 117 are connected to the general-purpose I / F 114 via a serial bus 115. An external storage device 110 is connected to the SATA I / F 118 via a serial bus 119. A display 113 is connected to the VC 111 via a serial bus 112. The CPU 101 displays a UI (user interface) provided by a program on the display 113, and inputs input information indicating a user's instruction obtained via the input device 116. The image processing device 1 shown in FIG. 1 is realized by, for example, a desktop PC.

[0012] Furthermore, the external storage device 110 is not limited to an HDD. For example, it may be an SSD (solid state drive). The external storage device 110 may also be realized by a medium (recording medium) and an external storage drive for accessing the medium. The medium may be a flexible disk (FD), CD-ROM, DVD, USB memory, MO, flash memory, or the like.

[0013] <Configuration of image processing device 1> Fig. 2 is a block diagram showing the configuration of the image processing device 1. The image processing device 1 functions as shown in Fig. 2 by the CPU 101 executing a program stored in the ROM 102 using the RAM 103 as a work memory. Note that it is not necessary for all of the processes shown below to be executed by the CPU 101, and the information processing device 1 may be configured so that part or all of the processes are executed by one or more processing circuits other than the CPU 101.

[0014] The image processing device 1 includes an image acquisition unit 201, a development item acquisition unit 202, a learning data generation unit 203, a development parameter estimation unit 204, a development processing unit 205, and a display control unit 206. The image acquisition unit 201 acquires image data such as a target image to be color matched, RAW data to be corrected, and RAW data to be used for learning from the imaging device 117, the ROM 102, or the external storage device 110. Hereinafter, RAW data may also be called image data. The development item acquisition unit 202 acquires processing items for performing development processing such as white balance and gamma curve for the RAW data. Hereinafter, the acquired processing items may be called development items or items. The learning data generation unit 203 generates learning image data to be used for generating a learning model for the items acquired by the development item acquisition unit 202. The development parameter estimation unit 204 generates a learning model based on the data generated by the learning data generation unit 203, and estimates development parameters based on the learning model. A development processing unit 205 performs development processing on the original RAW data acquired by the image acquisition unit 201, based on development parameters estimated by a development parameter estimation unit 204. A display control unit 206 controls a user interface that allows a user to input information required for processing, and the display of a processed image.

[0015] <Processing Executed by Image Processing Device 1> 3 is a flowchart showing the flow of processing executed by the image processing device 1. Hereinafter, each step (process) will be represented by adding an S before the reference symbol.

[0016] In S11, the display control unit 206 causes the display 113 to display a graphical user interface (hereinafter, GUI) for allowing the user to input information required for processing. The display control unit 206 also inputs instructions from the user via the GUI. An example of the GUI is shown in Fig. 4. First, the display control unit 206 displays a GUI for performing development processing on the display 113.

[0017] 4, a target image data selection button 1001 is a button that allows the user to input the selection of a target image to be color matched. A RAW image data selection button 1002 is a button that allows the user to input the selection of RAW image data. A developed image save button 1003 is a button that allows the user to input the selection of a save destination for the developed image. An image display area 1004 is an area that displays an image, and displays the image data selected with the target image data selection button 1001 on the left side, and the image data after development processing on the right side.

[0018] When the development parameter estimation button 1005 is pressed, a process for estimating development parameters is executed, and the estimated parameters are set in each of the development parameter setting sections 1007 to 1015 described below. When the development process execution button 1006 is pressed, development processing of the image selected by the RAW image data selection button 1002 is executed based on the set development parameters.

[0019] The black level setting unit 1007 sets development parameters that affect the brightness level of dark areas of an image. The black gamma setting unit 1008 sets development parameters that affect the tone curve of dark areas of an image. The black cast setting unit 1009 sets development parameters that affect the color of dark areas of an image. The knee point setting unit 1010 sets development parameters that affect the starting point of compression in high brightness areas. The slope setting unit 1011 sets development parameters that affect the slope of the compression curve in high brightness areas. The gamma curve setting unit 1012 sets development parameters that affect the brightness gradation of the entire image. The white balance setting unit 1013 sets development parameters that affect the color of the entire image. The color matrix setting unit 1014 sets development parameters that affect the color of the entire image. The color correction setting unit 1015 sets development parameters that affect the color of a specific hue.

[0020] A gamma curve display area 1016 displays a luminance histogram of pixel values ​​of the developed image and the set gamma curve.

[0021] In S12, the image acquisition unit 201 acquires, as input images, image data indicating a target image to be color - matched (hereinafter also referred to as target image data). Specifically, when the image acquisition unit 201 inputs, from the display control unit 206, information specifying an image designated as the target image according to a user instruction, it identifies the location where the target image data is stored and reads it from the ROM 102. The target image data has already been developed by a predetermined developing process and is composed of image data for each of the R (red), G (green), and B (blue) colors. Note that the order of S11 and S12 may be reversed.

[0022] In S13, the developing item acquisition unit 202 acquires items of developing parameters (hereinafter also referred to as developing items) to be used in the developing process. For example, it acquires items in which instructions from the user are input in each item of the GUI in FIG. 4. Note that the items are not limited to user input, and at least one predetermined item may be used. The items of developing parameters to be acquired will be described later.

[0023] In S14, the learning data generation unit 203 generates a learning data set for generating a learning model for each developing item. The details of the processing in the learning data generation unit 203 will be described later.

[0024] In S15, the developing parameter estimation unit 204 generates a learning model for each developing item acquired in S13 and estimates the developing parameters. The details of the processing in the developing parameter estimation unit 204 will be described later.

[0025] In S16, the image acquisition unit 201 acquires RAW data to be corrected to match the image characteristics of the target image.

[0026] In S17, the image processing apparatus 1 performs developing processing on the RAW data using the developing parameters estimated in S15, saves the developed image data, and ends the processing.

[0027] <Operation of the learning data generation unit 203 in S14> FIG. 5 is a flowchart showing the flow of the process executed by the learning data generating unit 203 in S14.

[0028] In S141, the learning data generation unit 203 acquires the development items of the development parameters acquired in S13. In this embodiment, the nine development items that affect the image characteristics are "black level", "black gamma", "black cast", "knee point", "tilt", "gamma curve", "white balance", "color matrix", and "color correction". However, the invention is not limited to these, and other items may be used, or only some of these items may be used. The development process for each item will be described.

[0029] The "knee point" and "slope" set the point at which the camera signal is compressed and the slope at that point so that the input signal value of the high-brightness parts of the subject fits within the camera's dynamic range to prevent overexposure, as shown in Figure 6. Setting the "slope" to the negative side makes the slope gentler, expanding the dynamic range but reducing the ability to express gradations. On the other hand, setting the "slope" to the positive side makes the slope steeper, narrowing the dynamic range but increasing the ability to express gradations.

[0030] "Black level" and "black gamma" are parameters that adjust the expression of black, and as shown in Figure 6, when the "black level" adjustment value is set to the positive side, the black stands out, and when the adjustment value is set to the negative side, the black is crushed and the image has strong contrast. "Black gamma" sets the brightness range and level at which the black gamma is effective, and when the level is set to the positive side, the image becomes brighter, and when set to the negative side, the image becomes darker.

[0031] To deal with the "black cast" issue, which occurs when black is tinted, the gain amount is set as an adjustment item for each of the R, G, and B signal values, and the color cast in dark areas of the image is alleviated by adjusting the R, G, and B gain ratio.

[0032] The "gamma curve" is a parameter that sets the gradation characteristics between the input signal value and the camera output signal value, and sets the overall brightness of the image.

[0033] "White balance" is a parameter that sets the color of the entire image by adjusting the gain ratio of R, G, and B of the image.

[0034] "Color matrix" stores multiple conversion matrices for converting specific colors in an image to target colors according to the number of target colors, and performs color conversion using the conversion matrix according to the target color, such as sRGB or AdobeRGB. "Color correction" is a parameter that adjusts the color of a specific hue, and adjusts the brightness, saturation, and hue values ​​by dividing the hue angle into six parts according to the hue wheel: R / G, R / B, G / R, G / B, B / R, and B / G.

[0035] The development items described in this embodiment are merely examples, and the items may be changed to match the development items published by the development processing application of the camera to be used.

[0036] In S142, the learning data generation unit 203 acquires a group of RAW image data prepared in advance as learning image data to be used for generating a learning model. In this embodiment, 1000 pieces of RAW data in various categories such as people and landscape images, sports and hobbies, and food are prepared and stored in advance, but images in a specific category may be stored according to the user's preferences, and the number of images to be prepared is not limited to this.

[0037] In S143, the learning data generation unit 203 performs development processing on the RAW data acquired in S142 based on the items acquired in S141. An example of development settings is shown below, but the range, increment, and numerical values ​​of the setting values ​​are not limited to this and may be freely set within the range published by the development processing application of the camera to be used.

[0038] First, for compression of high-brightness areas, the "knee point" is set in 5% increments within the range of 75% to 100% of the input signal value, and the "slope" is changed in 5 steps on the ± side to perform development processing using 125 different settings, generating 125 developed images with different reproductions of high-brightness areas. For black, the "black level" is set in 10 steps up to ±10 in increments of 2 RGB counts, and the "black gamma" is set in three brightness ranges L*=0-10, 0-15, and 0-20, with three curves with different gamma strengths set on the positive side and three on the negative side. In addition, for "black cast," seven sets of coefficients with different RGB ratios are set, and for black, development processing is performed with 1470 settings for the three combinations of "black level," "black gamma," and "black cast," generating 1470 developed images with different reproductions of black. For the "gamma curve," 10 curves with different brightness gradation characteristics are set and development processing is performed to generate 10 developed images with different gradation characteristics. For "white balance," 50 sets of coefficients with different RGB ratios are set and development is performed to generate 50 developed images with different white chromaticity. For "color matrix," 10 3x3 determinants with coefficients pre-calculated to achieve the desired target value are prepared, and each is developed to generate 10 developed images with different color reproduction. For "color correction," development is performed using a total of 750 settings that change brightness by ±2 steps, saturation by ±2 steps, and hue by ±2 steps for each of the six hue angles, generating 750 developed images with different color reproduction for each specific hue.

[0039] In S144, the learning data generation unit 203 saves data pairing the developed image developed in S143 and the development parameters used in the development process as learning data.

[0040] 7 is a diagram showing a model of the learning data, and in this embodiment, the corresponding relationship between the development parameters and the developed images developed with those development parameters is labeled on a one-to-one basis. This labeling is performed for each development item on 1000 pieces of RAW data.

[0041] In S145, the learning data generation unit 203 determines whether learning data has been generated for all development items. If not, the process returns to the process of S143. If so, the process ends.

[0042] <Operation of the development parameter estimation unit 204 in S15> FIG. 8 is a flowchart showing the flow of the process executed by the development parameter estimation unit 204 in S15.

[0043] In S151, the development parameter estimation unit 204 acquires the learning dataset for generating the learning model generated in S14.

[0044] In S152, the development parameter estimation unit 204 performs a learning process. Note that the learning process used in this embodiment uses a convolutional neural network (CNN), and the outline of the process is shown in FIG. 9. The configuration of the convolutional layer and the pooling layer used in this process uses VGG16 consisting of 16 layers. In the convolutional layer, filter processing (3x3) of the input image and non-linear processing using an activation function are performed. In the pooling layer, reduction processing of the feature map processed in the convolutional layer is performed. The activation function uses the sigmoid function in this embodiment, but other non-linear processes such as the ReLU function may be used. The classifier classifies the category of the development parameters based on the feature amount map processed by the convolutional layer and the pooling layer. The loss function uses cross entropy, compares the predicted category of the classifier with the true category, and optimizes the weight parameter that minimizes the loss function using the gradient method. The developed image generated in S14 is set as the input image, and the labeled development parameters are set as the correct data, and a learning model for classifying the development parameters of the input image is generated based on the set image data and label.

[0045] In S153, the development parameter estimation unit 204 determines whether a learning model has been generated for all development items. If not, the process returns to the process of S152. If so, the process proceeds to the process of S154.

[0046] In S154, the development parameter estimation unit 204 acquires a target image to be color matched.

[0047] In S155, the development parameter estimation unit 204 estimates development parameters such that the developed image approximates the image characteristics of the target image using the learning model generated in S153. The development parameters are estimated for each development item using the learning model generated for each development item. The estimated development parameters are set in each development parameter setting section 1007 to 1015, and are applied when performing development processing.

[0048] In S156, the development parameter estimation unit 204 stores the development parameters estimated in S155, and ends the processing.

[0049] As described above, according to this embodiment, the combination of development parameters and developed images is learned, and the learning results are used to estimate development parameters that will develop an image close to a desired image. This makes it possible to efficiently and automatically match colors between the camera that captured the desired image and other cameras. EXAMPLES

[0050] In the first embodiment, a method has been described in which one learning model is generated for each development parameter of a development item and this is applied to the estimation of the development parameters.

[0051] In the method of the first embodiment, there is a possibility that the optimal solution will not be obtained when multiple individually optimal development parameters are combined, so in the second embodiment, a method of generating a learning model using multiple development parameters will be described.

[0052] Specifically, in Example 1 regarding the setting of development parameters, as items for adjusting brightness reproduction, there are "gamma curve" for adjusting the overall tone, "black level" and "black gamma" for adjusting the tone of the dark part, "knee point" and "slope" for adjusting the tone of the highlight part. Since these determine the tone of the image brightness and interact with each other, in Example 2, no individual optimization is performed. First, the general tone is adjusted with the "gamma curve" item, and then, processing is performed to determine the detailed tone for the dark part and the highlight part.

[0053] Therefore, among the 16-layer network used in Example 1, the first 13 layers are trained using the dataset of the "gamma curve", and the remaining 3 layers are trained using the dataset corresponding to the local development items of the dark part and the highlight part.

[0054] Note that since the hardware configuration of the image processing apparatus 1 in this example is the same as that of Example 1, the description is omitted. In the following, the processing of the development parameter estimation unit 204 will be described as S25 newly in Example 2 instead of S15 in Example 1. Also, for the same configuration as the first embodiment, the same reference numerals will be used for description.

[0055] <Operation of the Development Parameter Estimation Unit 204 in S25> FIG. 10 is a flowchart showing the flow of processing executed by the development parameter estimation unit 204 in S25.

[0056] In S251, the development parameter estimation unit 204 acquires the learning data corresponding to the "gamma curve" from the learning dataset for generating the learning model generated in S14.

[0057] In S252, the development parameter estimation unit 204 performs learning processing. The VGG16 consisting of 16 layers in the network configuration is used with CNN. The details of the processing are the same as the method described in Example 1.

[0058] In S253, the development parameter estimation unit 204 fixes the first to thirteenth layers of the network to the parameters learned in S252, and replaces the three layers, the fourteenth to sixteenth layers, with new layers whose parameters have not yet been optimized.

[0059] In S254, the development parameter estimation unit 204 acquires learning data corresponding to the "black level" and "black gamma" related to black gradation reproduction from the learning data set for generating the learning model generated in S14.

[0060] In S255, the development parameter estimation unit 204 acquires learning data corresponding to the "knee point" and "slope" related to compression of high-brightness areas from the learning data set for generating the learning model generated in S14.

[0061] In S256, the development parameter estimation unit 204 trains the three layers from the 14th layer to the 16th layer using the training data acquired in S254 and S255.

[0062] In S257, the development parameter estimation unit 204 acquires a target image to be subjected to color matching.

[0063] In S258, the development parameter estimation unit 204 estimates development parameters from the target image using the learning model generated in S252 and S256. Note that in this embodiment, the first 13 layers are fixed, and the 14th to 16th layers are replaced with new layers to determine more detailed gradation reproduction, but the layer configuration is not limited to this, and for example, the 12th to 16th layers may be replaced and learned. Furthermore, the data set used for learning is not limited to this, and for example, if an item of the "color matrix" contributes to the gradation of brightness, this may be replaced with a new layer and learned.

[0064] In S259, the development parameter estimation unit 204 stores the development parameters estimated in S258, and ends the processing.

[0065] As described above, in Example 2, a learning model was generated using a plurality of development parameters for development parameters that interact with each other, and a method for estimating development parameters was described. By using a plurality of development parameters, it becomes possible to estimate an image closer to the post-development state, and the colors between cameras can be efficiently matched.

Example

[0066] In Example 3, as a method for more efficiently matching the color reproduction characteristics to the target image, a method will be described in which a learning data set is set as a learning model based on the difference value between the target image and the post-development image, and more optimal development parameters are derived. Note that since the hardware configuration of the image processing apparatus 1 in this example is the same as that in Example 1, the description thereof will be omitted. In the following, the process of the development parameter estimation unit 204 will be described as S35 newly in Example 3 instead of S15 in Example 1. Also, the same components as those in Example 1 will be described with the same reference numerals.

[0067] <Operation of the Development Parameter Estimation Unit 204 in S35> FIG. 11 is a flowchart showing the flow of processing executed by the development parameter estimation unit 204 in S35.

[0068] In S351, the development parameter estimation unit 204 acquires the development items acquired in S13.

[0069] In S352, the development parameter estimation unit 204 acquires the RAW data before the development process corresponding to the target image acquired in S12, and performs the development process by changing a plurality of development parameters for each development item.

[0070] In S353, the development parameter estimation unit 204 calculates the difference in pixel values ​​between the target image and the multiple developed images developed in S352. In this embodiment, the difference in pixel values ​​is calculated for each R, G, and B pixel for each coordinate position and averaged, but other statistical values ​​such as the median or mode may be used. Also, instead of the difference in pixel values, it is possible to calculate a luminance value from the pixel values ​​using a predetermined conversion formula such as sRGB or AdobeRGB, or a color value using a CIE-Lab conversion formula, and use the luminance difference or color difference value that is the difference value.

[0071] In S354, the development parameter estimation unit 204 calculates a variance value between the number of images for which the development parameters were changed in S352, based on the statistical value that indicates the difference from the target image calculated in S353. Note that although the variance value is used in this embodiment, other statistical values ​​such as standard deviation may be used. It is considered that the larger the variance, the higher the contribution rate of the development parameters of that development item to the target image, and the smaller the variance, the lower the contribution rate, and learning is performed so that classification by the learning model gives a higher weight to development items with a high contribution rate.

[0072] In S355, the development parameter estimation unit 204 determines whether or not the variance between the numbers of developed sheets has been calculated for each development item for all development items. If calculated, the process proceeds to S356, and if not, the process returns to S351.

[0073] In S356, the development parameter estimation unit 204 performs a learning process. A CNN is used, and the network configuration is VGG16 consisting of 16 layers. The details of the process are the same as those described in the first embodiment. As a method of setting the dataset used for learning, items with a small contribution of the development parameters are set in the lower layers of the learning model, and items with a large contribution of the development parameters are set in the upper layers to increase the learning weight, and learning is performed.

[0074] In S357, the development parameter estimation unit 204 acquires a target image to be color matched.

[0075] In S358, the development parameter estimation unit 204 estimates development parameters from the target image using the learning model generated in S356.

[0076] In S359, the development parameter estimation unit 204 stores the development parameters estimated in S358, and ends the processing.

[0077] As described above, according to the third embodiment, a learning model is generated so that the weight of the network increases in the order of the contribution rate of the correction target image to the target image, and development parameters are estimated. By increasing the optimization weight for development items that have a large impact on color reproduction, it is possible to efficiently match colors between cameras.

[0078] Other Examples In the first and second embodiments, a method for estimating development parameters related to color reproduction has been described as a method for efficiently matching colors between cameras. In addition to color reproduction, camera characteristics include sharpness and noise, and in order to match these characteristics, a learning dataset for development items such as sharpness and NR may be generated as shown in FIG. 12, and these development parameters may be estimated. For sharpness, the correspondence between images developed with multiple different sharpness intensities and their sharpness setting values ​​is stored in the dataset, and for NR, images developed with multiple different NR intensities and their NR development parameters are stored in the dataset. By performing the above-mentioned learning process using these datasets, it is possible to estimate development parameters related to sharpness and noise, not limited to color reproduction.

[0079] This embodiment can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

Claims

1. a first acquisition means for acquiring RAW image data and captured image data captured by a second imaging device different from a first imaging device that captured the RAW image data; an estimation means for estimating development parameters of the first imaging device using a learning model that has learned a correspondence relationship between development parameters used to develop RAW image data and an image developed using the development parameters, so that an image after development of the RAW image data approximates image characteristics of the captured image; a developing unit that develops the RAW image data using development parameters of the first imaging device, The image processing apparatus according to claim 1, wherein the learning model is a model generated for each item of development parameters used by the estimation means.

2. 2. The image processing apparatus according to claim 1, wherein the item is a development item related to color reproduction characteristics of the image.

3. Further, a second acquisition means for acquiring the item; a generating means for developing the RAW image data acquired by the first acquiring means using the plurality of development parameters for the items acquired by the second acquiring means, and generating the correspondence relationship; 3. The image processing device according to claim 1, further comprising:

4. The image processing device according to any one of claims 1 to 3, characterized in that the learning model is a model in which the items related to the reproduction of brightness of the entire image are set in a lower layer of the learning model and learning is performed, and the items related to the reproduction of brightness of local areas of the image are set in an upper layer of the learning model and learning is performed.

5. A program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 4.

6. a first acquisition step of acquiring RAW image data and captured image data captured by a second imaging device different from a first imaging device that captured the RAW image data; an estimation step of estimating development parameters of the first imaging device using a learning model that has learned a correspondence relationship between development parameters used to develop RAW image data and an image developed using the development parameters, so that an image after development of the RAW image data approximates image characteristics of the captured image; a developing step of developing the RAW image data using development parameters of the first imaging device, An image processing method, characterized in that the learning model is a model generated for each item of development parameters used in the estimation process.

Images