Image processing apparatus, method, and program

Abstract

To adjust the degree of lookup table conversion, while maintaining the effect of format conversion included in a LUT input by a user.SOLUTION: In an image processing apparatus, an image processing unit has: first acquisition means that acquires a first LUT for applying the effect of predetermined format conversion and target color conversion; second acquisition means that acquires a second LUT for applying, of the effect applied by the first LUT, the effect of the predetermined format conversion; setting means that sets the degree of the influences of the LUTs applied to a first image to be processed; composition means that combines the first LUT and the second LUT with each other on the basis of the degree of influences to create a third LUT; and processing means that performs color conversion on the first image by using the third LUT.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for performing image processing using a lookup table. [Background technology]

[0002] Look-up tables (LUTs) are commonly used to correct the color and contrast of images. Cameras also import LUTs from outside and apply them to captured images.

[0003] One of the uses of LUTs is to correct color and tone in image creation (hereafter referred to as look conversion). Users can easily convert images captured with a camera using LUTs they have created themselves or LUTs published on the web. However, the perception of color and tone is subjective, and the effects of LUTs may not be suitable for all users, and there is also a demand to adjust the strength of the LUT effect.

[0004] Patent Document 1 discloses a technique for adjusting the effect of the LUT. Specifically, Patent Document 1 discloses a technique for detecting scenes and synthesizing a correction table for each scene according to the reliability of multiple candidate scenes. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-165147 A Summary of the Invention [Problem to be solved by the invention]

[0006] In the above Patent Document 1, the camera judges the scene and adjusts the correction table. Therefore, the correction table does not necessarily perform conversion that matches the user's preferences. In addition to look conversion, the LUT can also be used to perform format conversion for converting gamma and color gamut. For example, when converting an image conforming to the PQ of ITU-R BT.2100 to an image conforming to ITU-R BT.709, the format conversion can be performed by applying an LUT that has the effect of the conversion. Furthermore, it is possible to include both the effect of the format conversion and the effect of the look conversion in one LUT. When adjusting the strength of the effect with such an LUT, it is desirable to be able to adjust only the strength of the look conversion while maintaining the effect of the format conversion.

[0007] As explained in the above example, when adjusting the effect of conversion for so-called image creation, it is necessary to take into account the influence of format conversion. [Means for solving the problem]

[0008] In order to solve this problem, for example, an image processing device according to the present invention has the following arrangement. A first acquisition means for acquiring a first LUT for applying the effect of a predetermined format conversion and a target color conversion; a second acquisition means for acquiring a second LUT that applies an effect of the predetermined format conversion from among effects of the first LUT; A setting means for setting an influence degree of the LUT to be applied to the first image to be processed; a synthesis means for synthesizing the first LUT and the second LUT based on the degree of influence to generate a third LUT; and a processing means for performing color conversion on the first image by using the third LUT. Effect of the Invention

[0009] According to the present invention, it is possible to adjust the effect of conversion for image creation while taking into account the influence of format conversion. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing the internal configuration of an imaging apparatus according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing the internal configuration of an image processing unit 24 according to the first embodiment. [Diagram 3] 4 is a flowchart showing a processing flow according to the first embodiment. [Figure 4] FIG. 4 is a view showing an example of LUT data according to the first embodiment. [Diagram 5] FIG. 11 is a block diagram showing the internal configuration of an image processing unit 24 according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] [First embodiment] FIG. 1 is a block diagram of an imaging device 100 according to the first embodiment. Examples of imaging devices include so-called digital cameras, as well as smartphones and tablet devices with imaging functions, medical equipment, and industrial equipment. In FIG. 1, an imaging lens 103 includes a lens group including a zoom lens, a focus lens, and a shift lens, and forms an image of a subject on an imaging surface of an imaging unit 32. An aperture 101 is an aperture used to adjust the amount of light, and an ND (Neutral Density) filter 104 is an ND (Neutral Density) filter that reduces light. The imaging unit 22 has a plurality of imaging elements arranged two-dimensionally on its imaging surface, which are composed of CCDs, CMOS elements, etc. that convert optical images into electrical signals. The imaging unit 22 also has functions such as controlling accumulation by an electronic shutter, changing analog gain, and changing readout speed. An A / D converter 23 converts an analog signal output from the imaging unit 22 into a digital signal.

[0013] The image processing unit 24 performs processes such as predetermined pixel interpolation and resizing such as reduction, detection of luminance information, color information, characteristic objects, and the like, color conversion, gamma correction, and digital gain addition, on the image data from the A / D converter 23 or the image data from the memory control unit 15. These image processing methods include image processing using a dedicated arithmetic circuit and image processing using an LUT application circuit. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and transmits the arithmetic results to the system control unit 50.

[0014] The system control unit 50 performs exposure control, distance measurement control, white balance control, etc., based on the above calculation results received from the image processing unit 24. This allows AF (autofocus) processing of a TTL (through-the-lens) method, AE (automatic exposure) processing, AWB (auto white balance) processing, etc. to be performed. In addition, the system control unit 50 performs image blur correction by operating the shift lens of the imaging lens 103 or by shifting the image in the image processing unit 24 in response to a movement or change in attitude of the imaging device 100 due to camera shake or the like detected by the gyro 40.

[0015] The output data from the A / D converter 23 is directly written into the memory 32 via the image processing unit 24 and the memory control unit 15. The memory 32 stores image data captured by the imaging unit 22 and converted into digital data by the A / D converter 23, and image data to be displayed on the display unit 28. The memory 32 is also used to temporarily store images that have been processed by the image processing unit 24, and then return the images to the image processing unit 24 for application of other image processing. The memory 32 has a storage capacity sufficient to store moving images and audio for a predetermined period of time.

[0016] The memory 32 also serves as a memory for image display (video memory). The D / A converter 13 converts the image display data stored in the memory 32 into an analog signal and supplies it to the display unit 28. In this way, the display image data written in the memory 32 is displayed by the display unit 28 via the D / A converter 13. The display unit 28 performs display according to the analog signal from the D / A converter 13 on a display device such as an LCD. The digital signal once A / D converted by the A / D converter 23 and stored in the memory 32 is converted to analog in the D / A converter 13 and sequentially transferred to the display unit 28 for display, thereby realizing an electronic viewfinder and enabling through-image display.

[0017] The non-volatile memory 56 is an electrically erasable and recordable memory, for example, an EEPROM. Constants, programs, etc. for the operation of the system control unit 50 are stored in the non-volatile memory 56. The programs referred to here are programs for executing various flowcharts described later.

[0018] The system control unit 50 controls the entire imaging device 100. By executing the programs recorded in the nonvolatile memory 56 described above, each process of this embodiment described below is performed. A RAM is used for the system memory 52, and constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like are expanded. The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 13, the display unit 28, and the like.

[0019] The system timer 53 is a timing unit that measures the time used for various controls and the time of a built-in clock. The mode change switch 60, the recording switch 61, and the operation unit 70 are operation means that transmit instruction inputs from the user to the system control unit 50.

[0020] The mode changeover switch 60 notifies the system control unit 50 of the operation mode, which may be a moving image recording mode, a still image recording mode, a playback mode, or the like. The system control unit 50 receives this notification and performs a process of transitioning to the corresponding operation mode. The moving image recording mode and the still image recording mode also have more detailed modes, which include an auto shooting mode, an auto scene determination mode, a manual mode, various scene modes in which shooting settings are made for each shooting scene, a program AE mode, a custom mode, and the like. By operating the mode changeover switch 60, it is possible to directly switch to any of these modes included in the moving image shooting mode. Alternatively, after switching to the moving image shooting mode once with the mode changeover switch 60, it is also possible to switch to any of these modes included in the moving image shooting mode using another operating member. The recording switch 61 is a switch for switching between a shooting standby state (OFF) and a shooting state (OB). When the recording switch 61 is turned ON, the system control unit 50 starts a series of operations from reading out a signal from the imaging unit 22 to writing moving image data to the external recording medium 150. When the recording switch 61 is turned OFF, the system control unit 50 ends writing moving image data to the external recording medium 150.

[0021] Each operating member of the operating unit 70 is assigned a function appropriate for each scene by selecting and operating various functional icons displayed on the display unit 28, and operates as various functional buttons. Examples of the functional buttons include an end button, a back button, an image forward button, a jump button, a filter button, and an attribute change button. For example, when the menu button is pressed, a menu screen in which various settings can be made is displayed on the display unit 28. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the four-way keys in the up, down, left, and right directions, and the SET button.

[0022] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to which electricity is applied, and detects whether a battery is attached, the type of battery, and the remaining battery power. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the necessary voltage to each unit including the external recording medium 150 for the necessary period.

[0023] The power supply unit 30 is made up of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li-ion battery, an AC adapter, etc. The I / F 18 is an interface with an external recording medium 150 such as a memory card or a hard disk, and an external display device 160. The external recording medium 150 is a recording medium such as a memory card for recording captured images and exchanging data with the outside, and a semiconductor memory or the like is used.

[0024] Next, the configuration of the image processing unit 24 in this embodiment will be described with reference to Fig. 2. The captured image data input from the A / D converter 23 is input to a basic image processing unit 241. The processing performed by the basic image processing unit 241 includes gain processing for correcting the brightness of the image data, white balance processing for correcting the white balance, color correction processing for correcting the color tone, and the like. The processing performed by the image processing unit 24 also includes edge enhancement processing for increasing the sharpness of the image and noise reduction processing for suppressing noise. The basic image processing unit 241 supplies the image data obtained by performing these image processes to the LUT processing unit 242.

[0025] The LUT processing unit 242 applies the LUT data to the image data input from the basic image processing unit 241 to convert the signal. The LUT processing unit 242 of this embodiment can perform conversion using influence-changed LUT data 504 generated based on the influence indicated by the influence 503. This influence-changed LUT data 504 is generated from the following information: user input LUT data 501 input by a user operation and targeted for use by the user, format conversion LUT data 502 corresponding to the format conversion included in the user input LUT data 501, and their influence 503 (described in detail later). The LUT processing unit 242 outputs the image data obtained by conversion using the influence-changed LUT data 504 to the memory 32 and the I / F 18.

[0026] Here, the user-input LUT data is LUT data held outside the imaging device 100 and input to the imaging device 100 by a user operation. For example, various LUTs that match the image quality and atmosphere that the user is aiming for are provided on the homepage of a camera manufacturer. In addition, there are also applications that allow the user to create an LUT by himself. The user can input new LUT data to the imaging device 100 by storing the provided LUT in the external storage medium 150 using a PC or the like and connecting the external storage medium 150 to the imaging device 100. Note that the method of inputting the LUT is not limited to this, and for example, if the I / F 18 has a network I / F function, the LUT may be input via a network based on a user operation. In addition, the imaging device 100 may have a function of generating an LUT based on a user operation and registering (inputting) it as a new LUT.

[0027] Next, the processing in this embodiment will be described with reference to the flowchart in Fig. 3. In the following description, it is assumed that the external recording medium 150 already stores user input LUT data desired by the user.

[0028] In S31, the system control unit 50 reads out the user input LUT data 501 stored in the external recording medium 150 to the system memory 52 via the I / F 18. This user input LUT data 501 is an LUT that represents any image conversion process that the user wants to apply, and is stored as text data or binary data. The user input LUT data 501 may be a one-dimensional LUT that applies the same conversion process to each of the RGB signals, or a three-dimensional LUT that uses vectors having three elements of the RGB signals as input and output. In addition, the color space that defines the conversion is not limited to RGB, and may be any color space such as YUV or Lab. In this embodiment, the description is based on the premise of an LUT that defines the conversion in a three-dimensional RGB space.

[0029] In S32, the system control unit 50 acquires format conversion information of the user input LUT data 501 input in S31. Usually, the LUT generates table values ​​by taking into account not only calculations for simply changing the color tone or atmosphere (also called look conversion or image creation) but also calculations for specific format conversion. Here, the format is information that indicates what kind of gamma and color gamut an image should be treated as. For example, an image conforming to ITU-R BT.709 is converted into numerical data by a calculation formula for gamma and color gamut defined in ITU-R BT.709. Therefore, a display conforming to ITU-R BT.709 displays an image based on the gamma and color gamut defined in ITU-R BT.709. In this step, the system control unit 50 acquires two formats, the conversion source format (first partial format conversion) and the conversion destination format (second partial format conversion) specified by the user input LUT data 501, and acquires format conversion information from the information.

[0030] First, in the case of an imaging device such as that of this embodiment, the format information of the source of conversion can be obtained from the system memory 52 because it is considered that the user has set it in advance via the operation unit 70. If only one format can be photographed, the user does not set it via the operation unit 70, but since the format is fixed, that information is used as the format information of the source of conversion. In the case of software that handles images photographed by an imaging device rather than an imaging device, it is possible to obtain the information from the metadata of the image or have the user specify it.

[0031] The destination format information is either input via the operation unit 70 as accompanying information when the user inputs the user-input LUT data 501, or is obtained by reading information added to the user-input LUT data 501.

[0032] In S33, the system control unit 50 generates format conversion LUT data 502 from the format conversion information acquired in S32.

[0033] As an example, consider a case where the format conversion information indicates conversion from a gamma and color gamut conforming to PQ (Perceptual Quantization) of ITU-R BT.2100 to a gamma and color gamut conforming to ITU-R BT.709. This conversion is realized by canceling the gamma of PQ, performing color gamut conversion, and converting to the gamma of BT.709. Therefore, the format conversion LUT data 502 can be generated by the following procedure. 1) Apply the inverse transformation of the PQ opto-electronic transfer function (OETF) described in ITU-R BT.2100 to untransformed LUT data having the same number of elements as the LUT data acquired in S31. 2) A matrix for converting the PQ color gamut specified in ITU-R BT.2100 to the color gamut specified in ITU-R BT.709 is calculated, and multiplied by the result of 1). 3) Apply the OETF described in ITU-R BT.709 to the results of 2).

[0034] The above 1) to 3) are procedures for generating the format conversion LUT data 502 for performing gamma conversion via an OETF, but there is also a method for performing gamma conversion via an electronic-opto transfer function (EOTF).

[0035] In S34, the system control unit 50 acquires the degree of influence, which indicates the strength of the effect of the user input LUT data 501 on the image. This is acquired by having the user input it via the operation unit 70. Alternatively, the initial degree of influence added to the LUT data acquired in S31 may be acquired. The degree of influence as the user input or accompanying information of the LUT data may be expressed in any manner. For example, it may be a value such as 0% to 100%, or may be expressed as weak, medium, or strong. In this embodiment, the user input LUT data 501 is set to the maximum effect (1.0 or 100%), and a value obtained by quantifying this as 0 (0%) to 1.0 (100%) is acquired as the degree of influence. In this embodiment, the intention of this degree of influence is the degree of the effect of so-called look conversion. That is, the user sets the degree of influence with the intention of, for example, making the effect of the look conversion strong or weak, suppressing the effect to about 50%, or applying the effect of 100%.

[0036] In S35, the system control unit 50 generates influence-changed LUT data 504 based on the influence acquired in S34, the user-input LUT data 501 acquired in S31, and the format conversion LUT data 502 acquired in S33. In this embodiment, a method of performing linear interpolation between the LUT data acquired in S31 and the format conversion LUT data 502 acquired in S33 based on the influence acquired in S34 and synthesizing the LUTs will be described.

[0037] 4(a) and (b) are examples of LUT data, where (a) in the figure is the format conversion LUT data 502 generated in S33, and (b) in the figure is the user input LUT data 501 acquired in S31. Linear interpolation is performed on all rows, but the following description focuses on the rows indicated by reference numbers 401 and 402. It should be understood that the same applies to the other rows.

[0038] If the influence level obtained in S34 is denoted as α (internally stored as a value between 0 and 1), the linear interpolation results R(α), G(α), and B(α) of each of the RGB colors in the rows indicated by reference numbers 401 and 402 are as follows: R(α)=0.0004×(1-α) + 0.8624×α = 0.8620α+0.0004 G(α)=0.3450×(1-α) + 1.0000×α = 0.6550α+0.3450 B(α)=0.0198×(1-α) + 0.8068×α = 0.7870α+0.0198

[0039] Therefore, if α=0.6, then R=0.5175..., G=0.7379..., and B=0.4290.... If α=0.3, then R=0.2590..., G=0.5414..., and B=0.2559.... By performing the same calculation for all rows, it is possible to generate influence-changed LUT data 504. This calculation indicates that, in the user-input LUT data 501 acquired in S31, the effect of the look conversion is changed to an effect multiplied by the influence α, while the effect of the format conversion does not change from the original effect. In this way, the influence-changed LUT data 504 becomes LUT data in which only the effect of the look conversion is adjusted by the influence.

[0040] In S36, the system control unit 50 applies the influence-changing LUT data 504 generated in S35 to the LUT processing unit 242 to execute image conversion. That is, the LUT processing unit 242 generates and outputs image data to which the effect of the influence-changing LUT data 504 has been applied to the image data to be processed supplied from the basic image processing unit 241. Thereafter, the system control unit 50 records the image data output from the LUT processing unit 242 as a file in the external storage medium 150, or displays it on the display unit 28 or the external display device 160.

[0041] In this embodiment, a method for generating an influence-changed LUT by synthesizing LUT data input by a user based on a user-desired influence level with LUT data generated from format conversion information corresponding to the LUT data has been described. According to this embodiment, the influence-changed LUT can change only the influence level of the look conversion while maintaining the effect of the format conversion, so that it is possible to prevent the image format from becoming inappropriate when the influence level is weakened, and a natural image can be obtained by adjusting only the strength of the LUT conversion.

[0042] [Second embodiment] In the above first embodiment, a method for directly interpolating LUT data input by a user and LUT data corresponding to the format conversion included in the LUT data input by a user according to the degree of influence has been described. However, in the first embodiment, the interpolation is performed in the space of the conversion destination format, so that the state of look conversion when the degree of influence is changed may differ depending on the format. Therefore, in the present second embodiment, a method for preventing a large difference in the appearance of the obtained image even when the conversion destination format is different by performing the interpolation according to the degree of influence in a degamma space will be described.

[0043] The overall flow is the same as in the first embodiment. Therefore, as in the first embodiment, the description will be given with reference to the flowchart in Fig. 3 and the block diagram in Fig. 2. Moreover, since the processes in this second embodiment from S31 to S34 in Fig. 3 are the same as those in the first embodiment, the description thereof will be omitted.

[0044] In S35, the system control unit 50 generates influence-changed LUT data 504 based on the format conversion LUT data 502, the user input LUT data 501, and the influence 503. However, unlike the first embodiment, the system control unit 50 performs interpolation based on the influence in a state excluding conversion processing from each LUT to the conversion destination format, and after the interpolation, applies conversion processing to the conversion destination format again. A specific example is as follows.

[0045] For example, consider a case where the source format of the user-input LUT data 501 is in compliance with PQ of ITU-R BT.2100, and the destination format has two patterns of LUTs, one in compliance with HLG (Hybrid Log Gamma) of ITU-R BT.2100 and the other in compliance with ITU-R BT.709. In this case, the EOTF of PQ described in ITU-R BT.2100 is set to F PQ , EOTF of HLG described in ITU-R BT.2100 is F HLG , EOTF described in ITU-R BT.709 is F 709 The conversion from the XYZ color space to the RGB color space according to the PQ color gamut definition in ITU-R BT.2100 is expressed as M PQ ,Conversion from XYZ color space to RGB color space according to the HLG color gamut definition of ITU-R BT.2100 HLG ,Conversion from XYZ color space to RGB color space according to the color gamut definition of ITU-R BT.709 709 Then, when the pixel value of the input image is x, the conversion from ITU-R BT.2100 PQ to ITU-R BT.2100 HLG and from ITU-R BT.2100 PQ to ITU-R BT.709 is as follows: For PQ to HLG conversion F HLG -1 (M HLG (M -1 PQ (F PQ (x)))) When converting from PQ to BT.709 F 709 -1 (M 709 (M -1 PQ (F PQ (x))))

[0046] The format conversion LUT data 502 conforms to the above formula. Moreover, since the user input LUT data 501 is the above format conversion plus an arbitrary conversion process, if the arbitrary conversion process is represented as C, the user input LUT data 501 will be as follows: For PQ to HLG conversion F HLG -1 (M HLG (CM PQ -1 (F PQ (x))))) When converting from PQ to BT.709 F 709 -1 (M 709 (CM PQ -1 (F PQ (x)))))

[0047] If the conversion processes other than the arbitrary conversion process C are reversible, the position of the arbitrary conversion process C is determined by the MPQ method as described above, regardless of the calculation order in which the user-input LUT data 501 is actually created. -1 It can be considered that it comes after the above. In actual definition, there may be cases where irreversible processing such as making less than 0 0 is included, so if it is replaced with processing that excludes these, the user input LUT data 501 can be considered to have been calculated in the above order. If the C of the PQ → HLG and PQ → BT.709 conversions is the same as above, images to which the respective user input LUT data 501 has been applied will look almost the same when viewed in a viewing environment suitable for each. This is because, when viewed in a suitable viewing environment, the display device changes the F HLG -1 (M HLG (x)), F 709 (M 709 (x)) are cancelled, and both C(M PQ -1 (F PQ (x))).

[0048] In this case, when interpolation is performed with the degree of influence set to α and the conversion destination format canceled, and then conversion to the conversion destination format is applied, the influence degree changed LUT data 504 is as follows: For PQ to HLG conversion F HLG -1 (M HLG (αC(M PQ -1 (F PQ (x)))+(1-α)(M PQ -1 (F PQ (x))))) When converting from PQ to BT.709 F 709 -1 (M 709 (αC(M PQ -1 (F PQ (x)))+(1-α)(M PQ -1 (F PQ (x)))))

[0049] Both are αC(M PQ -1 (F PQ (x)))+(1-α)(M PQ -1 (F PQ (x)), even if the strength of the LUT conversion is adjusted using an arbitrary influence level α, adjustment can be made so that the image appears almost the same regardless of the destination format.

[0050] In S36, the system control unit 50 applies the influence-changed LUT data 504 generated in S35 to the LUT processing unit 242, as in the first embodiment, and causes the LUT processing unit 242 to perform LUT conversion on the image data supplied from the basic image processing unit 241.

[0051] In the second embodiment, an influence-changing LUT is generated by performing an interpolation between LUTs without performing a conversion process to a conversion destination format, and then applying the conversion process to the conversion destination format again to generate an influence-changing LUT, and then applying the LUT. According to the second embodiment, it is possible to provide a user with a consistent influence-changing function that produces a similar appearance when the influence is the same for multiple user-input LUTs that perform conversion to the same appearance in different conversion destination formats.

[0052] [Third embodiment] In the first embodiment, the influence degree change is performed by performing interpolation between LUTs. However, there is also a method of realizing the change by performing interpolation between images. This method will be described as the third embodiment with reference to the block diagram of FIG.

[0053] The A / D converter 23 in FIG. 5 is the same as the A / D converter 23 in FIG. 2, and therefore description thereof will be omitted. The image processing unit 24 in FIG. 5 includes a format conversion processing unit 243. The format conversion processing unit 243 performs format conversion processing on the image input from the basic image processing unit 241 based on the format conversion information 505. As described in the first embodiment, if the source format and the destination format are known, conversion from the source format to the destination format can be performed by calculation. In the first embodiment, the format conversion LUT data 502 is generated by the calculation, but it is also possible for the format conversion processing unit 243 to convert the image itself input from the basic image processing unit 241. In addition, the LUT processing unit 242 is a block that performs conversion processing using an LUT on the image input from the basic image processing unit 241 as in FIG. 2, but in the third embodiment, unlike FIG. 2, the user input LUT data 501 is applied as it is. Then, the interpolation processing unit 244 synthesizes the images input from the format conversion processing unit 243 and the LUT processing unit 242 by interpolation based on the influence degree 503. With this configuration, the format-converted image output from the format conversion processing unit 243 and the LUT-applied image output from the LUT processing unit 242 are synthesized by interpolation in the interpolation processing unit 244. As a result, an image almost similar to that of the first embodiment can be obtained from the image processing unit 24. Compared to the first embodiment, the format conversion is not processed by the LUT but is calculated more accurately, so that image breakdown can be suppressed even when the degree of influence is small and the number of grids of the LUT is small.

[0054] In the first and second embodiments, the format conversion LUT data 502 is generated based only on the format conversion information, but it may also include other conversion processes. For example, when converting an image conforming to the PQ of ITU-R BT.2100 into an image conforming to ITU-R BT.709, a simple conversion often results in blown-out highlights and color saturation. Since such blown-out highlights and color saturation often impair the quality of the image, it is possible to include a process for mitigating or reducing blown-out highlights and color saturation in the format conversion LUT data 502. For example, it is possible to include a gradation compression process for blown-out highlights. Also, it is possible to include a process called gamut mapping for mapping colors outside the color gamut to colors within the color gamut for color saturation. By including these processes, the conversion becomes different from pure format conversion, but as long as the majority of the processes do not deviate significantly from pure format conversion, it is possible to improve the quality of the image obtained without giving a large sense of incongruity to the user. It is possible to include other processes in addition to the gradation compression process and gamut mapping.

[0055] In addition, in the first and second embodiments, the format conversion LUT data 502 is generated by a calculation based on the format conversion information. However, the format conversion information and the format conversion LUT data 502 may be linked together and stored in the non-volatile memory 56 in advance, and the linked data may be retrieved and used.

[0056] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0057] The disclosure of this specification includes the following image processing device, method, and program. (Item 1) A first acquisition means for acquiring a first LUT for applying the effect of a predetermined format conversion and a target color conversion; a second acquisition means for acquiring a second LUT that applies an effect of the predetermined format conversion from among effects of the first LUT; A setting means for setting an influence degree of the LUT to be applied to the first image to be processed; a synthesis means for synthesizing the first LUT and the second LUT based on the degree of influence to generate a third LUT; a processing means for performing color conversion on the first image by using the third LUT; 13. An image processing device comprising: (Item 2) The second LUT is an LUT for applying an effect of a conversion process different from the predetermined format conversion in addition to the effect of the predetermined format conversion. 2. The image processing device according to item 1, (Item 3) The predetermined format conversion is configured to include a first partial format conversion and a second partial format conversion; The synthesis means is generating a fourth LUT by removing the second partial format conversion from the first LUT, and a fifth LUT by removing the second partial format conversion from the second LUT; combining the fourth LUT and the fifth LUT based on the degree of influence; generating the third LUT by applying the second partial format conversion after the composition; 3. The image processing device according to item 1 or 2. (Item 4) The predetermined format conversion is specified by input from a user. 4. The image processing device according to any one of items 1 to 3, (Item 5) The predetermined format conversion is specified from information associated with the first LUT. 4. The image processing device according to any one of items 1 to 3, (Item 6) An acquisition means for acquiring a first LUT for applying the effect of a predetermined format conversion and a target color conversion; A determination means for determining an effect of the predetermined format conversion from among effects of the first LUT; A setting means for setting an influence degree of the LUT to be applied to the first image to be processed; a processing means for generating a second image from the first image based on the degree of influence, the first LUT, and the predetermined format conversion; An image processing device comprising: (Item 7) The processing means includes: The second image is generated by synthesizing a third image obtained by applying the first LUT to the first image and a fourth image obtained by applying the effect of the predetermined format conversion to the first image based on the degree of influence. 7. The image processing device according to item 6, (Item 8) The predetermined format conversion is configured to include a first partial format conversion and a second partial format conversion; The processing means includes: generating a third image obtained by applying a third LUT obtained by removing the second partial format conversion from the first LUT to the first image, and a fourth image obtained by applying a third partial format conversion obtained by removing the second partial format conversion from the predetermined format conversion to the first image; generating a fifth image by combining the third image and the fourth image based on the degree of influence; applying the second partial format conversion to the fifth image to generate the second image. 7. The image processing device according to item 6, (Item 9) 7. The image processing device according to item 1 or 6, further comprising an imaging means for acquiring the first image. (Item 10) A first acquisition step of acquiring a first LUT for applying the effect of a predetermined format conversion and a target color conversion; a second acquisition step of acquiring a second LUT that applies an effect of the predetermined format conversion from among effects of the first LUT; A setting step of setting an influence degree of the LUT to be applied to the first image to be processed; a combining step of combining the first LUT and the second LUT based on the degree of influence to generate a third LUT; a processing step of performing color conversion on the first image using the third LUT; 13. An image processing method comprising: (Item 11) obtaining a first LUT for applying the effect of a predetermined format conversion and a target color conversion; identifying an effect of the predetermined format conversion from among the effects of the first LUT; A setting step of setting an influence degree of the LUT to be applied to the first image to be processed; a processing step of generating a second image from the first image based on the degree of influence, the first LUT, and the predetermined format conversion; An image processing method comprising the steps of: (Item 12) 12. A program for causing a computer to execute each step of the method according to item 10 or 11 by reading and executing the program on the computer.

[0058] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0059] 22...imaging section, 23...A / D converter, 24...image processing section, 50...system control section, 52...system memory, 56...non-volatile memory, 60...mode change switch, 70...operation section, 100...imaging device, 241...basic image processing section, 242...LUT processing section, 243...format conversion processing section, 244...interpolation processing section, 501...user input LUT data, 502...format conversion LUT, 503...influence level, 504...influence level changed LUT data, 505...format conversion information

Claims

1. a first obtaining means for obtaining a first look-up table (LUT) for applying a predetermined format conversion and a target color conversion effect; a second acquisition means for acquiring a second LUT that applies the effect of the predetermined format conversion from among the effects of the first LUT; a setting means for setting the influence of the LUT to be applied to the first image to be processed; a combining means for combining the first LUT and the second LUT based on the degree of influence to generate a third LUT; a processing means for performing color conversion on the first image using the third LUT; 1. An image processing device comprising:

2. The second LUT is an LUT for applying the effect of a conversion process different from the predetermined format conversion in addition to the effect of the predetermined format conversion.

2. The image processing device according to claim 1, wherein:

3. the predetermined format conversion is configured to include a first partial format conversion and a second partial format conversion; The synthesis means is generating a fourth LUT by removing the second partial format conversion from the first LUT, and a fifth LUT by removing the second partial format conversion from the second LUT; combining the fourth LUT and the fifth LUT based on the degree of influence; generating the third LUT by applying the second partial format conversion after the combination; 2. The image processing device according to claim 1, wherein:

4. The predetermined format conversion is specified by input from a user.

2. The image processing device according to claim 1, wherein:

5. The predetermined format conversion is specified from information accompanying the first LUT.

2. The image processing device according to claim 1, wherein:

6. obtaining means for obtaining a first look-up table (LUT) for applying a predetermined format conversion and a target color conversion effect; a specifying means for specifying the effect of the predetermined format conversion from among the effects of the first LUT; a setting means for setting the influence of the LUT to be applied to the first image to be processed; a processing means for generating a second image from the first image based on the degree of influence, the first LUT, and the predetermined format conversion; 1. An image processing device comprising:

7. The processing means The second image is generated by combining a third image obtained by applying the first LUT to the first image and a fourth image obtained by applying the effect of the predetermined format conversion to the first image based on the degree of influence.

7. The image processing device according to claim 6,

8. The predetermined format conversion is configured to include a first partial format conversion and a second partial format conversion; The processing means generating a third image obtained by applying a third LUT obtained by removing the second partial format conversion from the first LUT to the first image, and a fourth image obtained by applying a third partial format conversion obtained by removing the second partial format conversion from the predetermined format conversion to the first image; generating a fifth image by combining the third image and the fourth image based on the degree of influence; generating the second image by applying the second partial format conversion to the fifth image; 7. The image processing device according to claim 6,

9. 10. The image processing apparatus according to claim 1, further comprising an image capturing means for capturing the first image.

10. a first obtaining step of obtaining a first look-up table (LUT) for applying a predetermined format conversion and a target color conversion effect; a second acquisition step of acquiring a second LUT that applies the effect of the predetermined format conversion from among the effects of the first LUT; a setting step of setting an influence degree of the LUT to be applied to the first image to be processed; a combining step of combining the first LUT and the second LUT based on the degree of influence to generate a third LUT; a processing step of performing color conversion on the first image using the third LUT; An image processing method comprising:

11. obtaining a first look-up table (LUT) for applying a predetermined format conversion and a target color conversion effect; a specifying step of specifying an effect of the predetermined format conversion from among the effects of the first LUT; a setting step of setting an influence degree of the LUT to be applied to the first image to be processed; a processing step of generating a second image from the first image based on the degree of influence, the first LUT, and the predetermined format conversion; An image processing method comprising:

12. A program that, when read and executed by a computer, causes the computer to execute each step of the image processing method according to claim 10 or 11.

Images