Lens device, image processing device, imaging device, lens device control method, image processing method, and program

Abstract

To provide a lens apparatus capable of reducing effects on hues of images even when the lens apparatuses attached to the image pickup apparatus are the same type.SOLUTION: A lens apparatus (100) detachably attachable to an image pickup apparatus (10) includes: notification means (213) configured to notify the image pickup apparatus of a type of the lens apparatus, and communication means (211) configured to communicate with the image pickup apparatus. Via communication with the image pickup apparatus, the communication means transmits setting information for setting a white detection area different from a white detection area that corresponds to the type of the lens apparatus and is stored in the image pickup apparatus.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an image processing device that reduces differences in color of captured images caused by the optical characteristics of an interchangeable lens. [Background technology]

[0002] It has been known that the color characteristics of interchangeable lenses differ, resulting in different colors in captured images depending on the type of interchangeable lens attached to a camera body. Patent Document 1 discloses an image processing device that reduces the difference in color by switching the calculation method of the white balance adjustment value depending on the type of interchangeable lens. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6041561 Summary of the Invention [Problem to be solved by the invention]

[0004] The image processing device disclosed in Patent Document 1 executes the same calculation method when it detects that the attached interchangeable lenses are of the same type. However, even if the interchangeable lenses are detected as being of the same type, the color characteristics of the interchangeable lenses may differ and affect the color of the image.

[0005] Therefore, an object of the present invention is to provide a lens device, an image processing device, an imaging device, a control method for the lens device, an image processing method, and a program that can reduce the impact on the color of an image even if the lens device attached to the imaging device is the same type of lens device. [Means for solving the problem]

[0006] A lens device according to one aspect of the present invention is a lens device that is detachable from an imaging device, and includes a notification unit that notifies the imaging device of a type of the lens device, and a communication unit that communicates with the imaging device, and the communication unit transmits setting information for setting a white detection range that is different from a white detection range stored in the imaging device in correspondence with the type of the lens device by communicating with the imaging device. When the lens device attached to the imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type. do.

[0007] According to another aspect of the present invention, there is provided an image processing device comprising: a determination unit that determines a type of lens device used to capture an image from among a plurality of types of lens devices having different color characteristics; an image processing unit that applies white balance adjustment to the image; and a communication unit that communicates with the lens device. When the image processing unit receives setting information for setting a white detection range different from a white detection range corresponding to the type of the lens device by communication with the lens device via the communication unit, the image processing unit sets a white detection range based on the setting information and applies the white balance adjustment. When the lens device attached to the imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type. do.

[0008] Other objects and features of the present invention will be described in the following embodiments. Effect of the Invention

[0009] According to the present invention, it is possible to provide a lens device, an image processing device, an imaging device, a control method for the lens device, an image processing method, and a program that can reduce the impact on the color of an image even if the lens device attached to the imaging device is the same type of lens device. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of a camera system according to an embodiment of the present invention. [Diagram 2] 5A and 5B are diagrams illustrating examples of color characteristics of an interchangeable lens in the present embodiment. [Diagram 3] FIG. 2 is a block diagram of a camera control unit and a lens control unit in the present embodiment. [Figure 4]5 is an explanatory diagram of a lens type determination unit in the present embodiment. FIG. [Diagram 5] 5 is a flowchart showing a lens type determination process in the present embodiment. [Figure 6] 11 is a table showing the relationship between lens type, white balance ID, and white balance selected by a camera body in this embodiment. [Figure 7] 5 is a flowchart showing lens communication of an interchangeable lens and development processing of an image processing unit in the present embodiment. [Figure 8] FIG. 11 is a diagram showing an example of a warning display in the present embodiment. [Figure 9] 5 is a diagram showing an example of a data structure of white balance coefficient data in the present embodiment. FIG. [Figure 10] 3 is a diagram illustrating a 3D-LUT in the present embodiment in a color space. FIG. [Figure 11] 3A to 3C are explanatory diagrams of a tetrahedral interpolation method as an example of a three-dimensional interpolation calculation in the present embodiment. [Figure 12] 10 is a flowchart showing another example of the development process of the image processing unit in the present embodiment. [Figure 13] 5A to 5C are diagrams illustrating an example of a data structure of a 3D-LUT according to a lens type in the present embodiment. [Figure 14] 10 is a flowchart showing yet another example of the development process of the image processing unit in the present embodiment. [Figure 15] 7A to 7C are diagrams illustrating examples of setting a white detection range according to a lens type in the present embodiment. [Figure 16] 5 is a flowchart illustrating an example of a manual white balance registration process and a manual white balance application process in the present embodiment. [Figure 17] FIG. 4 is a diagram showing an example of a method for classifying lens types from color characteristics according to the spectral transmittance of the lens in this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0012] Imaging devices such as digital cameras generate captured images by performing image processing such as white balance adjustment and color processing on RAW images acquired by image sensors such as CMOS sensors. Generally, white balance adjustment is a process of adjusting the levels of the R, G, and B signal values ​​acquired from the image sensor, and color processing is a color conversion process using color density and color tone adjustment and 3D-LUT (three-dimensional lookup table). White balance adjustment and color processing are processes for properly expressing the color of the subject without causing color cast or deterioration of color reproducibility of the captured image due to the influence of the light source and the spectral characteristics of the image sensor. The color of the captured image changes depending on the spectral transmittance of the lens. In particular, in the case of digital single-lens reflex cameras that allow lens replacement, there are multiple interchangeable lenses that can be attached to the camera body, and the spectral transmittance may differ depending on the type of interchangeable lens.

[0013] Here, the color characteristics of an interchangeable lens will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of the color characteristics of an interchangeable lens. FIG. 2 shows the color characteristics of a lens calculated from the spectral distribution of a light source under a certain light source and the spectral transmittance of the lens. In FIG. 2, lens type 1 (first type) is an interchangeable lens (first lens device) that can be attached to a mirrorless camera, and lens type 2 (second type) is an interchangeable lens (second lens device) that can be attached to a single-lens reflex camera. There are various types of lenses classified as lens type 1, and the color characteristics of each are distributed within a distribution area centered on point a in the figure. As with lens type 1, there are also various types of interchangeable lenses classified as lens type 2, and the color characteristics of each are distributed within a distribution area centered on point b in the figure.

[0014] As interchangeable lenses become more diverse and have a wide variety of color characteristics, a digital camera (camera body) that can attach and take pictures with all of these interchangeable lenses is conceivable. Even if the mount shapes of lens type 1 and lens type 2 are different (different types of interchangeable lenses), various interchangeable lenses can be attached by using an adapter that has a structure that converts the difference in mounts. In the case of such a camera body, the color characteristics of the attached interchangeable lenses differ, so the color of the image varies depending on the interchangeable lens used, and so it is necessary to reduce the difference in color of the image that differs depending on the interchangeable lens.

[0015] Furthermore, even if the interchangeable lenses have the same mount shape, i.e., are the same type (for example, lens type 1), it may be difficult to reduce size and cost if the camera body selects the white balance for lens type 1. For this reason, it is necessary to configure the camera body so that it can select the white balance for lens type 2 even if it is lens type 1.

[0016] First, a camera system (imaging system) 300 in this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram of the camera system 300. The camera system 300 is configured with a camera body (imaging device) 10 and an interchangeable lens (lens device) 100 that is detachable from the camera body 10. The camera body 10 and the interchangeable lens 100 each have a mount 1 that is equipped with electrical contacts for supplying power from the camera body 10 to the interchangeable lens 100 and for mutual communication.

[0017] The camera body 10 has an image sensor (imaging element) 11 such as a CMOS sensor or a CCD sensor. The image sensor 11 photoelectrically converts an optical image (subject image) formed by a lens (imaging optical system) 101 in the interchangeable lens 100 to output an electrical signal. The camera body 10 also has an A / D conversion unit 12 that converts an analog electrical signal output from the image sensor 11 into a digital signal, and an image processing unit 13 that performs various image processes on the digital signal output from the A / D conversion unit 12 to generate an image.

[0018] The image processing unit 13 functions as an image processing means for applying white balance adjustment to a captured image, and has a white balance processing unit 130, a color processing unit 131, and an other image processing unit 132. The white balance processing unit 130 applies gain to each of the red (R), green (G), and blue (B) signals constituting the input image to the image processing unit 13 to adjust the color ratio of R, G, and B. The color processing unit 131 adjusts color density and hue, and performs color conversion processing such as 3D-LUT. The other image processing unit 132 performs development processing such as pixel interpolation processing, brightness adjustment processing, and gamma processing. The image generated by the image processing unit 13 is displayed on the display unit 14 and recorded on the recording medium 15.

[0019] The camera body 10 also has a memory (storage unit) 16. The memory 16 functions as a buffer when processing images, and stores operation programs used by a camera control unit 18, which will be described later. The memory 16 also stores various data, which will be described later, used by a white balance processing unit 130 and a color processing unit 131. The camera body 10 also has an operation input unit 17 that includes a power switch for turning the power on and off, a shooting switch for starting image recording, and a selection / setting switch for setting various menus. The camera control unit 18 includes a microcomputer, and controls the image processing unit 13 and communication with the interchangeable lens 100 in response to signals from the operation input unit 17.

[0020] On the other hand, the interchangeable lens 100 has a lens 101, a lens driving unit 102, and a lens control unit 103. The lens driving unit 102 drives an actuator that moves or operates a focus lens, a zoom lens, an aperture, an anti-vibration lens, and the like (not shown) included in the lens 101. The lens control unit 103 has a microcomputer, and controls the lens driving unit 102 in response to a control signal received by communication from the camera control unit 18. Note that in this embodiment, the interchangeable lens 100 that can be attached to the camera body 10 will be described as having two types of interchangeable lenses, but the present invention is not limited to this and may be compatible with three or more types of interchangeable lenses.

[0021] Next, the configurations of the camera control unit 18 and the lens control unit 103, which are electrically connected via the mount 1, will be described with reference to Fig. 3. Fig. 3 is a block diagram of the camera control unit 18 and the lens control unit 103.

[0022] The LCLK terminal (1-1) is a terminal for a communication clock signal output from the camera body 10 to the interchangeable lens 100. The DCL terminal (1-2) is a terminal for communication data output from the camera body 10 to the interchangeable lens 100. The DLC terminal (1-3) is a terminal for communication data output from the interchangeable lens 100 to the camera body 10. The MIF terminal (1-4) is a terminal for detecting that the interchangeable lens 100 has been attached to the camera body 10. The camera microcomputer 20, which is a microcomputer within the camera control unit 18, detects that the interchangeable lens 100 has been attached to the camera body 10 based on the voltage of the MIF terminal.

[0023] The TYPE terminal (1-5) is a terminal for detecting the type of the interchangeable lens 100 attached to the camera body 10. In this embodiment, the TYPE terminal (1-5) and the lens type determination unit 213 (resistance RL) constitute a notification means for notifying the camera body 10 of the type of the interchangeable lens 100. The TYPE terminal (1-5) outputs a voltage according to the resistance RL to the camera body 10. The camera microcomputer 20 detects the type of the interchangeable lens 100 attached to the camera body 10 based on the voltage of the TYPE terminal (1-5). The VBAT terminal (1-6) is a terminal for supplying a driving power supply (VM) used for various operations except communication control from the camera body 10 to the interchangeable lens 100. The VDD terminal (1-7) is a terminal for supplying a communication control power supply (VDD) used for communication control from the camera body 10 to the interchangeable lens 100. The DGND terminal (1-8) is a terminal for connecting the communication control systems of the camera body 10 and the interchangeable lens 100 to ground. The PGND terminals (1-9) are terminals for connecting a mechanical drive system including a motor and the like provided in the camera body 10 and the interchangeable lens 100 to ground.

[0024] A plurality of types of interchangeable lenses 100, each of which has a different communication voltage with the camera body 10, are selectively attached to the camera body 10 of this embodiment. In the following explanation, a case will be explained in which the types of interchangeable lenses 100 identified by the camera body 10 based on the voltage of the TYPE terminal include a first lens device (lens type 1) and a second lens device (lens type 2) that has a different communication voltage from the first lens device.

[0025] The camera power supply unit 21 provided in the camera control unit 18 converts the battery voltage supplied from a battery (not shown) mounted on the camera body 10 into a voltage required for the operation of each circuit. At this time, the camera power supply unit 21 generates voltages V1, V2, V3, and VM. The first voltage V1 is a power supply voltage as a communication control power supply (VDD) for the first and second lens devices, and is also a communication voltage for the first lens device. The second voltage V2 is a communication voltage for the second lens device. The third voltage V3 is a power supply voltage as an operating power supply for the camera microcomputer 20. VM is a power supply voltage as a driving power supply for the first and second lens devices. When the power switch 22 is turned on, the camera microcomputer 20 starts supplying VDD and VM from the camera body 10 to the interchangeable lens 100. When the power switch 22 is turned off, the camera microcomputer 20 stops supplying VDD and VM from the camera body 10 to the interchangeable lens 100.

[0026] The camera microcomputer 20 communicates with the interchangeable lens 100 via a voltage conversion unit 23 serving as a voltage conversion means. The camera microcomputer 20 has an LCLK_OUT terminal that outputs a communication clock signal, a DCL_OUT terminal that outputs communication data to the interchangeable lens 100, and a DLC_IN terminal that receives input of communication data from the interchangeable lens 100. The communication clock signal and communication data correspond to communication signals. The camera microcomputer 20 functions as communication means (camera communication means) that communicates with the interchangeable lens 100.

[0027] The camera microcomputer 20 also has a MIF_IN terminal for detecting attachment of the interchangeable lens 100, a TYPE_IN terminal for identifying the type of the interchangeable lens 100, and a CNT_V_OUT terminal for outputting a communication voltage switching signal to the voltage conversion unit 23. The camera microcomputer 20 functions as a determination unit for determining the type of interchangeable lens used to capture an image from among a plurality of types of interchangeable lenses with different color characteristics. The camera microcomputer 20 also has a CNT_VDD_OUT terminal for outputting a power supply signal for the power switch 22, a connection terminal with the image processing unit 13, and a connection terminal with the operation input unit 17.

[0028] The lens microcomputer 211, which is a microcomputer in the lens control unit 103, communicates with the camera microcomputer 20 via the voltage conversion unit 23. The lens microcomputer 211 has an LCLK_IN terminal for inputting a communication clock signal, a DLC_OUT terminal for outputting communication data to the camera body 10, a DCL_IN terminal for receiving input of communication data from the camera body 10, and a connection terminal with the lens driving unit 102. The lens microcomputer 211 functions as a communication means (lens communication means) for communicating with the camera body 10. The lens control unit 103 also has a lens power supply unit 214 as a voltage generation means.

[0029] Next, detection of attachment of the interchangeable lens 100 to the camera body 10 will be described. The MIF_IN terminal of the camera microcomputer 20 is pulled up to the power supply by resistor R2 (100 KΩ), so its voltage value is H (High) when no lens is attached. However, when an interchangeable lens (first and second lens devices) 100 is attached, the MIF_IN terminal is connected to GND in the interchangeable lens 100, so its voltage value becomes L (Low) when the interchangeable lens 100 is attached regardless of the type of interchangeable lens 100.

[0030] Next, a configuration example of the lens type determination unit 213 as a determination means will be described with reference to Fig. 4. Fig. 4 is an explanatory diagram of the lens type determination unit 213. The lens type determination unit 213 has a resistor RL provided between a TYPE terminal (1-5) provided on the mount 1 and GND. A value according to the type of interchangeable lens is set as the resistance value of the resistor RL in advance. For example, the resistor RL provided in the first lens device shown in Fig. 4(a) is set to 2.4 kΩ, and the resistor RL provided in the second lens device shown in Fig. 4(b) is set to 27 kΩ.

[0031] In the camera body 10, a resistor R1 (for example, 10 KΩ) is connected as a camera-side resistor between the TYPE terminal of the mount 1 and the voltage (V3) of the operating power supply of the camera microcomputer 20, and the TYPE terminal is further connected to the TYPE_IN terminal of the camera microcomputer 20. The TYPE_IN terminal of the camera microcomputer 20 has an A / D conversion function (here, a 10-bit A / D conversion function).

[0032] Next, an operation of determining the type of interchangeable lens by the camera microcomputer 20 will be described. The camera microcomputer 20 determines the type of the attached interchangeable lens according to the voltage value input to the TYPE_IN terminal. Specifically, the camera microcomputer 20 performs A / D conversion of the input voltage value and performs lens type determination by comparing the A / D converted value with a lens type determination criterion that the camera microcomputer 20 has in advance.

[0033] For example, when the first lens device is attached, the A / D conversion value of the voltage input to the TYPE_IN terminal is determined to be approximately "0x00C6" by the resistance ratio RL / (R1+RL) of R1 (10KΩ) and RL (2.4KΩ). Therefore, the camera microcomputer 20 detects that the A / D conversion value of the TYPE_IN terminal is within the range of "0x0047 to 0x0145", which is the first lens type determination criterion, and determines that the attached interchangeable lens is the first lens device. On the other hand, when the second lens device is attached, the A / D conversion value of the voltage input to the TYPE_IN terminal is determined to be approximately "0x02EB" by the resistance ratio RL / (R1+RL) of R1 (10KΩ) and RL (27KΩ). Therefore, the camera microcomputer 20 detects that the A / D converted value of the TYPE_IN terminal is within the range of "0x026C to 0x036A", which is the second lens type determination criterion, and determines that the attached interchangeable lens is the second lens device.

[0034] As described above, the camera microcomputer 20 determines the type of the attached interchangeable lens 100 based on the voltage value (A / D converted value) input to the TYPE_IN terminal. Then, depending on the determination result of the type of the interchangeable lens 100, it controls the logic output from the CNT_V_OUT terminal. Specifically, if the camera microcomputer 20 determines that the attached interchangeable lens 100 is a first lens device based on the voltage value of the TYPE_IN terminal, the camera microcomputer 20 outputs H from the CNT_V_OUT terminal to control the communication voltage to V1. Also, if the camera microcomputer 20 determines that the attached interchangeable lens 100 is a second lens device based on the voltage value of the TYPE_IN terminal, the camera microcomputer 20 outputs L from the CNT_V_OUT terminal to control the communication voltage to V2.

[0035] When the camera microcomputer 20 detects a voltage value (A / D conversion value) of the TYPE_IN terminal that is outside the range of the first and second lens type determination criteria, it determines that an "incompatible lens" that is an interchangeable lens that is not compatible with the camera body 10 is attached. Alternatively, it reserves the determination because it is not possible to perform lens type determination normally. In these cases, the camera microcomputer 20 does not communicate with the interchangeable lens 100.

[0036] Next, the process of determining the type of the interchangeable lens 100 (lens type determination process) will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the lens type determination process. Each step in Fig. 5 is mainly executed by the camera microcomputer 20 in accordance with a computer program stored in the memory 16.

[0037] First, in step S60, the camera microcomputer 20 reads a voltage value H or L from the MIF_IN terminal. Then, in step S61, the camera microcomputer 20 reads a voltage value from the TYPE_IN terminal. Note that steps S60 and S61 do not have to be performed in this order, and for example, steps S60 and S61 may be performed simultaneously.

[0038] Next, in step S62, the camera microcomputer 20 determines the type of the attached interchangeable lens 100 based on the voltage value of the TYPE_IN terminal as described above. If the camera microcomputer 20 determines that the attached interchangeable lens 100 is the first lens device (lens type 1), it outputs H from the CNT_V_OUT terminal in step S63 to set the communication voltage to V1, and proceeds to step S63A. On the other hand, if the camera microcomputer 20 determines that the attached interchangeable lens 100 is the second lens device (lens type 2) in step S62, it outputs L from the CNT_V_OUT terminal in step S64 to set the communication voltage to V2, and proceeds to step S64A. Also, if the camera microcomputer 20 determines that the attached interchangeable lens 100 is neither the first nor the second lens device in step S62, it proceeds to step S65A for processing corresponding to "non-compatible lens" or reserved in step S65.

[0039] In steps S63A and S64A, the camera microcomputer 20 starts communication with the interchangeable lens 100 at the set communication voltage. In step S65A, the camera microcomputer 20 does not start communication with the attached interchangeable lens 100, and performs processing such as a warning to the user.

[0040] Next, in step S66, the camera microcomputer 20 determines whether or not a power-off interrupt has occurred due to an OFF operation of the power switch. If a power-off interrupt has occurred, the process proceeds to step S67, where the camera microcomputer 20 performs a power-off process. On the other hand, if a power-off interrupt has not occurred in step S66, the process proceeds to step S68. In step S68, the camera microcomputer 20 determines whether or not H has been input from the MIF_IN terminal, that is, whether or not the interchangeable lens 100 has been removed from the camera body 10. If H has been input from the MIF_IN terminal, the camera microcomputer 20 proceeds to step S69, where it stops communication with the interchangeable lens 100, and returns to step S60. On the other hand, if H has not been input from the MIF_IN terminal, the camera microcomputer 20 returns to step S66. By performing such communication processing, the camera control unit 18 can determine the type of the attached interchangeable lens 100.

[0041] In this embodiment, the method of determining the type of the attached interchangeable lens 100 has been described as being based on the voltage value input to the TYPE_IN terminal, but this is not limiting and other determination methods are also applicable. For example, it is also possible to determine the lens type by receiving information on the lens type from the interchangeable lens 100 immediately after step S63A, without performing the lens type determination and setting of the output voltage according to the lens type in steps S61 to S65.

[0042] Next, a process for reducing differences in color of a captured image due to the characteristics (color characteristics) of the interchangeable lens 100 in the development process by the image processing unit 13 will be described with reference to Fig. 6. Fig. 6 is a table showing the relationship between the lens type, white balance ID, and white balance selected by the camera body 10 in the development process by the image processing unit 13.

[0043] Conventionally, if the lens type is 1, the white balance selected by the camera body is also for lens type 1, and if the lens type is 2, the white balance selected by the camera body is also for lens type 2. However, if the camera body selects the white balance for lens type 1, it may be difficult to reduce the size and cost. Therefore, in this embodiment, an ID (white balance ID: WBID) is prepared to select the white balance of lens type 1 in order to allow the camera body to select the white balance of lens type 2 even though the lens type is 1. When WBID is 0, it indicates invalidity, and the same white balance as the lens type is selected. On the other hand, when the lens type is 1 and WBID=1, the white balance selected by the camera body is lens type 2. In addition, by providing another ID number such as WBID=2, it is also possible to select the white balance of lens type 3, which is different from lens type 1 and lens type 2.

[0044] Next, lens communication of the interchangeable lens 100 (lens control unit 103) and development processing of the image processing unit 13 will be described with reference to Figs. 7(a) and (b). Fig. 7(a) is a flowchart showing lens communication of the interchangeable lens 100. First, in step S90, lens communication is performed between the lens control unit 103 of the interchangeable lens 100 and the camera control unit 18 of the connected camera body 10. In the lens communication, the camera control unit 18 transmits information of the camera body 10 to the interchangeable lens 100, and the lens control unit 103 transmits information of the interchangeable lens 100 to the camera body 10. In a series of processes of lens communication, in step S91, the interchangeable lens 100 transmits WBID information (white balance ID information) to the camera body 10.

[0045] 7(b) is a flowchart showing the development process of the image processing unit 13. First, in step S100, the camera control unit 18 determines whether or not the lens type (the type of interchangeable lens attached to the camera body 10) is lens type 1. If the lens type is lens type 2, the process proceeds to step S101, where the image processing unit 13 performs white balance processing for lens type 2.

[0046] On the other hand, if the lens type is lens type 1 in step S100, the process proceeds to step S102, where the camera control unit 18 determines whether or not the WBID received from the interchangeable lens 100 is "1". If the WBID is "0", the process proceeds to step S103, where the image processing unit 13 performs white balance processing for lens type 1. On the other hand, if the WBID is "1", the process proceeds to step S104, where the image processing unit 13 checks whether or not it is possible to switch to white balance processing for lens type 2. If it is possible to switch to white balance processing for lens type 2, the process proceeds to step S101. In step S101, the image processing unit 13 determines that the lens type of the attached interchangeable lens 100 is lens type 1, but performs white balance processing for lens type 2. On the other hand, if it is impossible to switch to white balance processing for lens type 2, the process proceeds to step S105, where the image processing unit 13 displays a warning on the display unit 14.

[0047] Fig. 8 is a diagram showing an example of a warning display, showing a display that prompts the user to update the firmware of the camera body 10. The processes of steps S101 and S103 in Fig. 7(b) are processes performed by the white balance processing unit 130. Step S103 is a process of applying a white balance specialized for an interchangeable lens of lens type 1 to an image. Step S101 is a process of applying a white balance specialized for an interchangeable lens of lens type 2 to an image.

[0048] Generally, the camera body 10 stores white balance coefficient data corresponding to a plurality of light sources in advance, and the user can select the light source. Here, the white balance coefficient is a gain value applied to the R, G, and B signals that constitute an image. The R, G, and B white balance coefficients are WbGainR, WbGainG, and WbGainB, respectively, the R, G, and B values ​​before white balance adjustment are R0, G0, and B0, and the R, G, and B values ​​after white balance adjustment are R1, G1, and B1, respectively. In this case, the white balance adjustment process can be expressed as follows. White balance can be applied by performing the calculations according to the following equations (1) to (3) for all pixels in the image.

[0049] R1 = WbGainR × R0 (1) G1 = WbGainG × G0 (2) B1 = WbGainB × B0 (3) FIG. 9 is a diagram showing an example of the data structure of the white balance coefficient data stored in the memory 16. The memory 16 holds white balance coefficients corresponding to light sources such as sunlight, shade, and cloudy for each lens type. That is, when performing white balance application processing specialized for a lens of lens type 1, the white balance coefficients stored in the area for lens type 1 in FIG. 9 are used. Also, when performing white balance application processing specialized for a lens of lens type 2 in FIG. 9, the white balance coefficients stored in the area for lens type 2 are used. The white balance coefficients for lens type 1 and lens type 2 may be experimentally obtained in advance as optimal values ​​when using each lens. By switching the white balance coefficients to be used according to the lens type in this way, it is possible to reduce color casts in an image caused by the influence of color characteristics that differ for each lens type.

[0050] 7B, the color processing unit 131 adjusts the color density and hue of the image. Then, in step S107, the color processing unit 131 performs color space conversion processing using a 3D-LUT (three-dimensional lookup table) to reproduce the desired colors. The 3D-LUT is a table that defines the output R, G, and B values ​​for the input R, G, and B values, and is stored in the memory 16.

[0051] Here, we will explain a color space conversion method that uses a 3D-LUT and three-dimensional interpolation calculation in combination. An input digital image signal (R, G, B) is separated into upper bits and lower bits. The upper bits are used to extract multiple reference values ​​required for the interpolation calculation using the 3D-LUT. The lower bits are used as a weighting coefficient g for the interpolation calculation. Then, an interpolated value is calculated by a product-sum operation of the weighting coefficient g and the reference value extracted from the 3D-LUT.

[0052] Fig. 10 is a schematic diagram showing a state in which a three-dimensional input color space (RGB space) is divided into unit solids by dividing it into a limited number of vertices in each axis direction. The color data after color space conversion at the vertices of the unit solids is stored in the 3D-LUT as reference values. The upper bits of the three digital image signals are considered as the coordinates of the color space, and a unit solid to be used for the interpolation calculation is selected (for example, unit solid 1000 shown in Fig. 10), and the reference values ​​corresponding to the vertices of the unit solid are used for the interpolation calculation.

[0053] The 3D interpolation method used in this case is tetrahedral interpolation. Fig. 11 is an explanatory diagram of tetrahedral interpolation. Tetrahedral interpolation is performed as shown in Fig. 11(b) to (g). That is, a unit solid (for example, unit solid 1000 shown in Fig. 10) is divided into six tetrahedrons (hereinafter, the six divided tetrahedrons are referred to as Types 0 to 5, respectively), and the following formula is used to perform an interpolation calculation depending on which tetrahedron the input coordinates belong to.

[0054] In the following formula, the reference values ​​corresponding to the vertices of the unit solid shown in Fig. 11(a) are P0 to P7, respectively, and the weighting coefficient g is ΔR, ΔG, and ΔB. Also, which tetrahedron is selected from among the tetrahedrons of Type 0 to Type 5 is determined by the magnitude relationship of these weighting coefficients ΔR, ΔG, and ΔB.

[0055] In the case of Type 0 (ΔR>ΔG>ΔB) X=P0+(P1-P0)×ΔR+(P3-P0)×ΔG+(P7-P0)×ΔB...(4) In the case of Type 1 (ΔR>ΔB>ΔG) X=P0+(P1-P0)×ΔR+(P7-P0)×ΔG+(P5-P0)×ΔB (5) In the case of Type 2 (ΔG>ΔR>ΔB) X=P0+(P3-P0)×ΔR+(P2-P0)×ΔG+(P7-P0)×ΔB (6) In the case of Type 3 (ΔG>ΔB>ΔR) X=P0+(P7-P0)×ΔR+(P2-P0)×ΔG+(P6-P0)×ΔB (7) In the case of Type 4 (ΔB>ΔR>ΔG) X=P0+(P5-P0)×ΔR+(P7-P0)×ΔG+(P4-P0)×ΔB...(8) In the case of Type 5 (ΔB>ΔG>ΔR) X=P0+(P7-P0)×ΔR+(P6-P0)×ΔG+(P4-P0)×ΔB (9) By performing color space conversion processing using the 3D-LUT in step S104 in FIG. 7B, the development processing in the image processing unit 13 ends.

[0056] As another method of development processing performed by the image processing unit 13, as shown in Fig. 12, a 3D-LUT is stored for each lens type, and the 3D-LUT is switched depending on the lens used. Fig. 12(a) is a flowchart showing another example of development processing by the image processing unit. Note that in Fig. 12(a), for the sake of simplicity, the description of the warning display described in the flowchart of Fig. 7 is omitted. Steps S110 to S114 are the same as the processing flow described above.

[0057] In step S115, the camera control unit 18 determines the lens type. In step S116, the camera control unit 18 determines whether the WBID received from the interchangeable lens 100 is "1". In steps S117 and S118, the camera control unit 18 performs color space conversion processing using a 3D-LUT according to the lens type, based on the WBID information. The 3D-LUT used here is prepared in advance, taking into account differences in the color characteristics of the lenses, and by switching the 3D-LUT to be used according to the lens type, it becomes possible to match the color of the image in more detail even if the color characteristics of the lenses differ.

[0058] In this case, the data structure of the 3D-LUT held in memory 16 is a structure that holds one for lens type 1 and one for lens type 2, as shown in Fig. 13. Fig. 13 is a diagram showing an example of the data structure of a 3D-LUT according to the lens type. The 3D-LUT to be used is determined according to the lens type. Note that one block in the data structure shown in Fig. 13 corresponds to color data after color space conversion at the vertices of the unit solid shown in Fig. 10.

[0059] Also, by reflecting in advance in the 3D-LUT the color cast of the image due to the difference in the color characteristics of the lens, which is reduced by the white balance processing according to the lens type, the difference in the color characteristics of the lens can be absorbed by the 3D-LUT alone. In this case, the processing flow is as shown in FIG. 12(b). First, in step S120, the white balance processing unit 130 applies white balance processing that is independent of the lens type. Then, in step S121, the color processing unit 131 adjusts the color density and hue of the image. In the following steps S122 to S125, the camera control unit 18 uses the 3D-LUT according to the lens type.

[0060] As described above, by switching the white balance processing and 3D-LUT depending on the lens type, it is possible to reduce differences in image color caused by differences in the color characteristics of the lens.

[0061] Here, the switching of the white balance coefficient according to the lens type has been described as the white balance processing according to the lens type, but the conversion of the white balance coefficient can also be performed based on a conversion coefficient calculated in advance from the spectral transmittance of the lens. The processing flow in this case is as shown in Fig. 14. Fig. 14 is a flow chart showing yet another example of the development processing of the image processing unit 13. In Fig. 14, steps S131 to S133 are similar to the respective processes described above, and therefore their description will be omitted.

[0062] Prior to the process of applying white balance in step S131, in step S130, image processing unit 13 performs a conversion process of white balance coefficients according to the lens type. Camera body 10 holds only white balance coefficients corresponding to lens type 1, and holds adjustment values ​​AdjR, AdjG, and AdjB for converting the white balance coefficients of lens type 1 to white balance coefficients of lens type 2. This conversion is performed according to the following equations (10) to (12).

[0063] Here, WbGainR1, WbGainG1, and WbGainB1 are white balance coefficients corresponding to lens type 1 held by the camera, and WbGainR2, WbGainG2, and WbGainB2 are white balance coefficients after conversion.

[0064] WbGainR2=AdjR×WbGainR1 (10) WbGainG2=AdjG×WbGainG1 ···(11) WbGainB2=AdjB×WbGainB1 ···(12) This conversion coefficient may be set to a value that matches the color tone of the image after white balance is applied, based on the color characteristics due to the spectral transmittance of the lens.

[0065] By converting the white balance coefficients in this way, it becomes unnecessary to store the white balance coefficients for each lens type. If there are multiple lens types to be targeted, it is sufficient to store the conversion coefficients for each lens type.

[0066] In this embodiment, the white balance process is switched by using a white balance coefficient according to the light source at the time of shooting, but the present invention is not limited to this. For example, the process can be switched according to the lens type in the auto white balance process (AWB) that automatically determines the white balance coefficient from the image.

[0067] AWB is generally a process for reproducing white subjects in an image as white (matching RGB). Various methods have been proposed for AWB, one of which is to detect white subject areas in an image and determine the white balance coefficients to make those areas white. When detecting white subject areas in an image, a white detection area that is determined as white in color space is generally set, and the white subject area is detected based on whether it falls within that range. By switching this white detection area according to the lens type, appropriate AWB can be performed even if the spectral transmittance of the lens is different.

[0068] Here, an overview of AWB in this embodiment will be described with reference to Fig. 15. Fig. 15 is a diagram showing an example of setting a white detection range according to lens type. Fig. 15(a) shows a white detection area (first white detection area) of an interchangeable lens (first lens device) of lens type 1 (first type), and Fig. 15(b) shows a white detection area (second white detection area) of an interchangeable lens (second lens device) of lens type 2 (second type).

[0069] In FIG. 15(a), for each pixel constituting an input image before applying white balance, pixels whose UV values ​​obtained by converting RGB to YUV are included in area A or area B shown in the figure are treated as white subject areas. All pixels included in this white detection area are detected from within the image, and the white balance coefficient is calculated by averaging the RGB values ​​of the detected pixels. At that time, a weighted average with different weights for area A and area B is used. Specifically, the calculation is performed using the following equations (13) to (15). In equations (13) to (15), WbGainR, WbGainG, and WbGainB are the calculated white balance coefficients for RGB, and Ra, Ga, and Ba are the respective averages of the RGB values ​​of pixels belonging to area A of the white detection area. Also, Rb, Gb, and Bb are the respective averages of the RGB values ​​of pixels belonging to area B of the white detection area, and wA and wB are the weights for area A and area B.

[0070] WbGainR=(Ga×wA+Gb×wB) / (Ra×wA+Rb×wB) ···(13) WbGainG=1 (14) WbGainB=(Ga×wA+Gb×wB) / (Ba×wA+Bb×wB) ···(15) The white detection area used in such AWB processing is shown in FIG. 15(a) for lens type 1, and shown in FIG. 15(b) for lens type 2. The white detection area suitable for the lens characteristics can be experimentally obtained from images captured using that lens. The white detection area is determined so that the range of the subject included in the white detection area when captured using lens type 1 matches the range of the subject included in the white detection area when captured using lens type 2. By doing so, the subject area treated as a white subject is the same for lens type 1 and lens type 2, so the AWB calculation results match. If the white detection areas are not relatively the same for lens type 1 and lens type 2, the subject area used for AWB is different, so the AWB calculation results are different. Therefore, if the same white detection area is used when the color of the image differs depending on the lens, the results will not match. It is also clear from the above AWB principle that the results will not match even if the white balance coefficients calculated using the same white detection area are converted according to the spectral transmittance.

[0071] Such information regarding the white detection area according to the lens type may be stored as a parameter for each lens type, but it is also possible to store only the white detection area for one lens type as a parameter and calculate the white detection area for the other lens type using the conversion coefficient described above.

[0072] Next, a method for implementing the manual white balance function in this embodiment will be described with reference to Figs. 16(a) and (b). Figs. 16(a) and (b) are flowcharts showing examples of manual white balance registration processing and manual white balance application processing, respectively. The manual white balance function is a function in which a user photographs a subject area to be reproduced as white, registers a white balance coefficient determined from the image data in the camera body 10, and uses the white balance coefficient.

[0073] Once the white balance coefficients are registered in the camera body 10, they can be used in subsequent photographs, so if the lens used to photograph the image at the time of registration is different from the lens used to photograph the image at the time of use, it may not be possible to reproduce the appropriate white. Therefore, the camera body 10 of this embodiment performs the processing shown in Figures 16(a) and (b). Note that in Figures 16(a) and (b), in order to simplify the explanation, the description of the warning display in step S105 of Figure 7(b) is omitted.

[0074] 16(a) is a flowchart of the registration process of the white balance coefficients. First, in step S140, the camera control unit 18 calculates the white balance coefficients WbGainR, WbGainG, and WbGainB from the average values ​​R, G, and B of the RGB values ​​in the area near the center of the captured image according to the following equations (16) to (18).

[0075] WbGainR = G / R (16) WbGainG=1 (17) WbGainB=G / B (18) By using the RGB values ​​in the area near the center of the image in this manner, white balance coefficients are obtained that correspond to the subject area that should be reproduced as white, as photographed by the user.

[0076] Next, in step S141, the lens type used to capture the image from which the white balance coefficient was obtained is determined. If the lens type is lens type 1, the process proceeds to step S142. On the other hand, if the lens type is not lens type 1, the process proceeds to step S143. In step S142, the camera control unit 18 determines whether or not WBID is "1". If WBID is "1", the process proceeds to step S143. On the other hand, if WBID is not "1", the process proceeds to step S144.

[0077] In step S143, the camera control unit 18 performs a conversion process on the calculated white balance coefficients using conversion coefficients AdjR, AdjG, and AdjB according to the following equations (19) to (22), where WbGainR', WbGainG', and WbGainB' are white balance coefficients after conversion.

[0078] WbGainR'=WbGainR / AdjR (19) WbGainG'=WbGainG / AdjG ···(20) WbGainB'=WbGainB / AdjB (21) This conversion coefficient may be set to a value that matches the color tone of the image after white balance is applied, based on the color characteristics due to the spectral transmittance of the lens.

[0079] Then, in step S144, the camera control unit 18 stores the above-mentioned white balance coefficients in the memory 16, thereby registering them in the camera body 10.

[0080] 16(b) is a flowchart showing a process for applying white balance to an image using a registered white balance coefficient. First, in step S150, the camera control unit 18 reads out the manual white balance coefficient stored in the memory 16. Then, in step S151, the camera control unit 18 determines the lens type used to capture the image to which white balance is to be applied. If the lens type is lens type 1, the process proceeds to step S152. On the other hand, if the lens type is not lens type 1, the process proceeds to step S153.

[0081] In step S152, the camera control unit 18 determines whether or not WBID is "1". If WBID is "1", the process proceeds to step S153. On the other hand, if WBID is not "1", the process proceeds to step S154. If the lens type is 2 or WBID=1, the camera control unit 18 converts the white balance coefficients in step S153. The conversion coefficients AdjR, AdjG, and AdjB used in this case are set to the same values ​​as the conversion coefficients used when registering the white balance described above.

[0082] WbGainR=AdjR×WbGainR' (22) WbGainG=AdjG×WbGainG' (23) WbGainB=AdjB×WbGainB' (24) In other words, since the registered manual white balance coefficient is a value corresponding to lens type 1, the value is converted if the image was taken with lens type 2. In step S154, processing for applying the white balance coefficient is performed. In this way, by converting the white balance coefficient according to the lens type when registering and using manual white balance, it is possible to realize white balance processing that reproduces white appropriately regardless of the lens type.

[0083] In this embodiment, the two types of lens type 1 and lens type 2 have been described, but the same mechanism can be applied to three or more lens types. The method described in this embodiment is a method of switching parameters or converting coefficients depending on the lens type, so even if the number of target lens types increases, it can be easily handled by storing parameters for each lens type or storing conversion coefficients. Lens types may be classified according to the lens mount shape, or may be classified according to color characteristics corresponding to the spectral transmittance of the lens. FIG. 17 is a diagram showing an example of a method of classifying lens types according to color characteristics corresponding to the spectral transmittance of the lens. As shown in FIG. 17, it is also possible to classify lens types into groups of 1 to 5 according to the color characteristics of the lens. This makes it possible to reduce the difference in color of an image caused by lenses showing various color characteristics.

[0084] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0085] As described above, the communication means (lens microcomputer 211) of the lens device (interchangeable lens 100) transmits setting information for setting a white detection range different from the white detection range stored in the imaging device corresponding to the type of the lens device through communication with the imaging device (camera body 10). Preferably, when the attached lens device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type.

[0086] In this embodiment, when the image processing means (image processing unit 13) of the imaging device receives setting information for setting a white detection range different from the white detection range corresponding to the type of lens device, the image processing means sets the white detection range based on the setting information and applies white balance adjustment. Preferably, the image processing means calculates a white balance coefficient for white balance adjustment based on a white area detected from an image using the white detection range based on the setting information.

[0087] In this embodiment, when the image processing means receives setting information for setting a white detection range that is different from the white detection range corresponding to the type of lens device, and is unable to set a white detection range based on the setting information, it issues a warning.

[0088] According to each embodiment, it is possible to provide a lens device, an image processing device, an imaging device, a control method for the lens device, an image processing method, and a program that can reduce the impact on the color of an image even if the lens device attached to the imaging device is the same type of lens device.

[0089] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0090] 100 Interchangeable lenses (lens devices) 211 Lens microcomputer (communication means) 213 Lens type determination unit (notification means) 1-5 TYPE terminal (notification means)

Claims

1. A lens device that is detachable from an imaging device, a notification means for notifying the imaging device of the type of the lens device; A communication means for communicating with the imaging device, the communication means transmits, by communication with the imaging device, setting information for setting a white detection range that is different from a white detection range stored in the imaging device in accordance with a type of the lens device; A lens device characterized in that, when the lens device attached to the imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type.

2. the notification means has a resistor and a terminal for the imaging device to detect the type of the lens device, 2. The lens device according to claim 1, wherein the terminal outputs a voltage corresponding to the resistance to the imaging device.

3. 3. The lens device according to claim 1, wherein the types of the lens device are classified according to a mount shape of the lens device.

4. A determination unit that determines the type of lens device used to capture an image from among a plurality of types of lens devices having different color characteristics; image processing means for applying a white balance adjustment to the image; a communication means for communicating with the lens device; when the image processing means receives setting information for setting a white detection range different from a white detection range corresponding to a type of the lens device through communication with the lens device via the communication means, the image processing means sets a white detection range based on the setting information and applies the white balance adjustment; An image processing device characterized in that, when the lens device attached to an imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type.

5. 5. The image processing apparatus according to claim 4, wherein the image processing means calculates a white balance coefficient for the white balance adjustment based on a white area detected from the image using the white detection range based on the setting information.

6. A determination unit that determines the type of lens device used to capture an image from among a plurality of types of lens devices having different color characteristics; image processing means for applying a white balance adjustment to the image; a communication means for communicating with the lens device; when the image processing means receives setting information for setting a white detection range different from a white detection range corresponding to a type of the lens device through communication with the lens device via the communication means, and when the image processing means is unable to set a white detection range based on the setting information, it issues a warning; An image processing device characterized in that, when the lens device attached to an imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type.

7. An imaging device having a detachable lens device, An imaging element; An imaging device comprising: an image processing device according to claim 4 .

8. A method for controlling a lens device that is detachable from an imaging device, comprising: a notification step of notifying the imaging device of the type of the lens device; and a communication step of communicating with the imaging device, In the communication step, by communication with the imaging device, setting information for setting a white detection range different from a white detection range stored in the imaging device corresponding to a type of the lens device is transmitted, A method for controlling a lens device, characterized in that, when the lens device attached to the imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type.

9. a determining step of determining the type of lens device used to capture the image from among a plurality of types of lens devices having different color characteristics; an image processing step of applying a white balance adjustment to the image; and a communication step of communicating with the lens device, When the communication step receives setting information for setting a white detection range different from a white detection range corresponding to a type of the lens device, the image processing step sets a white detection range based on the setting information and applies the white balance adjustment; an image processing method characterized in that, when the lens device attached to the imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type.

10. a determining step of determining the type of lens device used to capture the image from among a plurality of types of lens devices having different color characteristics; an image processing step of applying a white balance adjustment to the image; and a communication step of communicating with the lens device, When the communication step receives setting information for setting a white detection range different from the white detection range corresponding to the type of the lens device but the white detection range cannot be set based on the setting information, a warning is issued in the image processing step. an image processing method characterized in that, when the lens device attached to the imaging device is a first lens device of a first type, the setting information is a signal for switching from a first white detection range corresponding to the first lens device to a second white detection range corresponding to a second lens device of a second type.

11. A program for causing a computer to execute the method for controlling a lens apparatus according to claim 8.

12. A program for causing a computer to execute the image processing method according to claim 9 or 10.

Images