Image pickup apparatus, control method for image pickup apparatus, and storage medium

Abstract

To provide a mechanism capable of reducing the flicker of the hue of a subject area.SOLUTION: An imaging apparatus 100 performs photographing by using a video light capable of changing a color temperature of a light source. An imaging apparatus 100 acquires a color temperature of environmental light from image data obtained by photographing, and determines a color temperature of a video light based on the acquired color temperature of the environmental light. Before applying the determined color temperature of the video light, the image capturing apparatus 100 calculates a predicted white balance coefficient corresponding to the color temperature of the video light. The imaging apparatus 100 applies the predicted white balance coefficient at the timing of applying the determined color temperature of the video light.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Conventionally, imaging devices with auto white balance control have been known as a technology for adjusting white balance (WB) based on signals output from an imaging element. In auto white balance control, white areas are detected from captured image data, a white balance coefficient is calculated from the average value of each color component across the entire screen, and the calculated white balance coefficient is applied to the entire screen.

[0003] Video lights are sometimes used to further adjust the white balance. A video light is an illumination device that continuously irradiates a subject with light during video shooting. Some video lights use LEDs as their light source, making it possible to freely change the color temperature of the light emitted. In video lights with variable color temperature, by comparing the color temperature of the area around the subject with the color temperature of the video light and adjusting the color temperature of the video light to match the color temperature of the area around the subject, it is possible to prevent deviations in the white balance adjustment and improve color reproducibility (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-118399 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned Patent Document 1 does not mention the timing of applying the color temperature of the video light, and there may be a time lag between the timing of applying the color temperature of the video light and the timing of applying the white balance coefficient. When such a time lag occurs, for example, the color of the subject area changes when the color temperature of the video light is switched, and then when the white balance coefficient is switched and the white balance follows, the color of the subject area changes further. This results in a flickering of the color of the subject area, where the color of the subject area changes in two stages.

[0006] An object of the present invention is to provide a mechanism that can reduce color flicker in a subject area. [Means for solving the problem]

[0007] In order to achieve the above object, an imaging device of the present invention is an imaging device that performs imaging using an illumination device that can change the color temperature of its light source, and includes: an acquisition means that acquires the color temperature of ambient light from image data obtained by imaging; a determination means that determines the color temperature of the illumination device based on the color temperature of the ambient light; a calculation means that calculates white balance coefficients for adjusting the white balance of the image data; and an application means that applies the determined color temperature of the video light and the calculated white balance coefficients, wherein the calculation means calculates a predicted white balance coefficient corresponding to the determined color temperature of the illumination device before the determined color temperature of the illumination device is applied, and the application means applies the predicted white balance coefficient calculated by the calculation means and corresponding to the determined color temperature of the illumination device in accordance with the timing to apply the determined color temperature of the illumination device. [Effects of the Invention]

[0008] According to the present invention, it is possible to reduce color flicker in the subject area. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram schematically showing the configuration of an imaging device according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating conventional auto white balance control. [Figure 3] 2 is a diagram for explaining changes in color tone between a subject area and a background area when a moving image is shot using the video light of FIG. 1. FIG. [Figure 4] 5 is a flowchart showing the procedure of a control process executed by a system control circuit of the imaging device according to the first embodiment. [Figure 5] 10 is a flowchart showing the procedure of a control process executed by a system control circuit of an image pickup apparatus according to a second embodiment. [Figure 6] 10 is an example of a graph showing the relationship between the absolute value of the difference in color temperature between ambient light and video light and the color temperature difference score. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following describes in detail embodiments of the present invention. The embodiments described below are examples for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is not limited to the following embodiments. Furthermore, the present invention may be configured by appropriately combining parts of each embodiment described below.

[0011] First, an imaging device and a control method thereof according to a first embodiment of the present invention will be described.

[0012] Fig. 1 is a block diagram showing a schematic configuration of an imaging device 100 according to the present embodiment. In Fig. 1, 10 is a photographing lens. 12 is a shutter equipped with an aperture function. 14 is an imaging element that converts an optical image of a subject into an electrical signal. 16 is an A / D converter that converts the analog signal output of the imaging element 14 into a digital signal.

[0013] A timing generation circuit 18 supplies clock signals and control signals to the image sensor 14, the A / D converter 16, and the D / A converter 26. The timing generation circuit 18 is controlled by a memory control circuit 22 and a system control circuit 50.

[0014] An image processing circuit 20 performs predetermined pixel interpolation and color conversion processing on data from the A / D converter 16 or data from the memory control circuit 22. The image processing circuit 20 also performs predetermined arithmetic processing using image data generated by capturing images and supplies the arithmetic results to a system control circuit 50. The image data may be, for example, still image data obtained by still image capture or image data for each frame constituting video data obtained by video capture. Based on the arithmetic results, the system control circuit 50 controls an exposure control unit 40 and a ranging control unit 42 to perform TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-flash) processing. Furthermore, the image processing circuit 20 performs TTL AWB (auto white balance) processing using the image data generated by capturing images.

[0015] Reference numeral 22 denotes a memory control circuit, which controls the A / D converter 16, timing generation circuit 18, image processing circuit 20, image display memory 24, D / A converter 26, memory 30, and compression / expansion circuit 32. Data output from the A / D converter 16 is supplied to the memory control circuit 22 via the image processing circuit 20, or is supplied directly to the memory control circuit 22. As a result, the data is written to the image display memory 24 or memory 30.

[0016] Reference numeral 24 denotes an image display memory. Reference numeral 26 denotes a D / A converter. Reference numeral 28 denotes an image display unit such as a TFT-LCD. The image data for display written to the image display memory 24 is supplied to the image display unit 28 via the D / A converter 26. As a result, an image is displayed on the image display unit 28. An electronic viewfinder function can be realized by sequentially displaying captured images on the image display unit 28. Furthermore, the image display unit 28 can be used for playback display functions and various other display functions. Furthermore, the image display unit 28 can arbitrarily turn on / off the display in response to instructions from the system control circuit 50, and turning off the display can significantly reduce the power consumption of the imaging device 100.

[0017] Reference numeral 30 denotes a memory that stores still image data and video data obtained by shooting. The memory 30 has a storage capacity sufficient to store a predetermined number of still image data and a predetermined period of video data. By using this memory 30, a large amount of still image data can be saved at high speed, even in the case of continuous shooting, in which multiple still image data are taken consecutively, or in the case of panoramic shooting. The memory 30 is also used as a working area for the system control circuit 50.

[0018] Reference numeral 32 denotes a compression / expansion circuit, which compresses or expands image data using adaptive discrete cosine transform (ADCT) or the like. The compression / expansion circuit 32 reads image data stored in the memory 30, performs compression or expansion processing on the data, and writes the processed image data into the memory 30.

[0019] Reference numeral 40 denotes an exposure control unit that controls the shutter 12, which has an aperture function, and can also perform flash dimming in cooperation with a strobe 48. Reference numeral 42 denotes a distance measurement control unit that controls focusing of the photographic lens 10. The exposure control unit 40 and distance measurement control unit 42 are controlled by a system control circuit 50 based on the results of calculations performed by the image processing circuit 20 on image data generated by imaging. This achieves TTL exposure control and distance measurement control. Reference numeral 44 denotes a zoom control unit that controls zooming of the photographic lens 10. Reference numeral 46 denotes a barrier control unit that controls the operation of the barrier (protection unit) 102.

[0020] Reference numeral 48 denotes a strobe as an illumination device, which has the function of projecting AF assist light to brightly illuminate the subject and make it easier to focus. The strobe 48 may be a single light source, or may be provided with multiple light sources such as a xenon tube for flashing light and LEDs for minute long-time light emission or video light, and each may be controlled by switching between them during main light emission.

[0021] Reference numeral 50 denotes a system control circuit that controls the entire image capture apparatus 100. Reference numeral 52 denotes a volatile memory that stores constants, variables, programs, etc. for the operation of the system control circuit 50. The volatile memory 52 also stores a program diagram used in AE. The program diagram is a table that defines the relationship between the aperture diameter and shutter speed control values ​​for each exposure value.

[0022] Reference numeral 54 denotes a display unit including a liquid crystal display device, a speaker, etc. The display unit 54 displays operating statuses, messages, etc. using characters, images, sounds, etc. in accordance with program execution by the system control circuit 50. One or more display units 54 are installed in easily visible positions near the operation unit of the imaging device 100, and are configured, for example, by a combination of an LCD, LED, sound-emitting elements, etc. Furthermore, a portion of the display unit 54 is installed within the optical viewfinder 104.

[0023] Of the contents displayed on the display unit 54, those displayed on the LCD or the like include single shot / continuous shooting display, self-timer display, compression ratio display, recording pixel count display, number of recorded frames display, number of remaining frames display, shutter speed display, aperture value display, etc. In addition, there are exposure compensation display, flash display, red-eye reduction display, macro shooting display, buzzer setting display, clock battery remaining capacity display, battery remaining capacity display, error display, multi-digit information display, recording medium 1200 attachment / detachment status display, communication I / F operation display, date and time display, etc. Furthermore, of the contents displayed on the display unit 54, those displayed within the optical viewfinder 104 include focus display, camera shake warning display, flash charging display, shutter speed display, aperture value display, exposure compensation display, etc.

[0024] Reference numeral 56 denotes an electrically erasable and recordable non-volatile memory, which is configured by, for example, an EEPROM.

[0025] Reference numerals 60, 62, 64, and 66 denote operation units through which the user instructs the system control circuit 50 to perform various operations, and are composed of, for example, one or a combination of switches, dials, touch panels, pointing with line-of-sight detection, voice recognition devices, etc. More specifically, reference numeral 60 denotes a mode dial switch. The mode dial switch 60 can switch between various modes, including power off, automatic exposure modes such as program AE mode, fully automatic mode, manual exposure mode, panoramic shooting mode, playback mode, multi-screen playback / erase mode, and PC connection mode.

[0026] Reference numeral 62 denotes a shutter switch SW1, which is turned on during operation of a shutter button (not shown) to instruct the start of operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing.

[0027] Shutter switch SW2 64 is turned on when the shutter button is operated, and starts an exposure process in which a signal read from the image sensor 14 is written as image data into memory 30 via the A / D converter 16 and memory control circuit 22. Furthermore, it instructs the start of a series of processes, including a development process using calculations in the image processing circuit 20 and memory control circuit 22, and a recording process in which image data is read from memory 30, compressed by the compression / expansion circuit 32, and written to the recording medium 1200.

[0028] Reference numeral 66 denotes an operation unit consisting of various buttons, a touch panel, etc. The operation unit 66 includes, for example, a menu button, a set button, a macro button, an image display on / off button, a multi-screen playback page break button, a flash setting button, a single shot / continuous shot / self-timer switching button, and a menu movement + (plus) button. The operation unit 66 also includes a menu movement - (minus) button, a playback image movement + (plus) button, a playback image - (minus) button, a shooting image quality selection button, an exposure compensation button, a date / time setting button, etc.

[0029] Reference numeral 80 denotes a power supply control unit, which is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to which electricity is applied, etc. The power supply control unit 80 detects whether a battery is installed, the battery type, and the remaining battery power, and controls the DC-DC converter based on the detection results and instructions from the system control circuit 50 to supply the required voltage for the required period to each unit, including the recording medium. Reference numerals 82 and 84 denote connectors. Reference numeral 86 denotes a power supply, which may be a primary battery such as an alkaline battery or lithium battery, a secondary battery such as a NiCd battery, NiMH battery, or Li battery, an AC adapter, etc.

[0030] Reference numerals 90 and 94 denote interfaces with recording media such as memory cards and hard disks. Reference numerals 92 and 96 denote connectors for connecting to recording media such as memory cards and hard disks. Reference numeral 98 denotes a recording medium attachment / detachment detection unit. The recording medium attachment / detachment detection unit 98 detects whether a recording medium 1200 is attached to the connector 92, and whether a recording medium 1210 is attached to the connector 96.

[0031] In this embodiment, the description is given assuming that there are two systems of interfaces and connectors for attaching recording media. Of course, the configuration may include either a single system or multiple systems of interfaces and connectors for attaching recording media. Also, the configuration may include a combination of interfaces and connectors of different standards. The interfaces and connectors may be configured using those that comply with standards such as PCMCIA cards and CF (CompactFlash (registered trademark)) cards.

[0032] Furthermore, if the interfaces 90 and 94 and the connectors 92 and 96 are configured using cards that comply with standards such as PCMCIA cards and CF (Compact Flash) cards, by connecting various communication cards such as LAN cards, modem cards, USB cards, IEEE1394 cards, P1284 cards, SCSI cards, and PHS communication cards, it is possible to transfer image data and management information attached to the image data between other computers and peripheral devices such as printers.

[0033] Reference numeral 102 denotes a barrier (protective means) that covers the imaging unit including the photographing lens 10 of the imaging device 100 to prevent the imaging unit from becoming dirty or damaged.

[0034] Reference numeral 104 denotes an optical viewfinder, and it is possible to take pictures using only the optical viewfinder, without using the electronic viewfinder function of the image display unit 28. Furthermore, the optical viewfinder 104 displays some of the display contents of the display unit 54, such as focus indication, camera shake warning indication, flash charging indication, shutter speed indication, aperture value indication, and exposure compensation indication.

[0035] Reference numeral 110 denotes a communication unit having various communication functions such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, wireless communication, etc. Reference numeral 112 denotes a connector that connects the imaging device 100 to other devices via the communication unit 110, or an antenna in the case of wireless communication.

[0036] Reference numeral 1200 denotes a recording medium such as a memory card or a hard disk. The recording medium 1200 includes a recording unit 1202 configured from a semiconductor memory, a magnetic disk, or the like, an interface 1204 for connecting with the image capturing device 100, and a connector 1206 for connecting with the image capturing device 100.

[0037] Reference numeral 1210 denotes a recording medium such as a memory card or a hard disk. The recording medium 1210 includes a recording unit 1212 configured from a semiconductor memory, a magnetic disk, or the like, an interface 1214 for connecting with the image capturing device 100, and a connector 1216 for connecting with the image capturing device 100.

[0038] Here, conventional auto white balance control by the image processing circuit 20 will be described. The analog signal output from the image sensor 14 is converted into a digital signal by the A / D converter 16 and divided into a plurality of blocks as shown in FIG. 2. Each block is made up of R, G, and B color pixels, and a color evaluation value (C x [I C y [i]) is calculated.

[0039]

number

[0040] Here, i is the number indicating the block. R[i], G[i], and B[i] are the average values ​​of the R, G, and B pixels contained in block i, respectively. Y[i] = (R[i] + 2G[i] + B[i] / 4).

[0041] where C x indicates the color temperature direction. Therefore, if only the block excluding the main subject area of ​​the image data is used, it can be simply expressed as C x may be used as the color temperature of the background, or if only the main subject is illuminated with strobe light, the color temperature of the background may be used as the color temperature of the ambient light.

[0042] Then, the calculated color evaluation value (C x[I C y If the pixel value [i]) is included in the preset white detection range, the block is determined to be white. Then, the integral values ​​(SumR, SumG, SumB) of the color pixels included in the block are calculated, and the white balance coefficient (WBC o _R, WBC o _G, WBC o _B) is calculated.

[0043]

number

[0044] Next, control of video shooting using a video light in the first embodiment will be described.

[0045] FIG. 3 is a diagram for explaining the change in color tone between the subject area and the background area when shooting a moving image using the video light of FIG.

[0046] In Figure 3(a), 300 is image data. 310 is a background region where ambient light is dominant. 320 is a main subject region, which is a simplified representation of a person as an example, with a video light illuminating the main subject.

[0047] Figure 3(a) shows an example of image data in which the color temperature of the ambient light and the color temperature of the video light illuminating the main subject are equal, and sufficient time has passed for the white balance to adjust. For example, assume that both color temperatures are 5000K (Kelvin). In this case, the white balance coefficient for 5000K is applied, and the colors of both the background and subject areas are appropriate.

[0048] Figure 3(b) is an example of image data immediately after the color temperature of the ambient light has changed from the state shown in Figure 3(a). Here, as an example, the color temperature of the ambient light is 3000K. In this case, the white balance coefficients for 5000K mentioned above are applied as is. Immediately after the color temperature of the ambient light changes, the subject is illuminated by a video light, so the color of the subject area is correct, but because no video light is illuminating areas other than the subject, the background area appears color-cast.

[0049] Figure 3(c) is an example of image data in which the color temperature of the video light is immediately adjusted from the state shown in Figure 3(b) to the same color temperature as the ambient light, 3000 K. At this point, the white balance has not yet been adjusted, and the white balance coefficients for 5000 K mentioned above are still applied, resulting in a color cast in both the background and main subject areas.

[0050] Figure 3(d) is an example of image data in the state shown in Figure 3(c) after sufficient time has passed for the white balance to adjust. In other words, the color temperature of both the ambient light and the video light remains at 3000K, and the white balance coefficient for 3000K has been applied to adjust the white balance, resulting in appropriate colors in both the background and main subject areas, just like in Figure 3(a).

[0051] In this embodiment, even if the color temperature of the video light is adjusted in accordance with changes in the color temperature of the ambient light, control is performed to prevent the state shown in Fig. 3(c) from occurring, thereby reducing color flicker in the subject area. Below, the operation of the video light (strobe 48) and white balance will be described according to the flowchart in Fig. 4.

[0052] 4 is a flowchart showing the procedure of control processing executed by the system control circuit 50 of the imaging device 100 of the first embodiment. The control processing of FIG. 4 is realized by the system control circuit 50 expanding a program recorded in the nonvolatile memory 56 into a work area of ​​the volatile memory 52, such as RAM, and executing the program. When the mode dial switch 60 is set to a mode allowing photography and the power supply 86 is turned on, the system control circuit 50 performs various photography preparation operations.

[0053] In FIG. 4, first, in S401, the system control circuit 50 determines whether or not to start shooting in video light mode. In video light mode shooting, moving images are shot with the video light turned on. Note that in S401, the system control circuit 50 determines to start shooting in video light mode when a predetermined operation is received from the user. The predetermined operation is, for example, turning on a video light switch (not shown), setting video light mode on using the operation unit 66, or pressing (on) the video light mode switch (not shown). The system control circuit 50 waits until it determines to start shooting in video light mode. If it determines to start shooting in video light mode (YES in S401), the process proceeds to S402.

[0054] In S402, the system control circuit 50 starts capturing moving images. That is, the imaging device 100 starts acquiring and recording image data. As described above, this image data is image data of frames that make up the moving image data obtained by capturing moving images.

[0055] Next, in S403, the system control circuit 50 acquires the color temperature of the ambient light from the acquired image data. The color temperature of the ambient light may be referred to as the color temperature of the background, and the following method is used to determine the background region. For example, the non-illuminated area of ​​the video light may be determined as the background region. The non-illuminated area is determined by comparing the pixel values ​​of corresponding pixels in image data obtained by video shooting, one with the video light turned off, with the other with the video light turned on, and determining the difference between the pixel values ​​of the corresponding pixels as being less than a predetermined value. Alternatively, the main subject region may be detected from the image data, and the area excluding the main subject region from this image data may be determined as the background region. Alternatively, the entire image data may be determined as the background region. The color temperature calculated from the background region thus determined is obtained as the color temperature of the ambient light.

[0056] Next, in S404, the system control circuit 50 acquires the color temperature of the video light. Note that here, the system control circuit 50 may acquire the color temperature of the video light directly from the video light (strobe 48), or may acquire the color temperature of the area illuminated by the video light. Also, if the video light is not yet turned on at this stage, the system control circuit 50 may use the color temperature of the ambient light acquired in S403 as the color temperature of the video light.

[0057] Next, in S405, the system control circuit 50 determines whether the color temperature of the ambient light has changed while the video light is on, or whether the video light has changed from an off state to an on state. Here, whether the color temperature of the ambient light has changed while the video light is on is determined to have changed, for example, if the color temperature of the ambient light has changed by more than a predetermined value relative to the color temperature of the video light and this change continues for more than a predetermined period of time. Whether the video light has changed from an off state to an on state may also be determined based on the brightness of the captured image data, or based on whether the power switch of the video light has been turned on.

[0058] If it is determined in S405 that the color temperature of the ambient light will not change while the video light is on, and that the video light will not change from an off state to an on state, the process returns to S401.

[0059] If it is determined in S405 that the color temperature of the ambient light has changed while the video light is on, or if it is determined that the video light has changed from an off state to an on state, the process proceeds to S406.

[0060] In S406, the system control circuit 50 determines the color temperature of the video light. The color temperature of the video light is determined based on the color temperature of the ambient light. For example, the color temperature of the video light may be determined to be the color temperature of the ambient light, or the average color temperature of the ambient light over a predetermined period of time in the past (predetermined past time) may be determined to be the color temperature of the video light.

[0061] Next, in S407, the system control circuit 50 calculates predicted white balance coefficients corresponding to the color temperature of the video light determined in S406. If the color temperature of the ambient light and the color temperature of the video light are exactly the same, the white balance coefficients for that color temperature should be used, but this is not always the case. For example, even if the color temperature of the ambient light detected from the image data is 5000K, the 5000K may be due to a single light source, or there may be multiple light sources and the average result may be 5000K. When there are multiple light sources, the main subject may also be illuminated by multiple light sources.

[0062] Therefore, in this embodiment, before applying the color temperature of the video light determined in S406, a predicted white balance coefficient (WBC) corresponding to the color temperature of the video light determined in S406 is calculated in S407. o _R, WBC o _G, WBC o _B) is calculated. o _R, WBC o _G, WBC o _B) is calculated using the following formulas (6) to (8).

[0063]

number

[0064] Note that α is the ratio of the amount of ambient light to the amount of video light in the main subject area, and the value decreases as the influence of video light increases within the range of 0 to 100. o _R, EWBC o _G, EWBC o _B is the white balance coefficient calculated from the color temperature of the ambient light obtained in S403. o _R, VWBC o _G, VWBC o _B is a white balance coefficient calculated from the color temperature of the video light obtained in S404.

[0065] Furthermore, if there is no change in the scene before and after switching the video light from its off state to its on state, the light intensity ratio α may be calculated from the luminance ratio in the main subject area before and after switching the video light from its off state to its on state. This allows an appropriate light intensity ratio α to be calculated when there is no change in the scene. Alternatively, without considering changes in the scene, the light intensity ratio α may be calculated using one or more of the exposure information used in video shooting, information indicating the position and size of the main subject area, information about the distance to the main subject, a distance map, the guide number of the video light, and the light output of the video light. This allows an appropriate light intensity ratio α to be calculated without considering changes in the scene.

[0066] Next, in S408, the system control circuit 50 simultaneously applies the color temperature of the video light and the predictive white balance coefficients. The color temperature of the video light here is the color temperature determined in S406, and the predictive white balance coefficients are the predictive white balance coefficients calculated in S407. In S408, the color temperature of the video light and the predictive white balance coefficients may be changed all at once, or they may be changed gradually. When changing gradually, for example, the amount of change in the color temperature of the video light and the predictive white balance coefficients may be divided by the same number of divisions, and the changes may be controlled to occur gradually over the same amount of time.

[0067] Next, in S409, the system control circuit 50 determines whether or not to end shooting in the video light mode. Note that in S409, if the system control circuit 50 receives a predetermined operation from the user, it determines to end shooting in the video light mode. In this case, this processing ends. The predetermined operation is, for example, turning off the video light switch (not shown), setting the video light mode to off using the operation unit 66, or pressing (off) the video light mode switch (not shown). On the other hand, if the predetermined operation is not received from the user, the system control circuit 50 determines not to end shooting in the video light mode. In this case, this processing returns to S402.

[0068] According to the embodiment described above, the predicted white balance coefficients calculated in S407 are applied in accordance with the timing at which the color temperature of the video light determined in S406 is applied. In other words, there is no time lag between the timing at which the color temperature of the video light is applied and the timing at which the white balance coefficients are applied, and this time lag does not cause the color of the subject area to change by two stages. This makes it possible to reduce flickering of the color of the subject area.

[0069] In the above-described embodiment, the color temperature of the ambient light is obtained from the background region of the image data, which is an area not illuminated by the video light. This makes it possible to obtain the color temperature of the ambient light that is not affected by the video light.

[0070] In the above-described embodiment, the color temperature of the ambient light is obtained from the background region of the image data, which is the region of the image data excluding the main subject region. This makes it possible to obtain the color temperature of the ambient light that is not affected by the video light when shooting with a video light illuminating only the main subject region.

[0071] In the above-described embodiment, the color temperature of the ambient light is obtained from the background region of the image data, which is the entire region of this image data. This makes it possible to obtain the color temperature of the ambient light even in a configuration that does not include a means for detecting non-illuminated regions of the video light or a means for detecting the main subject region.

[0072] Next, an imaging device and a control method thereof according to a second embodiment of the present invention will be described. The second embodiment is basically the same as the first embodiment in terms of configuration and operation, but differs from the first embodiment in that the color temperature of the video light and the predictive white balance coefficient are simultaneously applied when a preset condition is met. Therefore, a description of the overlapping configuration and operation will be omitted, and the following will describe the different configuration and operation.

[0073] In the first embodiment described above, when the color temperature of the video light is applied, the predictive white balance coefficient corresponding to the color temperature of the video light is also applied at the same time. In contrast, in the second embodiment, a condition is set for applying the color temperature of the video light and the predictive white balance coefficient at the same time.

[0074] FIG. 5 is a flowchart showing the procedure of control processing executed by the system control circuit 50 of the imaging device 100 of the second embodiment. The control processing of FIG. 5 is similar to the control processing of FIG. 4 described above, and the following will particularly describe the differences from the control processing of FIG. 4 described above. The control processing of FIG. 5, like the control processing of FIG. 4 described above, is realized by the system control circuit 50 expanding a program recorded in nonvolatile memory 56 into a work area of ​​volatile memory 52 such as RAM and executing it. When the power supply 86 is turned on with the mode dial switch 60 set to a mode allowing photography, the system control circuit 50 performs various photography preparation operations.

[0075] In FIG. 5, first, the above-mentioned steps S401 to S406 are performed.

[0076] Next, in S501, the system control circuit 50 determines whether to synchronize the color temperature of the video light with the timing for applying the predictive white balance coefficients. In S501, it is determined that the color temperature of the video light and the timing for applying the predictive white balance coefficients should be synchronized in the following cases: For example, when the difference between the color temperature of the ambient light and the color temperature of the video light is greater than a predetermined value; when the amount of change in the color temperature of the ambient light over a predetermined number of frames after the color temperature of the ambient light has changed by a greater than predetermined value is within a predetermined range; or when scene determination is performed using at least one of motion information and brightness information and it is determined that the scene has not changed. Note that, as a method for detecting motion information, for example, motion information may be detected from gyro information, or motion information may be detected from the movement of a point of interest using image data of a predetermined number of past frames.

[0077] If it is determined in S501 that the timing for applying the video light color temperature and the predicted white balance coefficients is to be synchronized, the process proceeds to S407, whereupon S408 and S409 are performed, and the process ends.

[0078] On the other hand, in S501, if none of the above cases apply, it is determined that the timing for applying the color temperature of the video light and the predicted white balance coefficients is not synchronized. In this case, the process proceeds to S502.

[0079] In S502, the system control circuit 50 applies the color temperature of the video light determined in S406. Next, in S503, the system control circuit 50 acquires image data of the frame after applying the color temperature of the video light.

[0080] Next, in S504, the system control circuit 50 calculates a white balance coefficient from the acquired image data and applies the calculated white balance coefficient. As explained in S408 above, the white balance coefficient can be changed all at once or gradually. After that, the process proceeds to S409.

[0081] In the second embodiment described above, when the difference between the color temperature of the ambient light and the color temperature of the video light is greater than a predetermined value, control is performed to synchronize the timing of applying the color temperature of the video light and the predictive white balance coefficients. Here, when the difference between the color temperature of the ambient light and the color temperature of the video light is greater than a predetermined value, the color temperature of the video light is adjusted to match the color temperature of the ambient light to prevent white balance adjustment errors and reproduce an appropriate white. However, even if the color temperature of the video light is adjusted, if an appropriate white balance coefficient corresponding to this color temperature is not applied, both the background region and the main subject region will be color-cast, as shown in FIG. 3(c) above. In contrast, in the second embodiment described above, when the difference between the color temperature of the ambient light and the color temperature of the video light is greater than a predetermined value, control is performed to synchronize the timing of applying the color temperature of the video light and the predictive white balance coefficients. This allows the color temperature of the video light to be adjusted and appropriate predictive white balance coefficients corresponding to the adjusted color temperature to be applied. As a result, both the background region and the main subject region can be controlled to have appropriate colors.

[0082] Furthermore, in the second embodiment described above, when the color temperature of the ambient light changes by more than a predetermined value, the timing for applying the predictive white balance coefficients is synchronized only when the amount of change in the color temperature of the ambient light between a predetermined number of frames is within a predetermined range. Here, when the color temperature of the ambient light changes by more than a predetermined value, if the amount of change in the color temperature of the ambient light between a predetermined number of frames is within a predetermined range, the change in the color temperature of the ambient light is relatively stable, and the color temperature of the ambient light can be accurately acquired in S403. With such an ambient light color temperature, appropriate predictive white balance coefficients can be calculated, and the background region and the main subject region can be controlled to appropriate colors. On the other hand, when the color temperature of the ambient light changes by more than a predetermined value, if the amount of change in the color temperature of the ambient light between a predetermined number of frames is outside the predetermined range, the change in the color temperature of the ambient light is not stable, and there is a concern that the color temperature of the ambient light cannot be accurately acquired in S403. With such an ambient light color temperature, there is a concern that appropriate predictive white balance coefficients cannot be calculated, and as a result, the background region and the main subject region cannot be controlled to appropriate colors. In contrast, in the second embodiment described above, when the color temperature of the ambient light changes by more than a predetermined amount, only when the amount of change in the color temperature of the ambient light between a predetermined number of frames is within a predetermined range, control is performed to synchronize the timing of applying the color temperature of the video light and the predictive white balance coefficients, thereby making it possible to suppress the implementation of this control when there is a concern that the color may not be controlled to an appropriate hue.

[0083] Furthermore, in the second embodiment described above, when the scene is not changing, control is performed to synchronize the timing of applying the color temperature of the video light and the predictive white balance coefficients. However, when the scene is changing, control is not performed to synchronize the timing of applying the color temperature of the video light and the predictive white balance coefficients. Here, when the scene is changing, there is a concern that the color temperature of the ambient light cannot be accurately acquired in S403. With such a color temperature of the ambient light, there is a concern that appropriate predictive white balance coefficients cannot be calculated, and as a result, the background region and the main subject region cannot be controlled to have appropriate colors. In contrast, in the second embodiment described above, when the scene is not changing, control is performed to synchronize the timing of applying the color temperature of the video light and the predictive white balance coefficients. However, when the scene is changing, control is not performed to synchronize the timing of applying the color temperature of the video light and the predictive white balance coefficients. This makes it possible to suppress the implementation of this control when there is a concern that appropriate color control will not be possible.

[0084] In the second embodiment described above, the information used in the determination in S501 may be used to calculate the predicted white balance coefficients in S407. An example of this will be described with reference to FIG.

[0085] Fig. 6 is an example of a graph showing the relationship between the absolute value of the color temperature difference between ambient light and video light and the color temperature difference score. As shown in Fig. 6, the color temperature difference score is 0 when the absolute value of the color temperature difference is in the range of 0 to less than 200. The color temperature difference score increases linearly when the absolute value of the color temperature difference is in the range of 200 or more and less than 600. The color temperature difference score is 100 when the absolute value of the color temperature difference is in the range of 600 or more.

[0086] The first predicted white balance coefficient (WBC) calculated by the above formulas (6) to (8) o _R, WBC o _G, WBC o _B) and the white balance coefficient before changing the color temperature of the video light (CWBC o _R, CWBCo _G, CWBC o Using the color temperature difference score β, the second predicted white balance coefficient (WBC o _R', WBC o _G', WBC o _B') is calculated.

[0087]

number

[0088] In this way, by using the information used in the determination in S501 to calculate the predicted white balance coefficients, it is possible to calculate predicted white balance coefficients that can be controlled to more appropriate colors while reducing color flicker in the subject area.

[0089] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0090] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An imaging device that takes pictures using a lighting device that can change the color temperature of its light source, comprising: an acquisition means for acquiring the color temperature of ambient light from image data obtained by shooting; a determination means for determining the color temperature of the lighting device based on the color temperature of the ambient light; a calculation means for calculating white balance coefficients for adjusting the white balance of the image data; and an application means for applying the determined color temperature of the video light and the calculated white balance coefficients, wherein the calculation means calculates a predicted white balance coefficient corresponding to the determined color temperature of the lighting device before the determined color temperature of the lighting device is applied, and the application means applies the predicted white balance coefficient calculated by the calculation means, which corresponds to the determined color temperature of the lighting device, in accordance with the timing of applying the determined color temperature of the lighting device. (Configuration 2) The imaging device according to Configuration 1, wherein the acquisition means acquires the color temperature of the ambient light from a background area in the image data, the background area being a non-illuminated area of ​​the lighting device. (Configuration 3) The imaging device according to Configuration 1, further comprising a detection means for detecting a main subject region from the image data, wherein the acquisition means acquires the color temperature of the ambient light from a background region in the image data, and the background region is a region of the image data excluding the main subject region. (Configuration 4) The imaging device according to configuration 1, wherein the acquisition means acquires the color temperature of the ambient light from a background region in the image data, and the background region is the entire region of the image data. (Configuration 5) The imaging device according to any one of configurations 1 to 4, wherein the determining means determines the color temperature of the ambient light to be the color temperature of the lighting device. (Configuration 6) The imaging device according to any one of configurations 1 to 5, wherein the determining means determines the average color temperature of the ambient light over a predetermined period of time in the past as the color temperature of the lighting device. (Configuration 7) The imaging device described in any one of configurations 1 to 6, characterized in that the calculation means calculates a predicted white balance coefficient corresponding to the determined color temperature of the lighting device based on the light intensity ratio of the ambient light and the lighting device in a predetermined area in the image data before the determined color temperature of the lighting device is applied. (Configuration 8) The imaging device according to configuration 7, wherein the light amount ratio is calculated based on a luminance ratio in the predetermined area before and after the lighting device is switched from an off state to an on state. (Configuration 9) The imaging device described in Configuration 7, characterized in that the light intensity ratio is calculated using one or more of the exposure information used in shooting the video, information indicating the position and size of the specified area, distance information to the subject corresponding to the specified area, a distance map, the guide number of the video light, and the light intensity of the video light. (Configuration 10) An imaging device described in any one of configurations 1 to 9, characterized in that when the difference between the color temperature of the ambient light and the color temperature of the lighting device is greater than a predetermined value, the application means applies a predicted white balance coefficient calculated by the calculation means, which corresponds to the determined color temperature of the lighting device, in accordance with the timing of applying the determined color temperature of the lighting device. (Configuration 11) The imaging device described in Configuration 10, wherein the calculation means uses information used to determine whether the difference between the color temperature of the ambient light and the color temperature of the lighting device is greater than a predetermined value in calculating the predicted white balance coefficients. (Configuration 12) An imaging device described in any one of configurations 1 to 11, characterized in that, when the color temperature of the ambient light changes by more than a predetermined value, only when the amount of change in the color temperature of the ambient light between predetermined frames is within a predetermined range, the application means applies a predicted white balance coefficient calculated by the calculation means, which corresponds to the color temperature of the determined lighting device, in accordance with the timing of applying the color temperature of the determined lighting device. (Configuration 13) The imaging device described in any one of Configurations 1 to 12, further comprising means for determining whether a scene has changed in the image data, wherein the application means, when it is determined that the scene has not changed, applies the predictive white balance coefficients calculated by the calculation means, which correspond to the determined color temperature of the lighting device, in accordance with the timing of applying the color temperature of the determined lighting device, and, when it is determined that the scene has changed, does not perform control to apply the predictive white balance coefficients calculated by the calculation means, which correspond to the determined color temperature of the lighting device, in accordance with the timing of applying the color temperature of the determined lighting device. [Explanation of symbols]

[0091] 50 System control circuit 100 Imaging device

Claims

1. An imaging device that takes pictures using an illumination device that can change the color temperature of the light source, an acquisition means for acquiring the color temperature of the ambient light from image data obtained by shooting; a determining means for determining a color temperature of the lighting device based on the color temperature of the ambient light; a calculation means for calculating a white balance coefficient for adjusting the white balance of the image data; applying means for applying the determined color temperature of the video light and the calculated white balance coefficient; the calculating means calculates predicted white balance coefficients corresponding to the determined lighting device color temperature before the determined lighting device color temperature is applied; the applying means applies the predicted white balance coefficient calculated by the calculating means, the predicted white balance coefficient corresponding to the determined color temperature of the lighting device, in accordance with a timing at which the determined color temperature of the lighting device is applied.

2. the acquiring means acquires the color temperature of the ambient light from a background region in the image data; 2. The imaging device according to claim 1, wherein the background area is a non-illuminated area of ​​the lighting device.

3. further comprising a detection means for detecting a main subject area from the image data; the acquiring means acquires the color temperature of the ambient light from a background region in the image data; 2. The imaging device according to claim 1, wherein the background region is a region of the image data excluding the main subject region.

4. the acquiring means acquires a color temperature of ambient light from a background region in the image data; 2. The imaging device according to claim 1, wherein the background area is the entire area of ​​the image data.

5. 2. The imaging device according to claim 1, wherein the determining means determines the color temperature of the ambient light to be the color temperature of the lighting device.

6. 2. The imaging device according to claim 1, wherein the determining means determines an average value of the color temperature of the ambient light over a predetermined period of time in the past as the color temperature of the lighting device.

7. 2. The imaging device according to claim 1, wherein the calculation means calculates a predicted white balance coefficient corresponding to the determined color temperature of the lighting device based on a light intensity ratio between the ambient light in a predetermined area of ​​the image data and the lighting device, before the determined color temperature of the lighting device is applied.

8. 8. The imaging device according to claim 7, wherein the light amount ratio is calculated based on a luminance ratio in the predetermined area before and after the lighting device is switched from an off state to an on state.

9. The imaging device described in claim 7, characterized in that the light intensity ratio is calculated using one or more of exposure information used in shooting the video, information indicating the position and size of the specified area, distance information to the subject corresponding to the specified area, a distance map, a guide number of the video light, and the light intensity of the video light.

10. 2. The imaging device according to claim 1, wherein, when a difference between the color temperature of the ambient light and the color temperature of the lighting device is greater than a predetermined value, the applying unit applies the predicted white balance coefficient calculated by the calculating unit, which predictive white balance coefficient corresponds to the determined color temperature of the lighting device, in accordance with a timing at which the determined color temperature of the lighting device is applied.

11. 11. The imaging device according to claim 10, wherein the calculation means uses information used to determine whether a difference between the color temperature of the ambient light and the color temperature of the lighting device is greater than a predetermined value in calculating the predicted white balance coefficients.

12. 2. The imaging device according to claim 1, wherein, when the color temperature of the ambient light changes by a greater amount than a predetermined value, only when the amount of change in the color temperature of the ambient light over a predetermined number of frames is within a predetermined range, the applying unit applies the predicted white balance coefficient calculated by the calculating unit, which predictive white balance coefficient corresponds to the determined color temperature of the lighting device, in accordance with a timing for applying the determined color temperature of the lighting device.

13. means for determining whether a scene has changed in the image data; 2. The imaging device according to claim 1, wherein, when it is determined that the scene has not changed, the applying unit applies the predictive white balance coefficients calculated by the calculating unit, which correspond to the determined color temperature of the lighting device, in accordance with the timing of applying the color temperature of the determined lighting device, and when it is determined that the scene has changed, the applying unit does not perform control to apply the predictive white balance coefficients calculated by the calculating unit, which correspond to the determined color temperature of the lighting device, in accordance with the timing of applying the color temperature of the determined lighting device.

14. A control method for an imaging device that captures images using an illumination device that can change the color temperature of the light source, comprising: A step of acquiring the color temperature of the ambient light from image data obtained by shooting; determining a color temperature of the lighting device based on the color temperature of the ambient light; calculating a white balance coefficient for adjusting the white balance of the image data; applying the determined lighting device color temperature and the calculated white balance coefficients; In the calculating step, predicted white balance coefficients corresponding to the determined lighting device color temperature are calculated before the determined lighting device color temperature is applied; a step of applying a predicted white balance coefficient calculated in the calculating step, the predicted white balance coefficient corresponding to the determined color temperature of the lighting device, in accordance with a timing at which the determined color temperature of the lighting device is applied;

15. A program that causes a computer to execute a control method for an imaging device that captures images using an illumination device that can change the color temperature of the light source, The method for controlling the imaging device includes: A step of acquiring the color temperature of the ambient light from image data obtained by shooting; determining a color temperature of the lighting device based on the color temperature of the ambient light; calculating a white balance coefficient for adjusting the white balance of the image data; applying the determined lighting device color temperature and the calculated white balance coefficients; In the calculating step, predicted white balance coefficients corresponding to the determined lighting device color temperature are calculated before the determined lighting device color temperature is applied; the applying step applies the predicted white balance coefficients calculated in the calculating step, the predicted white balance coefficients corresponding to the determined color temperature of the lighting device, in accordance with a timing at which the determined color temperature of the lighting device is applied.

Images