Imaging device, control method thereof, and program

Abstract

To provide an imaging device capable of preferable display in each mode in the imaging device having a plurality of auto white balance modes for changing the finish condition of an image.SOLUTION: An imaging device according to the present invention includes an imaging element, image processing means that processes an image signal from the imaging element, recording means that displays the image signal from the image processing means, display means that displays the image signal from the image processing means, and ambient light information acquisition means that acquires ambient light information, and further includes adjusting means that changes the image signal to be recorded in the recording means on the basis of the white balance setting, and adjusts the color balance of the image signal to be displayed on the display means on the basis of the white balance setting and the output of the ambient light information acquisition means.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a display control technique for an imaging device. [Background technology]

[0002] Imaging devices using an image sensor, such as digital cameras and digital video cameras, generally have an auto white balance control function that automatically detects areas of the captured image that are likely to be white and applies the calculated white balance coefficient to the entire screen. Furthermore, some commercially available imaging devices have multiple auto white balance modes that the user can select from. For example, some modes allow the user to select the desired image finish, such as a mode that prioritizes making the light source color whiter, or a mode that prioritizes preserving the reddish color of light sources such as incandescent bulbs to preserve the ambiance of the scene.

[0003] Furthermore, when the color temperature of the ambient light and the display differ, for example, if a user views a display with a high color temperature under illumination with a low color temperature after adapting to the ambient light, there is a problem in that the display appears to have a strong bluish cast. In response to this, Patent Document 1 discloses a method that allows the user to view a display with a preferred color by adjusting the white point of the display to be equivalent to the ambient light based on ambient light information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-95487 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional technology disclosed in the above-mentioned patent document does not take into account the possibility of multiple white balance modes that change the finished image quality. As a result, for example, in a mode that preserves the redness of an incandescent light source, the redness may become too strong, resulting in a color tone that is not intended by the user. [Means for solving the problem]

[0006] The imaging device according to the present invention has different imaging means and a different white balance adjustment method. 1st Auto White Balance Mode or a second auto white balance mode a white balance adjustment means for adjusting the white balance of an image signal based on the auto white balance mode set by the setting means; a recording control means for recording the image signal adjusted by the white balance adjustment means on a recording unit; a display control means for displaying the image signal adjusted by the white balance adjustment means on a display unit; and an acquisition means for acquiring ambient light information, wherein the display control means performs the following operations based on the auto white balance mode set by the setting means and the ambient light information acquired by the acquisition means: The image signal obtained by adjusting the color balance of the image signal adjusted by the white balance adjustment means is The above indication Department Shown in The display control means is capable of adjusting the color balance of the image signal adjusted by the white balance adjustment means differently when the auto white balance mode set by the setting means is the first auto white balance mode and when it is the second auto white balance mode. It is characterized by the following features. [Effects of the Invention]

[0007] According to the present invention, when there are multiple white balance modes, it is possible to display an image in a preferred color tone. [Brief explanation of the drawings]

[0008] [Figure 1] Block diagram showing the basic configuration of an imaging device [Figure 2] Diagram showing the white range [Figure 3] Diagram showing the difference in white tracking range depending on the auto white balance setting [Figure 4] Flowchart of processing during imaging [Figure 5]Schematic diagram of processing when AWB1 and DISP1 [Figure 6] Schematic diagram of processing when AWB2 and DISP1 [Figure 7] Schematic diagram of processing when AWB1 and DISP2 [Figure 8] Schematic diagram of processing when AWB2 and DISP2 DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0010] Example 1 FIG. 1 is a block diagram showing the basic configuration of an image pickup apparatus according to an embodiment of the present invention.

[0011] The imaging device 100 may be, for example, a camera such as a digital camera or a digital video camera, or may be an electronic device with a camera function such as a camera phone or a camera computer. The optical system 101 is an imaging optical system including a group of lenses, a shutter, an aperture, etc. The group of lenses includes a correction lens that corrects camera shake and the like, a focus lens, etc.

[0012] The optical system 101 forms an image of subject light on the image sensor 102 in accordance with a control signal from the CPU 103. The image sensor 102 is an imaging device such as a CCD (charge-coupled device) image sensor or a CMOS (complementary metal-oxide semiconductor) image sensor, and converts the light formed as an image through the optical system 101 into an image signal by photoelectric conversion. In this embodiment, the image sensor is a CMOS image sensor, and the description will be given assuming that the imaged light is converted into RGB image signal values.

[0013] The CPU 103 executes programs stored in advance in memory to control each component of the imaging device 100 in accordance with input signals and the like, thereby performing imaging control, display control, recording control, etc. The primary storage device 104 is a volatile storage device such as a RAM (Random Access Memory), which stores temporary data and is used as a work memory for the CPU 103. Information stored in the primary storage device 104 may be used by the image processing device 105 or may be recorded on a recording medium 106. The secondary storage device 107 is a non-volatile storage device such as an EEPROM (Electrically Erasable Programmable Read-Only Memory).

[0014] The recording medium 106, which is an example of a recording unit, records captured image data stored in the primary storage device 104. The recording medium 106 may also be a storage medium that is removable from the imaging device 100, such as a semiconductor memory card. The data recorded on the recording medium 106 can be read by an external device such as a PC (personal computer) by attaching the recording medium 106 to the external device. In other words, the imaging device 100 has an attachment / detachment mechanism and a read / write function for the recording medium 106.

[0015] The secondary storage device 107 stores programs (firmware) for controlling the imaging device 100 and various setting information, which are used by the CPU 103. The display unit 108 is composed of a liquid crystal display, an organic EL display, or the like, and displays a viewfinder image during shooting, a captured image, a GUI (Graphical User Interface) image for interactive operation, and the like. The display unit 108 is capable of changing the white point setting, and operates with a white point color temperature of 6500 K unless otherwise specified, although other color temperatures may be used.

[0016] The operation unit 109 is a group of input devices that accept user operations and transmit input information to the CPU 103, and includes, for example, buttons, levers, touch panels, etc. Input devices that use voice or line of sight can also be used for operation.

[0017] The imaging device 100 has a plurality of image processing patterns that the image processing device 105 applies to captured images, and these patterns can be set as shooting modes by the operation unit 109. The image processing device 105 performs various processes, including image processing known as development processing, as well as color tone adjustment according to the shooting mode. Note that at least some of the functions of the image processing device 105 may be realized by software processing by the CPU 103. The image processing device 105 also performs auto white balance processing, which will be described later. The division of roles among these hardware components is not particularly limited, and one piece of hardware may function as multiple processing means, or multiple pieces of hardware may work together to function as one processing means.

[0018] Next, the auto white balance processing of this embodiment will be described. The auto white balance processing of this embodiment calculates ambient light information from an image captured and applies a white balance adjustment value calculated from the ambient light information. The auto white balance processing of this embodiment has two control methods: auto white balance mode 1 (hereinafter referred to as AWB1) and auto white balance mode 2 (hereinafter referred to as AWB2). These modes use different white balance adjustment methods, resulting in different adjustment results even when the same subject is photographed under the same ambient light. AWB1 is a setting that aims to whiten the color of the light source (making the values ​​of each RGB color the same). On the other hand, AWB2 is a setting that aims to reproduce colors as seen by approximating the range of whitening achieved by auto white balance to the range of human color adaptation. When an image captured using the AWB2 setting is displayed on a display device with a white point setting and a color temperature of 6500K, white balance processing is performed to produce colors that are close to what the viewer sees.

[0019] Next, a method for acquiring ambient light information will be described. In this embodiment, the R / G and B / G values ​​of each pixel are calculated from the image signal values ​​calculated by the image processing device 105 from the image signal obtained by the image sensor 102. Of these values, the average value of pixels within a range previously determined as a white range is calculated and used as ambient light information around the imaging device. Here, the R, G, and B values ​​of the calculated ambient light image signal values ​​are designated as RL, GL, and BL, respectively. Figure 2 is an illustration of the white range. Figure 2 shows a color space with the R / G value on the X axis and the B / G value on the Y axis. The solid line 200 indicates the blackbody radiation axis, and the area inside the dotted line 201 indicates the white range.

[0020] Next, the difference between AWB1 and AWB2 will be explained using FIG. 3. The area indicated by dotted line 302 in FIG. 3 is the white tracking range of AWB1, and the area indicated by dotted line 303 in FIG. 3 is the white tracking range of AWB2. In this embodiment, the white tracking range of AWB1 and the white range are assumed to be equal. The white tracking range is the range in which white balance adjustment is performed, and light sources within the white tracking range are reproduced as white, while light sources outside the white tracking range are not reproduced as white. If the values ​​(RL / GL, BL / GL) calculated from the ambient light image signal values ​​RL, GL, and BL are outside the white tracking range, white tracking coefficients KW_R and KW_B are calculated to adjust the ambient light image signal values ​​RL and BL.

[0021] Next, a method for calculating the white tracking coefficients KW_R and KW_B will be described with reference to FIG.

[0022] When the auto white balance mode is AWB2, let's say the R / G and B / G values ​​calculated from the calculated ambient light image signal values ​​RL, GL, and BL are located at point X (R_X, B_X) in Figure 3. Here, point X is outside the white tracking range, so point Y, the closest point within the white tracking range (here, the R / G value of point Y is R_Y, and the B / G value is B_Y), is calculated. Then, the white tracking coefficients KW_R and KW_B are calculated using the following formulas. KW_R = R_Y / R_X KW_B = B_Y / B_X

[0023] When the R / G values ​​and B / G values ​​of the ambient light image signal values ​​RL, GL, BL are within the white tracking range, KW_R and KW_B are 1. The white tracking range of AWB2 is set to be closer to the human color adaptation range than the white tracking range of AWB1, and AWB2 is closer to what the human eye actually sees.

[0024] After obtaining the ambient light information, the white balance adjustment value is calculated using the following formula. WR, WG, and WB are the white balance adjustment values ​​by which the R, G, and B values ​​are multiplied, respectively, and MAX[RL, GL, BL] indicates the maximum value of RL, GL, and BL. WR = MAX[RL, GL, BL] / RL × KW_R WG = MAX[RL, GL, BL] / GL WB = MAX[RL, GL, BL] / BL × KW_B

[0025] Although the present embodiment is configured to acquire ambient light information from a captured image, the means for acquiring ambient light information is not limited to this. For example, if the lighting (smart lighting) is capable of acquiring color information using wireless communication, color information acquired using such means may be used as ambient light information. Furthermore, ambient light information may be acquired from color information obtained from an imaging element other than the imaging element that acquires the captured image.

[0026] Next, the display modes of this embodiment will be described. The imaging device 100 has two control modes for the display unit 108: display mode 1 (hereinafter, DISP1) and display mode 2 (hereinafter, DISP2). DISP1 is a setting that controls the image displayed on the display unit 108 so that the colors of the subject are closer to the colors seen by the user's eyes, regardless of ambient light information or auto white balance settings. This control aims to reduce the sense of discomfort felt by the user when looking at the display unit 108. On the other hand, DISP2 aims to provide a setting that makes it easy for the user to understand differences in color due to camera settings by displaying colors according to the camera settings on the display unit 108.

[0027] Here, in the imaging device 100, DISP1 and DISP2 can be selected for live view during imaging, but when images recorded on the recording medium 106 are played back and displayed, DISP2 is the only option.

[0028] Next, the processing performed during image capture in the imaging device of this embodiment will be described with reference to the flowchart of Figure 4 and schematic diagrams (Figures 5, 6, 7, and 8) of image processing according to auto white balance setting and display mode setting.

[0029] Here, FIG. 5 is a schematic diagram in which the auto white balance setting is AWB1 and the display mode setting is DISP1.

[0030] FIG. 6 is a schematic diagram in which the auto white balance setting is AWB2 and the display mode setting is DISP1.

[0031] FIG. 7 is a schematic diagram in which the auto white balance setting is AWB1 and the display mode setting is DISP2.

[0032] FIG. 8 is a schematic diagram in which the auto white balance setting is AWB2 and the display mode setting is DISP2.

[0033] Although the ambient light information and subject to be imaged are not limited, for the sake of simplicity, this embodiment will be described assuming that an image of a white subject is captured under ambient light with a color temperature of 3000 K. The processing in Fig. 4 is realized by the CPU 103 of the image capturing device controlling each unit of the device in accordance with various programs and input signals.

[0034] First, the image capturing apparatus 100 acquires the auto white balance setting (S401). The auto white balance setting is a setting for determining which control method, AWB1 or AWB2, is to be used, and is specified by the user via the operation unit 109.

[0035] Next, the imaging device 100 acquires a display mode setting (S402). The display mode setting is a setting for determining which control method, DISP1 or DISP2, is to be used, and is specified by the user via the operation unit 109.

[0036] Next, the image processing device 105 acquires ambient light information using the method described above (S403). Figures 5(a), 6(a), 7(a), and 8(a) are conceptual diagrams showing the ambient light of a color temperature of 3000K in this embodiment, expressed as the RGB color ratio (color balance). Because it is ambient light, the ratio remains the same regardless of the auto white balance setting or display mode setting.

[0037] Next, the CPU 103 references the acquired auto white balance setting and switches processing depending on the setting (S404). If the auto white balance setting is AWB1, the above-described AWB1 white balance adjustment is performed (S405). If the auto white balance setting is AWB2, the above-described AWB2 white balance adjustment is performed (S406). The color ratios after performing auto white balance are shown in Figures 5(b), 6(b), 7(b), and 8(b). Figures 5(b) and 7(b) use AWB1, so the color ratios are adjusted so that R, G, and B are equal. In contrast, Figures 6(b) and 8(b) use AWB2, so the colors remain as they appear to the eye.

[0038] Next, the image processing device 105 applies image processing such as gamma conversion and color conversion to the image signal after the processing of S405 or S406 (S407). Figures 5(c), 6(c), 7(c), and 8(c) are conceptual diagrams of the RGB color ratios of the image data after image processing.

[0039] Next, as a process for live view display, the CPU 103 switches the process depending on the combination of the display mode setting and the auto white balance setting (S408, S409, S410).

[0040] When the auto white balance setting is AWB1 and the display mode setting is DISP1, the CPU 103 brings the color temperature of the white point of the display unit 108 closer to the color temperature of the ambient light (S411). In this embodiment, the color temperature of the ambient light is set to 3000 K, so the CPU 103 controls the display unit 108 so that the color temperature of the white point of the display unit 108 is 3000 K. By controlling in this manner, it is possible to bring the color of a white subject displayed on the display unit 108 closer to the color of a white subject as seen by the human eye, as shown in FIG. 5(e).

[0041] When the auto white balance setting is AWB2 and the display mode setting is DISP1, the CPU 103 sets the color temperature of the white point of the display unit 108 to 6500K. In this embodiment, when AWB2 is selected, the processed image data will appear closer to the actual appearance when displayed on a display device with a color temperature of 6500K (S412). By controlling in this manner, it is possible to make the color of a white subject displayed on the display unit 108 closer to the color of a white subject as seen by the human eye, as shown in FIG. 6(e).

[0042] In the case of DISP2, the CPU 103 keeps the white point of the display unit 108 at the standard 6500K for display (S413, S414).

[0043] In this embodiment, when AWB1 is selected, if the human eye is adapted to ambient light of 3000K, the subject displayed on the display unit 108 will appear more bluish than when viewed directly with the human eye. However, this type of control is used because it is important to allow the user to confirm the difference between AWB1 and AWB2 when switching the auto white balance setting.

[0044] This concludes the description of the processing in this embodiment. In this way, in this embodiment, the color of the display unit is adjusted according to the white balance adjustment method. This makes it possible to reduce the sense of discomfort felt by the user viewing the live view.

[0045] <Other embodiments> The present invention can also be realized by executing the following process: That is, software (programs) that realize the functions of the above-described embodiments are supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the programs.

Claims

1. An imaging means; a setting means for setting a first auto white balance mode or a second auto white balance mode, which have different white balance adjustment methods; a white balance adjustment unit that adjusts the white balance of an image signal based on the auto white balance mode set by the setting unit; a recording control means for recording the image signal adjusted by the white balance adjustment means in a recording unit; a display control means for displaying the image signal adjusted by the white balance adjustment means on a display unit; and acquiring means for acquiring ambient light information, the display control means displays, on the display unit, an image signal obtained by adjusting the color balance of the image signal adjusted by the white balance adjustment means, based on the auto white balance mode set by the setting means and the ambient light information acquired by the acquisition means; the display control means is capable of adjusting the color balance of the image signal adjusted by the white balance adjustment means differently depending on whether the auto white balance mode set by the setting means is the first auto white balance mode or the second auto white balance mode.

2. 2. The imaging device according to claim 1, wherein the display control means adjusts the color balance by changing a white point of the display unit.

3. 3. The imaging device according to claim 1, wherein whether or not the color balance is adjusted by the display control means is set based on a user's operation.

4. 4. The imaging device according to claim 1, wherein the auto white balance modes include a first auto white balance mode in which adjustment is made so that the color of the light source becomes white, and a second auto white balance mode in which adjustment is made so that the color of the light source becomes the color seen by the user.

5. 5. The imaging device according to claim 1, wherein the display control means adjusts the color balance of an image displayed as a live view.

6. An imaging process; a setting step of setting a first auto white balance mode or a second auto white balance mode, which have different white balance adjustment methods; a white balance adjustment step of adjusting the white balance of an image signal based on the auto white balance mode set in the setting step; a recording step of recording the image signal adjusted in the white balance adjustment step in a recording unit; a display step of displaying the image signal adjusted in the white balance adjustment step on a display unit; and an acquisition step of acquiring ambient light information, In the display step, an image signal obtained by adjusting the color balance of the image signal adjusted in the white balance adjustment step is displayed on the display unit based on the auto white balance mode set in the setting step and the ambient light information acquired in the acquisition step. a display step of adjusting the color balance of the image signal adjusted in the white balance adjustment step, the display step being capable of adjusting the color balance of the image signal adjusted in the white balance adjustment step, depending on whether the auto white balance mode set in the setting step is the first auto white balance mode or the second auto white balance mode.

7. A computer-executable program that causes a computer to function as each of the means of the imaging device according to any one of claims 1 to 5.

Images