Image capture apparatus

Abstract

To adjust a white balance more suitably.SOLUTION: An image capture apparatus includes: an acquisition unit which acquires image data of a subject; an identification unit which identifies a color temperature of light from the subject on the basis of the image data of the subject acquired by the acquisition unit; an adjustment unit which adjusts a white balance of the image data of the subject on the basis of the color temperature identified by the identification unit; and a suppression unit which suppresses, when the image data includes image data of an LED light emitting body illuminated by a floodlight, a chroma of the image data white-balance-adjusted by the adjustment unit, on the basis of the color temperature identified by the identification unit.SELECTED DRAWING: Figure 2

Description

Incorporation by Reference

[0001] This application claims priority from Japanese Patent Application No. 2017-233354, filed on December 5, 2017, the contents of which are incorporated herein by reference. [Technical Field]

[0002] The present invention relates to an imaging device, an electronic device, and a saturation reduction program. [Background technology]

[0003] There is a technology that detects LED (Light Emitting Diode) light sources and performs color correction based on the type of LED light source detected. When using this technology to shoot in a stadium or similar, the colors of the LED signboards, which are LED light sources, are different from those of the ambient light, so if AWB (Auto White Balance) is applied to the ambient light, the image of the LED signboard will be colored and the color tone will not be natural. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-119780 Summary of the Invention

[0005] An imaging device that is one aspect of the technology disclosed in the present application includes an acquisition unit that acquires image data of a subject, an identification unit that identifies the color temperature of light from the subject based on the image data of the subject acquired by the acquisition unit, an adjustment unit that adjusts the white balance of the image data of the subject based on the color temperature identified by the identification unit, and a suppression unit that, if the image data of the subject includes image data of an LED light source illuminated by a lighting lamp, suppresses the saturation of the image data of the subject whose white balance has been adjusted by the adjustment unit based on the color temperature identified by the identification unit.

[0006] An electronic device that is one aspect of the technology disclosed in the present application has an acquisition unit that acquires image data of a subject, an identification unit that identifies the color temperature of light from the subject based on the image data of the subject acquired by the acquisition unit, a first adjustment unit that adjusts the color of the image data of the subject based on the color temperature identified by the identification unit, and a second adjustment unit that, when the image data of the subject includes image data of an LED light source illuminated by a lighting lamp, adjusts the color of the image data of the subject that has been adjusted by the first adjustment unit based on color information of the LED light source that has been adjusted by the adjustment unit.

[0007] In addition, the saturation suppression program, which is one aspect of the technology disclosed in the present application, causes a processor to execute an acquisition process to acquire image data of a subject, a specification process to identify the color temperature of light from the subject based on the image data of the subject acquired by the acquisition process, an adjustment process to adjust the white balance of the image data of the subject based on the color temperature identified by the specification process, and, if the image data of the subject includes image data of an LED light source illuminated by a lamp, a suppression process to suppress the saturation of the image data of the subject whose white balance has been adjusted by the adjustment process based on the color temperature identified by the specification process.

[0008] In addition, the saturation reduction program, which is another aspect of the technology disclosed in the present application, causes a processor to execute an acquisition process to acquire image data of a subject, a specification process to identify the color temperature of light from the subject based on the image data of the subject acquired by the acquisition process, a first adjustment process to adjust the color of the image data of the subject based on the color temperature identified by the specification process, and, if the image data of the subject includes image data of an LED light source illuminated by a lighting lamp, a second adjustment process to adjust the color of the image data of the subject adjusted by the first adjustment process based on the color information of the specific light source adjusted by the adjustment process. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing an example of a photographed scene. [Figure 2] FIG. 2 is an explanatory diagram showing an example of saturation suppression. [Figure 3] FIG. 3 is a block diagram showing an example of the hardware configuration of the imaging device. [Figure 4] FIG. 4 is a block diagram showing an example of the functional configuration of the imaging device. [Figure 5] FIG. 5 is an explanatory diagram showing the correspondence between color spaces and color temperatures. [Figure 6] FIG. 6 is an explanatory diagram showing the white balance adjustment and saturation suppression processing when the color of the ambient light is blue. [Figure 7] FIG. 7 is an explanatory diagram showing white balance adjustment and saturation suppression processing when the color of the ambient light is green (blue-green). [Figure 8] FIG. 8 is an explanatory diagram showing white balance adjustment and saturation suppression processing when the color of the ambient light is amber. [Figure 9] FIG. 9 is a flowchart 1 showing an example of a still image capturing process procedure by the imaging device of this embodiment. [Figure 10] FIG. 10 is a time chart showing an example of frame readout when a high-speed sampling rate is set for video shooting. [Figure 11] FIG. 11 is a flowchart 1 showing an example of a procedure for capturing a moving image using the imaging device of this embodiment. [Figure 12] FIG. 12 is an explanatory diagram showing an example of the system configuration of the imaging system. [Figure 13] FIG. 13 is a flowchart 2 showing an example of a still image capturing process procedure by the imaging device of this embodiment. [Figure 14] FIG. 14 is a flowchart 2 showing an example of a still image capturing process procedure by the imaging device of this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Shooting scene example> FIG. 1 is an explanatory diagram showing an example of a shooting scene. FIG. 1 shows an example of a scene in which a soccer player P playing soccer in an outdoor stadium 100 is being shot. The stadium 100 has a floodlight 101, a field (ground) 102, and an LED signboard 103. At night, the floodlight 101 is turned on, making it possible to play soccer. The floodlight 101 may be, for example, a mercury lamp, a metal hydride lamp, or a high-pressure sodium lamp. The floodlight 101 is a non-LED light source.

[0011] The LED signboard 103 has its light-emitting surface 103a facing the field 102 and is placed in a position visible to spectators watching the soccer game. The LED signboard 103 emits light in daylight or cool white with a color temperature of approximately 4500 to 5800 [K]. Advertisements expressed as text and figures are displayed on the light-emitting surface 103a, and the background of the advertisement may be white to make the advertisement easier for spectators to see.

[0012] FIG. 2 is an explanatory diagram showing an example of saturation suppression. Table 200 shows the relationship between the color of ambient light, the color temperature of ambient light, and saturation suppression processing. When the color of ambient light from the illuminating lamp 101 is blue, its color temperature is approximately 7000 to 8000 [K]. When the color of ambient light from the illuminating lamp 101 is green (blue-green), its color temperature is approximately 6000 to 7000 [K]. When the color of ambient light from the illuminating lamp 101 is amber, its color temperature is approximately 4000 [K].

[0013] 1, assume that an imaging device 110 such as a digital camera or digital video camera captures an image of soccer player P with the LED signboard 103 in the background, and obtains image data 201. The image data 201 is data representing an image in which soccer player P is the main subject, with the LED signboard 103 and spectators in the background.

[0014] In this case, the imaging device 110 performs white balance adjustment to make the white color of the light-emitting surface 103a of the LED signboard 103 whiter, but is affected by the ambient light from the illuminating lamp 101. Therefore, if the ambient light is blue, the image of the LED signboard 103, which should be white, appears amber. Also, if the ambient light is green (blue-green), the image of the LED signboard 103, which should be white, appears magenta. Similarly, if the ambient light is amber, the image of the LED signboard 103, which should be white, appears blue.

[0015] Therefore, the imaging device 110 reduces the saturation of the image of the LED sign 103, which has been colored amber, magenta, or blue by white balance adjustment, and converts it to white. This conversion process is called saturation reduction processing. (B) is image data 202 that has been subjected to saturation reduction processing from image data 201 before saturation reduction processing in (A). This makes it possible to optimize the white balance adjustment. Here, converting to white refers to processing that brings the color of the LED sign 103 closer to its original white color.

[0016] <Example of Hardware Configuration of Imaging Device 110> 3 is a block diagram showing an example of the hardware configuration of the imaging device 110. The imaging device 110 includes a lens unit 301, a shutter 302, a drive control circuit 303, an imaging element 304, an image processing circuit 305, a photometric sensor 306, a strobe 307, an input device 308, a processor 309, a memory 310, a recording medium 311, a display device 312, an acceleration sensor 313, a geomagnetic sensor 314, a communication IF, and a GPS receiver.

[0017] The drive control circuit 303, image processing circuit 305, input device 308, processor 309, memory 310, recording medium 311, acceleration sensor 313, geomagnetic sensor 314, communication IF (interface) 315, and GPS receiver 316 are connected to a bus 317.

[0018] The lens unit 301 has a condenser lens 301a, an aperture 301b, and a focus lens 301c. The condenser lens 301a condenses light from the subject and emits it toward the image sensor 304. The aperture 301b adjusts the amount of light from the condenser lens 301a. The focus lens 301c focuses on the subject by moving the focus lens 301c in the optical axis direction. The shutter 302 allows light from the lens unit 301 to pass through to the image sensor 304 only during exposure time and blocks light at other times.

[0019] The drive control circuit 303 controls the drive of the lens unit 301 and the shutter 302. Specifically, for example, the drive control circuit 303 adjusts the opening and closing of the diaphragm 301b, controls the movement of the focus lens 301c in the optical axis direction, and controls the opening and closing of the shutter 302.

[0020] The image sensor 304 photoelectrically converts light from a subject that has passed through the lens unit 301 and the shutter 302, and outputs an image signal as RAW data to the image processing circuit 305. The image sensor 304 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The image processing circuit 305 performs various image processing such as demosaic processing, noise removal, gradation correction, color correction, face detection, white balance adjustment, color temperature specification processing, saturation suppression processing, encoding processing, and decoding processing on the RAW data from the image sensor 304. The image processing circuit 305 is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array), for example.

[0022] The photometric sensor 306 is a sensor that measures the amount of light from the subject. The strobe 307 is a device that generates a flash of light. The photometric sensor 306 measures the amount of light reflected from the subject.

[0023] The input device 308 is a device that is operated by the user to give a trigger to the image capture device 110. The input device 308 includes, for example, a release button, a dial for setting a shooting mode, a button, and a touch panel.

[0024] The processor 309 controls the entire imaging device 110. The processor 309 also executes part of the image processing circuit 305 as a saturation suppression program. The saturation suppression program is stored in a volatile memory 310 or a non-volatile recording medium 311.

[0025] The memory 310 functions as a work area for the processor 309. The recording medium 311 is detachable from the image capturing device 110 and records various data such as the image data 201 and 202. The recording medium 311 may be built into the image capturing device 110. The memory 310 and the recording medium 311 are collectively referred to as a "recording device." A storage device is a non-temporary or temporary recording medium that stores various programs and data.

[0026] The display device 312 is a device that displays images (including still images, videos, and live view images) obtained by processing RAW data from the image sensor 304 in the image processing circuit 305, as well as various types of information. RAW data is also a type of image data. The display device 312 is a monitor provided on the back of the image capture device 110. The display device 312 also includes an electronic viewfinder.

[0027] The acceleration sensor 313 detects acceleration in three orthogonal axes, namely the X, Y, and Z directions, to detect the tilt of the image capture device 110. The geomagnetic sensor 314 is a sensor that detects the direction in which the image capture device 110 is facing. The acceleration sensor 313 and the geomagnetic sensor 314 can identify the direction in which the image capture device 110 is facing and the degree of tilt at which it is facing.

[0028] The communication IF 315 is a communication device that transmits and receives data. The GPS receiver 316 receives signals from GPS satellites and acquires the current position information (latitude and longitude information) of the image capture device 110.

[0029] <Example of functional configuration of imaging device 110> 4 is a block diagram showing an example of the functional configuration of the image capture device 110. The image capture device 110 has an acquisition unit 401, an identification unit 402, an adjustment unit 403, a suppression unit 404, and a detection unit 405. Specifically, the acquisition unit 401, the identification unit 402, the adjustment unit 403, the suppression unit 404, and the detection unit 405 are realized, for example, by causing the image processing circuit 305 shown in FIG. 3 or the processor 309 to execute a saturation suppression program recorded in a recording device.

[0030] The acquisition unit 401 acquires image data of a subject. Specifically, for example, the acquisition unit 401 acquires image data of a subject photographed under a specific light source, for example, under the illuminating lamp 101 shown in Fig. 1. This image data is image data 201 including an image of an LED signboard 103 as a specific light-emitting body illuminated by the illuminating lamp 101, as shown in Fig. 2(A).

[0031] The determination unit 402 determines the color temperature of light from the subject based on the image data acquired by the acquisition unit 401. Specifically, for example, the determination unit 402 divides the RGB image data acquired from the image sensor 304 into multiple blocks (one block is a group of one or more pixels) and converts each block into YCbCr image data represented by a luminance signal (Y component) and color difference signals (Cb component (blue) and Cr component (red)). Then, the determination unit 402 determines the color temperature corresponding to the color difference signals.

[0032] The luminance and color difference signals of each block may be the luminance and color difference signals of a pixel at a specific position (for example, the center or the upper left) within the group of pixels in the block, or may be the average, median, maximum, or minimum value of the luminance and color difference signals of the group of pixels in the block.

[0033] FIG. 5 is an explanatory diagram showing the correspondence between color spaces and color temperatures. Here, color space 500 is a coordinate system defined by color-difference signals Cb and Cr. In FIG. 5, the horizontal axis represents Cb and the vertical axis represents Cr. The direction from left to right on the horizontal axis Cb is the +Cb direction, and the direction from right to left is the -Cb direction. Similarly, the direction from bottom to top on the vertical axis Cr is the +Cr direction, and the direction from top to bottom is the -Cr direction (the same applies to FIGS. 6 to 8, which will be described later). In color space 500, the first quadrant represents magenta, the second quadrant represents amber, the third quadrant represents green, and the fourth quadrant represents blue. Note that the closer a color is to an adjacent quadrant, the more the color blends with the colors of the adjacent quadrants. Origin O represents achromatic color.

[0034] A color temperature curve 501 is set in the color space 500. The color temperature curve is a curve that indicates a preset color temperature, and has contour lines 502 that indicate the same color temperature in the direction that intersects with the curve. The color space 500 is recorded in a recording device in advance. If the color space 500 is not recorded in the recording device, the image processing circuit 305 or the processor 309 may refer to the color space 500 from a computer that has the color space 500 via the communication IF 315.

[0035] Once the color difference signals Cb, Cr of the image data 201 are identified, the identification unit 402 plots the color difference signals Cb, Cr in a color space 500 to identify saturations 503, 504 in the color space 500. The saturations of the image data 201 are plotted in the color space 500 for each block, but for simplicity of explanation, two saturations 503, 504 will be used as an example.

[0036] The identification unit 402 refers to a color temperature curve 501 set in a color space 500 and identifies a color temperature corresponding to saturation 503, 504. As a result, the identification unit 402 identifies an image area in the image data 201 with saturation 503 as a non-LED area that is the main subject (soccer player P) and the background (the spectators and field 102 excluding the LED signboard 103), and identifies an image area in the image data 201 with saturation 504 as the image area of ​​the LED signboard 103.

[0037] Returning to FIG. 4 , the adjustment unit 403 adjusts the white balance of the image data 201 acquired by the acquisition unit 401. The adjustment unit 403 normally adjusts the white balance so that white appears white. In such normal white balance adjustment, as described above, when the image data 201 is acquired, the adjustment unit 403 converts the image of the LED sign 103 to amber if the ambient light is blue, converts the image of the LED sign 103 to magenta if the ambient light is green (blue-green), and converts the image of the LED sign 103 to blue if the ambient light is amber, due to the influence (color temperature) of the ambient light from the illumination lamp 101.

[0038] 5, the adjustment unit 403 performs white balance adjustment on the image data 201, whereby saturation 503 (green in the third quadrant) in the color space 500 of the non-LED area is translated to the origin O indicating achromatic color by a correction amount x in the +Cb direction and a correction amount y in the +Cr direction, thereby becoming saturation 505. Similarly, saturation 504 in the color space 500 of the image area of ​​the LED signboard 103 is translated in the same way as the translation of saturation 503 to the origin O, following the white balance adjustment of saturation 503. Therefore, saturation 504 is converted to magenta in the first quadrant and becomes saturation 506, and the image of the white LED signboard 103 is colored magenta.

[0039] Returning to FIG. 4 , when image data 201 includes image data of a specific light-emitting body, the suppression unit 404 suppresses the saturation of the image data adjusted by the adjustment unit 403 based on the color temperature identified by the identification unit 402. Specifically, for example, when image data 201 includes image data of an LED signboard 103, the suppression unit 404 suppresses the saturation of the image data adjusted by the adjustment unit 403 based on the color temperature of the LED signboard 103 identified by the identification unit 402. More specifically, when the light from the subject has a color temperature indicating blue, green, or amber, the suppression unit 404 suppresses the saturation of the adjusted image data 201. Specific examples of the saturation suppression process performed by the suppression unit 404 will be described with reference to FIGS. 6 to 8.

[0040] 6 is an explanatory diagram showing white balance adjustment and saturation suppression processing when the color of the ambient light is blue. In FIG. 6, saturation 503 of the non-LED area is located in the blue fourth quadrant, and saturation 504 of the LED signboard 103 is also located in the blue fourth quadrant. Through the white balance adjustment by the adjustment unit 403, saturation 503 of the non-LED area is translated to the position of origin O (achromatic color) and becomes saturation 505. Similarly, saturation 504 of the LED signboard 103 follows the white balance adjustment of saturation 503 of the non-LED area and translates in the same way as saturation 503 of the non-LED area is translated to origin O.

[0041] Therefore, the saturation 504 of the LED sign 103 is converted to amber in the second quadrant to become saturation 506, and the image of the white LED sign 103 is colored amber. The suppression unit 404 then adjusts the values ​​of the color difference signals Cb and Cr so that the saturation 506 of the amber-colored LED sign 103 becomes the achromatic color of the origin O, resulting in a saturation of 600. This makes it possible to extract the amber from the image of the amber-colored LED sign 103 and convert it to white.

[0042] 7 is an explanatory diagram showing white balance adjustment and saturation suppression processing when the color of the ambient light is green (blue-green). In FIG. 7, saturation 503 of the non-LED area is located in the green of the third quadrant, and saturation 504 of the LED signboard 103 is located in the blue of the fourth quadrant. Through the white balance adjustment by the adjustment unit 403, saturation 503 of the non-LED area is translated to the position of origin O (achromatic color) and becomes saturation 505. Similarly, saturation 504 of the LED signboard 103 follows the white balance adjustment of saturation 503 of the non-LED area and translates in the same way as saturation 503 of the non-LED area is translated to origin O.

[0043] Therefore, the saturation 504 of the LED sign 103 is converted to magenta in the first quadrant to become saturation 506, and the image of the white LED sign 103 is colored magenta. The suppression unit 404 then adjusts the values ​​of the color difference signals Cb and Cr so that the saturation 506 of the magenta-colored LED sign 103 becomes the achromatic color of the origin O, resulting in a saturation of 600. This makes it possible to extract magenta from the image of the magenta-colored LED sign 103 and convert it to white.

[0044] 8 is an explanatory diagram showing white balance adjustment and saturation suppression processing when the color of the ambient light is amber. In FIG. 8, saturation 503 of the non-LED area is located in amber in the second quadrant, and saturation 504 of the LED signboard 103 is located in blue in the second quadrant. Through white balance adjustment by the adjustment unit 403, saturation 503 of the non-LED area is translated to the position of origin O (achromatic color) and becomes saturation 505. Meanwhile, saturation 504 of the LED signboard 103 follows the white balance adjustment of saturation 503 of the non-LED area and translates in the same way as saturation 503 of the non-LED area is translated to origin O.

[0045] Therefore, the saturation 504 of the LED sign 103 moves within the fourth quadrant to become saturation 506, and the image of the white LED sign 103 becomes blue. The suppression unit 404 then adjusts the values ​​of the color difference signals Cb and Cr so that the saturation 506 of the blue-tinted LED sign 103 becomes the achromatic color of the origin O, resulting in a saturation of 600. This makes it possible to extract blue from the image of the blue-tinted LED sign 103 and convert it to white.

[0046] In this way, the suppression unit 404 can remove the colors of ambient light (amber, magenta, blue) from the image area of ​​the LED signboard 103 by shifting only the saturation 506 of the image area of ​​the LED signboard 103 that has become colored due to the white balance adjustment by the adjustment unit 403 parallel to the origin O.

[0047] Furthermore, the position of the saturation 504 in the color space 500 identifies which image area in the image data 201 is the image of the LED signboard 103. Therefore, the suppression unit 404 can perform saturation suppression processing only on the image area of ​​the LED signboard 103.

[0048] Returning to FIG. 4, the detection unit 405 detects flicker F caused by a specific light-emitting body included in the subject from image data of the subject (for example, a live view image). Flicker F refers to a fine flickering phenomenon that occurs on a display. The specific light-emitting body is, for example, an LED signboard 103, which is a signboard that uses an LED as its light source. LEDs have a greater difference in flickering brightness than fluorescent lights, and the flickering cycle depends on the frequency of the AC power supply (50 Hz or 60 Hz). For example, if the frequency of the AC power supply is 50 Hz, the flickering cycle is the reciprocal of doubling the frequency (50 Hz), i.e., 1 / 100 sec.

[0049] For example, in rolling shutter control, the detection unit 405 sets the frame rate of the image sensor 304 so that the charge accumulation time for each line of the image sensor 304 is equal to or greater than the blinking cycle (1 / 100 [sec]) and is n times (n is an integer equal to or greater than 1) the blinking cycle (1 / 100 [sec]) (high-speed frame rate setting). As a result, the charge accumulation time for each line of the image sensor 304 differs from the cycle corresponding to the blinking cycle (1 / 100 [sec]), causing flicker F. Therefore, the detection unit 405 can detect flicker F.

[0050] In addition, the trigger for setting a high frame rate may be, for example, when photographing still images, when the power of the imaging device 110 is turned on and the capture of live view images begins or at any time thereafter, or when the user half-presses the release button.

[0051] In this embodiment, the detection of flicker F may be performed using existing technology. Furthermore, in this embodiment, rather than detecting flicker F and suppressing the flicker, the detection of flicker F is used to identify the presence of an LED light source such as the LED signboard 103. For this reason, the imaging device 110 detects the presence of an LED light source by intentionally creating a state in which flicker F can be detected using the detection unit 405.

[0052] Furthermore, since flicker F may also be detected from a background other than the LED signboard 103, for example, if the group of blocks from which saturation 504 has been obtained has a specific shape (for example, a horizontally long rectangle), the detection unit 405 may detect it as the LED signboard 103. This improves the accuracy of detecting the LED signboard 103.

[0053] The detection unit 405 detects flicker F using the image sensor 304, but may also use the photometry sensor 306. This makes it possible to detect an LED light source within a subject by detecting flicker F even when a live view image is not captured.

[0054] When the detection unit 405 detects flicker F, the suppression unit 404 determines that the LED signboard 103 is present within the subject. Therefore, when the white balance is adjusted by the adjustment unit 403, the image area of ​​the LED signboard 103 will be colored amber, magenta, or blue as described above. Therefore, the suppression unit 404 suppresses the saturation of the image data 201 adjusted by the adjustment unit 403 based on the color temperature identified by the identification unit 402, as shown in FIGS. 6 to 8 , to obtain image data 202.

[0055] <Still image shooting process procedure 1> 9 is a flowchart 1 showing an example of a still image capturing process procedure by the image capturing device 110 of this embodiment. Fig. 9 shows an example of a still image capturing process procedure when flicker F is detected by the detection unit 405. First, the image capturing device 110 determines whether or not it is in a specific capturing mode (step S901).

[0056] The specific shooting mode is a shooting mode that executes auto white balance and saturation suppression processing. The specific shooting mode is set, for example, by the user operating the input device 308 of the image capture device 110. If the specific shooting mode is not selected (step S901: No), the image capture device 110 will capture an image in another shooting mode (step S902).

[0057] On the other hand, if the specific shooting mode is selected (step S901: Yes), the image capturing device 110 sets the frame rate of the image sensor 304 to a high speed (high frame rate setting) (step S903). This makes it possible to detect flicker F when the subject includes an LED light source.

[0058] Next, the imaging device 110 detects flicker F using the detection unit 405 (step S904) and determines whether the release button has been pressed (step S905). If the release button has not been pressed (step S905: No), the process returns to step S904. If the release button has been pressed (step S905: Yes), the imaging device 110 acquires image data (step S906). If flicker F has been detected for the subject at the time the release button was pressed (step S904: Yes), horizontal stripe-like unevenness will be present in the acquired image data 201, and if flicker F has not been detected for the subject at the time the release button was pressed (step S904: No), horizontal stripe-like unevenness will not be present in the acquired image data.

[0059] Next, as shown in Fig. 5, the imaging device 110 uses the identification unit 402 to identify the color temperature of light from the subject based on the acquired image data (step S907). Then, as shown in Fig. 5, the imaging device 110 uses the adjustment unit 403 to adjust the white balance of the image data based on the identified color temperature (step S908). Thereafter, the imaging device 110 determines whether or not saturation reduction processing should be performed by the reduction unit 404 (step S909).

[0060] Specifically, for example, if flicker F is detected in step S904 (step S904: Yes), an LED light source is present within the subject. Therefore, the image of the LED signboard 103, which is an LED light source, will be colored by the white balance adjustment in step S908, so the image capture device 110 determines that saturation suppression processing should be performed (step S909: Yes). On the other hand, if flicker F is not detected (step S904: No), the LED signboard 103 is not present within the subject, so the image capture device 110 determines that saturation suppression processing should not be performed (step S909: No) and proceeds to step S911.

[0061] If it is determined that saturation suppression processing should be performed (step S909: Yes), the imaging device 110 causes the suppression unit 404 to perform the saturation suppression processing (step S910), as shown in Figures 6 to 8, and proceeds to step S911. On the other hand, if it is determined that saturation suppression processing should not be performed (step S909: No), the imaging device 110 proceeds to step S911 without performing the saturation suppression processing (step S910).

[0062] In step S911, the imaging device 110 stores the image data 202 in a recording device or displays it as an image on the display device 312 (step S911). When the saturation reduction process (step S910) is executed, for example, the image data 201 shown in FIG. 2A is converted into image data 202 in which the saturation of the image of the LED signboard 103 is reduced, as shown in FIG. 2B. This reduces the saturation of the color added to the image of the LED signboard 103, making it possible to reproduce the original color of the subject.

[0063] <Saturation reduction in video shooting> Next, we will explain how to suppress saturation when shooting moving images. The coloring of the image of the LED signboard 103 described above occurs not only when shooting still images but also when shooting moving images. Therefore, we will explain how to detect flicker F when shooting moving images, that is, how to detect LED light sources within the subject.

[0064] Figure 10 is a time chart showing an example of frame readout when a high-speed sampling rate is set for video shooting. In Figure 10, (A) shows the time axis for the start of exposure or frame readout, and (B) shows the time axis for the end of frame readout, i.e., the timing for recording to the recording device. The horizontal axis is the time axis.

[0065] When capturing moving images, the shooting process for recording one frame includes the steps from the start of exposure to the start of frame readout and frame recording. In this way, when capturing moving images, each frame is recorded to the recording device, and image capture device 110 uses the free time ST between the recording of a frame and the start of exposure for the next frame to quickly read out frames for detecting flicker F. Specifically, for example, when recording a frame, image capture device 110 sets the frame rate so that the charge accumulation time for each line of image capture element 304 is equal to or greater than the blinking cycle (1 / 100 [sec]) and is n times (n is a natural number) the blinking cycle (1 / 100 [sec]) (high-speed frame rate setting).

[0066] Furthermore, at the start of the next exposure, the frame rate is returned to the original frame rate for capturing moving images. By setting the high frame rate during the free time ST, the image capture device 110 causes the detection unit 405 to read out frames for detecting flicker F at high speed and detect flicker F from the frames.

[0067] <Video recording process step 1> FIG. 11 is a flowchart 1 showing an example of a moving image capturing process procedure by the image capturing device 110 of this embodiment. FIG. 11 shows an example of a moving image capturing process procedure when flicker F is detected by the detection unit 405. First, the image capturing device 110 determines whether exposure has started (step S1101). If exposure has not started (step S1101: No), the process proceeds to step S1112. In step S1111, the image capturing device 110 determines whether moving image capturing has ended (step S1112). If moving image capturing has ended (step S1111: Yes), the process ends this flowchart. If moving image capturing has not ended (step S1111: No), the process returns to step S1101.

[0068] If exposure is started in step S1101 (step S1101: Yes), the image capturing device 110 waits for a frame to be recorded (step S1102: No). If a frame has been recorded in the recording device (step S1102: Yes), the image capturing device 110 sets the frame rate of the image sensor 304 to a high speed (high frame rate setting) (step S1103). This causes flicker F to occur if the subject includes an LED light source.

[0069] Next, the image capturing device 110 detects flicker F by referring to the frames read after setting the frame rate to high speed (step S1104). If flicker F is not detected (step S1104: No), the process proceeds to step S1110. As a result, if flicker F is not detected, the processes of steps S1105 to S1109 can be omitted, thereby reducing the processing load during video capture processing.

[0070] If flicker F is detected in step S1104 (step S1104: Yes), an LED light source is present within the subject. Therefore, the image capture device 110 acquires the frame recorded in the recording device in step S1102: Yes (step S1105), and, as shown in FIG. 5, the identification unit 402 identifies the color temperature of the light from the subject based on the acquired frame (step S1106).

[0071] Then, the image capturing device 110 adjusts the white balance of the frame based on the identified color temperature by the adjustment unit 403 (step S1107), as shown in Fig. 5. Thereafter, the image capturing device 110 executes saturation suppression processing by the suppression unit 404, as shown in Figs. 6 to 8 (step S1108).

[0072] Then, the image capturing device 110 updates the frame acquired in step S1105 to the frame after saturation suppression (step S1109), and then returns the frame rate of the image capturing element 304 to the original rate (step S1110), and the process returns to step S1101.

[0073] In this way, even when shooting a moving image, by utilizing the free time ST between recording a frame and starting exposure of the next frame, it is possible to detect flicker F, that is, to detect an LED light source within a subject. Note that in FIG. 11, steps S1105 to S1109 are omitted if flicker F is not detected, but white balance adjustment may be performed to ensure consistency with the case where flicker F is detected. As a result, the white balance is adjusted for all frames regardless of whether flicker F is detected, and a moving image with reduced sense of incongruity between consecutive frames can be obtained.

[0074] <Example of using 3D model data for Stadium 100> Next, an example of using the 3D model data of the stadium 100 will be described. The 3D model data of the stadium 100 is data that models the 3D structure of the stadium 100, and models objects such as the field 102, lighting fixtures 101, LED signs 103, and spectator seats. Each object has 3D position information. By using the 3D model data of the stadium 100, it is possible to detect LED light sources within the subject when capturing a still image or video, without using flicker F detection.

[0075] 12 is an explanatory diagram showing an example of the system configuration of an imaging system. The imaging system is composed of a server, a router, and an imaging device 110. The imaging device 110 is communicably connected to the server via the router over a LAN (Local Area Network) or the Internet. The server stores 3D model data of the stadium 100. The router is installed in the stadium 100, for example, and communicably connected to the communication IF 315 of the imaging device 110 of a user visiting the stadium 100.

[0076] In this case, the imaging device 110 acquires image data (frames in the case of a video) by the acquisition unit 401 based on the imaging direction of the imaging device 110 in the facility identified by the current location information of the imaging device 110 and the three-dimensional model data of the facility including the specific light-emitting body in the current location information and the light source that illuminates the specific light-emitting body.

[0077] Current location information of the imaging device 110 is acquired by the GPS receiver 316 shown in FIG. 3. The specific light-emitting body is, for example, the LED sign 103 described above, the light source illuminating the specific light-emitting body is the illuminating lamp 101 described above, and the facility is the stadium 100 described above. The imaging direction of the imaging device 110 in the facility is acquired by the acceleration sensor 313 and geomagnetic sensor 314 shown in FIG. 3. The acceleration sensor 313 and geomagnetic sensor 314 can identify the direction and the tilt at which the imaging device 110 is facing. Therefore, by using the three-dimensional model data, the imaging device 110 can acquire the imaging direction from its current location.

[0078] <Still image shooting process procedure 2> Fig. 13 is a flowchart 2 showing an example of a still image capturing process procedure by the imaging device 110 of this embodiment. Fig. 13 shows an example of a still image capturing process procedure when using three-dimensional model data. Note that the same processes as in Fig. 9 are assigned the same step numbers, and their explanations will be omitted.

[0079] First, the imaging device 110 acquires current location information of the imaging device 110 using the acquisition unit 401 (step S1301). Next, the imaging device 110 acquires 3D model data of the facility (stadium 100) at its current location from the server (step S1302). The imaging device 110 uses the 3D model data of the facility to identify the shooting direction, which is the orientation of the lens at the current location of the imaging device 110 (step S1303). Then, the imaging device uses the 3D model data of the facility to estimate whether or not an LED signboard 103 is present within the subject in the shooting direction identified in step S1303 (step S1304).

[0080] Thereafter, if the release button is not pressed (step S905: No), the process returns to step S1301, and if the release button is pressed (step S905: Yes), the image capturing device 110 executes steps S906 to S911 shown in Fig. 9. This makes it possible to achieve saturation suppression without detecting flicker F from the subject.

[0081] <Video recording process step 2> Fig. 14 is a flowchart 2 showing an example of a still image capturing process procedure by the imaging device 110 of this embodiment. Fig. 14 shows an example of a video capturing process procedure when using three-dimensional model data. Note that the same processes as in Fig. 11 are assigned the same step numbers, and their explanations will be omitted.

[0082] In step S1102, if a frame has been recorded (step S1102: Yes), the imaging device 110 acquires current location information of the imaging device 110 using the GPS receiver 316, which is an example of the acquisition unit 401 (step S1401). Note that the imaging device 110 may acquire location information of the connected router 1202 from the router 1202 as the current location information of the imaging device 110.

[0083] Next, the imaging device 110 acquires 3D model data of the facility (stadium 100) at its current location from the server (step S1402). The imaging device 110 uses the 3D model data of the facility to identify the shooting direction, which is the orientation of the lens at the current location of the imaging device 110 (step S1403). The imaging device then uses the 3D model data of the facility to determine whether the LED signboard 103 is within the subject of the shooting direction identified in step S1303 (step S1404).

[0084] If there is no LED signboard 103 (step S1404: Yes), the process returns to step S1101. As a result, if no flicker F is detected, the processes of steps S1105 to S1109 can be omitted, thereby reducing the processing load during video shooting processing.

[0085] If it is determined in step S1404 that there is an LED signboard 103 (step S1404: Yes), the imaging device 110 executes the processes of steps S1105 to S1109 and returns to step S1101.

[0086] In this way, even if flicker F is not detected, LED light sources within a subject can be detected by utilizing the free time ST between recording a frame and starting exposure of the next frame during video shooting. Note that in FIG. 14, steps S1105 to S1109 are omitted if the LED signboard 103 is not detected, but white balance adjustment may be performed to ensure consistency with the case where the LED signboard 103 is detected. This adjusts the white balance for all frames regardless of whether the LED signboard 103 is detected, resulting in a video with a consistent look.

[0087] (1) As described above, the imaging device 110 includes an acquisition unit 401 that acquires image data of a subject, an identification unit 402 that identifies the color temperature of light from the subject based on the image data acquired by the acquisition unit 401, an adjustment unit 403 that adjusts the white balance of the image data based on the color temperature identified by the identification unit 402, and a suppression unit 404 that suppresses the saturation of the image data adjusted by the adjustment unit 403 when the image data includes image data of a specific light-emitting body (e.g., LED sign 103).

[0088] As a result, even if a specific light-emitting element that should have been adjusted to white by white balance adjustment is not adjusted to white and ends up being colored, the color can be removed by suppressing saturation, thereby optimizing the white balance adjustment.

[0089] Furthermore, when the image data includes image data of a specific light-emitting body (for example, the LED signboard 103), the suppression unit 404 may suppress the saturation of the image data adjusted by the adjustment unit 403 based on the color temperature identified by the identification unit 402, for example, when the light from the subject has a color temperature indicating blue, green, or amber. This makes it possible to suppress the saturation in the case of the identified color temperature.

[0090] (2) In the above (1), the image data includes an LED light source (for example, an LED signboard 103) as a specific light source illuminated by the lamp 101.

[0091] As a result, even if the LED light source, which should normally be adjusted to white by white balance adjustment, is not adjusted to white and ends up colored, the color can be removed by suppressing saturation, allowing you to get closer to the original color of the subject.

[0092] (3) In addition, in the above (1), the suppression unit 404 suppresses the saturation of the adjusted image data when the light from the subject has a color temperature that indicates blue, green, or amber.

[0093] If the light from the subject is blue, the image data of the specific light-emitting device (for example, LED sign 103) will be colored amber by white balance adjustment. If the light from the subject is green (or blue-green), the image data of the specific light-emitting device (for example, LED sign 103) will be colored magenta by white balance adjustment. If the light from the subject is amber, the image data of the specific light-emitting device (for example, LED sign 103) will be colored blue by white balance adjustment. In this way, saturation suppression processing can remove these colors from specific light-emitting devices that are colored amber, magenta, or blue after white balance adjustment, thereby optimizing the white balance adjustment.

[0094] (4) In the above (1), the suppression unit 404 suppresses the saturation of the image data of a specific light-emitting body (for example, the LED signboard 103) from among the adjusted image data.

[0095] By narrowing down the target of saturation suppression to image data of specific light-emitting bodies 301b, it is possible to suppress the influence on other image data of the soccer players P, spectators, the field 102, etc., which have already undergone white balance adjustment.

[0096] (5) In (1) above, the acquisition unit 401 acquires image data based on the imaging direction of the imaging device 110 in a facility identified by the current position of the imaging device 110 and three-dimensional model data of the facility (e.g., stadium 100) including a specific light-emitting object (e.g., LED sign 103) at the current position and a light source (e.g., lighting lamp 101) that illuminates the specific light-emitting object.

[0097] As a result, the user only needs to point the imaging device 110 at the subject, and the imaging device 110 can estimate the presence of a specific light-emitting object within the subject, thereby realizing saturation suppression processing for the specific light-emitting object. Furthermore, the facility is not limited to the stadium 100, as long as it is equipped with LED light sources such as the illuminating lamps 101 and the LED signboards 103. Furthermore, the facility is not limited to being outdoors, and may be an indoor facility (such as a gymnasium or arena) as long as it is equipped with LED light sources such as the illuminating lamps 101 and the LED signboards 103.

[0098] (6) In the above (1), the imaging device 110 has a detection unit 405 that detects flicker F caused by a specific light-emitting body (e.g., LED signboard 103) included in the subject from the image data of the subject, and when flicker F is detected by the detection unit 405, the suppression unit 404 suppresses the saturation of the image data adjusted by the adjustment unit 403 based on the color temperature identified by the identification unit 402.

[0099] The detection of flicker F is not used to suppress image flicker, but is used to identify the presence of a specific light-emitting object. For this reason, the image capture device 110 can detect the presence of a specific light-emitting object by intentionally creating a state in which flicker F can be detected using the detection unit 405. Furthermore, for image capture devices 110 that are equipped with a flicker F detection function for suppressing image flicker, the flicker F detection function can be diverted, allowing for a reduction in the number of parts, cost, and size.

[0100] (7) In the above (6), the detection unit 405 starts the operation of detecting flicker F when a predetermined operation input (for example, half-pressing the release button) is made.

[0101] This allows detection of flicker F, that is, detection of a specific light-emitting object, at the shooting timing intended by the user.

[0102] (8) In (6) above, the imaging device 110 is an imaging device 110 that captures moving images, and the detection unit 405 detects flicker F from the image data after the image data (frame) has been recorded and before exposure for recording the next image data of the image data is started (for example, during idle time ST).

[0103] This makes it possible to suppress the saturation of frames while shooting (recording) a moving image. Although the frame is updated in step S1109 in Fig. 11 and Fig. 14, the frame recorded in step S1102 may be retained and the frame after saturation suppression may also be recorded. This makes it possible to save both the original moving image and the moving image after white balance adjustment and saturation suppression.

[0104] (9) In the above (6), when the detection unit 405 detects flicker F, the identification unit 402 identifies the color temperature of the light from the subject based on the image data.

[0105] As a result, the color temperature of a frame is identified only when flicker F is detected. In other words, color temperature identification is omitted for frames in which flicker F is not detected, which reduces the processing load when shooting moving images.

[0106] Although the imaging device 110 described above has been described as a digital camera or a digital video camera, it may also be a smartphone or tablet equipped with a camera, a game console, an augmented reality head mount, a personal computer, or an in-vehicle drive recorder. Furthermore, the lens unit 301 and the imaging element 304 may be built into the imaging device 110, or may be detachable from the imaging device 110.

[0107] Furthermore, the saturation reduction program may be recorded on a recording medium (such as a CD-ROM) that can be read by the processor 309, in addition to the recording device of the image capture device 110. In this case, the recording medium is inserted into a personal computer connected to the image capture device 110, and the saturation reduction program is installed in the image capture device 110 via the personal computer.

[0108] The saturation reduction program may be recorded in a downloadable manner on a server that can communicate with the image capture device 110. Specifically, for example, the image capture device 110 downloads the saturation reduction program from the server. This allows the image capture device 110 to execute the processes according to the saturation reduction program, such as the white balance adjustment, saturation reduction processing, and flicker F detection described above. [Explanation of symbols]

[0109] 100 stadium, 101 lighting, 102 field, 103 LED sign (LED light source), 103a light-emitting surface, 110 imaging device, 201, 202 image data, 301 lens unit, 304 imaging element, 305 image processing circuit, 306 photometric sensor, 308 input device, 309 processor, 310 memory, 311 recording medium, 312 display device, 313 acceleration sensor, 314 geomagnetic sensor, 316 GPS receiver, 401 acquisition unit, 402 identification unit, 403 adjustment unit, 404 suppression unit, 405 detection unit, 500 color space, 501 color temperature curve, 502 contour line, 503 to 506, 600 saturation, F flicker, ST free time

Claims

[Claim 1] an acquisition unit that acquires image data of a subject; an identification unit that identifies a color temperature of light from the subject based on image data of the subject acquired by the acquisition unit; an adjustment unit that adjusts the white balance of the image data of the subject based on the color temperature specified by the specification unit; a suppression unit that suppresses the saturation of the image data of the subject, the white balance of which has been adjusted by the adjustment unit, based on the color temperature specified by the specification unit, when the image data of the subject includes image data of an LED light emitter illuminated by an illumination lamp; An imaging device having the above configuration.

Images