Image processing device, image processing method, and program

Abstract

To provide an image processing device which can control white balance to proper white balance even in removal of an infra-red cut-off filter (IRCF).SOLUTION: An image processing device comprises: detection means which detects the influence of infrared light to a color of an input image; calculation means which calculates a white balance control value on the basis of the input image; and control means which controls timing for calculating the white balance control value for the input image. The control means performs control of calculating the white balance control value at the first timing at which the detection result of the detection means changes and the second timing being at least one timing in a period in which it is detected that the detection means is influenced by the infrared light.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technology for processing images captured by an imaging device. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known an imaging device that performs white balance (hereinafter referred to as WB where appropriate) control while an imaging sensor captures infrared light, and outputs a color image.

[0003] Color images are typically designed assuming that only visible light is captured, i.e., that an infrared cutoff filter (IRCF) is inserted on the optical axis to block infrared light. However, some imaging devices have a mode that improves the sensitivity of the image sensor by removing the IRCF from the optical axis to capture infrared light, such as when shooting in low-light environments. However, as mentioned above, because the color of the imaging device is designed not to capture infrared light, capturing infrared light distorts the color of the color image. Specifically, the captured image appears reddish compared to when infrared light is not captured. In other words, the color reproducibility of the imaging device is reduced.

[0004] In response to this, Patent Document 1 discloses a technology for switching the white balance control method depending on the position of the IRCF. According to Patent Document 1, depending on the position of the IRCF, white balance control is switched between performing white balance control so that the ratio of the integrated values ​​of the RGB (red, green, blue) components of the entire screen becomes a pre-stored ratio, and performing white balance control so that the ratio becomes 1:1:1. This improves the color reproducibility of the imaging device.

[0005] Furthermore, with the recent increase in the sensitivity of image sensors, it has become possible to capture visible light components even if the exposure is lowered by the amount of infrared light captured when the IRCF is removed, making it less likely that color information of the subject will be lost when the IRCF is removed. In other words, the increased sensitivity of image sensors has made it easier to capture images that combine visible and infrared light components. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-130317 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the conventional technology disclosed in the above-mentioned patent document, when the infrared cut filter (IRCF) is removed, the white balance is likely to change if there is a change in the subject or light source. As a result, the white balance may change significantly during continuous shooting, which may cause image quality problems. In particular, in use cases such as surveillance cameras that continuously shoot a specific subject under specific conditions for a long period of time, if the white balance changes significantly, the color of the same subject may differ significantly depending on the timing of the shot, which may create an unnatural impression.

[0008] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image processing device that is capable of controlling the white balance to an appropriate level even when the IRCF is removed. [Means for solving the problem]

[0009] The image processing device of the present disclosure includes: Inserting and removing the infrared cut filter a detecting means for detecting a white balance; and a calculating means for calculating a white balance control value based on an input image. a measuring means for measuring the elapsed time from the timing when the white balance control value is calculated by the calculating means; The timing for calculating the white balance control value is decision and a control means for controlling the detection means. The removal of the infrared light cut filter is detected byThe first timing and a plurality of second timings based on the elapsed time measured by the measuring means after the first timing; and the white balance control value but calculation And As if and the control means determines the second timing so that the time elapsed from the timing at which the white balance control value is calculated becomes longer each time the white balance control value is calculated. It is characterized by: [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an imaging device that can control the white balance to an appropriate level even when the IRCF is removed. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a configuration diagram of an image processing device according to a first embodiment. [Figure 2] 1 is a flowchart showing a part of a processing flow of the present invention. [Figure 3] FIG. 4 is a diagram illustrating an example of white balance gain control in the first embodiment. [Figure 4] FIG. 10 is a configuration diagram of an image processing device according to a second embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of white balance gain control in the second embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of white balance gain control in the second embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of white balance gain control in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] The following embodiments do not limit the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). In addition, the present invention may be configured by appropriately combining parts of each of the embodiments described below. In the following embodiments, the same components are described with the same reference numerals.

[0014] First Embodiment FIG. 1 is a block diagram showing an example of the configuration of an image processing apparatus according to the first embodiment.

[0015] An image processing device according to a first embodiment will be described below with reference to Fig. 1. The image processing device of this embodiment is assumed to be a device built into or connected to an imaging device such as a digital camera or a surveillance camera.

[0016] The input image is an image captured by an imaging unit consisting of a lens and an imaging sensor (not shown). The input image is image data (or image signal) consisting of multiple pixels and includes information on multiple colors. The multiple colors are, for example, red (R), green (G), and blue (B). The image data is data corresponding to the amount of light that passes through color filters corresponding to each color provided on the imaging sensor (not shown) and is converted into an electrical signal by the imaging sensor. The color filters transmit not only visible light corresponding to red, green, and blue, but also some infrared light (invisible light). Therefore, typical imaging devices are equipped with an infrared cut-off filter (IRCF) to remove infrared light components, thereby obtaining an image that is closer to human vision. The imaging sensor is composed of imaging elements such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device).

[0017] The output image is an image in which the white balance has been appropriately corrected by multiplying the pixel values ​​of the input image by a white balance gain, which is the final white balance control value obtained as described below. In the following description, white balance will be referred to as WB where appropriate. As will be described in detail later, the image processing device of this embodiment obtains an output image in which the white balance has been appropriately corrected by performing white balance control (WB control) depending on whether the input image is affected by infrared light captured by the imaging sensor.

[0018] The feature acquisition unit 101 acquires feature amounts related to the color of the input image and outputs the feature amounts to the gain calculation unit 102. More specifically, when the input image is divided into a plurality of rectangular areas, the feature acquisition unit 101 acquires color information for each rectangular area determined by image data contained in each rectangular area. The color information is, for example, a representative value of the color difference signal for each rectangular area, and the representative value is, for example, an average value or a mode value.

[0019] The gain calculation unit 102 calculates a white balance control value according to the feature amount of the input image at a timing instructed by the gain calculation unit 102. For example, the gain calculation unit 102 acquires color information for each region from the feature amount acquisition unit 101 and calculates a representative value of the color information of the input image. Then, at a timing instructed by the timing control unit 105, the gain calculation unit 102 calculates a white balance gain (hereinafter referred to as WB gain) as a white balance control value such that the representative value of the color information of the output image becomes a predetermined target value. In this embodiment, the WB gain used for white balance control is, for example, a red gain (hereinafter referred to as Red gain) that adjusts the redness of the image and a blue gain (hereinafter referred to as Blue gain) that adjusts the blueness of the image. Information about the WB gain calculated by the gain calculation unit 102 is sent to the gain multiplication unit 106. Furthermore, at a timing other than the timing instructed by the timing control unit 105, the gain calculation unit 102 sends to the gain multiplication unit 106 a WB gain calculated at a timing instructed by the timing control unit 105 in the past.

[0020] The infrared light detection unit 103 detects the influence of infrared light on the color of the input image. That is, the infrared light detection unit 103 detects whether the color of the input image is influenced by infrared light captured by the image sensor, and outputs the detection result to the timing control unit 105. For example, when an IRCF (not shown) is inserted on the optical axis of the lens of the imaging unit, the infrared light detection unit 103 detects that the color of the input image is not influenced by infrared light. On the other hand, when the IRCF is not inserted on the optical axis of the lens of the imaging unit (removed from the optical axis), the infrared light detection unit 103 detects that the color of the input image is influenced by infrared light.

[0021] The illuminance change detection unit 104 detects changes in illuminance in the environment in which the input image was captured. That is, the illuminance change detection unit 104 acquires current illuminance information and detects a change in illuminance when the difference between the current illuminance information and the illuminance information at the timing when the infrared light detection unit 103 detects the effect of infrared light on the color of the input image is equal to or greater than a predetermined value. The illuminance change detection unit 104 then outputs the detection result to the timing control unit 105. The illuminance information may be calculated based on the brightness of the input image, calculated based on the exposure conditions, or calculated based on the read value of an illuminance sensor (not shown). Alternatively, the illuminance information may be the magnitude of bias in the distribution of color information for each region of the input image. This is because the lower the illuminance, the fewer color components of the subject there are, and the distribution of color information is biased toward specific color regions.

[0022] The timing control unit 105 instructs the timing for calculating the WB gain to the gain calculation unit 102. That is, the timing control unit 105 determines the timing for calculating the WB gain based on the detection result of the change in illuminance and the detection result of the influence of infrared light, and outputs that timing to the gain calculation unit 102. The timing control unit 105 outputs to the gain calculation unit 102 the timing at which the presence or absence of the influence of infrared light switches, and the timing at which the presence of the influence of infrared light is detected and the presence of a change in illuminance is detected, as the timing for calculating the WB gain.

[0023] The gain multiplication unit 106 acquires the WB gain from the gain calculation unit 105, and generates and outputs an output image on which white balance control has been performed by multiplying the input image by the WB gain.

[0024] Fig. 2 is a flowchart showing an example of the flow of the main parts of image processing executed by the image processing device of this embodiment. An example of image processing according to this embodiment will be described below with reference to the flowchart of Fig. 2. In the following description of the flowchart, processing steps will be represented by the symbol "S".

[0025] In step S01, the feature amount acquiring unit 101 acquires a feature amount related to the color of the input image.

[0026] In step S02, the infrared light detection unit 103 detects whether or not the color of the input image is affected by the infrared light captured by the image sensor.

[0027] In step S03, the illuminance change detection unit 104 detects a change in illuminance in the environment in which the input image was captured.

[0028] In step S04, the timing control unit 105 determines whether there has been a change in the effect of infrared light on the color of the input image. If it is determined that there has been a change, the process proceeds to step S07. If it is determined that there has not been a change, the process proceeds to step S05.

[0029] In step S05, if the infrared light detection unit 103 detects the effect of infrared light on the color of the input image, the timing control unit 105 advances the process to step S06. On the other hand, if the infrared light detection unit 103 does not detect the effect of infrared light on the color of the input image, the timing control unit 105 advances the process to the end.

[0030] In step S06, if the illuminance change detection unit 104 detects a change in illuminance, the timing control unit 105 advances the process to step S07. On the other hand, if the illuminance change detection unit 104 does not detect a change in illuminance, the timing control unit 105 advances the process to the end.

[0031] In step S07, the WB gain calculation unit 102 calculates a white balance gain (WB gain) based on the feature amount acquired in step S01, and the process ends.

[0032] The white balance gain control and its effects in this embodiment will be described below.

[0033] FIG. 3 is a diagram showing an example of WB control, which is an example of white balance gain control in this embodiment.

[0034] FIG. 3 shows an example of changes in illuminance and the effect of infrared light on the color of a captured image in a lighting environment before and after sunset as day turns to night, and corresponding WB gain control according to this embodiment.

[0035] As shown in FIG. 3 , illuminance decreases from dusk until sunset. After a sufficient amount of time has passed since sunset, the illuminance drops to its lowest level and stabilizes. During this time, when the illuminance falls below a predetermined value L1, the IRCF is removed to compensate for the lack of sensitivity of the image sensor. In the WB gain control of this embodiment, a first white setting process (white setting process 1) is performed when the illuminance falls below the predetermined value L1 and the IRCF is removed. The white setting process calculates a WB gain based on the captured image at the time the process is performed and fixes the WB gain to the calculated WB gain. Therefore, as shown in FIG. 3 , even if the WB gain was changed according to the captured image by the auto white balance (AWB) process before the IRCF was removed, the WB gain is fixed to WB1 after the IRCF is removed. WB1 is the WB gain calculated when the IRCF was removed, i.e., when the illuminance fell below the predetermined value L1. Furthermore, in the WB gain control of this embodiment, a second white setting process (white setting process 2) is performed when the illuminance becomes smaller than a predetermined value L2. That is, after the illuminance becomes smaller than the predetermined value L2, the WB gain is fixed to WB2, which is the WB gain calculated when the illuminance became smaller than the predetermined value L2.

[0036] When an IRCF is removed and the influence of infrared light on a captured image changes, the color of the captured image changes significantly, resulting in a significant change in the appropriate white balance. The image processing device of this embodiment performs white setting processing when the IRCF is removed, applying a WB gain that adjusts the white balance to an appropriate level in response to significant changes in the color of the captured image. Furthermore, because the WB gain is fixed, stable white balance control can be achieved regardless of the color of the captured object, even when capturing an image of a color-biased subject. Furthermore, if the illuminance of the shooting environment changes after the IRCF is removed, the balance between visible light and infrared light is disrupted, resulting in a change in the optimal white balance. If the WB gain is fixed in this situation, the device will be unable to adjust to the environmental change, resulting in a loss of white balance. The image processing device of this embodiment performs white setting processing again when it detects a change in the illuminance of the shooting environment after the IRCF is removed. By performing this processing in this manner, it is possible to maintain an appropriate white balance in response to changes in illuminance of the shooting environment after the IRCF is removed, and stable white balance control can be achieved regardless of the color of the subject.

[0037] The illuminance change detection unit 104 sequentially calculates (measures) temporal changes in illuminance after the time when the infrared light detection unit 103 detects the effect of infrared light on the color of the input image. Then, it may detect that the illuminance has stabilized to a constant value (approximately a constant value) and output the detection result to the timing control unit 105. When the illuminance has stabilized to a constant value, the appropriate white balance also becomes constant, so that appropriate white balance control can be achieved by performing white setting processing and fixing the WB gain when the illuminance has stabilized to a constant value.

[0038] <Second embodiment> The following describes an image processing apparatus according to the second embodiment. In the embodiment already described, the timing of performing the white setting process is controlled based on whether there is an influence of infrared light and whether there is a change in illuminance.

[0039] In contrast, the second embodiment controls the timing of white setting processing based on the presence or absence of the influence of infrared light and the elapsed time since the previous white setting processing was executed. Therefore, in the second embodiment, appropriate white balance control can be performed according to the speed of change in the lighting environment.

[0040] 4 is a block diagram showing an example of the functional configuration of an image processing apparatus according to the second embodiment. Note that the same functional units as those in the first embodiment are denoted by the same reference numerals, and their description will be omitted.

[0041] The timing control unit 205 instructs the gain calculation unit 102 on the timing to calculate the WB gain. That is, the timing control unit 105 determines the timing to calculate the WB gain based on the detection result of the influence of infrared light and the time elapsed since the previous WB gain was calculated, and outputs that timing to the gain calculation unit 102. The timing control unit 105 outputs to the gain calculation unit 102 the timing when the presence or absence of the influence of infrared light switches and the timing when a predetermined time has elapsed since the previous WB gain was calculated, as the timing to calculate the WB gain.

[0042] The time lapse calculation unit 207 calculates the time that has elapsed since the timing at which the WB gain was calculated immediately before. That is, the time lapse calculation unit 207 acquires the timing at which the WB gain was calculated from the timing control unit 205, calculates the time that has elapsed since that timing, and, when the elapsed time is equal to or greater than a predetermined time, notifies the timing control unit 205 that the predetermined time has elapsed.

[0043] The white balance control of this embodiment will be described with reference to FIG. 5. As shown in FIG. 5, in the WB gain control of this embodiment, a first white setting process (white setting process 1) is performed when the IRCF is removed. The white setting process calculates a WB gain based on the captured image at the time the process is performed, and fixes the WB gain to the calculated WB gain. Therefore, as shown in FIG. 5, after the IRCF is removed, the WB gain is fixed to WB1. Furthermore, in the WB gain control of this embodiment, a second white setting process (white setting process 2) is performed when a time T has elapsed since the first white setting process was performed. That is, the WB gain is fixed to WB2, which is the WB gain calculated when the time T has elapsed since the first white setting process was performed.

[0044] The effect of this embodiment will be described below. As shown in Fig. 5, there is a short-term change in illuminance after removing the IRCF, and if the white balance is controlled in accordance with this short-term change in illuminance, the short-term change in WB gain may cause the white balance to become unstable, potentially resulting in an unnatural image quality.

[0045] Therefore, in this embodiment, the first white setting process is performed when the IRCF is removed, and the second white setting process is performed when a time T has elapsed since the first white setting process was performed. By performing the process in this manner, it is possible to prevent white balance from being controlled in response to short-term changes in illuminance, and to achieve stable white balance control.

[0046] The time lapse calculation unit 207 may acquire illuminance information of the shooting environment, calculate the time during which a continuous change in illuminance has continued since the timing at which the previous WB gain was calculated, and notify the timing control unit 205 of the time at which the continuous change in illuminance has continued for a predetermined period of time or more. Alternatively, the time lapse calculation unit 207 may detect a discontinuous change in illuminance, suspend accumulation of the elapsed time while the discontinuous change in illuminance is detected, and resume accumulation of the elapsed time after the discontinuous change in illuminance is no longer detected. By doing so, it is possible to control the white balance to be more stable.

[0047] After calculating the WB gain when the IRCF is removed, the timing control unit 205 may output the timing for calculating the WB gain to the gain calculation unit 102 at multiple times. When calculating the WB gain at multiple times in the IRCF-removed state, the elapsed time from the timing at which the immediately preceding WB gain was calculated, which is notified by the time elapsed time calculation unit 207, may be changed each time the WB gain is calculated.

[0048] White balance control when timing is controlled in this manner will be described with reference to FIG. 6 . As shown in FIG. 6 , in the WB gain control of this embodiment, a first white setting process (white setting process 1) is performed when the IRCF is removed. The white setting process calculates a WB gain based on the captured image at the time the process is performed, and fixes the WB gain to the calculated WB gain. Therefore, as shown in FIG. 6 , the WB gain is fixed to WB1 after the IRCF is removed. Then, in the WB gain control of this embodiment, a second white setting process (white setting process 2) is performed when time T1 has elapsed since the first white setting process was performed. That is, the WB gain is fixed to WB2, which is the WB gain calculated when time T1 has elapsed since the first white setting process was performed. Furthermore, in the WB gain control of this embodiment, a third white setting process (white setting process 3) is performed when time T2 has elapsed since the second white setting process was performed. That is, the WB gain is fixed to WB3, which is the WB gain calculated when time T2 has elapsed since the second white setting process was executed. Here, T2 is set to a time longer than T1. That is, the time interval between white setting processes is controlled to become longer as the number of times the white setting process is repeated increases, starting from the time the IRCF is removed.

[0049] By implementing this in this way, white balance control that follows rapid changes in illuminance can be achieved by performing white setting processing at relatively short intervals immediately after IRCF removal, i.e., during the time period around sunset when illuminance changes rapidly.On the other hand, after sunset, when illuminance changes slowly, white setting processing is performed at relatively long intervals, enabling stable white balance control.

[0050] <Third embodiment> Next, an image processing device according to a third embodiment will be described with reference to Fig. 7. In the white balance control according to the third embodiment, the range of possible WB gains is narrowed each time WB gain calculation (white setting process) is repeated. That is, the WB gain is controlled so as to be less likely to change each time WB gain calculation (white setting process) is repeated.

[0051] The functional configuration of the image processing device according to this embodiment is the same as that of the first and second embodiments, and therefore will not be illustrated. Also, explanations of the same functional units as those of the first and second embodiments will be omitted, and only functional units having functions different from those of the first and second embodiments will be explained below.

[0052] The gain calculation unit 302 performs control so that the range of possible WB gains (effective range of WB gains) becomes smaller each time it repeats gain calculation (white setting process) at the timing instructed by the timing control unit 105 / 205, counting from the time the IRCF is removed. If the calculated WB gain is within the effective range, the gain calculation unit 102 sends the calculated WB gain to the gain multiplication unit 106. On the other hand, if the calculated WB gain is not within the effective range, the gain calculation unit 102 does not send the calculated WB gain to the gain multiplication unit 106, but instead outputs the previously calculated WB gain to the gain multiplication unit 106, thereby maintaining the WB gain.

[0053] White balance control in this embodiment will be described with reference to FIG. 7. As shown in FIG. 7, when an IRCF is inserted, a WB gain (e.g., P1) within the effective range A1 of the WB gain is applied. Next, in the first white setting process when the IRCF is removed, a WB gain (e.g., P2) within the effective range A2 of the WB gain, which is narrower than A1, is applied. This makes it easier to apply a white balance appropriate for the amount of infrared light captured after the IRCF is removed. Then, in the second white setting process executed after the first white setting process, a WB gain within the effective range A3 of the WB gain, which is narrower than A2 and includes the WB gain P2, is applied. That is, if a WB gain (e.g., P3) outside the effective range A3 of the WB gain is calculated in the second white setting process, P3 is not applied, and the WB gain P2 is maintained.

[0054] By implementing this in this way, white balance control that follows large changes in illuminance is achieved by performing white setting processing within a relatively wide effective range of WB gain immediately after IRCF removal, i.e., during the time period before and after sunset when illuminance changes significantly. On the other hand, by performing white setting processing within a relatively narrow effective range of WB gain during the time period after sunset when illuminance changes less, fluctuations in WB gain are suppressed, enabling stable white balance control.

[0055] While the image processing device of the above-described embodiment is applied to an imaging device, the image processing device may also be implemented by an information processing device (computer), such as a personal computer or smartphone, connected to the imaging device. In this case, the imaging device outputs to the computer raw data captured by the imaging unit, shooting parameters indicating exposure time, frame rate, exposure value, etc., as well as information indicating whether an IRCF is being used, i.e., information indicating whether infrared light is affecting the color of the input image. Note that the information indicating the presence or absence of infrared light influence may be input by the user. The computer then performs image processing similar to that described in the above-described embodiment. The computer in this example executes software program code that implements the image processing of this embodiment. Although the hardware configuration is not illustrated, the computer implementing the image processing device of this embodiment is configured to include a CPU, ROM, RAM (random access memory), an auxiliary storage device, a display unit, an operation unit, a communication I / F, a bus, etc. The CPU uses computer programs and data stored in the ROM and RAM to control the entire computer and perform the above-mentioned white balance control, etc. Furthermore, the image processing device of this embodiment may also be configured to include one or more dedicated hardware components separate from the CPU, and the dedicated hardware may perform at least some of the processing performed by the CPU. Examples of dedicated hardware include ASICs (application-specific integrated circuits), FPGAs (field-programmable gate arrays), and DSPs (digital signal processors). ROM stores programs that do not require modification. RAM temporarily stores programs and data supplied from the auxiliary storage device, as well as data supplied from the outside via a communication interface. The auxiliary storage device is composed of a hard disk drive or the like and stores various data such as image data, shooting parameters, and information indicating the presence or absence of the influence of infrared light. The display unit is composed of, for example, an LCD display or LED display, and displays a GUI for the user to operate the image processing device. The operation unit is composed of, for example, a keyboard, mouse, joystick, touch panel, etc., and inputs various instructions to the CPU in response to user operations.The CPU also functions as a display control unit that controls the display unit and an operation control unit that controls the operation unit. The communication I / F is used for communication with devices external to the image processing device. For example, if the image processing device is further connected to an external device via a wired connection, a communication cable is connected to the communication I / F. If the image processing device has the function of wirelessly communicating with an external device, the communication I / F is equipped with an antenna. The bus connects each unit of the image processing device to transmit information. In this embodiment, the external device connected to the image processing device is the above-mentioned imaging device, another information processing device, etc. Furthermore, although the display unit and the operation unit are assumed to be located inside the image processing device, at least one of the display unit and the operation unit may be located outside the image processing device as a separate device. Furthermore, the image processing device does not necessarily have to include a display unit or an operation unit.

[0056] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0057] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0058] The above-described embodiments are merely examples of specific implementations of the present invention, and should not be construed as limiting the technical scope of the present invention. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0059] 101 Feature acquisition unit 102 Gain calculation unit 103 Infrared light detection unit 104 Illuminance change detection unit 105 Timing control section 106 Gain multiplication unit

Claims

1. a detection means for detecting insertion or removal of the infrared cut filter; a calculation means for calculating a white balance control value based on an input image; a measuring means for measuring the elapsed time from the timing when the white balance control value is calculated by the calculating means; a control unit that determines a timing at which the calculation unit calculates the white balance control value, the control means determines whether the white balance control value is calculated by the calculation means at a first timing when the detection means detects the removal of the infrared light cut filter and at a plurality of second timings based on the elapsed time measured by the measurement means after the first timing, and The control means determines the second timing so that the time elapsed from the timing at which the white balance control value is calculated becomes longer each time the white balance control value is calculated.

1. An image processing device comprising:

2. 2. The image processing apparatus according to claim 1, wherein the control means sets the timing when a predetermined time or more has elapsed since the first timing as the first second timing.

3. 3. The image processing device according to claim 1, wherein the calculation means outputs the white balance control value calculated based on the input image when the white balance control value calculated based on the input image is within a predetermined range, and outputs a previously output white balance control value as is when the white balance control value calculated based on the input image is not within the predetermined range, and narrows the predetermined range each time the white balance control value is calculated.

4. An image processing method executed by an image processing device, a detection step of detecting insertion or removal of the infrared cut filter; a calculation step of calculating a white balance control value based on an input image; a measuring step of measuring an elapsed time from the timing when the white balance control value is calculated in the calculating step; a control step of determining a timing for calculating the white balance control value in the calculation step, the control step determines whether the white balance control value is calculated in the calculation step at a first timing at which removal of the infrared light cut filter is detected in the detection step and at a plurality of second timings based on elapsed times measured in the measurement step after the first timing; In the control step, the second timing is determined so that the time elapsed from the timing at which the white balance control value is calculated becomes longer each time the white balance control value is calculated. An image processing method comprising:

5. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 3.

Images