Control device, control method, and program

Abstract

To appropriately control automatic switching of an imaging mode of an imaging apparatus.SOLUTION: A control device comprises: evaluation value acquisition means for acquiring an evaluation value to be used for switching an imaging mode of an imaging apparatus on the basis of an exposure amount based on a photometric result in a photometric region which is set to the imaging apparatus; luminance acquisition means for acquiring luminance in a predetermined region of an image which is picked up by the imaging apparatus; histogram acquisition means for acquiring a luminance histogram of the image; and determination means for determining whether to switch the imaging mode to a first imaging mode or a second imaging mode. The determination means determines whether to switch the imaging mode to the first imaging mode or the second imaging mode based on the evaluation value, a threshold for the evaluation value, the luminance in the predetermined region, and the luminance histogram.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control technique for an imaging device. [Background technology]

[0002] Most imaging devices, such as surveillance cameras (hereafter referred to as surveillance cameras), are equipped with a function that automatically inserts and removes an infrared filter (hereafter referred to as IRCF) into the optical path depending on the brightness of the surrounding environment. The imaging mode in which an IRCF is inserted into the optical path in bright environments is called day mode, while the imaging mode in which an IRCF is removed from the optical path in dark environments is called night mode. The function that automatically switches between these modes depending on brightness is called the auto day / night function. In particular, in night mode, removing the IRCF from the optical path increases the infrared component of the light entering the imaging element, allowing for a lower minimum subject illumination required for image capture than when an IRCF is inserted into the optical path. Note that imaging devices with auto day / night function output color images in day mode and black and white images in night mode.

[0003] Furthermore, in recent years, technologies for switching between day mode and night mode according to various circumstances have also been proposed. Patent Document 1 discloses a method for acquiring disturbance information and weighting exposure data used for switching between day mode and night mode. Patent Document 2 discloses a method for correcting a threshold value when switching between day mode and night mode according to image dodging processing. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-121028 [Patent Document 2] Japanese Patent Application Laid-Open No. 2014-176069 Summary of the Invention [Problem to be solved by the invention]

[0005] Recently, surveillance cameras have also been equipped with a function for adjusting exposure to an area designated by the user. However, if the user sets the surveillance camera to adjust exposure to an area designated by the user in a bright environment, such as daytime, the surveillance camera may not automatically switch to night mode even when the entire environment becomes dark at night.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to make it possible to appropriately control automatic switching of the imaging mode of an imaging device. [Means for solving the problem]

[0007] The control device of the present invention is characterized by having an evaluation value acquisition means for acquiring an evaluation value used to switch the imaging mode of the imaging device based on an exposure amount based on the photometric result of a photometric area set for the imaging device, a brightness acquisition means for acquiring the brightness of a predetermined area of ​​an image captured by the imaging device, a histogram acquisition means for acquiring a brightness histogram of the image, and a determination means for determining whether to switch the imaging mode to a first imaging mode or a second imaging mode different from the first imaging mode based on the evaluation value, a threshold value for the evaluation value, the brightness of the predetermined area, and the brightness histogram. [Effects of the Invention]

[0008] According to the present invention, it is possible to appropriately control automatic switching of the imaging mode of an imaging device. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional configuration of a control device according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of an imaging unit. [Figure 3] FIG. 2 is a diagram illustrating an example of a main hardware configuration of a control device. [Figure 4]4 is a flowchart of processing in the control device of the first embodiment. [Figure 5] 10A and 10B are diagrams used to explain the switching operation between a day mode and a night mode. [Figure 6] FIG. 10 is a diagram illustrating an example of a functional configuration of a control device according to a second embodiment. [Figure 7] 10 is a flowchart of a process in a control device according to a second embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a luminance histogram. [Figure 9] FIG. 10 is a diagram showing an example of a GUI for selecting a priority mode between the entire image and a specified region. [Figure 10] FIG. 10 is a diagram showing an example of a GUI for selecting the priority between the entire image and a specified region. [Figure 11] FIG. 10 is a diagram showing an example of a pattern of a luminance histogram when there are many high luminances. [Figure 12] FIG. 10 is a diagram showing an example of a pattern of a luminance histogram when there are many low luminances. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). Furthermore, in the following embodiments, the same or similar configurations and processing steps are designated by the same reference numerals, and redundant explanations will be omitted.

[0011] First Embodiment FIG. 1 is a diagram illustrating an example of the functional configuration of a control device 100 according to a first embodiment that controls an imaging device. FIG. 2 is a diagram illustrating an example of the configuration of an imaging unit 101, which is an imaging device controlled by the control device 100. The imaging unit 101 of this embodiment is an imaging device that has a function capable of switching at least between a first imaging mode and a second imaging mode different from the first imaging mode. The first imaging mode is, for example, a mode in which imaging is performed with a predetermined optical element inserted into the optical path, and the second imaging mode is a mode in which imaging is performed with the predetermined optical element removed from the optical path. In this embodiment, an infrared ray cut filter (IRCF) is used as the predetermined optical element. In this embodiment, the first imaging mode is a day mode in which imaging is performed in a bright environment with an IRCF inserted into the optical path, and the second imaging mode is a night mode in which imaging is performed in a dark environment with the IRCF removed from the optical path. That is, the imaging unit 101 of this embodiment has an auto day / night function that can automatically switch between day mode and night mode. The control device 100 of this embodiment outputs a color image when the imaging unit 101 is in day mode (first imaging mode), and outputs a black and white image when the imaging unit 101 is in night mode (second imaging mode). Note that in this embodiment, a camera for surveillance (surveillance camera) is taken as an example of an application of the imaging unit 101 and the control device 100, but they can also be applied to digital still cameras, digital video cameras, and the like for purposes other than surveillance.

[0012] Before describing each functional configuration in Fig. 1, a schematic configuration of the imaging unit 101 will be described with reference to Fig. 2. As shown in Fig. 2, the imaging unit 101 is configured to include a lens 201, a filter unit 202, an image sensor 203, a CDS circuit 204, an AGC circuit 205, and an A / D conversion circuit 206. Note that Fig. 2 shows only the main components of the imaging unit 101, and the imaging unit 101 also includes components (not shown) that are included in a typical imaging device. The lens 201 is an imaging optical system including several lens groups, an aperture, etc., and takes in light from the outside and forms an image of a subject on the imaging surface of the imaging element 203 .

[0013] The filter unit 202 includes an optical element having predetermined optical characteristics and an insertion / removal switching mechanism that enables the optical element to be inserted or removed from the optical path from the lens 201 to the image sensor 203. In this embodiment, the optical element having predetermined optical characteristics is an infrared cutoff filter (IRCF) that removes wavelengths in the infrared region of light incident on the image sensor 203 via the lens 201. The human eye's color vision characteristic, which is its sensitivity to color, and its relative luminous efficiency characteristic, which is its sensitivity to brightness, are in the visible range of 380 nm to 780 nm, and it has almost no sensitivity in wavelengths longer than 700 nm. Therefore, in the image sensor 101 of this embodiment, in a high-illuminance environment, such as daytime, an IRCF is inserted into the optical path to block light in the near-infrared region, thereby adjusting the color reproducibility of the captured image to the color vision characteristics of the human eye. On the other hand, in a low-illuminance environment, such as nighttime, the image sensor 101 removes the IRCF from the optical path to allow light in the near-infrared region to pass, thereby increasing sensitivity during imaging. In this way, the imaging unit 101 of this embodiment has an insertion / removal switching function that inserts the IRCF into the optical path in a high-illumination environment, and removes the IRCF from the optical path in a low-illumination environment.

[0014] The insertion / removal switching function of the image capture unit 101 of this embodiment automatically switches between insertion and removal of the IRCF based on the day / night switching determination result made by the determination unit 104, which will be described later in FIG. 1. That is, the image capture unit 101 removes the IRCF from the optical path when the determination unit 104 determines that the mode should be switched to night mode, and inserts the IRCF into the optical path when the determination unit 104 determines that the mode should be switched to day mode. An example of an insertion / removal switching mechanism for inserting and removing the IRCF in the image capture unit 101 is a configuration in which the IRCF is inserted and removed by driving a stepping motor. There are various other well-known configurations for the insertion / removal switching mechanism, and any of these configurations may be used in this embodiment, and detailed illustrations and descriptions of these configurations will be omitted.

[0015] The image sensor 203 is an image sensor such as a CCD or CMOS, and converts the subject image formed on the imaging surface by the lens 201 into an electrical signal. The CDS circuit 204 is a correlated double sampling (CDS) circuit that reduces noise in an image. The AGC circuit 205 is an automatic gain control (AGC) circuit that automatically controls the gain of the imaging unit 101 . The A / D conversion circuit 206 converts the analog signal output from the AGC circuit 205 into a digital signal. The image data converted into a digital signal by the A / D conversion circuit 206 is output to the exposure control unit 102 in FIG.

[0016] The control device 100 of this embodiment has a function of adjusting exposure to an area arbitrarily designated by the user in addition to controlling the auto day / night function of the imaging unit 101. Fig. 1 is a functional block diagram showing the main functional configuration of the control device 100 of this embodiment, which has a function of controlling the auto day / night function of the imaging unit 101 and a function of adjusting exposure to an area designated by the user.

[0017] 1, the imaging unit 101 is an imaging device having the configuration shown in FIG. 2, and outputs image data obtained by capturing an image of a subject or the like to the exposure control unit 102. Note that, hereinafter, unless otherwise specified, image data handled in the imaging unit 101 and the control device 100 will be simply referred to as an image. The image captured by the imaging unit 101 is sent from the exposure control unit 102 to the image processing unit 105 via the brightness acquisition unit 107 and the determination unit 104, which will be described later. Note that the image captured by the imaging unit 101 may also be sent to the image processing unit 105 via the evaluation value acquisition unit 103 and the determination unit 104, which will be described later.

[0018] The image processing unit 105 performs predetermined image processing on the input image (image data), such as color conversion, white balance processing, and gamma processing that performs gradation conversion on the signal-processed image. The image processing unit 105 is configured, for example, with a DSP (Digital Signal Processing) or the like. In this embodiment, the image processing unit 105 generates a color image from an image captured by the imaging unit 101 in day mode, and generates and outputs a black-and-white image from an image captured by the imaging unit 101 in night mode.

[0019] The output unit 106 outputs the image output from the image processing unit 105 to an external device via a communication network. Examples of the external device connected to the control device 100 via the communication network include a personal computer, a smartphone, and a tablet terminal. The image output from the output unit 106 is displayed on a display provided in the external device or recorded in a storage device provided in the external device.

[0020] The area setting unit 108 sets an area arbitrarily designated by the user in an image captured by the imaging unit 101 as a photometric area for determining a target value for exposure during imaging. That is, in the control device 100 of this embodiment, the position and size of the photometric area set by the area setting unit 108 can be arbitrarily changed in response to an instruction from the user. The photometric area may be the entire image or a local area. Furthermore, the area setting unit 108 can also set a photometric area for the entire image and a local photometric area in a time-division manner. The area setting unit 108 then notifies the exposure control unit 102 of the position and size of the set photometric area.

[0021] The exposure control unit 102 sets a target value of exposure when the imaging unit 101 captures an image of a subject, etc., based on the photometry result of the photometry area set by the area setting unit 108, and performs exposure control so that the imaging unit 101 captures an image with an exposure amount corresponding to the target value. The imaging unit 101 captures an image with an exposure amount of the target value by setting at least one of the imaging conditions, such as the aperture value, shutter speed, and AGC gain value, in accordance with the target value of exposure amount set by the exposure control unit 102. The exposure control unit 102 also outputs information indicating the imaging conditions when the imaging unit 101 captured an image to the evaluation value acquisition unit 103, and outputs the image captured by the imaging unit 101 to the luminance acquisition unit 107.

[0022] The evaluation value acquisition unit 103 calculates an evaluation value based on the exposure amount determined by the exposure control unit 102, which is used by the downstream determination unit 104 for day / night switching determination to switch the imaging unit 101 between day mode and night mode. That is, the evaluation value acquisition unit 103 calculates the evaluation value used for day / night switching determination based on the exposure amount determined by the exposure control unit 102 in accordance with the photometry result of the photometry area set by the area setting unit 108. As described above, the photometry area set by the area setting unit 108 may be the entire image or a local area. Therefore, when the entire screen is set as the photometry area, the evaluation value is calculated based on the exposure amount determined by the exposure control unit 102 in accordance with the photometry result of the entire screen. On the other hand, when a local photometry area is set, the evaluation value is calculated based on the exposure amount determined by the exposure control unit 102 in accordance with the photometry result of the local photometry area.

[0023] Furthermore, the imaging unit 101 determines the shooting conditions for capturing an image so that the target exposure amount determined by the exposure control unit 102 is achieved. For this reason, in the present embodiment, the evaluation value acquisition unit 103 calculates an evaluation value used for determining day / night switching based on the shooting conditions for the imaging unit 101 capturing an image in accordance with the target exposure amount set by the exposure control unit 102. Specifically, the evaluation value acquisition unit 103 calculates the evaluation value by performing the calculation of the following formula (1) based on the shooting conditions acquired by the exposure control unit 102 from the imaging unit 101, namely, the aperture value, shutter speed, and AGC gain value.

[0024] SE=I+SS+G Equation (1)

[0025] In equation (1), SE is the evaluation value, I is the aperture value, SS is the shutter speed, and G is the AGC gain value. The evaluation value SE is a value that indicates the brightness of the exposure, in other words, the EV (Exposure Value). The EV is the sum of the aperture value (F-number), shutter speed (seconds), and AGC gain value (dB), each expressed logarithmically.

[0026] Furthermore, when the auto day / night function for the imaging unit 101 is on, the evaluation value acquisition unit 103 always calculates an evaluation value from the current shooting conditions of the imaging unit 101. Note that when the aperture value, shutter speed, and AGC gain value of the imaging unit 101 each reach their controllable maximum or minimum values, the evaluation value acquisition unit 103 may acquire the luminance of the image captured by the imaging unit 101 as the evaluation value. Then, the evaluation value acquisition unit 103 outputs the evaluation value calculated as described above to the determination unit 104.

[0027] The determination unit 104 compares the evaluation value calculated by the evaluation value acquisition unit 103 with a day / night switching threshold (hereinafter referred to as the threshold), and based on the comparison result, performs day / night switching determination to determine whether the imaging mode of the imaging unit 101 should be set to day mode or night mode. Here, the photometric area set by the area setting unit 108 is set to the entire image or a local area, as described above, so it is desirable to set different appropriate thresholds when the entire screen is set as the photometric area and when a local photometric area is set. For this reason, the determination unit 104 has a threshold setting function that sets a threshold appropriate for the photometric results of the entire screen when the entire screen is set as the photometric area, and sets a threshold appropriate for the photometric results of the local photometric area when a local photometric area is set. The threshold can also be set arbitrarily by the user.

[0028] For example, the determination unit 104 compares the current evaluation value calculated by the evaluation value acquisition unit 103 with a threshold value, and if the evaluation value is equal to or less than the threshold value, the determination unit 104 determines that the imaging mode of the imaging unit 101 should be set to night mode. If the current imaging mode of the imaging unit 101 is night mode, the determination unit 104 maintains that night mode. On the other hand, if the evaluation value exceeds the threshold value, the determination unit 104 determines that the imaging mode of the imaging unit 101 should be switched to day mode, and requests the luminance acquisition unit 107 to perform luminance calculation processing when switching from night mode to day mode.

[0029] When a luminance calculation process is requested by the determination unit 104, the luminance acquisition unit 107 calculates the luminance of a predetermined region of the image captured by the imaging unit 101. At this time, the luminance acquisition unit 107 first divides the image captured by the imaging unit 101 into multiple blocks and calculates the luminance for each block. The luminance for each block calculated by the luminance acquisition unit 107 may be the average luminance within the block, a weighted average luminance weighted for each block, or weighted according to the positional relationship of other blocks within the image. The luminance acquisition unit 107 then calculates the luminance of a predetermined region of the image captured by the imaging unit 101 based on the luminance calculated for each block. In this embodiment, the luminance acquisition unit 107 calculates the average luminance of the entire image and the average luminance of the photometric region set by the region setting unit 108 as the luminance of the predetermined region. Note that if the exposure control unit 102 has only a single photometric region capable of photometry, the luminance acquisition unit 107 may alternately calculate the luminance of the entire image and the luminance of the photometric region using time-division processing. Then, the luminance acquisition unit 107 outputs information on the luminance calculation result to the determination unit 104.

[0030] The determination unit 104 determines whether the average luminance of the entire image output from the luminance acquisition unit 107 and the average luminance of the photometric region set by the region setting unit 108 are within predetermined ranges. For example, if the image is expressed in 8-bit gradation, the luminance value ranges from 0 to 255. For example, the determination unit 104 determines that the average luminance of the entire image and the photometric region set by the region setting unit 108 are within the predetermined range if the average luminance is within a range of, for example, 120 to 140. Alternatively, for example, the determination unit 104 may calculate the luminance difference between the average luminance of the entire image and the photometric region set by the region setting unit 108, and determine that the luminance difference is within the predetermined range if the difference is within, for example, ±10. In particular, by determining whether the evaluation value calculated by the evaluation value acquisition unit 103 based on the exposure based on the shooting conditions deviates from the evaluation value calculated when the entire image is used as the photometric region, more stable day / night switching determination is possible. In this embodiment, if the average luminance of the entire image and the photometric region set by the region setting unit 108 are each within a predetermined range, or if the luminance difference is within a predetermined range, the determination unit 104 determines that there is no luminance difference in the actual environment, and sets the imaging mode of the imaging unit 101 to day mode. On the other hand, if the average luminance is not within the predetermined range, or if the luminance difference is not within the predetermined range, the determination unit 104 determines that the brightness of the actual environment has not been calculated correctly, and therefore maintains the current imaging mode of the imaging unit 101. For example, if the current imaging mode of the imaging unit 101 is night mode, the determination unit 104 maintains the night mode.

[0031] The determination unit 104 then outputs the determination result of the day / night switching as described above to the imaging unit 101. As a result, in the imaging unit 101, the insertion / removal of the IRCF in the filter unit 202 is switched according to the determination result of the day / night switching by the determination unit 104. In this embodiment, in the night mode, the IRCF is removed from the optical path of the optical system in the image capture unit 101 to increase the sensitivity for capturing infrared components, but it is also possible not to remove the IRCF from the optical path. In this case, the image processing unit 105 may output a color image or a black and white image from the image captured by the image capture unit 101, depending on the day / night switching determination result by the determination unit 104.

[0032] Fig. 3 is a diagram showing an example of the hardware configuration of the control device 100 including the imaging unit 101 of this embodiment. Fig. 3 shows only the main components of the various components included in the control device 100, and the control device 100 also includes other hardware components (not shown) that are included in a typical surveillance camera.

[0033] The imaging unit 101 has the configuration shown in FIG. The CPU 311 executes various programs, including a control program according to the present embodiment, stored in the ROM 312. In this embodiment, the CPU 311 executes the control program to implement functions related to the exposure control unit 102, evaluation value acquisition unit 103, determination unit 104, brightness acquisition unit 107, and area setting unit 108 shown in FIG. 1 . The DSP 316 corresponds to the image processing unit 105 shown in FIG. 1 . In the case of a second embodiment, which will be described later with reference to FIG. 6 and subsequent figures, the control program also includes a program implementing the function of the histogram acquisition unit 109 shown in FIG. 6 . The control device 100 may also include one or more dedicated hardware components separate from the CPU 311, and at least some of the processing performed by the CPU 311 may be performed by the dedicated hardware components. Examples of the dedicated hardware include an ASIC (application-specific integrated circuit), an FPGA (field-programmable gate array), and a DSP (digital signal processor). The ROM 312 has a storage area for storing a boot program, the control program according to the present embodiment, and the shooting conditions and various parameters of the imaging unit 101. The RAM 313 is used as a main memory, a work area, and other storage area for the CPU 311. The flash memory 314 stores image data and the like.

[0034] Under the control of the CPU 311, the drive unit 315 drives the focus lens and diaphragm included in the lens 201 of the imaging unit 101, drives the insertion / removal switching mechanism of the filter unit 202, and so on. The communication unit 318 executes communication processing between the control device 100 and a communication network, and also operates as the output unit 106 in FIG. 1 . Note that communication by the communication unit 318 may be performed via a wired network such as a LAN, or via a wireless network. The control device 100 acquires information such as instructions input by a user via a UI (user interface) in an external device such as the above-mentioned personal computer, via the communication unit 318. The information such as instructions input by the user includes information specifying the photometry area set in the area setting unit 108. Note that the imaging unit 101 may be an external imaging device separate from the control device 100, and in this case, the external imaging device is connected to the control device 100 via the communication unit 318.

[0035] Fig. 4 is a flowchart showing the flow of day / night switching control of the imaging unit 101 in the control device 100 shown in Fig. 1. Here, an example of the operation when the imaging mode of the imaging unit 101 is switched from night mode to day mode will be described, and an example in which a color image is output in day mode and a black-and-white image is output in night mode will be described. The operation of the flowchart in Fig. 4 is mainly realized by the imaging unit 101, exposure control unit 102, evaluation value acquisition unit 103, determination unit 104, image processing unit 105, output unit 106, luminance acquisition unit 107, and area setting unit 108.

[0036] First, in the process of step S401, the imaging unit 101 captures an image of a subject or the like in accordance with a target exposure value determined by the exposure control unit 102, and the evaluation value acquisition unit 103 calculates an evaluation value based on the shooting conditions of the imaging unit 101 for which exposure control has been performed by the exposure control unit 102. As explained in the above-mentioned formula (1), the evaluation value acquisition unit 103 calculates an evaluation value based on the shooting conditions of the imaging unit 101 for which exposure control has been performed by the exposure control unit 102 (aperture value, shutter speed, AGC gain value).

[0037] Next, in the process of step S402, the determination unit 104 compares the current evaluation value calculated by the evaluation value acquisition unit 103 with the threshold value for day / night switching, and performs day / night switching determination based on the comparison result.

[0038] Here, using Figure 5, we will explain the change in evaluation value when switching from night mode to day mode in a normal auto day / night function, and the correction function of the day / night switching threshold provided in the determination unit 104. In Figure 5, a solid line 511 represents the change in evaluation value (referred to as evaluation value 511) calculated by the evaluation value acquisition unit 103 based on the shooting conditions of the image capture unit 101, and a dash-dot line 512 represents the change in the day / night switching threshold (referred to as threshold 512). In Figure 5, a dotted line 501 represents the timing at which the image capture unit 101 transitions from day mode to night mode (referred to as transition timing 501), and a dotted line 502 represents the timing at which the image capture unit 101 transitions from night mode to day mode (referred to as transition timing 502). It is assumed that the interval in which the evaluation value 511 gradually decreases over time corresponds to a time period in which the brightness of the surrounding environment gradually becomes darker, while the interval in which the evaluation value 511 gradually increases over time corresponds to a time period in which the surrounding environment gradually becomes brighter.

[0039] When the brightness of the surrounding environment gradually decreases and the evaluation value 511 becomes equal to or less than the threshold value 512, the aforementioned determination unit 104 determines to switch the imaging mode of the imaging unit 101 to night mode. Based on this determination result, the imaging unit 101 transitions to night mode, in which the IRCF is removed from the optical path and imaging is performed. However, when the imaging unit 101 switches to night mode, the IRCF is removed from the optical path and infrared light components are captured, temporarily increasing the exposure amount. In this case, the exposure control unit 102 performs exposure control to reduce the exposure amount, and the evaluation value 511 calculated by the evaluation value acquisition unit 103 may temporarily increase and exceed the threshold value 512. In this case, the determination unit 104 may determine that the evaluation value 511 exceeds the threshold value 512 as a result of the imaging conditions of the imaging unit 101 being changed due to the increased brightness of the surrounding environment, resulting in an erroneous day / night determination. Therefore, when the determination unit 104 determines that the imaging mode should be switched to the night mode, the determination unit 104 corrects the threshold value 512 used for comparison with the evaluation value 511 to a high value (+Δ) corresponding to the increase in the infrared light component due to the removal of the IRCF, that is, sets it to a threshold value corresponding to the night mode.

[0040] Furthermore, when the surrounding environment gradually becomes brighter and the evaluation value 511 exceeds the threshold value 512, the determination unit 104 determines to switch the imaging mode of the imaging unit 101 to day mode. Based on this determination result, the imaging unit 101 transitions to day mode, in which an IRCF is inserted into the optical path to capture images. However, when the imaging unit 101 switches to day mode, the exposure amount temporarily decreases because the IRCF is inserted into the optical path and infrared light components are removed. In this case, the exposure control unit 102 performs exposure control to increase the exposure amount, and the evaluation value 511 calculated by the evaluation value acquisition unit 103 may temporarily decrease to below the threshold value 512. In this case, the determination unit 104 may determine that the evaluation value 511 is below the threshold value 512 as a result of the imaging conditions of the imaging unit 101 being changed due to the darkening of the surrounding environment, resulting in an erroneous day / night determination. For this reason, when the determination unit 104 determines that the imaging mode of the imaging unit 101 should be switched to day mode, the determination unit 104 corrects the threshold 512 used for comparison with the evaluation value to a value (-Δ) lower by the amount of the decrease in infrared light component due to the insertion of the IRCF, in other words, returns it to the threshold value corresponding to day mode. Note that even when the determination unit 104 determines that the imaging mode of the imaging unit 101 should be switched to day mode, the determination unit 104 may maintain the threshold value for night mode, which is higher by +Δ as described above.

[0041] If it is determined in step S402 that the evaluation value is equal to or less than the threshold value, the determination unit 104 sets the imaging mode of the imaging unit 101 to night mode in the process of step S403. In this embodiment, an operation when switching the imaging unit 101 from night mode to day mode is given as an example, and therefore in step S403, the determination unit 104 maintains the imaging mode of the imaging unit 101 in night mode.

[0042] On the other hand, if it is determined in step S402 that the evaluation value exceeds the threshold, the determination unit 104 requests a luminance calculation process from the luminance acquisition unit 107. In the process of step S404, the luminance acquisition unit 107 outputs to the determination unit 104 the luminance calculation result obtained by calculating the average luminance of the entire image captured by the exposure-controlled imaging unit 101 and the luminance calculation result obtained by calculating the average luminance of the photometric region set by the region setting unit 108.

[0043] Next, in the process of step S405, the determination unit 104 determines whether the average luminance of the entire image calculated by the luminance acquisition unit 107 and the average luminance of the photometric region set by the region setting unit 108 are each within a predetermined range, as described above. If the average luminances are each within the predetermined range, the determination unit 104 sets the imaging mode of the imaging unit 101 to day mode in the process of step S406. That is, if the luminance of the entire image and the luminance of the photometric region set by the region setting unit 108 are each within a predetermined range or the luminance difference is within a predetermined range, it can be determined that there is no luminance difference in the actual environment, and therefore the determination unit 104 sets the imaging mode to day mode.

[0044] On the other hand, if the luminance of the entire image and the luminance of the photometric area set by the area setting unit 108 are outside the predetermined ranges, the determination unit 104 maintains the imaging mode of the imaging unit 101 as processing in step S407. That is, if at least one of the luminance of the entire image and the luminance of the photometric area set by the area setting unit 108 is outside the predetermined range or the luminance difference is outside the predetermined range, it can be determined that there is a luminance difference in the actual environment, and this means that the brightness of the actual environment has not been calculated correctly. Therefore, the determination unit 104 maintains the imaging mode of the imaging unit 101. In this example, the imaging mode is maintained in night mode.

[0045] <Second embodiment> Next, a control device 600 according to a second embodiment will be described. Fig. 6 is a diagram showing an example of the functional configuration of a control device 600 according to the second embodiment. Note that in the control device 600 of the second embodiment, the imaging unit 101 has the same configuration as that shown in Fig. 2 described above, and the hardware configuration is the same as that shown in Fig. 3 described above, so illustration and description thereof will be omitted. In the following, the second embodiment will be described mainly with respect to configurations different from those of the first embodiment, and descriptions of configurations similar to those of the first embodiment will be omitted as appropriate. The control device 600 of the second embodiment includes a histogram acquisition unit 109 in addition to the functional units shown in FIG.

[0046] FIG. 7 is a flowchart showing the flow of day / night switching control of the image capture unit 101 in the control device 600 of the second embodiment. As in the first embodiment, the second embodiment will be described using an example in which the image capture mode of the image capture unit 101 is switched from night mode to day mode, and a color image is output in day mode and a black-and-white image is output in night mode. The operation of the flowchart in FIG. 7 is mainly realized by the image capture unit 101, exposure control unit 102, evaluation value acquisition unit 103, determination unit 104, image processing unit 105, output unit 106, luminance acquisition unit 107, area setting unit 108, and histogram acquisition unit 109. Note that steps S401 to S406 are the same processing steps as those described in the first embodiment, and therefore their description will be omitted. In the flowchart of FIG. 7, the process proceeds to step S701 after step S401, and to step S402 after step S704 or step S705. If it is determined in step S402 that the evaluation value exceeds the threshold value, the process proceeds to step S706.

[0047] After the process of step S401 described above, the process proceeds to step S701, where the histogram acquisition unit 109 calculates a luminance histogram based on the luminance calculated by the luminance acquisition unit 107. At this time, the histogram acquisition unit 109 calculates at least one of the luminance histogram of the entire image and the luminance histogram of the photometric region set by the region setting unit 108.

[0048] 8 is a diagram showing the results of calculating a luminance histogram from an image captured by the image capture unit 101 in an environment with relatively little luminance difference, with the vertical axis representing frequency and the horizontal axis representing luminance. For example, if the image captured by the image capture unit 101 is an image expressed in 8-bit gradation, the maximum luminance value is 255. However, FIG. 8 shows an example in which the luminance is grouped into 16 divisions rather than 255 divisions, and the luminance is averaged for each group. Of course, the number of divisions is not limited to 16, and the histogram acquisition unit 109 may calculate and generate a luminance histogram using any number of divisions.

[0049] Next, in the process of step S702, the determination unit 104 determines whether or not the priority mode is set in the control device 600 of this embodiment. If the determination unit 104 determines that the priority mode is set, the process proceeds to step S703, whereas if the determination unit 104 determines that the priority mode is not set, the process proceeds to step S704.

[0050] Here, the priority modes include an entire image priority mode that prioritizes the photometry results or luminance calculation results of the entire image when determining whether to switch between day and night, and a designated area priority mode that prioritizes the photometry results or luminance calculation results of the photometry area set by the area setting unit 108. In the control device 600 of this embodiment, the entire image priority mode and the designated area priority mode are set by a user specification or automatic specification by the control device 600. For example, the priority mode is set by the user in response to an input instruction from the user via a UI in an external device (such as a personal computer) connected to the communication unit 318 described above.

[0051] 9 is a diagram showing an example of a GUI (graphical user interface) 901 displayed on an external device such as a personal computer when a user sets a priority mode. As shown in FIG. 9, the GUI 901 provides a radio button 910 for setting the entire priority mode and a radio button 911 for setting the specified area priority mode. The user can set either the entire priority mode or the specified area priority mode by selecting either the radio button 910 or 911.

[0052] Furthermore, the UI used when the user sets the priority mode is not limited to the example shown in FIG. 9 , and a GUI 1001 such as that shown in FIG. 10 may be used. As shown in FIG. 10 , a slider is displayed in the GUI 1001, and the user can specify the ratio between the entire priority mode and the designated area priority mode by manipulating a knob 1011 and moving it to any position on the slide rail 1010. For example, when the knob 1011 is moved to the left end of the slide rail 1010, the entire priority mode is set to 100% and the designated area priority mode is set to 0%. For example, when the knob 1011 is moved to the right end of the slide rail 1010, the designated area priority mode is set to 100% and the entire priority mode is set to 0%. For example, when the knob 1011 is moved to the center position of the slide rail 1010, the entire priority mode and the designated area priority mode are each set to a ratio of 50%.

[0053] If it is determined in step S702 that the priority mode is set and the process proceeds to step S703, the determination unit 104 corrects (sets) the threshold value described above in accordance with the set priority mode. At this time, instead of correcting the threshold value, the evaluation value calculated by the evaluation value acquisition unit 103 may be corrected. For example, if the entire image priority mode is set in the GUI 901 of FIG. 9, the evaluation value acquisition unit 103 corrects the threshold value or evaluation value described above, prioritizing the photometry result of the entire image over the local photometry area set by the area setting unit 108. Also, for example, if the specified area priority mode is set in the GUI 901 of FIG. 9, the determination unit 104 corrects the threshold value or evaluation value, prioritizing the photometry result or luminance calculation result of the photometry area set by the area setting unit 108 over the entire image. Also, for example, if the ratio between the entire image mode and the entire image priority mode is set in the GUI 1001 of FIG. 10, the determination unit 104 corrects the threshold value or evaluation value based on the photometry result and luminance calculation result of the entire image and the local photometry area according to that ratio. After the process of step S703, the process of the control device 600 proceeds to step S704.

[0054] In step S704, the determination unit 104 determines whether the size of the photometric area set by the area setting unit 108 is equal to or smaller than a predetermined size relative to the size of the entire image. If the determination unit 104 determines that the size of the photometric area is equal to or smaller than a predetermined size relative to the size of the entire image, the process proceeds to step S705. On the other hand, if the determination unit 104 determines that the size is not equal to or smaller than a predetermined size, the process proceeds to step S402.

[0055] When the process proceeds to step S705, the determination unit 104 corrects the threshold or evaluation value by prioritizing the photometry results or luminance calculation results for the entire image. For example, if the local photometry area set by the area setting unit 108 is extremely small relative to the entire image, it will be more susceptible to the influence of light from, for example, vehicle headlights. In this case, correcting the threshold or evaluation value by prioritizing the photometry results or luminance calculation results for the entire image makes it less susceptible to the influence of light from, for example, vehicle headlights, and improves the stability of day / night switching determination. In this way, the determination unit 104 corrects the threshold or evaluation value according to the relative size of the photometry area set by the area setting unit 108 with respect to the size of the entire image, thereby improving the stability of day / night switching determination.

[0056] Furthermore, when the process proceeds to step S706 after it is determined in step S402 that the evaluation value exceeds the threshold, the determination unit 104 determines whether or not the luminance histogram calculated by the histogram acquisition unit 109 corresponds to a predetermined pattern. If the determination unit 104 determines that the luminance histogram corresponds to the predetermined pattern, the process proceeds to step S406, whereas if the determination unit 104 determines that the luminance histogram does not correspond to the predetermined pattern, the process proceeds to step S403.

[0057] 11 and 12 are diagrams showing examples of predetermined patterns of the luminance histogram in step S706. The predetermined pattern of the brightness histogram shown in Fig. 11 is a pattern that indicates that high brightness is relatively more prevalent than low brightness, and this is a pattern that occurs when the surrounding environment is bright. Therefore, when both the brightness histogram of the entire image and the brightness histogram of the local photometric area correspond to the predetermined pattern shown in Fig. 11, it can be determined that the entire image, including the local photometric area, is bright. Therefore, in step S406, the determination unit 104 sets the imaging mode of the imaging unit 101 to day mode.

[0058] On the other hand, Fig. 12 shows an example of a luminance histogram pattern when there are relatively more low luminances than high luminances. When the priority mode is the designated area priority mode, or when the designated area priority mode has a higher ratio than the entire area priority mode, and the luminance histogram has the pattern of Fig. 12, the determination unit 104 maintains the imaging mode of the imaging unit 101 even if the evaluation value exceeds the threshold. In this example of the present embodiment, the determination unit 104 maintains the imaging mode in the night mode.

[0059] The determination unit 104 may correct the threshold value or evaluation value described above based on the pattern of the luminance histogram. For example, when the luminance histogram pattern indicates that the entire image, including local photometric regions, is bright, as shown in Fig. 11, the determination unit 104 may determine that the image capture mode should be set to day mode. Also, when the luminance histogram pattern indicates that there are relatively more low luminances than high luminances, as shown in Fig. 12, the determination unit 104 may determine that the image capture mode should be set to day mode.

[0060] In the control device 600 of the second embodiment, by performing the processing of FIG. 7, it is possible to appropriately determine whether to switch between day and night in accordance with the pattern of the luminance histogram and the priority mode, even when a local photometry area is set by the area setting unit 108.

[0061] In the above-described embodiment, an example was given in which the image capturing unit 101 has a function of being able to switch between two image capturing modes, a first image capturing mode and a second image capturing mode, but the image capturing unit 101 may also have a function of being able to switch between a plurality of image capturing modes, such as a third image capturing mode and a fourth image capturing mode. Furthermore, in the above-described embodiment, an IRCF is given as the predetermined optical element, but other optical filters other than an IRCF, such as a polarizing filter or a neutral density filter, or a group of lenses that change the angle of view (focal length) of the image capturing unit 101, may also be used.

[0062] The present invention can also be realized by providing a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. The above-described embodiments are merely examples of specific embodiments for implementing the present invention, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0063] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) an evaluation value acquisition means for acquiring an evaluation value used to switch an imaging mode of the imaging device based on an exposure amount based on a photometry result of a photometry area set for the imaging device; a brightness acquisition means for acquiring brightness of a predetermined area of ​​an image captured by the imaging device; a histogram acquisition means for acquiring a luminance histogram of the image; a determination means for determining whether to switch the imaging mode to a first imaging mode or a second imaging mode different from the first imaging mode, based on the evaluation value, a threshold value for the evaluation value, the luminance of the predetermined region, and the luminance histogram; A control device comprising: (Configuration 2) the luminance acquisition means acquires, as the luminance of the predetermined region, the luminance of the entire image captured by the imaging device and the luminance of the photometric region; The control device according to configuration 1, characterized in that the determination means determines whether to switch the imaging mode to the first imaging mode or the second imaging mode based on a comparison result between the evaluation value and the threshold value and whether the luminance of the entire image and the photometric area is within a predetermined range or whether a difference in luminance between the entire image and the photometric area is within a predetermined range. (Configuration 3) The control device according to configuration 2, wherein even when the determination means determines to switch the imaging mode based on a comparison result between the evaluation value and the threshold value, the determination means determines to maintain the imaging mode when the luminance of the entire image and the photometric area is outside a predetermined range or when the difference in luminance between the entire image and the photometric area is outside a predetermined range. (Configuration 4) the luminance acquisition means acquires, as the luminance of the predetermined region, the luminance of the entire image captured by the imaging device and the luminance of the photometric region; The control device according to any one of configurations 1 to 3, wherein the histogram acquisition means acquires the luminance histogram based on at least one of the luminance of the entire image and the luminance of the photometric area. (Configuration 5) The control device according to configuration 4, wherein the determination means determines whether to switch the imaging mode to the first imaging mode or the second imaging mode based on a comparison result between the evaluation value and the threshold value and a pattern of the brightness histogram. (Configuration 6) The control device according to configuration 5, characterized in that even when the determination means determines to switch the imaging mode based on the comparison result between the evaluation value and the threshold value, it determines to maintain the imaging mode when the pattern of the luminance histogram of the photometric area is a pattern indicating that low luminances are relatively more prevalent than high luminances. (Configuration 7) 7. The control device according to any one of configurations 1 to 6, wherein the determining means corrects the threshold value or the evaluation value based on a pattern of the luminance histogram. (Configuration 8) 8. The control device according to any one of configurations 1 to 7, wherein the determining means sets the threshold value for the evaluation value. (Configuration 9) 9. The control device according to configuration 8, wherein the determining means sets the threshold value in response to an instruction from a user. (Configuration 10) 9. The control device according to configuration 8, wherein the determining means corrects the threshold value or the evaluation value in accordance with the size of the photometric area relative to the size of the entire image. (Configuration 11) The control device according to configuration 8, wherein the determination means corrects the threshold value or the evaluation value according to a priority mode indicating whether the first imaging mode or the second imaging mode is to be prioritized. (Configuration 12) The control device according to configuration 8, wherein the determination means obtains a ratio indicating which of the first imaging mode and the second imaging mode is to be prioritized, and corrects the threshold value or the evaluation value according to the ratio. (Configuration 13) 13. The control device according to any one of configurations 1 to 12, wherein the photometry area is the entire image captured by the imaging device or a local area of ​​the image. (Configuration 14) an exposure control means for controlling the exposure amount for the imaging device based on the photometry result of the photometry area; an area setting means for changing the photometric area; 14. The control device according to any one of configurations 1 to 13, comprising: (Configuration 15) 15. The control device according to configuration 14, wherein the area setting means changes the photometry area in a time-division manner. (Configuration 16) 15. The control device according to configuration 14, wherein the area setting means changes the photometry area in response to an instruction from a user. (Configuration 17) the exposure control means sets a target value of exposure when the imaging device captures an image based on the photometry result of the photometry area; The control device according to configuration 14, wherein the evaluation value acquisition means acquires, as the exposure amount, at least one of the shooting conditions of an aperture value, a shutter speed, and a gain value of an automatic gain control when the imaging device captures the image according to the target value of the exposure. (Configuration 18) 18. The control device according to any one of configurations 1 to 17, further comprising image processing means for generating a color image from an image captured by the imaging device when the imaging mode is the first imaging mode, and generating a black-and-white image from an image captured by the imaging device when the imaging mode is the second imaging mode. (Configuration 19) the imaging device has an insertion / removal unit for inserting and removing a predetermined optical element on an optical path, The control device according to configuration 18, wherein the first imaging mode is an imaging mode in which the predetermined optical element is inserted into the optical path and imaging is performed, and the second imaging mode is an imaging mode in which the predetermined optical element is removed from the optical path and imaging is performed. (Configuration 20) 20. The control device according to configuration 19, wherein the predetermined optical element is an infrared light removal filter. (Method 1) an evaluation value acquisition step of acquiring an evaluation value used for switching an imaging mode of the imaging device based on an exposure amount based on a photometry result of a photometry area set for the imaging device; a brightness acquisition step of acquiring brightness of a predetermined area of ​​an image captured by the imaging device; a histogram acquisition step of acquiring a luminance histogram of the image; a determination step of determining whether to switch the imaging mode to a first imaging mode or a second imaging mode different from the first imaging mode, based on the evaluation value, a threshold value for the evaluation value, the luminance of the predetermined region, and the luminance histogram; A control method comprising: (Program 1) A program that causes a computer to function as each means of the control device according to any one of configurations 1 to 20. [Explanation of symbols]

[0064] 100, 600: control device, 101: imaging unit, 102: exposure control unit, 103: evaluation value acquisition unit, 104: determination unit, 105: image processing unit, 106: output unit, 107: brightness acquisition unit, 108: area setting unit, 109: histogram acquisition unit

Claims

1. an evaluation value acquisition means for acquiring an evaluation value used to switch an imaging mode of the imaging device based on an exposure amount based on a photometry result of a photometry area set for the imaging device; a brightness acquisition means for acquiring brightness of a predetermined area of ​​an image captured by the imaging device; a histogram acquisition means for acquiring a luminance histogram of the image; a determination means for determining whether to switch the imaging mode to a first imaging mode or a second imaging mode different from the first imaging mode, based on the evaluation value, a threshold value for the evaluation value, the luminance of the predetermined region, and the luminance histogram; A control device comprising:

2. the luminance acquisition means acquires, as the luminance of the predetermined region, the luminance of the entire image captured by the imaging device and the luminance of the photometric region; 2. The control device according to claim 1, wherein the determination means determines whether to switch the imaging mode to the first imaging mode or the second imaging mode based on a comparison result between the evaluation value and the threshold value and whether the luminance of the entire image and the photometric area is within a predetermined range or whether a difference in luminance between the entire image and the photometric area is within a predetermined range.

3. The control device according to claim 2, characterized in that even when the determination means determines to switch the imaging mode based on the comparison result between the evaluation value and the threshold value, it determines to maintain the imaging mode when the luminance of the entire image and the photometric area is outside a predetermined range, or when the difference in luminance between the entire image and the photometric area is outside a predetermined range.

4. the luminance acquisition means acquires, as the luminance of the predetermined region, the luminance of the entire image captured by the imaging device and the luminance of the photometric region; 2. The control device according to claim 1, wherein the histogram acquisition means acquires the luminance histogram based on at least one of the luminance of the entire image and the luminance of the photometric area.

5. 5. The control device according to claim 4, wherein the determination means determines whether to switch the imaging mode to the first imaging mode or the second imaging mode based on a comparison result between the evaluation value and the threshold value and a pattern of the brightness histogram.

6. The control device according to claim 5, characterized in that even when the determination means determines to switch the imaging mode based on the comparison result between the evaluation value and the threshold value, it determines to maintain the imaging mode when the pattern of the luminance histogram of the photometric area is a pattern indicating that there are relatively more low luminances than high luminances.

7. 2. The control device according to claim 1, wherein the determining means corrects the threshold value or the evaluation value based on a pattern of the luminance histogram.

8. 2. The control device according to claim 1, wherein the determining means sets the threshold value for the evaluation value.

9. 9. The control device according to claim 8, wherein the determining means sets the threshold value in response to an instruction from a user.

10. 9. The control device according to claim 8, wherein the determining means corrects the threshold value or the evaluation value in accordance with the size of the photometric area relative to the size of the entire image.

11. 9. The control device according to claim 8, wherein the determining means corrects the threshold value or the evaluation value in accordance with a priority mode indicating whether the first imaging mode or the second imaging mode is to be prioritized.

12. 9. The control device according to claim 8, wherein the determining means obtains a ratio indicating which of the first imaging mode and the second imaging mode is to be prioritized, and corrects the threshold value or the evaluation value in accordance with the ratio.

13. The control device according to claim 1 , wherein the photometry area is the entire image captured by the imaging device or a local area of ​​the image.

14. an exposure control means for controlling the exposure amount for the imaging device based on the photometry result of the photometry area; an area setting means for changing the photometric area; 2. The control device according to claim 1, further comprising:

15. 15. The control device according to claim 14, wherein the area setting means changes the photometry area in a time-division manner.

16. 15. The control device according to claim 14, wherein the area setting means changes the photometry area in response to an instruction from a user.

17. the exposure control means sets a target value of exposure when the imaging device captures an image based on the photometry result of the photometry area; The control device according to claim 14, characterized in that the evaluation value acquisition means acquires, as the exposure amount, at least one of the shooting conditions, including an aperture value, a shutter speed, and a gain value of an automatic gain control, when the imaging device captures the image in accordance with the target value of exposure.

18. 18. The control device according to claim 1, further comprising an image processing means for generating a color image from an image captured by the imaging device when the imaging mode is the first imaging mode, and generating a black-and-white image from an image captured by the imaging device when the imaging mode is the second imaging mode.

19. the imaging device has an insertion / removal unit for inserting and removing a predetermined optical element on an optical path, 19. The control device according to claim 18, wherein the first imaging mode is an imaging mode in which imaging is performed by inserting the specified optical element into the optical path, and the second imaging mode is an imaging mode in which imaging is performed by removing the specified optical element from the optical path.

20. 20. The control device according to claim 19, wherein the predetermined optical element is an infrared light removal filter.

21. an evaluation value acquisition step of acquiring an evaluation value used for switching an imaging mode of the imaging device based on an exposure amount based on a photometry result of a photometry area set for the imaging device; a brightness acquisition step of acquiring brightness of a predetermined area of ​​an image captured by the imaging device; a histogram acquisition step of acquiring a luminance histogram of the image; a determination step of determining whether to switch the imaging mode to a first imaging mode or a second imaging mode different from the first imaging mode, based on the evaluation value, a threshold value for the evaluation value, the luminance of the predetermined region, and the luminance histogram; A control method comprising:

22. Computer, an evaluation value acquisition means for acquiring an evaluation value used to switch an imaging mode of the imaging device based on an exposure amount based on a photometry result of a photometry area set for the imaging device; a brightness acquisition means for acquiring brightness of a predetermined area of ​​an image captured by the imaging device; a histogram acquisition means for acquiring a luminance histogram of the image; a determination means for determining whether to switch the imaging mode to a first imaging mode or a second imaging mode different from the first imaging mode, based on the evaluation value, a threshold value for the evaluation value, the luminance of the predetermined region, and the luminance histogram; A program that functions as a control device having the above.

Images