Lighting control device, imaging device, control method, and storage medium

Abstract

The present application provides an illumination control device, an imaging device, a control method, and a storage medium. In order to realize an illumination control device and the like in which color reproducibility of an image and an S / N ratio are compatible at the time of infrared illumination, the illumination control device includes a visible light measurement unit configured to measure an amount of a visible light component, an infrared illumination unit configured to perform infrared illumination, and a control unit configured to cause an intensity of the infrared illumination performed by the infrared illumination unit to gradually change as the amount of the visible light component measured by the visible light measurement unit decreases from a first threshold value to a second threshold value.

Description

TECHNICAL FIELD

[0001] The present application relates to an illumination control device for infrared illumination, an imaging device, and a storage medium. BACKGROUND

[0002] There are models of monitoring cameras and the like that have a night mode in which imaging is performed with an infrared cut filter (to be referred to as an IRCF) removed from an imaging light path in low-illuminance imaging. Alternatively, operation of monitoring cameras and monitoring systems and the like that also perform imaging while using infrared illumination in a dark environment in which there is little external light or illumination has become widespread. Although visible light illumination can also be used for similar reasons, such visible light illumination can not be usable depending on the monitoring target or its installation environment, and therefore infrared illumination is used in such cases.

[0003] In the night mode of a general monitoring camera and the like, because the infrared component is removed due to the IRCF being removed from the imaging light path, a color different from the original color of the subject appears, and therefore a black-and-white image generated by removing the color component from the captured image is mainly output. On the other hand, users desire to recognize the color of the subject even in a dark environment in which there is little visible light.

[0004] For example, Japanese Patent Application 2017-63362 discloses a technology that divides an image into a plurality of regions, and controls a light source unit with respect to a region having a predetermined or smaller amount of invisible light (infrared light). In addition, Japanese Laid-Open 2017-5484 discloses a technology that adjusts image quality according to the ratio of visible light to invisible light.

[0005] However, according to the details of the technology of Japanese Patent Application 2017-63362, when the subject is irradiated with invisible light, the color component of the captured image is not considered, which leads to a deterioration problem in color reproduction of the subject. Furthermore, according to the details of Japanese Laid-Open 2017-5484, when the subject is irradiated with invisible light, the color of the subject cannot be reproduced.

[0006] Therefore, the present application aims to obtain an illumination control device and the like that can enable the color reproducibility of an image to be compatible with the S / N ratio at the time of infrared illumination. SUMMARY

[0007] An aspect of the present application is an illumination control apparatus including at least one processor or circuitry configured to function as: a visible light measurement unit configured to measure an amount of visible light component; an infrared illumination unit configured to perform infrared illumination; and a control unit configured to cause an intensity of the infrared illumination performed by the infrared illumination unit to gradually change as the amount of visible light component measured by the visible light measurement unit decreases from a first threshold to a second threshold.

[0008] Another aspect of the present application is an imaging apparatus having an imaging lens and an imaging unit, the imaging apparatus including: a visible light measurement unit configured to measure an amount of visible light component; an infrared illumination unit configured to perform infrared illumination; and a control unit configured to cause an intensity of the infrared illumination performed by the infrared illumination unit to gradually change as the amount of visible light component measured by the visible light measurement unit decreases from a first threshold to a second threshold, wherein the visible light measurement unit measures the amount of visible light component based on information acquired from the imaging unit.

[0009] Still another aspect of the present application is a control method including: measuring an amount of visible light component; and causing an intensity of infrared illumination performed by an infrared illumination unit to gradually change as the amount of visible light component measured in the measurement decreases from a first threshold to a second threshold.

[0010] Still another aspect of the present application is a non-transitory computer readable storage medium storing a program for executing a control method including: measuring an amount of visible light component; and causing an intensity of infrared illumination performed by an infrared illumination unit to gradually change as the amount of visible light component measured in the measurement decreases from a first threshold to a second threshold.

[0011] Other features of the present application will become apparent from the following description of embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a block diagram illustrating a structure of an illumination control apparatus according to a first embodiment.

[0013] Figure 2 is a flowchart illustrating processing of the illumination control apparatus according to the first embodiment.

[0014] Figure 3 is a graph illustrating a relationship between a threshold of visible light and an intensity of infrared illumination according to the first embodiment.

[0015] Figure 4 is a block diagram illustrating a structure of an imaging apparatus according to a second embodiment.

[0016] Figure 5This is a block diagram illustrating the structure of the camera unit according to the second embodiment.

[0017] Figure 6 This is a flowchart illustrating the processing of the camera device according to the second embodiment.

[0018] Figure 7 This is a distribution map of the chromaticity components in the image of a predetermined subject when the amount of visible light is sufficient, as shown in the second embodiment.

[0019] Figure 8 This is a distribution map of the chromaticity components in the image of a predetermined subject under the condition that the intensity of infrared illumination is more dominant than that of visible light in the second embodiment.

[0020] Figure 9 This is a graph showing the spectral sensitivity of a general color image sensor according to the second embodiment.

[0021] Figure 10 This is a graph showing the relationship between the visible light threshold and the intensity of infrared illumination according to the second embodiment.

[0022] Figure 11 It is shown in Figure 10 The table shows the states of IRCF 501, infrared illumination unit 100, and output unit 405 in each range from range A to range D in the figure.

[0023] Figure 12 This is a block diagram illustrating the structure of a camera system according to a third embodiment.

[0024] Figure 13 This is a flowchart illustrating the processing of the camera system according to the third embodiment.

[0025] Figure 14 This is a flowchart illustrating the processing of other parts of the camera system according to the third embodiment.

[0026] Figure 15 This is a diagram illustrating examples of various modes that can be set by the operation unit 1203 of the client device 1200 according to the third embodiment.

[0027] Figure 16 It is shown Figure 15 A table showing examples of combined states of IRCF, infrared illumination, and image output in various modes.

[0028] Figure 17 This is a diagram illustrating an example of a GUI for setting illuminance when automatically inserting or removing IRCFs according to a third embodiment.

[0029] Figure 18This is a diagram illustrating an example of a GUI for setting priorities when correcting illumination intensity in automatic adjustment of infrared illumination according to a third embodiment.

[0030] Figure 19 This is a diagram illustrating an example of a GUI for setting the priority of the white balance control range according to the third embodiment.

[0031] Figure 20 This illustrates the third embodiment in Figure 19 An example diagram showing the white balance control range set in the settings. Detailed Implementation

[0032] Hereinafter, preferred embodiments of the invention will be described with reference to the accompanying drawings. In the various figures, the same reference numerals are applied to the same components or elements, and repeated descriptions will be omitted or simplified.

[0033] Additionally, examples of using a webcam as a video recording device will be described in the examples. However, video recording devices include electronic devices with video recording capabilities, such as digital still cameras, digital video cameras, smartphones with cameras, tablet computers with cameras, or vehicle cameras, etc.

[0034] [First Embodiment]

[0035] First, refer to Figures 1 to 3 The structure and processing of the lighting control device according to the first embodiment will be described.

[0036] Figure 1 This is a block diagram illustrating the structure of a lighting control device according to a first embodiment. The lighting control device of this embodiment is used in combination with a webcam, used as an example of a camera device, and includes an infrared illumination unit 100, an illumination control unit 101, a visible light measurement unit 102, a judgment unit 103, and a storage unit 104, etc.

[0037] Infrared illumination unit 100 emits infrared light (e.g., light with infrared spectral characteristics of approximately 800 nm to 1000 nm) for infrared illumination. Illumination control unit 101 controls the illumination intensity of infrared illumination unit 100. In addition to controlling the illumination intensity of infrared illumination unit 100, illumination control unit 101 also controls various controllable parameters such as the timing of illumination intensity changes, the rate of illumination intensity changes, and illumination resolution.

[0038] The visible light measurement unit 102 measures the shooting environment, especially the amount of visible light component (visible light intensity) of the subject. The judgment unit 103 judges the visible light intensity, etc., measured by the visible light measurement unit 102, and controls the emitted light intensity, etc. of the infrared illumination unit 100 via the illumination control unit 101 based on the judgment result.

[0039] In addition, the judging unit 103 and the like have a CPU functioning as a computer to function as a control section (control unit) that controls the operation of each unit of the entire apparatus based on a computer program stored in the storage unit 104. Further, the judging unit 103 transmits a control signal to the imaging device. The storage unit 104 stores, in addition to the computer program, a plurality of illumination control patterns and predetermined threshold values and the like for use in judgment, which are provided in advance for the illumination control unit 101 to control the intensity of the infrared illumination unit 100.

[0040] Next, Figure 2 is a flowchart showing the processing of the illumination control apparatus according to the first embodiment, and the processing of the illumination control apparatus will be described using the flowchart of Figure 2 Further, the computer of the judging unit 103 and the like executes the computer program stored in the storage unit 104 to cause the operation of each step of Figure 2 to be performed. In addition, it is assumed that the imaging operation of the imaging device is performed during the operation of the flowchart shown in Figure 2

[0041] First, in step S201, the visible light measuring unit 102 measures the amount of the visible light component. The visible light measuring unit measures the illuminance (lux) of the environment using, for example, an illuminance sensor having sensitivity to the wavelength range of visible light. Further, the direction and range in which the visible light measuring unit 102 performs the measurement are set to a direction or a range substantially the same as the direction or the range of the radiation direction of the infrared illumination unit 100, and the direction and the range include the imaging direction and the imaging range of the imaging lens and the image sensor.

[0042] In step S202, the judging unit 103 judges whether or not the amount of the visible light component measured in step S201 is equal to or less than a first threshold value.

[0043] Here, the first threshold value is set to a value equivalent to a predetermined amount (brightness) of the visible light component, and if the value is equal to or less than the first threshold value, the value corresponds to, for example, the brightness at which the S / N ratio of the image deteriorates in the case where the imaging device for monitoring is used to take a color picture. In addition, if the value is set to be equal to or less than the first threshold value, an infrared cut filter (IRCF) installed on the imaging light path of the imaging lens is taken out (removed) in the imaging device, and a control signal is transmitted to the imaging device to generate a color image. Further, the first threshold value can be arbitrarily set by the user.

[0044] If the amount of the visible light component is greater than the first threshold value, the illumination control unit 101 causes the infrared illumination unit 100 to turn off the illumination (step S203).

[0045] ​If the amount of visible light component is judged to be smaller than or equal to the first threshold value in step S202, the judging unit 103 judges whether the amount of visible light component is equal to or larger than a second threshold value in step S204. If the amount of visible light component measured by the visible light measuring unit 102 is smaller than the second threshold value in step S204, the intensity of infrared illumination is raised to an output limit (upper limit) in step S205, and a control signal is sent to the camera to generate a black and white image. Here, the second threshold value is set to a predetermined brightness, for example, at which the subject cannot be recognized in the image when the camera for monitoring takes a color image, or the color information of the image cannot be recognized. Alternatively, the threshold value can be arbitrarily set by the user.

[0046] If the answer in step S204 is positive, the process proceeds to step S206. In step S206, the illumination control unit 101 performs control so that the intensity of infrared illumination performed by the infrared illumination unit 100 has a predetermined ratio with respect to the amount of visible light component measured by the visible light measuring unit 102. At this time, various parameters that can be controlled by the illumination control unit 101, such as the timing and rate of illumination intensity change, and the illumination resolution, can be changed. A detailed control example will be described using Figure 3 .

[0047] Here, Figure 3 is a graph showing the relationship between the threshold value of visible light and the intensity of infrared illumination according to the first embodiment. That is, Figure 3 shows an example of the first threshold value and the second threshold value judged by the judging unit 103 in steps S202 and S204, respectively, in the flowchart of Figure 2 . In addition, in Figure 3 , the output limit at the time when the illumination control unit 101 performs control in step S205 is indicated by a horizontal dotted line. The first threshold value is larger than the second threshold value. That is, the threshold values have a relationship in which the former has a larger amount of visible light component (the former means brighter). Furthermore, the upper limit of the illumination intensity of infrared illumination can be variable.

[0048] In addition, in step S206, the illumination control unit 101 controls the intensity of infrared illumination performed by the infrared illumination unit 100 so that the intensity gradually decreases as the amount of visible light component becomes smaller (the brightness decreases) between the first threshold value and the second threshold value. Here, in step S206, the illumination control unit 101 functions as a control unit that causes the intensity of infrared illumination performed by the infrared illumination unit to gradually change (decrease) as the amount of visible light component measured by the visible light measuring unit decreases from the first threshold value to the second threshold value.

[0049] That is, in step S206, control is performed so that the infrared light component and the visible light component are in a substantially constant ratio. In this case, a relationship in which the amount of the infrared light component is greater than the amount of the visible light component is desired. Thus, an image with a balanced S / N ratio and color reproducibility can be obtained. Further, control is desirably performed so that the saturation of the color image generated by the imaging device is emphasized in a case where the amount of the visible light component is between the first threshold value and the second threshold value.

[0050] Further, the distance measuring unit can measure the distance to the subject present in the radiation direction of the infrared illumination unit 100, and can control the amount of shift in the intensity of the infrared illumination in accordance with the measured distance to the subject. That is, for example, as indicated by the dotted line in FIG. 10, while maintaining the slope of the intensity of the infrared illumination in the portion from the first threshold value to the second threshold value, the intensity of the infrared illumination can be shifted so as to increase as the distance to the subject increases. In addition, while maintaining the slope characteristic of the infrared illumination intensity in the portion from the first threshold value to the second threshold value, the amount of shift can be adjusted by the user. Figure 3

[0051] Further, the output of the infrared illumination unit controlled by the illumination control unit 101 can be corrected in accordance with the characteristics of the infrared reflected on the subject. For example, the output of the infrared illumination unit controlled by the illumination control unit 101 can be controlled so as to relatively decrease as the reflectance of the subject with respect to the infrared increases. In addition, a plurality of illumination control modes of the illumination control unit 101 (a mode in which the intensity of the infrared illumination is changed in the portion from the first threshold value to the second threshold value) can be provided in advance and stored in the storage unit 104.

[0052] In the present embodiment, as described above, while the intensity of the infrared light is changed in accordance with the decrease in the amount of the visible light component in the luminance range from the first threshold value to the second threshold value, imaging is performed. Thus, compared to a conventional imaging device that uses infrared illumination to turn on or off the infrared light only, the amount of visible light information (color information) that can be acquired can increase in the luminance range from the first threshold value to the second threshold value.

[0053] Further, by radiating infrared light at a level lower than the first threshold value, the S / N ratio can be improved by the infrared sensitivity of the image sensor in a darker level. Thus, the color reproducibility of the captured image can be compatible with the S / N ratio. Further, by performing imaging while reducing the intensity of the infrared light in accordance with the amount of decrease in the visible light component in the luminance range from the first threshold value to the second threshold value, the balance between the S / N ratio and the color reproducibility can be appropriately maintained.

[0054] [Second Embodiment]

[0055] The second embodiment will be described below using Figure 4 and Figure 5 .​Figure 4 is a block diagram showing a configuration of an imaging device according to a second embodiment, and Figure 5 is a block diagram showing a configuration of an imaging unit 400 according to the second embodiment.

[0056] The description of the constituent components with reference signs 100 to 104 in Figure 4 will be omitted because these components are the same as those described in Figure 1 the first embodiment.

[0057] The imaging unit 400 of the imaging device 40 receives light from the outside to perform imaging. The imaging unit 400 includes an imaging lens 500 (constituted by a group including a plurality of lenses such as a zoom lens and a focus lens), an IRCF 501 that can be inserted in / from an imaging light path of the imaging lens 500, and an image sensor 502 such as a CCD or a CMOS.

[0058] For example, one of R, G, and B color filters is arranged in front of each pixel of the image sensor 502. In addition, the R, G, and B color filters are arranged in a so-called Bayer array in which the R, G, and B color filters are alternately arranged in a predetermined period. Thus, R, G, and B color signals are periodically read in a predetermined order by sequentially reading signals of each pixel of the image sensor. In addition, the imaging unit 400 includes a CDS circuit 503 that performs correlated double sampling (CDS) for noise reduction.

[0059] Further, the imaging unit 400 includes an automatic gain control (AGC) amplifier 504 that automatically performs gain control on a signal obtained from the camera. In addition, the imaging unit 400 includes an A / D conversion circuit 505 that converts an analog signal into a digital signal.

[0060] The image sensor 502 converts an object image formed via the imaging lens 500 serving as an imaging optical system into an electric signal, and the CDS circuit 503 performs correlated double sampling on the electric signal output from the image sensor 502. The AGC amplifier 504 performs automatic gain control on the electric signal output from the CDS circuit 503, and the A / D conversion circuit 505 converts an analog signal on which automatic gain control by the AGC amplifier 504 has been performed into a digital signal.

[0061] The IRCF driving unit 401 moves the IRCF 501 included in the imaging unit 400 in the vertical direction with respect to the imaging light path of the imaging lens 500, so that the IRCF is taken out of (removed from) or inserted into the imaging light path. In order to make the light path length in the case where the IRCF 501 is taken out of (removed from) the light path the same as the light path length when the IRCF 501 is inserted, a glass plate or the like capable of transmitting light of a spectral component from visible light components to infrared light can be inserted into the light path. The exposure control unit 402 controls exposure using an exposure parameter such as an exposure time (accumulation time) of the image sensor 502 of the imaging unit 400, a gain, or an aperture value.

[0062] The judging unit 103 uses the exposure control unit 402 to estimate the brightness (illuminance) of the subject (measure the amount of visible light components) using an exposure amount (aperture value and exposure time after exposure control), a gain, and luminance information of an image, and the like, to judge whether the IRCF 501 has been inserted or removed. That is, in this case, the exposure control unit 402 and the judging unit 103 function as a visible light measurement unit that measures the amount of visible light components based on information acquired from the imaging unit. Then, a control signal is transmitted to the IRCF driving unit 401 to control the insertion or removal of the IRCF 501 in accordance with the judgment result. The image processing unit 403 performs image processing such as gamma correction, white balance processing, edge enhancement, saturation adjustment, contrast adjustment, and noise reduction processing, and the like, on a captured image controlled by the exposure control unit 402.

[0063] Here, the judging unit 103 can switch the image to a color image or a black-and-white image using the image processing unit 403 in accordance with the control of the insertion and / or removal of the IRCF 501 using the IRCF driving unit 401 and the brightness threshold. That is, the image can be switched to a color image in the case where the IRCF 501 is inserted into the imaging light path, and switched to a black-and-white image in the case where the IRCF 501 is taken out of the imaging light path and the brightness (illuminance) is lower than the second threshold.

[0064] The calculating unit 404 calculates values of luminance and color components of an image processed by the image processing unit 403. The values include, for example, an average luminance value of the entire image, and chroma values of each predetermined divided area of the image, and the like. The output unit 405 outputs an image for which the values of luminance and color components have been calculated by the calculating unit 404.

[0065] The illumination control device having the infrared illumination unit 100 and the illumination control unit 101 can be provided inside the housing of the imaging device 40, or can be provided as a main body separate from the housing of the imaging device. In addition, the wavelength component of the infrared light to be radiated can be in a range from about 800 nm to 1000 nm in which the image sensor 502 has sensitivity.

[0066] Figure 6 is a flowchart showing the processing of the imaging device according to the second embodiment, and will be described with reference to Figures 4 to 11 The processing of the imaging device of Figure 6 will be described using the flowchart of Figure 4 . Further, the computer of the judging unit 103 and the like executes the computer program stored in the storage unit 104 to cause the operation of each step of Figure 6 .

[0067] First, in step S601, a color image is generated using the imaging unit 400, the exposure control unit 402, and the image processing unit 403. In step S602, the luminance (illuminance) of the subject in the shooting environment is estimated (measured). At this time, the illuminance can be estimated (measured) based on at least one of the aperture value, the exposure time, and the gain controlled by the exposure control unit 402, or the average luminance value of the image calculated by the calculation unit 404, and the like. Alternatively, the illuminance can be measured (estimated) using an illuminance sensor such as the visible light sensor described in the first embodiment.

[0068] The judging unit 103 judges whether the illuminance estimated (measured) in step S602 has a predetermined value or less (step S603). If the judging unit 103 judges that the illuminance has a value greater than a predetermined first threshold value in step S603, imaging is performed with the IRCF 501 of the imaging unit 400 inserted in the optical path (step S604). Although a color image is generally output in this case, the image can be in another output format (step S605). On the other hand, if the judging unit 103 judges that the illuminance has a value lower than or equal to the predetermined value (first threshold value) in step S603, the chroma components (first chroma components) of the image are measured with the IRCF 501 inserted in the optical path (step S606).

[0069] Here, Figure 7 is a distribution chart of the chroma components in the image of the predetermined subject in the second embodiment when the amount of visible light is sufficient. That is, Figure 7 is an example of a chart showing the distribution of the chroma (R-Y and B-Y) components for each region obtained by dividing the image of the imaging unit 400 that has shot the predetermined subject into N vertical regions and M horizontal regions, with the IRCF 501 inserted in the optical path in a bright shooting environment having an amount of visible light sufficient. The vertical axis is the R-Y axis, and the horizontal axis is the B-Y axis.

[0070] Further, Figure 8 is a distribution chart of the chroma components in the image of the predetermined subject in the second embodiment under the condition that the intensity of infrared light is dominant over visible light. That is,Figure 8 is a graph showing a distribution of chroma components of an image of the same subject as in Figure 7

[0071] Although the chroma components are uniformly distributed in Figure 7 , in Figure 8 , they converge in a local area around the center. This is because, in terms of the spectral sensitivity characteristic of a general image sensor shown in Figure 9 , the image sensor has substantially the same sensitivity regardless of color in the case of wavelengths longer than about 800 nm. Further, in Figure 9 is a graph showing the spectral sensitivity of a general color image sensor according to a second embodiment.

[0072] That is, since infrared has a greater intensity than visible light, the distribution of chroma components is more likely to be concentrated around the center, and thus the amount of chroma components tends to decrease (a non-color image is generated). Based on the change characteristic of the ratio of the intensity of the chroma components of the image to the intensity of the infrared components with respect to the intensity of the visible components, in the case where the IRCF 501 of the image pickup unit 400 is removed from the light path, the relative intensity of the infrared illumination with respect to the visible components can be estimated.

[0073] Further, there are various known techniques of estimating the ratio of the infrared components to the visible components from the chroma components of the image, and other known techniques can be used.

[0074] After the first chroma components are measured in the case where the IRCF 501 is inserted in step S606, in step S607, the IRCF 501 is removed from the light path using the IRCF driving unit 401. Further, the chroma components measured in step S606 (the first chroma components) can be, for example, the average of R-Y and B-Y obtained from a plurality of divided areas of the picture, and the like.

[0075] Then, in step S608, the judging unit 103 judges whether the first chroma components measured in step S606 have a value equal to or greater than a fourth threshold value. Here, the fourth threshold value is a chroma value corresponding to a predetermined luminance (the second threshold value in the first embodiment), for example, at which a subject cannot be recognized in an image obtained from color imaging of the image pickup apparatus 40, or color information cannot be recognized from the image. Further, the fourth threshold value can be arbitrarily set by a user. Alternatively, the fourth threshold value can be a chroma component value related to the color recognition limit of a color recognizer or human eyes when the saturation is emphasized in step S612 which will be described below.

[0076] ​If the first chrominance component is judged to have a value smaller than the fourth threshold value in step S608, the upper limit of the infrared illumination is raised to the output upper limit (step S613). At this time, the illumination control unit 101 does not need to forcibly control the intensity of the infrared illumination unit 100 to the output upper limit. In addition, the upper limit of the intensity of the illumination can be able to be changed according to the subject conditions such as the distance to the subject, the reflection characteristics of the subject, and the influence of the shield, and the like.

[0077] In addition, based on the fact that the amount of the chrominance component in the image at the time of insertion of the IRCF 501 is small (the image is substantially non-color), it can be estimated that the subject itself is almost non-color, and thus a black-and-white image is output by giving priority to the S / N ratio (step S614). On the other hand, if the first chrominance component has a value equal to or greater than the fourth threshold value in step S608, the calculation unit 404 calculates (measures) the chrominance component (second chrominance component) in step S609 with the IRCF 501 removed from the optical path.

[0078] Further, the chrominance component (second chrominance component) measured in step S609 can be, for example, the average value of R-Y and B-Y obtained from a plurality of divided regions of the picture, and the like. Then, in step S610, the judgment unit 103 judges whether or not the second chrominance component calculated in step S609 has a value smaller than or equal to a third threshold value and equal to or greater than a fourth threshold value.

[0079] Here, the third threshold value is a chrominance value corresponding to the luminance (the first threshold value in the first embodiment), and for example, if the second chrominance component has a value smaller than or equal to the third threshold value, the S / N ratio of the image deteriorates at the time when the monitoring camera is used to shoot a color picture at the luminance (the first threshold value in the first embodiment). In addition, if the second chrominance component has a value smaller than or equal to the third threshold value, the generation of the color image continues despite the removal of the IRCF 501 from the camera optical path. Further, the third threshold value can be arbitrarily set by the user. Further, the third threshold value and the fourth threshold value can be corrected respectively according to the amount of the first chrominance component with the IRCF 501 inserted. For example, the fourth threshold value can decrease as the amount of the first chrominance component increases. This is because, in this case, it is considered that the saturation of the subject is originally high.

[0080] If the second chrominance component is judged to have a value greater than the third threshold value and smaller than the fourth threshold value in step S610, the upper limit of the infrared illumination is raised to the output upper limit by giving priority to the sensitivity (step S613). Then, a black-and-white image is output by giving priority to the S / N ratio (step S614).

[0081] On the other hand, if the answer of step S610 is judged to be affirmative, the intensity of the illumination of the infrared illumination unit 100 is controlled according to the second chromaticity component so that the intensity is indicated by the solid line between the third threshold value and the fourth threshold value of Figure 10 (step S611).

[0082] Here, Figure 10 is a graph showing the relationship between the threshold value of the visible light and the intensity of the infrared illumination according to the second embodiment. In addition, Figure 11 is a table showing the states of the IRCF 501, the infrared illumination unit 100, and the output unit 405 in each of the range A to the range D in the graph of Figure 10

[0083] Further, at this time, as shown by the broken line in Figure 10 , a control mode in which the timing at which the infrared illumination unit 100 starts to radiate is shifted in the direction in which the amount of the chromaticity component decreases while maintaining the slope of the solid line portion can be stored in the storage unit 104 in advance, and this control mode can be used. Here, the control mode shown by the broken line is an example of a control mode in which the color reproducibility is more important than the S / N ratio compared to the control mode shown by the solid line.

[0084] Further, as another control mode, the intensity of the infrared illumination can be controlled so as to move upward to the right in the range B of Figure 10 . In this case, the color reproducibility is improved. These control modes and the control mode in which the color reproducibility is more important than the S / N ratio (the mode shown by the solid line of Figure 10 ) are stored in the storage unit 104 in advance. Which control mode is to be used can be selected by the user. In addition, Figure 10 , the third threshold value and the fourth threshold value can each be adjusted by the user. Further, the amount of the shift can be adjusted by the user while maintaining the slope characteristic of the infrared illumination intensity in the range B and the range C.

[0085] Further, in the present embodiment, because the color tone of the image tends to be weak due to the irradiation with the infrared illumination, the color saturation of the image is emphasized by the image processing unit 403 (step S612). Then, in steps S611 and S612, the illumination control unit 101 controls the intensity of the illumination of the infrared illumination unit 100 at a predetermined slope in the portion between the third threshold value and the fourth threshold value of Figure 10 , emphasizes the color saturation of the image, and outputs the image as a color image in step S605.

[0086] ​As described above, in the second embodiment as well, when the subject is imaged with the use of the imaging device that can insert or remove the IRCF and can control the intensity of the infrared illumination, the amount of the visible light information (color information) that can be acquired can be more increased compared to the related art, particularly in the chroma range from the third threshold value to the fourth threshold value.

[0087] Further, if the infrared light is radiated at a level where the chroma has a value lower than the third threshold value, the S / N ratio can be improved by the sensitivity of the infrared light to the image sensor at a relatively dark level. Therefore, the color reproducibility and the S / N ratio of the imaged image can be maintained.

[0088] [Third Embodiment]

[0089] Figure 12 is a block diagram showing the structure of the imaging system according to the third embodiment. Because Figure 12 the imaging device 40 and the infrared illumination unit 100 in Figure 4 perform similar processes to those described in Figure 12 , the description thereof will be omitted.

[0090] When the client device 1200 is connected to the imaging device 40 directly or via a network, Figure 12 the communication control unit 1201 in controls various types of communication. In addition, the imaging device 40 has a communication unit that is not shown as a communication section, so that it can communicate with the communication control unit 1201. Further, the communication control unit 1201 can communicate directly with the infrared illumination unit 100, or can communicate with the infrared illumination unit 100 via the imaging device 40. For example, the communication control unit 1201 can transmit a command for causing an instruction regarding each processing operation of the imaging device 40 to be executed to the imaging device 40, and the communication unit of the imaging device 40 receives the command from the outside via a network and applies it to each processing operation, the details of which are as follows.

[0091] Figure 12 the input image acquisition unit 1202 in acquires an image output from the imaging device 40, and causes a display unit that is not shown to display the image, for example. Figure 12 the operation unit 1203 in generates and changes each command to be transmitted to the imaging device 40 on a graphical user interface (GUI) displayed on a display unit that is not shown.

[0092] Figure 13 is a flowchart showing the process of the imaging system according to the third embodiment, and Figure 14is a flowchart showing other part processing of the imaging system according to the third embodiment. Reference will be made to Figures 4 to 20 Using Figure 13 and Figure 14 flowcharts, the processing of the imaging device of Figure 4 will be described. Further, a computer such as the judging unit 103 executes a computer program stored in the storage unit 104 and a program for controlling each unit of the client device 1200 and the like to cause the operation of each step of Figure 13 and Figure 14 to be performed.

[0093] In addition, since each of the steps in steps S601 to S614 of the flowcharts of Figure 13 and Figure 14 indicates processing similar to that of the same steps of the flowcharts in Figure 6 described in the second embodiment, the description thereof will be omitted.

[0094] In step S1301, it is judged whether the current mode is "automatic switching mode (linked to illumination)". Here, Figure 15 is a diagram showing examples of each mode that the operation unit 1203 of the client device 1200 according to the third embodiment can set.

[0095] In the example of Figure 15 , the user can select any one mode from "automatic switching mode", "infrared color mode", "color mode", and "monochrome mode". In addition, when "automatic switching mode" is selected, the mode can be switched to "linked to illumination" by clicking a check box for selecting whether to link the mode to infrared illumination. Further, Figure 16 is a table showing examples of the combination state of the IRCF, infrared illumination, and image output in each mode of Figure 15 . In the present embodiment, a plurality of modes having different combinations of the control state of the infrared illumination, the insertion / removal state of the infrared cut filter with respect to the imaging light path of the imaging lens, and the image output state are stored as in the diagram, and the modes are selectable.

[0096] If "automatic switching mode (linked to illumination)" is not selected in step S1301, it is judged whether "automatic switching mode" has been selected (step S1302).

[0097] If it is not judged that "automatic switching mode" has been selected in step S1302, any one of "infrared color mode", "color mode", and "monochrome mode" has been selected. Therefore, the output is performed in a manner that the IRCF, infrared illumination, and image output are combined according to each mode of Figure 16 (step S1303).

[0098] On the other hand, if it is determined in step S1302 that "automatic switching mode" has been selected, then the insertion / removal of IRCFs and automatic switching of image output are performed based on the illuminance (step S1304). The automatic insertion and removal of IRCFs based on the illuminance of the shooting environment ensures that when the illuminance is brighter than a predetermined threshold, a color image is output with IRCFs inserted, and when the illuminance is darker than the predetermined threshold, infrared components are absorbed by removing IRCFs. Furthermore, the image output when IRCFs are removed can be either a black and white or a color image. After performing steps S1303 and S1304, Figure 13 and Figure 14 The process is complete.

[0099] If "Automatic switching mode (linked to lighting)" has been selected in step S1301, then the illuminance is estimated in step S602, and then the illuminance is obtained for the user in situations such as... Figure 17 The illuminance of the IRCF switch specified on the GUI shown (step S1305). Figure 17 This diagram illustrates an example of a GUI for setting illumination to automatically insert or remove an IRCF according to a third embodiment, where the brightness for IRCF insertion / removal can be set to decrease to the left and increase to the right. That is, in this embodiment, the brightness when inserting or removing (removing) an infrared cutoff filter can be selected by the user.

[0100] Next, after steps S603 to S610, the user's... Figure 18 The priority for correcting the intensity of infrared illumination is specified on the GUI shown (step S1306). Figure 18 This is a diagram illustrating an example of a GUI for setting the priority when correcting illumination intensity in the automatic adjustment of infrared illumination according to the third embodiment. Figure 18 In the middle, the infrared illumination is set to decrease to the left to prioritize color reproducibility, and the infrared illumination is set to increase to the right to prioritize sensitivity.

[0101] Specifically, according to the second embodiment Figure 10 Under the infrared illumination intensity in ranges B and C of the figure, Figure 18 When color reproduction is prioritized on the GUI, the intensity of infrared illumination in ranges B and C is corrected to be weaker (e.g., the curve shown by the dashed line). Conversely, when sensitivity is prioritized, the intensity of infrared illumination in ranges B and C can be corrected to be stronger (e.g., the curve shown by the solid line). That is, in this embodiment, the intensity of infrared illumination is correctable according to the priority set by the user.

[0102] Next, after the step S611, the white balance control range specified by the user on the GUI shown in Figure 19 is acquired (step S1307). Figure 19 is a diagram showing an example of a GUI that sets the priority of the white balance control range according to the third embodiment. In Figure 19 , the white balance control range can be set to narrow as the GUI is slid to the left to consider color more important, and to widen as the GUI is slid to the right to consider white more important.

[0103] Specifically, when the general automatic white balance control is performed, as shown in the diagram of Figure 20 , the white balance processing control range of the image processing unit 403 is changed. Figure 20 is a diagram showing an example of setting the white balance control range in Figure 19 according to the third embodiment. In Figure 20 , for example, in a case where the white balance control range is set to be wide, the white balance gains (R gain and B gain) are controlled with the range of 1901. In addition, for example, in a case where the white balance control range is set to be narrow, the white balance gains are controlled in the range of 1902.

[0104] Further, according to the above description, the image processing unit 403 functions as a white balance control section, and the user can select the white balance control range on the GUI of Figure 19 using the white balance control section. That is, the imaging device 40 can receive a command from the outside via a network, and control at least one of the intensity of the infrared illumination, the insertion / removal of the infrared cut filter with respect to the imaging optical path of the imaging lens, and the white balance.

[0105] As described above, in the third embodiment, in particular, in the chroma range from the third threshold value to the fourth threshold value, the amount of the acquired visible light information (color information) increases, and the image can be output while further reflecting the image quality preferred by the user therein.

[0106] Although the present application has been described with reference to exemplary embodiments, it is to be understood that the application is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be given the broadest interpretation to encompass all such modifications and equivalent structures and functions. In addition, as part or all of the control according to the present embodiment, a computer program that implements the functions of the above-described embodiments can be supplied to the lighting control device through a network or various storage media. Then, a computer (or CPU or MPU or the like) of the lighting control device can be configured to read and execute the program. In this case, the program and the storage medium that stores the program constitute the present application.

[0107] This application claims the benefit of Japanese Patent Application No. 2021-064246, filed April 5, 2021, and Japanese Patent Application No. 2022-009795, filed January 26, 2022, both of which are hereby incorporated by reference herein in their entirety.

Claims

1. An imaging apparatus having an imaging lens and an imaging unit, the imaging apparatus comprising: an infrared cut filter inserted in or removed from an imaging light path of the lens; a visible light measuring unit configured to measure an amount of a visible light component; an infrared illumination unit configured to perform infrared illumination; a control unit configured to control an intensity of infrared illumination performed by the infrared illumination unit, wherein the infrared cut filter is removed from the imaging light path in a case where the amount of the visible light component is less than or equal to a threshold value, wherein the imaging apparatus further comprises a calculation unit configured to: calculate a first chroma component from an image in a case where the infrared cut filter is inserted in the imaging light path; and calculate a second chroma component from the image in a case where the infrared cut filter is removed from the imaging light path, and wherein the control unit controls the intensity of infrared illumination performed by the infrared illumination unit based on the second chroma component in a case where i) the first chroma component is equal to or greater than a first chroma threshold value, and ii) the second chroma component is equal to or greater than the first chroma threshold value and less than or equal to a second chroma threshold value.

2. The imaging apparatus according to claim 1, wherein the visible light measuring unit measures the amount of the visible light component in accordance with at least one of an exposure amount of the imaging unit, luminance information of an image acquired from the imaging unit, and a gain of the imaging unit.

3. The camera of claim 1, wherein a color image is generated in a case where the infrared cut filter is inserted in the imaging light path, and the color image or a monochrome image is generated in accordance with the first chroma component, the second chroma component, and the intensity of infrared illumination radiated by the infrared illumination unit in a case where the infrared cut filter is removed from the imaging light path.

4. The imaging apparatus according to claim 3, wherein in a case where the first chroma component or the second chroma component is less than the first chroma threshold value, the intensity of the infrared illumination unit is boosted to an upper limit.

5. The imaging apparatus according to claim 1, wherein a plurality of modes having different combinations of a control state of the infrared illumination, an insertion / removal state of the infrared cut filter with respect to the imaging light path of the imaging lens, and an image output state are stored, and the modes are selectable.

6. A control method comprising: measuring an amount of a visible light component; controlling an intensity of infrared illumination performed by an infrared illumination unit, wherein the infrared cut filter is removed from an imaging light path in a case where the amount of the visible light component is less than or equal to a threshold value, wherein a first chroma component is calculated from an image in a case where the infrared cut filter is inserted in the imaging light path; and a second chroma component is calculated from the image in a case where the infrared cut filter is removed from the imaging light path, and wherein, i) in case said first chroma component is equal to or larger than a first chroma threshold, and ii) in case said second chroma component is equal to or larger than said first chroma threshold and is smaller than or equal to a second chroma threshold, the intensity of the infrared illumination by said infrared illumination unit is controlled based on said second chroma component.

7. A computer readable storage medium storing a program which, when executed by a computer, implements the steps of the control method according to claim 6.

8. A computer program product comprising a program which, when executed by a computer, implements the steps of the control method according to claim 6.

Images