Backlight correction device, operation method of the backlight correction device, operation program of the backlight correction device, and imaging apparatus

Abstract

To provide a backlight correction device capable of performing an appropriate backlight correction, an operation method of the backlight correction device, an operation program of the backlight correction device, and an imaging apparatus.SOLUTION: A first determination part performs a first determination for determining so that a photographing scene of an image is a backlight scene when a difference between a representative value of a region exposure value in a reference region that becomes a reference of a backlight correction in the image and a representative value of a region exposure value in a peripheral region of the reference region is a first threshold value or larger that is previously set. An exposure control part performs the backlight correction by performing an exposure correction to a direction where a whole brightness of the image becomes bright in the case where it is determined by the first determination that it is the backlight scene. A second determination part performs a second determination determining whether or not there is a saturated region in the image. The first determination part sets the first threshold value to a value that is lower than a value in the case where it is determined that it is the backlight scene by the first determination, and there is the saturation region by the second determination as compared with the case where it is determined that there is no saturation region.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The technique disclosed herein relates to a backlight compensation device, an operating method for a backlight compensation device, an operating program for a backlight compensation device, and an imaging device. [Background technology]

[0002] Patent Document 1 describes an imaging device that includes an imaging unit that captures an image of a subject and generates image data, a first detection unit that detects a face area corresponding to the subject's face in the image data, a second detection unit that detects a subject area corresponding to at least a part of the subject in the image data, and a control unit that controls exposure by photometry of the image data. When the face area is detected by the first detection unit, the control unit corrects the exposure according to the photometry of the detected face area. Then, when the face area is not detected by the first detection unit thereafter and a subject area corresponding to the subject of the detected face area is detected by the second detection unit, the control unit continues to correct the exposure according to the photometry of the detected face area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6817590 Summary of the Invention

[0004] One embodiment of the technique of the present disclosure provides a backlight correction device capable of performing appropriate backlight correction, a method for operating a backlight correction device, an operating program for a backlight correction device, and an imaging device. [Means for solving the problem]

[0005] The backlight correction device disclosed herein includes a processor, and when the difference in brightness between a reference area in an image that serves as the basis for backlight correction and an area surrounding the reference area is equal to or greater than a predetermined first threshold, the processor makes a first determination to determine that the scene in the image is a backlit scene in which light hits the subject from behind, and when the first determination determines that the scene is a backlit scene, the processor performs backlight correction by adjusting the exposure in a direction that makes the overall brightness of the image brighter, and makes a second determination to determine whether or not the image has any saturated areas, and when the first determination determines that the scene is a backlit scene and the second determination determines that there are saturated areas, the processor sets the first threshold to a lower value than when it is determined that there are no saturated areas.

[0006] It is preferable that the processor subjects the surrounding area to the second determination.

[0007] It is preferable that the processor makes the second determination based on a comparison result between a difference in brightness between a peripheral region among a plurality of divided regions obtained by dividing the image and the entire image, and a preset second threshold value.

[0008] The processor preferably determines that there is a saturated region when there is a specified number or more of surrounding regions whose brightness difference from the entire image is equal to or greater than a second threshold value.

[0009] The reference region is preferably any one of the central region of the image, the region in which the subject is detected, and a region designated by the user.

[0010] The imaging device of the present disclosure includes the above-described backlight correction device.

[0011] The imaging device of the present disclosure is preferably a surveillance camera.

[0012] The operation method of the backlight correction device disclosed herein includes making a first determination that the scene in the image is a backlit scene in which light hits a subject from behind, when the difference in brightness between a reference area in an image that serves as a basis for backlight correction and an area surrounding the reference area is equal to or greater than a preset threshold; if the first determination determines that the scene is a backlit scene, performing backlight correction by adjusting the exposure in a direction that makes the overall brightness of the image brighter; making a second determination that determines whether the image has any saturated areas; and, if the first determination determines that the scene is a backlit scene and the second determination determines that there are saturated areas, setting the threshold to a lower value than when it is determined that there are no saturated areas.

[0013] The operating program for the backlight correction device disclosed herein causes a computer to execute processes including: making a first determination that the scene in the image is a backlit scene in which light hits the subject from behind, when the difference in brightness between a reference area in an image that serves as the basis for backlight correction and an area surrounding the reference area is equal to or greater than a preset threshold; if the first determination determines that the scene is a backlit scene, performing backlight correction by adjusting the exposure in a direction that makes the overall brightness of the image brighter; making a second determination that determines whether the image has any saturated areas; and, if the first determination determines that the scene is a backlit scene and the second determination determines that there are saturated areas, setting the threshold to a lower value than when it is determined that there are no saturated areas. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing a surveillance camera system. [Diagram 2] FIG. 1 is a block diagram of a surveillance camera. [Diagram 3] FIG. 2 is a block diagram of a control unit. [Figure 4] FIG. 2 is a block diagram showing a processing unit of a CPU. [Diagram 5] 10 is a flowchart showing a processing procedure of an exposure value calculation unit. [Figure 6] FIG. [Figure 7]FIG. 11 is a diagram showing exposure value information. [Figure 8] 13 is a flowchart showing a processing procedure of a first determination unit. [Figure 9] 13 is a flowchart showing a procedure for setting a first threshold value. [Figure 10] FIG. 1 is a diagram showing a reference region and a surrounding region. [Figure 11] 13 is a flowchart showing a processing procedure of a second determination unit. [Figure 12] FIG. 4 is a diagram illustrating a process of an exposure control unit. [Figure 13] 13 is a flowchart showing a processing procedure of a CPU of the surveillance camera. [Figure 14] FIG. 7 is a schematic diagram of the brightness of a region along line XX of the image shown in FIG. 6, and is a diagram showing a comparative example in which the first threshold value is fixed to a standard value. [Figure 15] FIG. 7 is a schematic diagram of the brightness of the region along line XX of the image shown in FIG. 6, showing an example in which the first threshold value is changed to a standard value and a value lower than the standard value. [Figure 16] FIG. 13 is a diagram showing an example in which a face region is used as a reference region. [Figure 17] FIG. 13 is a diagram showing an example in which a region specified by a user is set as a reference region. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] [First embodiment] As an example, as shown in FIG. 1, a surveillance camera system 10 includes a surveillance camera 11 and a management device 12. The surveillance camera 11 is installed on the street or inside a building, and captures the state of the street or inside the building. The management device 12 is installed in a control room where a user U, who is the administrator of the surveillance camera 11, is always present. The management device 12 has a monitor 13 that displays an image 65 (see FIG. 4 and FIG. 6) from the surveillance camera 11, and an input device 14 such as a keyboard and a mouse for inputting various operation instructions to the surveillance camera 11. The surveillance camera 11 and the management device 12 are connected via a network 15. The network 15 is, for example, a WAN (Wide Area Network) such as the Internet or a public communication network. Although only one surveillance camera 11 is connected to the management device 12 in FIG. 1, a plurality of surveillance cameras 11 may be connected to the management device 12.

[0016] As an example, as shown in FIG. 2, the surveillance camera 11 includes an imaging optical system 20 and an imaging element 21. The imaging optical system 20 includes a plurality of lenses for forming an image of subject light on the imaging element 21. Specifically, the imaging optical system 20 includes an objective lens 22, a focus lens 23, and a zoom lens 24. These lenses 22-24 are arranged in this order from the object side (subject side) toward the image formation side (image element 21 side). Although simplified in FIG. 1, each of the lenses 22-24 is actually a lens group in which a plurality of lenses are combined. The imaging optical system 20 also includes an aperture 25. The aperture 25 is arranged on the closest image formation side of the imaging optical system 20. The focus lens 23 is provided with a focus lens driving mechanism 26, the zoom lens 24 is provided with a zoom lens driving mechanism 27, and the aperture 25 is provided with an aperture opening adjustment mechanism 28.

[0017] Focus lens driving mechanism 26 holds focus lens 23 and includes a focus cam ring having a cam groove formed on its outer periphery, a focus motor that moves the focus cam ring along the optical axis OA by rotating the focus cam ring about the optical axis OA, a driver for the focus motor, etc. Similarly, zoom lens driving mechanism 27 holds zoom lens 24 and includes a zoom cam ring having a cam groove formed on its outer periphery, a zoom motor that moves the zoom cam ring along the optical axis OA by rotating the zoom cam ring about the optical axis OA, a driver for the zoom motor, etc.

[0018] The diaphragm 25 is a so-called iris diaphragm that is composed of a combination of multiple diaphragm blades. The diaphragm 25 simultaneously moves multiple diaphragm blades using a cam mechanism to open and close a central opening formed by the inner edges of the multiple diaphragm blades, that is, to change the opening degree of the opening (hereinafter referred to as the opening degree of the diaphragm 25), thereby adjusting the amount of light passing through. The diaphragm opening adjustment mechanism 28 includes a diaphragm motor that opens and closes the diaphragm blades, a driver for the diaphragm motor, and the like.

[0019] The focus motor, zoom motor, and aperture motor are, for example, stepping motors. In this case, the positions of the focus lens 23 and the zoom lens 24 on the optical axis OA, and the opening degree of the aperture 25 can be derived from the drive amounts of the focus motor, zoom motor, and aperture motor. Note that instead of the drive amounts of the focus motor and the zoom motor, a position sensor may be provided to detect the positions of the focus lens 23 and the zoom lens 24.

[0020] The motors (focus motor, zoom motor, and aperture motor) or electric components such as drivers of each of the drive mechanisms 26-28 are connected to the control unit 29. The electric components of each of the drive mechanisms 26-28 are driven under the control of the control unit 29. More specifically, the control unit 29 issues drive signals according to various operation instructions input via the input device 14 of the management device 12 to drive the electric components of each of the drive mechanisms 26-28. For example, when an operation instruction to change the angle of view to the telephoto side is input, the control unit 29 issues a drive signal to a driver of the zoom motor of the zoom lens drive mechanism 27 to move the zoom lens 24 to the telephoto side.

[0021] The focus motor, zoom motor, and diaphragm motor output their drive amounts to the control unit 29. The control unit 29 derives the positions of the focus lens 23 and zoom lens 24 on the optical axis OA and the opening degree of the diaphragm 25 from the drive amounts.

[0022] The imaging element 21 is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and has an imaging surface that captures subject light. The imaging surface is formed by a plurality of pixels arranged two-dimensionally. The pixels accumulate signal charges corresponding to the subject light, and output an image signal (voltage signal) corresponding to the signal charges. The imaging element 21 is arranged so that the center of the imaging surface coincides with the optical axis OA, and the imaging surface is perpendicular to the optical axis OA. Note that "coincidence" and "orthogonality" here refer to perfect coincidence and orthogonality, as well as coincidence and orthogonality that include errors generally permitted in the technical field to which the technology of the present disclosure belongs.

[0023] An imaging element driver 30 is connected to the imaging element 21. The imaging element driver 30 is connected to the control unit 29. Under the control of the control unit 29, the imaging element driver 30 controls the timing of imaging of subject light by the imaging element 21 by, for example, supplying a vertical scanning signal and a horizontal scanning signal to the imaging element 21. The imaging element driver 30 also sets a gain to be given to an image signal output from a pixel, which gain corresponds to ISO (International Organization for Standardization) sensitivity.

[0024] A shutter 31 is provided between the imaging optical system 20 and the imaging element 21. The shutter 31 is, for example, a focal plane shutter having a front curtain and a rear curtain. A shutter drive mechanism 32 is connected to the shutter 31. The shutter drive mechanism 32 includes an electromagnet that holds the front curtain and the rear curtain and releases the hold to cause the front curtain and the rear curtain to run, and a driver for the electromagnet. The shutter drive mechanism 32 is driven under the control of the control unit 29 to open and close the shutter 31. Note that the shutter is not limited to the exemplified mechanical shutter 31, and may be an electronic shutter. In particular, electronic shutters are often used in surveillance cameras.

[0025] The control unit 29 is connected to each unit, such as an image input controller 33, an image memory 34, and an image processing unit 35, via a bus line 36. In addition, a VRAM (Video Random Access Memory) 37, an image distribution control unit 38, and an instruction receiving unit 39 are also connected to the bus line 36. Although not shown in the figure, the bus line 36 is also connected to a power supply unit that supplies power to each unit of the surveillance camera 11, an external communication I / F (Interface) that communicates with an external device via a connection terminal such as a USB (Universal Serial Bus) terminal, a wireless communication I / F, etc.

[0026] An image 65 based on an image signal obtained by capturing an image of the subject light is input from the imaging element 21 to the image input controller 33. The image input controller 33 outputs the image 65 to the image memory 34. The image memory 34 is, for example, a Synchronous Dynamic Random Access Memory (SDRAM), and temporarily stores the image 65.

[0027] The image processing unit 35 reads out an unprocessed image 65 from the image memory 34. The image processing unit 35 performs various image processing on the image 65. The various image processing includes, for example, offset correction processing, sensitivity correction processing, pixel interpolation processing, white balance correction processing, gamma correction processing, demosaic processing, luminance signal and color difference signal generation processing, contour enhancement processing, color correction processing, etc. The image processing unit 35 writes the image 65 after the various image processing back to the image memory 34.

[0028] The VRAM 37 receives images 65 after various image processing, which are to be displayed on the monitor 13 of the management device 12, from the image memory 34. The VRAM 37 has an area for storing two consecutive frames of images 65. The images 65 stored in the VRAM 37 are successively rewritten with new images 65. The VRAM 37 sequentially outputs the newer image 65 of the two consecutive frames of images 65 to the image delivery control unit 38.

[0029] The image delivery control unit 38 controls delivery of the image 65 from the VRAM 37 to the management device 12. This allows the user U to view the image 65 through the monitor 13 of the management device 12. The display frame rate of the image 65 is, for example, 5 fps (frames per seconds) to 30 fps.

[0030] The instruction receiving unit 39 receives various operation instructions input by the user U via the input device 14 of the management apparatus 12. The instruction receiving unit 39 outputs the received various operation instructions to the control unit 29 via the bus line 36.

[0031] The various operational instructions include, in addition to the above-mentioned operational instruction for changing the angle of view, an operational instruction for manually setting the aperture of the aperture 25, the ISO sensitivity, and the shutter speed of the shutter 31. There are also operational instructions for taking a still image or a video of the image 65, or for changing the orientation of the surveillance camera 11 by operating a pan-tilt mechanism (not shown).

[0032] 3, the control unit 29 includes a storage 45, a CPU (Central Processing Unit) 46, and a memory 47. The storage 45, the CPU 46, and the memory 47 are interconnected via a bus line 48. The control unit 29 is an example of a "backlight correction device" and a "computer" according to the techniques of the present disclosure.

[0033] The storage 45 is a non-volatile storage device such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). The storage 45 stores various programs and various data associated with the programs. Instead of the EEPROM, the storage 45 may be a Ferroelectric Random Access Memory (FeRAM) or a Magnetoresistive Random Access Memory (MRAM).

[0034] The memory 47 is a work memory for the CPU 46 to execute processing. The CPU 46 loads a program stored in the storage 45 into the memory 47 and executes processing according to the program. In this way, the CPU 46 comprehensively controls each part of the surveillance camera 11. The CPU 46 is an example of a "processor" according to the technology of the present disclosure. The memory 47 may be built into the CPU 46.

[0035] As an example, as shown in FIG. 4, an operating program 50 is stored in the storage 45. The operating program 50 is a program for causing the CPU 46 to execute backlight correction and the like. In other words, the operating program 50 is an example of an "operating program of a backlight correction device" according to the technology of the present disclosure. In addition to the operating program 50, the storage 45 also stores a first threshold 51, a second threshold 52, and a program diagram 53. The first threshold 51 is an example of a "threshold" according to the technology of the present disclosure.

[0036] When the operating program 50 is started, the CPU 46 cooperates with the memory 47 and the like to function as an image acquisition section 55, an exposure value calculation section 56, a first determination section 57, a second determination section 58, and an exposure control section 59.

[0037] The image acquisition unit 55 acquires the processed image 65 by reading it from the image memory 34. The image acquisition unit 55 outputs the image 65 to the exposure value calculation unit 56.

[0038] The exposure value calculation unit 56 calculates an exposure value according to the brightness of the image 65. The exposure value calculation unit 56 generates exposure value information 66 that summarizes the results of the exposure value calculation. The exposure value calculation unit 56 outputs the exposure value information 66 to the first determination unit 57, the second determination unit 58, and the exposure control unit 59.

[0039] The first threshold value 51 is input to the first determination unit 57. The first determination unit 57 performs a first determination using the first threshold value 51 and exposure value information 66. The first determination is a determination of whether or not the shooting scene of the image 65 is a backlit scene. A backlit scene is a shooting scene in which light hits a subject from behind, making the surroundings of the subject appear bright and the subject appear dark. The first determination unit 57 outputs a first determination result 67, which is the result of the first determination, to the exposure control unit 59.

[0040] The second threshold value 52 is input to the second determination unit 58. The second determination unit 58 performs a second determination using the second threshold value 52 and exposure value information 66. The second determination is a determination as to whether or not there is a saturated area in the image 65 (hereinafter referred to as a saturated area). A saturated area is an area where the image signals of each color of RGB reach their upper limit values ​​and appear white, that is, a so-called blown-out white area. The second determination unit 58 outputs a second determination result 68, which is the result of the second determination, to the first determination unit 57.

[0041] The exposure control unit 59 receives the program chart 53. The exposure control unit 59 performs exposure control to obtain a standard exposure image 65 based on the program chart 53, exposure value information 66, and a first determination result 67. More specifically, the exposure control unit 59 derives the aperture of the diaphragm 25, the ISO sensitivity, and the shutter speed of the shutter 31 to obtain the standard exposure image 65 from the program chart 53 and the overall exposure value of the exposure value information 66 (see FIG. 7). The exposure control unit 59 outputs drive signals for setting the derived aperture of the diaphragm 25, the ISO sensitivity, and the shutter speed to the aperture opening adjustment mechanism 28, the image sensor driver 30, and the shutter drive mechanism 32. The standard exposure image 65 refers to an image with the same brightness as when actually viewed by the eye.

[0042] The exposure value calculation unit 56 calculates the exposure value in the procedure shown in FIG. 5 as an example. First, the exposure value calculation unit 56 calculates a representative value of brightness for each of a plurality of divided regions 70 obtained by dividing an image 65 as shown in FIG. 6 as an example (step ST10). The representative value of brightness is, for example, the average value or median value of luminance signals generated from image signals output by pixels in the divided regions 70. FIG. 6 illustrates an example of an image 65 showing a family walking between two standing trees on a lawn in a park with the sun shining behind them. FIG. 6 also illustrates an example of a case where the image 65 is divided into 16×16=256 equal parts, and 256 divided regions 70 are prepared. Note that the number of divided regions 70 is not limited to 256 as illustrated. It may be 8×8=64 or 24×24=576.

[0043] The exposure value calculation unit 56 calculates a weighted average value of the representative brightness value for each divided region 70 (step ST11). The weighted average value is calculated by adding values ​​obtained by multiplying the representative brightness value of each divided region 70 by a weight preset for each divided region 70, and dividing the value thus obtained by the number of divided regions 70. The weights are set, for example, so that the values ​​gradually decrease from the center of the image 65 toward the periphery.

[0044] The exposure value calculation unit 56 calculates the exposure value of each divided region 70 (hereinafter, referred to as the region exposure value) based on the representative value of the brightness of each divided region 70 (step ST12). The exposure value calculation unit 56 also calculates the overall exposure value of the image 65 (hereinafter, referred to as the overall exposure value) based on the weighted average value (step ST12). As an example, as shown in FIG. 7, the exposure value calculation unit 56 generates and outputs the region exposure value and the overall exposure value together as exposure value information 66 (step ST13). Note that the numbers such as "0101" and "0102" assigned to the divided regions in FIG. 7 are a sequence of numbers assigned to the matrix of the divided regions 70 in FIG. 6. For example, "0403" represents the divided region 70 in the fourth row and third column, and "0709" represents the divided region 70 in the seventh row and ninth column.

[0045] The first determination unit 57 performs the first determination in the procedure shown in FIG. 8 and FIG. 9 as an example. First, the first determination unit 57 sets the first threshold 51 (step ST20). As shown in FIG. 9, if the exposure control unit 59 has performed exposure correction according to a backlit scene in the previous frame, and the second determination result 68 in the previous frame indicates that the image 65 has a saturated region (YES in both steps ST201 and ST202), the first determination unit 57 sets the first threshold 51 to a value lower than the standard value (step ST203). On the other hand, if the exposure control unit 59 has not performed exposure correction according to a backlit scene in the previous frame (NO in step ST201), the first determination unit 57 sets the first threshold 51 to the standard value (step ST204). In the previous frame, the exposure control unit 59 performs exposure correction according to a backlit scene, but if the second judgment result 68 in the previous frame indicates that there is no saturated area in the image 65 (YES in step ST201, NO in step ST202), the first judgment unit 57 also sets the first threshold 51 to a standard value (step ST204). Note that the standard value of the first threshold 51 is, for example, 2 EV, and a value lower than the standard value is, for example, 0.5 EV.

[0046] In FIG. 8, after setting the first threshold value 51, the first determination unit 57 calculates a reference region serving as a reference for backlight correction and a representative value of the regional exposure values ​​of the peripheral regions of the reference region based on the regional exposure value of the exposure value information 66 (step ST21). As shown by solid lines and hatching in FIG. 10 as an example, the reference region is 4×4=16 divided regions 70 in the center of the image 65, that is, the central region of the image 65. In terms of numbers, the reference region is the divided regions 70 of "0707", "0708", "0709", "0710", "0807", "0808", "0809", "0810", "0907", "0908", "0909", "0910", "1007", "1008", "1009", and "1010". On the other hand, the peripheral regions are the 240 divided regions 70 other than the above 16 divided regions 70. The representative value of the area exposure value of the reference area is the average value, median, etc. of the area exposure values ​​of the above 16 divided areas 70. Similarly, the representative value of the area exposure value of the peripheral area is the average value, median, etc. of the area exposure values ​​of the 240 divided areas 70 other than the above 16 divided areas 70.

[0047] The first determination unit 57 calculates the difference between the representative values ​​of the area exposure values ​​of the reference area and the surrounding area by subtracting the representative value of the area exposure values ​​of the reference area from the representative value of the area exposure values ​​of the surrounding area (step ST22). This difference between the representative values ​​of the area exposure values ​​of the reference area and the surrounding area is larger when the shooting scene of the image 65 is a backlit scene than when it is not. The difference between the representative values ​​of the area exposure values ​​of the reference area and the surrounding area is an example of the "difference in brightness between the reference area and the surrounding area" according to the technology of the present disclosure. Note that the "difference in brightness between the reference area and the surrounding area" may also be the difference in the representative values ​​of the luminance signals of the reference area and the surrounding area.

[0048] The first determination unit 57 compares the difference between the representative values ​​of the area exposure values ​​of the reference area and the peripheral area with the first threshold value 51 set in step ST20 (step ST23). If the difference between the representative values ​​of the area exposure values ​​of the reference area and the peripheral area is equal to or greater than the first threshold value 51 (YES in step ST24), the first determination unit 57 determines that the scene captured in the image 65 is a backlit scene (step ST25). On the other hand, if the difference between the representative values ​​of the area exposure values ​​of the reference area and the peripheral area is less than the first threshold value 51 (NO in step ST24), the first determination unit 57 determines that the scene captured in the image 65 is not a backlit scene (step ST26). The first determination unit 57 outputs the first determination result 67 (step ST27).

[0049] The second determination unit 58 performs the second determination in the procedure shown in FIG. 11 as an example. First, the second determination unit 58 calculates the difference between the area exposure value of the peripheral area and the overall exposure value for each peripheral area by subtracting the overall exposure value from the area exposure value of each peripheral area (step ST30). As can be seen from the processing of step ST30, only the peripheral area is the subject of the second determination, and the reference area is excluded from the subject of the second determination. The difference between the area exposure value of the peripheral area and the overall exposure value is an example of the "difference in brightness between the peripheral area and the entire image" according to the technology of the present disclosure. Note that the "difference in brightness between the peripheral area and the entire image" may be the difference between the representative value of the luminance signal of the peripheral area and the representative value of the luminance signal of the entire image 65.

[0050] The second determination unit 58 compares the difference between the area exposure value and the overall exposure value of the surrounding area with the second threshold value 52 (step ST31), and counts the number of surrounding areas whose difference between the area exposure value and the overall exposure value is equal to or greater than the second threshold value 52 (step ST32).

[0051] If the number of surrounding areas in which the difference between the area exposure value and the overall exposure value is equal to or greater than the second threshold value 52 is equal to or greater than the specified number (YES in step ST33), the second determination unit 58 determines that the image 65 has a saturated area (step ST34). On the other hand, if the number of surrounding areas in which the difference between the area exposure value and the overall exposure value is equal to or greater than the second threshold value 52 is less than the specified number (NO in step ST33), the second determination unit 58 determines that the image 65 does not have a saturated area (step ST35). The second determination unit 58 outputs a second determination result 68 (step ST36). The second threshold value 52 is, for example, 2.5 EV, and the specified number is, for example, 10.

[0052] 12(A), when the first determination result 67 indicates that the scene in which the image 65 was taken is a backlit scene, the exposure control unit 59 performs exposure correction. On the other hand, when the first determination result 67 indicates that the scene in which the image 65 was taken is not a backlit scene, as shown in (B), the exposure control unit 59 does not perform exposure correction.

[0053] Exposure compensation performed when the photographing scene of the image 65 is a backlit scene is a process of raising the overall brightness of the image 65, and is a process of correcting the backlight. More specifically, exposure compensation is a process of correcting the aperture of the diaphragm 25, the ISO sensitivity, and the shutter speed of the shutter 31 derived from the program chart 53 and the overall exposure value of the exposure value information 66 in a direction that makes the overall brightness of the image 65 brighter. Correcting the overall brightness of the image 65 in a direction that makes the aperture of the diaphragm 25 larger, the ISO sensitivity set higher, and the shutter speed slower. The degree to which the overall brightness of the image 65 is raised by the exposure compensation is, for example, about +2EV in terms of exposure value.

[0054] Next, the operation of the above configuration will be described with reference to the flowchart shown in Fig. 13 as an example. The CPU 46 of the control unit 29 functions as an image acquisition unit 55, an exposure value calculation unit 56, a first determination unit 57, a second determination unit 58, and an exposure control unit 59 when the operating program 50 is started, as shown in Fig. 4.

[0055] In the surveillance camera 11, the subject light focused by the imaging optical system 20 is captured by the imaging element 21, and an image 65 is output from the imaging element 21. The image 65 is subjected to various image processing by the image processing unit 35. The image 65 subjected to various image processing is delivered to the management device 12 under the control of the image delivery control unit 38, and is displayed on the monitor 13 of the management device 12. The image 65 is also acquired by being read out by the image acquisition unit 55 (step ST100). The image 65 is output from the image acquisition unit 55 to the exposure value calculation unit 56.

[0056] 5, the exposure value calculation unit 56 calculates an area exposure value which is an exposure value of each divided area, and an overall exposure value which is an exposure value of the entire image 65 (step ST110). Exposure value information 66 including the area exposure values ​​and the overall exposure value is output from the exposure value calculation unit 56 to the first determination unit 57, the second determination unit 58, and the exposure control unit 59.

[0057] 8 and 9, the first determination unit 57 performs a first determination as to whether or not the photographed scene of the image 65 is a backlit scene (step ST120). The first determination result 67 is output from the first determination unit 57 to the exposure control unit 59.

[0058] In the first judgment, the first threshold 51 is set to a standard value or a value lower than the standard value based on the second judgment result 68 by the second judgment unit 58 as to whether or not the image 65 has a saturated region. Specifically, the first threshold 51 is set to the standard value when the exposure control unit 59 has not performed exposure correction according to a backlit scene in the previous frame, and when the exposure control unit 59 has performed exposure correction according to a backlit scene in the previous frame but the second judgment result 68 for the previous frame indicates that the image 65 has no saturated region. On the other hand, when the exposure control unit 59 has performed exposure correction according to a backlit scene in the previous frame and the second judgment result 68 for the previous frame indicates that the image 65 has a saturated region, the first threshold 51 is set to a value lower than the standard value. In addition, if the second judgment unit 58 judges that the image 65 has a saturated area, and the first judgment unit 57 sets the first threshold 51 to a value lower than the standard value, the shooting scene changes and the second judgment unit 58 judges that the image 65 does not have a saturated area, the first judgment unit 57 resets the first threshold 51 to the standard value.

[0059] 11, the second determination unit 58 performs a second determination as to whether or not there is a saturated region in the image 65 (step ST130). The second determination result 68 is output from the second determination unit 58 to the first determination unit 57.

[0060] Exposure control section 59 performs exposure control based on program chart 53 and the overall exposure value (step ST140). At this time, as shown in Fig. 12(A), if first judgment result 67 indicates that the scene in which image 65 was photographed is a backlit scene, exposure correction is performed in the direction of increasing the overall brightness of image 65. The processes of steps ST100 to ST140 are repeated as long as surveillance camera 11 is not turned off (NO in step ST150).

[0061] As described above, the first determination unit 57 of the CPU 46 of the control unit 29 performs a first determination to determine that the photographed scene of the image 65 is a backlit scene when the difference between the representative value of the regional exposure value of the reference region serving as the reference for backlight compensation in the image 65 and the representative value of the regional exposure value of the region surrounding the reference region is equal to or greater than the first threshold value 51 set in advance. When the exposure control unit 59 determines that the scene is a backlit scene by the first determination, it performs backlight compensation by correcting the exposure in a direction that makes the overall brightness of the image 65 brighter. The second determination unit 58 performs a second determination to determine whether or not the image 65 has a saturated region. When the first determination unit 57 determines that the scene is a backlit scene by the first determination and determines that there is a saturated region by the second determination, it sets the first threshold value 51 to a lower value than when it determines that there is no saturated region.

[0062] Fig. 14 and Fig. 15 are schematic diagrams showing the brightness of the area along the line XX of the image 65 shown in Fig. 6. Fig. 14 shows a case where the first threshold 51 is fixed to a standard value (comparative example). In contrast, Fig. 15 shows a case where the first threshold is changed to the standard value and a value lower than the standard value (this example).

[0063] The brightness of the image 65 is low in the area of ​​the two trees and the family, and reaches the upper limit and is saturated in the other areas. The area of ​​the family corresponds to the central area, i.e., the reference area. The difference ΔEV between the representative value EVX of the area exposure value of the reference area and the representative value EVY of the area exposure value of the peripheral area is equal to or greater than the first threshold value (standard value) TH1. Therefore, in the case of the situation shown in FIG. 14 and FIG. 15, the first determination unit 57 determines that the shooting scene of the image 65 is a backlit scene. The exposure control unit 59 performs exposure correction. This increases the overall brightness of the image 65. However, the saturated area cannot be increased any further, and only the brightness of the area other than the saturated area including the reference area is increased. Then, the value of the representative value EVX of the area exposure value of the reference area increases, and accordingly the difference ΔEV between the representative value EVX of the area exposure value of the reference area and the representative value EVY of the area exposure value of the peripheral area decreases.

[0064] In the comparative example shown in Fig. 14, the first threshold value 51 is fixed at the standard value TH1. Therefore, when exposure compensation is performed and the value of EVX increases, and ΔEV decreases accordingly, ΔEV becomes less than TH1. In this case, the first determination unit 57 erroneously determines that the scene captured in the image 65 is not a backlit scene, even though it is actually a backlit scene. In this case, the exposure control unit 59 cancels the exposure compensation. Thereafter, the implementation and cancellation of this exposure compensation is repeated until the scene changes, causing the overall brightness of the image 65 to change rapidly.

[0065] In contrast, in this example shown in Fig. 15, the first threshold value 51 is set to a value TH1L lower than the standard value TH1. Therefore, even if the exposure correction is performed and the EVX value increases, and ΔEV decreases accordingly, the first determination unit 57 can correctly determine that the scene in which the image 65 was captured is a backlit scene. This makes it possible to avoid a situation in which the exposure correction is repeatedly performed and released, as in the comparative example, causing the overall brightness of the image 65 to change rapidly. Therefore, it becomes possible to perform appropriate backlight correction.

[0066] 11, the second determination unit 58 performs the second determination on the surrounding area, thereby avoiding unnecessary processing.

[0067] 11, the second determination unit 58 performs the second determination based on a comparison result between the difference between the regional exposure value and the overall exposure value of a peripheral region among a plurality of divided regions 70 obtained by dividing the image 65, and a preset second threshold value 52. This can improve the validity of the second determination result 68.

[0068] 11, the second determination result 68 determines that there is a saturated region when there is a specified number or more of surrounding regions where the difference between the regional exposure value and the overall exposure value is equal to or greater than the second threshold value 52. This makes it possible to further improve the validity of the second determination result 68.

[0069] 10, the reference area is the central area of ​​the image 65. Conventionally, there is a high probability that a subject of interest will appear in the central area. Therefore, if the central area is used as the reference area, more appropriate backlight correction can be performed.

[0070] The surveillance camera 11 is particularly required to have stable image quality for the image 65 displayed on the monitor 13, since the surveillance camera 11 is used to clearly display accidents, crimes, and the like and to notify the user U of emergencies. For this reason, the present example using the surveillance camera 11 as the imaging device can greatly exhibit the effect of being able to perform appropriate backlight correction.

[0071] The imaging device according to the technique of the present disclosure is not limited to the exemplified surveillance camera 11. It may be a single-lens reflex digital camera, a compact digital camera, a smartphone, or a tablet terminal.

[0072] (Variation 1) The reference area is not limited to the central area of ​​the example image 65. As an example, as shown in FIG. 16, a face area 75 of a person detected by a known face detection function may be the reference area. In this case, the area other than the face area 75 becomes the peripheral area. The face area 75 is an example of a "area where a subject is detected" according to the technology of the present disclosure. The area where a subject is detected may be a person's torso area, an animal's face or torso area, or the front or torso area of ​​a vehicle such as a car, a train, or an airplane.

[0073] (Variation 2) 17, an area 78 designated by a user U (hereinafter, referred to as a designated area) may be used as the reference area. In this case, the area other than the designated area 78 is the peripheral area. Note that the designated area 78 is an area in which automatic focus adjustment is performed based on an image signal of the designated area 78, for example.

[0074] Only when the first determination is made that the scene is a backlit scene, the second determination may be made.

[0075] The function of the backlight correction device may be performed by the management device 12 .

[0076] In the above embodiment, the following various processors can be used as the hardware structure of the processing units that execute various processes, such as the image processing unit 35, image distribution control unit 38, instruction receiving unit 39, image acquisition unit 55, exposure value calculation unit 56, first determination unit 57, second determination unit 58, and exposure control unit 59. The various processors include the CPU 46, which is a general-purpose processor that executes software (operation program 50) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and / or a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0077] A single processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA). Also, multiple processing units may be composed of a single processor.

[0078] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System On Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0079] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0080] From the above description, the technology described in the following supplementary paragraphs can be understood.

[0081] [Additional note 1] A processor is provided. The processor, performing a first determination that, when a difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is equal to or greater than a first threshold value set in advance, the scene in which the image is captured is a backlit scene in which light hits a subject from behind; When it is determined that the scene is a backlit scene by the first determination, the backlight correction is performed by performing exposure correction in a direction in which the overall brightness of the image becomes brighter; performing a second determination of whether the image has any saturated regions; When the first determination determines that the scene is a backlit scene and the second determination determines that the saturated region exists, the first threshold is set to a value lower than when the saturated region is determined not to exist. Backlight correction device. [Additional note 2] The processor, The backlight correction device according to claim 1, wherein the peripheral area is a target of the second determination. [Additional note 3] The processor, The backlight correction device according to claim 2, wherein the second determination is made based on a result of comparing a difference in brightness between the peripheral region among a plurality of divided regions obtained by dividing the image and the entire image with a second threshold value set in advance. [Additional note 4] The processor, The backlight correction device according to claim 3, wherein if there are a specified number or more of the surrounding areas whose brightness difference from the entire image is equal to or greater than the second threshold, it is determined that the saturated area exists. [Additional note 5] 5. The backlight correction device according to claim 1, wherein the reference area is one of a central area of ​​the image, an area in which the subject is detected, and an area designated by a user. [Additional note 6] Item 6. An imaging device comprising the backlight correction device according to any one of items 1 to 5. [Additional note 7] 7. The imaging device according to claim 6, which is a surveillance camera.

[0082] The technology of the present disclosure can be appropriately combined with the above-mentioned various embodiments and / or various modified examples. In addition, it is not limited to the above-mentioned embodiments, and various configurations can be adopted without departing from the gist of the present disclosure. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores a program, in addition to a program.

[0083] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0084] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0085] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0086] 10. Surveillance Camera System 11. Surveillance Cameras 12 Management device 13 Monitor 14 Input Devices 15 Network 20 Imaging optical system 21 Image sensor 22 Objective Lens 23 Focus Lens 24 Zoom Lens 25 Aperture 26 Focus lens drive mechanism 27 Zoom lens drive mechanism 28 Aperture opening adjustment mechanism 29 Control Unit 30 Image sensor driver 31 Shutter 32 Shutter drive mechanism 33 Image Input Controller 34 Image Memory 35 Image Processing Section 36, 48 Bus Lines 37 VRAM 38 Image distribution control section 39 Instruction Reception Department 45 Storage 46 CPU 47 Memory 50 Operating Program 51 First Threshold 52 Second Threshold 53 Program Diagram 55 Image acquisition unit 56 Exposure value calculation section 57 1st Judgment Section 58 Second judgment part 59 Exposure control section 65 images 66 Exposure Value Information 67 First judgment result 68 Second judgment result 70 Split area 75 Face Area 78 User-specified area (specified area) Typical area exposure value for EVX reference area EVY Representative area exposure value for the surrounding area OA optical axis ST10, ST11, ST12, ST13, ST20, ST21, ST22, ST23, ST24, ST25, ST26, ST27, ST30, ST31, ST32, ST33, ST34, ST35, ST36, ST100, ST110, ST120, ST130, ST140, ST150, ST201, ST202, ST203, ST204 Steps TH1 First threshold (standard value) TH1L First threshold (value lower than standard value) U User ΔEV The difference between the representative value of the area exposure value of the reference area and the representative value of the area exposure value of the surrounding area

Claims

1. A processor is provided. The processor, performing a first determination that, when a difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is equal to or greater than a first threshold value set in advance, the scene in which the image is captured is a backlit scene in which light hits a subject from behind; When it is determined that the scene is a backlit scene by the first determination, the backlight correction is performed by performing exposure correction in a direction in which the overall brightness of the image becomes brighter; performing a second determination of whether the image has any saturated regions; When the first determination determines that the scene is a backlit scene and the second determination determines that the saturated region exists, the first threshold is set to a value lower than when the saturated region is determined not to exist. Backlight correction device.

2. The processor, The backlight correction device according to claim 1 , wherein the peripheral area is a target of the second determination.

3. The processor, The backlight correction device according to claim 2 , wherein the second judgment is made based on a comparison result between a difference in brightness between the peripheral area of ​​a plurality of divided areas into which the image is divided and the entire image, and a predetermined second threshold value.

4. The processor, 4. The backlight correction device according to claim 3, wherein the presence of the saturated region is determined when a prescribed number of the surrounding regions have a difference in brightness from the entire image equal to or greater than the second threshold value.

5. The backlight correction device according to claim 1 , wherein the reference area is any one of a central area of ​​the image, an area in which the subject is detected, and an area designated by a user.

6. An imaging device comprising the backlight correction device according to claim 1 .

7. 7. The imaging device according to claim 6, which is a surveillance camera.

8. performing a first determination in which, when a difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is equal to or greater than a preset threshold, the photographing scene of the image is determined to be a backlit scene in which light hits a subject from behind; when it is determined that the scene is a backlit scene by the first determination, performing the backlight correction by correcting the exposure in a direction in which the overall brightness of the image becomes brighter; performing a second determination of whether the image has any saturated regions; and when the first determination determines that the scene is a backlit scene and the second determination determines that the saturated region exists, the threshold is set to a lower value than when the saturated region is determined not to exist; A method for operating a backlight correction device comprising:

9. performing a first determination in which, when a difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is equal to or greater than a preset threshold, the photographing scene of the image is determined to be a backlit scene in which light hits a subject from behind; when it is determined that the scene is a backlit scene by the first determination, performing the backlight correction by correcting the exposure in a direction in which the overall brightness of the image becomes brighter; performing a second determination of whether the image has any saturated regions; and when the first determination determines that the scene is a backlit scene and the second determination determines that the saturated region exists, the threshold is set to a lower value than when the saturated region is determined not to exist; An operating program for a backlight correction device that causes a computer to execute a process including the steps of:

Images