Backlight correction device, operation method of backlight correction device, storage medium, and imaging device

Abstract

The invention provides a backlight correction device capable of performing appropriate backlight correction, an operation method of the backlight correction device, a storage medium, and an imaging device. When a difference between a representative value of a region exposure value of a reference region, which is a reference for backlight correction in an image, and a representative value of a region exposure value of a peripheral region of the reference region is equal to or greater than a preset first threshold value, a first determination unit performs a first determination for determining that a photographic scene of the image is a backlight scene. When it is determined by the first determination that the scene is a backlight scene, the exposure control unit performs backlight correction by performing exposure correction in a direction in which the brightness of the entire image is increased. The second determination unit performs a second determination to determine whether or not there is a saturated region in the image. When it is determined by the first determination that the scene is a backlight scene and it is determined by the second determination that the saturation region exists, the first determination unit sets the first threshold value to a value lower than that in a case where it is determined that the saturation region does not exist.

Description

Technical Field

[0001] The technology of the present invention relates to a backlight correction device, an operation method of the backlight correction device, an operation program of the backlight correction device and a camera device. Background Art

[0002] Patent document 1 describes an imaging device, which includes: an imaging unit that captures a subject and generates image data; a first detection unit that detects a facial region corresponding to the face of the subject in the image data; a second detection unit that detects a subject region corresponding to at least a portion of the subject in the image data; and a control unit that controls exposure by photometry related to the image data. When the first detection unit detects a facial region, the control unit corrects the exposure based on the photometry of the detected facial region. And, when the first detection unit does not detect a facial region thereafter and the second detection unit detects a subject region corresponding to the subject of the detected facial region, the control unit continues to correct the exposure corresponding to the photometry of the detected facial region.

[0003] Patent Document 1: Japanese Patent No. 6817590 Summary of the invention

[0004] One embodiment of the technology according to the present invention provides a backlight correction device capable of performing appropriate backlight correction, an operating method of the backlight correction device, an operating program of the backlight correction device, and an imaging device.

[0005] The backlight correction device of the present invention includes a processor, which performs the following processing: when the difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is greater than a pre-set first threshold value, a first determination is made to determine that the photographic scene of the image is a backlight scene in which light is irradiated from behind a subject; when it is determined by the first determination that it is a backlight scene, backlight correction is performed by performing exposure correction in a direction that increases the brightness of the entire image; a second determination is made to determine whether there is a saturated area in the image; and when it is determined by the first determination that it is a backlight scene and by the second determination that there is a saturated area, the first threshold is set to a value lower than a value in the case where it is determined that there is no saturated area.

[0006] Preferably, the processor takes the surrounding area as an object of the second determination.

[0007] Preferably, the processor performs the second determination based on a comparison result between a brightness difference between a peripheral region and the entire image among a plurality of divided regions obtained by dividing the image and a preset second threshold value.

[0008] Preferably, the processor determines that there is a saturated area when there are a predetermined number or more of peripheral areas whose brightness difference with the entire image is equal to or greater than a second threshold.

[0009] Preferably, the reference area is any one of a central area of ​​the image, an area where the subject is detected, and an area designated by a user.

[0010] The imaging device of the present invention includes the above-mentioned backlight correction device.

[0011] Preferably, the camera device of the present invention is a surveillance camera.

[0012] The operating method of the backlight correction device of the present invention includes the following steps: when the difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is greater than a preset threshold value, a first determination is made to determine that the photographic scene of the image is a backlight scene in which light is irradiated from behind a subject; when it is determined by the first determination that it is a backlight scene, backlight correction is performed by performing exposure correction in a direction that increases the brightness of the entire image; a second determination is made to determine whether there is a saturated area in the image; and when it is determined by the first determination that it is a backlight scene and by the second determination that there is a saturated area, the threshold is set to a value lower than a value in the case where it is determined that there is no saturated area.

[0013] The operating procedure of the backlight correction device of the present invention causes a computer to execute a process including the following steps: when the difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is greater than a preset threshold value, a first determination is made to determine that the photographic scene of the image is a backlight scene in which light is irradiated from behind a subject; when it is determined by the first determination that it is a backlight scene, backlight correction is performed by performing exposure correction in a direction that increases the brightness of the entire image; a second determination is made to determine whether there is a saturated area in the image; and when it is determined by the first determination that it is a backlight scene and by the second determination that there is a saturated area, the threshold is set to a value lower than a value in the case where it is determined that there is no saturated area. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a diagram showing a surveillance camera system.

[0015] Figure 2 It is a block diagram of a surveillance camera.

[0016] Figure 3 This is a block diagram of the control unit.

[0017] Figure 4 This is a block diagram showing a processing unit of a CPU.

[0018] Figure 5 This is a flowchart showing the processing procedure of the exposure value calculation unit.

[0019] Figure 6 is a diagram showing segmented regions.

[0020] Figure 7 It is a graph showing exposure value information.

[0021] Figure 8 This is a flowchart showing the processing procedure of the first determination unit.

[0022] Fig. 9 This is a flowchart showing the procedure for setting the first threshold value.

[0023] Fig.10 It is a diagram showing a reference area and a peripheral area.

[0024] Fig.11 This is a flowchart showing the processing procedure of the second determination unit.

[0025] Fig.12 It is a diagram showing the processing of the exposure control unit.

[0026] Fig.13 This is a flowchart showing the processing procedure of the CPU of the surveillance camera.

[0027] Fig.14 yes Figure 6 1 is a schematic diagram showing the brightness of a region along the line XX of an image shown, and is a diagram showing a comparative example in which the first threshold value is fixed to a standard value.

[0028] Fig.15 yes Figure 6 1 is a schematic diagram showing the brightness of the area along the XX line of the image shown, and is a diagram showing the present example in which the first threshold setting is changed to a standard value and a value lower than the standard value.

[0029] Fig.16 The figure shows an example in which the face area is used as the reference area.

[0030] Fig.17 This is a diagram showing an example in which a region designated by a user is used as a reference region. DETAILED DESCRIPTION

[0031] [First embodiment]

[0032] As an example, Figure 1 As shown in FIG. 1 , the monitoring camera system 10 includes a monitoring camera 11 and a management device 12. The monitoring camera 11 is arranged on a street or in a building to capture the situation on the street or in the building. The management device 12 is installed in a control room where a user U who is the manager of the monitoring camera 11 resides. The management device 12 has a display 65 (refer to FIG. 1 ) for displaying the image 65 from the monitoring camera 11. Figure 4 and Figure 6) and an input device 14 such as a keyboard and a mouse for inputting various operation instructions to the monitoring camera 11. The monitoring camera 11 and the management device 12 are connected via a network 15. The network 15 is, for example, a WAN (Wide Area Network) such as the Internet or a public communication network. Figure 1 In the example, only one monitoring camera 11 is connected to the management device 12 , but a plurality of monitoring cameras 11 may be connected to the management device 12 .

[0033] As an example, Figure 2 As shown, the surveillance camera 11 includes an imaging optical system 20 and an imaging element 21. The imaging optical system 20 includes a plurality of lenses for imaging the subject light on the imaging element 21. Specifically, the imaging optical system 20 includes an objective lens 22, a focus lens 23, and a zoom lens 24. These lenses 22 to 24 are arranged in order from the object side (the subject side) toward the imaging side (the imaging element 21 side). Figure 1 Although simplified in FIG. 1 , each lens 22 to 24 is actually a lens group composed of a plurality of lenses. The imaging optical system 20 further includes an aperture 25. The aperture 25 is disposed on the imaging optical system 20 on the most imaging side. The focusing lens 23 is provided with a focusing lens driving mechanism 26, the zoom lens 24 is provided with a zoom lens driving mechanism 27, and the aperture 25 is provided with an aperture opening adjustment mechanism 28.

[0034] The focus lens driving mechanism 26 includes a focus cam ring that holds the focus lens 23 and has a cam groove formed on its outer circumference, a focus motor that moves the focus cam ring along the optical axis OA by rotating the focus cam around the optical axis OA, a driver for the focus motor, etc. The zoom lens driving mechanism 27 also includes a zoom cam ring that holds the zoom lens 24 and has a cam groove formed on its outer circumference, a zoom motor that moves the zoom cam ring along the optical axis OA by rotating the zoom cam around the optical axis OA, a driver for the zoom motor, etc.

[0035] The aperture 25 is a so-called iris aperture composed of a combination of a plurality of aperture blades. The aperture 25 uses a cam mechanism to simultaneously move the plurality of aperture blades to open and close the central opening formed by the inner edges of the plurality of aperture blades, that is, to change the opening degree of the opening (hereinafter referred to as the aperture degree of the aperture 25), thereby adjusting the amount of light passing therethrough. The aperture opening degree adjustment mechanism 28 includes an aperture motor for opening and closing the aperture blades and a driver for the aperture motor.

[0036] The focusing motor, the zooming motor, and the diaphragm motor are, for example, stepping motors. In this case, the position of the focus lens 23 on the optical axis OA, the position of the zoom lens 24, and the aperture 25 can be derived from the driving amounts of the focusing motor, the zooming motor, and the diaphragm motor. Alternatively, a position sensor may be provided to detect the position of the focus lens 23 and the position of the zoom lens 24 instead of the driving amounts of the focusing motor and the zoom motor.

[0037] The electric components such as the motors (focusing motor, zooming motor, and iris motor) or drivers of the drive mechanisms 26 to 28 are connected to the control unit 29. The electric components of the drive mechanisms 26 to 28 are driven under the control of the control unit 29. More specifically, the control unit 29 issues a drive signal corresponding to various operation instructions inputted via the input device 14 of the management device 12, and drives the electric components of the drive mechanisms 26 to 28. For example, when an operation instruction to change the angle of view to the telephoto side is inputted, the control unit 29 issues a drive signal to the driver of the zoom motor of the zoom lens drive mechanism 27, and moves the zoom lens 24 to the telephoto side.

[0038] The focus motor, zoom motor, and diaphragm motor output drive amounts to the control unit 29. The control unit 29 derives the position of the focus lens 23, the position of the zoom lens 24, and the aperture of the diaphragm 25 on the optical axis OA based on the drive amounts.

[0039] The imaging element 21 is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and has an imaging surface for capturing subject light. The imaging surface is formed by a plurality of pixels arranged in a two-dimensional shape. The pixels accumulate signal charges corresponding to the subject light, and output image signals (voltage signals) corresponding to the signal charges. The imaging element 21 is configured so that the center of the imaging surface coincides with the optical axis OA and the imaging surface is orthogonal to the optical axis OA. In addition, the "coincidence" and "orthogonality" mentioned here refer not only to complete coincidence and orthogonality, but also to coincidence and orthogonality including the meaning of errors generally allowed in the technical field to which the technology of the present invention belongs.

[0040] The imaging element 21 is connected to an imaging element driver 30. The imaging element driver 30 is connected to the control unit 29. The imaging element driver 30 supplies a vertical scanning signal and a horizontal scanning signal to the imaging element 21 under the control of the control unit 29 to control the imaging timing of the imaging element 21 capturing the subject light. In addition, the imaging element driver 30 sets a gain corresponding to the ISO (International Organization for Standardization) sensitivity given to the image signal output from the pixel.

[0041] A shutter 31 is provided between the imaging optical system 20 and the imaging element 21. The shutter 31 is, for example, a focal plane shutter having a front curtain and a rear curtain. A shutter drive mechanism 32 is connected to the shutter 31. The shutter drive mechanism 32 includes an electromagnet that holds the front curtain and the rear curtain and releases the holding to allow the front curtain and the rear curtain to move, and a driver thereof. The shutter drive mechanism 32 is driven under the control of the control unit 29 to open and close the shutter 31. In addition, the shutter is not limited to the mechanical shutter 31 exemplified, and may be an electronic shutter. In particular, electronic shutters are often used in surveillance cameras.

[0042] The control unit 29 is connected to each unit such as the image input controller 33, the image memory 34, and the image processing unit 35 through the bus 36. The bus 36 is also connected to a VRAM (Video Random Access Memory) 37, an image distribution control unit 38, and an instruction receiving unit 39. In addition, although not shown in the figure, the bus 36 is also connected to a power supply unit that supplies power to each unit of the monitoring camera 11, an external communication I / F (Interface) or a wireless communication I / F that communicates with an external device via a connection terminal such as a USB (Universal Serial Bus) terminal, and the like.

[0043] An image 65 based on an image signal obtained by capturing subject light is input from the imaging element 21 to the image input controller 33. The image input controller 33 outputs the image 65 to the image memory 34. The image memory 34 is, for example, an SDRAM (Synchronous Dynamic Random Access Memory) and temporarily stores the image 65.

[0044] The image processing unit 35 reads out the unprocessed image 65 from the image memory 34. The image processing unit 35 performs various image processing on the image 65. The various image processing includes, for example, offset correction processing, sensitivity correction processing, pixel interpolation processing, white balance correction processing, gamma correction processing, demosaicing processing, brightness signal and color difference signal generation processing, outline emphasis processing, color correction processing, etc. The image processing unit 35 writes the image 65 after various image processing back to the image memory 34.

[0045] The image 65 after various image processing for display on the monitor 13 of the management device 12 is input from the image memory 34 to the VRAM 37. The VRAM 37 has an area for storing two consecutive frames of the image 65. The image 65 stored in the VRAM 37 is sequentially rewritten to a new image 65. The VRAM 37 sequentially outputs the newer image 65 of the two consecutive frames of the image 65 to the image distribution control unit 38.

[0046] The image distribution control unit 38 controls distribution of the image 65 from the VRAM 37 to the management device 12. Thus, the user U can visually recognize the image 65 through the monitor 13 of the management device 12. The display frame rate of the image 65 is, for example, 5 fps (frames per second) to 30 fps.

[0047] The instruction accepting unit 39 accepts various operation instructions inputted by the user U via the input device 14 of the management apparatus 12 . The instruction accepting unit 39 outputs the accepted various operation instructions to the control unit 29 via the bus 36 .

[0048] In addition to the above-mentioned operation instructions for changing the viewing angle, the various operation instructions include operation instructions for manually setting the aperture 25 opening, ISO sensitivity, and shutter speed of the shutter 31. In addition, there are operation instructions for shooting still images or moving images of the image 65, or operation instructions for changing the orientation of the monitoring camera 11 by operating a pan / tilt mechanism (not shown).

[0049] As an example, Figure 3 As shown, the control unit 29 includes a storage device 45, a CPU (Central Processing Unit) 46, and a memory 47. The storage device 45, CPU 46, and memory 47 are connected to each other via a bus 48. The control unit 29 is an example of a "backlight correction device" and a "computer" involved in the technology of the present invention.

[0050] The storage device 45 is a nonvolatile storage device such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). The storage device 45 stores various programs and various data accompanying the various programs. In addition, FeRAM (Ferroelectric Random Access Memory) or MRAM (Magnetoresistive Random Access Memory) can be used as the storage device 45 instead of EEPROM.

[0051] The memory 47 is a working memory for the CPU 46 to perform processing. The CPU 46 loads the program stored in the storage device 45 into the memory 47 and performs processing according to the program. Thus, the CPU 46 uniformly controls each part of the monitoring camera 11. The CPU 46 is an example of a "processor" involved in the technology of the present invention. In addition, the memory 47 can also be built into the CPU 46.

[0052] As an example, Figure 4 As shown, the storage device 45 stores an operation program 50. The operation program 50 is a program for the CPU 46 to perform backlight correction and the like. That is, the operation program 50 is an example of the "operation program of the backlight correction device" involved in the technology of the present invention. In addition to the operation program 50, the storage device 45 also stores a first threshold value 51, a second threshold value 52, and a program diagram 53. The first threshold value 51 is an example of the "threshold value" involved in the technology of the present invention.

[0053] When the operation program 50 is activated, the CPU 46 cooperates with the memory 47 and the like to function as the image acquisition unit 55 , the exposure value calculation unit 56 , the first determination unit 57 , the second determination unit 58 , and the exposure control unit 59 .

[0054] The image acquisition section 55 acquires the image 65 by reading out the image 65 subjected to the image processing from the image memory 34. The image acquisition section 55 outputs the image 65 to the exposure value calculation section 56.

[0055] The exposure value calculation unit 56 calculates an exposure value corresponding to the brightness of the image 65. The exposure value calculation unit 56 generates exposure value information 66 summarizing the calculation result of the exposure value. The exposure value calculation unit 56 outputs the exposure value information 66 to the first determination unit 57, the second determination unit 58, and the exposure control unit 59.

[0056] The first threshold value 51 is input to the first determination unit 57. The first determination unit 57 performs a first determination using the first threshold value 51 and the exposure value information 66. The first determination is a determination of whether the photographic scene of the image 65 is a backlit scene. The backlit scene is a photographic scene in which light is irradiated from behind the subject, causing the periphery of the subject to become brighter and the subject to become darker. The first determination unit 57 outputs a first determination result 67 as a result of the first determination to the exposure control unit 59.

[0057] The second threshold value 52 is input to the second determination unit 58. The second determination unit 58 performs a second determination using the second threshold value 52 and the exposure value information 66. The second determination is a determination of whether there is a saturated region (hereinafter referred to as a saturated region) in the image 65. The saturated region is a so-called whitened region where the image signal of each RGB color reaches the upper limit value and appears white. The second determination unit 58 outputs a second determination result 68 as the result of the second determination to the first determination unit 57.

[0058] The sequence diagram 53 is input to the exposure control unit 59. The exposure control unit 59 performs exposure control for obtaining the image 65 with standard exposure according to the sequence diagram 53, the exposure value information 66, and the first determination result 67. More specifically, the exposure control unit 59 controls the exposure of the image 65 according to the sequence diagram 53 and the overall exposure value of the exposure value information 66 (see Figure 7) to derive the aperture 25 opening, ISO sensitivity, and shutter speed of the shutter 31 for obtaining the standard exposure image 65. The exposure control unit 59 outputs a drive signal for setting the aperture 25 opening, ISO sensitivity, and shutter speed to be derived to the aperture opening adjustment mechanism 28, the imaging element driver 30, and the shutter drive mechanism 32. In addition, the standard exposure image 65 refers to an image with the same brightness as when actually observed with the naked eye.

[0059] As an example, the exposure value calculation unit 56 follows Figure 5 The exposure value is calculated in the order shown. First, the exposure value calculation unit 56 calculates the exposure value as an example. Figure 6 The representative value of brightness of each of the plurality of segmented regions 70 obtained by segmenting the image 65 is obtained (step ST10). The representative value of brightness is, for example, an average value or a median value of a brightness signal generated by an image signal output by pixels in the segmented region 70. Figure 6 , an image 65 is shown showing a family walking between two trees on a lawn in a park with sunlight behind them. Figure 6 In FIG. 1 , the case where the image 65 is divided into 16×16=256 equal parts is illustrated, thereby preparing 256 divided regions 70. The number of divided regions 70 is not limited to the illustrated 256. It may be 8×8=64 or 24×24=576.

[0060] The exposure value calculation unit 56 calculates a weighted average value of the representative value of the brightness of each segmented area 70 (step ST11). The weighted average value is calculated by adding a value obtained by multiplying a weight set in advance for each segmented area 70 by the representative value of the brightness of each segmented area 70, and dividing the value obtained by the number of segmented areas 70. The weight is set, for example, so that the value decreases from the center of the image 65 toward the periphery.

[0061] The exposure value calculation unit 56 calculates the exposure value of each divided area 70 (hereinafter referred to as the regional exposure value) based on the representative value of the brightness of each divided area 70 (step ST12). In addition, the exposure value calculation unit 56 calculates the exposure value of the entire image 65 (hereinafter referred to as the overall exposure value) based on the weighted average value (step ST12). As an example, Figure 7 As shown in FIG. 1 , the exposure value calculation unit 56 generates the regional exposure value and the overall exposure value as exposure value information 66 and outputs the information (step ST13). Figure 7 In the example, the segmented areas are marked with numbers such as "0101", "0102", etc. Figure 6 , which is a list of numbers assigned to the rows and columns of the divided regions 70. For example, "0403" indicates the divided region 70 at the 4th row and the 3rd column, and "0709" indicates the divided region 70 at the 7th row and the 9th column.

[0062] As an example, the first determination unit 57 follows Figure 8 and Fig. 9 The first determination is performed in the order shown. First, the first determination unit 57 sets the first threshold value 51 (step ST20). Fig. 9 As shown, when the exposure control unit 59 has implemented the exposure correction corresponding to the backlit scene in the previous frame and the second determination result 68 in the previous frame is that there is a saturated area in the image 65 (both "Yes" in step ST201 and step ST202), the first determination unit 57 sets the first threshold 51 to a value lower than the standard value (step ST203). On the other hand, when the exposure control unit 59 has not implemented the exposure correction corresponding to the backlit scene in the previous frame ("No" in step ST201), the first determination unit 57 sets the first threshold 51 to the standard value (step ST204). When the exposure control unit 59 has implemented the exposure correction corresponding to the backlit scene in the previous frame but the second determination result 68 in the previous frame is that there is no saturated area in the image 65 ("Yes" in step ST201, "No" in step ST202), the first determination unit 57 also sets the first threshold 51 to the standard value (step ST204). In addition, the standard value of the first threshold value 51 is, for example, 2EV, and the value lower than the standard value is, for example, 0.5EV.

[0063] exist Figure 8 In step ST21, after setting the first threshold value 51, the first determination unit 57 calculates representative values ​​of the regional exposure values ​​of the reference area and the peripheral area of ​​the reference area, which are the references for backlight correction, based on the regional exposure values ​​of the exposure value information 66 (step ST21). Fig.10 As shown by the solid line and hatching, the reference area is 4×4=16 segmented areas 70 in the center of the image 65, that is, the central area of ​​the image 65. In terms of numbers, the reference areas are "0707", "0708", "0709", "0710", "0807", "0808", "0809", "0810", "0907", "0908", "0909", "0910", "1007", "1008", "1009", and "1010" segmented areas 70. On the other hand, the peripheral area is 240 segmented areas 70 other than the above 16 segmented areas 70. The representative value of the regional exposure value of the reference area is the average value, the central value, etc. of the regional exposure values ​​of the above 16 segmented areas 70. Similarly, the representative value of the regional exposure value of the peripheral area is the average value, the central value, etc. of the regional exposure values ​​of the 240 segmented areas 70 other than the above 16 segmented areas 70.

[0064] The first determination unit 57 calculates the difference between the representative values ​​of the regional exposure values ​​of the reference area and the peripheral area by subtracting the representative value of the regional exposure value of the reference area from the representative value of the regional exposure value of the peripheral area (step ST22). The difference between the representative values ​​of the regional exposure values ​​of the reference area and the peripheral area is greater when the photographic scene of the image 65 is a backlit scene than when it is not a backlit scene. The difference between the representative values ​​of the regional exposure values ​​of the reference area and the peripheral area is an example of the "difference in brightness between the reference area and the peripheral area" involved in the technology of the present invention. In addition, as the "difference in brightness between the reference area and the peripheral area", it can also be the difference in representative values ​​of the brightness signals of the reference area and the peripheral area.

[0065] The first determination unit 57 compares the difference between the representative values ​​of the area exposure values ​​of the reference area and the peripheral area with the first threshold value 51 set in step ST20 (step ST23). When the difference between the representative values ​​of the area exposure values ​​of the reference area and the peripheral area is greater than the first threshold value 51 ("Yes" in step ST24), the first determination unit 57 determines that the photographic scene of the image 65 is a backlit scene (step ST25). On the other hand, when the difference between the representative values ​​of the area exposure values ​​of the reference area and the peripheral area is less than the first threshold value 51 ("No" in step ST24), the first determination unit 57 determines that the photographic scene of the image 65 is not a backlit scene (step ST26). The first determination unit 57 outputs the first determination result 67 (step ST27).

[0066] As an example, the second determination unit 58 Fig.11 The second determination is performed in the order shown. First, the second determination unit 58 calculates the difference between the regional exposure value of the peripheral area and the overall exposure value for each peripheral area by subtracting the overall exposure value from the regional exposure value of each peripheral area (step ST30). As can be seen from the processing of step ST30, the object of the second determination is only the peripheral area, and the reference area is excluded from the object of the second determination. The difference between the regional exposure value and the overall exposure value of the peripheral area is an example of the "difference in brightness between the peripheral area and the image as a whole" involved in the technology of the present invention. In addition, as the "difference in brightness between the peripheral area and the image as a whole", it can also be the difference between the representative value of the brightness signal of the peripheral area and the representative value of the brightness signal of the image 65 as a whole.

[0067] The second determination unit 58 compares the difference between the regional exposure value and the overall exposure value of the peripheral area with the second threshold 52 (step ST31 ), and then counts the peripheral areas where the difference between the regional exposure value and the overall exposure value is greater than or equal to the second threshold 52 (step ST32 ).

[0068] When the number of peripheral regions where the difference between the regional exposure value and the overall exposure value is greater than or equal to the second threshold value 52 is greater than or equal to the prescribed number ("Yes" in step ST33), the second determination unit 58 determines that a saturated region exists in the image 65 (step ST34). On the other hand, when the number of peripheral regions where the difference between the regional exposure value and the overall exposure value is greater than or equal to the second threshold value 52 is less than or equal to the prescribed number ("No" in step ST33), the second determination unit 58 determines that there is no saturated region in the image 65 (step ST35). The second determination unit 58 outputs the second determination result 68 (step ST36). In addition, the second threshold value 52 is, for example, 2.5EV, and the prescribed number is, for example, 10.

[0069] As an example, Fig.12 As shown in (A), when the first determination result 67 indicates that the photographing scene of the image 65 is a backlit scene, the exposure control unit 59 performs exposure correction. Fig.12 As shown in (B), when the first determination result 67 indicates that the photographing scene of the image 65 is not a backlit scene, the exposure control unit 59 does not perform exposure correction.

[0070] The exposure correction performed when the photographic scene of the image 65 is a backlit scene is a process of increasing the overall brightness of the image 65, which is a process of correcting the backlight. More specifically, the exposure correction is a process of correcting the aperture 25 opening, ISO sensitivity, and shutter speed of the shutter 31 derived from the overall exposure value of the program diagram 53 and the exposure value information 66 in the direction of increasing the overall brightness of the image 65. The correction in the direction of increasing the overall brightness of the image 65 is a correction in the direction of increasing the aperture 25 opening, in the direction of setting a high ISO sensitivity, and in the direction of slowing down the shutter speed. In addition, the degree of increase in the overall brightness of the image 65 by the exposure correction is, for example, about +2EV in terms of exposure value conversion.

[0071] Next, as an example, refer to Fig.13 The flowchart shown in FIG. 1 illustrates the function of the above structure. Figure 4 As shown, the CPU 46 of the control unit 29 activates the operation program 50 to thereby function as the image acquisition unit 55 , the exposure value calculation unit 56 , the first determination unit 57 , the second determination unit 58 , and the exposure control unit 59 .

[0072] In the monitoring camera 11, the subject light formed by the imaging optical system 20 is captured by the imaging element 21, and the image 65 is output from the imaging element 21. The image processing unit 35 performs various image processing on the image 65. The image 65 subjected to various image processing is distributed to the management device 12 under the control of the image distribution control unit 38, and is displayed on the monitor 13 of the management device 12. Then, the image 65 is read out by the image acquisition unit 55 to acquire the image 65 (step ST100). The image 65 is output from the image acquisition unit 55 to the exposure value calculation unit 56.

[0073] In the exposure value calculation unit 56, according to Figure 5 The regional exposure value as the exposure value of each divided area and the overall exposure value as the exposure value of the entire image 65 are calculated in the order shown (step ST110). The exposure value calculation unit 56 outputs the exposure value information 66 including the regional exposure value and the overall exposure value to the first determination unit 57, the second determination unit 58 and the exposure control unit 59.

[0074] In the first determination unit 57, according to Figure 8 and Fig. 9 The first determination result 67 is output from the first determination unit 57 to the exposure control unit 59 , as to whether the shooting scene of the image 65 is a backlit scene or not (step ST120 ).

[0075] In the first determination, the first threshold value 51 is set to a standard value or a value lower than the standard value based on the second determination result 68 of the second determination unit 58 as to whether or not a saturated region exists in the image 65. Specifically, when the exposure control unit 59 has not performed exposure correction corresponding to a backlit scene in the previous frame, and when the exposure control unit 59 has performed exposure correction corresponding to a backlit scene in the previous frame but the second determination result 68 in the previous frame indicates that there is no saturated region in the image 65, the first threshold value 51 is set to the standard value. On the other hand, when the exposure control unit 59 has performed exposure correction corresponding to a backlit scene in the previous frame and the second determination result 68 in the previous frame indicates that there is a saturated region in the image 65, the first threshold value 51 is set to a value lower than the standard value. In addition, when the second determination unit 58 determines that there is a saturated area in the image 65 and the first threshold 51 is set to a value lower than the standard value in the first determination unit 57, the photographic scene changes, and when the second determination unit 58 determines that there is no saturated area in the image 65, the first threshold 51 is reset to the standard value in the first determination unit 57.

[0076] In the second determination unit 58, according to Fig.11The second determination as to whether or not a saturated region exists in the image 65 is performed in the order shown (step ST130 ). The second determination result 68 is output from the second determination unit 58 to the first determination unit 57 .

[0077] The exposure control unit 59 performs exposure control according to the program chart 53 and the overall exposure value (step ST140). Fig.12 As shown in (A), when the first determination result 67 indicates that the photographic scene of the image 65 is a backlit scene, exposure correction is performed in the direction of increasing the overall brightness of the image 65. If the power of the monitoring camera 11 is not turned off ("No" in step ST150), the processing of these steps ST100 to ST140 is repeatedly performed.

[0078] As described above, when the difference between the representative value of the area exposure value of the reference area serving as the reference for backlight correction in the image 65 and the representative value of the area exposure value of the peripheral area of ​​the reference area is greater than the preset first threshold value 51, the first determination unit 57 of the CPU 46 of the control unit 29 makes the first determination that the photographic scene of the image 65 is a backlight scene. When it is determined to be a backlight scene by the first determination, the exposure control unit 59 performs backlight correction by performing exposure correction in the direction of increasing the brightness of the entire image 65. The second determination unit 58 makes the second determination to determine whether there is a saturated area in the image 65. When it is determined to be a backlight scene by the first determination and it is determined to be a saturated area by the second determination, the first determination unit 57 sets the first threshold value 51 to a value lower than the case where it is determined that there is no saturated area.

[0079] Fig.14 and Fig.15 Shown in Figure 6 The image 65 shown is a schematic diagram showing the brightness of the area along the line XX. Fig.14 , the case where the first threshold value 51 is fixed to a standard value (comparative example) is shown. Fig.15 2 shows a case where the first threshold value setting is changed to a standard value and a value lower than the standard value (this example).

[0080] The brightness of the image 65 will be lower in the area of ​​the two trees and the family, and will reach the upper limit and saturate in other areas. The area of ​​the family is equivalent to the central area, that is, the reference area. And the difference ΔEV between the representative value EVX of the regional exposure value of the reference area and the representative value EVY of the regional exposure value of the peripheral area is greater than the first threshold value (standard value) TH1. Therefore, in Fig.14 and Fig.15In the case of the situation shown in FIG. 1 , the first determination unit 57 determines that the photographic scene of the image 65 is a backlit scene. The exposure control unit 59 performs exposure correction. As a result, the brightness of the entire image 65 is increased. However, the brightness of the saturated area cannot be further increased, so only the brightness of the area outside the saturated area including the reference area is increased. In this way, the value of the representative value EVX of the regional exposure value of the reference area increases, and accordingly, the difference ΔEV between the representative value EVX of the regional exposure value of the reference area and the representative value EVY of the regional exposure value of the peripheral area decreases.

[0081] exist Fig.14 In the comparative example shown, the first threshold value 51 is fixed to the standard value TH1. Therefore, if the exposure correction is performed to increase the value of EVX, and ΔEV is reduced accordingly, ΔEV will be less than TH1. In this way, although the shooting scene of the image 65 is actually a backlit scene, the first determination unit 57 will mistakenly determine that it is not a backlit scene. In this case, the exposure control unit 59 will cancel the exposure correction. Thereafter, the exposure correction will be repeatedly performed and canceled until the shooting scene changes, and the overall brightness of the image 65 will change dazzlingly.

[0082] On the other hand, Fig.15 In the example shown, the first threshold value 51 is set to a value TH1L lower than the standard value TH1. Therefore, even if the exposure correction is performed to increase the value of EVX, and ΔEV is reduced accordingly, the first determination unit 57 can accurately determine that the photographing scene of the image 65 is a backlit scene. Therefore, it is possible to avoid a situation where the brightness of the entire image 65 changes drastically due to repeated implementation and cancellation of exposure correction as in the comparative example. Therefore, appropriate backlight correction can be performed.

[0083] like Fig.11 As shown in FIG. 1 , the second determination unit 58 uses the surrounding area as the object of the second determination. Therefore, it is not necessary to perform unnecessary processing.

[0084] like Fig.11 As shown, the second determination unit 58 performs the second determination based on the comparison result between the difference between the regional exposure value of the peripheral area and the overall exposure value in the plurality of segmented areas 70 obtained by segmenting the image 65 and the preset second threshold 52. Therefore, the validity of the second determination result 68 can be improved.

[0085] And, if Fig.11 As shown, when there are a predetermined number or more of peripheral areas where the difference between the regional exposure value and the overall exposure value is greater than the second threshold value 52, the second determination result 68 determines that there is a saturated area. Therefore, the validity of the second determination result 68 can be further improved.

[0086] like Fig.10As shown, the reference area is the central area of ​​the image 65. Generally, the subject of interest is more likely to be reflected in the central area. Therefore, if the central area is used as the reference area, more appropriate backlight correction can be performed.

[0087] In view of the use of clearly displaying accidents, crimes, etc. to notify the user U of an emergency, the monitoring camera 11 is particularly required to have stable image quality of the image 65 projected on the monitor 13. Therefore, the present example using the monitoring camera 11 as the imaging device can greatly demonstrate the effect of being able to perform appropriate backlight correction.

[0088] The imaging device according to the technology of the present invention is not limited to the monitoring camera 11 exemplified, but may be a single-lens reflex digital camera, a compact digital camera, a smartphone, or a tablet terminal.

[0089] (Variant 1)

[0090] The reference area is not limited to the central area of ​​the illustrated image 65. Fig.16 As shown in FIG. 1 , a human face region 75 detected by a known face detection function may be used as a reference region. In this case, the region other than the face region 75 becomes a peripheral region. The face region 75 is an example of a “region where a subject is detected” according to the technology of the present invention. In addition, the region where a subject is detected may be a human body region, an animal face or body region, or a front or fuselage region of a vehicle such as a car, a train, or an airplane.

[0091] (Variant 2)

[0092] And, as an example, Fig.17 As shown, an area (hereinafter referred to as a designated area) 78 designated by a user U may be used as a reference area. In this case, an area other than the designated area 78 becomes a peripheral area. The designated area 78 is, for example, an area for automatic focusing based on an image signal of the designated area 78.

[0093] The second determination may be performed only when it is determined by the first determination that the scene is backlit.

[0094] The management device 12 may also be responsible for the function of the backlight correction device.

[0095] In the above-mentioned embodiment, for example, various processors shown below can be used as the hardware structure of the processing unit (Processing Unit) that performs various processes such as the image processing unit 35, the image distribution control unit 38, the instruction receiving unit 39, the image acquisition unit 55, the exposure value calculation unit 56, the first determination unit 57, the second determination unit 58, and the exposure control unit 59. The various processors include a CPU 46 that is a general-purpose processor that performs the functions of various processing units by executing software (operation program 50), and in addition, a programmable logic device (PLD) such as an FPGA (Field Programmable Gate Array) that is a processor whose circuit structure can be changed after manufacturing and / or an ASIC (Application Specific Integrated Circuit) that is a dedicated circuit such as a processor having a circuit structure specially designed to perform specific processing.

[0096] A processing unit may be composed of one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs and / or a combination of a CPU and an FPGA). Furthermore, multiple processing units may be composed of one processor.

[0097] As an example of a plurality of processing units composed of one processor, first, there is a method represented by computers such as clients and servers: a processor is composed of a combination of one or more CPUs and software, and the functions of multiple processing units are performed by the processor. Secondly, there is a method represented by a system on chip (System On Chip: SoC) etc.: a processor that realizes the functions of the entire system including multiple processing units by a single IC (Integrated Circuit) chip is used. In this way, various processing units are configured as a hardware structure using one or more of the above-mentioned various processors.

[0098] Furthermore, as the hardware configuration of these various processors, more specifically, a circuit (Circuitry) formed by combining circuit elements such as semiconductor elements can be used.

[0099] Based on the above description, the techniques described in the following supplementary notes can be understood.

[0100] [Supplementary Item 1]

[0101] A backlight correction device comprises a processor,

[0102] The processor performs the following processing:

[0103] When the difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is greater than a first threshold value set in advance, a first determination is made that the photographic scene of the image is a backlight scene in which light is irradiated from behind the subject;

[0104] When it is determined by the first determination that the scene is the backlit scene, performing the backlight correction by performing exposure correction in a direction of increasing the brightness of the entire image;

[0105] Performing a second determination of whether or not a saturated area exists in the image; and

[0106] When it is determined by the first determination that the scene is the backlit scene and when it is determined by the second determination that the saturated area exists, the first threshold is set to a value lower than when it is determined that the saturated area does not exist.

[0107] [Supplementary Item 2]

[0108] The backlight correction device according to Supplementary Item 1, wherein:

[0109] The processor uses the surrounding area as a target for the second determination.

[0110] [Supplementary item 3]

[0111] The backlight correction device according to Supplementary Item 2, wherein:

[0112] The processor performs the second determination based on a comparison result between a brightness difference between the peripheral area and the entire image among a plurality of divided areas obtained by dividing the image and a preset second threshold value.

[0113] [Supplementary Item 4]

[0114] The backlight correction device according to Supplementary Item 3, wherein:

[0115] The processor determines that the saturated region exists when there are a predetermined number or more of the surrounding regions whose brightness difference with the entire image is equal to or greater than the second threshold.

[0116] [Supplementary Note 5]

[0117] The backlight correction device according to any one of supplementary items 1 to 4, wherein the reference area is any one of a central area of ​​the image, an area where the subject is detected, and an area specified by a user.

[0118] [Supplementary Item 6]

[0119] A camera device comprising the backlight correction device according to any one of Supplementary Items 1 to 5.

[0120] [Supplementary Note 7]

[0121] The camera device according to supplementary item 6 is a surveillance camera.

[0122] The technology of the present invention may also appropriately combine the above-mentioned various embodiments and / or various variations. Furthermore, it is certainly not limited to the above-mentioned embodiments, and various structures may be adopted within the scope of the main purpose. Furthermore, the technology of the present invention relates to a program, and in addition, it also relates to a storage medium that does not temporarily store a program.

[0123] The recorded contents and illustrated contents shown above are detailed descriptions of the parts involved in the technology of the present invention, and are only an example of the technology of the present invention. For example, the description of the above-mentioned structure, function, action and effect is a description of an example of the structure, function, action and effect of the parts involved in the technology of the present invention. Therefore, of course, it is also possible to delete unnecessary parts, add new elements or replace the recorded contents and illustrated contents shown above without departing from the main purpose of the technology of the present invention. In addition, in order to avoid trouble and facilitate understanding of the parts involved in the technology of the present invention, the description of technical common sense that does not need to be specifically explained when implementing the technology of the present invention is omitted in the recorded contents and illustrated contents shown above.

[0124] In this specification, "A and / or B" has the same meaning as "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. In addition, in this specification, when three or more items are expressed in association with "and / or", the same concept as "A and / or B" also applies.

[0125] All documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent as if each document, patent application, or technical standard was specifically and individually described as being incorporated by reference.

[0126] Explanation of symbols

[0127] 10-monitoring camera system, 11-monitoring camera, 12-management device, 13-monitor, 14-input device, 15-network, 20-camera optical system, 21-imaging element, 22-objective lens, 23-focusing lens, 24-zoom lens, 25-aperture, 26-focusing lens driving mechanism, 27-zoom lens driving mechanism, 28-aperture opening adjustment mechanism, 29-control unit, 30-imaging element driver, 31-shutter, 32-shutter driving mechanism, 33-image input controller, 34-image memory, 35-image processing unit, 36, 48-bus, 37-VRAM, 38-image distribution control unit, 39-instruction receiving unit, 45-storage device, 46-CPU, 47-memory, 50-operation program, 51-first threshold value, 52-second threshold value, 53 -Program diagram, 55-image acquisition unit, 56-exposure value calculation unit, 57-first judgment unit, 58-second judgment unit, 59-exposure control unit, 65-image, 66-exposure value information, 67-first judgment result, 68-second judgment result, 70-segmented area, 75-face area, 78-user-specified area (specified area), EVX-representative value of the area exposure value of the reference area, EVY-representative value of the area exposure value of the peripheral area, OA-optical axis, ST10, ST11, ST12, ST13, ST20, ST21, ST22, ST23, ST24, ST25, ST26, ST27, ST30, ST31, ST32, ST33, ST34, ST35, ST36, ST100, ST110, ST120, ST 130, ST140, ST150, ST201, ST202, ST203, ST204-steps, TH1-first threshold value (standard value), TH1L-first threshold value (value lower than the standard value), U-user, ΔEV-difference between the representative value of the regional exposure value of the reference area and the representative value of the regional exposure value of the surrounding area.

Claims

1. A backlight correction device, comprising a processor, The processor performs the following processing: performing a first determination, wherein, when a difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is greater than a first threshold value set in advance, determining that the photographic scene of the image is a backlit scene in which light is irradiated from behind a subject; When it is determined by the first determination that the scene is backlit, performing the backlight correction by performing exposure correction in a direction of increasing the brightness of the entire image; Performing a second determination of whether there is a saturated area in the image; and When it is determined by the first determination that the scene is the backlit scene and when it is determined by the second determination that the saturated area exists, the first threshold is set to a value lower than when it is determined that the saturated area does not exist.

2. The backlight correction device according to claim 1, wherein: The processor uses the surrounding area as a target for the second determination.

3. The backlight correction device according to claim 2, wherein: The processor performs the second determination based on a comparison result between a brightness difference between the peripheral area and the entire image among a plurality of divided areas obtained by dividing the image and a preset second threshold value.

4. The backlight correction device according to claim 3, wherein: The processor determines that the saturated region exists when there are a predetermined number or more of the surrounding regions whose brightness difference with the entire image is equal to or greater than the second threshold.

5. The backlight correction device according to any one of claims 1 to 4, wherein: The reference area is any one of a central area of ​​the image, an area where the subject is detected, and an area designated by a user. 6 . An imaging device comprising the backlight correction device according to claim 1 . The camera device according to claim 6 , which is a surveillance camera.

8. A method for operating a backlight correction device, comprising the following steps: performing a first determination, wherein, when a difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is greater than a preset threshold, determining that the photographic scene of the image is a backlit scene in which light is irradiated from behind a subject; When it is determined by the first determination that the scene is backlit, performing the backlight correction by performing exposure correction in a direction of increasing the brightness of the entire image; Performing a second determination of whether there is a saturated area in the image; and When it is determined by the first determination that the scene is the backlit scene and when it is determined by the second determination that the saturated area exists, the threshold is set to a value lower than when it is determined that the saturated area does not exist.

9. A computer-readable storage medium storing an operating program of a backlight correction device, wherein the operating program causes a computer to execute a process comprising the following steps: performing a first determination, wherein, when a difference in brightness between a reference area serving as a reference for backlight correction in an image and a peripheral area of ​​the reference area is greater than a preset threshold, determining that the photographic scene of the image is a backlit scene in which light is irradiated from behind a subject; When it is determined by the first determination that the scene is backlit, performing the backlight correction by performing exposure correction in a direction of increasing the brightness of the entire image; Performing a second determination of whether there is a saturated area in the image; and When it is determined by the first determination that the scene is the backlit scene and when it is determined by the second determination that the saturated area exists, the threshold is set to a value lower than when it is determined that the saturated area does not exist.

Images