Imaging control device and control method thereof

Abstract

To make it possible to obtain a stable exposure for a subject.SOLUTION: An imaging control device for controlling imaging by an imaging apparatus includes detection means for detecting a feature part included in an image captured by the imaging apparatus, a determination means for determining an exposure area to be used for exposure calculation on the basis of a subject area surrounding a subject that includes the detected feature area. The determination means determines the exposure area by enlarging the subject area in a first direction.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to exposure control for imaging. [Background technology]

[0002] In recent years, a technology has been proposed that detects a specific area of ​​a subject from an image captured by an imaging device, and sets shooting settings such as exposure and image quality based on information related to the detected area. For example, Patent Document 1 discloses a technology that detects a predetermined target image portion from a subject image included in image data, and determines an automatic exposure (AE) target area that determines the brightness of the target image portion based on the detected target image portion. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4567618 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology of Patent Document 1, the AE target area is inside the target image part (such as a face), so the area of ​​the AE target area is narrow, making it difficult to maintain stable exposure depending on the scene. For example, in a scene with a lot of noise or a lot of movement, the exposure becomes unstable, and the obtained image becomes brighter or darker.

[0005] The present invention has been made in consideration of such problems, and has an object to provide a technique that enables stable exposure to a subject. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, an imaging control device according to the present invention has the following configuration. Included in the captured image captured by the imaging device The subject A detection means for detecting a characteristic portion; The features detected by the detection means Department Surrounded Domain a determining means for determining an exposure area to be subjected to exposure calculation based on the area; having The determining means Record Expand the area in the first direction and shrinking the image in a second direction different from the first direction. The exposed area is determined by: Alternatively, an imaging control device that controls imaging by the imaging device includes: Included in the captured image captured by the imaging device The subject A detection means for detecting a characteristic portion; The features detected by the detection means Department Surrounded Domain determining means for determining a weight distribution of exposure calculation for each region; before Record a calculation means for calculating an exposure value based on the weights determined for the regions; having The determining means expands an initial weight distribution set based on a section in which the subject exists in a first direction. and shrinking the image in a second direction different from the first direction. The weight distribution is determined by: Effect of the Invention

[0007] According to the present invention, it is possible to provide a technique that enables stable exposure to a subject. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an overall configuration of an imaging control system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of the surveillance camera. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of a client device. [Figure 4] FIG. 2 is a block diagram showing a functional configuration of a client device. [Diagram 5] FIG. 13 is a diagram showing an example of a captured image in which a human body region is underexposed and becomes dark. [Figure 6] FIG. 13 is a diagram showing an example of a captured image in which a human body region is appropriately exposed; [Figure 7] 1A to 1C are diagrams illustrating examples of captured images with different degrees of backlighting. [Figure 8] 4 is a flowchart of an exposure control process in the first embodiment. [Figure 9] FIG. 11 is a block diagram showing a functional configuration of a client device according to a second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a captured image in which a face and human body area are detected. [Figure 11] FIG. 13 is a diagram showing an example of weighting for detected face and human body regions. [Figure 12] 10 is a flowchart of an exposure control process in a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] (First embodiment) As a first embodiment of an imaging control device according to the present invention, a client device that controls imaging by a surveillance camera will be described below as an example.

[0011] <System configuration> FIG. 1 is a block diagram illustrating an example of the overall configuration of an imaging control system 100 according to the first embodiment.

[0012] The imaging control system 100 includes a surveillance camera 101, a network 102, a client device 103, an input device 104, and a display device 105. The surveillance camera 101 is an imaging device for acquiring moving images, and is a device capable of imaging a subject and processing the images. The surveillance camera 101 and the client device 103 are connected to each other via the network 102 in a state in which they can communicate with each other. The client device 103 is connected to the input device 104 and the display device 105 in a state in which they can communicate with each other. The client device 103 is a device that processes various information, and therefore may be called an information processing device. In addition, the client device 103 is a device that controls the imaging of the surveillance camera 101, and therefore may be called an imaging control device.

[0013] The input device 104 is composed of a mouse, a keyboard, etc., and is operated by a user who operates the client device 103. The display device 105 is a device equipped with a monitor and the like that displays images received from the client device 103. The display device 105 can also function as a graphical user interface (GUI) by installing a touch panel on its surface. In this case, the display device 105 can also function as an input device that inputs instructions, information, data, etc. to the client device 103.

[0014] In FIG. 1, the client device 103, the input device 104, and the display device 105 are depicted as independent devices, but are not limited to such a configuration. For example, the client device 103 and the display device 105 may be integrated, or the input device 104 and the display device 105 may be integrated. Also, the client device 103, the input device 104, and the display device 105 may be integrated. When the client device 103 and the display device 105 are integrated, the integrated device takes the form of, for example, a personal computer, a tablet terminal, or a smartphone.

[0015] One or more of the functional blocks shown in FIG. 4, which will be described later, may be realized by hardware such as an application specific integrated circuit (ASIC) or a programmable logic array (PLA). They may also be realized by a programmable processor such as a central processing unit (CPU) or an MPU executing a software program. MPU is an abbreviation for Micro-Processing Unit. They may also be realized by a combination of software and hardware. Therefore, even if different functional blocks are described as operating subjects in the following description, they may be realized by the same hardware as the subject.

[0016] -Surveillance camera hardware configuration 2 is a block diagram showing the hardware configuration of the surveillance camera 101. The surveillance camera 101 includes an imaging optical system 201 and an imaging element 202. The surveillance camera 101 also includes a CPU 203, a ROM 204, a RAM 205, an imaging system control unit 206, a communication control unit 207, an A / D conversion unit 208, an image processing unit 209, an encoder unit 210, and a network interface (I / F) 211. These elements are interconnected by a system bus 212. ROM is an abbreviation for Read Only Memory. RAM is an abbreviation for Random Access Memory. A / D is an abbreviation for Analog / Digital.

[0017] The imaging optical system 201 is a group of optical components that includes a zoom lens, a focus lens, a blur correction lens, an aperture, a shutter, etc., and collects optical information of a subject. The imaging optical system 201 is connected to the imaging element 202 so that an optical image formed by the imaging optical system 201 is guided to the imaging element 202.

[0018] The imaging element 202 is a charge-accumulation type solid-state imaging element such as a CMOS or CCD that converts an optical image collected by the imaging optical system 201 into a current value (signal value). For example, it is possible to obtain color information by combining it with a color filter or the like. CMOS is an abbreviation for Complementary Metal Oxide Semiconductor. CCD is an abbreviation for Charge-Coupled Device. The imaging element 202 is connected to an A / D conversion unit 208.

[0019] The CPU 203 is a control unit that performs overall control of the operation of the surveillance camera 101. The CPU 203 reads instructions stored in the ROM 204 or RAM 205, and executes processing in accordance with the results of the read instructions.

[0020] The imaging system control unit 206 controls each unit of the surveillance camera 101 based on instructions from the CPU 203. For example, the imaging system control unit 206 controls the imaging optical system 201 in terms of focus control, shutter control, aperture adjustment, and the like.

[0021] The communication control unit 207 performs control for transmitting control commands (control signals) to each unit of the security camera 101 from the client device 103 to the CPU 203 through communication with the client device 103 .

[0022] The A / D conversion unit 208 converts the optical image detected by the imaging element 202 into a digital signal (image data). The A / D conversion unit 208 transmits the obtained digital signal to an image processing unit 209. The image processing unit 209 performs various image processing on the image data of the digital signal received from the imaging element 202. The image processing unit 209 is connected to an encoder unit 210. The encoder unit 210 performs processing to convert the image data processed by the image processing unit 209 into a file format such as Motion JPEG, H.264, or H.265. The encoder unit 210 is connected to a network I / F 211.

[0023] The network I / F 211 is an interface used for communication with external devices such as the client device 103 via the network 102 , and is controlled by the communication control unit 207 .

[0024] The network 102 is a network that connects the surveillance camera 101 and the client device 103. The network 102 is composed of a plurality of routers, switches, cables, etc. that comply with a communication standard such as Ethernet (registered trademark). However, the network 102 may be of any communication standard, scale, configuration, etc. as long as it is capable of performing communication between the surveillance camera 101 and the client device 103. For example, the network 102 may be composed of the Internet, a wired LAN (Local Area Network), a wireless LAN, a WAN (Wide Area Network), etc.

[0025] - Client device hardware configuration 3 is a block diagram showing the hardware configuration of client device 103. Client device 103 includes a CPU 301, a main storage device 302, an auxiliary storage device 303, an input I / F 304, an output I / F 305, and a network I / F 306. These elements are connected to each other via a system bus 307 so as to be able to communicate with each other.

[0026] The CPU 301 is a central processing unit that performs overall control of the operation of the client device 103. Note that the CPU 301 may also perform overall control of the surveillance camera 101 via the network 102.

[0027] The main memory device 302 is a memory device such as a RAM that functions as a temporary storage location for data of the CPU 301. For example, the main memory device 302 is used to store patterns for pattern matching (patterns corresponding to facial features or human body features) used when the client device 103 performs face detection or human body detection.

[0028] The auxiliary storage device 303 is a storage device such as an HDD, ROM, or SSD that stores various programs, various setting data, etc. HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Drive.

[0029] The input I / F 304 is an interface used when the client device 103 receives an input (signal) from the input device 104 etc. The output I / F 305 is an interface used when the client device 103 outputs information (signal) to the display device 105 etc. The network I / F 306 is an interface used for communication with an external device such as the surveillance camera 101 via the network 102.

[0030] Functional configuration of client device Fig. 4 is a block diagram showing the functional configuration of client device 103. Various functions of client device 103 are realized by CPU 301 executing processes based on programs stored in auxiliary storage device 303. In other words, each functional unit shown in Fig. 4 is a function that can be executed by CPU 301, and each unit is synonymous with CPU 301. However, each functional unit shown in Fig. 4 may be realized by hardware (or software) other than CPU 301.

[0031] The client device 103 includes an input information acquisition unit 401, a communication control unit 402, an input image acquisition unit 403, a camera information acquisition unit 404, and a detection method setting unit 405. The client device 103 includes a subject detection unit 406, an exposure determination unit 408, and a display control unit 409.

[0032] The input signal acquisition unit 401 accepts input from a user via the input device 104. The communication control unit 402 executes control for receiving images transmitted from the surveillance camera 101 (i.e., images captured by the surveillance camera 101) via the network 102. The communication control unit 402 also executes control for transmitting control commands from the client device 103 to the surveillance camera 101 via the network 102.

[0033] The input image acquisition unit 403 acquires an image received from the surveillance camera 101 via the communication control unit 402 as an image to be subjected to subject detection processing. Details of the detection processing will be described later. The camera information acquisition unit 404 acquires camera information (imaging information) when an object is imaged by the surveillance camera 101 via the communication control unit 402. The camera information is various information when an object is imaged and an image is acquired, such as exposure parameters such as aperture value, shutter speed, and gain. It also includes an exposure correction value that changes an exposure target value used during AE (Auto Exposure).

[0034] The detection method setting unit 405 sets a predetermined (appropriate) detection method from among various detection methods including detection of a face area (face detection) and detection of a human body area (human body detection) for the image acquired by the input image acquisition unit 403. Here, the human body detection does not necessarily need to detect the whole body, and may be detection of the upper body, a bust-up, or a head area including the face.

[0035] Here, the detection method setting unit 405 sets (selects) a face detection method or a human body detection method, but is not limited to such settings. For example, a detection method for detecting a characteristic region of a part of a person, such as a part of the upper body, head, eyes, nose, mouth, etc. of a person, may be set (selected). In addition, although the subject to be detected here is a person, a configuration may be possible that is capable of detecting a specific region related to a predetermined subject other than a person. For example, a configuration may be possible that is capable of detecting a predetermined subject preset in the client device 103, such as an animal's face or a car.

[0036] The subject detection unit 406 gives priority to detecting a face region in an image when face detection is set by the detection method setting unit 405. It gives priority to detecting a human body region in an image when human body detection is set by the detection method setting unit 405. It calculates the coordinates (vertex coordinates, width, height, etc.) of the detected face region or human body region.

[0037] An exposure area setting section 407 sets an exposure area to be subjected to AE based on the face area or human body area calculated by the subject detection section 406. Setting of the exposure area will be described later with reference to Figs. 5 to 7.

[0038] An exposure determination unit 408 determines an exposure level based on image information of the subject area obtained from the exposure area set by the exposure area setting unit 407, and transmits the exposure level to the surveillance camera 101 from the communication control unit 402, and exposure control is performed via the control unit 207. As a method of adjusting the exposure level, at least one of parameters that affect exposure, such as aperture value, shutter speed, gain, and exposure compensation value, may be used.

[0039] The display control unit 409 , in accordance with an instruction from the CPU 301 , outputs the captured image in which the exposure correction determined by the exposure determination unit 408 is reflected to the display device 105 .

[0040] <Control of exposed area> Next, details of the setting of the exposed region in the exposed region setting unit 407 will be described with reference to Fig. 5 to Fig. 7. Here, the exposed region means a region that is a target of exposure calculation in AE.

[0041] 5 is a diagram showing an example of a captured image in which the human body region is underexposed and has dark spots. Such a captured image can be acquired when capturing an image from a dark place such as indoors toward a bright outdoor area. In other words, when a subject is captured indoors (in a dark place), the subject is backlit, resulting in underexposure of the face and human body, resulting in a so-called dark spots.

[0042] When it is desired to adjust the exposure to the face region or human body region, it is necessary to set the exposure region appropriately. Here, the frame calculated by the subject detection unit 406 is a rectangular region including the head of a person as shown in FIG. 5. However, as shown by the hatched region in FIG. 5, it may be large or out of position. In that case, the influence of the portion of the hatched region that is not the face region or human body region becomes large, and the exposure of the face region or human body region may not be appropriate. In addition, simply making the exposure region small (setting it to a part of the face region or human body region) narrows the exposure region, and the brightness of the face region or human body region is not stable in a scene with a lot of noise or movement. As a result, the exposure may fluctuate between a state where the exposure is adjusted to the background of the human body region (FIG. 5) and a state where the exposure is adjusted to the human body region (FIG. 6).

[0043] Fig. 6 is a diagram showing an example of a captured image in which the human body region is properly exposed. In order to suppress the above-mentioned fluctuation in exposure and accurately adjust the exposure of the face region and human body region, it is possible to set an exposure region as shown in Fig. 6. That is, if the main subject is a person, one of the four sides of the frame (face, human body region) calculated by subject detection unit 406 in the direction in which the lower half of the person's body is present is moved so as to include the neck or chest of the person.

[0044] This allows the exposure area to be set to the main subject (person) without including the background such as the shaded area in FIG. 5. In addition, since the area of ​​the exposure area can be increased, stable exposure can be achieved even in scenes with a lot of noise and movement. When a larger area is calculated from the subject detection unit 406, the horizontal width of the face or human body can be reduced and shifted in the direction in which the main subject is present, allowing exposure to be adjusted more accurately to the face area or human body area. Note that in FIG. 6, all of the remaining three sides are moved toward the center (reducing the area surrounded by the frame), but only one side may be moved. In some cases, the remaining three sides may not need to be moved.

[0045] In order to achieve stable exposure, it is possible to set the minimum area of ​​the exposure area in advance and set it so that it is equal to or larger than the set minimum area. It is better to change the minimum area according to the degree of movement of the subject and the amount of noise. Furthermore, to reduce the influence of the color of the clothes and to match the exposure to the face area or human body area, it is better to detect the position of the neck or shoulders and expand the exposure area to that area. In order to expand the area appropriately regardless of whether it is an adult or a child, it is better to expand it in proportion to the size of the face area or human body area.

[0046] Fig. 7 shows examples of captured images with different degrees of backlighting. Both Fig. 7(a) and Fig. 7(b) show captured images with proper exposure for the face area. However, Fig. 7(a) shows an example of a captured image with not so strong backlighting, while Fig. 7(b) shows an example of a captured image with quite strong backlighting. When the backlighting becomes strong as in Fig. 7(b), the saturated area may leak into the face area or human body area due to optical flare or sensor leakage.

[0047] In the case of Fig. 7(b), it is advisable to set the horizontal width of the exposed area of ​​the face or human body relatively narrower than that of Fig. 7(a). That is, a backlight determination is performed to determine the degree of backlight in the captured image, and the greater the degree of backlight, the greater the degree of reduction. However, this narrows the area of ​​the exposed area, so it is advisable to also increase the amount of expansion in the direction in which the torso is present. For example, it is advisable to determine the exposed area so that it has approximately the same area as the subject area, which is the frame calculated by the subject detection unit 406.

[0048] When the subject detection unit 406 detects a face region, the torso is often located vertically downward from the face region. Therefore, the exposed region may be expanded in a direction equivalent to the vertically downward direction in the captured image. Also, a configuration may be made in which the direction in which the main subject exists is determined from the positional relationship between the face region and the human body region. For example, the face region and the human body region are compared, and the direction in which they protrude the most is the human torso region. Therefore, the direction in which the main subject exists is the direction of the torso region.

[0049] <Device Operation> 8 is a flowchart of the exposure control process in the first embodiment. Here, the subject detection process and the exposure determination process are particularly described. The process of this flowchart is started in response to input of a captured image of a subject from the surveillance camera 101 to the client device 103 via the network 102.

[0050] In the imaging control system 100, it is assumed that the powers of the surveillance camera 101, client device 103, input device 104, and display device 105 are turned on, and a connection (communication) is established between the surveillance camera 101 and the client device 103. It is also assumed that the surveillance camera 101 captures an image of a subject at a predetermined update period, the surveillance camera 101 transmits image data to the client device 103, and the display device 105 displays the image.

[0051] In step S801, the detection method setting unit 405 sets face detection in the subject detection unit 406, and the subject detection unit 406 performs face detection processing on the input image. Patterns corresponding to facial features and human body features are stored in advance in the main storage device 302 of the client device 103, and the subject detection unit 406 detects a face area included in the captured image by pattern matching based on the patterns.

[0052] In step S802, the subject detection unit 406 determines whether a face area is detected in the image in the face detection process executed in S801. If no face area is detected, the process proceeds to S803, and if at least one face area is detected, the process proceeds to S805.

[0053] In step S803, the detection method setting unit 405 sets the subject detection unit 406 to detect a human body, and the subject detection unit 406 performs a human body detection process on the input image. Here, when detecting a face area, the face can generally be detected with high accuracy, and the face area of ​​the subject can be clearly distinguished from the area other than the face area. However, if the face direction, face size, face brightness, etc. are not suitable conditions for face detection, the face area cannot be detected accurately. In contrast, when performing human body detection, the area where a person exists can be detected regardless of the face direction, face size, face brightness, etc. As described above, human body detection does not necessarily need to detect the entire body, and may be the upper body, bust up, or head area including the face. FIG. 10 shows an example in which the face area and the human body area are detected.

[0054] Furthermore, when a pattern matching method is adopted as a method for detecting a subject, a pattern (classifier) ​​created using statistical learning may be used as a pattern used in pattern matching. Alternatively, subject detection may be performed using a method other than pattern matching. For example, subject detection may be performed using a brightness gradient in a local region. In other words, the method for detecting a subject is not limited to a specific detection method, and various methods such as detection based on machine learning or detection based on distance information may be adopted.

[0055] In step S804, the subject detection unit 406 determines whether a human body region is detected in the image in the human body detection process executed in S803. If a human body region is not detected, this process ends, and if a human body region is detected, the process proceeds to S806.

[0056] In step S805, the exposure determination unit 408 sets the attention area to be used in the following step S808 to the face area detected in S801. On the other hand, in step S806, the exposure determination unit 408 sets the attention area to be used in the following step S808 to the human body area detected in S803.

[0057] In step S807, an exposure area is set from the set attention area. In step S808, the exposure determination unit 408 calculates an average luminance value of the exposure area set in S807. Specifically, the exposure determination unit 408 applies information regarding the number of attention areas (the number of detected faces or human bodies) set in S805 or S806, the positions of the attention areas, and the sizes of the attention areas to Equation (1) to perform the calculation.

[0058]

number

[0059] Here, I(x, y) represents the brightness value of the two-dimensional coordinate position (x, y) in the horizontal direction (x-axis direction) and vertical direction (y-axis direction) in the image. In addition, s represents the index of the region of interest, f represents the number of regions of interest, (v, h) represents the center coordinate of the region of interest, k represents the size of the region of interest in the horizontal direction, and l represents the size of the region of interest in the vertical direction.

[0060] In step S809, the exposure determination unit 408 determines the exposure correction amount EV based on the average luminance value calculated in S808. correction First, the exposure determination unit 408 determines the average luminance value I object and the target brightness value of the region of interest I object target The difference value ΔDiff between the target luminance value I object target For example, the value may be arbitrarily set by the user, or may be set to a value that increases the accuracy in comparison with the authentication accuracy during face authentication.

[0061]

number

[0062] Next, the exposure determination unit 408 calculates the correction amount EV correction In addition, EV currentis an APEX-converted EV value based on the subject brightness value (BV value), and is set based on a program diagram relating to exposure control that is stored in advance in the client device 103.

[0063]

number

[0064] Here, the parameter β is the current exposure value EV current This is a coefficient that affects the degree (speed) of exposure correction when correcting exposure to the underexposure or overexposure side, with the value of parameter β at the center. By setting the value of parameter β to a large value, the processing speed (or time) required to reach the target value will be faster, but if an erroneous judgment occurs in the detection result or if the subject detection is unstable, the brightness of the entire screen will fluctuate sharply. On the other hand, if the value of parameter β is set to a small value, the processing speed (or time) required to reach the target exposure will be slower, but it will be more robust to erroneous detection and shooting conditions. This parameter β is used to set the current exposure value EV when the difference ΔDiff is equal to or greater than the set threshold Th. current is set as the exposure compensation value for

[0065] In step S810, the camera information acquisition unit 404 acquires information on the current exposure setting value from the camera. Here, an exposure correction value is acquired. In step S811, the exposure determination unit 408 communicates with the surveillance camera 101 via the communication control unit 402, and sets the exposure correction value calculated in step S809 in the surveillance camera 101.

[0066] In step S812, the subject detection unit 406 performs face detection in the same manner as in S801. In step S813, the subject detection unit 406 determines whether a face area has been detected in the image in the face detection process executed in S812. If a face area has been detected, the process proceeds to S814, and if a face area has not been detected, the process proceeds to S815.

[0067] In step S814, the exposure determination unit 408 communicates with the surveillance camera 101 via the communication control unit 402, sets the surveillance camera 101 to maintain the exposure compensation value set in S811, and ends this process. On the other hand, in step S815, the exposure determination unit 408 communicates with the surveillance camera 101 via the communication control unit 402, sets the exposure compensation value acquired in S810 in the surveillance camera 101, and ends this process.

[0068] As described above, according to the first embodiment, an appropriate exposure area is set from a face area or a human body area, and exposure control is performed based on the exposure area. In particular, the size of the exposure area (exposure frame) targeted for AE is changed according to the state of the subject in the captured image. This makes it possible to stably set an appropriate exposure control value for face detection even in a strong backlighting state, such as a store entrance or stadium gate with a glass door. This makes it possible to stably perform face detection, and quickly capture the faces of store visitors or visitor subjects (people). Second embodiment In the second embodiment, a form in which a weight is set for an area within a frame detected by the subject detection unit 406 and exposure control is performed will be described.

[0069] <System configuration> The overall configuration of the system in the second embodiment and the hardware configurations of the surveillance camera 101 and the client device 103 are similar to those in the first embodiment (FIGS. 1 to 3), and therefore a detailed description will be omitted.

[0070] Functional configuration of client device 9 is a block diagram showing the functional configuration of a client device in the second embodiment. Since components other than an exposure weight setting unit 907 are similar to those in the first embodiment (FIG. 4), the same reference numerals are used and detailed description is omitted.

[0071] The exposure weight setting unit 907 sets a weight distribution for calculating exposure in face regions or human body regions detected by the subject detection unit 406. Then, the exposure determination unit 408 calculates the average luminance value of the region taking the weights into consideration. Specifically, the exposure determination unit 408 applies information regarding the number of attention regions (the number of detected faces or human bodies), the positions of the attention regions, the sizes of the attention regions, and the set weights to Equation (4) to perform the calculation.

[0072]

number

[0073] Here, I(x, y) represents the brightness value of the two-dimensional coordinate position (x, y) in the horizontal direction (x-axis direction) and vertical direction (y-axis direction) in the image. Also, s represents the index of the area of ​​interest, and f represents the number of areas of interest. (v, h) represents the center coordinate of the area of ​​interest, k represents the size of the area of ​​interest in the horizontal direction, and l represents the size of the area of ​​interest in the vertical direction. M(x, y) represents the weight of the two-dimensional coordinate position (x, y).

[0074] Fig. 10 is a diagram showing an example of a captured image in which a face and human body region has been detected. Fig. 10(a) shows the detected face region, and Fig. 10(b) shows the detected human body region. As described in the first embodiment, the frame calculated by the subject detection unit 406 may be large or out of position, as shown by the hatched area in Fig. 5. In such a case, the influence of the hatched area that is not a face region or human body region may become large, and the exposure of the face region or human body region may not be appropriate.

[0075] FIG. 11 is a diagram showing an example of weighting for detected face and human body regions. In order to suppress the above-mentioned fluctuation in exposure and accurately adjust the exposure of the face and human body regions, it is possible to set weighting as shown in FIG. 11(a). That is, if the main subject is a person, it is possible to give a high weight to the area in which the face or human body actually exists within the frame (face and human body area) calculated by the subject detection unit 406. Here, an initial weight distribution is set such that the section in which the person's head (face) exists has a weight of 1 and the section in which the head does not exist has a weight of 0. Here, a weight distribution is set in four directions, up, down, left, and right.

[0076] Also, as shown in FIG. 11(b), weighting may be performed by expanding in the direction of the torso. That is, the weight distribution is determined by expanding the initial weight distribution in the direction of the torso (one direction in which the lower body exists). This increases the area of ​​the exposed region, making it possible to achieve stable exposure even in scenes with a lot of noise and movement. When a larger region is calculated by the subject detection unit 406, the weight is set to the center in the lateral direction of the face or human body, and the weight is shifted in the direction in which the main subject exists. That is, the weight distribution is moved toward the center for the remaining three directions. This makes it possible to more accurately adjust the exposure to the face region or human body region.

[0077] In order to achieve stable exposure, it is possible to set the minimum weight integral value of the exposure area in advance and set it so that it is equal to or greater than the minimum weight integral value. It is more preferable to change the minimum weight integral value according to the degree of subject movement or noise.

[0078] When the subject detection unit 406 detects a face region, the torso is often located vertically downward from the face region. Therefore, the weighting may be expanded in a direction corresponding to the vertically downward direction in the captured image. Also, the direction in which the main subject exists may be determined from the positional relationship between the face region and the human body region. For example, the face region and the human body region are compared, and the direction in which they protrude significantly is the human torso region. Therefore, the direction in which the main subject exists is the direction of the torso region.

[0079] Also, as in the first embodiment, when the backlight is strong as in Fig. 7(b), it is advisable to set the horizontal width of the weight relatively narrower than that in Fig. 7(a). However, since this narrows the area of ​​the exposed region, it is also advisable to widen the weight in the direction in which the torso is present. Furthermore, although the weight is set as a discrete value (0 or 1) in Fig. 11, it is also possible to set it to a continuous value.

[0080] 12 is a flowchart of the exposure control process in the second embodiment. The same parts as those in the first embodiment (FIG. 8) are given the same reference numerals and detailed description will be omitted.

[0081] In step S1207, the exposure weight setting unit 907 sets a weight from the set attention area. In step S1208, the exposure determination unit 408 calculates a luminance value for exposure using the weight set in S1207. The above-mentioned formula (4) is applied to calculate the luminance value.

[0082] In step S1209, the exposure determination unit 408 determines the exposure brightness value I object and the target brightness value of the region of interest I object target If the difference value ΔDiff is equal to or greater than the set threshold value Th, the current exposure value EV current is set as the exposure compensation value for

[0083] As described above, according to the second embodiment, an appropriate exposure area is set from a face area or a human body area, and exposure control is performed based on the exposure area. In particular, the weight of the exposure area targeted for AE is changed according to the state of the subject in the captured image. This makes it possible to stably set an appropriate exposure control value even in a backlit condition, as in the first embodiment.

[0084] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0085] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0086] 100 Imaging control system; 101 Surveillance camera; 102 Network; 103 Client device; 104 Input device; 105 Display device

Claims

1. An imaging control device that controls imaging by an imaging device, a detection means for detecting a feature of a subject included in an image captured by the imaging device; a determining means for determining an exposure area to be subjected to exposure calculation based on an area surrounding the characteristic portion detected by the detecting means; having The determining means determines the exposed area by expanding the area in a first direction and shrinking the area in a second direction different from the first direction. An imaging control device comprising:

2. The determining means determines the exposed region so as to have substantially the same area as the region.

2. The imaging control device according to claim 1.

3. The determining means determines the exposed area so as to have a predetermined area or more.

3. The imaging control device according to claim 1, wherein the imaging control device is a digital camera.

4. the subject is a person, The feature is at least one of an upper body, a head, a face, an eye, a nose, and a mouth; The first direction is a direction in which the lower half of the person's body is present.

4. The imaging control device according to claim 1, wherein the imaging control device further comprises:

5. the region is a rectangular region including a person's head, The determining means determines the exposed area by moving one of the four sides of the area in a direction in which the lower half of the person's body is present so as to include the neck or chest of the person and by moving at least one of the remaining three sides toward the center of the area.

5. The imaging control device according to claim 4.

6. The imaging device further includes a determination unit that determines the first direction based on the feature of the object including the feature portion detected by the detection unit.

6. The imaging control device according to claim 1, wherein the imaging control device further comprises:

7. The method further includes a backlight determination unit for determining a degree of backlight in the captured image, The determining means increases a degree of reduction of the area in the second direction as the degree of backlighting increases.

2. The imaging control device according to claim 1.

8. An imaging control device that controls imaging by an imaging device, a detection means for detecting a feature of a subject included in an image captured by the imaging device; a determining means for determining a weight distribution for exposure calculation for an area surrounding the feature detected by the detecting means; a calculation means for calculating an exposure value based on the weights determined for the regions; having The determining means determines the weight distribution by expanding an initial weight distribution set based on the section in which the subject exists in a first direction and shrinking the initial weight distribution in a second direction different from the first direction. An imaging control device comprising:

9. The determining means determines the weight distribution so that the weight integral value is substantially the same as the weight integral value of the initial weight distribution.

9. The imaging control device according to claim 8.

10. The determining means determines the weight distribution so as to be equal to or greater than a predetermined weight integral value.

10. The imaging control device according to claim 8, wherein the imaging control device is a digital camera.

11. the subject is a person, The feature is at least one of an upper body, a head, an eye, a nose, and a mouth; The first direction is a direction in which the lower half of the person's body is present.

11. The imaging control device according to claim 8, wherein the imaging control device comprises:

12. the region is a rectangular region including a person's head, the initial weight distribution is set as a distribution in which a section in which a person's head is present in the region has a weight of 1 and a section in which a head is not present has a weight of 0; The determining means determines the weight distribution by expanding a section of weight 1 in a direction in which the lower half of the person's body is present among the four directions corresponding to the four sides of the region so as to include the neck or chest of the person, and by moving a section of weight 1 in at least one of the remaining three directions toward the center of the region. The imaging control device according to claim 11 .

13. The imaging device further includes a determination unit that determines the first direction based on the feature of the object including the feature portion detected by the detection unit.

13. The imaging control device according to claim 8, wherein the imaging control device comprises:

14. The method further includes a backlight determination unit for determining a degree of backlight in the captured image, The determining means increases a degree of reduction in the second direction with respect to the initial weight distribution as the degree of backlighting increases.

9. The imaging control device according to claim 8.

15. A control method for an imaging control device that controls imaging by an imaging device, comprising: a detection step of detecting a feature portion of a subject included in an image captured by the imaging device; a determination step of determining an exposure area to be subjected to exposure calculation based on an area surrounding the characteristic portion detected by the detection step; Including, In the determining step, the exposed area is determined by expanding the area in a first direction and shrinking the area in a second direction different from the first direction. A control method comprising:

16. A control method for an imaging control device that controls imaging by an imaging device, comprising: a detection step of detecting a feature portion of a subject included in an image captured by the imaging device; a determining step of determining a weight distribution for exposure calculation for an area surrounding the feature detected by the detecting step; a calculation step of calculating an exposure value based on the weights determined for the regions; Including, In the determining step, the weight distribution is determined by expanding an initial weight distribution set based on the section in which the subject exists in a first direction and shrinking the initial weight distribution in a second direction different from the first direction. A control method comprising:

17. A program for causing a computer to function as each of the means of the imaging control device according to any one of claims 1 to 14.

Images