Information processing device, imaging device, method, program, and storage medium

Abstract

To perform exposure control suitable for detection of a photographic object.SOLUTION: An information processing apparatus includes: acquisition means configured to acquire a captured image; face detection means configured to detect a face area from the image; human body detection means configured to detect a human body area from the image if the face is not detected by the face detection means; determination means configured to determine a target exposure value on the basis of luminance of the face area detected by the face detection means or the human body area detected by the human body detection means; and output means configured to output the exposure value determined by the determination means. The determination means maintains the exposure value until the face area is newly detected by the face detection means or the human body region is newly detected by the body detection means, if a predetermined condition is satisfied.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an imaging apparatus, a method, a program, and a storage medium.

Background Art

[0002] Conventionally, a technique is known in which a face region of a subject is detected from a photographed screen, and image quality adjustment of a photographed image is performed based on information related to the face region (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to perform exposure control suitable for detection of a subject.

Means for Solving the Problems

[0005] The present invention solves the above problems by the following solution means.

[0006] An information processing apparatus according to one aspect of the present invention includes an acquisition unit that acquires an imaged being image, a face detection unit that detects a face region from the image, performing face region detection a body detection unit that detects a body region from the recorded image, a determination unit that determines a target exposure value based on the luminance of the face region detected by the face detection unit or the body region detected by the body detection unit, and an output unit that outputs the before determined recorded exposure value. The determination unit is configured to capture an image with the recorded determined recorded exposure value and the dew captured image face confirmation for determining whether to maintain the determined exposure value according to the result of the face region detection for the recorded dewWhen the process is invalidated, until a face area is newly detected by the face detection means or a human body area is detected by the human body detection means, newly maintain the exposure value determined until a human body area is detected by the human body detection means. and when the face confirmation process is enabled and no face region is detected in the face confirmation process, not maintaining the determined exposure value 。

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Modes for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following embodiments do not limit the present invention, and not all combinations of the features described in this embodiment are essential for the solution means of the present invention. The configuration of the embodiment can be appropriately modified or changed according to the specifications of the apparatus to which the present invention is applied and various conditions (usage conditions, usage environments, etc.). The technical scope of the present invention is determined by the scope of the claims and is not limited by the following individual embodiments. Also, a configuration may be formed by appropriately combining a part of each of the embodiments described later.

[0009] Note that one or more of the functional blocks shown in the drawings described later may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or an MPU executing software. Also, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as the operating entity, the same hardware can be realized as the main body. ASIC is an abbreviation for Application Specific Integrated Circuit. CPU is an abbreviation for Central Processing Unit. MPU is an abbreviation for Micro-Processing Unit.

[0010] ≪First Embodiment≫ (Basic Configuration) FIG. 1 is a diagram exemplarily explaining the configuration of an imaging control system 100 according to the first embodiment of the present invention.

[0011] The imaging control system 100 includes a surveillance camera 101, a network 102, a client device 103, an input device 104, and a display device 105. The surveillance camera 101 is an imaging device for acquiring moving images and is a device capable of imaging a subject and performing image processing. The surveillance camera 101 and the client device 103 are connected in a state where they can communicate with each other via the network 102. The client device 103 is connected in a state where it can communicate with the input device 104 and the display device 105. The client device 103 is a device (information processing device) that processes various types of information. Further, the client device 103 also functions as an imaging control device that controls the imaging of the surveillance camera 101.

[0012] The input device 104 is composed of a mouse, a keyboard, etc. and is operated by the user of the client device 103.

[0013] The display device 105 is a device equipped with a monitor or the like that displays the image received from the client device 103. Note that the display device 105 can also function as a UI (User Interface) such as a touch panel. In this case, the display device 105 also functions as an input device for inputting instructions, information, data, etc. to the client device 103.

[0014] In FIG. 1, the client device 103, the input device 104, and the display device 105 are depicted as independent devices, but the present embodiment is not limited to such a configuration. For example, the client device 103 and the display device 105 may be integrated, or the input device 104 and the display device 105 may be integrated. Further, the client device 103, the input device 104, and the display device 105 may be integrated. When the client device 103 and the display device 105 are integrated, the integrated device takes the form of, for example, a personal computer, a tablet terminal, or a smartphone.

[0015] (Structure of the surveillance camera) FIG. 2 is a block diagram exemplarily explaining the internal configuration of the surveillance camera 101. The surveillance camera 101 includes an imaging optical system 201, an image sensor 202, a camera CPU 203, a ROM 204, a RAM 205, an imaging system control unit 206, a communication control unit 207, an A / D conversion unit 208, an image processing unit 209, an encoder unit 210, and a network I / F 211. Each part (203 to 211) of the surveillance camera 101 is interconnected by a system bus 212. ROM is the abbreviation of Read Only Memory. RAM is the abbreviation of Random Access Memory. A / D is the abbreviation of Analog / Digital. I / F is the abbreviation of interface.

[0016] The imaging optical system 201 is composed of a zoom lens, a focus lens, an anti-shake lens, a diaphragm, a shutter, etc., and is an optical member group that condenses the optical information of the subject. The imaging optical system 201 is connected to the image sensor 202.

[0017] The image sensor 202 is a charge accumulation type solid-state image sensor such as a CMOS or a CCD that converts the light beam condensed by the imaging optical system 201 into a current value (signal value), and is an imaging unit that acquires color information by combining with a color filter, etc. CMOS is the abbreviation of Complementary Metal Oxide Semiconductor. CCD is the abbreviation of Charge-Coupled Device. The image sensor 202 is connected to the A / D conversion unit 208.

[0018] The camera CPU 203 is a control unit that comprehensively controls the operation of the surveillance camera 101. The camera CPU 203 reads the instructions stored in the ROM 204 and the RAM 205, and executes processing according to the results.

[0019] The imaging system control unit 206 controls each part of the surveillance camera 101 based on an instruction from the camera CPU 203. For example, the imaging system control unit 206 performs controls such as focus control, shutter control, and diaphragm adjustment on the imaging optical system 201.

[0020] The communication control unit 207 performs control to transmit a control command (control signal) from the client device 103 to each part of the surveillance camera 101 to the camera CPU 203 through communication with the client device 103.

[0021] The A / D conversion unit 208 converts the amount of light of the subject detected by the imaging device 202 into a digital signal (image data). The A / D conversion unit 208 transmits the digital signal to the image processing unit 209.

[0022] The image processing unit 209 performs image processing on the image data of the digital signal received from the imaging device 202. The image processing unit 209 is connected to the encoder unit 210.

[0023] The encoder unit 210 performs processing to convert the image data processed by the image processing unit 209 into a file format such as Motion Jpeg, H264, or H265. The encoder unit 210 is connected to the network I / F 211.

[0024] The network I / F 211 is an interface used for communication via the network 102 with an external device such as the client device 103 and is controlled by the communication control unit 207.

[0025] The network 102 is an IP network that connects the surveillance camera 101 and the client device 103. The network 102 is composed of a plurality of routers, switches, cables, etc. that conform to a communication standard such as Ethernet (trademark). In this embodiment, the network 102 only needs to be able to communicate between the surveillance camera 101 and the client device 103, regardless of its communication standard, scale, configuration, etc. For example, the network 102 may be composed of the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), etc.

[0026] (Configuration of the client device) FIG. 3 is a block diagram exemplarily illustrating the internal configuration of the client device 103.

[0027] The client device 103 includes a client CPU 301, a main storage device 302, an auxiliary storage device 303, an input I / F 304, an output I / F 305, and a network I / F 306. Each element of the client device 103 is communicably connected to each other via a system bus 307.

[0028] The client CPU 301 is a central processing unit that comprehensively controls the operation of the client device 103. Note that the client CPU 301 may execute overall control of the monitoring camera 101 via the network 102.

[0029] The main storage device 302 is a storage device such as a RAM that functions as a temporary storage location for data of the client CPU 301. For example, the main storage device 302 stores in advance patterns for pattern matching (patterns corresponding to characteristic parts of a face or a human body) used when the client device 103 performs face detection or human body detection.

[0030] The auxiliary storage device 303 is a storage device such as an HDD, a ROM, or an SSD that stores various programs, various setting data, and the like. HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Device.

[0031] The input I / F 304 is an interface used when the client device 103 receives an input (signal) from an input device 104 or the like.

[0032] The output I / F 305 is an interface used when the client device 103 outputs information (signal) to a display device 105 or the like.

[0033] The network I / F 306 is an interface used for communication via the network 102 with an external device such as the monitoring camera 101.

[0034] The client CPU 301 executes processing based on a program stored in the auxiliary storage device 303, thereby realizing the functions and processing of the client device 103 shown in FIG. 4. Details thereof will be described later.

[0035] (Functions of the client device) FIG. 4 is a diagram exemplarily explaining the functions executed by the client device 103. In other words, each part (function block) illustrated in FIG. 4 is a function that can be executed by the client CPU 301.

[0036] As shown in FIG. 4, the client CPU 301 of the client device 103 includes an input information acquisition unit 401, a communication control unit 402, an input image acquisition unit 403, a camera information acquisition unit 404, and a detection method setting unit 405. The client CPU 301 also includes a subject detection unit 406, an exposure determination unit 407, and a display control unit 408. Note that the client device 103 may execute the functions of each part 401 to 408 illustrated in FIG. 4 by hardware (or software) different from the client CPU 301.

[0037] The input signal acquisition unit 401 receives an input from the user via the input device 104.

[0038] The communication control unit 402 executes control for receiving an image transmitted from the monitoring camera 101 (an image captured by the monitoring camera 101) via the network 102. The communication control unit 402 also executes control for transmitting a control command from the client device 103 to the monitoring camera 101 via the network 102.

[0039] The input image acquisition unit 403 acquires, as an image to be subjected to subject detection processing (an image to which subject detection processing is applied), the image received from the monitoring camera 101 via the communication control unit 402. Details of the detection processing will be described later.

[0040] The camera information acquisition unit 404 acquires camera information (imaging information) when the monitoring camera 101 images a subject via the communication control unit 402. The camera information (imaging information) is various information when imaging a subject to obtain an image. The camera information is, for example, exposure parameters such as aperture values.

[0041] The detection method setting unit 405 sets a predetermined (appropriate) detection method from various detection methods including face area detection (face detection) and human body area detection (human body detection) for the image acquired by the input image acquisition unit 403. When performing face detection, the subject detection unit 406 described later preferentially detects the face area in the image. When performing human body detection, the subject detection unit 406 preferentially detects the human body area in the image.

[0042] In this embodiment, it is assumed that the detection method setting unit 405 sets (selects) the detection method for face detection or the detection method for human body detection. Note that this embodiment is not limited to such settings. For example, a detection method for detecting a feature area of a part of a person such as the upper body, head, eyes, nose, mouth, etc. of a person may be set (made selectable).

[0043] Also, in this embodiment, the subject to be detected is a person, but a configuration capable of detecting a specific area related to a predetermined subject other than a person may be used. For example, a configuration capable of detecting a predetermined subject preset in the client device 103 such as the face of an animal or an automobile may be used.

[0044] The exposure determination unit 407 determines an exposure value based on the image information of the subject area obtained from the subject detection unit 406, transmits (outputs) the aforementioned exposure value to the monitoring camera 101 from the communication control unit 402, and exposure control (exposure adjustment) is executed via the control unit 207. Details of the processing flow related to the camera information acquisition unit 404, the detection method setting unit 405, the subject detection unit 406, and the exposure determination unit 407 will be described later with reference to the flowchart of FIG. 5.

[0045] The display control unit 408 outputs, in accordance with an instruction from the CPU 301, a captured image in which the exposure correction determined by the exposure determination unit 407 is reflected, to the display device 105.

[0046] (Subject detection process · Exposure determination process) Hereinafter, with reference to the flowchart shown in FIG. 5, the exposure control process according to the present embodiment will be described.

[0047] In the imaging control system 100 of FIG. 1, it is assumed that the power supplies of the surveillance camera 101, the client device 103, the input device 104, and the display device 105 are turned on, and the connection (communication) between the surveillance camera 101 and the client device 103 is established.

[0048] Also, in this state, it is assumed that imaging of the subject by the surveillance camera 101, transmission of image data from the surveillance camera 101 to the client device 103, and image display on the display device 105 are repeated at a predetermined update cycle.

[0049] Then, in response to the input of the captured image of the subject from the surveillance camera 101 to the client device 103 via the network 102, the process of the flowchart in FIG. 5 is started by the client CPU 301.

[0050] First, in step S501, the detection means setting unit 405 sets face detection for the subject detection unit 406, and the subject detection unit 406 performs face detection processing on the input image. In the main storage device 302 of the client device 103, respective patterns corresponding to the characteristic portions of the face and the characteristic portions of the human body are stored in advance, and the subject detection unit 406 detects the face region by pattern matching based on the pattern.

[0051] Next, in step S502, the subject detection unit 406 determines whether a face region is detected in the face detection processing executed in step S501. If no face region is detected, the process proceeds to step S503, and if at least one or more face regions are detected, the process proceeds to step S505.

[0052] In step S503, the detection means setting unit 405 sets the detection of a human body for the subject detection unit 406, and the subject detection unit 406 performs human body detection processing on the input image.

[0053] In face detection, when conditions suitable for face detection such as the orientation of the face, the size of the face, and the brightness of the face are not met, the face region cannot be accurately detected. In contrast, human body detection can detect the region where a person exists regardless of the orientation of the face, the size of the face, the brightness of the face, etc.

[0054] Note that the human body detection in the present invention does not necessarily need to detect the whole body, and may be the upper body, bust-up, or the head region including the face. FIG. 7 shows detection examples of the face region and the human body region.

[0055] Also, when adopting the pattern matching method as the subject detection method, as the pattern used in pattern matching, a pattern (discriminator) created using statistical learning may be used. Alternatively, subject detection may be performed by a method other than pattern matching. For example, subject detection may be performed using the luminance gradient within a local region. That is, the subject detection method is not limited to a specific detection method, and various methods such as detection based on machine learning and detection based on distance information can be adopted.

[0056] In step S504, the subject detection unit 406 determines whether a human body region is detected in the human body detection processing executed in step S503. If the human body region is not detected, this process ends, and if it is detected, the process proceeds to step S506.

[0057] In step S505, the exposure determination unit 407 sets the attention region to be used in the subsequent step S507 to the face region detected in step S501.

[0058] In step S506, the exposure determination unit 407 sets the attention region to be used in the subsequent step S507 to the human body region detected in step S503.

[0059] In step S507, the exposure determination unit 407 calculates the average luminance value of the target area set in step S505 or step S506. Specifically, the exposure determination unit 407 applies information on the number of target areas (the number of detected faces or human bodies), the positions of the target areas, and the sizes of the target areas set in step S505 or step S506 to the following formula (1).

[0060]

Equation

[0061] Here, I(x, y) represents the luminance value at the two-dimensional coordinate position (x, y) in the horizontal direction (x-axis direction) and the vertical direction (y-axis direction) within the image. Also, f represents the number of target areas, (v, h) represents the center coordinates of the target area, k represents the size of the target area in the horizontal direction, and l represents the detected size of the target area in the vertical direction.

[0062] In step S508, the exposure determination unit 407 determines the exposure value EVcorrection as the exposure target based on the average luminance value calculated in step S507. First, the exposure determination unit 407 calculates the difference value between the average luminance value Iobject of the target area calculated in step S507 and the target luminance value Iobject target of the target area as shown in formula (2). Note that the target luminance value Iobject target of the target area may be arbitrarily set by the user, for example, or may be set to a value that improves the accuracy in light of the authentication accuracy at the time of face authentication.

[0063]

Equation

[0064] Next, the exposure determination unit 407 determines the exposure value EVcorrection as shown in Equation (3). Note that EVcurrent is the EV value in APEX conversion based on the subject luminance value (BV value), and is set based on the program diagram related to exposure control stored in advance in the client device 103.

[0065]

Equation

[0066] Here, the parameter β is a coefficient that affects the degree (speed) of correction when correcting the exposure on the underexposure side or overexposure side of the exposure, centered on the current exposure value EVcurrent. By setting the value of the parameter β large, the processing speed (or time) required to reach the exposure target value becomes high, but when a false determination occurs in the detection result or when the detection of the subject is unstable, the brightness of the entire screen fluctuates steeply. On the other hand, when the value of the parameter β is set small, the processing speed (or time) required for the exposure to reach the target becomes slow, but it becomes robust against false detection and shooting conditions. This parameter β is set as the exposure correction value for the current exposure value EVcurrent when the difference ΔDiff is equal to or greater than the set threshold Th.

[0067] In step S509, the camera information acquisition unit 404 acquires information on the current exposure value from the camera.

[0068] In step S510, the exposure determination unit 407 communicates with the surveillance camera 101 via the communication control unit 402, and sets the exposure value calculated in step S508 in the surveillance camera 101.

[0069] In step S511, the subject detection unit 406 performs face detection in the same manner as in step S501.

[0070] In step S512, the subject detection unit 406 determines whether a face region is detected in the image in the face detection process executed in step S511. If a face region is detected, the process proceeds to step S513; if no face region is detected, the process proceeds to step S514.

[0071] In step S513, the exposure determination unit 407 communicates with the surveillance camera 101 via the communication control unit 402, sets the surveillance camera 101 to maintain (fix) the exposure value set in step S510, and ends this process. Specifically, it is maintained until a face or a human body is detected again and a new exposure target value is set.

[0072] In step S514, the exposure determination unit 407 communicates with the surveillance camera 101 via the communication control unit 402, returns the exposure target to the exposure value (exposure setting value) obtained in step S509, and ends this process. That is, the exposure value set in step S510 is discarded. Thereafter, based on the luminance value of the entire image, the exposure target is sequentially updated to perform exposure control. Note that the luminance value does not necessarily have to be that of the entire image, and the luminance values of one or more regions (predetermined regions) preset in the image can be used. As described above, in the imaging control system 100 of the present embodiment, exposure control is performed based on the face region or the human body region, and when a face can be detected as a result, the exposure value is maintained. By maintaining the exposure value in this way, it is possible to maintain an exposure suitable for face detection. For example, even in a strong backlight state such as a store entrance or a stadium gate that employs a glass door, it is possible to maintain an exposure value appropriate for face detection. Therefore, it is possible to quickly capture the faces of customers or visitors (persons) as subjects.

[0073] Also, in the present embodiment, exposure control is performed based on the face region or the human body region, and when a face cannot be detected as a result, the original exposure value is restored. Therefore, even when appropriate exposure control is not performed due to misdetection of a face or a human body, the exposure value is not maintained.

[0074] ≪Second Embodiment≫ Hereinafter, with reference to FIGS. 6 to 7, a second embodiment of the present invention will be described. In this embodiment, a method of performing exposure control more suitable for face recognition by performing exposure control again based on the face region when exposure control is performed based on the human body region in the first embodiment will be described. Regarding the same configurations and processes as in the first embodiment, the same reference numerals are given and detailed descriptions are omitted.

[0075] FIG. 6 is a flowchart of this embodiment. Hereinafter, with reference to the flowchart shown in FIG. 6, the exposure control process according to this embodiment will be described. Note that the description of the start timing of the process is the same as that in the first embodiment and will be omitted. Also, the processes of S501 to S514 are the same as those in the first embodiment, and the description will be omitted.

[0076] In step S601, the exposure determination unit 407 sets the human body detection flag indicating that a human body has been detected to OFF. This process is performed when it is determined in step S502 that a face has been detected. In the subsequent steps S505 and later, exposure control is performed based on the face region as described in the first embodiment.

[0077] In step S602, the exposure determination unit 407 sets the human body detection flag indicating that a human body has been detected to ON. This process is performed when it is determined in step S504 that a human body has been detected. In the subsequent steps S506 and later, exposure control is performed based on the human body region as described in the first embodiment.

[0078] In step S603, the exposure determination unit 407 determines whether the human body detection flag is OFF. If it is OFF, the process proceeds to step S513. If not (human body detection flag ON), the process proceeds to step S601. That is, when exposure control is performed based on the face region, the human body detection flag is OFF, so the process proceeds to step S513, where the exposure control (exposure value) is maintained and the process ends. On the other hand, when exposure control is performed based on the human body region, the human body detection flag is ON, so the process returns to step S601, and exposure control based on the face region is performed. In this case, in step S505, the face region detected in step S511 is set as the target region.

[0079] As shown in FIG. 7, the human body region may include regions other than the face, such as clothes. Therefore, when maintaining the exposure value based on the human body region as in the first embodiment, there is an advantage that the process is simple and has excellent responsiveness. However, it may not always result in an exposure suitable for the face region.

[0080] According to the second embodiment, when exposure control is performed based on the human body region and as a result the face is detected, exposure control is performed again based on the face region. By performing the process as in this embodiment, an exposure suitable for the face region is set, so that an exposure more suitable for face recognition can be maintained.

[0081] ≪Third Embodiment≫ Hereinafter, with reference to FIG. 8, the third embodiment of the present invention will be described. As described so far, in the first and second embodiments, an exposure value suitable for face detection can be maintained. However, for example, when a store has few customers and the face or human body is not detected for a long time, and the brightness inside and outside the store changes due to changes in time, weather, or lighting, the maintained exposure value may not be suitable for face detection. In particular, when there is a large change in brightness such that the backlight state and the frontlight state are switched, if the exposure value is maintained, the face or human body may be greatly overexposed or underexposed, and the detection accuracy may be significantly reduced.

[0082] Therefore, in this embodiment, a method for releasing the maintenance of the exposure value in response to a change in the brightness of the shooting environment will be described. Regarding the same configurations and processes as in the first and second embodiments, the same reference numerals will be given and detailed descriptions will be omitted.

[0083] FIG. 8 is a flowchart of this embodiment. Hereinafter, with reference to the flowchart illustrated in FIG. 8, the exposure control process according to this embodiment will be described. Note that the description of the start timing of the process is the same as that in the first embodiment, so it will be omitted. Also, the processes of S501 to S514 are the same as those in the first embodiment, so the description will be omitted.

[0084] In step S801, the exposure determination unit 407 sets an exposure maintenance flag indicating that the maintenance of exposure control has been performed to ON. Note that the exposure maintenance flag is a static variable and its content is carried over to the next exposure control process.

[0085] In step S802, the exposure determination unit 407 calculates the average luminance value of the entire screen. This process is executed when both step S502 and step S504 are determined to be No. That is, when neither a face nor a human body is detected and there is no person as the subject, the average luminance value of the entire screen is calculated.

[0086] In step S803, the exposure determination unit 407 determines whether the exposure maintenance flag is ON. If it is ON, the process proceeds to step S804; otherwise, the process proceeds to step S806.

[0087] In step S804, the exposure determination unit 407 saves the average luminance value of the entire screen calculated in step S802 as a reference luminance value (reference value). Note that the reference luminance value is a static variable and its content is carried over to the next exposure control process.

[0088] In step S805, the exposure determination unit 407 sets the exposure maintenance flag to OFF and ends this process.

[0089] On the other hand, in step S806, the exposure determination unit 407 calculates a difference value between the average luminance value of the entire screen calculated in step S802 and the reference luminance value stored in step S804. In the present embodiment, when step S804 has not been executed even once, the difference value is set to 0.

[0090] In step S807, the exposure determination unit 407 determines whether or not the difference value calculated in step S806 is equal to or greater than a preset threshold value. If so, the process proceeds to step S808; otherwise, this process ends. The threshold value may be set, for example, based on the range of average luminance values that can be detected by the subject detection unit 406 by photographing various face and human body brightness patterns in advance with the surveillance camera 101 while changing exposure settings, lighting, etc.

[0091] In step S808, the exposure determination unit 407 communicates with the surveillance camera 101 via the communication control unit 402, resets the exposure value of the surveillance camera 101 to a specified value that the surveillance camera 101 has, and ends this process. The reset of the exposure value may be performed, for example, by setting EVcorrection in Equation (3) to 0.

[0092] As described above, according to the third embodiment, the maintenance of the exposure value is released in response to a change in the brightness of the shooting environment. By performing processing as in this embodiment, even when there is a large change in the brightness of the shooting environment, the exposure value is reset, so that a decrease in the detection accuracy of the face or human body can be suppressed.

[0093] In the present embodiment, for the sake of simplicity of explanation, an example based on the first embodiment is shown, but it can also be combined with the second embodiment. In that case, step S801 may be executed after step S513 in the second embodiment, and steps S802 to S808 may be executed when it is determined No in step S504.

[0094] In addition, in this embodiment, a method using the average luminance value of the entire screen is shown, but it is not necessarily so. For example, a weighted average luminance value obtained by weighting the detected region of a face or a human body, or the average luminance value of the vicinity region where a face or a human body is detected may be used.

[0095] ≪Fourth Embodiment≫ In the above-described embodiment, an example in which the exposure value is maintained when a face is detected after exposure control has been described. However, depending on conditions such as the angle of view (shooting range) of the camera and the moving speed of the subject, the subject may move out of the angle of view (shooting range) before the face detection process is performed. In that case, the exposure value of the exposure control performed based on the luminance of the region of interest (face / human body) is not maintained.

[0096] In this embodiment, when the face confirmation process is not effective (invalid), the exposure value is maintained regardless of whether a face is detected. Note that the same components and processes as those in the above-described embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

[0097] FIG. 9 is a flowchart of this embodiment. Hereinafter, the exposure control process according to this embodiment will be described with reference to the flowchart shown in FIG. 9. Note that the description of the start timing of the process is omitted because it is the same as that in the first embodiment. In addition, the processes of S501 to S514 are the same as those in the first embodiment, and thus the description thereof is omitted.

[0098] In step S901, the exposure determination unit 407 determines whether the face confirmation process is effective. Here, the face confirmation process is the face detection process for the image after the exposure control in step S510. If the face confirmation process is effective, the process proceeds to step S511, and the face confirmation process (S511-512) is performed. On the other hand, if the face confirmation process is not effective, the process proceeds to step S513, and the exposure value is maintained (exposure is fixed). That is, the exposure value is maintained when a predetermined condition is satisfied. Here, the predetermined condition is that the face confirmation process is invalid (NO in step S901), or the face confirmation process is effective and the face region is detected (YES in step S512).

[0099] Note that the face confirmation process may be set to invalid if the subject moves out of the angle of view (shooting range) before the face confirmation process is completed, depending on conditions such as the installation status of the camera, the angle of view (shooting range), and the subject speed, and set to valid otherwise.

[0100] Note that the face confirmation process can also be automatically set according to the ratio of the presence or absence of face detection in step S512. For example, when the ratio of the presence of face detection is low, it is determined that the environment is such that the subject is likely to move out of the angle of view, and the face confirmation process is enabled. Otherwise, the face confirmation process is disabled.

[0101] The exposure fixed in step 513 is maintained until a new face or human body is detected. That is, the fixed exposure is released when a new face or human body is detected. Note that the fixed exposure may be released after a predetermined time has elapsed since the start of fixing (maintaining) the exposure. After releasing the fixed exposure, the exposure value is calculated based on the luminance of a predetermined area (for example, the entire image or an area near the center of the image) of the captured image, and exposure control is performed.

[0102] ≪Other Embodiments≫ In the above embodiment, an example of detecting a face or a human body has been described, but the present invention is not limited to this. The object to be detected may be a specific object, for example, an automobile and its license plate.

[0103] Also, in the above embodiment, an example of performing exposure control of the surveillance camera 101 by the client device 103 via the network 102 has been described, but the surveillance camera 101 itself may perform exposure control.

[0104] Further, part or all of the control in the present invention may be supplied to an imaging device or an information processing device via a network or various storage media as a program (software) that realizes the functions of the above-described embodiments. Then, a computer (or a CPU, MPU, etc.) in the imaging device or the information processing device may read and execute the program. In that case, the program and the storage medium storing the program will constitute the present invention.

[0105] Also, it can be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0106] As described above, the embodiments of the present invention have been described. However, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist thereof.

Explanation of Reference Numerals

[0107] 101 Surveillance camera 103 Client device 104 Input device 105 Display device

Claims

1. An acquisition means for acquiring a captured image; a face detection means for detecting a face area from the image; a human body detection means for detecting a human body region from the image; a determination means for determining a target exposure value based on the luminance of the face area detected by the face detection means or the luminance of the human body area detected by the human body detection means; an output means for outputting the determined exposure value; having The determining means is maintain the determined exposure value until a face area is newly detected by the face detection means or a human body area is newly detected by the human body detection means, when a face confirmation process for determining whether to maintain the determined exposure value depending on a result of the face area detection for an image captured with the determined exposure value is disabled; When the face confirmation process is enabled and a face area is not detected in the face confirmation process, the determined exposure value is not maintained.

23. An information processing apparatus comprising:

2. When the face confirmation process is enabled and a face area is detected in the face confirmation process, the determining means maintains the determined exposure value until a face area is newly detected by the face detection means or a human body area is newly detected by the human body detection means.

2. The information processing apparatus according to claim 1,

3. 3. The information processing device according to claim 1, wherein the determining means, when not maintaining the determined exposure value, discards the determined exposure value and determines a new exposure value based on the luminance of a predetermined area of ​​the captured image.

4. The information processing device according to any one of claims 1 to 3, characterized in that if a predetermined time has elapsed since the start of maintaining the determined exposure value, a state in which neither a face region nor a human body region is detected, the determination means cancels the maintenance of the determined exposure value and determines a new exposure value based on the brightness of a predetermined region of the captured image.

5. a storage means for storing an average luminance of a predetermined area of ​​an image as a reference value when neither a face area nor a human body area is detected by the face detection means or the human body detection means after the determination means starts to maintain the determined exposure value; a calculation means for calculating a difference value between a current average luminance of the predetermined region and the reference value when neither a face region nor a human body region is detected by the face detection means or the human body detection means again after the storage means has stored the reference value; and The information processing device according to any one of claims 1 to 4, characterized in that, when the difference value calculated by the calculation means is equal to or greater than a threshold value, the determination means cancels the maintenance of the determined exposure value and determines a new exposure value based on the brightness of a specified area of ​​the captured image.

6. 6. The information processing apparatus according to claim 3, wherein the predetermined area is the entire area of ​​a captured image.

7. 6. The information processing apparatus according to claim 3, wherein the predetermined area is an area near the center of a captured image.

8. The information processing device according to any one of claims 1 to 7, characterized in that the determination means determines the target exposure value by correcting the current exposure value so that a difference between the luminance of the face area detected by the face detection means or the human body area detected by the human body detection means and a target luminance falls within a predetermined range.

9. 9. The information processing apparatus according to claim 1, wherein the human body region is any one of a whole body, an upper body, a bust-up, and a head region including a face.

10. An imaging apparatus comprising the information processing apparatus according to claim 1 .

11. An acquisition step of acquiring a captured image; a face detection step of detecting a face area from the image; a human body detection step of detecting a human body region from the image; a determination step of determining a target exposure value based on the luminance of the face area detected in the face detection step or the luminance of the human body area detected in the human body detection step; an output step of outputting the determined exposure value; having maintain the determined exposure value until a face area is newly detected by the face detection step or a human body area is newly detected by the human body detection step, when a face confirmation process for determining whether to maintain the determined exposure value depending on a result of the face area detection for an image captured with the determined exposure value is disabled; When the face confirmation process is enabled and a face area is not detected in the face confirmation process, the determined exposure value is not maintained. A method comprising:

12. A program for causing a computer to execute a method, the method comprising: An acquisition step of acquiring a captured image; a face detection step of detecting a face area from the image; a human body detection step of detecting a human body region from the image; a determination step of determining a target exposure value based on the luminance of the face area detected in the face detection step or the luminance of the human body area detected in the human body detection step; an output step of outputting the determined exposure value; having maintain the determined exposure value until a face area is newly detected by the face detection step or a human body area is newly detected by the human body detection step, when a face confirmation process for determining whether to maintain the determined exposure value depending on a result of the face area detection for an image captured with the determined exposure value is disabled; When the face confirmation process is enabled and a face area is not detected in the face confirmation process, the determined exposure value is not maintained. A program characterized by:

13. A computer-readable storage medium storing the program according to claim 12.

Images