Control device, control method, and storage medium for controlling an image pickup device

Abstract

A control device, a control method, and a storage medium control an imaging device. A control device that controls an imaging device includes a determination unit configured to determine whether or not a subject in an image captured by the imaging device is an authenticated subject, and a control unit configured to control exposure of the image by targeting the subject in a case where the determination unit determines that the subject is not an authenticated subject.

Description

Technical Field

[0001] This invention relates to a camera control device, camera control method, and storage medium that controls the exposure of a camera device based on the subject authentication result. Background Technology

[0002] Japanese Patent Application Publication No. 4254873 discusses a technique for detecting facial regions of a subject from a captured image and then setting camera conditions based on the importance of the detected face. Summary of the Invention

[0003] According to one aspect of this disclosure, a control device for controlling a camera device is provided, comprising: a determining unit configured to determine whether a subject in an image captured by the camera device is an authenticated subject; and a control unit configured to control the exposure of an image by targeting the subject if the determining unit determines that the subject is not an authenticated subject.

[0004] Other features of the invention will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0005] Figure 1 This is a block diagram illustrating the configuration of a camera control system according to a first exemplary embodiment of the present invention.

[0006] Figure 2 This is a block diagram illustrating the internal configuration of a surveillance camera according to a first exemplary embodiment.

[0007] Figure 3 This is a block diagram illustrating the internal configuration of a client device according to a first exemplary embodiment.

[0008] Figure 4 It is shown by Figure 3 The diagram shows the functions performed by the client device.

[0009] Figure 5 This is a flowchart illustrating the exposure control process according to a first exemplary embodiment.

[0010] Figure 6 This is a flowchart illustrating a facial authentication process according to a first exemplary embodiment.

[0011] Figure 7A and 7B These are diagrams showing the face detection area and the human body detection area, respectively.

[0012] Figure 8 This is a flowchart illustrating the exposure control process according to a second exemplary embodiment. Detailed Implementation

[0013] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The following exemplary embodiments do not limit the invention, and not all combinations of features described in the exemplary embodiments are necessary for the solutions of the invention. Configurations according to the exemplary embodiments may be appropriately modified or changed based on the specifications of the apparatus to which any exemplary embodiment is applied and various conditions (e.g., usage conditions, usage environment). The scope of the invention is defined by the claims and is not limited to the various exemplary embodiments. Furthermore, portions of the exemplary embodiments may be appropriately combined.

[0014] Figure 4 One or more functional blocks shown (described below) can be implemented in hardware such as application-specific integrated circuits (ASICs) or programmable logic devices (PLAs), or can be implemented in software by a programmable processor such as a central processing unit (CPU) or a microprocessor unit (MPU). Furthermore, Figure 4 One or more functional blocks shown can be implemented using a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as operational entities, the functional blocks can be implemented by the same hardware.

[0015] Reference Figures 1 to 7B A first exemplary embodiment of the present invention is described.

[0016] <Basic Configuration>

[0017] Figure 1 The configuration of a camera control system 100 according to this exemplary embodiment is shown.

[0018] The camera control system 100 includes a surveillance camera 101, a network 102, a client device 103, an input device 104, and a display device 105. The surveillance camera 101 is a camera device for acquiring moving images and is capable of capturing images of a subject and performing image processing. Each of the surveillance camera 101, client device 103, input device 104, and display device 105 includes communication functionality. The surveillance camera 101 and client device 103 are communicatively interconnected via the network 102. The client device 103 is communicatively connected to the input device 104 and the display device 105. The client device 103 processes various types of information and can therefore be referred to as an information processing device. The client device 103 also controls the imaging of the surveillance camera 101 (e.g., controls exposure). Therefore, the client device 103 can be referred to as a camera control device. For example, the subject is a person.

[0019] The input device 104 includes a mouse and a keyboard, and is operated by the user of the client device 103.

[0020] Display device 105 includes a monitor for displaying images received from client device 103. Additionally, display device 105 can be used as a user interface (UI) such as a touch panel. In this case, display device 105 can also be used as an input device for inputting instructions, information, and data to client device 103.

[0021] exist Figure 1 In this embodiment, client device 103, input device 104, and display device 105 are shown as separate devices, but this exemplary embodiment is not limited to this configuration. For example, client device 103 and display device 105 may be integrated into one device, or input device 104 and display device 105 may be integrated into one device. Alternatively, client device 103, input device 104, and display device 105 may be integrated into one device. When client device 103 and display device 105 are integrated into one device, the integrated device may take the form of, for example, a personal computer, tablet terminal, or smartphone.

[0022] <Configuration of Surveillance Cameras>

[0023] Figure 2 This is a block diagram showing the internal configuration of a surveillance camera 101. The surveillance camera 101 includes a camera optical system 201, an image sensor 202, a CPU (or camera CPU) 203, a read-only memory (ROM) 204, a random access memory (RAM) 205, a camera system control unit 206, a communication control unit 207, an analog-to-digital (A / D) conversion unit 208, an image processing unit 209, an encoder unit 210, and a network interface (I / F) 211. The components 203 to 211 included in the surveillance camera 101 are interconnected via a system bus 212.

[0024] The camera optical system 201 includes a zoom lens, a focusing lens, a blur correction lens, an aperture, and a shutter, and is an assembly of optical components that collect optical information about the subject. The camera optical system 201 is connected to the image sensor 202.

[0025] Image sensor 202 is a charge storage type solid-state image sensor, such as a complementary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor, which converts the light flux collected by the imaging optical system 201 into a current value (signal value). Image sensor 202 is an imaging unit configured to acquire color information by combining with color filters, etc. Image sensor 202 is connected to A / D conversion unit 208.

[0026] The camera CPU 203 is a control unit configured to comprehensively control the operation of the surveillance camera 101. The camera CPU 203 reads commands stored in ROM 204 or RAM 205 to perform processing based on the read results.

[0027] The camera system control unit 206 controls the components of the surveillance camera 101 based on instructions from the camera CPU 203. For example, the camera system control unit 206 performs control over the camera optical system 201, such as focus control, shutter control, and aperture adjustment.

[0028] The communication control unit 207 performs control by sending control commands (control signals) from the client device 103 to the components of the surveillance camera 101 to the camera CPU 203 via communication with the client device 103.

[0029] The A / D conversion unit 208 converts the current value received from the image sensor 202 into a digital signal (image data). The A / D conversion unit 208 then sends the digital signal to the image processing unit 209.

[0030] The image processing unit 209 performs image processing on the image data (i.e., the digital signal received from the A / D conversion unit 208). The image processing unit 209 is connected to the encoder unit 210.

[0031] Encoder unit 210 performs processing to convert image data processed by image processing unit 209 into file formats such as Motion JPEG, H.264, or H.265. Encoder unit 210 is connected to network I / F 211.

[0032] Network I / F 211 is used to communicate with external devices such as client device 103 via network 102 and is controlled by communication control unit 207.

[0033] Network 102 is an Internet Protocol (IP) network used to connect surveillance camera 101 and client device 103. Network 102 includes multiple routers, switches, and cables conforming to communication standards such as Ethernet (registered trademark). In this exemplary embodiment, the communication standards, size, and configuration of network 102 are not particularly limited, as long as network 102 enables communication between surveillance camera 101 and client device 103. For example, network 102 can be the Internet, a wired local area network (LAN), a wireless LAN, or a wide area network (WAN).

[0034] <Client device configuration>

[0035] Figure 3 This is a block diagram showing the internal configuration of the client device 103.

[0036] The client device 103 includes a CPU (or client CPU) 301, a main storage device 302, an auxiliary storage device 303, an input I / F 304, an output I / F 305, and a network I / F 306. The components of the client device 103 are communicatively connected to each other via a system bus 307.

[0037] The client CPU 301 provides integrated control over the operation of the client device 103. The client CPU 301 can also control the surveillance camera 101 via the network 102.

[0038] The main storage device 302 serves as a temporary data storage area, such as RAM, for the client CPU 301. For example, the main storage device 302 pre-stores patterns (patterns corresponding to facial and human features) for pattern matching used when the client device 103 performs face detection or human detection.

[0039] Auxiliary storage device 303 is, for example, a hard disk drive (HDD), ROM, or solid-state drive (SSD), and stores various programs and settings data. Auxiliary storage device 303 stores a list of authentication identifiers (IDs) used to maintain past authentication history information. Auxiliary storage device 303 also stores a database (facial authentication database) that associates pre-registered facial images with person information.

[0040] The client device 103 uses the input I / F 304 when it receives input (signals) from the input device 104, etc.

[0041] The output I / F 305 is used when the client device 103 outputs information (signals) to the display device 105, etc.

[0042] Network I / F 306 is used to communicate with external devices such as surveillance camera 101 via network 102.

[0043] Figure 4 The functions and processing of the client device 103 shown are implemented by the client CPU 301 executing processing based on programs stored in the auxiliary storage device 303. (Refer to...) Figure 4 Describe its details.

[0044] <Functions of the Client Device>

[0045] Figure 4 This illustrates the functions performed by the client device 103. In other words, Figure 4 The components (function blocks) shown can be executed by the client CPU 301, and these components are synonymous with the client CPU 301.

[0046] like Figure 4As shown, the client CPU 301 of the client device 103 includes an input information acquisition unit (input signal acquisition unit) 401, a communication control unit 402, an input image acquisition unit 403, a camera information acquisition unit 404, and a detection method setting unit 405. The client CPU 301 also includes a subject detection unit 406, an exposure determination unit 407, a display control unit 408, and a face authentication processing unit 409. The client device 103 can execute different hardware (or software) than the client CPU 301. Figure 4 The functions of units 401 to 409 shown.

[0047] The input information acquisition unit 401 receives input from the user through the input device 104.

[0048] The communication control unit 402 performs control to receive images (images captured by the surveillance camera 101) transmitted from the surveillance camera 101 via the network 102. Furthermore, the communication control unit 402 performs control to send control commands from the client device 103 to the surveillance camera 101 via the network 102.

[0049] The input image acquisition unit 403 acquires the image received from the surveillance camera 101 via the communication control unit 402, and uses it as the target image (the image to which the subject detection processing is applied) for subject detection processing. The details of the detection processing will be described below.

[0050] The camera information acquisition unit 404 acquires camera information (video information) to be used when the monitoring camera 101 captures images of the subject via the communication control unit 402. Camera information (video information) includes various information used when capturing and acquiring images of the subject. Camera information includes exposure parameters, such as aperture value.

[0051] The detection method setting unit 405 sets a predetermined detection method (appropriate detection method) for the image acquired by the input image acquisition unit 403 from various detection methods, including face region detection (face detection) and human body region detection (human body detection). In face detection, the subject detection unit 406 prioritizes detecting the face region in the image. In human body detection, the subject detection unit 406 prioritizes detecting the human body region in the image. The face region is the region that includes the face, and the human body region is the region that includes the human body.

[0052] In this exemplary embodiment, the detection method setting unit 405 sets (selects) face detection or human body detection as the detection method. This exemplary embodiment is not limited to this setting. For example, the detection method setting unit 405 may set (select) a detection method for detecting, for example, the upper body, head, or partial facial regions such as eyes, nose, or mouth, of a person. Furthermore, in this exemplary embodiment, the subject to be detected is a person, but specific areas related to a predetermined subject other than a person can be detected. For example, predetermined subjects preset in the client device 103, such as an animal's face or a car (vehicle), can be detected.

[0053] The exposure determination unit 407 determines the exposure level based on image information about the subject area obtained from the subject detection unit 406, and the communication control unit 402 sends the determined exposure level to the monitoring camera 101. The monitoring camera 101 then performs exposure control via the communication control unit 207. (Refer to the following...) Figure 5 The flowchart describes the detailed processing related to the camera information acquisition unit 404, the detection method setting unit 405, the subject detection unit 406, and the exposure determination unit 407.

[0054] The display control unit 408, in response to instructions from the client CPU 301, outputs the captured image, reflecting the exposure correction determined by the exposure determination unit 407, to the display device 105.

[0055] The face authentication processing unit 409, in response to an instruction from the client CPU 301, performs face authentication processing on the captured image reflecting the exposure correction determined by the exposure determination unit 407. In the face authentication processing, the captured image is compared with a database of pre-registered facial images and person information, and the person information with the highest facial image similarity exceeding a predetermined value is output. Feature information matching or a neural network can be used to calculate the similarity. In this exemplary embodiment, the subject to be authenticated is a human face, but the subject to be authenticated is not limited to this.

[0056] For example, the face of an animal or the license plate of a car can be authenticated by a pre-defined subject preset in the client device 103. Furthermore, the face authentication processing unit 409 is not necessarily included in the client device 103. The face authentication processing unit 409 may be included in a device externally connected to the client device via network 102.

[0057] Subject detection and exposure determination processing

[0058] Reference Figure 5 The flowchart in the document describes the exposure control process according to this exemplary embodiment. Figure 1In the illustrated camera control system 100, it is assumed that the surveillance camera 101, client device 103, input device 104, and display device 105 are turned on, and a connection (communication) is established between the surveillance camera 101 and the client device 103. In this state, it is also assumed that images of the subject are repeatedly captured by the surveillance camera 101, image data is transmitted from the surveillance camera 101 to the client device 103, and images are displayed on the display device 105 at predetermined update cycles. It is also assumed that in response to the input of captured images of the subject from the surveillance camera 101 to the client device 103 via network 102, the client CPU 301 starts... Figure 5 The processing in the flowchart.

[0059] In step S501, the detection method setting unit 405 sets face detection in the subject detection unit 406, and the subject detection unit 406 performs face detection processing on the input image.

[0060] In step S502, the detection method setting unit 405 sets human body detection in the subject detection unit 406, and the subject detection unit 406 performs human body detection processing on the input image.

[0061] The main storage device 302 of the client device 103 pre-stores patterns corresponding to facial features and patterns corresponding to human body features. The subject detection unit 406 detects facial regions based on the patterns through pattern matching.

[0062] Typically, when detecting facial areas, faces can be detected with high precision, and the facial area and other areas of the subject can be clearly distinguished. However, if conditions such as the orientation, size, and brightness of the face are unsuitable for facial detection, the facial area cannot be detected accurately. Conversely, when detecting human bodies, the area where the person is present can be detected regardless of the orientation, size, and brightness of the face. In the human body detection according to this exemplary embodiment, it is not necessary to detect the entire body; the upper body, the body above the chest, or the head area (e.g., the area above the shoulders) can be detected.

[0063] Figure 7A An example of facial region detection is shown. Figure 7B An example of detection of human body regions is shown.

[0064] Figure 7B The human body region in this context refers to a portion of the upper body (i.e., the body above the chest). The upper body includes the face. The human body region is not limited to... Figure 7BThe area shown. For example, the human body area can be the entire body or upper body. In cases where the face in an image is darkened due to backlighting, the human body area can be the head area. The head area is the region where the shape of the head can be identified, and includes, for example, the shoulders, neck, and face. Figure 7A and Figure 7B The illustration shows a scenario where a surveillance camera 101 is installed at the entrance or gate of a building such as an office building or stadium, and the surveillance camera 101 captures images of people entering the building from the outside. The building is not limited to office buildings and stadiums; for example, it could be a parking lot. In the case of a parking lot, the subject is, for example, a vehicle entering the parking lot. When the vehicle is the subject, the area to be detected is, for example, the vehicle body area (including the vehicle body and the surrounding area) or the license plate area (including the vehicle's license plate and the surrounding area).

[0065] When using pattern matching as a subject detection method, the pattern (classifier) ​​generated using statistical learning can be used as the pattern for pattern matching. Alternatively, methods other than pattern matching can be used to detect the subject. For example, the subject can be detected by using the brightness gradient in a local region. That is, the subject detection method is not limited to a specific detection method and can employ various methods such as machine learning-based detection methods and distance-based detection methods.

[0066] In step S503, the facial authentication processing unit 409 determines whether a face or a human body is detected.

[0067] If a face or human body is detected ("Yes" in step S503), the process proceeds to step S504. If no face or human body is detected ("No" in step S503), the process ends.

[0068] In step S504, the facial authentication processing unit 409 performs facial authentication processing. Figure 6 This example illustrates facial authentication processing as a subroutine. Now, refer to... Figure 6 This describes the facial authentication process. Facial authentication is performed on all faces and human bodies detected by the subject detection unit 406.

[0069] like Figure 6 As shown, in step S601, the face authentication processing unit 409 acquires a face image, a face ID, and a body ID from the subject detection unit 406. If no face is detected, the face authentication processing unit 409 acquires neither a face image nor a face ID. The face ID is a value generated based on facial features; each person has one face ID. The same applies to the body ID. That is, the body ID is a value generated based on the characteristics of the human body; each person has one body ID.

[0070] In step S602, the facial authentication processing unit 409 determines whether the facial ID or human ID has not been authenticated in the past. This is determined by referring to a list of authentication IDs that maintains past authentication history information. In other words, in step S602, the facial authentication processing unit 409 determines whether the detected subject has been previously authenticated. If the facial authentication processing unit 409 determines that the detected subject has not been authenticated in the past ("Yes" in step S602), the process proceeds to step S603. If the facial authentication processing unit 409 determines that the detected subject has been authenticated in the past ("No" in step S603), the process proceeds to step S604.

[0071] In step S603, the facial authentication processing unit 409 performs facial authentication processing. In this process, feature information is extracted from the facial image, and attribute information is output as the authentication result. This attribute information is associated with the feature information that has the highest similarity to the extracted feature information in a pre-registered database that links feature information and attribute information such as names. If the similarity does not reach a predetermined threshold, authentication is deemed unsuccessful. Furthermore, authentication is also deemed unsuccessful if no facial image is acquired, i.e., if no face is detected.

[0072] In step S604, the face authentication processing unit 409 notifies the exposure determination unit 407 (by sending a signal indicating that the face or human body has been authenticated) that the face or human body detected by the subject detection unit 406 has been authenticated. Then the processing ends.

[0073] In step S605, the facial authentication processing unit 409 determines whether the facial authentication in step S603 was successful. If the facial authentication processing unit 409 determines that the facial authentication was successful ("Yes" in step S605), the process proceeds to step S606. If the facial authentication processing unit 409 determines that the facial authentication was unsuccessful ("No" in step S605), the process ends.

[0074] In step S606, the face authentication processing unit 409 adds the face ID and the corresponding human body ID to the authentication ID list indicating that the face has been authenticated. Then the processing ends.

[0075] The above describes the subroutines related to facial authentication processing. Now return. Figure 5 The exposure control process will be described again.

[0076] In step S505, a region of interest is set to be used in step S506. Of the subject's face (or multiple faces) and body (or multiple bodies) that did not receive an authentication notification in step S504, at least one is set as a region of interest. If multiple faces or bodies exist and a face or body was set as a region of interest in the processing of the previous frame of the captured image, that face or body is preferentially set as a region of interest. Otherwise, the region of interest is determined according to the application purpose, for example, based on the detection order, distance from the surveillance camera 101, and position in the image.

[0077] The following description uses the case where surveillance camera 101 is installed near the entrance of a building to perform walkthrough authentication as an example. In walkthrough authentication, people closer to surveillance camera 101 pass through it faster. Therefore, when using distance from surveillance camera 101, it is desirable to prioritize exposure control processing for people closer to surveillance camera 101. In other words, when controlling exposure for multiple subjects, the order of exposure control for multiple subjects can be determined based on the distance between surveillance camera (video equipment) 101 and each subject. When using position in the image, if surveillance camera 101 is installed looking down at a person, it is assumed that a person located at the bottom of the image is closer to surveillance camera 101. Therefore, it is desirable to give this person higher priority. In other words, when controlling exposure for multiple subjects, the order of exposure control for multiple subjects can be determined based on the position of each subject in the image.

[0078] Furthermore, when the face and body of the same subject have already been detected, it is desirable to prioritize the facial area.

[0079] In step S506, the exposure determination unit 407 calculates the average brightness value I of the area of ​​interest set in step S505. object More specifically, information about the location of the region of interest and information about the size of the region of interest are applied to the following expression (1).

[0080]

[0081] In expression (1), I(x, y) represents the brightness value at the two-dimensional coordinate position (x, y) in the horizontal (x-axis direction) and vertical (y-axis direction) directions of the image. Additionally, (v, h) represents the center coordinates of the region of interest, k represents the size of the region of interest detected in the horizontal direction, and l represents the size of the region of interest detected in the vertical direction. The left side of equation (1) represents the average brightness value I of the region of interest. object .

[0082] In step S507, the exposure determination unit 407 determines the average brightness value I calculated in step S506. object Determine the exposure correction value EV correction First, the exposure determination unit 407 calculates the average brightness value I of the area of ​​interest calculated in step S506 using expression (2). object Target brightness value I of the region of interest object target The difference ΔDiff between them. For example, the target brightness value I of the region of interest. object target It can be set arbitrarily by the user, or it can be set to a value that improves accuracy by taking into account the authentication accuracy in facial recognition.

[0083]

[0084] Next, the exposure determination unit 407 determines the exposure correction value EV using expression (3). correction Current exposure value (EV) current It is an EV value converted from the subject brightness value (BV) according to the Additive System of Photographic Exposure (APEX), and is set based on a program diagram related to exposure control and pre-stored in the client device 103.

[0085]

[0086] In expression (3), parameter β is the value at the current exposure value EV. current This parameter β is a coefficient that affects the degree (speed) of exposure correction when adjusting for underexposure or overexposure. Setting a larger value for β results in faster (shorter) processing time to reach the target exposure. However, in cases of misjudgment in the detection results or unstable subject detection, the overall brightness of the image can change drastically. Conversely, setting a smaller value for β results in slower (longer) processing time to reach the target exposure, but this process is more robust to error detection and shooting conditions. When the difference ΔDiff falls outside the set threshold Th, the parameter β is set to the current exposure value EV. current The exposure correction value.

[0087] In step S508, the exposure determination unit 407 communicates with the monitoring camera 101 through the communication control unit 402, and sets the exposure correction value EV calculated in step S507 in the monitoring camera 101. correction Then the process ends.

[0088] As described above, in this exemplary embodiment, faces and bodies whose facial authentication has been completed by the facial authentication processing unit 409 are determined not to be exposure control targets, and exposure control is performed on faces and bodies whose facial authentication processing has not been completed. Therefore, even when multiple people are present in the image, the target individuals for exposure control are sequentially reduced based on the facial authentication results, thereby enabling excellent (effective) exposure control for facial authentication. Furthermore, each facial authentication result is managed in association not only with the face ID but also with the body ID (identification information). Therefore, even when a face cannot be detected due to changes in exposure or facial orientation, repeated exposure control for people who have completed facial authentication can be avoided.

[0089] The following will refer to Figure 8 A second exemplary embodiment of the present invention is described. In the first exemplary embodiment, in cases where the person subject to exposure control is not registered in the facial authentication database, exposure control is not applied to another person until the person under exposure control passes through the surveillance camera 101. Thus, depending on the installation conditions or purpose of the surveillance camera 101, exposure control to others is delayed, which affects facial authentication. In this exemplary embodiment, it is determined whether the target person (subject) subject to exposure control is not registered in the database. If it is determined that the target person is not registered in the database, exposure control to the target person is interrupted (exposure control based on exposure correction values ​​will not be performed), and exposure control is transferred to exposure control of another person.

[0090] The same reference numerals are used to denote components and processes similar to those in the first exemplary embodiment, and will not be described again.

[0091] The processing in steps S501 to S504 and S506 to S508 Figure 3 The processing is the same as in the first exemplary embodiment. After step S504, the processing proceeds to step S801.

[0092] In step S801, the exposure determination unit 407 sets the area of ​​interest to be used in steps S506 and S802. In step S801, unlike step S505 according to the first exemplary embodiment, an unregistered personnel list (described below) is referenced, and subjects with IDs present in the unregistered personnel list are determined not to be the target for setting the area of ​​interest.

[0093] Furthermore, when the same subject's face and body have already been detected, the facial area is prioritized as the area of ​​interest.

[0094] In step S802, the exposure determination unit 407 determines whether the face ID corresponding to the area of ​​interest set (selected) in step S801 is the same as the face ID of the previous frame. If the face ID is the same as the face ID of the previous frame ("Yes" in step S802), the process proceeds to step S803. If the face ID is different from the face ID of the previous frame ("No" in step S802), the process proceeds to step S506. If the face ID does not exist, that is, if the face area is not set as an area of ​​interest, the process also proceeds to step S506.

[0095] In step S803, the exposure determination unit 407 determines whether the exposure control performed for the previous frame in step S508 has reached the target value (exposure control has been completed). More specifically, based on the exposure correction value EV in step S507... correction Is it equal to the current exposure value EV? current The system determines whether at least one of the operations for the aperture of the camera optical system 201 and the shutter speed and gain of the image sensor 202 has been completed. If the exposure determination unit 407 determines that the exposure control has been completed ("Yes" in step S803), the process proceeds to step S804. If the exposure determination unit 407 determines that the exposure control has not been completed ("No" in step S803), the process proceeds to step S506.

[0096] In step S804, the exposure determination unit 407 determines whether the face ID corresponding to the area of ​​interest set in step S801 has been selected continuously for the latest predetermined number of times or more. The predetermined number of times is set as the interval from the completion of exposure control to the completion of face authentication. That is, even if exposure control is completed, if face authentication has not been completed for the predetermined number of times (if authentication is not completed within the predetermined time), the person is determined to be not registered in the face authentication database. If the exposure determination unit 407 determines that the face ID has been selected continuously for the predetermined number of times or more (step S804 is "Yes"), the process proceeds to step S805. If the exposure determination unit 407 determines that the face ID has not been selected continuously for the predetermined number of times or more (step S804 is "No"), the process proceeds to step S506.

[0097] In step S805, the exposure determination unit 407 adds (registers) the face ID and the corresponding human body ID to the unregistered people list, which correspond to the area of ​​interest set in step S801. The unregistered people list is stored, for example, in an auxiliary storage device 303. That is, the auxiliary storage device 303 registers information (face ID and human body ID) about subjects that cannot be authenticated within a predetermined time.

[0098] In step S806, the exposure determination unit 407 determines whether another face or another human body exists in the captured image. If the exposure determination unit 407 determines that another face or another human body exists in the captured image (step S806 is "Yes"), the process returns to step S801. If the exposure determination unit 407 determines that another face or another human body does not exist in the captured image (step S806 is "No"), the process ends.

[0099] In this exemplary embodiment, the exposure determination unit 407 determines whether a person is unregistered in step S804 based on whether the same facial ID has been selected more than a predetermined number of times. Alternatively, other methods can be used for determination. For example, facial authentication similarity of the corresponding facial ID can be received from the facial authentication processing unit 409, and determination can be made based on whether the low facial authentication similarity value persists. Alternatively, if facial authentication is not completed even if facial orientation is detected and the face is substantially facing forward, the person can be determined to be unregistered.

[0100] If a person is identified as an unregistered person (in the case where the subject is added to the unregistered person list), the exposure determination unit 407 can send the facial image to the facial authentication processing unit 409 as a registration image. This is because exposure control suitable for the person's face is performed during determination, and the facial image is suitable as a registration image. Furthermore, when the facial orientation is detected and the face is substantially facing forward, a more suitable facial image can be transmitted.

[0101] In the first and second exemplary embodiments described above, the surveillance camera 101 is described as an example of a camera device, but the exemplary embodiments of the present invention can be applied to any camera device other than a surveillance camera.

[0102] Other embodiments

[0103] The embodiments of the present invention can also be implemented by providing software (programs) that perform the functions of the above embodiments to a system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessor unit (MPU) of the system or device reads out and executes the program.

[0104] While the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the claims should be interpreted as broadly as possible to encompass all such variations and equivalent structures and functions.

Claims

1. A control device that controls an imaging device, the control device comprising: a determination unit configured to determine whether each of subjects in an image captured by the imaging device is an authenticated subject; an authentication unit configured to authenticate a subject that is not an authenticated subject in the image based on feature information extracted from a face image of the subject determined by the determination unit; and a control unit configured to control exposure of the image targeting a subject that the authentication unit has successfully authenticated, and not to control exposure of the image targeting a subject that the determination unit has determined to be an authenticated subject and a subject that the authentication unit has not successfully authenticated, wherein, in a case where the determination unit determines that a subject that is not an authenticated subject is successfully authenticated by the authentication unit, the control unit controls exposure of the image based on luminance of the subject and adds a face ID and a body ID to an authenticated ID list; in a case where the determination unit determines that a subject is an authenticated subject, the control unit notifies an exposure unit that the subject has been authenticated.

2. The control device according to claim 1, further comprising a detection unit configured to detect the subjects from the image captured by the imaging device.

3. The control device according to claim 1, wherein the determination unit determines whether the authentication unit completes authentication of the subject within a predetermined time, and wherein, in a case where the determination unit determines that the authentication unit does not complete authentication of the subject within the predetermined time, the control unit does not control exposure of the image targeting the subject.

4. The control device according to claim 3, further comprising a registration unit configured to register information about the subject in a case where the authentication unit does not complete authentication of the subject within the predetermined time, wherein the authentication unit authenticates the subject based on a determination of whether the subject is a subject registered by the registration unit.

5. The control device of claim 1, wherein, in a case where the determination unit determines that a plurality of subjects that are not authenticated subjects are successfully authenticated by the authentication unit, the control unit determines an order in which the plurality of subjects are controlled in exposure by the control unit based on distances between each of the plurality of subjects and the imaging device.

6. The control device of claim 1, wherein, in a case where the determination unit determines that a plurality of subjects that are not authenticated subjects are successfully authenticated by the authentication unit, the control unit determines an order in which the plurality of subjects are controlled in exposure by the control unit based on positions of each of the plurality of subjects in the image.

7. The control device according to any one of claims 1 to 6, wherein, the subjects are people, and wherein the control unit controls exposure of the image targeting a region in the image that includes at least a part of a body of a person.

8. The control device according to claim 7, wherein the control unit controls exposure of the image targeting a region in the image that includes one of a full body, an upper body, a body above a chest, and a head of a person.

9. The control device according to any one of claims 1 to 6, wherein, the subject is a vehicle, and wherein the control unit controls exposure of the image by targeting a region of the image including a vehicle body or a region of the image including a vehicle license plate.

10. A control method for an imaging device, the control method comprising: determining whether each subject in an image captured by the imaging device is an authenticated subject; authenticating a subject that is not an authenticated subject in the image based on feature information extracted from a face image of the subject that is determined to be not an authenticated subject; and controlling exposure of the image targeting a subject that is authenticated successfully, and not targeting a subject that is determined to be an authenticated subject and a subject that is not authenticated successfully, wherein in a case where a subject that is determined to be not an authenticated subject is authenticated successfully by the authentication unit, exposure of the image is controlled based on luminance of the subject, and a face ID and a body ID are added to an authenticated authentication ID list; in a case where a subject is determined to be an authenticated subject, the exposure unit is notified that the subject has been authenticated.

11. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method for controlling an imaging device, the method further comprising: determining whether each subject in an image captured by the imaging device is an authenticated subject; authenticating a subject that is not an authenticated subject in the image based on feature information extracted from a face image of the subject that is determined to be not an authenticated subject; and controlling exposure of the image targeting a subject that is authenticated successfully, and not targeting a subject that is determined to be an authenticated subject and a subject that is not authenticated successfully, wherein in a case where a subject that is determined to be not an authenticated subject is authenticated successfully by the authentication unit, exposure of the image is controlled based on luminance of the subject, and a face ID and a body ID are added to an authenticated authentication ID list; in a case where a subject is determined to be an authenticated subject, the exposure unit is notified that the subject has been authenticated.

Images