Information processing device, authentication system, authentication method, program, and storage medium

Abstract

To provide an information processor, an authentication system, an authentication method, a program, and a storage medium that suppress output of erroneous authentication due to an image being adjusted.SOLUTION: In an authentication system, at least one imaging device, an image recording server, an image analysis server, and a terminal device are communicably connected via a network. Therein, an image analysis server 104, which is an information processor, comprises: an acquisition unit 301 that acquires a first image imaged by the imaging device; a detection unit 302 that detects a subject from the first image; an authentication unit 308 that, when the detection unit 302 detects the subject, authenticates the subject on the basis of information on the first image and information on a second image stored in a storage unit; a determination unit 303 that determines whether the first image is an image being adjusted; and an output unit 310 that, when the determination unit 303 determines that the first image is the image being adjusted, does not output an authentication result of the authentication unit 308, or otherwise, when it determines that the first image is not the image being adjusted, outputs the authentication result.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an authentication system, an authentication method, a program, and a storage medium. [Background technology]

[0002] A technology has been disclosed in the past that switches between face detection and face identification depending on the photographing function (Patent Document 1). Also, a technology has been disclosed in which an authentication system has a means for adjusting exposure (Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2008-270896 [Patent Document 2] Patent Publication No. 2007-94535 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to suppress the output of erroneous authentication due to an image that is being adjusted. [Means for solving the problem]

[0005] In order to solve the above problem, an information processing device according to one aspect of the present invention is an information processing device capable of communicating with an imaging device, and includes: an acquisition unit that acquires a first image captured by the imaging device; a detection unit that detects a subject from the first image; an authentication unit that, when the detection unit detects the subject, authenticates the subject based on information related to the first image and information related to a second image stored in a storage unit; a determination unit that determines whether the first image is an image under adjustment; and an output unit that, when the determination unit determines that the first image is an image under adjustment, does not output an authentication result from the authentication unit, and, when the determination unit determines that the first image is not an image under adjustment, outputs the authentication result. The image processing device further includes a storage unit, wherein the storage unit stores an authentication result of the authentication unit when the determination unit determines that the first image is an image being adjusted, and the output unit outputs the authentication result stored in the storage unit when the determination unit determines that the first image is an image being adjusted and one or more subjects are out of frame of the first image.do. [Effects of the Invention]

[0006] According to the present invention, it is possible to suppress the output of erroneous authentication due to an image that is being adjusted. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an authentication system according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing the hardware configuration of each device according to the first embodiment. [Figure 3] FIG. 1 is a diagram showing an example of the functional configuration of an information processing apparatus according to a first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of the functional configuration of an operation terminal according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing an example of the image adjustment operation of the authentication system according to the first embodiment. [Figure 6] 5 is a flowchart showing an example of the operation of the information processing device according to the first embodiment. [Figure 7] 5A and 5B are diagrams showing an example of how an authentication result is output according to an image adjustment state by the information processing apparatus according to the first embodiment. [Figure 8] 10 is a flowchart showing an example of the operation of the information processing device according to the second embodiment. [Figure 9] 10A and 10B are diagrams showing an example of how an authentication result is output in accordance with an image adjustment state by an information processing apparatus according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is not limited to the following embodiments. Furthermore, a configuration may be made by appropriately combining parts of each embodiment described below.

[0009] <Embodiment 1> (System Configuration) FIG. 1 is a block diagram showing an example of the configuration of a face authentication system according to this embodiment.

[0010] The authentication system 100 has at least one imaging device 102 and information processing devices 103, 104, and 105. The information processing device 103 can be configured as a recording server, the information processing device 104 as an image analysis server, and the information processing device 105 as an operation terminal. These information processing devices may be configured as physically independent devices or may be configured as an integrated device.

[0011] The imaging device 102 and information processing devices 103, 104, and 105 are communicably connected via a network 101. The network 101 is, for example, a local area network (LAN) or a wide area network (WAN). The physical connection to the network 101 may be wired or wireless. Furthermore, in FIG. 1, one or two of the imaging device 102 and the information processing devices 103, 104, and 105 are connected to the network 101, but the number of connected devices is not limited to the number shown in FIG. 1.

[0012] The imaging device 102 captures an image of a subject existing in a space to be recognized at a predetermined angle of view. The imaging device 102 has a function of transmitting the captured image (also simply referred to as an image) to information processing devices 103, 104, and 105 via a network 101.

[0013] The image recording server 103 receives, saves, and accumulates images transmitted from the imaging device 102 via the network 101. In addition, the recording server 103 transmits the captured images it has saved to each device in response to requests received from the image analysis server 104 and the operation terminal 105.

[0014] The image analysis server 104 receives an image of a subject to be detected from a captured image from the operation terminal 105 via the network, generates feature amounts necessary for subject face detection processing, and stores the feature amounts as feature information used to detect the subject's face. In this embodiment, the subject is described as a person.

[0015] When the image analysis server 104 detects a face to be detected in a captured image, it generates authentication notification information and transmits the generated authentication notification information to the operation terminal 105. The authentication notification information may be transmitted based on a request from the operation terminal 105, or, if the operation terminal 105 is configured to receive detection results in real time, the image analysis server 104 may transmit the analysis results to the operation terminal 105 each time the analysis process is completed. In the following description of this embodiment, it is assumed that the image analysis server 104 receives images captured by the network camera 102 via the image recording server 103, but various processes can be performed in the same way even when the images are received directly from the network camera 102. Furthermore, in the following description, it is assumed that the image analysis server 104 holds the analysis results and registration information, but the information may also be stored in another database via a network or in the operation terminal 105.

[0016] The operation terminal 105 has a display device (display) and has a display control function for playing back and displaying captured images received from the camera 102 and image data recorded in the image recording server 103, and for displaying authentication notification information received from the image analysis server 104. The operation terminal 105 also has a user interface and input unit for face detection, and has a function for operating to set parameters related to image analysis processing for face detection, set registration information, and set various parameters for authentication notification information.

[0017] (Device configuration) The device configuration of the imaging device according to this embodiment will be described. Fig. 2 is a diagram showing an example of the hardware configuration of the imaging device 102. The imaging device 102 can be configured to include a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an external memory 204, an imaging unit 205, an input unit 206, a communication I / F 207, and a system bus 208. The imaging device 102 may further include other components.

[0018] The CPU 201 is a control unit that performs overall control of the operation of the imaging device 102, and controls the operation of each component (202 to 207) via a system bus 208. The ROM 202 is a non-volatile memory that stores control programs and the like required for the CPU 201 to execute processing. Note that the programs may be stored in the external memory 204 or a removable storage medium (not shown). The RAM 203 functions as the main memory, work area, and the like of the CPU 201. That is, when executing processing, the CPU 201 loads the necessary programs and the like from the ROM 202 into the RAM 203, and executes the programs and the like to realize various functional operations.

[0019] The external memory 204 stores, for example, various data and various information required when the CPU 201 performs processing using a program. The external memory 204 also stores, for example, various data and various information obtained when the CPU 201 performs processing using a program. The imaging unit 205 captures an image of a face and generates a captured image.

[0020] The imaging unit 205 can be configured to include an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device), an A / D converter, an image processing unit, and the like.

[0021] The input unit 206 is composed of a power button, a setting button, etc., and an operator of the imaging device 102 can give instructions to the imaging device 102 via the input unit 206. The communication I / F 207 is an interface for communicating with an external device (here, the information processing device 103). The communication I / F 207 is, for example, a LAN interface. The system bus 208 communicatively connects the CPU 201, ROM 202, RAM 203, external memory 204, imaging unit 205, input unit 206, and communication I / F 207. The functions of each unit of the imaging device 102 are realized when the CPU 201 executes a program stored in the ROM 202 or external memory 204.

[0022] In this embodiment, the hardware configuration of each of the information processing devices 103 to 105 may be similar to the hardware configuration shown in FIG. 2. However, in that case, the information processing device may be provided with a hardware configuration such as a display unit. The information processing device may be an information processing device such as a tablet or smartphone that has both an imaging unit 205 and a display unit. Alternatively, the information processing device may be an information processing device such as a PC that has a display unit instead of the imaging unit 205. Here, the display unit is configured as a monitor such as a liquid crystal display (LCD). Furthermore, the information processing devices 103 to 105 are provided with a pointing device such as a keyboard or a mouse as the input unit 206, and are configured so that an operator can give instructions to the information processing devices 103 to 105. Furthermore, the external memory 204 may include a database for storing various information.

[0023] The functional configuration of an information processing device according to this embodiment will be described. Fig. 3 is a block diagram showing an example of the functional configuration of an image analysis server (information processing device). The image analysis server 104 includes an acquisition unit 301, a detection unit 302, a determination unit 303, an adjustment unit 304, a feature acquisition unit 305, a subject information registration unit 306, a storage unit 307, an authentication unit 308, an authentication result information management unit 309, an output unit 310, a creation unit 311, and a notification control unit 312.

[0024] Each functional component shown in Fig. 3 can be realized by the CPU 201 of the image analysis server 104 executing a corresponding processing program. However, at least some of the components shown in Fig. 3 may be configured to be realized by dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 201, but some of the functions may be installed in other devices. For example, some of the functions may be installed in the imaging device 102, or in other information processing devices including the image recording server 103. Furthermore, in this embodiment, a case will be described in which the image analysis server 104 mainly detects a human face, but the subject to be detected is not limited to a human face.

[0025] The acquisition unit 301 receives captured images transmitted from the imaging device 102 via the image recording server 103, expands and decodes the images, and acquires images (moving images or still images) (first images). The acquisition unit 301 sequentially sends the acquired images to the detection unit 302. In this embodiment, the captured images supplied from the imaging device 102 may be still images or moving images. The source of the captured images is not particularly limited. The source may be the imaging device 102, the image recording server 103, or another imaging device that supplies captured images via a wired or wireless connection. The acquisition unit 301 may also acquire captured images from a memory within the same device (e.g., the external memory 204). In the following description, a case will be described in which the acquisition unit 301 sends one image to the detection unit 302, regardless of whether the acquisition unit 301 has acquired a moving image or a still image. In the former case, the one image corresponds to each frame constituting the moving image, and in the latter case, the one image corresponds to a still image. In the case of a moving image, the same process may be repeated for each frame. Alternatively, the image capturing device 102 may transmit information indicating whether the first image is being adjusted, and the acquiring unit 301 may acquire the information indicating whether the first image is being adjusted together with the first image.

[0026] The detection unit 302 performs subject detection processing on the captured image received by the acquisition unit 301 using moving object detection, a matching pattern dictionary, etc. As a result, a subject such as a face or a human body is detected in an area of ​​the captured image, and a subject area in the captured image that includes the subject is extracted.

[0027] Here, the subject detection process in this embodiment may be any method that can detect a subject from an image and manage and track an area containing the detected subject using an identifier such as an ID that can identify the area. A method may be used that detects a moving object and manages the area using an ID that identifies the moving object, or a method may be used that detects a human body area using Ω tracking and tracks the human body area. Alternatively, a method may be used that detects a face area rather than a moving object or human body area and tracks the same face. Furthermore, if a human body is detected first during image adjustment, tracking of the human body may be continued even if there are frames in which face detection is temporarily difficult due to tracking the human body and the face is temporarily hidden or due to changes in the image during image adjustment in video shooting, and the same person's face may continue to be identified.

[0028] The detection unit 302 detects a subject for each image and assigns a unique ID (IDentifier) ​​as an identifier for the subject region. If the subject has been detected continuously in past images under specified conditions, the same ID as in the past images is assigned. If the subject has no continuity with past images, a different ID from that in the past images is assigned and the detected subject region is managed. For example, even if a subject is not detected for a certain period of time (number of images), if it is determined to be the same subject as a person in a past image, the same ID as the past image is assigned. As another example, even if the orientation or size of the subject in an image changes and it is difficult to detect the subject region in some images, it is possible to consider it to be the same person by estimating moving body movement, etc., and assign the same ID.

[0029] In the above-described manner, the acquisition unit 301 detects and tracks the subject in the images that it continuously acquires.

[0030] The determination unit 303 determines whether a captured image (first image) captured by the imaging device 102 in the authentication system according to this embodiment is under adjustment. This determination may be made by receiving information indicating whether image adjustment is underway from the imaging device 102, or may be made in response to an instruction from the adjustment unit 304, which will be described later. The adjustment period is defined as the period from when it is determined that an image unsuitable for authentication has been acquired for the area of ​​the subject to be authenticated in the captured image captured by the imaging device 102, to when an image adjustment instruction is issued to the imaging device 102 and it is determined that an image suitable for authentication that does not require image adjustment has been acquired.

[0031] The adjustment unit 304 controls adjustment of the captured image (first image). For example, when the image captured by the image capture device 102 is affected by backlight or other factors, making it impossible to properly identify the subject, the adjustment unit 304 may instruct AE correction. If the subject is a face, the adjustment unit 304 calculates the average luminance value of each pixel in the face area used for authentication. Then, the necessity of image adjustment is determined based on whether the calculated average luminance value is within a predetermined value range that does not interfere with authentication, and the adjustment unit 304 transitions between a normal shooting state and an image adjustment in progress state. If multiple subjects are present in the captured image, the determination can be made based on the average luminance values ​​of the multiple subjects. Although AE correction has been used as an example of the adjustment in progress state in this embodiment, other image adjustment means can be used in the same way as long as they determine and adjust an image suitable for subject authentication. When adjusting the captured image using a function of the image capture device 102, such as AE correction, the adjustment unit 304 instructs the image capture device 102 to adjust the captured image, and the image capture device 102 adjusts the captured image. When the captured image acquired by the imaging device 102 is acquired by the image analysis server 104 and then subjected to various image processing to perform image adjustment, the image adjustment unit 304 can also instruct the acquisition unit 301 to adjust the captured image.

[0032] The feature acquisition unit 305 performs a feature extraction process to extract features (information about the first image) based on a predetermined calculation from the image of the subject (first image) detected by the detection unit 302. In this embodiment, facial features are extracted as the features. The feature of the subject may be any feature that can be used for authenticating the subject.

[0033] The subject information registration unit 306 receives information including a facial image to be authenticated from the operation terminal 105 via the LAN 101, generates features (information about the second image) necessary for face detection processing, and stores the features in the storage unit 307 as facial feature information (hereinafter referred to as "facial feature information"). The feature generation here can be performed in the same manner as the processing performed by the feature acquisition unit 305. Furthermore, the subject information registration unit 306 stores information about the facial pose relative to the direction of gravity based on a user input in the storage unit 307 as facial pose information of the registered facial image (information about the second image). Any information can be used as the facial pose information of the registered facial image as long as it can identify the tendency of facial deformation based on the facial pose in real space. The facial pose can be expressed as a three-dimensional vector, or the facial direction can be specified using terms that indicate general human body orientations, such as standing, supine, facing up, and prone. Alternatively, the desired subject image (second image) can simply be stored in the storage unit 307.

[0034] The storage unit 307 stores the facial feature amounts and image of the registration target and face pose information of the registered image generated by the subject information registration unit 306, together with information such as the registration name and registration time, as facial feature information (information about the second image). The information stored in the storage unit 307 is not limited to facial feature information. It may also be information about the subject's overall body shape or characteristic parts (arms, legs, etc.). Furthermore, as described above, the information may be image data of the subject's image (second image), or a structure that associates them.

[0035] The authentication unit 308 compares the feature amount (information about the first image) acquired by the feature amount acquisition unit 305 with the feature amount (information about the second image) of the face to be authenticated stored and managed by the storage unit 307, and calculates the similarity. If the calculated similarity is equal to or greater than a predetermined threshold, it determines that the captured face resembles the face to be detected, and recognizes that the face (or human body) to be verified has been detected. In other words, it authenticates the face detected from the image against the registered facial feature information. The authentication unit 308 also temporarily stores authentication information for each ID of the detected subject.

[0036] In the case of a fixed camera, the authentication unit 308 performs authentication using a face pose appropriate for authentication set in the camera face pose information and pose information of the registered face image (information about the second image). A three-dimensional vector may be used as the pose relative to the direction of gravity. An authentication method may be selected based on information that allows the pose relative to the direction of gravity to be matched, such as a horizontal orientation or a vertically upward orientation. Information about the pose may also be converted into a three-dimensional vector and used for determination. Furthermore, authentication performed by the authentication unit 308 is not limited to the face. Authentication may also be based on the subject's overall body shape or characteristic body parts (legs, arms, etc.).

[0037] In the case of a camera such as a pan-tilt-zoom camera whose shooting direction can be changed, or a camera with an orientation sensor, the facial orientation relative to the direction of gravity can be calculated from the orientation of the camera when the authentication image was captured, and any facial orientation can be used for authentication judgment. Also, even if the camera's orientation and shooting direction are variable, it can be operated by determining a facial pose suitable for authentication, just like a fixed camera. In this case, authentication judgment is made based on two conditions: the camera is in an orientation suitable for facial recognition, and the facial pose relative to the direction of gravity is suitable for authentication.

[0038] The authentication result information management unit 309 stores and manages authentication result information for subjects. It stores an image of the facial area at the time of authentication in association with the ID of the tracked subject, and stores and manages the authentication result for each frame image. Information saved as an authentication result includes, for example, information identifying the registered subject, such as the ID of the registered person, authentication information, such as the degree of similarity with the facial image of the registered person, the authenticated frame image, and information about the camera. When tracking of the subject is basically completed, the authentication result information is deleted for each tracking target after the necessary processing is completed, but it may also be stored in a permanent manner in a storage device. Alternatively, the authentication result information may be temporarily stored in memory until a desired timing during operation of the software of the authentication system according to this embodiment.

[0039] The output unit 310 outputs the authentication result by the authentication unit 308. The authentication result may be a determination result as to whether or not the information of the detected subject is similar to the information of the subject stored in the storage unit 307, or information of the subject determined to be similar to the information of the detected subject. The authentication result may include information on the degree of similarity in addition to this information, or may include information on the first image and information on the second image. The output authentication result may also include information indicating whether the authentication unit 308 has authenticated the image being adjusted.

[0040] The creation unit 311 creates authentication notification information for notifying that the face of the detection target has been detected. The authentication notification information is created based on the authentication result output from the output unit 310. The authentication notification information may include information such as the camera settings of the imaging device 102 when capturing the first image, and settings related to tracking and authentication. The output unit 310 and the creation unit 311 may be reversed. That is, the creation unit 311 may create the authentication notification information based on the authentication result of the authentication unit 308, and the output unit 310 may output the authentication notification information.

[0041] The notification control unit 312 determines whether or not to notify the user of the notification information created by the creation unit 311, and controls the notification to the user. Whether or not to notify the user of the notification information is controlled based on the authentication result output from the output unit 310 or the notification information created by the creation unit 311.

[0042] FIG. 4 is a block diagram showing an example of the functional configuration of the operation terminal 105 according to this embodiment. The operation terminal 105 includes an authentication notification receiving unit 401, a setting input unit 403, a setting communication unit 404, and an authentication notification unit 405. The setting input unit 403 inputs face-related information, including an image of a face to be detected, and authentication notification information. The setting communication unit 404 transmits the input information to the image analysis server 104 via the LAN 101. The authentication notification receiving unit 401 receives the authentication notification information from the image analysis server 104 via the LAN 101. The image acquisition unit 402 acquires, from the image recording server 103, an image captured at the alarm occurrence time of the received authentication notification information. The authentication notification unit 405 displays an authentication notification display screen on the display, including the received authentication notification information and a captured image related to the authentication notification, to notify the operator of the authentication. A procedure for using an AE correction function in combination as an example of an image adjustment function for a captured image will be described.

[0043] (Operation description) The operation of the imaging system according to this embodiment will be described. Fig. 5 is a flowchart showing an example of the operation of the authentication system according to this embodiment. The operation shown in this flowchart starts when a program stored in a ROM or the like in the imaging device or information processing device is loaded into RAM and the loaded program is read by a CPU.

[0044] In step S501, image adjustment of the imaging device 102 is started. This may be started when the acquisition unit 301 in the image analysis server 104 starts acquiring images from the imaging device 102. Image adjustment may also be started at a timing of the user's choice, such as when an input unit such as the imaging device 102, the recording server 103, the image analysis server 104, or a UI displayed on the operation terminal 105 is operated.

[0045] In step S502, the continuation of the image adjustment function is confirmed, and if an instruction to stop the image adjustment function has not been input, the process proceeds to step S503, and if an instruction to stop the image adjustment function has been input, the operation is terminated.

[0046] In step S503, the acquisition unit 301 acquires an image captured by the image capturing device 102, and the process proceeds to step S504. Here, in this embodiment, the image is acquired for the image adjustment function, but the acquired image may be shared with functions other than the image adjustment function, such as subject recognition and recording, or the acquired captured image may be used so that the implementation procedure of this embodiment is incorporated into the procedure for using images for other functions.

[0047] In S504, the detection unit 302 detects a face or an object correlated with the position of the face in the image acquired by the acquisition unit 301. Then, for example, it checks whether an object, such as a human body or a wheelchair, that is correlated with the face or the position of the face to be authenticated as the authentication target is present in the acquired image. It then determines whether the area of ​​the face image is within a predetermined brightness range. The brightness value can be determined, for example, by the average brightness value of each pixel in the face image area, but any method can be used as long as it can determine the need for AE correction to obtain an image appropriate for face authentication. Here, in the case of an object correlated with a face, the position of the face is estimated based on the shape and posture of the object. For example, in the case of a human body, the position and size of the head are estimated using a method such as skeletal estimation, and even an image in which a face is not clearly detected is subject to AE correction.

[0048] The authentication system according to this embodiment is not limited to methods for detecting only faces. For example, when a photograph is taken under strong backlighting conditions, if a human body is detected but the face is unclear, the face is located near the head of the human body, and the face can be detected by adjusting the image of the photographing device. To ultimately obtain the desired face image for face recognition, it is also possible to track and adjust the image of a non-face object, such as a human body. If a wheelchair can be detected in addition to a human body, it is possible to track a moving object with a regular positional relationship to the face, allowing the approximate position of the head or face to be estimated, and the face image can be adjusted for face recognition. If a face or an object correlated with a face is detected and it is determined that a face that requires image adjustment as a subject for face recognition is present, the process proceeds to step S505.

[0049] In step S505, it is determined whether the image is being adjusted. If the image is not being adjusted, the process proceeds to S506, where the image is transitioned to an image being adjusted state, and then the process proceeds to S507. If the image is already being adjusted, the process proceeds directly to step S507.

[0050] Here, we will explain the image adjustment in progress state. When authenticating a subject, such as a face, image adjustments such as AE correction may be required in severe backlit conditions. Image adjustments, such as AE correction, require repeated correction instructions until the desired image is acquired, but this requires time. In an image captured in the image adjustment in progress state, it may not be possible to obtain a normal image of a subject, such as a face, and some images may result in erroneous subject recognition. When detecting a subject from a captured image, if the subject's image area does not fall within a predetermined value range and image adjustment is required, the image capture device enters the image adjustment in progress state, during which image adjustment is being performed. For example, if an image adjustment instruction is sent to the capture device during video capture, such as by sending an AE correction value to the camera via command, the images of the continuously captured frames will change midway through. If the subject image area falls within the predetermined value range and an image that does not interfere with subject recognition is acquired, the state transitions to the normal capture state. In the normal capture state, no image adjustment instructions are issued, and normal subject recognition functions can be performed. When adjusting an image during video shooting, the effect of the image adjustment instruction is checked and the next correction instruction is issued repeatedly, and this continues until the desired image is obtained.

[0051] In step S507, the adjustment unit 304 instructs the image adjustment to the image capture device 102 in the image adjustment state. In this embodiment, AE correction is instructed using a predetermined correction value. The correction value may be any value, but it is desirable to use a correction value that completes image adjustment as quickly as possible. In this embodiment, AE correction is used as an example, but in image adjustment to acquire a subject area image suitable for subject recognition, any value according to the characteristics of each image adjustment method is used. Once the image adjustment instruction is sent to the image capture device 102 in step S507, the process proceeds to step S502.

[0052] In step S503, if it is determined that there is no face or a subject image correlated with a face in the image, or that image adjustment is not required for the subject as a face recognition target, the process proceeds to step S508.

[0053] In step S508, it is determined whether or not the image adjustment in progress state needs to be ended. If the subject to be face recognition is not present in the acquired image and the image adjustment in progress state is not in progress, the process proceeds to S502, and the next image is acquired. If the image adjustment in progress state is already in progress and the normal shooting state is in progress, the process determines whether or not to end the image adjustment in progress state. For example, if the image adjustment in progress state is to be ended after confirming that the subject to be image adjustment is not present in multiple consecutive frame images, the image adjustment in progress state does not change and the process proceeds to step S502 until it is confirmed that the subject is not present in a predetermined number of frame images. If the image adjustment in progress state is to be ended immediately after one frame image, or if the subject to be image adjustment is not present in a predetermined number of consecutive frame images, the process proceeds to step S509.

[0054] In step S509, the image adjustment state is transitioned to the normal shooting state, the image adjustment process is ended, and the process proceeds to step S502.

[0055] In step S502, if the user has not input an instruction to end image adjustment, the process continues from S503 to S509 while changing the state related to image adjustment if a subject to be subjected to face recognition exists in the acquired image.

[0056] Fig. 6 is a flowchart showing the operation of the information processing device according to this embodiment. The flowchart in Fig. 6 is based on the premise that the situation starts when backlighting becomes strong during shooting, making it difficult to detect people. This flowchart starts operation when a program stored in a ROM or the like is loaded into RAM and executed by the CPU (step S601).

[0057] In step S601, the acquisition unit 301 acquires a captured image (first image) transmitted from the image capture device .

[0058] In step S602, the detection unit 302 detects a subject (for example, a human body or a face) from the first image acquired by the acquisition unit 301. If a subject is detected, the process proceeds to step S603, and if a subject is not detected, the process proceeds to step S605 as the subject to be authenticated is not present within the angle of view of the image capture device 102.

[0059] In step S603, authentication unit 308 authenticates the subject detected by detection unit 302 based on information about the first image and information about the second image. More specifically, information about the second image stored in storage unit 307 is compared with information about the first image. A similarity between the information about the detected subject and the information about the subject stored in storage unit 307 is calculated. If the calculated similarity is equal to or greater than a predetermined threshold, the information about the detected subject is determined to be similar to the information about the subject stored in storage unit 307. The authentication result by authentication unit 308 may be a determination result as to whether the information about the detected subject is similar to the information about the subject stored in storage unit 307, or information about the subject determined to be similar to the information about the detected subject. The authentication result may include similarity information in addition to this information, or may include information about the first image and information about the second image.

[0060] In step S604, the determination unit 303 determines whether the first image acquired by the acquisition unit 301 is an image being adjusted (for example, AE being adjusted). The determination unit 303 makes this determination based on information instructing image adjustment output from the adjustment unit 304, information indicating whether the image is being adjusted received from the imaging device 102, etc. If the first image is being adjusted, the authentication unit 308 has performed authentication on the image being adjusted, so it determines that the authentication result is uncertain, and proceeds to step S606.

[0061] In step S605, the output unit 310 outputs the authentication result obtained by the authentication unit 308. As described above, in step S605, the creation unit 311 may create authentication notification information based on the authentication result obtained by the authentication unit 308, and the output unit 310 may output the created authentication notification information.

[0062] In step S606, it is determined whether or not to continue authenticating the subject. If the authentication of the subject is to be continued, the process returns to step S601 and an image is acquired again. If the authentication of the subject is not to be continued, the operation ends.

[0063] 7 shows an example in which the information processing device (authentication system) according to this embodiment suppresses the output of erroneous authentication. In this example, the authentication unit 308 erroneously recognizes a subject who is actually Mr. A as Mr. B while the image is being adjusted. If the determination unit 303 determines that the image is being adjusted, the authentication unit 308 does not output an authentication result. Therefore, even if the authentication unit 308 recognizes Mr. B with a similarity above a predetermined value, the authentication result is not output, and the output of erroneous authentication can be suppressed. Therefore, the user is not notified of the erroneous authentication, and malfunctions such as opening a gate in response to the result of erroneous authentication at an authentication gate can also be suppressed.

[0064] On the other hand, if the image is not being adjusted, the authentication result is output. In the example of Figure 7, Mr. A is correctly identified in the normal shooting state (when the image is not being adjusted), and the authentication result is also output.

[0065] In this way, the authentication result of the authentication unit 308 for the image being adjusted is not output. Therefore, according to the information processing device (authentication system) according to this embodiment, it is possible to prevent the output of erroneous authentication caused by the image being adjusted.

[0066] <Embodiment 2> In this embodiment, a case where the subject can be recognized only in the image being adjusted in the first embodiment will be described.

[0067] In this embodiment, if the subject can only be authenticated using the image being adjusted, the authentication result cannot be output even if the subject cannot be correctly authenticated using the image being adjusted. According to this embodiment, it is possible to prevent the subject's authentication result from being omitted from being output. Note that the configuration of the authentication system according to this embodiment and the device configuration and functional configuration of the information processing device according to this embodiment are the same as those of embodiment 1, and therefore description thereof will be omitted.

[0068] (Operation description) 8 is a flowchart showing an example of the operation of the information processing device (authentication system) according to this embodiment. Note that the operations in steps S801 to S805 and S809 are the same as those in the first embodiment, and therefore will not be described again.

[0069] If it is determined in step S804 that the image (first image) captured by the image capture device 102 is currently being adjusted, the authentication result by the authentication unit 308 is stored in the storage unit 307 in step S806.

[0070] In step S806, the first image is compared with the image captured in the frame preceding the first image to determine whether one or more subjects have left the frame. If it is determined that one or more subjects have left the frame, the process proceeds to step S808; otherwise, the process proceeds to step S809. Whether one or more subjects have left the frame can also be determined by comparing the authentication result of the first image with the authentication result of the previous frame stored in storage unit 307 and determining the difference between the authenticated subject IDs.

[0071] In step S808, output unit 310 outputs the authentication result of the frame-out subject (or authentication notification information created by creation unit 311). By comparing the authentication result of the first image with the authentication result of the image captured in the frame before the first image, the ID that was not authenticated is identified. This makes it possible to identify which subject is the frame-out subject. Output unit 310 acquires and outputs the authentication result corresponding to the ID of the frame-out subject from the authentication result of the previous frame stored in storage unit 307. Note that the authentication result corresponding to the ID of the subject does not need to be acquired from the authentication result of the previous frame. For example, the authentication result corresponding to the ID of the subject may be searched for among the authentication results stored in storage unit 307, and any authentication result may be acquired and output. In particular, it is desirable to output the authentication result with the highest similarity among the multiple authentication results corresponding to the ID of the subject stored in storage unit 307.

[0072] FIG. 9 shows an example in which the information processing device (authentication system) according to this embodiment prevents omission of the subject's authentication result from being output. Here, it is assumed that the authentication unit 308 authenticated C during image adjustment, and that the authentication was successful. Since the authentication result may be uncertain during image adjustment, the authentication result is accumulated in the storage unit 307. Thereafter, in a normal shooting state (a state in which the image is not being adjusted), C, whose ID is the same as that authenticated during adjustment, may be framed out without being authenticated. In this case, the output unit 310 outputs the authentication result during image adjustment that was accumulated in the storage unit 307. As described above, it is desirable to output the authentication result with the highest similarity among the authentication results stored in the storage unit 307.

[0073] The information processing device (authentication system) according to this embodiment can prevent omissions in output of the subject authentication result. Furthermore, the authentication result output from the output unit 310 may include information indicating whether the authentication unit 308 has authenticated an image during adjustment. Therefore, when the authentication result is notified by the notification control unit 312, the user is notified along with the similarity that authentication was performed during image adjustment, allowing the user to confirm the accuracy of the authentication result. Furthermore, the authentication gate can be configured to open if the similarity is equal to or greater than a predetermined threshold when an authentication result during image adjustment is output.

[0074] <Other embodiments> The present invention can be realized by a process of reading and executing a program that realizes one or more functions of the above-described first embodiment. This program is supplied to a system or device via a network or a storage medium, and is read and executed by one or more processors in the computer of the system or device. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0075] Each functional component shown in Fig. 3 can be realized by the CPU 201 of the image analysis server 104 executing a corresponding processing program. However, at least some of the components shown in Fig. 3 may be configured to be realized by dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 201. [Explanation of symbols]

[0076] 100 Imaging device 103 Image recording server (information processing device) 104 Image analysis server (information processing device) 105 Operation terminal (information processing device) 301 Acquisition Department 302 Detection unit 303 Judgment section 308 Authentication Department 310 Output section

Claims

1. An information processing device capable of communicating with an imaging device, an acquisition unit that acquires a first image captured by the imaging device; a detection unit that detects a subject from the first image; an authentication unit that, when the detection unit detects the subject, authenticates the subject based on information about the first image and information about a second image stored in a storage unit; a determination unit that determines whether the first image is an image being adjusted; an output unit that does not output an authentication result from the authentication unit when the determination unit determines that the first image is an image being adjusted, and that outputs the authentication result when the determination unit determines that the first image is not an image being adjusted; and The storage unit is further included, the storage unit stores an authentication result of the authentication unit when the determination unit determines that the first image is an image being adjusted; The output unit outputs the authentication result stored in the memory unit when the determination unit determines that the first image is an image being adjusted and one or more subjects are out of frame from the first image.

2. 2 . The information processing apparatus according to claim 1 , wherein the authentication unit calculates a similarity between the information about the first image and the information about the second image, and authenticates the subject based on the similarity.

3. The information processing apparatus according to claim 1 , further comprising a notification control unit that notifies a user of an authentication result of the authentication unit.

4. An authentication system having an imaging device and an information processing device, an acquisition unit that acquires a first image captured by the imaging device; a detection unit that detects a subject from the first image; an authentication unit that, when the detection unit detects the subject, authenticates the subject based on information about the first image and information about a second image stored in a storage unit; a determination unit that determines whether the first image is an image being adjusted; an output unit that does not output an authentication result from the authentication unit when the determination unit determines that the first image is an image being adjusted, and that outputs the authentication result when the determination unit determines that the first image is not an image being adjusted; and The storage unit is further included, the storage unit stores an authentication result of the authentication unit when the determination unit determines that the first image is an image being adjusted; An authentication system characterized in that the output unit outputs the authentication result stored in the memory unit when the judgment unit judges that the first image is an image being adjusted and one or more subjects are out of frame from the first image.

5. an acquisition step of acquiring a first image captured by an imaging device; a detecting step of detecting a subject from the first image; an authentication step of authenticating the subject based on information about the first image and information about the second image stored in the storage step when the subject is detected in the detection step; a determining step of determining whether the first image is an image being adjusted; an output step of not outputting an authentication result from the authentication step when it is determined in the determination step that the first image is an image under adjustment, and outputting the authentication result when it is determined in the determination step that the first image is not an image under adjustment; and The method further comprises the storing step, In the storing step, if it is determined in the determining step that the first image is an image being adjusted, an authentication result in the authenticating step is stored; An authentication method characterized in that in the output process, if the determination process determines that the first image is an image being adjusted and one or more subjects are out of frame from the first image, the authentication result accumulated in the storage process is output.

6. 6. The authentication method according to claim 5, wherein the authentication step calculates a similarity between the information about the first image and the information about the second image, and authenticates the subject based on the similarity.

7. 7. The authentication method according to claim 5, further comprising a notification control step of notifying a user of the authentication result of said authentication step.

8. A program for causing a computer to execute the authentication method according to any one of claims 5 to 7.

9. A computer-readable storage medium storing the program according to claim 8.

Images