Information processing device, authentication system, control method, program, and storage medium

Abstract

To authenticate a subject in a shorter time even when there are multiple subjects in an image.SOLUTION: An information processing device has: an acquisition part 401 for acquiring images; a detection part 402 for detecting a subject area from the image acquired by the acquisition part 401; a determination part 405 for determining the subject area targeted by image adjustment from multiple subject areas when the detection part 402 detects the multiple subject areas from the image; an identification part 404 for identifying the subject area satisfying a prescribed condition from the multiple subject areas detected by the detection part 402 during the image adjustment; an authentication part 406 for authenticating the image in the subject area satisfying the prescribed condition identified by the identification part 404; and an output part 410 for outputting an authentication result by the authentication part 406.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an authentication system, a control method, a program, and a storage medium. [Background technology]

[0002] Patent Document 1 discloses a technique for switching between face detection and face identification, and Patent Document 2 discloses a technique in which an authentication system has means for adjusting exposure and lighting. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2008-270896 [Patent Document 2] Patent Publication No. 2007-94535 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to perform authentication of subjects in a shorter time even when there are multiple subjects in a captured image. [Means for solving the problem]

[0005] In order to solve the above problem, an information processing device according to one aspect of the present invention includes an acquisition unit that acquires an image, a detection unit that detects a subject region from the image acquired by the acquisition unit, a determination unit that, when the detection unit detects multiple subject regions from the image, determines a subject region to be subjected to image adjustment from the multiple subject regions, an identification unit that, during image adjustment, identifies a subject region that satisfies a predetermined condition from the multiple subject regions detected by the detection unit, an authentication unit that authenticates an image of the subject region that satisfies the predetermined condition identified by the identification unit, and an output unit that outputs an authentication result by the authentication unit. The determining unit determines, from among the plurality of subject regions, a subject region in which a difference between an average value of luminance of the image of the subject region and an average value of the image acquired by the acquiring unit is the largest, as the subject region to be subjected to the image adjustment. do. [Effects of the Invention]

[0006] According to the present invention, even when there are multiple subjects in an image, the subjects can be authenticated in a shorter time. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration according to a first embodiment. [Figure 2] FIG. 1 is a diagram showing an example of the device configuration of an imaging device according to a first embodiment. [Figure 3] FIG. 1 is a diagram showing an example of the device configuration of an information processing device according to a first embodiment. [Figure 4] FIG. 1 is a diagram showing an example of the functional configuration of an information processing apparatus according to a first embodiment. [Figure 5] 5 is a flowchart showing an example of the operation of the information processing device according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of authentication by the information processing apparatus according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is not limited to the following embodiments. Furthermore, a configuration may be made by appropriately combining parts of each embodiment described below.

[0009] <Embodiment 1> (System Configuration) 1 is a diagram showing an example of the configuration of an imaging system including an imaging device according to this embodiment. The imaging system 100 includes an imaging device 101, a network 102, an information processing device 103, a display device 104 (display unit), and an input device (input unit) 105.

[0010] The imaging device 101 can communicate with the information processing device 103 via a network 102. The imaging device 101 captures an image of a subject to generate an image, and transmits the captured image to the information processing device 103 via the network 102. A display device 104 and an input device 105 are connected to the information processing device 103, and the image sent from the imaging device 101 is displayed on the display device 104. The input device 105 is a keyboard, a mouse, or the like, and is used to operate the information processing device 103. Note that operations on the information processing device 103 include setting imaging conditions for the imaging device 101 via the network 102 and operating PTZ (Pan Tilt Zoom).

[0011] In this embodiment, the information processing device 103, the display device 104, and the input device 105 are separate entities, but the information processing device 103, the display device 104, and the input device 105 may be integrated into one device, as in a notebook PC with a touch panel display. Also, the imaging device 101 and the information processing device 103 do not need to be connected via the network 102, and may be directly connected to one another. Furthermore, the imaging device 101, the information processing device 103, the display device 104, and the input device 105 may all be integrated into one device, as in a consumer camera with a touch panel display.

[0012] (Device configuration) 2 is a diagram showing an example of the configuration of an imaging device according to this embodiment. The imaging device 101 includes an imaging unit 201, an encoder unit 202, a network I / F 203, a CPU (Central Processing Unit) 204, a RAM (Random Access Memory) 205, and a ROM (Read Only Memory) 206. The imaging optical system 200 is detachably provided in the imaging device 101.

[0013] The imaging optical system 200 is a lens that focuses light from a subject onto an imaging surface of an imaging element 201a (described later), and is composed of, for example, a zoom lens, a focus lens, and a blur correction lens.

[0014] In this embodiment, the imaging device 101 and the imaging optical system 200 are provided as separate, detachable bodies, but the imaging device 101 may have the imaging optical system 200, as in a lens-integrated camera.

[0015] The imaging unit 201 captures an image of a subject using the imaging optical system 200 and generates an image. The imaging unit 201 includes an imaging element 201a, an amplifier 201b, and an image processing unit 201c.

[0016] The image sensor 201a converts light from a subject, which is focused on an imaging surface by the imaging optical system 200, into an electrical signal for each pixel and outputs the signal. The image sensor 201a is an IC chip in which pixels, each made of a photoelectric conversion element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor, are arranged in a matrix. The image sensor 201a is sensitive mainly to visible light, with each pixel having high sensitivity to red (R), green (G), or blue (B), but also has some sensitivity to infrared light. This allows it to clearly capture an image of a subject that is bright with infrared light, such as during times of day when sunlight is present or in a place illuminated by infrared lighting.

[0017] The amplifier 201b amplifies the electrical signal output from the image sensor 201a and outputs the amplified signal.

[0018] The image processing unit 201c performs A / D conversion on the analog electrical signal output from the amplifier 201b to a digital signal, and performs image processing including demosaicing, white balance processing, gamma processing, etc. to generate a digital image.

[0019] The encoder unit 202 encodes the image data output from the imaging unit 201 (image processing unit 201c) into a predetermined file format such as Motion JPEG, H264, or H265.

[0020] The network I / F 203 transmits the image data that has been encoded by the encoder unit 202 to the information processing device 103 via the network 102. The encoded image data may be stored in an internal storage device such as a RAM 205 or a ROM 206 (described later) or in a removable storage medium (not shown) such as an SD card. The image data may also be stored in these storage devices after being output by the image processing unit 201c. In this case, the image data is saved as RAW data before encoding.

[0021] The CPU 204 is a central processing unit that controls the image capturing apparatus 101 .

[0022] The RAM 205 temporarily stores computer programs executed by the CPU 204. The RAM 205 also provides a work area used when the CPU 204 executes processing. The RAM 205 also functions as a frame memory and a buffer memory.

[0023] The ROM 206 stores a program for the CPU 204 to control the image capturing apparatus 101 and the like.

[0024] 3 is a diagram showing an example of the configuration of an information processing device 103 connected to the imaging device according to this embodiment via the Internet. The information processing device 103 is an information processing device having a CPU 301, a RAM 302, a ROM 303, an input I / F 304, an output I / F 305, and a network I / F 406.

[0025] The CPU 301 is a central processing unit that controls the information processing device 103 .

[0026] The RAM 302 stores programs and the like for the CPU 301 to control the information processing device 103 .

[0027] The ROM 303 stores a program for the CPU 301 to control the information processing device 103 and the like.

[0028] The input I / F 304 is an interface that is connected to the input device 105 and that accepts operations for the information processing device 103 input by the user via the input device 105 .

[0029] The output I / F 306 is an interface that is connected to the display device 104 to display the image output from the imaging device 101 on the display device 104 .

[0030] The network I / F 306 is an interface that connects to the image capturing apparatus 101 via the network 102 , and is used to input operation information for the image capturing apparatus 101 and to receive images output from the image capturing apparatus 101 .

[0031] (Functional configuration) FIG. 4 is a diagram illustrating an example of the functional configuration of information processing according to this embodiment. For the software-implemented functions of the functional blocks illustrated in FIG. 4, a program for providing the function of each functional block is stored in a memory such as the ROM 303. The program is then loaded into the RAM 302 and executed by the CPU 301, thereby implementing the functions. For the hardware-implemented functions, a dedicated circuit may be automatically generated on the FPGA from a program for implementing the function of each functional block, for example, using a predetermined compiler. FPGA stands for Field Programmable Gate Array. Alternatively, a gate array circuit may be formed in a similar manner to an FPGA, and implemented as hardware. Alternatively, the functions may be implemented using an ASIC (Application Specific Integrated Circuit). Note that the functional block configuration illustrated in FIG. 4 is merely an example; multiple functional blocks may constitute a single functional block, or any functional block may be divided into blocks performing multiple functions.

[0032] The information processing device 103 includes an acquisition unit 401 , a detection unit 402 , an extraction unit 403 , an identification unit 404 , a determination unit 405 , an authentication unit 406 , a storage unit 407 , a setting unit 408 , a creation unit 409 , an output unit 410 , and a management unit 411 .

[0033] The acquisition unit 401 acquires an image captured by the imaging device 101 .

[0034] The detection unit 402 detects a subject area from the image acquired by the acquisition unit 401. In this embodiment, a case will be described in which the subject area is a face area, but the subject area may be an area of ​​an entire person, or an area of ​​an animal or an object.

[0035] The extraction unit 403 extracts feature amounts of the subject region detected by the detection unit 402. In this embodiment, feature amounts relating to facial features such as the orientation and size of the face are extracted.

[0036] The identification unit 404 identifies a subject area to be authenticated by the authentication unit 406, which will be described later. More specifically, it identifies a subject area that satisfies a predetermined condition from among the subject areas detected by the detection unit 402. In this embodiment, the predetermined condition is whether the luminance of the subject area is within a predetermined luminance range. The predetermined condition may also be based on the amount of noise or contrast value of the subject area.

[0037] The determination unit 405 determines a subject area to be subjected to image adjustment from the subject areas detected by the detection unit 402. In particular, when the detection unit 402 detects multiple subject areas, the determination unit 405 determines a subject area to be subjected to image adjustment. The subject area to be subjected to image adjustment determined by the determination unit 405 is output to the image capture device 101. The image capture device 101 performs image adjustment based on image information of the subject area determined by the determination unit 405. Image adjustment is, for example, control of aperture and exposure time. Other image adjustments include post-processing such as white balance correction and NR (Noise Reduction) processing. Furthermore, image adjustment is not limited to adjustment by the image capture device 101, but may also include image adjustment by the information processing device 103.

[0038] The authentication unit 406 authenticates the image of the subject region identified by the identification unit 404. It also authenticates the image of the subject region in the adjusted image determined by the determination unit 405. The image is authenticated by comparing the feature amounts of the subject region stored in the storage unit 407 (described later) with the feature amounts of the subject region extracted by the extraction unit 403, and determining whether a subject similar to the feature amounts of the subject stored in the storage unit 407 has been detected.

[0039] The storage unit 407 stores the feature amounts of the image of the subject region. In this embodiment, it stores the feature amounts of the face. It is a database that stores, as necessary, feature amounts stored in advance by a user or a designer, feature amounts of the image of the subject region authenticated by the authentication unit 406, and the like, and is referenced when the authentication unit 406 executes the authentication process.

[0040] The setting unit 408 performs settings related to the authentication notification content according to the authentication result by the authentication unit 408 and the authentication notification of the image capture device 101. The setting unit 408 registers the authentication notification settings in the management unit 411, which will be described later.

[0041] The creating unit 409 creates authentication notification information for notifying the authentication result by the authenticating unit 406 based on the authentication notification setting set by the setting unit 408 .

[0042] The output unit 410 outputs the authentication notification information and the authentication result itself created by the creation unit 409. The output authentication result is output to the display device 104 or the like via the output I / F 305, and can be used by notifying the user of the authentication result, for example.

[0043] The management unit 411 comprehensively manages information such as the feature amount of the image of the subject area stored in the storage unit 407, authentication notification settings, and authentication results.

[0044] (Operation description) 5 is a flowchart showing an example of the operation of the information processing device according to this embodiment. The operation of this flowchart starts when the CPU 301 in the information processing device 103 reads a program stored in a storage medium such as the RAM 302 or the ROM 303.

[0045] In step S501, the acquisition unit 401 acquires an image, which may be an image captured by the imaging device 101 or an image stored in an external storage device.

[0046] In step S502, the detection unit 402 detects a subject area from the image acquired by the acquisition unit 401. If a subject area is not detected, it is determined that no subject exists in the image and the process returns to step S501. If a subject area is detected, the process proceeds to step S503.

[0047] In step S503, it is determined whether the image acquired in step S501 is an image being adjusted. That is, it is determined whether the image capturing device 101 was adjusting the image when the acquired image was captured. In this embodiment, it is determined whether the image capturing device 101 was performing exposure control. If the image was captured during exposure control, the process proceeds to step S504; if not, the process proceeds to step S506.

[0048] In step S504, the identification unit 404 identifies a subject area that satisfies a predetermined condition. Of the subject areas detected in step S502, a subject area that satisfies the predetermined condition is identified. The predetermined condition may be the range of brightness of the image of the subject area, the amount of noise, the contrast value, etc. For example, a subject area is identified where the average brightness of the image in the subject area falls within a predetermined range. Here, it is desirable that the predetermined range is a brightness that is neither too bright nor too dark so that authentication can be performed by the authentication unit 406. Note that the range can be changed by the user or designer.

[0049] In step S505, the authentication unit 406 authenticates the subject region identified by the identification unit 404. More specifically, the extraction unit 403 extracts the feature amount of the subject region and compares it with the feature amount of the subject region stored in the storage unit 407 to calculate the similarity. If the similarity is equal to or greater than a predetermined value, it is determined that the same subject as the subject stored in the storage unit 407 has been detected. Furthermore, in step S505, the output unit 410 outputs the authentication result by the authentication unit 406.

[0050] In step S506, it is determined whether there is a subject area to be subjected to image adjustment determined in step S508 (described later). Because it is determined in step S503 that the image is not being adjusted, it is determined that the image acquired by acquisition unit 401 has either been adjusted or that a subject area to be subjected to image adjustment has not been determined in step S508 (described later). If it is determined that there is a subject area to be subjected to image adjustment, it is determined that image adjustment in that subject area has been completed, and the process proceeds to step S507. On the other hand, if it is determined that there is no subject area to be subjected to image adjustment, the process proceeds to step S508.

[0051] In step S507, the image of the subject region for which image adjustment has been completed is authenticated by authentication unit 406. The authentication method and the method of outputting the authentication result are the same as in step S505, and therefore will not be described here.

[0052] In step S508, the determination unit 405 determines a subject area to be subjected to image adjustment. If the detection unit 402 detects only one subject area in step S502, that subject area is determined as the subject area to be subjected to image adjustment. If the detection unit 402 detects multiple subject areas, one subject area is determined as the subject area to be subjected to image adjustment as a priority. Methods for determining a subject area to be subjected to image adjustment will be described below. The first method is a method for determining a subject area based on a user instruction. Images of multiple subject areas detected by the detection unit 402 are displayed on the display device 104, and the user selects a subject area to be subjected to image adjustment via the input device 105. An operation input to the input device 105 receives information about the selected subject area via the input I / F 305. The determination unit 405 determines the subject area selected by the user as the subject area to be subjected to image adjustment. The second method is a method for determining a subject area based on image feature amounts of each subject area. In step S508, the extraction unit 403 extracts image feature amounts of each subject area. The extracted features include, for example, the size and orientation of the face. Therefore, the subject area with the largest face size or the subject area with the face facing forward is determined as the subject area to be adjusted. The third method is to determine the subject area based on the size of each subject area. Of the multiple detected subject areas, the subject area with the largest subject area is determined as the subject area to be adjusted. The fourth method is to determine the subject area based on the position of each subject area and the motion vector of the image. For example, if a subject area is located near the edge of the image, it is likely to fall outside the angle of view, so it is prioritized as the subject area to be adjusted. Also, even if it is near the edge of the image, if a subject area is moving toward the center of the image, the latter is more likely to fall outside the angle of view than a subject area moving toward the edge of the image, so the latter is prioritized as the subject area to be adjusted. In this way, the subject area to be adjusted is determined based on at least one of the position of each subject area and the motion vector of the image. The fifth method is to determine the subject area based on the luminance of each subject area.For example, the average luminance value for each subject area is calculated, and the subject area with the largest difference from the average luminance value of the entire image is determined as the subject area for image adjustment, or the subject area with the largest or smallest average luminance value is determined as the subject area for image adjustment. In other words, the subject area with the lowest visibility is determined as the subject area for image adjustment. Information about the subject area determined by the determination unit 404 is sent to the imaging device 101 via the network 102. The imaging device 101 performs image adjustment (exposure control) based on the subject area information received from the information processing device 103 (information about the subject area that is the subject of image adjustment) to improve the visibility of the target subject area.

[0053] In step S509, it is determined whether or not to continue the authentication operation. For example, if the user performs an operation to stop the authentication operation on the input device 105, the authentication operation is terminated. If no such operation has been performed, the process returns to step S501.

[0054] 6A, 6B, and 6C are diagrams showing an example of the authentication operation of the information processing device according to this embodiment. The images (a), (b), and (c) are arranged in chronological order, and show images acquired by the acquisition unit 401. For simplicity of explanation, only faces are shown in the images.

[0055] Fig. 6(a) shows the image before image adjustment, and the dashed line indicates the authenticated subject area. Since Fig. 6(a) is not the image being adjusted, the process proceeds to step S506. Since there is no subject area to be subject to image adjustment in step S506, the subject area to be subject to image adjustment is determined in S508. Here, it is assumed that the subject area 601 with the lowest average brightness is determined to be the subject to image adjustment. The border line indicates that the subject area is the subject to image adjustment.

[0056] 6(b) shows an image during image adjustment. Therefore, in step S504, subject areas that satisfy predetermined conditions are recognized. That is, images of subject areas whose average brightness is within a predetermined range are recognized. Here, subject areas 602 to 604 are recognized.

[0057] 6(c) shows the image after image adjustment is complete. Because a subject to be adjusted is present in step S506, the subject area 601 is authenticated. Once the subject area to be adjusted is authenticated, it is removed from the subject areas to be adjusted, and the subject area to be adjusted is re-determined from the detected subject areas.

[0058] According to the information processing device of this embodiment, even when there are multiple subjects in an image, it is possible to perform subject authentication in a shorter time.

[0059] <Other embodiments> The present invention can be realized by a process of reading and executing a program that realizes one or more functions of the above-described first embodiment. This program is supplied to a system or device via a network or a storage medium, and is read and executed by one or more processors in the computer of the system or device. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0060] 401 Acquisition Department 402 Detection unit 403 Extraction part 404 Specific part 405 Decision Section 406 Authentication Section 407 Storage section 408 Settings 409 Creation Department 410 Output section 411 Management Department

Claims

1. an acquisition unit that acquires an image; a detection unit that detects a subject area from the image acquired by the acquisition unit; a determination unit that, when the detection unit detects a plurality of subject regions from the image, determines a subject region to be subjected to image adjustment from the plurality of subject regions; an identification unit that identifies a subject area that satisfies a predetermined condition from the plurality of subject areas detected by the detection unit during adjustment of the image; an authentication unit that authenticates an image of the subject area that satisfies the predetermined condition identified by the identification unit; an output unit that outputs an authentication result by the authentication unit; and The information processing device is characterized in that the determination unit determines, among the multiple subject areas, the subject area in which the difference between the average brightness of the image of the subject area and the average brightness of the image acquired by the acquisition unit is the largest as the subject area to be subjected to the image adjustment.

2. An acquisition unit that acquires an image; a detection unit that detects a subject area from the image acquired by the acquisition unit; a determination unit that, when the detection unit detects a plurality of subject regions from the image, determines a subject region to be subjected to image adjustment from the plurality of subject regions; an identification unit that identifies a subject area that satisfies a predetermined condition from the plurality of subject areas detected by the detection unit during adjustment of the image; an authentication unit that authenticates an image of the subject area that satisfies the predetermined condition identified by the identification unit; an output unit that outputs an authentication result by the authentication unit; and The information processing device is characterized in that the determination unit determines, from among the plurality of subject regions, a subject region having a minimum or maximum average value of image luminance as the subject region to be subjected to the image adjustment.

3. 3 . The information processing apparatus according to claim 1 , wherein the determination unit determines the subject area to be subjected to the image adjustment based on the luminance of the image in each of the plurality of subject areas.

4. an extraction unit that extracts a feature amount of an image of the subject area detected by the detection unit; The information processing apparatus according to claim 1 , wherein the determining unit determines the subject region to be subjected to the image adjustment based on the feature amount extracted from the plurality of subject regions by the extracting unit.

5. 5. The information processing apparatus according to claim 1, wherein the predetermined condition is that an average value of luminance of the image of the subject region is within a predetermined range.

6. 6. The information processing apparatus according to claim 1, wherein the image adjustment includes exposure control.

7. An authentication system comprising the information processing device according to claim 1 and an imaging device.

8. an acquisition step of acquiring an image; a detection step of detecting a subject area from the image acquired in the acquisition step; a determining step of determining, when a plurality of subject regions are detected from the image in the detecting step, a subject region to be subjected to image adjustment from the plurality of subject regions; a specifying step of specifying a subject area that satisfies a predetermined condition from the plurality of subject areas detected in the detecting step during the adjustment of the image; an authentication step of authenticating an image of the subject area that satisfies the predetermined condition identified in the identification step; an output step of outputting the authentication result in the authentication step; and A control method for an information processing device, characterized in that in the determination process, the subject area among the multiple subject areas in which the difference between the average brightness of the image of the subject area and the average brightness of the image acquired in the acquisition process is the largest is determined to be the subject area to be subjected to the image adjustment.

9. An acquisition step of acquiring an image; a detection step of detecting a subject area from the image acquired in the acquisition step; a determining step of determining, when a plurality of subject regions are detected from the image in the detecting step, a subject region to be subjected to image adjustment from the plurality of subject regions; a specifying step of specifying a subject area that satisfies a predetermined condition from the plurality of subject areas detected in the detecting step during the adjustment of the image; an authentication step of authenticating an image of the subject area that satisfies the predetermined condition identified in the identification step; an output step of outputting the authentication result in the authentication step; and A control method for an information processing device, characterized in that in the determination step, the subject area among the plurality of subject areas whose average value of image brightness of the subject area is the smallest or largest is determined to be the subject area to be subjected to the image adjustment.

10. 10. The method of claim 8, wherein the determining step determines the subject area to be subjected to the image adjustment based on the luminance of the image in each of the plurality of subject areas.

11. The method further includes an extraction step of extracting a feature amount of the image of the subject area detected in the detection step, 11. The control method for an information processing device according to claim 8, wherein the determination step determines a subject area to be subjected to the image adjustment based on the feature values ​​extracted from the plurality of subject areas in the extraction step.

12. 12. The method of controlling an information processing apparatus according to claim 8, wherein the predetermined condition is that an average value of luminance of the image of the subject region is within a predetermined range.

13. 13. The method of controlling an information processing apparatus according to claim 8, wherein the image adjustment includes exposure control.

14. A program for causing a computer to execute the method for controlling an information processing device according to any one of claims 8 to 13.

15. A computer-readable storage medium storing the program according to claim 14.

Images