Authentication device, authentication method, and program

Abstract

An information processing system according to one aspect comprises: acquisition means that acquires a plurality of images obtained by photographing an authentication target; generation means that generates composite information of an iris by synthesizing partial information of the iris of the authentication target appearing in each of the plurality of images; authentication means that performs iris authentication of the authentication target using the composite information; condition determination means that determines whether or not a photographing condition is satisfied; and operation detection means that detects an operation serving as a trigger for the iris authentication of the authentication target. The acquisition means acquires the plurality of images during a period from when the condition determination means determines that the photographing condition is satisfied to when the operation detection means detects the operation serving as the trigger. The generation means generates the composite information when the operation detection means detects the operation serving as the trigger.

Description

Authentication Device, Authentication Method, and Program

[0001] This disclosure relates to an authentication device, an authentication method, and a program.

[0002] Technologies related to iris authentication have been evolving. For example, Patent Document 1 describes that an authentication system captures a plurality of images of an authentication target using a plurality of cameras and processes at least one of the plurality of images for iris authentication.

[0003] Japanese Patent Application Laid-Open No. 2007-504562 [[ID=I0]]

[0004] This disclosure aims to improve the technologies described in the prior art documents.

[0005] An authentication device according to one aspect includes: an acquisition unit that acquires a plurality of images of an authentication target; a generation unit that generates composite information of the iris by synthesizing partial information of the iris of the authentication target reflected in each of the plurality of images; an authentication unit that performs iris authentication of the authentication target using the composite information; a condition determination unit that determines whether or not shooting conditions are satisfied; and an operation detection unit that detects an operation that triggers the iris authentication of the authentication target, wherein the acquisition unit acquires the plurality of images between when the condition determination unit determines that the shooting conditions are satisfied and when the operation detection unit detects the operation that triggers the authentication, and the generation unit generates the composite information when the operation detection unit detects the operation that triggers the authentication.

[0006] An authentication method according to one aspect is an authentication method executed by a computer, which includes: determining whether or not shooting conditions are satisfied; acquiring a plurality of images of an authentication target between when it is determined that the shooting conditions are satisfied and when an operation that triggers the iris authentication of the authentication target is detected; generating composite information of the iris by synthesizing partial information of the iris of the authentication target reflected in each of the plurality of images when the operation that triggers the authentication is detected; and performing iris authentication of the authentication target using the composite information.

[0007] A program according to one embodiment is a program that causes a computer to perform the following actions: determine whether or not the shooting conditions are met; acquire multiple images of the object to be authenticated until it is determined that the shooting conditions are met and an action that triggers iris authentication of the object to be authenticated is detected; generate composite iris information by combining partial information of the iris of the object to be authenticated as seen in each of the multiple images when the trigger action is detected; and perform iris authentication of the object to be authenticated using the composite information.

[0008] This is a block diagram showing an example of an authentication device related to this disclosure. This is a flowchart showing a typical example of an authentication device's processing. This is a block diagram showing an example of a payment system related to this disclosure. This is a floor plan of the store interior. This is a block diagram showing an example of a cash register terminal related to this disclosure. This is a block diagram showing an example of an authentication device related to this disclosure. This is a flowchart showing a typical example of an authentication device's processing. This is a flowchart showing a typical example of an authentication device's processing. This is a block diagram showing another example of a payment system related to this disclosure. This is a floor plan of the store interior. This is a block diagram showing an example of an entrance terminal related to this disclosure. This is a block diagram showing an example of an exit terminal related to this disclosure. This is a block diagram showing an example of an authentication device related to this disclosure. This is a flowchart showing a typical example of an authentication device's processing. This is a block diagram showing an example of an authentication device related to this disclosure. This is a block diagram showing an example of the hardware configuration of an information processing device on which the processing related to this disclosure is performed.

[0009] Embodiments of this disclosure will be described below with reference to the drawings. Note that the following descriptions and drawings in the embodiments have been omitted and simplified as appropriate for clarity of explanation. Furthermore, unless otherwise specified, in this disclosure, when "at least one of the multiple items" is defined for multiple items, that definition may mean any single item, or it may mean any multiple items including all of the items.

[0010] Each drawing referenced in an embodiment is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate.

[0011] This disclosure applies mutatis mutandis to the contents disclosed in International Publication No. 2024 / 095362, filed by the same applicant.

[0012] Embodiment 1 [Description of Configuration] Figure 1 is a block diagram showing an example of an authentication device according to this disclosure. The authentication device 100 comprises a condition determination unit 102, an acquisition unit 104, an operation detection unit 106, a generation unit 108, and an authentication unit 110.

[0013] The condition determination unit 102 determines whether the shooting conditions are met. The shooting conditions are predetermined conditions relating to the object to be authenticated. The shooting conditions are, for example, that the object to be authenticated has performed a predetermined action, or that some kind of authentication has been performed on the object to be authenticated, but are not limited to these. The condition determination unit 102 may determine whether the shooting conditions are met based, for example, on an image captured by a camera, a detection result from a sensor, an operation of an input terminal, or an authentication result from an authentication terminal. This camera, sensor, input terminal, or authentication terminal may be mounted on the authentication device 100, or it may be provided outside the authentication device 100.

[0014] The acquisition unit 104 acquires multiple images of the object to be authenticated. The acquisition unit 104 may acquire multiple images taken by one or more cameras mounted on the authentication device 100 or provided outside the authentication device 100. One or more cameras are provided in a predetermined area, such as a store, and take pictures of the object to be authenticated, such as a person, that is present in the predetermined area.

[0015] The motion detection unit 106 detects an action that triggers iris authentication of the subject to be authenticated. The triggering action may be, for example, an action that indicates the subject intending to purchase a product, or an action that indicates the subject entering an area requiring a security check, but is not limited to these. The motion detection unit 106 may also determine whether the shooting conditions are met based on, for example, an image captured by a camera, a detection result from a sensor, an operation of an input terminal, or an authentication result from an authentication terminal. This camera, sensor, input terminal, or authentication terminal may be mounted on the authentication device 100 or provided outside the authentication device 100.

[0016] The acquisition unit 104 acquires multiple images between the time the condition determination unit 102 determines that the shooting conditions are met and the time the motion detection unit 106 detects a triggering action. The acquisition unit 104 may also acquire multiple images actually captured by the camera during the period from the time it is determined that the shooting conditions are met to the time the triggering action is detected.

[0017] The generation unit 108 generates composite iris information by combining partial information of the iris to be authenticated that is captured in each of the multiple images acquired by the acquisition unit 104. When the motion detection unit 106 detects a triggering motion, the generation unit 108 generates composite information using the multiple images acquired.

[0018] For example, the generation unit 108 may generate composite information by combining partial information of the iris of the person to be authenticated, which has been identified as belonging to the same person. Partial iris information refers to information about a portion of the iris that can be used for iris authentication. The generation unit 108 connects multiple partial iris information to generate composite information that covers a larger portion of the iris area of ​​the person to be authenticated than the partial information. The composite information is generated by combining the partial information. The composite information may be information that represents the entire iris area, or it may be information about the iris that has a sufficient area or proportion of the entire iris area to be used for iris authentication. Partial iris information may be, for example, feature quantities used for iris authentication or information from a captured iris image of a portion of the iris area. Composite information may be, for example, feature quantities used for iris authentication or information from a captured iris image of all or part of the iris area. Information about iris feature quantities may be, for example, information about the iris pattern. The partial iris information and composite information may be information from either the left or right eye of the person to be authenticated, or information from both eyes.

[0019] The authentication unit 110 performs iris authentication of the subject to be authenticated using the composite information generated by the generation unit 108. For example, the authentication unit 110 compares the feature information, which is composite information of the left and right eyes to be authenticated, with the feature information of the left and right eyes to be compared, for each eye. The feature information to be compared may be stored in a database (DB) inside or outside the authentication device 100. If the composite information generated by the generation unit 108 is iris image information, the authentication unit 110 can convert the composite information into feature information and use the converted feature information for iris authentication.

[0020] The authentication unit 110 calculates the similarity between the feature of the item to be authenticated and the feature of the item to be compared for each eye. If there is a feature of the iris of the item to be compared that is more similar than the threshold for both the left and right eyes, the authentication unit 110 determines that iris authentication has been successful. The authentication unit 110 determines that the item to be authenticated is the item to be compared. At this time, the authentication unit 110 may acquire and use personal information of the item to be compared that is linked to the item to be compared (for example, information indicating personal attributes such as name, age, gender, etc., or payment information such as credit card information). On the other hand, if there is no iris feature of the item to be compared that is more similar than the threshold for either the left or right eye, the authentication unit 110 determines that iris authentication has failed.

[0021] [Flowchart Description] Figure 2 is a flowchart showing a typical example of the authentication device 100's processing, and this flowchart provides an overview of the authentication device 100's processing. Details of each processing step are as described above, so explanations will be omitted as appropriate.

[0022] First, the condition determination unit 102 determines whether or not the shooting conditions are met (step S12). If the condition determination unit 102 determines that the shooting conditions are not met (No in step S12), the determination in step S12 is performed again. On the other hand, if the condition determination unit 102 determines that the shooting conditions are met (Yes in step S12), the acquisition unit 104 acquires multiple images of the object to be authenticated (step S14).

[0023] The motion detection unit 106 determines whether or not it has detected an action that triggers iris authentication for the subject to be authenticated (step S16). If the motion detection unit 106 determines that it has not detected a trigger action (No in step S16), the processing from step S14 onwards is executed again. On the other hand, if the motion detection unit 106 determines that it has detected a trigger action (Yes in step S16), the generation unit 108 generates composite iris information by combining partial iris information from multiple images (step S18). The authentication unit 110 performs iris authentication for the subject to be authenticated using the composite information (step S20).

[0024] [Explanation of Effects] When an authentication device actually performs iris authentication, the timing of acquiring the image used for iris authentication becomes an issue. If it is difficult to acquire the image at the appropriate time, the captured image may not be suitable for iris authentication, and the authentication device may not be able to perform iris authentication.

[0025] However, as described above, the authentication device 100 acquires multiple images of the object to be authenticated when the shooting conditions are met. Furthermore, when the authentication device 100 detects an action that triggers iris authentication, it generates iris synthesis information and performs iris authentication of the object to be authenticated. In other words, the authentication device 100 performs iris authentication when necessary, using images taken under conditions where iris authentication is considered likely to be performed. Therefore, the authentication device 100 can perform iris authentication more reliably. However, it should be noted that this effect is only one of several effects that the multiple embodiments disclosed herein aim to achieve.

[0026] Furthermore, the authentication device 100 generates composite iris information used for authentication by combining partial iris information from each of multiple images. Therefore, compared to a case where the authentication device generates a single image suitable for authentication when the person to be authenticated remains still facing the camera for a predetermined time and then acquires iris information from that image, the person to be authenticated can acquire iris information unconsciously. This reduces the effort required of the person to be authenticated when taking images. In addition, even if the person to be authenticated does not notice the camera, does not make eye contact with the camera, or does not know how to perform iris authentication, the authentication device 100 can acquire the iris information of the person to be authenticated by combining partial iris information from each of multiple images, thereby enabling iris authentication.

[0027] The authentication device 100 may be configured as a single computer device (information processing device), or as a distributed system having multiple computer devices. In a distributed system, the processing performed by the authentication device 100 can be divided and executed by multiple computer devices. In other words, the authentication device 100 may be realized by distributing each part of the authentication device 100 across two or more computer devices, and enabling communication between the two or more computer devices.

[0028] The authentication device 100 may have some or all of its components installed on a cloud server built on the cloud, or on another type of virtualized server generated using virtualization technology, etc. Functions other than those installed on servers such as cloud servers or virtualized servers are located at the edge.

[0029] Each embodiment described below discloses a specific example of the authentication device 100 described in Embodiment 1. The following describes an example in which the authentication device 100 is applied to authentication for product purchase in a store. However, the specific examples of the authentication device 100 shown in Embodiment 1 are not limited to those described below. For example, the authentication device 100 may be used for person authentication in facilities requiring security, such as event venues, airports, and research laboratories, or for other person authentication applications. Furthermore, the configurations and processes described below are illustrative and not limiting.

[0030] Embodiment 2 [Description of Configuration] Figure 3 is a block diagram showing an example of a payment system according to this disclosure. The payment system S1 comprises cash register terminals 200A and 200B, an authentication device 300, and an iris recognition DB 1. Hereinafter, cash register terminals 200A and 200B may be collectively referred to as cash register terminal 200. The cash register terminals 200, the authentication device 300, and the iris recognition DB 1 are connected to each other. The payment system S1 is applied to a system in which customers make payments for their purchases themselves in a store, a so-called "self-checkout" system, as shown below.

[0031] Figure 4 is a floor plan of the interior of a store where a cash register terminal 200 is installed. The store T1 shown in Figure 4 has a roughly rectangular floor. A customer entrance E1 is provided on one side of the floor, and product display shelves H1, H2, and H3 are provided on the remaining three sides. A product display shelf H4 is also provided in the center of the floor. A counter U1 is provided near display shelf H3, and the cash register terminal 200 is placed on counter U1. Inside store T1, there are customers P1 and P2 looking at products, and customer P3 near the cash register terminal 200A. Customer P3 is about to make a payment for a purchase using the cash register terminal 200A. On the opposite side of counter U1 from customer P3, there is staff member T1 to assist customers as needed.

[0032] Figure 5 is a block diagram showing an example of a cash register terminal according to this disclosure. The cash register terminal 200 comprises a shooting unit 202, an input display unit 204, a product reading unit 206, a lighting unit 208, an audio output unit 210, a control unit 212, and a transmitting / receiving unit 214. The cash register terminal 200 is a self-checkout payment terminal that allows customers to pay for their purchases themselves.

[0033] The imaging unit 202 has a camera that captures multiple images of customers operating the cash register terminal 200 by photographing the vicinity of the cash register terminal 200. The imaging unit 202 captures images of customers at predetermined intervals, for example. The imaging unit 202 includes a lens, an image sensor, etc., as hardware components. The imaging unit 202 may have a function to control one or more of the following: magnification, resolution, frame rate, quantization bits, bitrate, and camera angle. The video data may also include time information at the time of capture. In this example, the imaging unit 202 has a near-infrared camera and captures near-infrared images. However, the imaging unit 202 may capture non-visible light images (e.g., infrared images) that are not near-infrared images. Alternatively, the imaging unit 202 may capture visible light images. In other words, the imaging unit 202 may have a non-visible light camera or a visible light camera that is not a near-infrared camera. If the imaging unit 202 is a visible light camera, the camera may be a low-resolution camera. The installation location of the imaging unit 202 is not particularly limited, but it may be installed in a location that makes it easy to photograph the front of the customer's face as they operate the cash register terminal 200, for example, near the input display unit 204 of the cash register terminal 200. The imaging unit 202 may also be installed separately from the cash register terminal 200.

[0034] The input display unit 204 has a touch panel used by the customer for payment. The input display unit 204 first displays a payment start button for the customer to initiate payment. Pressing this payment start button corresponds to an action that is determined to satisfy the shooting conditions. After the payment start button is pressed by the customer, the input display unit 204 displays product information read by the product reading unit 206. Product information includes, for example, the product name and its price. The input display unit 204 displays the read product information each time a new product is read by the product reading unit 206.

[0035] Furthermore, the input display unit 204 displays a payment confirmation button along with the read product information. The payment confirmation button is a button for the customer to input their intention to pay for all products read by the product reading unit 206. Pressing this payment confirmation button corresponds to the action that triggers iris authentication for the products being authenticated. When the payment confirmation button is pressed by the customer and the payment confirmation instruction is received, the authentication device 300 performs iris authentication as described later.

[0036] The input and display unit 204 may be configured as separate input and display units. For example, the payment start button and payment confirmation button may be provided as physical input buttons on the cash register terminal 200.

[0037] The product reading unit 206 acquires product information by reading product identification information attached to the product. Product identification information is attached to the product, for example, as a barcode or an IC (Integrated Circuit) tag. The product reading unit 206 reads the product identification information, for example, by having a barcode scanner or an RFID (registered trademark) reader. For example, a customer may move the product in front of the product reading unit 206 to allow the product reading unit 206 to read the product identification information of that product.

[0038] The lighting unit 208 notifies staff member T1 whether or not payment has been made at the cash register terminal 200 by changing the on / off state or the pattern of the lighting. Changing the pattern of the lighting includes changing at least the interval between turning on or off, or changing the color of the light emitted.

[0039] Furthermore, the illumination unit 208 may be provided near the imaging unit 202. As described later, the authentication device 300 generates composite iris information using partial iris information captured by the imaging unit 202. However, if iris authentication using the generated composite information is not possible in the authentication device 300, the imaging unit 202 needs to capture an image for iris authentication again. In that case, the illumination unit 208 can make the presence of the imaging unit 202 known to the customer by turning on and flashing its lights. When the customer, having recognized the presence of the imaging unit 202, faces the imaging unit 202, the imaging unit 202 can capture an image suitable for iris authentication.

[0040] The audio output unit 210 includes a speaker. The audio output unit 210 assists the customer in making a payment by outputting audio information to the customer. For example, when the customer presses the payment start button, the audio output unit 210 outputs the audio information, "Payment for product purchase will now begin." For example, when the customer presses the payment confirmation button, the audio output unit 210 outputs the audio information, "Payment confirmed. Proceeding to authentication procedure."

[0041] The control unit 212 controls the start and stop of the above-mentioned processing in the imaging unit 202 to the audio output unit 210. The control unit 212 can control the start and stop of the above-mentioned processing based on a control signal transmitted from, for example, the authentication device 300.

[0042] The transmitting / receiving unit 214 receives control signals transmitted from the authentication device 300. The transmitting / receiving unit 214 also transmits operation signals indicating customer operations performed on the cash register terminal 200, product information read by the product reading unit 206, and images captured by the imaging unit 202 to the authentication device 300.

[0043] In addition, when the imaging unit 202 needs to capture an iris authentication image again (hereinafter, also simply referred to as the case of re-capture), the control unit 212 may cause the input display unit 204 to output a display for guiding the customer to recognize the presence of the imaging unit 202 and face the front of the imaging unit 202. The display information is, for example, information such as "Please face the front of the camera". The control unit 212 may output voice information indicating the same content to the voice output unit 210.

[0044] Further, the control unit 212 may determine whether it is necessary to change the position or orientation of the customer's face or body so as to obtain a more preferable iris authentication image by analyzing the iris of the customer reflected in the image captured by the imaging unit 202. When it is necessary to change the position or orientation of the customer's face or body, the control unit 212 may display information indicating how much change is required via the input display unit 204. The control unit 212 may cause the voice output unit 210 to output voice information indicating the same content.

[0045] In addition, when re-capture is necessary in the imaging unit 202, the control unit 212 may control the lighting or lighting mode of the lighting unit 208 to make the customer recognize the presence of the imaging unit 202 and turn the customer's face in the direction of the imaging unit 202.

[0046] FIG. 6 is a block diagram showing an example of the authentication device according to this disclosure. The authentication device 300 includes a condition determination unit 302, an image acquisition unit 304, an operation detection unit 306, a product information acquisition unit 308, a generation unit 310, a quality estimation unit 312, an iris authentication unit 314, a guidance unit 316, and a settlement unit 318. The authentication device 300 is, for example, a server provided at a location separated from the store T1.

[0047] The condition determination unit 302 determines whether or not the payment start button displayed on the input display unit 204 of the cash register terminal 200 has been pressed based on the operation signal acquired from the cash register terminal 200. When it is determined that the payment start button has been pressed, the condition determination unit 302 determines that the shooting conditions are satisfied. In that case, the condition determination unit 302 generates a control signal for starting the shooting of the imaging unit 202, and transmits the control signal to the cash register terminal 200. In response to the control signal, the control unit 212 of the cash register terminal 200 controls the imaging unit 202 to start shooting and execute shooting at a predetermined interval.

[0048] The image acquisition unit 304 acquires a plurality of images captured by the imaging unit 202 after the imaging unit 202 starts shooting until the shooting is completed. As described above, the imaging unit 202 starts shooting when the payment start button displayed on the input display unit 204 is pressed. Also, as will be described later, the imaging unit 202 ends shooting when the payment confirmation button displayed on the input display unit 204 is pressed. Further, the image acquisition unit 304 acquires the captured images in the case of reshooting by the imaging unit 202. The image acquisition unit 304 extracts information on the iris portion from the acquired images, thereby acquiring partial information on the customer's iris for each image.

[0049] The operation detection unit 306 determines whether or not the payment confirmation button displayed on the input display unit 204 of the cash register terminal 200 has been pressed based on the operation signal acquired from the cash register terminal 200. When it is determined that the payment confirmation button has been pressed, the operation detection unit 306 determines that an operation serving as a trigger for iris authentication has been performed. In that case, the operation detection unit 306 generates a control signal for ending the shooting of the imaging unit 202, and transmits the control signal to the cash register terminal 200. In response to the control signal, the control unit 212 of the cash register terminal 200 controls the imaging unit 202 to end shooting.

[0050] The product information acquisition unit 308 acquires the product information read by the product reading unit 206 from when the payment start button displayed on the input display unit 204 is pressed until the payment confirmation button is pressed.

[0051] When the operation detection unit 306 detects that the payment confirmation button displayed on the input display unit 204 has been pressed, the generation unit 310 determines whether iris authentication is possible using partial information of the customer's iris shown in each of the multiple images acquired by the image acquisition unit 304. For example, the generation unit 310 may synthesize partial information of the iris to be authenticated shown in each of the multiple images acquired by the image acquisition unit 304 to generate synthesized iris information. The generation unit 310 then determines whether iris authentication by the iris authentication unit 314 is possible with respect to the generated synthesized information. For example, the generation unit 310 may determine that iris authentication is possible if the generated synthesized information is larger than a predetermined size, and that iris authentication is not possible if the synthesized information is smaller than the predetermined size. The predetermined size may be, for example, the size of the diameter of an average human iris.

[0052] The method by which the generation unit 310 generates composite information is the same as the method by which the generation unit 108 generates composite information in Embodiment 1, so a description will be omitted. If there are overlapping regions in the iris, the generation unit 310 selects partial information from one image for the overlapping regions and uses the selected partial information to generate composite information for the iris. The generation unit 310 can determine the presence or absence of overlapping regions and select partial information in the following two, but is not limited to these.

[0053] (1) The generation unit 310 may use information about the area around the customer's eyes to identify the position of the partial information in the iris in each of the multiple images acquired by the image acquisition unit 304. The information about the area around the eyes may be, for example, information about the shape of the eye, such as the edge of the eye, the orbit, the outer corner of the eye, or the hollow around the eye, such as "periocular". Based on the identification result, the generation unit 310 determines whether there are any overlapping regions in the iris among the partial information in each of the multiple images. If there are overlapping regions in the iris, the generation unit 310 selects the partial information of one image for the overlapping region.

[0054] (2) The generation unit 310 may perform geometric correction using a set reference point on the partial information of at least one of the multiple images acquired by the image acquisition unit 304. After performing this geometric correction, the generation unit 310 determines whether there are any overlapping regions in the iris among the partial information of each of the multiple images. If there are overlapping regions in the iris, the generation unit 310 generates composite information by selecting the partial information of one image for the overlapping region.

[0055] Geometric correction may, for example, involve generating a "rubber sheet," which is a sheet formed by deforming the iris information, which has an overall annular shape, into a plate-like shape. The generation unit 310 can determine whether or not the partial information is duplicated by determining where the partial information in each of the multiple images corresponds to on the sheet.

[0056] The quality estimation unit 312 estimates the quality of each of the multiple images acquired by the image acquisition unit 304. The generation unit 310 selects the partial information of the image with the highest quality estimated by the quality estimation unit 312 in the overlapping regions of the iris from the partial information of each of the multiple images. The generation unit 310 generates composite information by combining the partial information selected in the overlapping regions of the iris with the partial information relating to the non-overlapping regions of the iris.

[0057] The quality of each of the multiple images estimated by the quality estimation unit 312 may be, for example, as follows, but is not limited to these.

[0058] Quality may include one or more of various elements (quality elements) in the captured image, such as resolution, image quality, degree of focus blur, degree of motion blur, brightness, lightness, degree of glasses reflection, and degree of noise in the area indicating the target part. Quality may be represented, for example, by a score corresponding to one or more predetermined quality elements (for example, a score that decreases as the focus blur increases). Alternatively, quality may be represented by a vector composed of the values ​​of one or more quality elements. The score or vector value of the quality described above is changed depending on whether or not it is easy for the iris authentication unit 314 to authenticate.

[0059] Resolution may be the number of pixels equal to the iris diameter or iris radius. The degree of focus blur and the degree of motion blur may be values ​​representing the magnitude of focus blur and the magnitude of motion blur, respectively. The degree of glasses reflection may include at least one of the following: the intensity of light reflected by the glasses, the area on which reflection is applied to the target part used for biometric authentication, etc. The degree of noise may include at least one of the following: the magnitude of the noise (e.g., the total amount of overall noise, the average of overall noise, etc.) and values ​​representing the distribution of the noise (variance, standard deviation, etc.).

[0060] Furthermore, the quality estimation unit 312 may calculate the values ​​of the quality elements to be estimated for each image using the quality score calculation method described in International Publication No. 2024 / 095362.

[0061] Furthermore, the generation unit 310 may select partial information from each of the multiple images acquired by the image acquisition unit 304 that has a size greater than or equal to a predetermined size in areas that overlap in the iris. For example, the generation unit 310 may select partial information from each of the multiple images that has a size greater than or equal to about half the size of the iris region. The quality estimation unit 312 estimates the quality of the selected partial information that has a size greater than or equal to a predetermined size. The generation unit 310 generates the composite information as described above using the partial information of the image with the highest quality estimated by the quality estimation unit 312.

[0062] Furthermore, even if the imaging unit 202 captures multiple images of the same area of ​​the iris, the size of the area may differ from image to image if the customer's position moves. In this case, the generation unit 310 adjusts the size of the partial information in multiple images by enlarging or reducing the size of the partial information in at least one of the images. After adjusting the size of the partial information in this way, the generation unit 310 determines whether or not there are overlapping areas, as described above.

[0063] The iris authentication unit 314 performs iris authentication of the customer using the synthesized information generated by the generation unit 310, when the generation unit 310 determines that iris authentication is possible with respect to the synthesized information generated by the generation unit 310. The method by which the iris authentication unit 314 performs iris authentication is the same as the method by which the authentication unit 110 performs iris authentication in Embodiment 1, so a description is omitted.

[0064] The iris recognition database 1 stores information on the iris characteristics of service registrants and information on their online payments, linked together. The information on the service registrants' characteristics is the information used for comparison with the composite information in iris recognition, and is information that can identify the registrant to be recognized. The information on iris characteristics is the information used for iris recognition, and is, for example, information on the iris pattern. The information on iris characteristics is stored in the iris recognition database 1 for each eye. The online payment information is the information necessary for the registrant to make a payment, and may be, but is not limited to, credit card information or electronic money information. The iris recognition unit 314 can compare the composite information generated by the generation unit 310 with the comparison-target feature information stored in the iris recognition database 1 by referring to the iris recognition database 1.

[0065] Furthermore, if iris authentication is not possible with respect to the composite information generated by the generation unit 310, the imaging unit 202 needs to re-capture an image for iris authentication. In that case, the iris authentication unit 314 generates a control signal to the cash register terminal 200 to perform a re-capture and transmits the control signal to the cash register terminal 200. In response to the control signal, the control unit 212 of the cash register terminal 200 controls the imaging unit 202 to resume imaging.

[0066] After transmitting the above control signal, the iris authentication unit 314 acquires the image captured again by the imaging unit 202 via the image acquisition unit 304. The iris authentication unit 314 determines whether or not iris authentication is possible using the iris information contained in the captured image. Here, the iris authentication unit 314 may determine whether or not iris authentication is possible using the iris information contained in the captured image as is. Alternatively, the iris authentication unit 314 may generate new composite information by combining the iris information contained in the captured image with the composite information previously generated by the generation unit 310. The method for generating this composite information is the same as the method for generating composite information of the generation unit 310 described above, so a detailed explanation is omitted. The iris authentication unit 314 may also determine whether or not iris authentication is possible using the newly generated composite information. The determination of whether or not iris authentication is possible performed by the iris authentication unit 314 is the same as the determination of whether or not iris authentication is possible performed by the generation unit 310 as described above, so a detailed explanation is omitted.

[0067] The iris authentication unit 314 performs iris authentication using the iris information contained in the captured image if iris authentication is possible using that iris information. On the other hand, if iris authentication is not possible using that iris information, the iris authentication unit 314 generates a control signal to the cash register terminal 200 to perform a re-shoot and transmits the control signal to the cash register terminal 200. Similarly, the iris authentication unit 314 determines whether iris authentication is possible using the iris information contained in the newly captured image. The iris authentication unit 314 repeats this process until it is determined that iris authentication is possible.

[0068] The guidance unit 316 generates a control signal to guide the customer, who is the subject of the photograph, to look at the iris recognition camera 202 if iris recognition is not possible with respect to the composite information generated by the generation unit 310. The guidance unit 316 may also generate a similar control signal if iris recognition is not possible using the iris information contained in the image captured again in response to the control signal from the iris recognition unit 314. In response to this control signal, the control unit 212 causes the input display unit 204 to display a message guiding the customer to recognize the presence of the camera 202 and face it directly. The control unit 212 may also cause the audio output unit 210 to output audio information for similar guidance in response to this control signal. Furthermore, the control unit 212 may display information via the input display unit 204 indicating how much the position or orientation of the customer's face or body needs to be changed in response to this control signal. The control unit 212 may also cause the audio output unit 210 to output audio information indicating similar content. Furthermore, in response to this control signal, the control unit 212 may control whether or not the lighting unit 208 is lit, or the manner in which it is lit, for guidance purposes.

[0069] If the iris authentication unit 314 performs iris authentication on the customer and the authentication is successful, the settlement unit 318 retrieves online payment information stored in the iris authentication DB1 that is linked to the registered user whose iris authentication was successful. The settlement unit 318 uses the online payment information and the product information retrieved by the product information retrieval unit 308 to execute the payment. For example, the settlement unit 318 may execute the payment by sending the online payment information and product information to a payment server (not shown).

[0070] Furthermore, if the customer does not match any of the registered users stored in the iris recognition DB1, that is, if the customer cannot be authenticated, the settlement unit 318 may generate a control signal to cause the input display unit 204 to display that the settlement could not be completed. In response to this control signal, the control unit 212 causes the input display unit 204 to display that the settlement could not be completed.

[0071] [Flowchart Description] Figures 7A and 7B are flowcharts showing a typical example of the authentication device 300's processing, and these flowcharts explain the general overview of the authentication device 300's processing. Details of each processing step are as described above, so explanations will be omitted as appropriate. The following example describes a case where iris authentication and payment are performed for customer P3 shown in Figure 4.

[0072] First, the condition determination unit 302 determines whether or not the payment start button displayed on the input display unit 204 has been pressed (step S32). That is, the condition determination unit 302 determines whether or not the conditions for photographing customer P3 have been met. If the condition determination unit 302 determines that the payment start button has not been pressed (No in step S32), the process from step S32 onward is executed again. On the other hand, if the condition determination unit 302 determines that the payment start button has been pressed (Yes in step S32), the image acquisition unit 304 instructs the photography unit 202 to start photographing customer P3. This allows the photography unit 202 to photograph customer P3 multiple times when they are near the cash register terminal 200. The image acquisition unit 304 acquires multiple images of customer P3, who is the subject of authentication (step S34). Then, the image acquisition unit 304 extracts iris information from the acquired images to obtain partial iris information of customer P3 for each image (step S36).

[0073] The operation detection unit 306 determines whether the payment confirmation button displayed on the input display unit 204 of the cash register terminal 200 has been pressed (i.e., whether payment confirmation has been instructed) (step S38). In other words, the operation detection unit 306 determines whether or not it has detected an action that triggers iris authentication by customer P3. If the operation detection unit 306 determines that the payment confirmation button has not been pressed (No. in step S38), the processing from step S34 onwards is executed again.

[0074] On the other hand, if the operation detection unit 306 determines that the payment confirmation button has been pressed (Yes in step S38), the operation detection unit 306 instructs the shooting unit 202 to stop shooting customer P3. The product information acquisition unit 308 acquires product information transmitted from the cash register terminal 200 from the time the payment start button is pressed until the payment confirmation button is pressed (step S40). The generation unit 310 generates iris composite information by combining partial iris information from multiple images acquired from the time the payment start button is pressed until the payment confirmation button is pressed. Then, the generation unit 310 uses the generated composite information to determine whether or not iris authentication by the iris authentication unit 314 is possible (step S42).

[0075] If the generation unit 310 determines that iris authentication using the generated composite information is not possible (No. in step S42), the iris authentication unit 314 instructs the imaging unit 202 to photograph customer P3 again. The guidance unit 316 also instructs the input display unit 204, the illumination unit 208, and the audio output unit 210 to view the iris authentication imaging unit 202 (step S44). The image acquisition unit 304 acquires an image of customer P3 as a result (step S46).

[0076] The iris authentication unit 314 determines whether or not iris authentication is possible using the iris information contained in the image acquired in step S46 (step S48). If the iris authentication unit 314 determines that iris authentication is not possible using the iris information (No. in step S48), it instructs the imaging unit 202 to photograph customer P3 again. Then, the processing from step S44 onwards is executed again.

[0077] On the other hand, in step S42, if the generation unit 310 determines that iris authentication is possible using the generated composite information (Yes in step S42), the iris authentication unit 314 performs iris authentication of customer P3 using the generated composite information (step S50). Also, if the iris authentication unit 314 determines that iris authentication is possible using the iris information acquired in step S46 (Yes in step S48), it performs iris authentication of customer P3 using the acquired iris information (step S50).

[0078] If, as a result of iris authentication in step S50, it is determined that customer P3 matches a specific registered person stored in the iris authentication DB1 (i.e., iris authentication is successful), the settlement unit 318 obtains the online payment information of customer P3. Then, the settlement unit 318 uses the online payment information and the product information obtained in step S40 to execute the payment (step S52).

[0079] The cash register terminal 200 may also have other imaging units besides the imaging unit 202 (hereinafter also referred to as "other imaging units"). In step S44, when the customer P3 is to be photographed again, the iris authentication unit 314 may be controlled to have the other imaging units photograph the customer P3.

[0080] [Explanation of Effects] As described above, the authentication device 300 acquires multiple images of the customer when the payment start button is pressed and the payment procedure is initiated. Furthermore, when the payment confirmation button is pressed and the instruction to confirm the payment is given, the authentication device 300 generates composite iris information and performs iris authentication of the person to be authenticated. In other words, the authentication device 300 can acquire images taken when the customer is located near the cash register terminal 200 and when iris authentication is considered likely to be performed. In addition, the authentication device 300 performs iris authentication when payment confirmation is performed, that is, when iris authentication is required. Therefore, the authentication device 300 can perform iris authentication more reliably.

[0081] Furthermore, as shown in (1), the generation unit 310 may use information about the area around the eye to be authenticated to determine the position of the partial information in the iris in each of the multiple acquired images. Also, as shown in (2), the generation unit 310 may perform geometric correction using a set reference point on at least one of the partial information in the multiple images acquired by the image acquisition unit 304. By this determination, the generation unit 310 can accurately determine the position of the partial information in the iris. Therefore, the generation unit 310 can accurately determine whether or not there are overlapping regions in the iris among the partial information in each of the multiple images. As a result, the generation unit 310 can generate accurate iris composite information, which contributes to performing accurate iris authentication.

[0082] Furthermore, the generation unit 310 may generate composite information by selecting the image portion information with the highest quality estimated by the quality estimation unit 312 in areas of overlapping iris regions among the partial information of each of the multiple images. Therefore, the generation unit 310 can generate composite information more suitable for iris authentication, thereby contributing to accurate iris authentication.

[0083] Furthermore, the generation unit 310 may select multiple images from among the partial information in each of the multiple images, where the overlapping regions in the iris have partial information of a predetermined size or larger. The generation unit 310 generates the composite information by selecting the partial information of the image with the highest quality estimated by the quality estimation unit 312 from among the selected multiple images. Therefore, the generation unit 310 can generate composite information that is more suitable for iris authentication, thus contributing to accurate iris authentication.

[0084] Furthermore, the generation unit 310 may determine whether iris authentication is possible using the synthesized information. If it is determined that iris authentication is not possible, the guidance unit 316 guides the object to be authenticated to look at the iris authentication imaging device. As a result, the authentication device 300 can acquire an image that enables iris authentication even when iris authentication using synthesized information is not possible, thus ensuring reliable iris authentication.

[0085] In the above example, a self-checkout system was described in which the customer themselves has the product identification information of the product read by the product reading unit 206. However, the authentication device 300 can also be applied in the same way when a store employee has the product identification information of the product read by the product reading unit 206. In this case, in step S32, the condition determination unit 302 may determine whether or not the store employee has pressed the payment start button to start the payment. Also, in step S38, the operation detection unit 306 may determine whether or not the store employee has pressed the payment confirmation button to confirm the payment.

[0086] Furthermore, the operation detection unit 306 may detect that the cash register terminal 200 has received a payment confirmation instruction as an action that triggers iris authentication. This allows the authentication device 300 to apply iris authentication for payment purposes.

[0087] Furthermore, an infrared camera can be used to photograph the object being authenticated. Therefore, the camera can be miniaturized and mounted on the POS terminal 200.

[0088] Embodiment 3 (3A) The following describes a variation of Embodiment 2. Embodiment 2 described a case where payment is made by a customer or staff member using the cash register terminal 200. However, Embodiment 3 describes another example where the cash register terminal 200 is not present. Points already explained in Actual Embodiment 2 will be omitted as appropriate, and the explanation will focus on the processing specific to Embodiment 3.

[0089] Figure 8 is a block diagram showing another example of the payment system described in this disclosure. The payment system S2 comprises cameras MA, MB, ..., MG, an entrance terminal 400, an exit terminal 500, an authentication device 600, an iris recognition DB1, and a facial recognition DB2. Hereinafter, cameras MA, MB, ..., MG will also be collectively referred to as camera M. Each element shown in the payment system S2 is connected to one another. The payment system S2 is applied to a system that allows customers to make payments for product purchases in an unmanned store without operating a terminal, as shown below.

[0090] Figure 9 is a floor plan of the interior of a store where cameras MA, MB, ..., MG, an entrance terminal 400, and an exit terminal 500 are installed. The store T2 shown in Figure 9 has a roughly rectangular floor. A gate G1 is provided at one of the four corners of the floor as a customer entrance. Gate G1 is a gate whose opening and closing can be controlled by the entrance terminal 400 located near Gate G1. When Gate G1 opens, customers can enter the store T2. Display shelves H1, H2, and H3 are provided on three sides of the floor. A display shelf H4 is also provided in the center of the floor. Gate G2 is provided at the corner of the floor opposite Gate G1, flanking display shelf H4, as a customer exit. Gate G2 is a gate whose opening and closing can be controlled by the authentication device 600. When Gate G2 opens, customers can exit the store T2.

[0091] Furthermore, cameras MA and MB are provided at both ends of display shelf H1, respectively. Separately spaced cameras MC and MD are provided on display shelf H2, respectively. Cameras ME and MF are provided at both ends of display shelf H3, respectively. Near the center of display shelf H4, camera MG is provided opposite display shelf H1, and camera MH is provided opposite display shelf H3. Cameras MA through MH are near-infrared cameras and photograph people in the vicinity of each camera M. Each camera M transmits the image it has captured to the authentication device 600.

[0092] Furthermore, an exit terminal 500 equipped with a camera unit 502 is provided near gate G2. The camera unit 502 can photograph customers when they enter the AR area near gate G2.

[0093] Inside store T2, there are customers P4, P5, and P6 looking at products, as well as customer P7 moving to area AR. Customer P7 is attempting to complete payment for a product purchase by moving to area AR.

[0094] Figure 10 is a block diagram showing an example of an entrance terminal according to this disclosure. The entrance terminal 400 comprises a camera unit 402, a facial recognition unit 404, an opening / closing control unit 406, and a transmission unit 408. The entrance terminal 400 is a terminal that controls a customer's entry into store T2.

[0095] The imaging unit 402 has a camera that captures images of customers near gate G1 by photographing the vicinity of the entrance terminal 400. A detailed explanation of the other parts of the imaging unit 402 is the same as that of the imaging unit 202 and will therefore be omitted.

[0096] The facial recognition unit 404 performs facial recognition of a person in the image captured by the imaging unit 402. Specifically, the facial recognition unit 404 detects the face image (i.e., information about the face) of a person in the captured image and calculates information about the facial features of the person based on that face image. The facial recognition unit 404 compares the calculated facial feature information with the facial feature information stored in the facial recognition DB2 by referring to the facial recognition DB2.

[0097] Furthermore, the facial recognition DB2 stores information on the facial features of service registrants. This facial feature information is information that can identify the registrant to be authenticated. This facial feature information is used for facial recognition, and for example, the positions of feature points such as the eyes, nose, and mouth in a facial image, and the distances between feature points are defined as "features."

[0098] For example, the facial recognition unit 404 calculates the similarity between the calculated feature quantities and the facial feature quantities stored in the facial recognition DB2. The facial recognition DB2 stores the facial feature quantities of service registrants. If there are facial feature quantities in the facial recognition DB2 with a similarity greater than a threshold, the facial recognition unit 404 determines that facial recognition was successful. Also, if the feature quantities are feature vectors, the facial recognition unit 404 calculates the distance between the calculated feature vectors and the facial feature vectors in the facial recognition DB2. If there are facial feature vectors in the facial recognition DB2 with a distance less than a threshold, the facial recognition unit 404 determines that the feature vectors match and that facial recognition was successful. On the other hand, if there are no facial feature quantities in the facial recognition DB2 with a similarity greater than a threshold, or no facial feature vectors in the facial recognition DB2 with a distance less than a threshold, the facial recognition unit 404 determines that facial recognition failed.

[0099] Furthermore, the facial recognition DB2 may store information on the facial features of service registrants and identification information that uniquely identifies the registrants, linked together. If facial recognition by the facial recognition unit 404 is successful, the facial recognition unit 404 transmits the identification information of the registrant whose facial recognition was successful to the authentication device 600 via the transmission unit 408.

[0100] The opening / closing control unit 406 controls the opening and closing of gate G1. For example, the opening / closing control unit 406 normally keeps gate G1 closed, and when the facial recognition unit 404 determines that facial recognition has been successful, it controls gate G1 to open. The opening / closing control unit 406 may also close gate G1 after a predetermined period of time has elapsed since opening it.

[0101] When facial recognition by the facial recognition unit 404 is successful and gate G1 is open, the transmitting unit 408 transmits authentication success information to the authentication device 600 indicating that facial recognition was successful. Successful facial recognition corresponds to the action of triggering iris recognition of the person to be authenticated. In addition, as described above, when facial recognition by the facial recognition unit 404 is successful, the transmitting unit 408 may also transmit the identification information of the registered person whose facial recognition was successful to the authentication device 600.

[0102] Figure 11 is a block diagram showing an example of an exit terminal according to this disclosure. The exit terminal 500 comprises a camera unit 502, a display unit 504, an audio output unit 506, a control unit 508, and a transmitting / receiving unit 510. The exit terminal 500 is a terminal that controls a customer's exit from store T2.

[0103] The imaging unit 502 has a camera that captures images of customers near gate G2 (for example, within the area AR) by photographing the vicinity of the exit terminal 500. A detailed description of the other parts of the imaging unit 502 is the same as that of the imaging unit 202 and will therefore be omitted.

[0104] The display unit 504 can output a display that guides the customer to recognize the presence of the imaging unit 502 and face the front of the imaging unit 502, in response to control by the control unit 508. The display unit 504 has, for example, a display.

[0105] The audio output unit 506 can output audio information indicating the same content in response to the control unit 508. The audio output unit 506 may include, for example, a speaker.

[0106] Furthermore, the control unit 508 may analyze the customer's iris as seen in the image captured by the imaging unit 502 to determine whether it is necessary to change the position or orientation of the customer's face or body in order to obtain a more favorable image for iris authentication. If it is necessary to change the position or orientation of the customer's face or body, the control unit 508 may display information indicating the extent to which the change is necessary via the display unit 504. The control unit 508 may also output audio information indicating the same content to the audio output unit 506.

[0107] The transmitting / receiving unit 510 receives control signals transmitted from the authentication device 600. The transmitting / receiving unit 510 also transmits the image captured by the imaging unit 202 to the authentication device 600.

[0108] Figure 12 is a block diagram showing an example of an authentication device according to this disclosure. The authentication device 600 comprises a condition determination unit 602, an image acquisition unit 604, a person identification unit 606, an arrival detection unit 608, a product detection unit 610, a generation unit 612, a quality estimation unit 614, an iris authentication unit 616, a guidance unit 618, a payment unit 620, and an opening / closing control unit 622. The authentication device 600 is a server installed, for example, at a location separated from the store T2.

[0109] The condition determination unit 602 determines whether or not it has received authentication success information from the entrance terminal 400. If it determines that it has received authentication success information, the condition determination unit 602 determines that the shooting conditions have been met. In that case, the condition determination unit 602 controls the image acquisition unit 604 to start acquiring images taken by each camera M inside the store T2.

[0110] The image acquisition unit 604 acquires multiple images captured by each camera M. Furthermore, the image acquisition unit 604 also acquires images captured by the shooting unit 502. The image acquisition unit 604 extracts information about the iris from the acquired images to obtain partial information about the customer's iris for each image.

[0111] The person identification unit 606 classifies images of the same person from the images acquired by the image acquisition unit 604. For example, as shown in Figure 9, when customers P4 to P7 enter store T2, the person identification unit 606 classifies the images of each customer from the images acquired by the image acquisition unit 604. An example of image classification will be explained below.

[0112] (i) The person identification unit 606 may function as a face recognition unit. The person identification unit 606 performs face recognition of a person in the captured image. Specifically, the person identification unit 606 detects the face image of a person (i.e., information about the face) in the captured image and calculates information about the person's facial features based on that face image. The person identification unit 606 compares the calculated facial feature information with the facial feature information stored in the facial recognition DB2 by referring to the facial recognition DB2. The details of this process are the same as the details of the process of the face recognition unit 404, so the explanation is omitted.

[0113] The person identification unit 606 then identifies the image in which it is determined that the face of the same registered person is visible. For a given customer (i.e., a registered user of the system), after receiving authentication success information, the person identification unit 606 identifies the image in which it is determined that the face of that customer is visible in each of the acquired images until the arrival detection unit 608 determines that the customer has arrived in area AR. For example, in Figure 9, customer P7's face is photographed by each camera M from the time they pass through gate G1 until they arrive in area AR. The person identification unit 606 classifies the images taken by each camera M using facial recognition to identify the image in which customer P7's face is visible. Using a similar method, the person identification unit 606 identifies the images in which the faces of customers P4 to P6 are visible from the images taken by each camera M.

[0114] Furthermore, the person identification unit 606 may use the facial feature information stored in the facial recognition DB2, which is linked to the registered person's identification information received from the entrance terminal 400, as a target for comparison with the facial feature information of the person appearing in the image. If there are multiple people whose identification information is received from the entrance terminal 400, the person identification unit 606 may use the facial feature information of those multiple people stored in the facial recognition DB2 as a target for comparison with the facial feature information of the person appearing in the image. For example, in the example shown in Figure 9, the person identification unit 606 can determine which of the customers P4 to P7 is appearing in the image by comparing the feature information related to the identification information of customers P4 to P7 received from the entrance terminal 400 with the facial feature information of the face appearing in the image of camera M. As a result, the person identification unit 606 only needs to perform facial recognition using a limited portion of the information stored in the facial recognition DB2 as a comparison target. Therefore, the facial recognition processing of the person identification unit 606 can be completed earlier.

[0115] (ii) The person identification unit 606 may use the images captured by each camera M to track the movement of the photographed customer within the store T2. For example, the person identification unit 606 may use map information of the store T2 floor, information on the position and field of view of each camera M, etc., to track the movement of each customer within the store T2. By doing so, the person identification unit 606 can determine whether the customer appearing in each of the captured images is the same person.

[0116] Furthermore, in (ii), the person identification unit 606 may use one of the images taken of the customer determined to be the same person to perform the facial recognition shown in (i) and determine which registered person the customer corresponds to. Alternatively, the person identification unit 606 may identify the registered person's identification information stored in the facial recognition DB2 as a result of the determination.

[0117] The arrival detection unit 608 detects whether or not a customer has arrived in the AR area by analyzing the image captured by the imaging unit 502. If it determines that a customer has arrived in the AR area, the arrival detection unit 608 determines that an action that triggers iris authentication has been performed.

[0118] At this time, the person identification unit 606 determines whether the customer who has arrived in the area AR is the same person as the customer captured in the images taken by each camera M. The person identification unit 606 may also function as a face recognition unit in the same manner as above to determine which registered person the customer who has arrived in the area AR and is captured by the camera 502 corresponds to. In the example shown in Figure 9, the person identification unit 606 can determine which of the customers P4 to P7 is captured in the image by comparing the feature information related to the identification information of customers P4 to P7 received from the entrance terminal 400 with the feature information of the faces captured in the images taken by the camera 502. In addition, the person identification unit 606 may identify the registered person's identification information stored in the face recognition DB2 as a result of the determination.

[0119] The product detection unit 610 analyzes the image captured by the imaging unit 502 to determine whether a customer within the AR area is holding a product. If the customer is holding a product, the product detection unit 610 identifies the product information (e.g., product name and price) of the product shown in the image. The product detection unit 610 may use a trained AI (Artificial Intelligence) model to identify the product information of the product. The AI ​​model is trained by inputting training data, which includes sample images showing products and correct labels indicating the product information corresponding to the sample images. The product detection unit 610 inputs the image captured by the imaging unit 502 to the trained AI model. Based on the input information, the AI ​​model outputs estimated product information for the product.

[0120] When a predetermined customer arrives in the area AR, the generation unit 612 acquires multiple images of the predetermined customer identified by the person identification unit 606 during the period from when the predetermined customer is authenticated at the entrance terminal 400 until they arrive in the area AR. The generation unit 612 then determines whether iris authentication is possible using partial information of the predetermined customer's iris as seen in each of the multiple images identified by the person identification unit 606. For example, the generation unit 612 may synthesize the partial information of the iris to be authenticated as seen in each of the multiple identified images to generate synthesized iris information. The generation unit 612 then determines whether iris authentication by the iris authentication unit 616 is possible with respect to the generated synthesized information. A detailed explanation of this determination is the same as in Embodiment 2, so the explanation is omitted.

[0121] The method by which the generation unit 612 generates the composite information is the same as the method by which the generation unit 310 generates the composite information in Embodiment 2, so a description will be omitted.

[0122] The quality estimation unit 614 estimates the quality of each of a set of images of a given customer in the same manner as the quality estimation unit 312 in Embodiment 2. A detailed explanation of the quality estimated by the quality estimation unit 614 is the same as in Embodiment 2, so the explanation is omitted.

[0123] The generation unit 612 selects the image with the highest quality estimated by the quality estimation unit 614 from among the partial information of each of the multiple images, in the overlapping regions of the iris. The generation unit 612 generates composite information by combining the partial information selected in the overlapping regions of the iris with the partial information relating to the non-overlapping regions of the iris.

[0124] Furthermore, similar to the generation unit 310 in Embodiment 2, the generation unit 612 may select partial information having a size greater than or equal to a predetermined size in the regions that overlap in the iris from among the partial information in each of the multiple images.

[0125] Furthermore, in Embodiment 3, there are multiple cameras M for photographing customers, and the relative positions of the customer and the cameras M at the time of shooting are different. Therefore, compared to Embodiment 2, even when the same area of ​​the iris is captured in multiple images, it is likely that the size of the area will often differ from image to image. In this case, the generation unit 612 adjusts the size of the partial information in multiple images by enlarging or reducing the size of the partial information in at least one of the images. After adjusting the size of the partial information in this way, the generation unit 612 determines whether or not there are overlapping areas, as described above.

[0126] The iris authentication unit 616 performs iris authentication of the customer using the synthesized information generated by the generation unit 612, when the generation unit 612 determines that iris authentication is possible with respect to the synthesized information generated by the generation unit 612. The method by which the iris authentication unit 616 performs iris authentication is the same as the method by which the iris authentication unit 314 performs iris authentication in Embodiment 2, so a description is omitted. The description of the iris authentication DB1 is also the same as in Embodiment 2, so a description is omitted.

[0127] Furthermore, if iris authentication is not possible with respect to the composite information generated by the generation unit 612, the imaging unit 502 needs to capture an image for iris authentication. In that case, the iris authentication unit 616 generates a control signal to the exit terminal 500 to perform imaging and transmits the control signal to the exit terminal 500. In response to the control signal, the control unit 508 of the exit terminal 500 controls the imaging unit 502 to perform imaging.

[0128] After transmitting the above-mentioned control signal, the iris authentication unit 616 acquires the image captured by the imaging unit 502 via the image acquisition unit 604. The iris authentication unit 616 determines whether or not iris authentication is possible using the iris information contained in the captured image. Here, the iris authentication unit 616 may determine whether or not iris authentication is possible using the iris information contained in the captured image as is. Alternatively, the iris authentication unit 616 may generate new composite information by combining the iris information contained in the captured image with the composite information previously generated by the generation unit 612. The determination performed by the iris authentication unit 616 is the same as the determination performed by the iris authentication unit 314 according to Embodiment 2, so a detailed explanation is omitted.

[0129] The iris authentication unit 616 performs iris authentication using the iris information contained in the captured image if iris authentication is possible using that iris information. On the other hand, if iris authentication is not possible using that iris information, the iris authentication unit 616 generates a control signal to the exit terminal 500 to perform re-shooting and transmits the control signal to the exit terminal 500. Similarly, the iris authentication unit 616 determines whether iris authentication is possible using the iris information contained in the newly captured image. The iris authentication unit 616 repeats this process until it is determined that iris authentication is possible.

[0130] The guidance unit 618 generates a control signal to guide the customer, who is the subject of the photograph, to look at the iris recognition camera 502 if iris recognition is not possible with respect to the composite information generated by the generation unit 612. The guidance unit 618 may also generate a similar control signal if iris recognition is not possible using the iris information contained in the image captured in response to the control signal of the iris recognition unit 616. In response to this control signal, the control unit 508 causes the display unit 504 to display information guiding the customer to recognize the presence of the camera 502 and face it directly. The control unit 212 may also cause the audio output unit 506 to output audio information for similar guidance in response to this control signal. Furthermore, the control unit 508 may also cause the display unit 504 to display information indicating how much the position or orientation of the customer's face or body needs to be changed in response to this control signal. The control unit 508 may also cause the audio output unit 506 to output audio information indicating similar content.

[0131] If the iris authentication unit 616 performs iris authentication on the customer and the authentication is successful, the settlement unit 620 retrieves online payment information stored in the iris authentication DB1 that is linked to the registered user for whom the iris authentication was successful. The settlement unit 620 uses the online payment information and the product information obtained by the product detection unit 610 to execute the payment. The details of the payment executed by the settlement unit 620 are the same as those of the settlement unit 318 in Embodiment 2, so the explanation is omitted.

[0132] Furthermore, if the customer does not match any of the registered users stored in the iris recognition DB1, that is, if the customer cannot be authenticated, the settlement unit 620 may generate a control signal to cause the display unit 504 to display that the settlement could not be completed. In response to this control signal, the control unit 508 causes the display unit 504 to display that the settlement could not be completed.

[0133] The opening / closing control unit 622 controls the opening and closing of gate G2. For example, the opening / closing control unit 622 normally keeps gate G2 closed, and controls gate G2 to open after the settlement unit 620 has completed the settlement. The opening / closing control unit 622 may also close gate G2 after a predetermined period of time has elapsed since opening it. Furthermore, if the settlement unit 620 is unable to complete the settlement, the opening / closing control unit 622 may keep gate G2 closed. In addition, if the product detection unit 610 determines that a customer in area AR does not possess any products, the opening / closing control unit 622 may control gate G2 to open it from a closed state.

[0134] [Flowchart Description] Figures 13A and 13B are flowcharts showing a typical example of the authentication device 600's processing, and these flowcharts explain the general overview of the authentication device 600's processing. Details of each processing step are as described above, so explanations will be omitted as appropriate. The following example describes a case where iris authentication and payment are performed for customer P7 shown in Figure 9.

[0135] First, the condition determination unit 602 determines whether or not it has received authentication success information from the entrance terminal 400 (i.e., whether authentication has been completed) (step S62). That is, the condition determination unit 602 determines whether or not the conditions for photographing the customer have been met. If the condition determination unit 602 determines that it has not received authentication success information from the entrance terminal 400 (No in step S62), the processing from step S62 onwards is executed again. On the other hand, if the condition determination unit 602 determines that it has received authentication success information from the entrance terminal 400 (Yes in step S62), the image acquisition unit 604 acquires multiple images taken by each camera M (step S64). The image acquisition unit 604 acquires partial iris information for each image by extracting iris information from the acquired images (step S66). The person identification unit 606 identifies the images taken of the same person, customer P7, from among the images acquired by the image acquisition unit 604.

[0136] The arrival detection unit 608 determines whether or not customer P7 has arrived at area AR (i.e., the exit) (step S68). That is, the arrival detection unit 608 determines whether or not it has detected an action that triggers iris authentication by customer P7. If the arrival detection unit 608 determines that customer P7 has not arrived at area AR (No. in step S68), the processing from step S64 onwards is executed again.

[0137] On the other hand, if the arrival detection unit 608 determines that customer P7 has arrived in area AR (Yes in step S68), the product detection unit 610 analyzes the image captured by the imaging unit 502 to detect whether customer P7 possesses any products (step S70). If it is detected that customer P7 does not possess any products (No in step S70), the opening / closing control unit 622 controls gate G2 to open it from a closed state (step S84). As a result, customer P7 who does not intend to purchase any products can exit store T2.

[0138] On the other hand, if it is detected that customer P7 is in possession of a product (Yes in step S70), the product detection unit 610 detects product information of the product that customer P7 is in possession of. The generation unit 612 acquires multiple images of customer P7 that have been identified by the person identification unit 606 during the period from when customer P7 is authenticated at the entrance terminal 400 until they arrive at the area AR. The generation unit 612 then generates iris composite information using partial information of customer P7's iris that appears in each of the multiple images identified by the person identification unit 606. The generation unit 612 uses the generated composite information to determine whether or not iris authentication by the iris authentication unit 616 is possible (step S72).

[0139] If the generation unit 612 determines that iris authentication using the generated composite information is not possible (No. in step S72), the iris authentication unit 616 instructs the imaging unit 502 to photograph customer P7. The guidance unit 618 also instructs the display unit 504 and the audio output unit 506 to view the iris authentication imaging unit 502 (step S74). The image acquisition unit 604 acquires an image of customer P7 as a result (step S76).

[0140] The iris authentication unit 616 determines whether iris authentication is possible using the iris information contained in the image acquired in step S76 (step S78). If the iris authentication unit 616 determines that iris authentication is not possible using the iris information (No. in step S78), it instructs the imaging unit 502 to photograph customer P7 again. Then, the processing from step S74 onwards is executed again.

[0141] On the other hand, in step S72, if the generation unit 612 determines that iris authentication is possible using the generated composite information (Yes in step S72), the iris authentication unit 616 performs iris authentication of customer P7 using the generated composite information (step S80). Also, if the iris authentication unit 616 determines that iris authentication is possible using the iris information acquired in step S76 (Yes in step S78), it performs iris authentication of customer P7 using the acquired iris information (step S80).

[0142] If, as a result of iris authentication in step S80, it is determined that customer P7 matches a specific registered person stored in the iris authentication DB1 (i.e., iris authentication is successful), the settlement unit 620 obtains the online payment information of customer P7. The settlement unit 620 then uses the online payment information and the product information obtained in step S70 to execute the payment (step S82). After the settlement by the settlement unit 620 is completed, the opening / closing control unit 622 controls the gate G2 to open it from the closed state (step S84).

[0143] [Explanation of Effects] As described above, the authentication device 600 generates iris synthesis information when a customer arrives at the exit and performs iris authentication for the customer to be authenticated. In other words, the authentication device 600 performs iris authentication when it is required when shopping at an unmanned store. Therefore, the authentication device 600 can perform iris authentication processing only when necessary and suppress the execution of unnecessary processing. In addition, the authentication device 600 can eliminate the need for cash register operations, etc., during customer payment.

[0144] Furthermore, the generation unit 612 may generate composite information of a customer by combining partial iris information of the same customer identified by the person identification unit 606. Therefore, the generation unit 612 can accurately generate composite information relating to the iris of the same person, thus contributing to accurate iris authentication.

[0145] Furthermore, camera M, which is an infrared camera, may capture infrared images including the facial image and iris image of the customer to be authenticated. This allows the authentication device 600 to classify the customer in the image using the customer's facial image. The authentication device 600 can also generate composite information for the classified customers using the iris images captured in the image. Therefore, the authentication device 600 can generate composite iris information without using multiple types of cameras. This can lead to a reduction in the cost of the payment system. It also becomes possible to install a large number of cameras M within the store.

[0146] At the entrance to store T2, personal authentication other than facial recognition may be performed. For example, the entrance terminal 400 may perform authentication using the customer's ID card. The card reader of the entrance terminal 400 reads the information on the customer's ID card. The entrance terminal 400 compares the information on the read ID card with the information on the registered user's ID card stored in the dedicated database. If the entrance terminal 400 finds an ID card in the dedicated database that matches the information on the read ID card, it determines that authentication has been successful and opens gate G1. The dedicated database may also store the registered user's ID card information and the registered user's identification information linked together. The authentication device 600 can obtain the identification information of the registered user who has been successfully authenticated from the dedicated database. The authentication device 600 can then perform the same processing as described above.

[0147] Furthermore, personal authentication at the entrance of store T2 is not necessarily required. For example, a motion sensor may be installed at the entrance of store T2 instead of the entrance terminal 400. The motion sensor can use infrared, ultrasonic, microwave, or the like for detection. When the motion sensor detects a customer entering store T2, the motion sensor transmits a detection signal to the authentication device 600. The condition determination unit 602 determines whether or not it has received a detection signal from the motion sensor. If it determines that it has received a detection signal, the condition determination unit 602 determines that the shooting conditions have been met. The authentication device 600 can then perform the same processing as described above.

[0148] Furthermore, the exit terminal 500 may have functions such as facial recognition and authentication using an ID card. For example, the exit terminal 500 performs facial recognition of a person in an image captured by the imaging unit 502. The exit terminal 500 compares the facial feature information calculated based on the facial image with the facial feature information stored in the facial recognition DB2 by referring to the facial recognition DB2. The details of this facial recognition are the same as those of the facial recognition performed by the entrance terminal 400, so the explanation is omitted. If facial recognition is successful, the exit terminal 500 transmits information indicating that facial recognition was successful to the authentication device 600 via the transmitting / receiving unit 510. The exit terminal 500 may also transmit the identification information of the registered person whose facial recognition was successful to the authentication device 600. The exit terminal 500 can also perform authentication using an ID card instead of facial recognition in the same manner as the entrance terminal 400 described above.

[0149] When the arrival detection unit 608 receives information from the exit terminal 500 indicating that facial recognition was successful, it determines that customer P7 has arrived at area AR (i.e., the exit). The authentication device 600 can then perform the same processing as described above.

[0150] In step S70 of Figure 13A, if it is detected that customer P7 does not possess the product (No. in step S70), the information acquired or generated regarding customer P7 in Figures 13A and 13B may not be stored in the authentication device 600 and may be discarded. The information acquired or generated regarding customer P7 in Figures 13A and 13B includes images of customer P7 taken by each camera M, partial iris information extracted for customer P7, and composite iris information generated for customer P7.

[0151] However, the authentication device 600 does not have to discard the information described above. For example, the iris authentication DB1 may store information on the iris characteristics of the service registrant, the registrant's online payment information, and information indicating whether or not to save the information acquired by the authentication device 600, all linked together. The authentication device 600 refers to the iris authentication DB1 and, when it performs iris authentication for customer P7, refers to the information indicating whether or not to save the information acquired by the authentication device 600. If the information is not to be saved, the authentication device 600 discards the information described above. On the other hand, if the information is to be saved, the authentication device 600 does not discard the information described above. For example, this information may be used for marketing, etc., together with product information detected by the product detection unit 610 as data indicating customer P7's visit to store T2.

[0152] As another example, the authentication device 600 may store the iris synthesis information generated for customer P7 and the identification information of registered customer P7 as customer P7's visit history to store T2 within the authentication device 600 for a certain period (e.g., one day) after customer P7 leaves store T2. The authentication device 600 determines whether customer P7 has come to store T2 again by comparing the identification information of the registered person transmitted to the authentication device 600 by the entrance terminal 400 with the identification information of customer P7 in the visit history. Alternatively, the authentication device 600 may perform the comparison using the identification information of the registered person obtained from the facial recognition performed by the person identification unit 606 using an image captured by camera M, instead of the identification information of the registered person transmitted by the entrance terminal 400.

[0153] If the two items being compared match, the authentication device 600 determines that customer P7 has come to store T2 again. At this time, the generation unit 612 may generate composite information of customer P7 using the iris composite information from the visit history and partial information of customer P7 captured by camera M when customer P7 came to store T2 again. This allows the authentication device 600 to generate composite information of customer P7 with higher accuracy than if it were to generate composite information using only partial information of customer P7 captured by camera M.

[0154] (3B) Section (3B) will explain the variations in (3A). Below, we will focus on explaining the differences from (3A), and omit explanations of other points.

[0155] Figure 14 is a block diagram showing an example of an authentication device according to this disclosure. Compared to the authentication device 700 of (3A), the authentication device 700 further includes a gaze detection unit 624.

[0156] The gaze detection unit 624 detects the direction of the gaze of customers in multiple images captured by each camera M acquired by the image acquisition unit 604. The gaze detection unit 624 performs this detection using images of the area around the eyes (for example, the area around the eyes including the eyelids). The gaze detection unit 624 also detects the direction of the gaze of customers in images captured by the shooting unit 502.

[0157] The generation unit 612 acquires the gaze direction of a predetermined customer and partial information of the predetermined customer's iris, as detected by the gaze detection unit 624, for each of the multiple images of a predetermined customer identified by the person identification unit 606. The generation unit 612 may use the partial information of images relating to gaze directions where the gaze direction is less than or equal to a predetermined proportion, with priority over the partial information of images relating to other gaze directions, to generate composite information. "Priority use" means that the probability of it being used in generating the composite information is higher. As a result, the generation unit 612 can generate composite information of the iris that is closer to perfect.

[0158] Furthermore, in step S72, if it is determined that iris authentication by the iris authentication unit 616 is not possible using the generated composite information, the generation unit 612 may determine the missing parts in the generated composite information by comparing it with the information of the entire iris. Then, in step S74, the guidance unit 618 instructs the display unit 504 and the audio output unit 506 to guide the customer P7 to move the position or orientation of their face so that the missing parts are captured by the imaging unit 502.

[0159] For example, the generation unit 612 determines that the generated composite information is missing the lower part of the overall iris information. At this time, in step S74, the guidance unit 618 instructs the display unit 504 and the audio output unit 506 to guide customer P7 to look upwards. For example, the display unit 504 displays "Please look upwards" in response to a control signal from the authentication device 700. As a result, the imaging unit 502 can photograph the lower part of customer P7's iris when customer P7 looks upwards. The iris authentication unit 616 can generate new composite information using the captured partial information of the lower part of the iris and the previously generated composite information. In step S78, the authentication device 700 can perform iris authentication using the new composite information.

[0160] Furthermore, the gaze detection unit 624 may identify products or other items located in the direction of the customer's gaze in each image by analyzing the images captured by each camera M. The product detection unit 610 identifies product information of products located in the direction of the gaze in the same manner as described above. The authentication device 700 may store the product information of the identified products internally. Alternatively, the authentication device 700 may store the personal information of the customer who viewed the product, associated with the product information of the identified product, internally. The customer's personal information is stored in the iris recognition DB 1 or face recognition DB 2, linked to the characteristic quantities of each registered user. In either the iris recognition or face recognition process described above, the authentication device 700 obtains the personal information of the customer whose authentication was successful by referring to the iris recognition DB 1 or face recognition DB 2. This information stored in the authentication device 700 may be used for marketing or other purposes.

[0161] Although this disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, which can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0162] The authentication device 300 according to Embodiment 2 and the authentication device 600 according to Embodiment 3 may be configured as a single computer device, similar to the authentication device 100 according to Embodiment 1, or as a distributed system having multiple computer devices. For example, some components of the authentication device 300 may be provided on the cash register terminal 200 or other devices. Some components of the authentication device 600 may be provided on the entrance terminal 400, the exit terminal 500 or other devices. Furthermore, some or all of these systems may be provided on a cloud server built on the cloud, or on other types of virtualized servers generated using virtualization technology, etc.

[0163] In the embodiments described above, this disclosure has been described as a hardware configuration, but this disclosure is not limited thereto. This disclosure can also be realized by having a processor in a computer execute a computer program to perform the processing of the devices constituting the system described in the embodiments described above.

[0164] Figure 15 is a block diagram showing an example of the hardware configuration of an information processing device (in other words, a computer) on which the authentication device 100 according to Embodiment 1 or the devices according to Embodiments 2 to 3 are executed. Referring to Figure 15, the information processing device 90 includes a signal processing circuit 91, a processor 92, and a memory 93.

[0165] The signal processing circuit 91 is a circuit for processing signals in accordance with the control of the processor 92. The signal processing circuit 91 may also include a communication circuit for receiving signals from a transmitting device.

[0166] The processor 92 is connected to the memory 93 and performs the processing of the system described in the above embodiment by reading and executing a computer program from the memory 93. As an example of the processor 92, one of the following may be used: CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field-Programmable Gate Array), DSP (Digital Signal Processor), or ASIC (Application Specific Integrated Circuit), or multiple of these may be used in parallel.

[0167] The memory 93 is composed of volatile memory, non-volatile memory, or a combination thereof. There is not limited to one memory 93, but multiple memory units may be provided. The volatile memory may be, for example, RAM (Random Access Memory) such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The non-volatile memory may be, for example, ROM (Read Only Memory) such as PROM (Programmable Random Only Memory) or EPROM (Erasable Programmable Read Only Memory), flash memory, or SSD (Solid State Drive).

[0168] Memory 93 is used to store one or more instructions. Here, one or more instructions are stored in memory 93 as a program. The processor 92 can perform the processing described in the above embodiment by reading and executing these programs from memory 93.

[0169] The memory 93 may include not only memory located outside the processor 92, but also memory built into the processor 92. Furthermore, the memory 93 may include storage located separately from the processors that make up the processor 92. In this case, the processor 92 can access the memory 93 via an I / O (Input / Output) interface.

[0170] As described above, one or more processors in each system or device of the above embodiments execute one or more programs that include a set of instructions for causing a computer to perform the algorithm described with reference to the drawings. The information processing described in each embodiment can be realized by executing the programs.

[0171] The program, when loaded into a computer, includes a set of instructions or software code for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrical, optical, acoustic or other forms of propagating signals. The temporary computer-readable medium or a communication medium can supply the program to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0172] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) An authentication device comprising: an acquisition means for acquiring a plurality of images of a target for authentication; a generation means for generating composite iris information by combining partial information of the iris of the target for authentication that is captured in each of the plurality of images; an authentication means for performing iris authentication of the target for authentication using the composite information; a condition determination means for determining whether or not the shooting conditions are met; and an action detection means for detecting an action that triggers iris authentication of the target for authentication, wherein the acquisition means acquires the plurality of images between the time the condition determination means determines that the shooting conditions are met and the time the action detection means detects the trigger action, and the generation means generates the composite information when the action detection means detects the trigger action. (Note 2) The authentication device according to Note 1, wherein the generation means uses information about the area around the eye to be authenticated to identify the position of the partial information in the iris in each of the plurality of images, determines whether there are any overlapping regions in the iris among the partial information in each of the plurality of images based on the identification result, and if there are overlapping regions in the iris, generates the composite information by selecting the partial information of one image for the overlapping region. (Note 3) The authentication device according to Note 1, wherein the generation means performs geometric correction using a set reference point on the partial information of at least one of the plurality of images, determines whether there are any overlapping regions in the iris among the partial information in each of the plurality of images after the execution of the geometric correction, and if there are overlapping regions in the iris, generates the composite information by selecting the partial information of one image for the overlapping region. (Note 4) The authentication device further comprises a quality estimation means for estimating the quality of each of the plurality of images, and the generation means generates the composite information by selecting the partial information of the image with the highest quality estimated by the quality estimation means in the region of the iris that overlaps with the partial information of each of the plurality of images.(Note 5) The authentication device according to Note 4, wherein the generation means selects from the partial information in each of the plurality of images the partial information having a size greater than or equal to a predetermined size in the overlapping region of the iris, and from the selected plurality of images the partial information of the image with the highest quality estimated by the quality estimation means generates the composite information. (Note 6) The authentication device according to any one of Notes 1 to 5, further comprising: determination means for determining whether iris authentication is possible using the composite information; and guidance means for guiding the authentication target having the iris to view the iris authentication imaging device when the determination means determines that iris authentication is not possible. (Note 7) The authentication device according to any one of Notes 1 to 6, wherein the authentication device further comprises identification means for identifying the same authentication target that appears in the plurality of images when the acquisition means acquires the plurality of images from a plurality of imaging devices, and the generation means generates the composite information of the same authentication target by combining the partial information of the iris of the identified same authentication target. (Note 8) The authentication device according to any one of Notes 1 to 7, wherein the authentication device is connected to a payment terminal that receives instructions to confirm payment, and the operation detection means detects that the payment terminal has received instructions to confirm payment as the trigger operation. (Note 9) The authentication device according to any one of Notes 1 to 7, wherein the operation detection means detects that the object to be authenticated has moved to a predetermined area as the trigger operation. (Note 10) The authentication device according to any one of Notes 1 to 9, wherein the plurality of images are infrared images including a face image and an iris image of the object to be authenticated, taken by an infrared camera. (Note 11) An authentication method performed by a computer, comprising: determining whether or not the shooting conditions are met; acquiring a plurality of images of the object to be authenticated from the time it is determined that the shooting conditions are met until an operation that triggers iris authentication of the object to be authenticated is detected; generating composite iris information by combining partial information of the iris of the object to be authenticated as seen in each of the plurality of images when the trigger operation is detected; and performing iris authentication of the object to be authenticated using the composite information.(Note 12) A program that causes a computer to perform the following actions: determine whether or not the shooting conditions are met; acquire multiple images of the object to be authenticated until an action that triggers iris authentication of the object to be authenticated is detected after determining that the shooting conditions are met; generate composite iris information by combining partial information of the iris of the object to be authenticated as seen in each of the multiple images when the trigger action is detected; and perform iris authentication of the object to be authenticated using the composite information.

[0173] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 10 that are dependent on Appendice 1 may also be dependent on Appendices 11 and 12 in the same way as those described in Appendices 2 to 10. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0174] Although this disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, which can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0175] 100 Authentication device 102 Condition determination unit 104 Acquisition unit 106 Motion detection unit 108 Generation unit 110 Authentication unit S1, S2 Payment system DB1 Iris recognition DB DB2 Face recognition DB 200 Register terminal 202 Shooting unit 204 Input display unit 206 Product reading unit 208 Lighting unit 210 Audio output unit 212 Control unit 214 Transmit / receive unit 300 Authentication device 302 Condition determination unit 304 Image acquisition unit 306 Operation detection unit 308 Product information acquisition unit 310 Generation unit 312 Quality estimation unit 314 Iris recognition unit 316 Guidance unit 318 Payment unit 400 Entrance terminal 402 Shooting unit 404 Face recognition unit 406 Open / close control unit 408 Transmit unit 500 Exit terminal 502, shooting unit 504, display unit 506, audio output unit 508, control unit 510, transmitting / receiving unit 600, 700, authentication device 602, condition determination unit 604, image acquisition unit 606, person identification unit 608, arrival detection unit 610, product detection unit 612, generation unit 614, quality estimation unit 616, iris authentication unit 618, guidance unit 620, payment unit 622, opening / closing control unit 624, gaze detection unit

Claims

1. An authentication device comprising: an acquisition means for acquiring multiple images of a target for authentication; a generation means for generating composite iris information by combining partial information of the iris of the target for authentication as seen in each of the multiple images; an authentication means for performing iris authentication of the target for authentication using the composite information; a condition determination means for determining whether or not the shooting conditions are met; and an action detection means for detecting an action that triggers iris authentication of the target for authentication, wherein the acquisition means acquires the multiple images between the time the condition determination means determines that the shooting conditions are met and the time the action detection means detects the trigger action, and the generation means generates the composite information when the action detection means detects the trigger action.

2. The authentication device according to claim 1, wherein the generation means uses information about the area around the eye to be authenticated to identify the position of the partial information in the iris in each of the plurality of images, determines whether there are any overlapping regions in the iris among the partial information in each of the plurality of images based on the identification result, and if there are overlapping regions in the iris, generates the composite information by selecting the partial information of one image for the overlapping region.

3. The authentication device according to claim 1, wherein the generation means performs geometric correction on the partial information of at least one of the plurality of images using a set reference point, and after performing the geometric correction, determines whether there are any overlapping regions in the iris among the partial information of each of the plurality of images, and if there are overlapping regions in the iris, generates the composite information by selecting the partial information of one image for the overlapping region.

4. The authentication device further comprises a quality estimation means for estimating the quality of each of the plurality of images, and the generation means generates the composite information by selecting the partial information of the image with the highest quality estimated by the quality estimation means in the region of the iris that overlaps with the partial information of each of the plurality of images.

5. The authentication device according to claim 4, wherein the generation means selects from the partial information in each of the plurality of images the partial information having a size greater than or equal to a predetermined size in the overlapping region in the iris, and from the selected plurality of images the partial information of the image with the highest quality estimated by the quality estimation means generates the composite information.

6. The authentication device according to any one of claims 1 to 5, further comprising: determination means for determining whether iris authentication is possible using the synthesized information; and guidance means for guiding the authentication target having an iris to view an iris authentication imaging device when the determination means determines that iris authentication is not possible.

7. The authentication device according to any one of claims 1 to 6, wherein the authentication device further comprises a identification means for identifying the same authentication target that appears in the plurality of images when the acquisition means acquires the plurality of images from the plurality of imaging devices, and the generation means generates the combined information of the same authentication target by combining the partial information of the iris of the identified same authentication target.

8. The authentication device according to any one of claims 1 to 7, wherein the authentication device is connected to a payment terminal that receives instructions to confirm payment, and the operation detection means detects that the payment terminal has received instructions to confirm payment as the trigger operation.

9. The authentication device according to any one of claims 1 to 7, wherein the motion detection means detects that the object to be authenticated has moved to a predetermined area as the triggering motion.

10. The authentication device according to any one of claims 1 to 9, wherein the plurality of images are infrared images including the face image and iris image of the subject to authentication, captured by an infrared camera.

11. A computer-based authentication method comprising: determining whether or not the shooting conditions are met; acquiring multiple images of the object to be authenticated from the time it is determined that the shooting conditions are met until an action that triggers iris authentication of the object to be authenticated is detected; generating composite iris information by combining partial information of the iris of the object to be authenticated as seen in each of the multiple images when the triggering action is detected; and performing iris authentication of the object to be authenticated using the composite information.

12. A program that causes a computer to perform the following actions: determine whether or not the shooting conditions are met; acquire multiple images of the object to be authenticated until an action that triggers iris authentication of the object to be authenticated is detected after determining that the shooting conditions are met; generate composite iris information by combining partial information of the iris of the object to be authenticated as seen in each of the multiple images when the trigger action is detected; and perform iris authentication of the object to be authenticated using the composite information.

Images