Information processing device, information processing method, and program

Abstract

An information processing device according to one aspect comprises: a generation means that generates projection information indicating at least one of a position and an orientation preferable for imaging an imaging target with respect to an imaging device that images the imaging target; a projection means that projects the information; and a notification means that, on the basis of an image obtained by imaging the imaging target and the projected information, performs notification used for guiding a person so that the projected information is projected in a predetermined position and orientation with respect to the imaging target.

Description

Information Processing Apparatus, Information Processing Method and Program

[0001] The present disclosure relates to an information processing apparatus, an information processing method and a program.

[0002] Technologies related to biometric authentication have been developed. For example, Patent Document 1 discloses that an authentication apparatus calculates a difference between a relative position of a hand with respect to an imaging apparatus when capturing an image of the hand and a relative position of the hand with respect to the imaging apparatus when capturing a registered image of the hand, and outputs guidance information for reducing the difference. The authentication apparatus projects an arrow or a dot onto the hand as the guidance information.

[0003] International Publication No. 2010 / 086993

[0004] An object of the present disclosure is to improve the technology described in the prior art documents.

[0005] An information processing apparatus according to one aspect comprises: a generation unit configured to generate projection information indicating at least one of a position and an orientation preferable for capturing an imaging target, for an imaging apparatus that captures the imaging target; a projection unit configured to project the information; a notification unit configured to perform notification used for guiding a person so that the projected information is projected at a predetermined position and in a predetermined orientation with respect to the imaging target, based on an image obtained by capturing the imaging target and the projected information.

[0006] An information processing method according to one aspect is an information processing method executed by a computer, the method comprising: generating projection information indicating at least one of a position and an orientation preferable for capturing an imaging target, for an imaging apparatus that captures the imaging target; projecting the information; performing notification used for guiding a person so that the projected information is projected at a predetermined position and in a predetermined orientation with respect to the imaging target, based on an image obtained by capturing the imaging target and the projected information.

[0007] A program according to one embodiment is a program that causes a computer to perform the following actions: generate projection information indicating at least one of a preferred position or orientation for photographing a subject to a photographing device that photographs the subject; project the information; and, based on the image obtained from the photographed subject and the projected information, provide notification used to guide a person so that the projected information is projected at a predetermined position and orientation relative to the subject.

[0008] This is a block diagram showing an example of an information processing device related to this disclosure. This is a flowchart showing a typical example of processing by the information processing device. This is a block diagram showing an example of an authentication system related to this disclosure. This is a flowchart showing a typical example of processing by the authentication device. This is a block diagram showing another example of an authentication system related to this disclosure. This is a block diagram showing an example of the hardware configuration of an information processing device on which the processing related to this disclosure is performed.

[0009] Embodiments of this disclosure will be described below with reference to the drawings. Note that the following descriptions and drawings in the embodiments have been omitted and simplified as appropriate for clarity of explanation. Furthermore, unless otherwise specified, in this disclosure, when "at least one of the multiple items" is defined for multiple items, that definition may mean any single item, or it may mean any multiple items including all of the items.

[0010] Each drawing referenced in an embodiment is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate.

[0011] Embodiment 1 [Description of Configuration] Figure 1 is a block diagram showing an example of an information processing device according to this disclosure. The information processing device 100 comprises a generation unit 102, a projection unit 104, and a notification unit 106.

[0012] The generation unit 102 generates projection information (hereinafter also referred to as projection information) that indicates at least one of the preferred position or orientation of the subject relative to the imaging device that photographs the subject. Hereafter, a person (hereinafter also referred to as subject) will be used as an example of the subject. However, the subject is not limited to a person. The preferred position of the subject relative to the imaging device will also be referred to as the appropriate position of the subject. The preferred orientation of the subject relative to the imaging device will also be referred to as the appropriate orientation of the subject. For example, the generation unit 102 may generate projection information that has been stored in advance. As another example, the generation unit 102 may generate projection information that indicates at least one of the preferred position or orientation of the subject relative to the imaging device based on the position information of the imaging device that photographs the subject. As yet another example, the generation unit 102 may generate projection information using not only the position information of the imaging device but also the information of the direction the imaging device is facing.

[0013] An appropriate position for a subject is a position in which the imaging device can capture an image suitable for a predetermined image-based processing (e.g., authentication). An appropriate position for a subject may satisfy at least one of the following conditions: - The subject is less than a predetermined distance D1 and at least a predetermined distance D2 from the imaging device (D1 > D2) - The subject is positioned in the front direction of the imaging device (i.e., the imaging device can photograph the subject) In other words, this is a condition relating to at least one of the magnitude or direction of the vector connecting the subject and the imaging device. The front direction of the imaging device is defined, for example, as the direction in which the lens of the imaging device is facing forward.

[0014] The appropriate orientation of a subject is the orientation of the subject that allows the imaging device to capture an image suitable for a predetermined image-based processing (e.g., authentication). The appropriate orientation of a subject means, for example, that the imaging device can capture the subject, and that when the imaging device and the subject are viewed from the side or from above, the angle between the front direction of the imaging device and the front direction of the subject is below a predetermined threshold. The front direction of the subject is defined, for example, as the direction in which a part of the subject's body (e.g., face or torso) is facing forward. In the following, the front direction of the imaging device and the front direction of the subject will also be referred to simply as the orientation of the imaging device and the orientation of the subject.

[0015] The imaging device is any device capable of photographing an object, such as a camera or a device equipped with a camera module. For example, the imaging device may determine its position information using radio waves received from positioning satellites, using a positioning method such as the Global Navigation Satellite System (GNSS). As another example, a measurement sensor that measures the movement of the imaging device may acquire movement information indicating the movement history of the imaging device. The measurement sensor is any sensor capable of detecting the movement of the imaging device. The measurement sensor may be an inertial sensor such as a gyro sensor or an accelerometer, but the specific examples of measurement sensors are not limited to these. The measurement sensor may be installed on the imaging device or at a location separated from the imaging device. The imaging device or another device derives the position information of the imaging device using the movement information. The generation unit 102 generates projection information as described above based on the derived position information.

[0016] Projected information is information that, when projected at any location, becomes visible to the subject, the holder of the recording device (hereinafter simply referred to as the holder), or any other person. Projected information may also be any display information such as symbols, logos, marks, or illustrations. Symbols may include numbers or letters. Marks may include figures. Illustrations may include illustrations of characters.

[0017] For example, the projected information may be a mark containing straight lines. A mark containing straight lines may be a mark containing multiple straight lines, where each line intersects with the others at approximately right angles (e.g., a cross mark, a grid, etc.). As another example, a mark containing straight lines may be composed of a polygon containing multiple straight lines. If the projected information is projected onto a subject to indicate at least one of the subject's appropriate position or orientation, it is assumed that the appearance of the projected information will differ depending on whether the subject is positioned in the appropriate position or orientation or not.

[0018] In particular, when the mark includes a straight line, it is assumed that the distortion in the appearance of the projected line will differ significantly depending on whether the subject is positioned in the appropriate position or orientation or not. Therefore, by visually inspecting the mark, the holder can easily determine whether the subject is positioned in the appropriate position or orientation. However, the mark is not limited to the above example. For example, the mark may be an arrow indicating the appropriate position or orientation of the subject. Alternatively, the mark may be a figure such as a circle indicating the appropriate position of the subject.

[0019] As another example, the projection information may be an illustration indicating the appropriate orientation of the subject. For example, the illustration may show the direction of the person's face. Alternatively, the projection information may be textual information indicating at least one of the appropriate position or orientation of the subject.

[0020] The projected information may include a single display piece or multiple display pieces. Multiple display pieces may be, for example, information that combines multiple symbols, logos, marks, or illustrations. By projecting multiple display pieces, information about both the appropriate position and orientation of the subject is displayed. Furthermore, the projected information may be static, which does not change over time, or it may be a dynamic mark that changes over time. For example, the generation unit 102 may set the projected information so that the mark is projected in a blinking state in order to emphasize the appropriate position of the subject.

[0021] The projection unit 104 projects the information generated by the generation unit 102. For example, the projection unit 104 projects the projection information onto a person, space, a plane, or a curved surface. As a first example, the projection unit 104 may project the projection information onto the forehead of a person. However, the projection unit 104 may project the projection information onto a part of the person other than the forehead. As a second example, the projection unit 104 may project the projection information onto space where no object exists. In this case, the projection unit 104 can project the projection information using technologies such as holograms or fog displays. This eliminates the need for a physical display to project the projection information. The projection unit 104 may also display the field of view of the imaging device as projection information. As a third example, the projection unit 104 may project the projection information onto a plane such as a display or the floor. However, the examples of projection by the projection unit 104 are not limited to these.

[0022] The projection unit 104 may project projection information to a position (appropriate position) relative to the imaging device that is favorable for photographing the subject. In this case, when the subject moves to the position on which the projection information is projected, the imaging device can capture an image favorable for predetermined processing. For example, the projection unit 104 may project text information such as "Please stand in this position" onto the floor surface located at the appropriate position of the subject. Alternatively, if the subject is already in the appropriate position, the projection unit 104 may project a cross mark as projection information onto a part of the subject (e.g., the forehead). Alternatively, the projection unit 104 may project a human-shaped mark or a rectangular frame, etc., as projection information at a spatial position corresponding to the appropriate position of the subject, using technologies such as holograms or fog displays. However, the projection unit 104 may project projection information to a position other than the appropriate position of the subject. For example, the projection unit 104 may project text information such as "Please stand to the right of this display" onto a display adjacent to the appropriate position of the subject. In this case, if the subject moves to the right of the position where the projection information is projected (the position on the display), the imaging device can capture an image suitable for the predetermined processing. The generation unit 102 generates the projection information described above to indicate the appropriate position of the subject.

[0023] The notification unit 106 performs notifications used to guide a person so that the projection information is projected at a predetermined position and orientation relative to the person. Here, the notification unit 106 performs notifications based on images of the person and the projection information that have been captured. Images of the person and the projection information are captured by a camera or other device. The person to be guided is at least one of the person being guided or the person holding the object. Depending on the shooting conditions, any position and orientation can be set as the predetermined position and orientation.

[0024] For example, the notification unit 106 can understand the positional relationship between the subject and the projection information by analyzing the captured image. The notification unit 106 then determines whether the positional relationship between the subject and the projection information is appropriate.

[0025] For example, if projection information is projected onto the appropriate location of the subject, the notification unit 106 uses the image to determine whether the position of the projection information in the image is the same as the position of the subject. As an example, suppose the projection unit 104 projects the text information "Please stand in this position" onto the floor surface located at the appropriate location of the subject. In this case, the notification unit 106 analyzes the image to determine whether the subject is at the position where the projection information was projected (the appropriate location). As another example, suppose the projection unit 104 projects a cross mark onto a part of the subject as projection information. In this case, the notification unit 106 analyzes the projection state of the cross mark in the image to determine whether the subject is at the appropriate location. Details of this will be described later in Embodiment 2.

[0026] On the other hand, if projection information is projected to a location different from the appropriate location of the subject, the notification unit 106 uses the image to determine whether the subject's location is the same as the appropriate location. For example, suppose the projection unit 104 projects the text information "Please stand to the right of this display" as projection information onto a display close to the appropriate location of the subject. In this case, the notification unit 106 analyzes the image to determine whether the subject is to the right of the display (the appropriate location). For example, the notification unit 106 determines that the subject is in the appropriate location if the subject is within a predetermined range that includes the appropriate location. On the other hand, if the subject is not within a predetermined range that includes the appropriate location, the notification unit 106 determines that the subject is not in the appropriate location. The notification unit 106 may also determine whether a person is visible in the image and, if so, identify that person as the subject.

[0027] If the notification unit 106 determines that the subject is not in their appropriate location (for example, the location indicated by the projection information), it issues a notification used to guide a person so that the subject moves to their appropriate location. At this time, the subject may move to their appropriate location by noticing the notification themselves. Alternatively, another person, such as a holder, may notice the notification and guide the subject to move to their appropriate location.

[0028] As another example, if the system determines that the subject is not in the appropriate position, the notification unit 106 issues a notification used to guide the holder so that the appropriate position for the subject moves to the subject's position. When the holder notices the notification and moves the position of the imaging device, the projection information generated by the generation unit 102 is projected to a different location than before. As a result, the appropriate position for the subject can be made to be the same as the subject's position.

[0029] If the system determines that the subject is not in the correct position, the notification unit 106 may use the image to derive the distance difference between the subject's current position and the correct position. The notification unit 106 then notifies the person of the derived distance.

[0030] On the other hand, if the notification unit 106 determines that the subject is in the appropriate location, it does not need to issue a notification used to guide the person. However, the notification unit 106 may issue a notification indicating that the subject is already in the appropriate location.

[0031] Furthermore, the notification unit 106 may analyze the captured image to determine whether the subject's orientation is the same as the subject's appropriate orientation (i.e., the orientation indicated by the projection information). For example, the notification unit 106 may determine that the subject's orientation is the same as the subject's appropriate orientation if the angle between the subject's orientation and the subject's appropriate orientation is below a predetermined threshold.

[0032] If the notification unit 106 determines that the orientation of the subject differs from the appropriate orientation, it will issue a notification used to guide the person so that the subject's orientation changes to the appropriate orientation. At this time, the subject may change their orientation by noticing the notification. Alternatively, a person other than the subject, such as a holder, may notice the notification and guide the subject to change their orientation.

[0033] As another example, if the notification unit 106 determines that the subject's orientation is different from the appropriate orientation, it will issue a notification used to guide the holder to change the subject's orientation to the appropriate orientation. The holder will notice the notification and move the orientation of the imaging device, thereby making the subject's orientation the same as the subject's orientation.

[0034] If the subject's orientation differs from the appropriate orientation, the notification unit 106 may use the image to derive the angle difference between the subject's orientation and the appropriate orientation. The notification unit 106 then notifies the person of the derived angle.

[0035] On the other hand, if the notification unit 106 determines that the subject's orientation is the same as the subject's appropriate orientation, it does not need to perform a notification used to guide the person. However, the notification unit 106 may perform a notification indicating that the subject's orientation is already the same as the subject's appropriate orientation.

[0036] Furthermore, the notification unit 106 may use projection information displayed in the image to determine whether the orientation of the subject is the same as the appropriate orientation for the subject. Details of this will be described later in Embodiment 2.

[0037] The notification unit 106 notifies a person of information for notification (hereinafter also referred to as notification information) by at least one of the following means: display, sound, or vibration. For example, notification information is information that can be made visible to the target person, holder, or other person by being displayed at any location. Notification information may also be any display information such as symbols, logos, marks, or illustrations. Symbols include numbers or letters. Marks include figures. Illustrations may be illustrations showing characters.

[0038] For example, the notification information may indicate the direction of movement from the subject's current location to their appropriate location. Such notification information may be represented as a mark, illustration, or text. For example, the notification information may be an arrow indicating the direction of movement from the subject's current location to their appropriate location. The direction of movement may be indicated by either qualitative or quantitative information.

[0039] The notification unit 106 may determine the orientation of the subject by performing image processing on the captured image. The notification unit 106 may then derive the direction of movement from the subject's current position to the appropriate position, as viewed from the subject. For example, if the direction of movement is forward from the subject's perspective, the notification unit 106 may display the text information "Forward" as notification information. Alternatively, if the direction of movement is backward from the subject's perspective, the notification unit 106 may display the text information "Backward" as notification information.

[0040] The notification unit 106 may indicate the distance traveled from the subject's current location to the appropriate location using numbers or letters as notification information. Alternatively, the notification unit 106 may use an image to detect a landmark object located near the appropriate location of the subject. The notification unit 106 displays textual information indicating the detected object as notification information.

[0041] As another example, the notification information may indicate a movement direction from the current orientation of the subject to an appropriate orientation of the subject. The movement direction may be indicated by either qualitative information or quantitative information. For example, the notification information may be an arrow indicating a movement direction for moving the face of the subject to an appropriate orientation. The notification information may be an illustration showing an appropriate face orientation. Furthermore, the notification information may be numerical or text information indicating the movement direction of the face orientation. As an example, the notification information may be text information stating "Turn your face a little more to the right".

[0042] Furthermore, the notification unit 106 may, as the notification information, indicate the movement angle from the current orientation of the subject to the appropriate orientation of the subject by using numbers or characters.

[0043] The notification information may indicate a plurality of pieces of information selected from the movement direction or movement distance from the current position of the subject to the appropriate position of the subject, or the movement direction or movement angle from the current orientation of the subject to the appropriate orientation of the subject.

[0044] Furthermore, in order to move the photographing device, the notification information may indicate at least one of a movement direction or a movement distance from the appropriate position of the subject to the current position of the subject, or a movement direction or a movement angle from the appropriate orientation of the subject to the current orientation of the subject. In this case, examples of the notification information notified by the notification unit 106 to the holder are as described above.

[0045] When it is determined that the subject is already at an appropriate position, the notification unit 106 may, for example, display the text "Position OK" as notification information indicating that the subject is already at an appropriate position. Even when it is determined that the orientation of the subject matches the appropriate orientation of the subject, the notification unit 106 may, for example, display the text "Orientation OK" as notification information indicating that the orientation of the subject is already appropriate. Both "Position OK" and "Orientation OK" may be displayed.

[0046] The notification unit 106 may project notification information onto any of a subject, a space, a flat surface, or a curved surface. Since the method by which the notification unit 106 projects notification information is the same as the method by which the projection unit 104 projects projection information, description thereof is omitted. Note that the notification unit 106 may project notification information onto a location different from the location onto which projection information is projected, or may project notification information onto the same location as projection information.

[0047] As another example, the notification unit 106 may cause notification information to be displayed by changing a light emission pattern of a light emitting device communicatively connected to the information processing apparatus 100. The light emission pattern indicated by the light emitting device is a pattern in which at least any one of, for example, a light emission color, light emission intensity, a light emission period, a non-light emission period, or a light emission site in the light emitting device is different from each other. The notification unit 106 can change a blinking pattern of light emission by changing at least any one of the light emission period or the non-light emission period. Each light emission pattern indicates at least any one of a movement direction or a movement distance from the position of the subject to an appropriate position of the subject, or a movement direction or a movement angle from the orientation of the subject to an appropriate orientation of the subject. For example, the light emission pattern may display character or numeral information. When the subject or a holder views the light emission pattern, the position or orientation of the subject is changed. Accordingly, the position or orientation of the subject can be adjusted to the appropriate position or orientation of the subject.

[0048] However, each light emission pattern may indicate at least any one of a movement direction or a movement distance from the appropriate position of the subject to the position of the subject, or a movement direction or a movement angle from the appropriate orientation of the subject to the orientation of the subject. When the holder views the light emission pattern, the position or orientation of an imaging apparatus is changed. Accordingly, the appropriate position or orientation of the subject can be adjusted to the position or orientation of the subject.

[0049] Furthermore, some light emission patterns may indicate at least any one of that the subject is already at the appropriate position of the subject, or that the orientation of the subject is already appropriate.

[0050] Furthermore, the notification unit 106 may change the mode of illumination notification according to the illuminance around the target person. Details of this will be described later in Embodiment 3.

[0051] The notification unit 106 may notify by changing the vibration pattern of a vibration generator that is communicatively connected to the information processing device 100. The vibration generator may be held by the holder or by another person. The vibration generator may be installed on the imaging device or installed separately from the imaging device. If the vibration generator is installed separately from the imaging device, it is held or worn by the holder. The vibration pattern shown by the vibration device is, for example, a pattern in which the frequency of vibration, the magnitude of vibration (i.e., amplitude), the duration of vibration, the period without vibration, or at least one of the vibration points on the vibration generator are different from each other. Each vibration pattern indicates at least one of the direction or distance of movement from the subject's position to the appropriate position of the subject, or the direction or angle of movement from the subject's orientation to the appropriate orientation of the subject. By sensing this vibration pattern, the holder can move at least one of the position or orientation of the imaging device to a preferred position or orientation.

[0052] The notification unit 106 may output notification information as audio information to a speaker that is communicatively connected to the information processing device 100. The speaker is provided in close proximity to the target person or holder. By outputting the notification information as audio, the target person or holder can be guided to move their position or orientation in a desirable direction.

[0053] Furthermore, when the notification unit 106 projects notification information, the projection unit 104 and the notification unit 106 may be provided in the information processing device 100 as the same component. However, the projection unit 104 and the notification unit 106 may be provided in the information processing device 100 as different components.

[0054] [Flowchart Description] Figure 2 is a flowchart showing a typical example of the information processing device 100, and this flowchart explains the overview of the information processing device 100's processes. Details of each process are as described above, so explanations will be omitted as appropriate.

[0055] The generation unit 102 generates projection information (step S12). The projection unit 104 projects the projection information generated in step S12 (step S14). The notification unit 106 provides notification used to guide a person, based on the image in which the subject and projection information have been captured, so that the projection information is projected to a predetermined position and orientation relative to the subject (step S16).

[0056] [Explanation of Effect] When an imaging device photographs a subject, it is possible that the subject may not be in a desirable position or orientation relative to the imaging device. In this case, it may be difficult for the imaging device to photograph the subject in a desirable position or orientation. Therefore, the shooting, or a series of processes including shooting, may take a long time.

[0057] However, as described above, the information processing device 100 can perform notification used to guide a person so that the projection information is projected at a predetermined position and orientation relative to the target, based on the captured image of the target and projection information. In response to the guidance, at least one of the position or orientation of the target or the shooting device is moved. For example, the subject may change their own position or orientation by checking the notification information. Alternatively, the person performing the shooting may be guided to change the position or orientation of the subject by checking the notification information. In this way, the information processing device 100 can ensure that shooting is performed reliably and shorten the time required for shooting.

[0058] Furthermore, the information processing device 100 performs guidance based on the captured image, including not only the target to be photographed but also projection information. Therefore, compared to the case where guidance is performed based only on the image of the target to be photographed, the information processing device 100 can guide the target to be photographed or the photographing device more accurately.

[0059] The information processing device 100 may be configured as a single computer device or as a distributed system having multiple computer devices. In a distributed system, the processing performed by the information processing device 100 can be shared and executed by multiple computer devices. In other words, the information processing device 100 may be realized by distributing each part of the information processing device 100 across two or more computer devices, and enabling communication between the two or more computer devices.

[0060] The information processing device 100 may have some or all of its components installed on a cloud server built on the cloud, or on other types of virtualized servers generated using virtualization technology, etc. Functions other than those installed on servers such as cloud servers or virtualized servers are located at the edge.

[0061] Each of the embodiments described below discloses specific examples of the information processing device 100 described in Embodiment 1. Below, we describe an example in which the information processing device 100 is applied to iris recognition in a store. However, the specific examples of the information processing device 100 shown in Embodiment 1 are not limited to those shown below. For example, the information processing device 100 may be used for person authentication in facilities requiring security, such as event venues, airports, and research laboratories, or for other person authentication purposes. Furthermore, the configurations and processes described below are illustrative and not limiting.

[0062] Embodiment 2 [Description of Configuration] Figure 3 is a block diagram showing an example of the authentication system according to this disclosure. The authentication system S1 comprises an authentication device 200, a shooting device C1, and a display D1. The authentication system S1 photographs a customer in a store and uses the captured image to perform facial recognition or iris recognition of the customer.

[0063] The imaging device C1 is a device that can be held and freely moved by the store staff performing customer authentication. The imaging device C1 has a camera module and, in accordance with the control of the authentication device 200, takes pictures of the object to be photographed and the customer being authenticated. The captured images are output to the authentication device 200.

[0064] Display D1 is connected to the authentication device 200. Images captured by the camera C1 are displayed on display D1 for both staff and customers. If necessary, notification information is also displayed on display D1.

[0065] The authentication device 200 authenticates the customer appearing in the image captured by the camera C1. The authentication device 200 can also guide the customer or the staff holding the camera C1 to move them to a preferred position and orientation for authentication photography.

[0066] The authentication device 200 includes an acquisition unit 202, a generation unit 204, a projection unit 206, a determination unit 208, a notification unit 210, a shooting control unit 212, an authentication unit 214, and a storage unit 216. Each unit will be described below.

[0067] The acquisition unit 202 acquires information about the images taken by the customer from the imaging device C1. The acquisition unit 202 is an interface that is connected to the imaging device C1 by wireless or wired connection.

[0068] The generation unit 204 generates projection information indicating the preferred position and orientation of the object to be photographed relative to the imaging device. The details of the processing of this generation unit 204 are the same as the details of the processing of the generation unit 102 described in Embodiment 1, so a description is omitted. The projection information includes, for example, a cross mark.

[0069] The projection unit 206 projects the projection information generated by the generation unit 204. At this time, as described in Embodiment 1, the projection unit 206 may project the projection information onto a space where no object exists, a plane, or a curved surface. Alternatively, the projection unit 206 may project the projection information onto the customer's forehead as a position preferred for photographing the customer. At this time, the staff can suggest to the customer that they maintain their position and orientation.

[0070] When projection information is projected, the acquisition unit 202 acquires information about the customer and the image in which the projection information has been captured from the shooting device C1. When the projection unit 206 projects the projection information onto the customer's forehead, the determination unit 208 determines, based on the image in which the customer and the projection information have been captured, whether or not the projection information has been properly projected onto the customer's forehead. If the customer's position and orientation are both appropriate for shooting, the determination unit 208 determines that the projection information has been properly projected onto the customer's forehead. On the other hand, if at least one of the customer's position or orientation is not appropriate for shooting, the determination unit 208 determines that the projection information has not been properly projected onto the customer's forehead.

[0071] For example, the determination unit 208 performs image processing on the captured image to compare the position where the projection information is projected in the image with the position of the customer. In other words, the determination unit 208 determines whether or not the projection information is properly projected onto the customer's forehead. If the projection information is projected onto the customer's forehead, the determination unit 208 determines that the customer is in a suitable position for shooting. The determination unit 208 may also determine that the customer is in a suitable position for shooting if the projection information is projected onto the customer's forehead at a predetermined size. If the projection information is not projected onto the customer's forehead, the determination unit 208 determines that the customer is not in a suitable position for shooting. The determination unit 208 may also determine that the customer is not in a suitable position for shooting if the projection information is not projected onto the customer's forehead at a predetermined size. For example, the determination unit 208 may determine that the customer is in a suitable position for shooting if the area of ​​the projection information projected onto the customer's forehead is greater than or equal to a predetermined threshold T1 and less than a predetermined threshold T2 (T2 > T1). The determination unit 208 determines that the customer is not in a suitable position for shooting if the area of ​​the projection information is any other value.

[0072] Furthermore, the determination unit 208 compares the projection state of the projection information shown in the image with the projection state of the appropriate projection information stored in the storage unit 216. The determination unit 208 calculates at least one of the degree of stretching or angular deformation of the projection state of the projection information shown in the actual image relative to the projection state of the appropriate projection information. The determination unit 208 may, for example, calculate the degree of stretching or angular deformation using a displacement vector from the projection state of the appropriate projection information to the projection state of the projection information shown in the image. If at least one of the calculated degree of stretching or angular deformation is above a predetermined threshold, the determination unit 208 determines that the customer's orientation is not suitable for shooting.

[0073] For example, even if projection information is projected onto the customer's forehead, if the customer's face is not facing the front of the imaging device C1, the projection information will be distorted compared to when the customer's face is facing the front of the imaging device C1. The determination unit 208 can detect this projection state and determine that the customer's orientation is not suitable for imaging. On the other hand, the determination unit 208 determines that the customer's orientation is suitable for imaging if both the calculated degree of stretching or angle are below a predetermined threshold.

[0074] The following describes further examples of the process by which the determination unit 208 determines whether or not projection information is properly projected onto the customer's forehead. The determination unit 208 detects the customer's position and orientation relative to the imaging device C1 as seen in the image acquired by the acquisition unit 202. Specifically, the determination unit 208 detects the position and orientation of the customer's face as seen in the image. The customer's orientation is as described in Embodiment 1. Since any known method can be used for detecting the customer's position and orientation performed by the determination unit 208, a detailed explanation is omitted.

[0075] The determination unit 208 derives a difference F1 between the detected customer position and the position on which the projection information is projected (the appropriate customer position). The determination unit 208 also derives a difference F2 between the customer orientation detected by the determination unit 208 and the orientation indicated by the projection information (the appropriate customer orientation). Difference F1 is expressed as a distance of 0 or more, and difference F2 is expressed as an angle of 0 or more. Difference F2 is the angle between the front direction of the imaging device C1 and the front direction of the customer when the imaging device C1 and the customer are viewed from above or the side.

[0076] The above position and orientation information may be expressed using, for example, latitude and longitude. However, the examples of position and orientation information are not limited to these. For example, if the photograph is taken indoors, the position and orientation information may be expressed using a coordinate system set up for indoor use.

[0077] The determination unit 208 determines whether the position difference F1 is less than a predetermined threshold Th1. The determination unit 208 also determines whether the orientation difference F2 is less than a predetermined threshold Th2. If F1 < Th1, the customer is positioned in a suitable location for imaging. If F2 < Th2, the imaging device C1 is facing the customer in a suitable orientation for imaging. If at least one of F1 or F2 does not satisfy this condition, the customer is not in a state suitable for imaging. On the other hand, if both F1 < Th1 and F2 < Th2 are satisfied, the customer is in a state suitable for imaging. The determination unit 208 uses this method to determine, for example, whether projection information is properly projected onto the customer's forehead.

[0078] Furthermore, the authentication device 200 or the imaging device C1 may be equipped with a light that illuminates a position appropriate for imaging by the imaging device C1. When the light is emitted, the projected information becomes more clearly visible. Therefore, the determination unit 208 can more accurately determine whether or not the projected information is properly projected onto the customer's forehead based on the image of the customer and the projected information that has been captured.

[0079] If the determination unit 208 determines that the projection information is properly projected onto the customer's forehead, the notification unit 210 projects the words "Shooting OK" onto the customer's forehead using the projection unit 206 as notification information. On the other hand, if the determination unit 208 determines that the projection information is not properly projected onto the customer's forehead, the notification unit 210 uses the projection unit 206 to project notification information such that the projection information is projected at a predetermined position and orientation relative to the customer. A detailed explanation of the content of this notification information is as described in Embodiment 1. The notification unit 210 may project the notification information onto the customer's forehead or onto another location.

[0080] The notification unit 210 may also notify the customer of the direction and distance of movement from the customer's position at the time of detection to the appropriate customer position, and the direction and angle of movement from the customer's orientation at the time of detection to the appropriate customer orientation. By visually confirming the projected notification information, staff can guide the customer to a preferred position and orientation for authentication photography.

[0081] Furthermore, the notification unit 210 may display notification information on the display D1. For example, if the determination unit 208 determines that the projection information is properly projected onto the customer's forehead, the notification unit 210 will display the words "Shooting OK" on the display D1 as notification information. On the other hand, if the determination unit 208 determines that the projection information is not properly projected onto the customer's forehead, the notification unit 210 will display notification information on the display D1 that will project the information to a predetermined position and orientation relative to the customer. By visually confirming the notification information, the staff and the customer can adjust the customer's position and orientation accordingly. In other words, the customer can move so that the notification information is projected onto their forehead.

[0082] Furthermore, the image acquired by the acquisition unit 202 may be displayed on display D1 along with the notification information. The notification unit 210 may display at least one of the arrows or numerical values ​​indicating the appropriate position and orientation based on the photographed customer on the screen of display D1 as notification information. This makes it easier for staff and customers to understand how to optimize the customer's position and orientation.

[0083] As another example, the notification unit 210 may notify information such as the direction and distance of movement from the appropriate customer position to the customer position at detection, and the direction and angle of movement from the appropriate customer orientation to the customer orientation at detection. In this case, the staff member holding the imaging device C1 can move the imaging device C1 to a position and orientation favorable for imaging by moving based on the notification information. In this case as well, the notification information may be displayed on the display D1.

[0084] If the determination unit 208 determines that the projection information is properly projected onto the customer's forehead, the shooting control unit 212 controls the shooting device C1 to automatically take a picture. The shooting control unit 212 may also use information from a previously taken image of the customer to set at least one of the shooting parameters of the shooting device C1 when the shooting device C1 takes an automatic picture. The shooting parameters are, for example, at least one of focus, exposure time, or gain, but are not limited to these. This allows the authentication device 200 to take a picture of the customer's face or iris in a state suitable for authentication when taking an automatic picture.

[0085] However, automatic shooting by the shooting device C1 under the control of the shooting control unit 212 is not required. For example, if the determination unit 208 determines that the projection information is properly projected onto the customer's forehead and the notification unit 210 outputs the notification information "Shooting OK", the staff recognizes that the customer is in a suitable position and orientation for shooting by visually checking the notification information. Then, the staff can operate the shooting device C1 to capture an image of the customer suitable for authentication.

[0086] The authentication unit 214 performs customer facial recognition or iris recognition by using the image acquired by the acquisition unit 202 at the timing when the shooting control unit 212 controls the shooting. The authentication unit 214 can automatically perform authentication when the shooting device C1 automatically performs shooting under the control of the shooting control unit 212.

[0087] When performing facial recognition, the authentication unit 214 detects the customer's face image in the captured image and calculates information about the customer's facial features based on that face image. The authentication unit 214 compares the calculated facial feature information with the facial feature information stored in the storage unit 216 by referring to the storage unit 216.

[0088] The memory unit 216 stores information on the facial features of service registrants (hereinafter also simply referred to as registrants). This facial feature information is information that can identify the registrant to be authenticated. This facial feature information is used for facial recognition, and for example, the positions of feature points such as the eyes, nose, and mouth in a facial image, and the distances between feature points are defined as "features."

[0089] As an example, the authentication unit 214 calculates the similarity between the calculated feature quantities and the face feature quantities stored in the memory unit 216. The memory unit 216 stores the face feature quantities of the registered user. If there are face feature quantities in the memory unit 216 with a similarity greater than a threshold, the authentication unit 214 determines that face authentication was successful. If the feature quantities are feature vectors, the authentication unit 214 calculates the distance between the calculated feature vectors and the face feature vectors in the memory unit 216. If there are face feature vectors in the memory unit 216 with a distance less than a threshold, the authentication unit 214 determines that the feature vectors match and that face authentication was successful. On the other hand, if there are no face feature quantities in the memory unit 216 with a similarity greater than a threshold, or no face feature vectors in the memory unit 216 with a distance less than a threshold, the authentication unit 214 determines that face authentication failed.

[0090] When performing iris authentication, the authentication unit 214 detects the customer's iris image in the captured image and calculates information on the customer's left and right eyes based on that iris image. The authentication unit 214 then refers to the storage unit 216 and compares the calculated information on the left and right eyes with the information on the registered user's left and right eyes stored in the storage unit 216, for each eye.

[0091] The authentication unit 214 calculates the similarity between the calculated feature quantities and the registered user's feature quantities for each eye. If there are registered user iris feature quantities with a similarity greater than the threshold for both the left and right eyes, the authentication unit 214 determines that iris authentication has been successful. The authentication unit 214 then determines that the person to be authenticated is the registered user. On the other hand, if there are no registered user iris feature quantities with a similarity greater than the threshold for either the left or right eye, the authentication unit 214 determines that iris authentication has failed.

[0092] If facial recognition or iris recognition is successful, the authentication unit 214 may acquire and use the registered person's personal information stored in the storage unit 216, which is linked to the registered person's characteristic information. Personal information includes, for example, information indicating the individual's attributes such as name, age, and gender, as well as payment-related information such as credit card information and account number.

[0093] The memory unit 216 may store information regarding the appropriate customer's location and orientation, as well as information regarding the projection state of appropriate projection information. The memory unit 216 may also store the registered user's feature quantities used in facial recognition or iris recognition. However, instead of the memory unit 216, the registered user's feature quantities used in facial recognition or iris recognition may be stored in a database (DB) outside the authentication device 200. The authentication unit 214 can access the DB to obtain the registered user's feature quantities and perform facial recognition or iris recognition.

[0094] [Flowchart Description] Figures 4A and 4B are flowcharts showing typical processing examples of the authentication device 200. These flowcharts provide an overview of the processing of the authentication device 200. Details of each process are as described above, so explanations will be omitted as appropriate.

[0095] The acquisition unit 202 acquires images taken by the customer from the shooting device C1 (step S22). The generation unit 204 generates projection information indicating the preferred position and orientation of the customer relative to the shooting device C1 (step S24). The projection unit 206 projects the projection information generated in step S24 (step S26).

[0096] The determination unit 208 determines whether the projection information is properly projected onto the customer's forehead based on the image of the customer and projection information acquired by the acquisition unit 202 (step S28). If the projection information is not properly projected (No. in step S28), the notification unit 210 generates notification information such that the projection information is projected onto the customer at a predetermined position and orientation (step S30). The projection unit 206 projects the notification information generated in step S30 (step S32).

[0097] After the processing in step S32, the authentication device 200 returns to step S22 and executes the process. In this manner, the authentication device 200 repeatedly executes the processes in steps S22 to S32 until the projection information is appropriately projected onto the customer's forehead. As the projection unit 206 projects the notification information, the information regarding the direction and distance of movement from the appropriate customer position to the customer position at detection, and the direction and angle of movement from the appropriate customer orientation to the customer orientation at detection, changes in response to the movement of the customer or the camera C1. By repeatedly executing the processes in steps S22 to S32, the authentication device 200 can generate and project notification information that follows the movement of the customer or the camera C1.

[0098] On the other hand, if the projection information is projected correctly (Yes in step S28), the notification unit 210 generates the text information "Shooting OK" as notification information (step S34). The projection unit 206 projects the notification information generated in step S34 onto the customer's forehead (step S36).

[0099] The shooting control unit 212 controls the shooting device C1 to automatically take a picture (step S38). The authentication unit 214 uses the image captured under the control of the shooting control unit 212 to perform facial recognition or iris recognition of the customer (step S40).

[0100] [Explanation of Effects] Authentication systems that photograph individuals and use the captured images for authentication are common in various fields. In this scenario, a scenario where an authenticator holds a camera in their hand and the authentication system uses the captured image is also conceivable. For authentication to be successful, it is desirable for the authenticator to accurately photograph the individual.

[0101] However, the certifier may not know the appropriate distance between the camera and the subject, or the orientation of the subject or camera, necessary for accurately capturing the subject. In particular, when iris recognition is performed, the distance between the camera and the subject, and the orientation of the subject or camera, required to enable authentication are within a limited range. Therefore, if the certifier is unfamiliar with the process, the time it takes from the start of shooting to capturing an image suitable for authentication may be long, potentially resulting in a longer authentication process.

[0102] In this disclosure, the authentication device 200 can perform notification based on the captured image of the customer and the projection information, so that the projection information is appropriately projected onto the customer. Staff can easily move the customer or the camera C1 to a position and orientation suitable for shooting according to the notification information. Furthermore, the authentication device 200 can reduce the time from the start of shooting to capturing an image suitable for authentication.

[0103] Furthermore, the determination unit 208 may determine whether the projection information is properly projected onto the customer's forehead. In other words, the determination unit 208 determines the state of projection of the projection information onto the customer's forehead. If the projection information is not properly projected onto the customer's forehead, the notification unit 210 provides notification to guide the person so that the projection information is projected onto the customer's forehead in the appropriate orientation. By setting the system so that the projection information is projected onto the customer's forehead under conditions suitable for shooting, staff can easily determine whether the customer is in a suitable state for shooting. Because staff can easily perform the shooting, the authentication device 200 can shorten the time from the start of shooting to capturing an image suitable for authentication.

[0104] Furthermore, the shooting control unit 212 may control the shooting device C1 to automatically photograph the customer when the projection information is appropriately projected onto the customer's forehead. Since staff do not need to photograph the customer, the authentication device 200 can shorten the time it takes to capture an image suitable for authentication.

[0105] Furthermore, the determination unit 208 may compare the projection state of the projection information displayed in the image with the projection state of the appropriate projection information. The notification unit 210 provides notification to change the orientation of the customer or the camera C1 based on the comparison result. By performing the comparison process, the authentication device 200 can provide more accurate notification to change the orientation of the customer or the camera C1 to the appropriate orientation.

[0106] Furthermore, the determination unit 208 may compare the position where the projection information displayed in the image is projected with the customer's position. Based on the comparison result, the notification unit 210 provides notification to change the position of the customer or the imaging device C1. By performing the comparison process, the authentication device 200 can provide more accurate notification to change the position of the customer or the imaging device C1 to an appropriate position.

[0107] Furthermore, the authentication unit 214 may perform iris authentication of the customer using the image captured by the imaging device C1 after notification by the notification unit 210. At this time, the generation unit 204 generates projection information indicating at least one of the customer's position or orientation relative to the imaging device C1 that is favorable for iris authentication. The projection unit 206 projects the generated projection information. The notification unit 210 also performs notifications used to guide the customer so that the projection information is projected at a predetermined position and orientation favorable for the customer's iris authentication. Therefore, even when performing iris authentication, the authentication device 200 can shorten the time from the start of shooting to capturing an image suitable for authentication.

[0108] The processing of the authentication device 200 shown in Embodiment 2 is applicable not only to iris recognition or facial recognition, but also to fingerprint, vein, or palm print recognition. This is because, in fingerprint, vein, or palm print recognition, it is assumed that the staff will move the customer or the imaging device C1 to an appropriate position and orientation when taking the authentication image.

[0109] Embodiment 3 In Embodiment 3, we will further describe the expected variations regarding notification. Note that we will omit explanations of points already described in Actual Embodiment 2 as appropriate, and focus on explaining the processing specific to Embodiment 3.

[0110] [Description of Configuration] Figure 5 is a block diagram showing another example of the authentication system according to this disclosure. In the authentication system S2 in Figure 5, an authentication device 300 is provided in place of the authentication device 200. The authentication system S2 also further includes an illuminance sensor L1. The authentication system S2 also includes a light-emitting device I1 in place of the display D1. However, the authentication system S2 may further include a display D1.

[0111] The illuminance sensor L1 is any type of sensor that detects illuminance and is installed around the customer. The illuminance data detected by the illuminance sensor L1 is output to the illuminance detection unit 302.

[0112] The light-emitting device I1 has multiple light-emitting elements. The light-emitting elements may be, for example, LEDs (Light Emitting Diodes), or other types of light-emitting elements. The light-emitting device I1 may have multiple light-emitting elements that emit the same color, or the multiple light-emitting elements may each emit a different color.

[0113] The authentication device 300 can change the light emission pattern of each light-emitting element of the light-emitting device I1 by controlling the light-emitting device I1. The multiple light emission patterns are patterns in which at least one of the following is different from one another: light emission color, light emission intensity, duration of light emission, duration of non-light emission, or the light-emitting parts in the light-emitting device I1, as shown in Embodiment 1.

[0114] The authentication device 300 includes an illuminance detection unit 302 in addition to the acquisition unit 202 to the storage unit 216 shown in Figure 3. The illuminance detection unit 302 detects the illuminance around the customer using data acquired from an illuminance sensor L1 installed around the customer. The notification unit 210 changes the notification mode according to the detected illuminance.

[0115] In detail, the notification unit 210 determines whether the detected illuminance is above a predetermined threshold. If the illuminance is above the predetermined threshold, the notification unit 210 changes the light emission color of one or more light-emitting elements of the light-emitting device I1 according to the determination result of the determination unit 208. On the other hand, if the illuminance is below the predetermined threshold, the notification unit 210 changes at least one of the light emission intensity, the duration of emission, or the duration of non-emission of one or more light-emitting elements of the light-emitting device I1 according to the determination result of the determination unit 208. In this way, the notification unit 210 sets the light emission mode used to guide customers so that the light emission mode can be identified from the perspective of the customer or holder.

[0116] As another example, the notification unit 210 may control the light-emitting device I1 to emit light in a predetermined first light color when the detected illuminance is above a predetermined threshold. The first light color may be at least one of the following: red, green, blue, orange, etc. On the other hand, if the detected illuminance is below a predetermined threshold, the notification unit 210 may control the light-emitting device I1 to emit light in a second light color different from the first light color. The second light color may be at least one of the following: white, yellow, yellow-green, etc. For example, the first light color is a color that is expected to be easily visible during the day, and the second light color is a color that is expected to be easily visible at night.

[0117] Furthermore, the notification unit 210 may have two or more thresholds to determine the detected illuminance. For example, if the detected illuminance is equal to or greater than the first threshold Th1, the notification unit 210 may change the emission color of one or more light-emitting elements of the light-emitting device I1 according to the determination result of the determination unit 208. If the detected illuminance is less than the first threshold Th1 and equal to or greater than the second threshold Th2 (Th1 > Th2), the notification unit 210 may change the emission intensity of one or more light-emitting elements of the light-emitting device I1 according to the determination result of the determination unit 208. If the detected illuminance is less than the second threshold Th2, the notification unit 210 may change the period during which the light-emitting device I1 emits light or does not emit light according to the determination result of the determination unit 208.

[0118] The notification unit 210 may perform the emission of light by the light-emitting device I1 and the notification of the notification information shown in Embodiment 2 in parallel.

[0119] [Explanation of Effects] As described above, the illuminance detection unit 302 detects the illuminance around the customer who is the subject of the photograph, and the notification unit 210 can change the notification method according to the detected illuminance. As a result, the authentication device 300 can perform notification in a manner that is easily understood by the person according to the illuminance. By having the person perform the actions necessary for photography according to the notification, the authentication device 300 can shorten the time from the start of shooting to the capture of an image suitable for authentication.

[0120] Furthermore, the notification unit 210 may provide notification used to guide customers by changing the light emission color of the light-emitting device I1 when the detected illuminance is above a predetermined threshold. On the other hand, the notification unit 210 may provide notification used to guide customers by changing at least one of the light emission intensity, the duration of emission, or the duration of non-emission of the light-emitting device I1 when the detected illuminance is below a predetermined threshold. The notification unit 210 can provide notification in a manner that is easily understood by people, regardless of whether the illuminance is bright or dark. As a result, the authentication device 300 can shorten the time from the start of shooting to the capture of an image suitable for authentication.

[0121] The authentication device 300 of Embodiment 3 may be configured as a single computer device, similar to the information processing device 100 of Embodiment 1, or as a distributed system having multiple computer devices. For example, some of the components of the authentication device 300 may be provided on other devices. Furthermore, some or all of the parts of the authentication device 300 may be provided on a cloud server built on the cloud, or on other types of virtualized servers generated using virtualization technology, etc.

[0122] Although this disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, which can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0123] In the embodiments described above, this disclosure has been described as a hardware configuration, but this disclosure is not limited thereto. This disclosure can also be realized by having a processor in a computer execute a computer program to perform the processing of the devices constituting the system described in the embodiments described above.

[0124] Figure 6 is a block diagram showing an example of the hardware configuration of an information processing device (in other words, a computer) on which the processing of the information processing device 100 according to Embodiment 1, or the authentication device according to Embodiment 2 or 3, is performed. Referring to Figure 6, the information processing device 90 includes a signal processing circuit 91, a processor 92, and a memory 93.

[0125] The signal processing circuit 91 is a circuit for processing signals in accordance with the control of the processor 92. The signal processing circuit 91 may also include a communication circuit for receiving signals from a transmitting device.

[0126] The processor 92 is connected to the memory 93 and performs the processing of the system described in the above embodiment by reading and executing a computer program from the memory 93. As an example of the processor 92, one of the following may be used: CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field-Programmable Gate Array), DSP (Digital Signal Processor), or ASIC (Application Specific Integrated Circuit), or multiple of these may be used in parallel.

[0127] The memory 93 is composed of volatile memory, non-volatile memory, or a combination thereof. There is not limited to one memory 93, but multiple memory units may be provided. The volatile memory may be, for example, RAM (Random Access Memory) such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The non-volatile memory may be, for example, ROM (Read Only Memory) such as PROM (Programmable Random Only Memory) or EPROM (Erasable Programmable Read Only Memory), flash memory, or SSD (Solid State Drive).

[0128] Memory 93 is used to store one or more instructions. Here, one or more instructions are stored in memory 93 as a program. The processor 92 can perform the processing described in the above embodiment by reading and executing these programs from memory 93.

[0129] The memory 93 may include not only memory located outside the processor 92, but also memory built into the processor 92. Furthermore, the memory 93 may include storage located separately from the processors that make up the processor 92. In this case, the processor 92 can access the memory 93 via an I / O (Input / Output) interface.

[0130] As described above, one or more processors in each system or device of the above embodiments execute one or more programs that include a set of instructions for causing a computer to perform the algorithm described with reference to the drawings. The information processing described in each embodiment can be realized by executing the programs.

[0131] The program, when loaded into a computer, includes a set of instructions or software code for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically or otherwise propagating signals. The temporary computer-readable medium or a communication medium can supply the program to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0132] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) An information processing device comprising: generation means for generating projection information indicating at least one of a preferred position or orientation for photographing a subject to be photographed on a photographing device for photographing the subject; projection means for projecting the information; and notification means used to guide a person so that the projected information is projected onto the subject to be photographed at a predetermined position and orientation, based on an image taken of the subject to be photographed and the projected information. (Note 2) The information processing device according to Note 1, further comprising determination means for determining the state of projection of the information onto the forehead of a person who is the subject to be photographed, and the notification means for guiding the person so that the information is projected onto the forehead of the subject to be photographed at a predetermined orientation, according to the determined projection state. (Note 3) The information processing device according to Note 2, further comprising shooting control means for controlling the photographing device to automatically photograph the subject when the information is projected onto the forehead of the subject to be photographed at a predetermined orientation. (Note 4) The information processing device further comprises comparison means for comparing the projection state of the information reflected in the image with an appropriate projection state of the information, and the notification means provides notification to change the orientation of the target to be photographed or the photographing device based on the comparison result, as described in any one of Notes 1 to 3. (Note 5) The information processing device further comprises comparison means for comparing the position on which the information reflected in the image is projected with the position of the target to be photographed, and the notification means provides notification to change the position of the target to be photographed or the photographing device based on the comparison result, as described in any one of Notes 1 to 4. (Note 6) The information processing device further comprises detection means for detecting the illuminance around the target to be photographed, and the notification means changes the mode of notification according to the illuminance, as described in any one of Notes 1 to 5.(Note 7) The information processing apparatus according to Note 6, wherein the notification means provides notification used to guide the person by changing the light emission color of the light-emitting device when the illuminance is above a predetermined threshold, and provides notification used to guide the person by changing at least one of the light emission intensity, the period of emission, or the period of non-emission of the light-emitting device when the illuminance is below a predetermined threshold. (Note 8) The information processing apparatus according to any one of Notes 1 to 7, wherein the information processing apparatus further comprises authentication means that performs iris authentication of the person being photographed using an image captured by the photographing device after notification by the notification means has been made, the generation means generates projection information indicating at least one of a preferred position or orientation for iris authentication of the subject being photographed, and the notification means provides notification used to guide the person so that the projected information is projected at a predetermined position and orientation preferred for iris authentication of the subject being photographed. (Note 9) A computer-based information processing method comprising: generating projection information for a camera that photographs a subject, indicating at least one of a preferred position or orientation for photographing the subject; projecting the information; and providing notification used to guide a person so that the projected information is projected at a predetermined position and orientation relative to the subject, based on an image of the subject and the projected information. (Note 10) A program that causes a computer to perform the following actions: generating projection information for a camera that photographs a subject, indicating at least one of a preferred position or orientation for photographing the subject; projecting the information; and providing notification used to guide a person so that the projected information is projected at a predetermined position and orientation relative to the subject, based on an image of the subject and the projected information.

[0133] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 8 that are dependent on Appendice 1 may also be dependent on Appendices 9 and 10 in the same manner as those described in Appendices 2 to 8. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0134] Although this disclosure has been described above with reference to embodiments, this disclosure is not limited to the embodiments described above. Various modifications to the structure and details of this disclosure are possible, which can be understood by those skilled in the art within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0135] 100 Information processing device 102 Generation unit 104 Projection unit 106 Notification unit S1, S2 Authentication system C1 Shooting device D1 Display L1 Illuminance sensor I1 Light-emitting device 200, 300 Authentication device 202 Acquisition unit 204 Generation unit 206 Projection unit 208 Judgment unit 210 Notification unit 212 Shooting control unit 214 Authentication unit 216 Storage unit 302 Illuminance detection unit

Claims

1. An information processing device comprising: generation means for generating projection information indicating at least one of a preferred position or orientation for photographing a subject to be photographed on a photographing device for photographing the subject; projection means for projecting the information; and notification means for providing notification used to guide a person so that the projected information is projected at a predetermined position and orientation relative to the subject to be photographed, based on an image taken of the subject and the projected information.

2. The information processing device further comprises determination means for determining the state of projection of the information onto the forehead of the person who is the subject of the photograph, and the notification means provides notification to guide the person so that the information is projected onto the forehead of the subject of the photograph in a predetermined orientation, according to the determined projection state, as described in claim 1.

3. The information processing apparatus according to claim 2, further comprising a shooting control means for controlling the shooting device to automatically photograph the target when the information is projected onto the forehead of the target in a predetermined orientation.

4. The information processing device further comprises a comparison means for comparing the projection state of the information reflected in the image with an appropriate projection state of the information, and the notification means provides notification to change the orientation of the object to be photographed or the photographing device based on the comparison result, according to any one of claims 1 to 3.

5. The information processing device further comprises a comparison means for comparing the position on which the information reflected in the image is projected with the position of the object to be photographed, and the notification means provides notification to change the position of the object to be photographed or the photographing device based on the comparison result, the information processing device according to any one of claims 1 to 4.

6. The information processing device further comprises detection means for detecting the illuminance around the object to be photographed, and the notification means changes the mode of notification according to the illuminance, as described in any one of claims 1 to 5.

7. The information processing apparatus according to claim 6, wherein the notification means provides notification used to guide the person by changing the light emission color of the light-emitting device when the illuminance is above a predetermined threshold, and provides notification used to guide the person by changing at least one of the light emission intensity, the period of emission, or the period of non-emission of the light-emitting device when the illuminance is below a predetermined threshold.

8. The information processing device further comprises authentication means that, after notification by the notification means, performs iris authentication of the person who is the subject of the photograph using an image taken by the photographing device, the generation means generates projection information indicating at least one of a position or orientation preferred for iris authentication of the subject of the photograph, and the notification means performs notification used to guide the person so that the projected information is projected at a predetermined position and orientation preferred for iris authentication of the subject of the photograph, according to any one of claims 1 to 7.

9. A computer-based information processing method comprising: generating projection information for a camera that photographs a subject, indicating at least one of a preferred position or orientation for photographing the subject; projecting the information; and providing guidance for a person, based on an image of the subject and the projected information, so that the projected information is projected at a predetermined position and orientation relative to the subject.

10. A program that causes a computer to perform the following actions: generate projection information indicating at least one of a preferred position or orientation for photographing a subject with respect to a photographing device that photographs the subject; project the information; and, based on the image obtained from the photograph of the subject and the projected information, provide guidance to a person so that the projected information is projected at a predetermined position and orientation relative to the subject.

Images