Facial recognition system, information processing device, and information processing method

Abstract

To provide a face authentication system, an information processor, and an information processing method that enable smooth face authentication.SOLUTION: A face authentication system has: a guidance processing part that performs a visual display to guide the gaze of a subject for face authentication in a direction of a camera; an acquisition part that acquires a face image that is a region of a face of the subject from a photographed image obtained by photographing the subject by the camera; and a face authentication part that executes face authentication processing on the subject based on a face image of a registrant registered in advance and the face image acquired by the acquisition part.SELECTED DRAWING: Figure 1

Description

[[Technical Field]]

[0001] The present disclosure relates to a face authentication system, an information processing apparatus, and an information processing method. [[Background Art]]

[0002] In recent years, the use of face authentication for personal authentication and person entry / exit management has been studied. For example, in facilities such as offices and schools where entry of pre-registered persons is permitted and entry of unregistered persons is restricted, studies have been conducted on using face authentication to authenticate that a person attempting to enter (e.g., an authentication subject) is an authorized person.

[0003] Patent Document 1 describes a face authentication system comprising: detection means for detecting an authentication subject; imaging means for capturing an image of the detected authentication subject and generating moving image data; feature extraction means for extracting facial features of the authentication subject based on the moving image data and generating feature information; authentication means for performing authentication based on the feature information; and guidance means for guiding the authentication subject to turn their face toward the imaging means for a predetermined period of time or longer when the feature extraction means fails to extract the facial features of the authentication subject. [[Prior Art Documents]] [[Patent Documents]]

[0004] [[Patent Document 1]] Japanese Unexamined Patent Application Publication No. 2010-218039 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0005] However, there remains room for study on methods for smoothly performing face authentication regardless of who the authentication subject (e.g., an entrant) is.

[0006] Non-limiting embodiments of this disclosure contribute to the provision of a facial recognition system, an information processing device, and an information processing method that can perform facial recognition smoothly. [Means for solving the problem]

[0007] A facial recognition system according to one embodiment of the present disclosure includes: a guidance processing unit that provides a visual display to guide the gaze of a person to be facially recognized toward the camera; an acquisition unit that acquires a facial image, which is the region of the face of the person, from an image captured by the camera; and a facial recognition unit that performs facial recognition processing for the person based on a pre-registered facial image of a registered person and the facial image acquired by the acquisition unit.

[0008] An information processing device according to one embodiment of the present disclosure includes: a guidance processing unit that causes a visual display on a display unit to guide the gaze of a person subject to facial recognition towards the camera; an acquisition unit that acquires a face image, which is the region of the face of the person subject, from an image captured by the camera; and a facial recognition unit that performs facial recognition processing for the person subject based on a pre-registered face image of a registered person and the face image acquired by the acquisition unit.

[0009] An information processing method according to one embodiment of the present disclosure displays a visual display on a display unit that guides the gaze of the subject of facial recognition towards the camera, acquires a facial image which is the region of the subject's face from an image captured by the camera, and performs facial recognition processing for the subject based on a pre-registered registered facial image and the facial image acquired by the camera.

[0010] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0011] Non-limiting embodiments of this disclosure enable smooth facial recognition.

[0012] Further advantages and effects of one embodiment of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by several embodiments and features described in the specification and drawings, but not all of them are necessarily provided in order to obtain one or more identical features. [Brief explanation of the drawing]

[0013] [Figure 1] A diagram showing an example of a facial recognition system according to Embodiment 1 of this disclosure. [Figure 2] A diagram showing an example of information registration. [Figure 3] A diagram showing an example of user information. [Figure 4] A flowchart showing an example of the processing flow according to Embodiment 1 of this disclosure. [Figure 5] Sequence diagram showing an example of the processing flow according to Embodiment 1 of this disclosure [Figure 6] Figure showing the first example of a guided image. [Figure 7] Figure showing a second example of a guided image. [Figure 8] This figure shows an example of how authentication results are displayed on the display unit. [Figure 9] A diagram showing an example of a facial recognition system according to Embodiment 2 of this disclosure. [Figure 10] A diagram showing an example of age and guided images. [Figure 11] A flowchart illustrating a first example of the processing flow according to Embodiment 2 of this disclosure. [Figure 12] Sequence diagram showing a first example of the processing flow according to Embodiment 2 of this disclosure [Figure 13] A flowchart illustrating a second example of the processing flow according to Embodiment 2 of this disclosure. [Figure 14A] Sequence diagram showing a second example of the processing flow according to Embodiment 2 of this disclosure [Figure 14B]Sequence diagram showing a variation of the second example of the processing flow according to Embodiment 2 of the present disclosure [Figure 15] Flowchart showing an example of the processing flow according to Variation 1 of Embodiment 2 of the present disclosure [Figure 16] Flowchart showing an example of the processing flow according to Variation 2 of Embodiment 2 of the present disclosure [Figure 17] Sequence diagram showing an example of the processing flow according to Variation 2 of Embodiment 2 of the present disclosure [Figure 18] Flowchart showing an example of the processing flow according to Variation 3 of Embodiment 2 of the present disclosure [Figure 19] Diagram showing an example of a face authentication system according to Embodiment 3 of the present disclosure [Figure 20] Diagram showing an example of information in an entry / exit management database [Figure 21] Sequence diagram showing an example of the processing flow according to Embodiment 3 of the present disclosure DETAILED DESCRIPTION OF EMBODIMENTS

[0014] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functions are denoted by the same reference numerals, and redundant description is omitted.

[0015] <Findings that led to the present disclosure> In facilities such as offices and schools where entry of pre-registered persons is permitted and entry of unregistered persons is restricted, the use of face authentication to authenticate that a person attempting to enter (e.g., a person to be authenticated) is an authorized person has been studied.

[0016] In a facial recognition system that authenticates individuals using facial recognition (hereinafter referred to as a facial recognition system), a camera captures an image including the facial region of the person to be authenticated, and an information processing device extracts facial information (e.g., facial features) from the captured image (hereinafter referred to as the captured image). The extracted facial information is then compared and matched with the facial information of individuals that have been registered in advance to determine whether or not the person to be authenticated is a person who has been registered in advance. In a facial recognition system, if the camera captures an image from the front direction with the person to be authenticated looking at the camera, facial information can be appropriately extracted from the captured image, allowing for smooth facial recognition.

[0017] Beyond offices and schools, there are plans to use facial recognition to authenticate whether a person being authenticated is an authorized individual (for example, a preschooler) in facilities attended by young children, such as kindergartens and daycare centers. However, because young children have difficulty consciously directing their gaze towards a camera, facial recognition may not work smoothly in facilities attended by young children.

[0018] For example, the facial recognition system described in Patent Document 1 includes guidance means that, when the feature extraction means fails to extract the facial features of the person to be authenticated, guides the person to face their face toward the imaging means for a predetermined period of time or longer.

[0019] However, since young children may not be able to read the message displayed or understand the meaning of the audio, the facial recognition system described in Patent Document 1 may result in them not facing the camera, which could hinder smooth facial recognition.

[0020] Therefore, in the following embodiments, we will describe a facial recognition system, an information processing device, and an information processing method that can perform facial recognition smoothly even when the person to be authenticated is a person who cannot read the message display or understand the meaning of the voice (for example, a young child).

[0021] The facial recognition systems described in the following embodiments are explained using an example of their implementation in a kindergarten, where facial recognition is used to identify which registered person a child is when they enter and / or leave the facility. In this example, the primary subjects of authentication are infants. Infants may also be referred to as children or kindergarteners.

[0022] Furthermore, in each of the following embodiments, the children belonging to the kindergarten, their guardians, and facility staff are pre-registered individuals and are referred to as "users" of the facial recognition system. The person whose face is to be recognized is referred to as the "person to be recognized." In other words, the facial recognition system shown in each of the following embodiments uses facial recognition to identify which of the users the person to be recognized is and to determine whether the identified user is a person permitted to enter or leave the kindergarten. In the following examples, all users are assumed to be persons permitted to enter or leave the kindergarten.

[0023] (Embodiment 1) <Example of system configuration> Figure 1 shows an example of a facial recognition system according to this embodiment 1. Figure 1 shows a server device 10, an authentication terminal 20 connected to the server device 10, and a registration terminal 30 connected to the server device 10. The server device 10 and the authentication terminal 20 may be directly connected by wire or wireless, or they may be connected via a network such as the Internet. Similarly, the server device 10 and the registration terminal 30 may be directly connected by wire or wireless, or they may be connected via a network such as the Internet.

[0024] The server device 10 may or may not be installed within the kindergarten. Based on the information received from the registration terminal 30 (which may be referred to as a user registration request), the server device 10 registers information related to facial recognition processing (which may be referred to as a user information). The server device 10 also performs facial recognition processing based on the registered user information and the information requesting facial recognition processing received from the authentication terminal 20 (which may be referred to as a facial recognition request).

[0025] The authentication terminal 20 is installed, for example, at the entrance of a kindergarten facility. The authentication terminal 20 may be stationary, portable, or mobile. The authentication terminal 20 is used when a person to be authenticated performs authentication. The authentication terminal 20 captures an area including the face of the person to be authenticated who is attempting to enter the facility. During image capture, the authentication terminal 20 displays a guidance image as an example of guidance processing to guide the person to be authenticated's gaze towards the camera. Based on the captured image, the authentication terminal 20 generates a face authentication request for one or more people to be authenticated included in the captured image and sends the face authentication request to the server device 10. The authentication terminal 20 receives the authentication result from the server device 10 and displays the authentication result.

[0026] The registration terminal 30 is a smartphone, tablet, PC (personal computer), etc., owned by the user. The registration terminal 30 is a terminal that the user (for example, a parent of a young child) operates and inputs the user's information. The registration terminal 30 transmits a user registration request to the server device 10, which includes personal information entered by the user and facial information extracted from images taken by the registration terminal 30. Note that the registration terminal 30 is not limited to one owned by the user. For example, the registration terminal 30 may be installed within the kindergarten. For example, the authentication terminal 20 and the registration terminal 30 may be the same terminal.

[0027] The server device 10 includes a communication control unit 101, a user registration unit 102, a facial recognition unit 103, and a storage unit 104. The user registration unit 102 and the facial recognition unit 103 of the server device 10 may be collectively referred to as the processing unit of the server device 10.

[0028] The communication control unit 101 controls communication between the server device 10 and the authentication terminal 20, and communication between the server device 10 and the registration terminal 30. For example, the communication control unit 101 receives a user registration request from the registration terminal 30 and outputs the received user registration request to the user registration unit 102. The communication control unit 101 also receives a facial recognition request from the authentication terminal 20 and outputs the received facial recognition request to the facial recognition unit 103. The communication control unit 101 may be divided into a transmitting unit that processes the transmission of signals (or information) and a receiving unit that processes the reception of signals (or information). Furthermore, the communication between the server device 10 and the authentication terminal 20, and the communication between the server device 10 and the registration terminal 30 may be wireless communication, wired communication, or a combination of both.

[0029] The user registration unit 102 registers users based on user registration requests obtained from the registration terminal 30. For example, the user registration unit 102 extracts facial information (e.g., facial features) from the facial image included in the user registration request. Extraction of facial information may be equivalent to conversion to facial information. The facial image corresponds to the facial region of a person extracted from an image containing a person. The user registration unit 102 issues identification information (hereinafter referred to as ID) to identify each user, generates user information including the user ID, the user's personal information, and the user's facial information, and registers the generated user information in the user information database 104a of the storage unit 104.

[0030] The facial recognition unit 103 performs facial recognition processing based on user information and a facial recognition request. The facial recognition unit 103 performs facial recognition processing based on the facial image of the person to be authenticated, which is included in the facial recognition request, and the facial image of the user that has been registered in advance. The facial recognition unit 103 extracts the facial information of the person to be authenticated from the facial image of the person to be authenticated, which is included in the facial recognition request. The facial recognition unit 103 compares the facial information of the person to be authenticated with the facial information registered as user information and calculates a score indicating the similarity of the facial information. The facial recognition unit 103 calculates a score for each user, and if the highest score among the calculated scores is above the threshold, it determines that the user corresponding to the highest score is the person to be authenticated. If a user who is the person to be authenticated is identified, it corresponds to successful authentication of the person to be authenticated. Also, if the highest score among the calculated scores is below the threshold, the facial recognition unit 103 determines that there is no user who is the person to be authenticated. If a user who is the person to be authenticated is not identified, it corresponds to authentication failure of the person to be authenticated.

[0031] The facial recognition unit 103 transmits information indicating either a successful or unsuccessful authentication result to the authentication terminal 20 via the communication control unit 101.

[0032] If the facial recognition unit 103 successfully authenticates, it may transmit information indicating the authentication result to a device (for example, a smartphone) owned by the guardian of the user who is the person to be authenticated, via the communication control unit 101.

[0033] The facial recognition unit 103 may also compare the facial image of the person to be authenticated with the facial image registered as user information and calculate a score indicating the similarity of the facial images.

[0034] The authentication terminal 20 includes a communication control unit 201, a person detection unit 202, a face information acquisition unit 203, a guidance image control unit 204, a camera 205, a display unit 206, and a storage unit 207. The person detection unit 202, the face information acquisition unit 203, and the guidance image control unit 204 of the authentication terminal 20 may be collectively referred to as the processing unit of the authentication terminal 20.

[0035] The communication control unit 201 controls communication between the server device 10 and the authentication terminal 20. The communication control unit 201 may be divided into a transmitting unit that processes the transmission of signals (or information) and a receiving unit that processes the reception of signals (or information).

[0036] The person detection unit 202 acquires images captured by the camera 205 (hereinafter referred to as "captured images"), performs image analysis on the captured images, and detects people included in the captured images.

[0037] The method for detecting a person is not particularly limited. For example, a person may be detected if, based on the captured image, known image recognition technology is used to detect that at least part of a face is included, or if a moving object is included in the captured image. Furthermore, a person may be detected using information other than the captured image. For example, a human presence sensor may be provided, and a person may be detected based on the detection information from the human presence sensor.

[0038] The facial information acquisition unit 203, when a person is detected by the person detection unit 202, extracts the facial region of the person included in the captured image and acquires a facial image. The person included in the captured image is considered to be a person to be authenticated.

[0039] The image used to acquire the face image may be the most suitable image among multiple images taken by camera 205. The method for determining the most suitable image is not particularly limited, but for example, among multiple images, the image in which the face is facing closest to the front, the face is the largest, the shape of the facial features (eyes, nose, mouth) is the best, the size of the facial features is the largest, and the clarity of the facial features is the best may be determined as the most suitable image.

[0040] The facial information acquisition unit 203 then generates a facial authentication request that includes the facial image of the person to be authenticated in order to determine who the person to be authenticated is (for example, whether or not they are a person authorized to enter the kindergarten (i.e., a user)), and sends the facial authentication request to the server device 10 via the communication control unit 201.

[0041] The guidance image control unit 204 controls the display of guidance images on the display unit 206. For example, the guidance image control unit 204 controls the display of a visual display on the display unit 206 that guides the gaze of the person being authenticated toward the camera 205. Based on the guidance images stored in the storage unit 207, the guidance image control unit 204 selects and changes guidance images. By displaying guidance images on the display unit 206, the guidance image control unit 204 guides the gaze of the person being authenticated toward the camera 205, enabling the camera 205 to photograph the person being authenticated from the front. In addition to controlling guidance images, the guidance image control unit 204 may also control guidance by voice or other means. Specific examples of guidance images will be described later.

[0042] Camera 205 is located near the display unit 206. For example, camera 205 is positioned to photograph the person being authenticated from the front as they look towards the display unit 206. Camera 205 captures a still image or a moving image and outputs the captured image to the face information acquisition unit 203. Alternatively, if a person is detected by the captured image, the captured image may be output to the person detection unit 202.

[0043] The display unit 206 displays information to be presented to the person being authenticated. For example, the display unit 206 displays a guidance image that directs the user's gaze to the camera. An example of the display on the display unit 206, including the guidance image, will be described later.

[0044] The memory unit 207 stores information related to the guidance images.

[0045] The registration terminal 30 includes a communication control unit 301, a face information acquisition unit 302, and a display unit 303.

[0046] The communication control unit 301 controls communication between the server device 10 and the registered terminal 30. The communication control unit 301 may be divided into a transmitting unit that processes the transmission of signals (or information) and a receiving unit that processes the reception of signals (or information).

[0047] The face information acquisition unit 302 extracts the facial region from the image captured by the camera (not shown in Figure 1) mounted on the registration terminal 30 using known face detection techniques and acquires a face image.

[0048] The display unit 303 displays information prompting the user operating the registration terminal 30 to register information.

[0049] The facial information acquisition unit 302 generates a user registration request that includes the acquired facial image and information entered by the user (for example, personal information such as name and age), and transmits the generated user registration request to the server device 10 via the communication control unit 301.

[0050] In the example described above, the authentication terminal 20 and the registration terminal 30 transmit information including a facial image to the server device 10, and the server device 10 extracts facial information from the facial image. However, this disclosure is not limited to this example. For example, the authentication terminal 20 and the registration terminal 30 may extract facial information from the facial image. In this case, the authentication terminal 20 and the registration terminal 30 may transmit information including facial information to the server device 10 instead of a facial image.

[0051] Note that the facial recognition system may include device configurations not shown in Figure 1, and the device configurations shown in Figure 1 may be excluded from the facial recognition system. For example, multiple authentication terminals 20 and / or multiple registration terminals 30 may be connected to the server device 10.

[0052] Furthermore, some of the device configurations shown in Figure 1 may be integrated with other device configurations. For example, the authentication terminal 20 and the registration terminal 30 may be a single device. In this case, users may register personal information, etc., using the authentication terminal 20 installed in the kindergarten.

[0053] Furthermore, some functions of the device configuration shown in Figure 1 may be omitted or included in other devices. For example, instead of having a camera 205 and a display unit 206, the authentication terminal 20 may be configured to connect to an external camera and a display unit. In this case, the authentication terminal 20 may control the shooting by the external camera and acquire images from the external camera, or it may control the display on the external display unit (for example, the display of guidance images).

[0054] Next, we will explain how to register user information using the registration terminal 30.

[0055] Figure 2 shows an example of information registration. In Figure 2, a child and their guardian are shown as users registering information using the registration terminal 30.

[0056] Parents enter personal information such as the child's (infant's) name, age, gender, and the name of the class at the kindergarten they belong to, via the registration terminal 30, and also enter their own personal information. In addition, parents operate the camera installed on the registration terminal 30 to take pictures of themselves and their child's faces.

[0057] The face information acquisition unit 302 of the registration terminal 30 acquires a face image from the image captured by the camera. The face information acquisition unit 302 then transmits a user registration request, which includes the acquired face image and information entered by the user (for example, personal information such as name and age), to the server device 10 via the communication control unit 301.

[0058] Figure 3 shows an example of user information. Figure 3 displays the user IDs, facial features, and personal information of each registered user (infant, guardian, and facility staff member).

[0059] As shown in Figure 3, users are distinguished into infants, guardians, and facility staff. Also, as shown in Figure 3, some users are associated with a family ID to identify their family and a facility staff ID to identify facility staff. An infant and their guardian are associated with the same family ID. Furthermore, if siblings belong to the same kindergarten, the siblings are associated with the same family ID.

[0060] The user registration unit 102 of the server device 10 generates a database of user information shown in Figure 3 based on the user registration request obtained from the registration terminal 30. The facial recognition unit 103 performs facial recognition processing based on the user information shown in Figure 3.

[0061] Figure 4 is a flowchart showing an example of the processing flow according to this embodiment 1. The flow shown in Figure 4 may start, for example, when the power of the authentication terminal 20 is turned on.

[0062] The person detection unit 202 of the authentication terminal 20 determines whether or not it was able to detect a person (step 101 (S101)).

[0063] If no person is detected (NO in S101), the process in S101 is executed again.

[0064] If a person is detected (YES in S101), the guidance image control unit 204 displays a guidance image on the display unit 206 (S102).

[0065] The camera 205 of the authentication terminal 20 captures a face (S103). The face information acquisition unit 203 acquires a face image from the captured image and sends a face authentication request including the acquired face image to the server device 10.

[0066] The facial recognition unit 103 of the server device 10 performs facial recognition processing and determines whether or not it has succeeded in authenticating the person to be authenticated in response to the facial recognition request (S104).

[0067] If authentication is successful (YES in S104), the facial recognition unit 103 generates an authentication result indicating that authentication was successful (S105).

[0068] If authentication is unsuccessful (NO in S104), the facial recognition unit 103 generates an authentication result indicating that authentication failed (S106).

[0069] The facial recognition unit 103 sends the authentication result to the authentication terminal 20 (S107). Then, the flow shown in Figure 4 ends. After the flow ends, the process in S101 starts again.

[0070] Next, we will explain the sequence of interactions between devices corresponding to the flowchart shown in Figure 4.

[0071] Figure 5 is a sequence diagram showing an example of the processing flow according to this embodiment 1.

[0072] After the person detection unit 202 of the authentication terminal 20 detects a person (after S201), the guidance image control unit 204 displays a guidance image on the display unit 206 (S202).

[0073] The facial information acquisition unit 203 acquires a facial image from the captured image (S203).

[0074] The facial information acquisition unit 203 generates a facial recognition request and sends the facial recognition request to the server device 10 (S204).

[0075] The facial recognition unit 102 of the server device 10 performs facial recognition processing (S205). The facial recognition unit 102 sends an authentication result indicating whether or not the facial recognition processing was successful to the authentication terminal 20 (S206).

[0076] The display unit 206 of the authentication terminal 20 displays the authentication result (S207). Then, the sequence shown in Figure 5 ends.

[0077] <Example of a guidance image> Next, we will explain an example of a guide image.

[0078] Figure 6 shows a first example of a guidance image. In the example in Figure 6, the display unit 206 displays the captured image (in the example in Figure 6, the image of the person being authenticated) taken by the camera 205, and the guidance image is displayed around the captured image.

[0079] The guidance image in Figure 6 includes multiple star symbols. In the example in Figure 6, the multiple star symbols are displayed sequentially in the direction of approaching camera 205 so that the gaze of the person being photographed (i.e., the person being authenticated) is directed towards camera 205.

[0080] For example, the guided images, numbered 1 through 5 stars, move along a trajectory from star 1 to star 5, reaching the position of camera 205.

[0081] As shown in Figure 6, a guidance image is displayed, which guides the person being authenticated to look towards camera 205, allowing camera 205 to capture an image of the person being authenticated from the front. By capturing an image from the front, facial recognition can be performed appropriately.

[0082] Figure 7 shows a second example of a guidance image. Figure 7 illustrates two guidance image patterns, Pattern A and Pattern B. In the two pattern examples in Figure 7, the display unit 206 displays the captured image taken by the camera 205, and the guidance image is displayed around the captured image.

[0083] The two guided images in Figure 7 include star trails similar to those in Figure 6, as well as a face frame to decorate the face. The face frame is displayed in the area surrounding the position where the face appears in the captured image (approximately the center in the example in Figure 7).

[0084] As shown in Figure 7, a guidance image is displayed, causing the person being authenticated to consciously change their posture and gaze so that their face is within the guidance image. As a result, the person being authenticated is guided to direct their gaze towards the camera 205, allowing the camera 205 to capture an image of the person being authenticated from the front. By capturing an image from the front, facial recognition can be performed appropriately. The guidance image control unit 204 may also control the display of the guidance image based on the configuration of the authentication terminal 20 (for example, the position where the camera 205 is installed, the size and shape of the display unit 206). For example, the information on the position where the camera 205 is installed is the relative position of the camera 205 with respect to the authentication terminal 20. The information on the relative position of the camera 205 with respect to the authentication terminal 20 may be, for example, above, below, or diagonally to the upper right of the authentication terminal 20. The information on the position where the camera 205 is installed may be stored in the storage unit 207 as a setting file for the authentication terminal 20 (not shown). The guidance image control unit 204 may obtain a configuration file containing information about the location where the camera 205 is installed from the storage unit 207, and control the position of the guidance image, the size of the guidance image, the direction of movement of the guidance image, etc. This makes it possible to handle cases where the position of the camera 205 is variable, and cases where the camera 205 is configured separately from the authentication terminal 20 and installed externally.

[0085] Note that the guidance images are not limited to the examples described above. For example, the guidance images do not have to be fixed and may be changed. Also, they only need to be able to guide the gaze of the person being authenticated. The following describes variations in guidance images and variations in guidance processes, including guidance other than images.

[0086] <Variations of induction> For example, the guidance image may be changed daily. Different character images may be set as the guidance image for each day of the week.

[0087] For example, the face frame shown in Figure 7 may be replaced with the character's face frame.

[0088] For example, an animation may be displayed on the display unit 206, and the animation may guide the gaze of the person being authenticated.

[0089] Additionally, for example, the display may be made to move during the first few seconds after an object is detected, thereby guiding the gaze of the person being authenticated.

[0090] Furthermore, for example, the capture of images for facial recognition and the taking of facial photos for recreational purposes may be performed in an integrated manner. For example, an action for taking a facial photo may be performed, and facial recognition may be performed at the same time as a photograph of the person to be authenticated is taken. Before taking the photograph, a countdown indicating the timing of the photograph may be displayed. Alternatively, an audio countdown may be output.

[0091] Furthermore, game elements may be added to guide the gaze of the person being authenticated. A game may be displayed on the display unit 206 that guides the gaze towards the camera 205.

[0092] Furthermore, the display unit 206 may display text information that is updated daily (for example, fortune-telling information).

[0093] Alternatively, a touch panel-type operation unit may be superimposed on the display unit 206, and information prompting operation on the operation unit (for example, an image of a character) may be displayed on the display unit 206. In this case, when the infant being authenticated touches the image of the character displayed on the display unit 206, the infant's gaze is directed towards the camera 205, enabling proper facial recognition.

[0094] Furthermore, the display unit 206 may show information that the child, who is the subject of authentication, should check upon entering the room. This helps the child to become accustomed to checking the information upon entering the room, thereby directing their gaze towards the camera 205 and enabling proper facial recognition.

[0095] Furthermore, objects that children may be interested in (e.g., stuffed animals, dolls, toys, etc.) may be placed around the authentication terminal 20. This allows the child to be guided towards the objects of interest, directing their gaze towards the camera 205 and enabling appropriate facial recognition processing. In addition, the objects that children may be interested in may be equipped with sensors (heat-sensing sensors or contact-sensing sensors), and the camera 205 may take a picture when it is detected that the child has touched or approached the object. Alternatively, the camera 205 may be mounted on the objects that children may be interested in.

[0096] Furthermore, the display unit 206 may display a standby screen, and when an object is detected, it may change from displaying the standby screen to displaying a specific image (for example, a specific animation). The specific animation may, for example, prompt a child to play an activity that will capture their interest (for example, "peek-a-boo"). In this case, while the specific animation is displayed on the display unit 206, the child's gaze will be directed towards the camera 205, allowing for proper facial recognition.

[0097] In addition to displaying guidance images on the display unit 206, guidance may also be provided through audio output and / or image movement. For example, if a guidance image is displayed on the display unit 206 and the infant touches the guidance image, an audio output may be generated and the guidance image may start moving as an animation. This directs the infant's gaze towards the camera 205, enabling proper facial recognition.

[0098] Furthermore, the content of the guidance image on the display unit 206 may be changed depending on the date and time. For example, on days when specific events are held (e.g., Hina Matsuri, Children's Day, sports day, etc.), an image corresponding to the specific event (e.g., Hina dolls, carp streamers, etc.) may be set as the guidance image.

[0099] Additionally, for example, a special display (e.g., a birthday message) may be added to the birthday of a registered user who qualifies as a verified user. Alternatively, a special audio (e.g., a birthday song) may be played.

[0100] <Example of how authentication results are displayed> Figure 8 shows an example of how the authentication result is displayed on the display unit. Figure 8 shows patterns A and B of the authentication result display when authentication is successful.

[0101] As shown in Pattern A of Figure 8, if authentication is successful, a message may be displayed to the authenticated person. In addition, audio may be output along with the message.

[0102] Furthermore, as shown in Pattern B of Figure 8, a character may be displayed along with a message to the person being authenticated. The character is not particularly limited, but may be a character of the person being authenticated's preference that has been registered in advance. In addition, sound may be output along with the message and the character.

[0103] In this embodiment 1, the face recognition system includes a guidance image control unit 204 (an example of a guidance processing unit) that provides a visual display to guide the gaze of the person to be authenticated toward the camera 205, a face information acquisition unit 203 (an example of an acquisition unit) that acquires a face image, which is the facial region of the person to be authenticated, from an image captured by the camera 205, and a face recognition unit 103 that performs face recognition processing for the person to be authenticated based on a pre-registered face image of a registered person and the face image of the person to be authenticated acquired by the face information acquisition unit 203.

[0104] According to the above-described embodiment 1, even when the person to be authenticated is a child who cannot read the message display or understand the meaning of the voice, facial recognition can be performed smoothly by displaying a specific guidance image to guide the person's gaze toward the camera 205.

[0105] In this embodiment 1, the facial recognition system has an authentication terminal 20 (an example of a first information processing device) and a server device 10 (an example of a second information processing device), and the authentication terminal 20 has a communication control unit 201, a person detection unit 202, a facial information acquisition unit 203, a guidance image control unit 204, a camera 205, a display unit 206, and a storage unit 207, and the server device 10 has a communication control unit 101, a user registration unit 102, a facial recognition unit 103, and a storage unit 104. However, this disclosure is not limited to these examples. For example, the facial recognition system may be configured by a single information processing device that includes the configuration of the authentication terminal 20 and the configuration of the server device 10 described above. Alternatively, the facial recognition system may have two or more information processing devices, and the configuration of the authentication terminal 20 and the configuration of the server device 10 described above may be included in any of the two or more information processing devices.

[0106] For example, the face recognition unit 103 may not be included in the server device 10, but in the authentication terminal 20. In this case, the authentication terminal 20 includes a guidance image control unit 204 (an example of a guidance processing unit) that provides a visual display to guide the gaze of the person to be authenticated toward the camera 205, a face information acquisition unit 203 (an example of an acquisition unit) that acquires a face image, which is the region of the face of the person to be authenticated, from an image captured by the camera 205, and a face recognition unit that performs face recognition processing for the person to be authenticated based on a pre-registered face image of a registered person and the face image of the person to be authenticated acquired by the face information acquisition unit 203.

[0107] (Embodiment 2) <Example of system configuration> Figure 9 shows an example of a facial recognition system according to this second embodiment. In Figure 9, components similar to those in Figure 1 are given the same reference numerals and their descriptions may be omitted. Figure 9 shows a server device 40, an authentication terminal 20 connected to the server device 40, and a registration terminal 30.

[0108] Server device 40 is an addition to server device 10 shown in Figure 1, with the addition of an age estimation unit 401, a database of age and guidance images 402, and a holding / carrying determination unit 403. The following describes these added configurations.

[0109] The age estimation unit 401 estimates the age of the person to be authenticated. For example, the age estimation unit 401 acquires a captured image including the person to be authenticated from the authentication terminal 20 and estimates the age of the person to be authenticated based on image analysis of the acquired captured image.

[0110] The method for estimating age is not particularly limited. For example, the age estimation unit 401 may estimate age based on at least a portion of the facial features of the person being authenticated included in the captured image. Alternatively, the age estimation unit 401 may estimate age based on at least one of the size of the person being authenticated included in the captured image (e.g., height) and the posture of the person being authenticated (e.g., whether or not they can walk upright and the speed of their movement). Alternatively, age estimation may be performed based on learning using AI (Artificial Intelligence).

[0111] The age estimation unit 401 refers to the age and guidance image database 402 and determines a guidance image corresponding to the estimated age. The age estimation unit 401 transmits the determined guidance image to the authentication terminal 20 via the communication control unit 101.

[0112] The guidance image control unit 204 of the authentication terminal 20 may control the display of guidance images acquired from the server device 40. The guidance image control unit 204 may also determine the content of the display according to the estimated age of the person being authenticated. For example, the age and guidance image database 402 may be included in the authentication terminal 20 instead of the server device 40. The guidance image control unit 204 may then acquire information on the estimated age of the person being authenticated from the age estimation unit 401, refer to the age and guidance image database 402, and determine a guidance image corresponding to the estimated age of the person being authenticated.

[0113] The age and guided image database 402 is a database that stores ages in association with age-appropriate guided images. Age-appropriate guided images may be registered in advance.

[0114] The holding determination unit 403 determines whether the infant being authenticated is being held by their guardian. The state in which the infant is being held by their guardian will be simply referred to as "being held" below. For example, the holding determination unit 403 acquires a captured image including the person being authenticated from the authentication terminal 20 and determines whether the infant is being held based on image analysis of the acquired captured image.

[0115] The method for determining whether someone is being held is not particularly limited. For example, the holding determination unit 403 may determine whether there are two people in the captured image based on the size and position of the facial features included in the captured image, and if there are two people, it may determine whether the two people are in a holding position based on the relative positions of their facial features. Alternatively, the holding determination unit 403 may determine whether there are two people in the captured image based on the silhouette (or posture) of the person included in the captured image, and if there are two people, it may determine whether the two people are in a holding position based on their silhouettes. Alternatively, the holding determination may be performed based on learning using AI.

[0116] The baby-carrying detection unit 403 transmits information to the authentication terminal 20 indicating that the baby is being carried if it is determined that the baby is being carried. Conversely, the baby-carrying detection unit 403 transmits information to the authentication terminal 20 indicating that the baby is not being carried if it is determined that the baby is not being carried.

[0117] When the face information acquisition unit 203 receives information indicating that a person is being held (hereinafter sometimes referred to as the "holding determination result"), it acquires a face image from the face of at least one of the two people being held in the captured image. The target for acquiring a face image is determined based on predetermined criteria. For example, the target for acquiring a face image may be the larger face, the face from which it is easier to acquire a face image from the captured image, the face with a relatively large number of facial features, the face of the older person (e.g., the guardian), or the face of the younger person (e.g., the infant). The age may be estimated by the age estimation unit 401. If the captured image contains two or more people, the face image acquisition unit 203 may acquire face images of each of the two or more people.

[0118] The facial information acquisition unit 203 then generates a facial authentication request that includes the facial image of the person to be authenticated in order to determine who the person to be authenticated is (for example, whether or not they are a person authorized to pass through), and transmits the facial authentication request to the server device 40 via the communication control unit 201.

[0119] The facial recognition unit 103 performs facial recognition processing based on user information and facial recognition requests. When the facial recognition unit 103 identifies a user who is the target of authentication, it determines whether the two people being held are family members based on the identified user's family ID. If the two people being held are family members, the facial recognition unit 103 determines that the infant being authenticated has been successfully recognized. The facial recognition unit 103 may also determine whether the identified user is an infant (preschooler) belonging to a kindergarten based on the identified user's user information.

[0120] Furthermore, if the carrying detection unit 403 determines that there are two or more people in the captured image and that they are not being carried, for example, if two or more people are lined up in the captured image, the face information acquisition unit 203 receives information indicating that the multiple people are not being carried. In this case, the face information acquisition unit 203 acquires a face image from the face of at least one of the multiple people included in the captured image. The target for acquiring a face image may be, for example, a large face, a face from which a face image can be easily acquired from the captured image, a face with a relatively large number of facial features, or the face of an older person or a younger person among the multiple people.

[0121] Furthermore, if it is determined that there are two or more people in the captured image and that they are not being held, and the facial recognition unit 103 identifies at least one user who is a target for authentication, it determines whether the identified user is a guardian or a facility staff member based on the user information of the identified user. If the facial recognition unit 103 determines that the identified user is a guardian or a facility staff member (an example of a second registered user), it instructs the authentication terminal 20 to display a message prompting the user to perform facial recognition of the child (an example of a first registered user).

[0122] In the example shown in Figure 9, both the age estimation unit 401 and the holding determination unit 403 are included in the server device 40. However, either the age estimation unit 401 or the holding determination unit 403 may be included in the server device 40. Alternatively, the age estimation unit 401 and / or the holding determination unit 403 may be included in the authentication terminal 20 instead of the server device 40.

[0123] Furthermore, in the example described above, the carrying detection unit 403 determined whether or not there were two or more people in the captured image. However, the authentication terminal 20 (for example, the person detection unit 202) may also determine whether or not there were two or more people in the captured image. If there were two or more people in the captured image, the person detection unit 202 may request a carrying detection from the carrying detection unit 403 of the server device 40.

[0124] <Database of age and guided images> Next, we will explain the information stored in the database 402, which contains age and guided images.

[0125] Figure 10 shows an example of age and guidance images. Figure 10 shows examples of guidance images associated with ages under 3 years old and ages 3 years and older. In Figure 10, touch sounds are also associated with the guidance images as an example. The touch sound is, for example, the sound output when the person being authenticated operates the authentication terminal 20. Furthermore, in Figure 10, the guidance images and touch sounds are set for each day of the week as an example.

[0126] The information shown in Figure 10 is stored in the age and guide image database 402. Then, as described above, the age estimation unit 401 refers to the age and guide image database 402 to determine the guide image corresponding to the estimated age.

[0127] For example, if the age estimation unit 401 determines that the person to be authenticated is 2 years old on a Monday, it will select a "chick" as the guidance image. The age estimation unit 401 will then send the "chick" image to the authentication terminal 20 as the guidance image. In this case, the age estimation unit 401 will also send "sound 11" to the authentication terminal 20 as the touch sound.

[0128] By setting appropriate guidance images according to age, it is possible to more effectively guide the gaze of young children and perform facial recognition smoothly.

[0129] Next, the processing flow according to this second embodiment will be described. In the following, as the first example, an example in which the age estimation unit 401 performs age estimation will be described, and as the second example, an example in which the carrying determination unit 403 performs carrying determination will be described.

[0130] <Example 1 (Example of age estimation)> Figure 11 is a flowchart showing a first example of the processing flow according to this second embodiment. In Figure 11, the same processes as in Figure 4 are given the same reference numerals and their descriptions are omitted. The flow shown in Figure 11 may start, for example, when the power of the authentication terminal 20 is turned on.

[0131] The person detection unit 202 of the authentication terminal 20 determines whether or not it was able to detect a person (S101).

[0132] If the person detection unit 202 of the authentication terminal 20 detects a person (YES in S101), the person detection unit 202 of the authentication terminal 20 transmits the captured image to the server device 40, and the age estimation unit 401 of the server device 40 estimates the age of the person to be authenticated (S301).

[0133] The age estimation unit 401 determines a guidance image based on the estimated age (S302). The age estimation unit 401 transmits the determined guidance image to the authentication terminal 20.

[0134] The guidance image control unit 204 displays the guidance image received from the server device 40 (S303). Then, the processing from S103 onwards is executed. The process from S103 onwards is the same as in Figure 4, so the explanation is omitted.

[0135] Next, we will explain the sequence of interactions between devices corresponding to the flowchart shown in Figure 11.

[0136] Figure 12 is a sequence diagram showing a first example of the processing flow according to this second embodiment. In Figure 12, the same processes as in Figure 5 are given the same reference numerals as in Figure 5, and their explanations are omitted.

[0137] After the person detection unit 202 of the authentication terminal 20 detects a person (after S201), the person detection unit 202 sends information requesting age estimation (age estimation request) to the server device 40 (S401).

[0138] The age estimation unit 401 of the server device 40 performs age estimation (S402). The age estimation unit 401 determines a guidance image corresponding to the estimated age (S403). Then, the age estimation unit 401 transmits the determined guidance image (S404).

[0139] The guidance image control unit 204 displays the received guidance image (S202). Then, processing from S203 onwards is executed. The process from S203 onwards is the same as in Figure 5, so the explanation is omitted.

[0140] <Example 2 (Example of determining whether to hold the baby)> Figure 13 is a flowchart showing a second example of the processing flow according to this second embodiment. In Figure 13, the same processes as in Figure 4 are given the same reference numerals and their descriptions are omitted. The flow shown in Figure 13 may start, for example, when the power of the authentication terminal 20 is turned on.

[0141] The person detection unit 202 of the authentication terminal 20 determines whether or not it was able to detect a person (S101).

[0142] If the person detection unit 202 of the authentication terminal 20 can detect a person (YES in S101), the person detection unit 202 determines whether or not there are two or more people detected (S501).

[0143] If the number of detected individuals is less than two (NO in S501), for example, if only one individual is detected, the same processing as that shown in Figure 4 from S102 onwards is executed.

[0144] If two or more people are detected (YES in S501), the person detection unit 202 transmits the captured image to the server device 40, and the holding determination unit 403 of the server device 40 determines whether or not two or more people are being held (S502).

[0145] If the person is being held (YES in S502), the holding determination unit 403 sends information indicating that the person is being held to the authentication terminal 20, and the camera 205 of the authentication terminal 20 takes a picture of the face (S503). The face information acquisition unit 203 may acquire a face image of at least one of the two people being held (for example, the face of the person with the larger face size) from the captured image, and send a face authentication request including the acquired face image to the server device 40.

[0146] The facial recognition unit 103 of the server device 40 performs facial recognition processing and determines whether or not it has succeeded in facial recognition of the person to be authenticated in response to the facial recognition request (S504).

[0147] If authentication is successful (YES in S504), the facial recognition unit 103 determines whether the user identified as the person to be authenticated is a child or not (S505). For example, if the user information of the identified user includes information indicating that the user is a child (e.g., age, kindergarten class, etc.), the unit may determine that the identified user is a child.

[0148] If the user identified as the person to be authenticated is an infant (YES in S505), the facial recognition unit 103 determines that the authentication of the infant themselves was successful (S506). After S506, the facial recognition unit 103 transmits an authentication result indicating successful authentication (S510).

[0149] If the user identified as the person to be authenticated is not an infant (NO in S505), the facial recognition unit 103 determines whether the user identified as the person to be authenticated has a family ID (S507). In this case, the person to be authenticated is not an infant, but an infant's guardian or facility staff member. The person to be authenticated who has a family ID is the infant's guardian, and the person to be authenticated who does not have a family ID is a facility staff member.

[0150] If a family ID is available (YES in S507), that is, if the user identified as the person to be authenticated is the guardian of the infant, the facial recognition unit 103 considers the successful authentication of the guardian among the two people holding the infant as the successful authentication of the infant themselves (i.e., the guardian's child) (S508).

[0151] After S508, the facial recognition unit 103 transmits an authentication result indicating successful authentication (S510).

[0152] If two or more people are not holding the child (NO in S502), if facial recognition is unsuccessful (NO in S504), or if the person does not have a family ID (NO in S507), the facial recognition unit 103 determines that authentication has failed (S509).

[0153] After S509, the facial recognition unit 103 sends an authentication result indicating authentication failure (S510). Then, the flow shown in Figure 13 ends.

[0154] Next, we will explain the sequence of interactions between devices corresponding to the flowchart shown in Figure 13.

[0155] Figure 14A is a sequence diagram showing a second example of the processing flow according to this second embodiment. Figure 14A is a sequence diagram of the case in which it is determined whether or not a user identified as an authentication target in S507 of Figure 13 has a family ID. Note that in Figure 14A, the explanation of some of the processes in Figure 13 may be omitted.

[0156] After the person detection unit 202 of the authentication terminal 20 detects two or more people (after S601), the person detection unit 202 sends a request to the server device 40 to determine if the person is being held, including the captured image (S602).

[0157] The holding detection unit 403 of the server device 40 performs a holding detection (S603).

[0158] The baby-carrying detection unit 403 transmits the baby-carrying detection result to the authentication terminal 20 (S604).

[0159] The face information acquisition unit 203 acquires a face image from the captured image, from the face of at least one of the two people who are being held (S606). The face information acquisition unit 203 sends a face authentication request containing the acquired face information to the server device 40 (S607).

[0160] The facial recognition unit 102 of the server device 40 performs facial recognition processing (S608). The facial recognition unit 102 authenticates the child of the person to be authenticated based on the family ID (S609). The facial recognition unit 102 sends an authentication result indicating whether or not the facial recognition processing was successful to the authentication terminal 20 (S610).

[0161] The display unit 206 of the authentication terminal 20 displays the authentication result (S611). Then, the sequence shown in Figure 14 ends.

[0162] In the second example, facial recognition is performed on at least one of the two or more individuals to be authenticated, but not on the remaining individuals. In this case, for example, after S611, information prompting the authenticated individual whose facial recognition was successful (e.g., the guardian) to perform facial recognition on the remaining individuals (e.g., the infant) may be displayed on the display unit 206.

[0163] Figure 14B is a sequence diagram showing a variation of a second example of the processing flow according to this second embodiment. Figure 14B is a sequence diagram for the case where the user identified as the person to be authenticated in S505 of Figure 13 is an infant. Note that in Figure 14B, the explanation of some of the processes in Figure 13 may be omitted. Also, in Figure 14B, processes similar to those in Figure 14A are given the same numbering and their explanations are omitted.

[0164] The facial recognition unit 102 of the server device 40 performs facial recognition processing (S608). The facial recognition unit 102 authenticates that the person to be authenticated is the infant (S621). The facial recognition unit 102 transmits the authentication result indicating that the person to be authenticated is the infant to the authentication terminal 20 (S610).

[0165] In the second example described above, the server device 10 determines whether the first person to be authenticated (e.g., a guardian) is holding the second person to be authenticated (e.g., the guardian's child) among the multiple persons to be authenticated included in the captured image. If the server device 10 determines that the first person to be authenticated is holding the second person to be authenticated and the authentication of the first person to be authenticated is successful, then the server device 10 determines whether the first person to be authenticated has a parent-child ID. Having a parent-child ID means that the first person to be authenticated is the guardian of one of the users. If the first person to be authenticated has a parent-child ID, the server device 10 determines that the second person to be authenticated is an infant who has registered the first person to be authenticated as their guardian.

[0166] When a child is being held, they may not be able to face camera 205 properly, for example, if they have fallen asleep, which could prevent proper facial recognition. Furthermore, even if a facial image of the held child is acquired, facial recognition based on the facial information may fail. According to the second example, even in situations where facial recognition is not possible properly, such as when a child is being held, the system can still make favorable decisions and perform facial recognition smoothly.

[0167] In the following, this second embodiment will describe variations of facial recognition in various cases, in addition to the two examples described above.

[0168] <Variation 1: Cases where two or more people are photographed> Variation 1 involves cases where two or more people are photographed, and the captured image includes two or more people (for example, the person to be authenticated). In Variation 1, if the captured image includes two people, facial recognition is performed on the face of the person with the larger face size.

[0169] Figure 15 is a flowchart showing an example of the processing flow related to variation 1 of this embodiment 2. In Figure 15, the same processing as in Figure 4 is given the same reference numerals as in Figure 4, and its explanation is omitted. The flow shown in Figure 15 may start, for example, when the power of the authentication terminal 20 is turned on.

[0170] The person detection unit 202 of the authentication terminal 20 determines whether or not it was able to detect a person (S101).

[0171] If the person detection unit 202 of the authentication terminal 20 can detect a person (YES in S101), the person detection unit 202 determines whether or not there are two or more people detected (S701).

[0172] If the number of detected individuals is less than two (NO in S701), for example, if only one individual is detected, the same processing as that shown in Figure 4 from S102 onwards is executed.

[0173] If two or more people are detected (YES in S701), the camera 205 of the authentication terminal 20 takes a picture of their faces (S702). In this case, the captured image may include the faces of two or more people.

[0174] The face information acquisition unit 203 compares the face sizes of two or more people from the captured image and detects the one with the larger face size (S703). The face information acquisition unit 203 then acquires the face information of the person with the larger face size and sends a face authentication request containing the acquired face information to the server device 40.

[0175] The facial recognition unit 103 of the server device 40 performs facial recognition processing and determines whether or not it has succeeded in facial recognition of the person to be authenticated in response to the facial recognition request (S704).

[0176] If authentication is successful (YES in S704), the facial recognition unit 103 generates an authentication result indicating that authentication was successful (S705).

[0177] If authentication is unsuccessful (NO in S704), the facial recognition unit 103 generates an authentication result indicating that authentication failed (S706).

[0178] The facial recognition unit 103 transmits the authentication result to the authentication terminal 20 (S707). Then, the flow shown in Figure 15 is completed.

[0179] In the flow shown in Figure 15, facial recognition is performed on at least one of the two or more people to be authenticated, but not on the remaining people. In this case, for example, after S707, information prompting the person whose facial recognition was successful (e.g., the guardian) to perform facial recognition on the remaining people (e.g., the infant) may be displayed on the display unit 206.

[0180] In the flow shown in Figure 15, facial recognition is performed on at least one of the two or more people to be authenticated, but not on the remaining people. In this case, after successful authentication (after YES in S704), a family ID determination may be performed, similar to S505 in Figure 13. As a result, the facial recognition unit 103 may consider that two or more people included in the captured image are family members of the person who was successfully authenticated, and determine that authentication was successful.

[0181] Furthermore, while the flow shown in Figure 15 illustrates an example where the larger face size is detected among two or more people in S703, this disclosure is not limited to this. For example, the smaller face size may be detected among two or more people. Alternatively, the younger age may be detected among two or more people. For example, if an infant and their guardian are included among two or more people, the infant can be authenticated as the person to be authenticated by detecting the person with the smaller face size or the younger age among the two or more people. Note that age estimation may be performed by the age estimation unit 401 of the server device 40, as shown in Figure 9.

[0182] Furthermore, while the flow shown in Figure 15 illustrates an example where facial recognition is performed on at least one person out of two or more individuals, this disclosure is not limited to this. For example, facial recognition processing for two or more individuals may be performed simultaneously or in parallel. Alternatively, facial recognition processing for two or more individuals may be performed at high speed using time-sharing. In this case, instead of the process in S703 of Figure 15 which detects the larger face, the facial images of each of the two or more individuals may be acquired, and a facial recognition request containing each facial image may be sent to the server device 10. The server device 10 may then perform facial recognition processing for each of the multiple facial images.

[0183] <Variation 2: Case of a toddler with siblings> In early childhood, there may be little individual variation in facial information (e.g., facial features). In particular, facial information may be similar among siblings in early childhood. Variation 2 describes facial recognition of such siblings when they attend the same kindergarten.

[0184] Figure 16 is a flowchart showing an example of the processing flow related to variation 2 of this embodiment 2. In Figure 16, the same processing as in Figure 4 is given the same reference numerals as in Figure 4, and its explanation is omitted. The flow shown in Figure 16 may start, for example, when the power of the authentication terminal 20 is turned on.

[0185] The camera 205 of the authentication terminal 20 captures a face (S103). The face information acquisition unit 203 acquires face information from the captured image and sends a face authentication request containing the acquired face information to the server device 40.

[0186] The face recognition unit 103 of the server device 40 compares the face information of the person to be authenticated with the face information of each user during the face recognition process and calculates the similarity. The face recognition unit 103 then determines whether the similarity of the face with the highest similarity (Face Matching Rank 1) is above a threshold (S901).

[0187] If the similarity of the first-ranked face in face matching is above the threshold (YES in S901), the face recognition unit 103 determines whether the difference between the similarity of the first-ranked face with the highest similarity and the similarity of the second-ranked face with the next highest similarity is above the threshold (S902).

[0188] If the difference between the similarity of the first-ranked face and the similarity of the second-ranked face is greater than or equal to a threshold (YES in S902), the face recognition unit 103 generates an authentication result indicating that authentication was successful (S905).

[0189] If the difference between the similarity of the first-ranked facial match and the similarity of the second-ranked facial match is not greater than or equal to a threshold (NO in S902), the facial recognition unit 103 determines whether the user corresponding to the first-ranked facial match and the user corresponding to the second-ranked facial match have the same family ID (S903).

[0190] If the user corresponding to face matching 1 and the user corresponding to face matching 2 have the same family ID (YES in S903), the facial recognition unit 103 validates face matching 1 (S904). In this case, the user corresponding to face matching 1 is the person to be authenticated. The facial recognition unit 103 then generates an authentication result indicating that authentication was successful (S905).

[0191] If the similarity of the first-ranked face in face matching is not above the threshold (NO in S901), or if the user corresponding to the first-ranked face in face matching and the user corresponding to the second-ranked face in face matching do not have the same family ID (NO in S903), the face authentication unit 103 generates an authentication result indicating that authentication failed (S906).

[0192] The facial recognition unit 103 transmits the authentication result to the authentication terminal 20 (S907). Then, the flow shown in Figure 16 is completed.

[0193] Next, we will explain the sequence of interactions between devices corresponding to the flowchart shown in Figure 16.

[0194] Figure 17 is a sequence diagram showing an example of the processing flow related to variation 2 of this embodiment 2. In Figure 17, processes similar to those in Figure 5 are given the same reference numerals as in Figure 5, and their explanations are omitted.

[0195] The facial information acquisition unit 203 generates a facial recognition request and sends the facial recognition request to the server device 40 (S204).

[0196] The face recognition unit 102 of the server device 40 performs face recognition processing (S1001). Here, the face recognition unit 103 determines whether the similarity of the face with the highest similarity (Face Match 1) is above a threshold, and if the similarity of the face with the highest similarity (Face Match 1) is above the threshold, it determines whether the difference between the similarity of Face Match 1 and the similarity of the next highest similar face (Face Match 2) is above the threshold.

[0197] If the difference between the similarity of the first-ranked facial match and the similarity of the second-ranked facial match is not greater than or equal to a threshold, the facial recognition unit 103 determines the family ID (S1002). Then, if the two users corresponding to the similarity of the first-ranked facial match and the similarity of the second-ranked facial match have the same family ID, the first-ranked facial match is made valid (S1003). When the first-ranked facial match is made valid, it is determined that the user corresponding to the first-ranked facial match is the person to be authenticated, and authentication is successful.

[0198] The facial recognition unit 102 transmits an authentication result indicating whether or not the facial recognition process was successful to the authentication terminal 20 (S1004).

[0199] The display unit 206 of the authentication terminal 20 displays the authentication result (S1005). Then, the sequence shown in Figure 17 ends.

[0200] <Variation 3: Cases involving facial recognition for individuals other than infants> Variation 3 describes cases where the person being authenticated (the person included in the photograph) is a parent or facility staff member other than an infant.

[0201] Figure 18 is a flowchart showing an example of the processing flow related to variation 3 of this embodiment 2. In Figure 18, the same processing as in Figure 4 is given the same reference numerals as in Figure 4, and its explanation is omitted. The flow shown in Figure 18 may start, for example, when the power of the authentication terminal 20 is turned on.

[0202] The camera 205 of the authentication terminal 20 captures a face (S103). The face information acquisition unit 203 acquires face information from the captured image and sends a face authentication request containing the acquired face information to the server device 40.

[0203] The facial recognition unit 103 of the server device 40 performs facial recognition processing and determines whether or not it has succeeded in authenticating the person to be authenticated in response to the facial recognition request (S1101).

[0204] If authentication is unsuccessful (NO in S1101), the facial recognition unit 103 generates an authentication result indicating that authentication failed (S1103).

[0205] If authentication is successful (YES in S1101), it is determined whether the user who is the target of authentication has a facility staff ID or a guardian ID (S1102).

[0206] If a user who is subject to authentication does not have a facility staff ID or guardian ID (NO in S1102), the facial recognition unit 103 generates an authentication result indicating that authentication failed (S1103). However, for example, if it can be identified that a user who does not have a facility staff ID or guardian ID is a child, the facial recognition unit 103 may generate an authentication result indicating that authentication of the child was successful.

[0207] The facial recognition unit 103 transmits the authentication result to the authentication terminal 20 (S1104).

[0208] If the user who is subject to authentication has a facility staff ID or guardian ID (YES in S1102), the facial recognition unit 103 instructs the authentication terminal 20 to display a message prompting the user to perform facial recognition on the child (S1105). The display unit 206 of the authentication terminal that receives the instruction displays a message prompting the child to perform facial recognition.

[0209] As described above, according to this embodiment 2, even in various cases such as when the captured image includes two or more people, when the person to be authenticated is one of siblings belonging to the same facility, or when the captured image includes facility staff or guardians, facial recognition of the infant to be authenticated can be performed smoothly.

[0210] (Embodiment 3) Embodiment 3 describes an example in which an access control device for managing entry into and / or exit from a facility is added to the system configuration shown in Embodiment 1.

[0211] <Example of system configuration> Figure 19 shows an example of a facial recognition system according to this third embodiment. In Figure 19, components similar to those in Figure 1 are given the same reference numerals and their descriptions may be omitted. Figure 19 shows a server device 10, an authentication terminal 20 connected to the server device 10, a registration terminal 30, and an access control device 50.

[0212] The access control device 50 connects to the server device 10 and receives authentication results from the server device 10. Based on the received authentication results, the access control device 50 manages the entry and exit of users.

[0213] The access control device 50 includes a communication control unit 501, an access information processing unit 502, and a storage unit 503.

[0214] The communication control unit 501 controls communication between the server device 10 and the access control device 50. The communication control unit 501 may be divided into a transmitting unit that processes the transmission of signals (or information) and a receiving unit that processes the reception of signals (or information).

[0215] The access control unit 502 generates access information for each user based on the authentication results obtained from the server device 10, and stores the generated access information in the access management database 503a of the storage unit 503. Access information includes, for example, the time a user enters the facility and the time a user leaves the facility. The time a user enters and the time a user leaves may correspond to the time when facial recognition was successful upon entry and the time when facial recognition was successful upon exit, respectively.

[0216] The entry / exit information processing unit 502 manages the time when users enter and leave the room, and may identify users who do not enter by a predetermined time (e.g., scheduled entry time) and / or users who do not leave by a predetermined time (e.g., scheduled exit time), and issue an alert. For example, the alert may be sent to a terminal owned by a facility staff member, or to a terminal owned by the guardian of the identified user.

[0217] In Figure 19, an example is shown where the access control device 50 is included in the facial recognition system as a separate device from the server device 10, the authentication terminal 20, and the registration terminal 30. However, the configuration of the access control device 50 (an example of the access control unit) may be included in either the server device 10 or the authentication terminal 20.

[0218] Next, I will explain the access control database.

[0219] Figure 20 shows an example of information in the access control database. In Figure 20, the user ID, the user's name, and the group are shown in association. In addition, as an example of access information, Figure 20 shows the time of arrival (e.g., time of entry into the facility) and the time of departure (e.g., time of departure from the facility) for each user. The arrival time and departure time are included in the authentication results for arrival and departure, respectively.

[0220] Furthermore, as shown in the example in Figure 20, it may be possible to store information for each user regarding whether or not they have reported being absent or late.

[0221] As shown in Figure 20, access control information is managed in the access control database.

[0222] Figure 21 is a sequence diagram showing an example of the processing flow according to this third embodiment. In Figure 21, the same reference numerals are used for processes similar to those in Figure 5, and their explanations are omitted.

[0223] After performing the facial recognition process (after S205), the facial recognition unit 102 of the server device 10 sends an authentication result indicating whether or not the facial recognition process was successful to the authentication terminal 20 (S206), and then sends the authentication result to the access control device 50 (S1201).

[0224] The access control device 50 registers the time of arrival and / or departure based on the authentication result (S1202). Then, the sequence shown in Figure 21 ends.

[0225] As described above, according to this embodiment 3, access control based on facial recognition can be stored in a database, and access control can be performed appropriately.

[0226] The embodiments and variations described above may be used in combination or selectively switched between.

[0227] In each of the embodiments described above, the guidance of gaze may include guidance of posture and orientation in addition to gaze.

[0228] The facial recognition systems described in each embodiment above were explained using an example where they are introduced in a kindergarten as an example of a facility, and authenticate the child by using facial recognition to identify which of the registered users the child is when entering and / or leaving the facility. However, this disclosure is not limited to this. For example, this disclosure may be introduced in facilities other than kindergartens (e.g., daycare centers, hospitals, or nursing homes), and the person to be authenticated and the user may be a person other than a child (e.g., the elderly, people with visual impairments, etc.).

[0229] For example, the facial recognition systems described in each of the embodiments above may be introduced into a school bus used to transport children to and from kindergarten. In this case, facial recognition processing of the child is performed when the child boards and / or alights from the bus.

[0230] If facial recognition is performed when boarding or alighting, children who successfully undergo facial recognition may be registered as having attended kindergarten. The server device 10 and / or authentication terminal 20 may compare the absence notification that a child will be absent from kindergarten with the authentication result, and if there are registered children (i.e., children attending kindergarten) who have not submitted an absence notification and have not been successfully authenticated, an alert may be issued to the kindergarten staff. The alert to the kindergarten staff may be issued from the authentication terminal 20 or from a terminal owned by the kindergarten staff. The alert may also be issued to the guardian of the child in question.

[0231] If facial recognition processing is performed both when boarding and when alighting, children who successfully undergo facial recognition both boarding and alighting may be registered as having attended the daycare. In addition, the server device 10 and / or the authentication terminal 20 may issue an alert to the daycare staff if there are children who successfully undergo facial recognition when boarding but fail to do so when alighting, or if there is no facial recognition result when alighting. The alert to the daycare staff may be issued from the authentication terminal 20 or from a terminal owned by the daycare staff. The alert may also be issued to the guardian of the child in question.

[0232] The flowcharts and sequence diagrams shown in each of the embodiments described above do not limit the execution of processes to the order shown. For example, each process shown in the flowchart and sequence may be executed in a different order than shown, and multiple processes may be executed in parallel or simultaneously. The order of the multiple processes shown may also be changed. Furthermore, some of the processes shown in the sequence may be temporarily omitted.

[0233] In the embodiments described above, the notation "...part" used for each component may be replaced with other notations such as "...means," "...circuitry," "...assembly," "...device," "...unit," or "...module."

[0234] <Summary of Embodiments> A facial recognition system according to one embodiment of the present disclosure includes: a guidance processing unit that provides a visual display to guide the gaze of a person to be facially recognized toward the camera; an acquisition unit that acquires a facial image, which is the region of the face of the person, from an image captured by the camera; and a facial recognition unit that performs facial recognition processing for the person based on a pre-registered facial image of a registered person and the facial image acquired by the acquisition unit.

[0235] In this facial recognition system, the guidance processing unit determines the content of the display according to the estimated age of the subject.

[0236] In this facial recognition system, the acquisition unit determines whether the captured image contains multiple subjects, and if it contains multiple subjects, it determines at least one subject from among the multiple subjects based on predetermined criteria.

[0237] In this facial recognition system, the facial recognition unit determines the similarity between the registered person's facial image and the target person's facial image. If the difference between the highest first similarity and the next highest second similarity is less than a threshold, the unit checks whether a specific relationship exists between the first registered person corresponding to the first similarity and the second registered person corresponding to the second similarity. If a specific relationship exists between the first registered person and the second registered person, the unit determines that the target person is the first registered person.

[0238] In this facial recognition system, the acquisition unit estimates whether the captured image contains multiple subjects, and if it contains multiple subjects, it estimates the age of each of the subjects, determines the subject with the youngest age, and acquires the facial image of the determined subject from among the multiple subjects based on the information about the determined subject.

[0239] This facial recognition system includes a display unit that, when the authentication of the subject is successful, displays a specific image associated with the subject and registered in that image to the subject.

[0240] In this facial recognition system, the facial recognition system has an access control unit that manages whether or not the registered person has entered a specific facility and / or left the specific facility.

[0241] In this facial recognition system, if the result of the facial recognition process indicates that the subject is a second registrant and not a first registrant, the facial recognition unit performs a process to request the subject to perform the facial recognition process for the first registrant.

[0242] In this facial recognition system, the facial recognition unit determines whether a first subject is holding a second subject among the multiple subjects included in the captured image, and if it determines that the first subject is holding the second subject and the authentication of the first subject is successful, it determines whether the first subject is the guardian of one of the registered persons, and if the first subject is the guardian of one of the registered persons, it determines that the second subject is an infant who has registered the first subject as their guardian.

[0243] An information processing device according to one embodiment of the present disclosure includes: a guidance processing unit that causes a visual display on a display unit to guide the gaze of a person subject to facial recognition towards the camera; an acquisition unit that acquires a face image, which is the region of the face of the person subject, from an image captured by the camera; and a facial recognition unit that performs facial recognition processing for the person subject based on a pre-registered face image of a registered person and the face image acquired by the acquisition unit.

[0244] An information processing method according to one embodiment of the present disclosure displays a visual display on a display unit that guides the gaze of the subject of facial recognition towards the camera, acquires a facial image which is the region of the subject's face from an image captured by the camera, and performs facial recognition processing for the subject based on a pre-registered registered facial image and the facial image acquired by the camera.

[0245] This disclosure can be implemented using software, hardware, or software integrated with hardware.

[0246] Each functional block used in the description of the above embodiments may be implemented partially or entirely as an integrated circuit (LSI), and each process described in the above embodiments may be controlled partially or entirely by a single LSI or a combination of LSIs. An LSI may consist of individual chips, or it may consist of a single chip that includes some or all of the functional blocks. An LSI may have data inputs and outputs. Depending on the degree of integration, LSIs may be referred to as ICs, system LSIs, super LSIs, or ultra LSIs.

[0247] The method of integration is not limited to LSIs; it may also be implemented using dedicated circuits, general-purpose processors, or dedicated processors. Furthermore, FPGAs (Field Programmable Gate Arrays) that can be programmed after LSI manufacturing, or reconfigurable processors that allow for the reconfiguration of the connections and settings of circuit cells within the LSI, may also be used. This disclosure may be implemented as digital or analog processing.

[0248] Furthermore, if advancements in semiconductor technology or other derived technologies lead to the emergence of integrated circuit technologies that replace LSIs, then naturally, it would be possible to use those technologies to integrate functional blocks. The application of biotechnology, for example, is a possibility.

[0249] This disclosure is applicable to all types of devices, systems, and equipment having communication capabilities (collectively referred to as communication equipment). Communication equipment may include a radio transceiver and a processing / control circuit. A radio transceiver may include a receiver and a transmitter, or both as functions. A radio transceiver (transmitter, receiver) may include an RF (Radio Frequency) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or similar. Non-exclusive examples of communication devices include telephones (mobile phones, smartphones, etc.), tablets, personal computers (PCs) (laptops, desktops, notebooks, etc.), cameras (digital still / video cameras, etc.), digital players (digital audio / video players, etc.), wearable devices (wearable cameras, smartwatches, tracking devices, etc.), game consoles, digital book readers, telehealth / telemedicine devices, vehicles or mobile transport with communication capabilities (cars, airplanes, ships, etc.), and combinations of the above-mentioned devices.

[0250] Communication devices are not limited to portable or movable devices, but also include all kinds of non-portable or fixed devices, devices, and systems, such as smart home devices (appliances, lighting equipment, smart meters or measuring instruments, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0251] Furthermore, in recent years, Cyber-Physical Systems (CPS), a new concept in IoT (Internet of Things) technology that creates new added value through information linkage between the physical and cyber spaces, has been attracting attention. This CPS concept can also be adopted in the above-described embodiment.

[0252] In other words, as a basic configuration of CPS, for example, edge servers located in physical space and cloud servers located in cyberspace are connected via a network, and processing can be distributed and performed by the processors installed on both servers. Here, it is preferable that each processing data generated on the edge server or cloud server is generated on a standardized platform, and by using such a standardized platform, it is possible to improve efficiency when building systems that include various diverse groups of sensors and IoT application software.

[0253] Communication includes data communication via cellular systems, wireless LAN systems, and communication satellite systems, as well as data communication using combinations of these.

[0254] Furthermore, the communication device also includes devices such as controllers and sensors that are connected to or linked to a communication device that performs the communication functions described in this disclosure. For example, this includes controllers and sensors that generate control signals and data signals used by the communication device that performs the communication functions of the communication device.

[0255] Furthermore, communication equipment includes infrastructure facilities such as base stations, access points, and any other devices, devices, and systems that communicate with or control the aforementioned non-limited types of equipment.

[0256] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of this disclosure. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the disclosure.

[0257] The specific examples of this disclosure have been described in detail above, but these are merely illustrative and do not limit the scope of the claims. The technologies described in the claims include various modifications and changes to the specific examples described above. [Industrial applicability]

[0258] One embodiment of the present disclosure is suitable for a facial recognition system. [Explanation of Symbols]

[0259] 10, 40 Server Devices 20 Authentication terminals 30 Registered terminals 101 Communication Control Unit 102 User Registration Section 103 Facial Recognition Section 104 Storage section 201 Communication Control Unit 202 Person Detection Unit 203 Facial Information Acquisition Unit 204 Guidance Image Control Unit 205 Camera 206 Display section 207 Memory section 301 Communication Control Unit 302 Face Information Acquisition Unit 303 Display section 401 Age Estimation Section 402 Age and Guided Image Database 403 Baby-holding determination unit

Claims

1. A guidance processing unit that displays a visual indicator to guide the gaze of the person being recognized by facial recognition towards the camera, The camera acquires a facial image, which is the facial region of the subject, from the captured image of the subject, Based on the registered face image of the registrant and the face image acquired by the acquisition unit, a face recognition process is performed for the target person, and the similarity between the registered face image and the target person's face image is determined. If the difference between the highest first similarity and the next highest second similarity is less than a threshold, check whether a specific relationship exists between the first registrant corresponding to the first similarity and the second registrant corresponding to the second similarity. A facial recognition unit determines that the subject is the first registrant if a specific relationship exists between the first registrant and the second registrant. A facial recognition system that has the following features.

2. A guidance processing unit that displays a visual indicator to guide the gaze of the person being recognized by facial recognition towards the camera, The acquisition unit acquires a facial image, which is the facial region of the subject, from the image captured by the camera, estimates whether the image contains multiple subjects, estimates the age of each of the multiple subjects if multiple subjects are included, determines the youngest subject, and acquires a facial image of the determined subject from among the multiple subjects based on the information about the determined subject. A facial recognition unit performs facial recognition processing on the subject based on a facial image of a registered person that has been registered in advance and the facial image acquired by the acquisition unit. A facial recognition system that has the following features.

3. A guidance processing unit that displays a visual indicator to guide the gaze of the person being recognized by facial recognition towards the camera, The camera acquires a facial image, which is the facial region of the subject, from the captured image of the subject, Based on the facial image of a registered person registered in advance and the facial image acquired by the acquisition unit, the facial recognition unit performs facial recognition processing on the subject, determines whether the first subject is holding the second subject among the multiple subjects included in the captured image, determines whether the first subject is the guardian of one of the registered persons, and determines that the second subject is an infant who has been registered with the first subject as their guardian. A facial recognition system that has the following features.

4. The induction processing unit, The content of the display is determined according to the estimated age of the subject. The facial recognition system according to claims 1 to 3.

5. The acquisition unit is, Determine whether the aforementioned captured image contains multiple subjects, If the aforementioned multiple subjects are included, then at least one subject will be selected from among the aforementioned multiple subjects based on predetermined criteria. The facial recognition system according to claims 1 to 3.

6. If the authentication of the subject is successful, the display unit has a display that is controlled to show a specific display associated with the subject. The facial recognition system according to claims 1 to 3.

7. The aforementioned facial recognition system is The system has an access control unit that manages whether the registered person has entered a specific facility and / or left the specific facility. The facial recognition system according to claims 1 to 3.

8. The aforementioned facial recognition unit is If the result of the facial recognition process indicates that the subject is a second registrant who is not the first registrant and is the subject's guardian or facility staff, the process is performed to request the subject to perform the facial recognition process of the first registrant. The facial recognition system according to claims 1 to 3.

9. A guidance processing unit that displays a visual indicator on the display unit to guide the gaze of the person being recognized by facial recognition towards the camera, The camera acquires a facial image, which is the facial region of the subject, from the captured image of the subject, A facial recognition unit performs facial recognition processing on the subject based on a pre-registered facial image of a registrant and a facial image acquired by the acquisition unit, determines the similarity between the registrant's facial image and the subject's facial image, and if the difference between the highest first similarity and the next highest second similarity is less than a threshold, checks whether a specific relationship exists between the first registrant corresponding to the first similarity and the second registrant corresponding to the second similarity, and if a specific relationship exists between the first registrant and the second registrant, determines that the subject is the first registrant. An information processing device equipped with the following features.

10. A visual indicator is displayed on the screen to guide the gaze of the person being recognized by facial recognition towards the camera. The camera acquires a facial image, which is the facial region of the subject, from the captured image of the subject. Based on the facial image of a pre-registered registrant and the facial image acquired by the camera, a facial recognition process is performed on the subject, the similarity between the registrant's facial image and the subject's facial image is determined, and if the difference between the highest first similarity and the next highest second similarity is less than a threshold, it is checked whether a specific relationship exists between the first registrant corresponding to the first similarity and the second registrant corresponding to the second similarity, and if a specific relationship exists between the first registrant and the second registrant, it is determined that the subject is the first registrant. Information processing methods.

11. A guidance processing unit that displays a visual indicator to guide the gaze of the person being recognized by facial recognition towards the camera, The acquisition unit acquires a facial image, which is the facial region of the subject, from the image captured by the camera, estimates whether the image contains multiple subjects, estimates the age of each of the multiple subjects if multiple subjects are included, determines the youngest subject, and acquires a facial image of the determined subject from among the multiple subjects based on the information about the determined subject. A facial recognition unit performs facial recognition processing on the subject based on a facial image of a registered person that has been registered in advance and the facial image acquired by the acquisition unit. An information processing device having

12. A visual indicator is displayed on the screen to guide the gaze of the person being recognized by facial recognition towards the camera. The camera acquires a facial image, which is the facial region of the subject, from the captured image of the subject, estimates whether the captured image contains multiple subjects, estimates the age of each of the multiple subjects, determines the youngest subject, and based on the information of the determined subject, acquires the facial image of the determined subject from among the multiple subjects. Based on the facial image of a registered user that has been registered in advance and the facial image acquired by the camera, the system performs facial recognition processing on the subject. Information processing methods.

13. A guidance processing unit that displays a visual indicator to guide the gaze of the person being recognized by facial recognition towards the camera, The camera acquires a facial image, which is the facial region of the subject, from the captured image of the subject, Based on the facial image of a registered person registered in advance and the facial image acquired by the acquisition unit, the facial recognition unit performs facial recognition processing on the subject, determines whether the first subject is holding the second subject among the multiple subjects included in the captured image, determines whether the first subject is the guardian of one of the registered persons, and determines that the second subject is an infant who has been registered with the first subject as their guardian. An information processing device having

14. A visual indicator is displayed on the screen to guide the gaze of the person being recognized by facial recognition towards the camera. The camera acquires a facial image, which is the facial region of the subject, from the captured image of the subject. Based on the facial image of a registered person registered in advance and the acquired facial image, a facial recognition process is performed on the subject, and it is determined whether the first subject is holding the second subject among the multiple subjects included in the captured image, and if it is determined that the first subject is holding the second subject and the authentication of the first subject is successful, it is determined whether the first subject is the guardian of one of the registered persons, and if the first subject is the guardian of one of the registered persons, it is determined that the second subject is an infant who has been registered with the first subject as their guardian. Information processing methods.

Images