Information processing device, information processing method, and recording medium

Abstract

This information processing device includes: an image acquisition means for acquiring an image including a facial recognition region set in a ticket gate region of a railway station and through which users of the railway station pass; a facial authentication means for authenticating a user's face on the basis of the image; and an information processing means for recording, in a system related to an automated ticket gate, information which is for using the automated ticket gate and is linked to a registered individual corresponding to the user identified by the facial recognition, and information indicating the user's entry into the railway station and / or the user's exit from the railway station.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] As an example of this type of device, a device has been proposed that is capable of performing ticket inspection processing using a medium such as a ticket and ticket inspection processing using facial matching, and performs ticket inspection processing for one passerby using either the medium or facial matching (see Patent Document 1).Other prior art documents related to this disclosure include Patent Documents 2 and 3.

[0003] JP 2022-144170 A JP 2022-048502 A JP 2021-149257 A

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques described in prior art documents.

[0005] One aspect of an information processing device includes an image acquisition means that is set in a ticket gate area of ​​a railway station and acquires an image including a facial recognition area through which users of the railway station pass, a facial recognition means that performs facial recognition of the user based on the image, and an information processing means that registers, in a system related to the automatic ticket gate, information for using the automatic ticket gate linked to a registered person corresponding to the user identified by the facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station.

[0006] One aspect of the information processing method is an information processing method that acquires an image that includes a facial recognition area that is set in a ticket gate area of ​​a railway station and through which users of the railway station pass, performs facial recognition of the user based on the image, and registers information for using the automatic ticket gate linked to the registered person corresponding to the user identified by the facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station, in a system related to the automatic ticket gate.

[0007] One aspect of the recording medium has a computer program recorded on it for causing a computer to execute an information processing method that acquires an image that includes a facial recognition area that is set in a ticket gate area of ​​a railway station and through which users of the railway station pass, performs facial recognition of the user based on the image, and registers information for using the automatic ticket gate linked to the registered person corresponding to the user identified by the facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station, in a system related to the automatic ticket gate.

[0008] FIG. 1 is a diagram showing an example of a ticket gate system. FIG. 2 is a diagram showing an example of a face information database. FIG. 3 is a block diagram showing an example of the configuration of an information processing device. FIG. 4 is a diagram showing an example of a ticket gate area of ​​a railway station. FIG. 5 is a flowchart showing the operation of the information processing device according to the first embodiment. FIG. 6 is a block diagram showing another configuration of the information processing device. FIG. 7 is a diagram for explaining person tracking processing. FIG. 8 is a diagram showing an example of an ID correspondence table.

[0009] <First embodiment> A first embodiment relating to an information processing device, an information processing method, and a recording medium will be described with reference to Figures 1 to 5. Hereinafter, the first embodiment relating to an information processing device, an information processing method, and a recording medium will be described using an information processing device 10 provided in a ticket gate system 1.

[0010] (Outline of Ticket Gating System) An outline of the ticket gate system according to the first embodiment will be described with reference to Fig. 1. In Fig. 1, the ticket gate system 1 includes a station server 41 installed at railway station A, a station server 42 installed at railway station B, a center server 50, and an information database 70 (hereinafter referred to as "face information DB 70" as appropriate). The station servers 41 and 42, the center server 50, and the face information DB 70 are connected via a network NW. The network NW is, for example, a wide area network such as the Internet.

[0011] The ticket gate system 1 may include information processing devices 10 and 10a, and automatic ticket gates 31 and 32. The information processing device 10 and the automatic ticket gate 31 may be installed at station A. Note that it is not necessary for either the information processing device 10 or the automatic ticket gate 31 to be installed at station A. In other words, at least one of the information processing device 10 and the automatic ticket gate 31 may be installed at station A. The information processing device 10a and the automatic ticket gate 32 may be installed at station B. Note that it is not necessary for either the information processing device 10a or the automatic ticket gate 32 to be installed at station B. In other words, at least one of the information processing device 10a and the automatic ticket gate 32 may be installed at station B.

[0012] At station A, the station server 41, the information processing device 10, and the automatic ticket gate 31 may be connected via a narrow area network such as a local area network (LAN). At station B, the station server 42, the information processing device 10a, and the automatic ticket gate 32 may be connected via a narrow area network such as a LAN.

[0013] The automatic ticket gates 31 and 32 are ticket gate devices that can use at least one of magnetic tickets, IC (Integrated Circuit) tickets, and mobile tickets, for example. Various existing aspects can be applied to the automatic ticket gates 31 and 32, so detailed explanations thereof will be omitted.

[0014] The station server 41 may obtain information from the automatic ticket gate 31 indicating that a user holding an IC ticket or mobile ticket has entered station A. In this case, the station server 41 may link the identification information related to the IC ticket or mobile ticket with the information indicating entry into station A and transmit the linked information to the center server 50. The center server 50 may register the information transmitted from the station server 41 in the entry / exit record 51.

[0015] The station server 41 may obtain information from the automatic ticket gate 31 indicating that a user holding an IC ticket or mobile ticket has exited at station A. In this case, the station server 41 may link the identification information related to the IC ticket or mobile ticket with the information indicating the exit from station A and transmit the linked information to the center server 50. The center server 50 may register the information transmitted from the station server 41 in the entry / exit record 51.

[0016] The station server 42 may obtain information from the ticket gate device indicating that a user holding an IC ticket or mobile ticket has entered station B. In this case, the station server 42 may link the identification information related to the IC ticket or mobile ticket with the information indicating entry into station B and transmit the linked information to the center server 50. The center server 50 may register the information transmitted from the station server 42 in the entry / exit record 51.

[0017] The station server 42 may obtain information from the ticket gate device indicating that a user holding an IC ticket or mobile ticket has exited at station B. In this case, the station server 42 may link the identification information related to the IC ticket or mobile ticket with the information indicating the exit at station B and transmit the linked information to the center server 50. The center server 50 may register the information transmitted from the station server 42 in the entry / exit record 51.

[0018] For example, suppose a user enters station A, travels from station A to station B by train, and exits at station B. In this case, the entry / exit record 51 of the center server 50 may register that the user entered station A and exited station B. The center server 50 may calculate the railway fare (e.g., fare) from station A to station B. The center server 50 may pay the calculated railway fare using payment information related to the user. The payment information may be, for example, at least one of credit card information, debit card information, prepaid card information, IC card information, and account information.

[0019] The facial information DB 60 stores information about people who wish to use the ticket gate system 1. Among people who wish to use the ticket gate system 1, those whose information is registered in the facial information DB 60 will be referred to as "registrants" as appropriate. The facial information DB 60 stores authentication IDs as identification information for identifying each of the multiple registrants. Information about each of the multiple registrants is linked to the authentication ID and registered in the facial information DB 70.

[0020] Here, the facial information DB 60 will be further described with reference to FIG. 2 . As shown in FIG. 2 , facial information and passenger ticket information may be linked to an authentication ID associated with a registrant. The facial information may be, for example, a feature value indicating the facial characteristics of the registrant, or information for identifying the feature value (e.g., a file name). Alternatively, the facial information may be, for example, a facial image including the face of the registrant, or information for identifying the facial image (e.g., a file name). The passenger ticket information is information regarding at least one of an IC ticket and a mobile ticket that can be used at an automatic ticket gate (e.g., automatic ticket gates 31 and 32). Therefore, the passenger ticket information can be said to be information for using an automatic ticket gate (e.g., automatic ticket gates 31 and 32).

[0021] The configurations and operations of the automatic ticket gates 31 and 32, the station servers 41 and 42, and the center server 50 may be similar to those of existing automatic ticket gate systems. Therefore, the ticket gate system 1 can be said to be a system that includes the information processing devices 10 and 10a and the face information DB 60 in addition to the existing automatic ticket gate system.

[0022] (Information Processing Device) The information processing device 10 will be described with reference to Fig. 3. The information processing device 10a may have the same configuration as the information processing device 10. Therefore, the description of the information processing device 10a will be omitted.

[0023] The information processing device 10 performs facial authentication on a user (in other words, a person to be authenticated) passing through the ticket gate area of ​​Station A. The information processing device 10 detects that a registered person corresponding to the user identified by facial authentication has entered Station A or exited Station A. In other words, the information processing device 10 detects the entry and exit of a user at a railway station (e.g., Station A) in the same way as an automatic ticket gate (e.g., automatic ticket gates 31 and 32).

[0024] 3, the information processing device 10 includes an image acquisition unit 11, a face authentication unit 12, an information processing unit 13, and a face information database 14 (hereinafter referred to as "face information DB 14" as appropriate). The face information DB 14 may store the same information as that registered in the face information DB 60. The face information DB 14 may be updated at a predetermined interval so that the information registered in the face information DB 14 is the same as the information registered in the face information DB 60.

[0025] The ticket gate area of ​​Station A will now be described with reference to Figure 4. The ticket gate area includes an area where automatic ticket gates 31 are installed and a facial recognition ticket gate where the entrance and exit of users is detected by the information processing device 10. Note that two or more automatic ticket gates 31 may be installed at Station A. Note that, although Figure 4 shows the automatic ticket gate 31 installed adjacent to the facial recognition ticket gate, the arrangement of the automatic ticket gate 31 and the facial recognition ticket gate is not limited to this.

[0026] The facial recognition ticket gate has a facial recognition area Ar through which at least some of the users of Station A pass. As shown in Figure 4, users entering Station A pass through the facial recognition area Ar from the bottom to the top of the drawing. Users exiting Station A pass through the facial recognition area Ar from the top to the bottom of the drawing.

[0027] The facial recognition area Ar is a virtual area. Therefore, for example, a mark indicating the facial recognition area Ar does not need to be drawn at Station A. However, for example, a mark indicating the facial recognition area Ar may be drawn at Station A. The facial recognition area through which users entering Station A pass and the facial recognition area through which users leaving Station A pass may be set separately.

[0028] The camera CAM is installed so as to be able to capture an image of the facial authentication area Ar. Note that the camera CAM does not have to be installed so that the entire facial authentication area Ar is included within its angle of view. In other words, the camera CAM may be installed so as to be able to capture an image of at least a part of the facial authentication area Ar. The camera CAM may be installed in a position that does not interfere with the passage of users at Station A (for example, a position higher than the heads of users).

[0029] The camera CAM may be a high-resolution camera such as a 4K resolution camera. The camera CAM may be connected to the information processing device 10 via a cable, or may be connected to the information processing device 10 using wireless communication technology. Alternatively, the camera CAM may be connected to the station server 41 via a cable, or may be connected to the station server 41 using wireless communication technology. In addition to the camera CAM, another camera capable of capturing an image of at least a portion of the face authentication area Ar may be installed at station A. In other words, two or more cameras capable of capturing an image of at least a portion of the face authentication area Ar may be installed at station A.

[0030] 3 , the image acquisition unit 11 of the information processing device 10 acquires an image including the face authentication area Ar, which is generated by the camera CAM capturing an image of the face authentication area Ar. The image acquisition unit 11 may acquire the image directly from the camera CAM, or may acquire the image from the camera CAM via the station server 41.

[0031] The face authentication unit 12 performs face authentication based on the image acquired by the image acquisition unit 11. Note that various existing methods such as a two-dimensional (2D) authentication method and a three-dimensional (3D) authentication method can be applied to face authentication. An example of processing related to face authentication will be described below.

[0032] The facial authentication unit 12 may detect the face of a user (hereinafter referred to as the "person to be authenticated") included in the image. The facial authentication unit 12 may calculate facial feature amounts indicating the characteristics of the detected face. The facial authentication unit 12 may use the calculated facial feature amounts and the face information DB 14 to identify a registered person corresponding to the person to be authenticated.

[0033] The face authentication unit 12 may calculate a matching score from the calculated facial feature amount (i.e., the facial feature amount related to the face of the person to be authenticated) and the facial feature amount based on the facial information of one registrant registered in the face information DB 14. If the calculated matching score is greater than a predetermined threshold, the face authentication unit 12 may determine that the one registrant is a registrant corresponding to the person to be authenticated.

[0034] If the calculated matching score is smaller than a predetermined threshold, the face authentication unit 12 may calculate a matching score from the calculated facial feature amount and facial feature amounts based on the facial information of other registrants registered in the face information DB 14. In this case, the face authentication unit 12 may determine that the one registrant is not a registrant corresponding to the person to be authenticated. Note that if the calculated matching score is "equal" to the predetermined threshold, it may be treated as including either case.

[0035] When a registered person corresponding to a user passing through the facial authentication area Ar from the outside to the inside of station A is identified, the information processing unit 13 may determine that the registered person has entered station A. When a registered person corresponding to a user passing through the facial authentication area Ar from the inside to the outside of station A is identified, the information processing unit 13 may determine that the registered person has exited station A.

[0036] Note that, when the facial recognition area through which a user entering station A passes and the facial recognition area through which a user exiting station A passes are set separately, the information processing unit 13 may determine that a registrant identified by facial recognition performed based on an image including the facial recognition area through which a user entering station A passes is a registrant entering station A. In this case, the information processing unit 13 may determine that a registrant identified by facial recognition performed based on an image including the facial recognition area through which a user exiting station A passes is a registrant exiting station A.

[0037] The information processing unit 13 obtains from the face information DB 14 ticket information linked to the authentication ID of the registered person corresponding to the person to be authenticated (in other words, the user of Station A) identified by the face authentication unit 12.

[0038] If the registrant corresponds to an authenticated person who entered station A, the information processing unit 13 may link the ticket information (for example, information relating to at least one of an IC ticket and a mobile ticket) with information indicating entry into station A and transmit the linked information to the station server 41. The station server 41 may transmit the ticket information and information indicating entry into station A transmitted from the information processing unit 13 to the center server 50. As a result, the entry of the registrant into station A may be registered in the entry / exit record 51.

[0039] If the registrant corresponds to an authenticated person who exited from station A, the information processing unit 13 may link the ticket information (for example, information relating to at least one of an IC ticket and a mobile ticket) with information indicating that the person exited from station A and transmit the linked information to the station server 41. The station server 41 may transmit the ticket information transmitted from the information processing unit 13 and the information indicating that the person exited from station A to the center server 50. As a result, the fact that the registrant exited from station A may be registered in the entry / exit record 51.

[0040] Next, the operation of the information processing device 10 will be described with reference to the flowchart in Fig. 5. In Fig. 5, the image acquisition unit 11 acquires an image including a face recognition area Ar (step S101). The face recognition unit 12 performs face recognition based on the image acquired in the processing of step S101 (step S102).

[0041] The information processing unit 13 determines whether the registrant identified by the processing of step S102 is entering or exiting station A (step S103). If it is determined in the processing of step S103 that the registrant is entering station A (step S103: Entry), the information processing unit 13 acquires ticket information linked to the registrant. Then, the information processing unit 13 may associate the acquired ticket information with information indicating entry into station A and transmit the information to the station server 41. As a result, the entry of the registrant into station A may be registered in the entry / exit record 51 (step S104).

[0042] If it is determined in the processing of step S103 that the registrant will exit from station A (step S103: exit), the information processing unit 13 acquires ticket information linked to the registrant. Then, the information processing unit 13 may associate the acquired ticket information with information indicating that the registrant has exited from station A, and transmit the information to the station server 41. As a result, the fact that the registrant has exited from station A may be registered in the entry / exit record 51 (step S105).

[0043] In this way, the information processing device 10 performs an information processing method in which an image is acquired that includes a facial recognition area that is set in the ticket gate area of ​​a railway station and through which railway station users pass, and performs facial recognition of the user based on the image, and registers information for using the automatic ticket gate linked to the registered person corresponding to the user identified by facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station, in a system related to the automatic ticket gate.

[0044] The information processing device 10 described above may be realized by a computer reading a computer program recorded on a recording medium. In this case, it can be said that the recording medium stores a computer program for causing the computer to execute an information processing method for acquiring an image including a facial recognition area that is set in a ticket gate area of ​​a railway station and through which railway station users pass, performing facial recognition of the user based on the image, and registering, in a system related to the automatic ticket gate, information for using an automatic ticket gate linked to a registered person corresponding to the user identified by the facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station.

[0045] (Technical Effect) When an information processing device (e.g., information processing device 10) that detects entry and exit of a railway station using facial recognition is introduced to railway stations, it is highly unlikely that the information processing device will be introduced to all railway stations at the same time. For example, in the early stages of introduction of the information processing device, the information processing device and an existing system related to automatic ticket gates will coexist. This could result in a situation where an information processing device is introduced to one railway station while only automatic ticket gates are installed at other railway stations. In such a situation, if no countermeasures are taken, when a user who enters one railway station using facial recognition passes through an automatic ticket gate at another railway station and exits, there is a possibility that railway fare (e.g., fare) cannot be properly collected from the user.

[0046] For example, the information processing device 10 identifies at least one of a user entering station A and a user exiting station A through facial recognition. Then, the information processing device 10 links the ticket information of the user identified through facial recognition (for example, information relating to at least one of an IC card and a mobile card) with one of information indicating entry into station A and information indicating exit from station A, and transmits the linked information to the station server 41. As a result, the entry / exit record 51 of the center server 50 registers that the user has entered station A or exited station A. Therefore, it can be said that the information processing device 10 registers that the user has entered station A or exited station A in the system related to the automatic ticket gate.

[0047] The above-described operation of the information processing device 10 integrates information related to entering and exiting railway stations detected by facial recognition and information related to entering and exiting railway stations detected by an automatic ticket gate. Therefore, even if a user enters one railway station using facial recognition and then exits another railway station by passing through an automatic ticket gate, railway charges (e.g., fares) can be appropriately collected from the user.

[0048] Second Embodiment A second embodiment relating to an information processing device, an information processing method, and a recording medium will be described with reference to Figures 6 to 8. In the second embodiment, the ticket gate system 1 may include an information processing device 20 instead of the information processing device 10. Below, the second embodiment relating to the information processing device, the information processing method, and the recording medium will be described using the information processing device 20. Note that, with regard to the second embodiment, descriptions that overlap with the description of the first embodiment described above will be omitted as appropriate.

[0049] In the second embodiment, the information processing device 20 may be installed, for example, at station A. At station A, the information processing device 20 may be connected to a station server 41 via a narrow area network such as a LAN. Note that, for example, an information processing device similar to the information processing device 20 may be installed at station B.

[0050] 6, the information processing device 20 includes a calculation device 21, a storage device 22, and a communication device 23. The information processing device 20 may further include an input device 24 and an output device 25. The information processing device 20 may include a face information database 26 (hereinafter, appropriately referred to as "face information DB 26"). Note that the information processing device 20 does not necessarily include at least one of the input device 24 and the output device 25. In the information processing device 20, the calculation device 21, the storage device 22, the communication device 23, the input device 24, the output device 25, and the face information DB 26 may be connected via a data bus 27.

[0051] The face information DB 26 may store the same information as that stored in the face information DB 60. The face information DB 26 may be updated at a predetermined interval so that the information stored in the face information DB 26 is the same as that stored in the face information DB 60.

[0052] The arithmetic device 21 may include, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a TPU (Tensor Processing Unit), and a quantum processor.

[0053] The storage device 22 may include, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and an optical disk array. In other words, the storage device 22 may include a non-transitory recording medium. The storage device 22 is capable of storing desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that is temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program.

[0054] The communication device 23 may be capable of communicating with devices external to the information processing device 20. The communication device 23 may perform wired communication or wireless communication.

[0055] The input device 24 is a device capable of accepting information input to the information processing device 20 from outside. The input device 24 may include an operation device (e.g., a keyboard, a mouse, a touch panel, etc.) that can be operated by an operator of the information processing device 20. The input device 24 may include a recording medium reading device that can read information recorded on a recording medium that is detachable from the information processing device 20, such as a USB (Universal Serial Bus) memory. Note that when information is input to the information processing device 20 via the communication device 23 (in other words, when the information processing device 20 acquires information via the communication device 23), the communication device 23 may function as an input device.

[0056] The output device 25 is a device capable of outputting information to the outside of the information processing device 20. The output device 25 may output visual information such as text or images, auditory information such as sound, or tactile information such as vibration, as the information. The output device 25 may include, for example, at least one of a display, a speaker, a printer, and a vibration motor. The output device 25 may be capable of outputting information to a recording medium detachable from the information processing device 20, such as a USB memory. Note that when the information processing device 20 outputs information via the communication device 23, the communication device 23 may function as the output device.

[0057] The computing device 21 may have an image acquisition unit 211, a tracking unit 212, a face authentication unit 213, and an information processing unit 214 as logically realized functional blocks or as physically realized processing circuits.

[0058] At least one of the image acquisition unit 211, the tracking unit 212, the face authentication unit 213, and the information processing unit 214 may be realized in a format in which a logical functional block and a physical processing circuit (i.e., hardware) are mixed. When at least some of the image acquisition unit 211, the tracking unit 212, the face authentication unit 213, and the information processing unit 214 are functional blocks, at least some of the image acquisition unit 211, the tracking unit 212, the face authentication unit 213, and the information processing unit 214 may be realized by the calculation device 21 executing a predetermined computer program.

[0059] The arithmetic device 21 may acquire (in other words, read) the predetermined computer program from the storage device 22. The arithmetic device 21 may read the predetermined computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (not shown) included in the information processing device 20. The arithmetic device 21 may acquire (in other words, download or read) the predetermined computer program from a device (not shown) external to the information processing device 20 via the communication device 23. Note that the recording medium for recording the predetermined computer program executed by the arithmetic device 21 may be at least one of an optical disk, a magnetic medium, a magneto-optical disk, a semiconductor memory, and any other medium capable of storing a program.

[0060] In addition, the "image acquisition unit 211", "face authentication unit 213", "information processing unit 214", and "face information DB 26" are components corresponding to the "image acquisition unit 11", "face authentication unit 12", "information processing unit 13", and "face information DB 14" in the first embodiment described above, respectively.

[0061] (Operation of Information Processing Device) The information processing device 20 performs face authentication of the person to be authenticated (i.e., a user of Station A) based on the image captured by the camera CAM. The information processing device 20 may perform face authentication of the person to be authenticated while the person is passing through the face authentication area Ar. In other words, the information processing device 20 may be a device capable of performing walk-through face authentication.

[0062] Furthermore, if there are no devices installed in or around the facial authentication area Ar that can physically obstruct the passage of the person to be authenticated, the facial authentication method performed by the information processing device 20 may be referred to as gateless facial authentication.

[0063] The image acquisition unit 211 acquires an image including the face authentication area Ar captured by the camera CAM. The tracking unit 212 performs tracking processing of one or more subjects (in other words, users of station A) passing through the face authentication area Ar, using the image acquired by the image acquisition unit 211. Furthermore, the face authentication unit 213 performs face authentication processing of one or more subjects passing through the face authentication area Ar, using the image acquired by the image acquisition unit 211.

[0064] As a specific example of the tracking process performed by the tracking unit 212, a case where the tracking unit 212 performs tracking process on a person P1 as a person to be authenticated who passes through the face authentication area Ar will be described with reference to Fig. 7. Also, as a specific example of the face authentication process performed by the face authentication unit 213, a case where the face authentication unit 213 performs face authentication process on a person P1 as a person to be authenticated will be described.

[0065] 7 are images including person P1 corresponding to the person to be authenticated (in other words, images in which person P1 is captured). Image IMG1 is an image generated by camera CAM capturing an image of face authentication area Ar at time t1. Image IMG2 is an image generated by camera CAM capturing an image of face authentication area Ar at time t2, which is later than time t1. Image IMG3 is an image generated by camera CAM capturing an image of face authentication area Ar at time t3, which is later than time t2.

[0066] Each of images IMG1, IMG2, and IMG3 may be an image corresponding to one frame of a video. In this case, image IMG2 does not have to be an image corresponding to the frame immediately following the frame corresponding to image IMG1. In other words, image IMG2 may be an image corresponding to a frame that is two or more frames after the frame corresponding to image IMG1. Similarly, image IMG3 does not have to be an image corresponding to the frame immediately following the frame corresponding to image IMG2. In other words, image IMG3 may be an image corresponding to a frame that is two or more frames after the frame corresponding to image IMG2.

[0067] The tracking unit 212 detects the head of person P1 included in image IMG1 from image IMG1. Note that the method for detecting a person's head from an image is an existing technique, so a detailed description thereof will be omitted. Based on the detected head of person P1, the tracking unit 212 sets an area including the head of person P1 as a tracking area TA1.

[0068] When the tracking unit 212 sets the tracking area TA1, it sets a tracking ID, which is identification information for identifying the person P1 related to the tracking area TA1, to the person P1. The tracking unit 212 calculates the position of the person P1 (for example, the position of the person P1's feet) based on the tracking area TA1. Note that since existing technology can be applied to a method for calculating the position of a subject included in an image from the image, a detailed description thereof will be omitted. The tracking unit 212 associates the tracking ID with the position of the person P1 and registers them in an ID correspondence table 221 stored in the storage device 22.

[0069] When the tracking area TA1 is set, the tracking unit 212 determines whether or not the face of person P1 is captured in the tracking area TA1. In other words, the tracking unit 212 performs face detection on the tracking area TA1. When it is determined that the face of person P1 is captured in the tracking area TA1, the tracking unit 212 generates a face image including the face area of ​​person P1. The tracking unit 212 associates the generated face image with the tracking ID related to person P1 and transmits it to the face authentication unit 213. Note that when it is determined that the face of person P1 is not captured in the tracking area TA1, the tracking unit 212 does not need to generate a face image.

[0070] The face authentication unit 213 performs face authentication processing using the face image transmitted from the tracking unit 212. For example, the face authentication unit 213 may calculate a face feature amount indicating features related to the face of the person P1 from the face image. The face authentication unit 213 may calculate a matching score from the calculated face feature amount and a face feature amount based on the face information of one registrant registered in the face information DB 26.

[0071] If the calculated matching score is greater than a predetermined threshold, the face authentication unit 213 determines that face authentication has been successful. In this case, one registrant corresponding to the person (here, person P1) indicated by the face image transmitted from the tracking unit 212 is identified from among the multiple registrants registered in the face information DB 26. On the other hand, if the calculated matching score is less than the predetermined threshold, the face authentication unit 213 determines that face authentication has failed. In this case, it is highly likely that the person corresponding to the person (here, person P1) indicated by the face image transmitted from the tracking unit 212 is not registered in the face information DB 26. Note that if the calculated matching score and the predetermined threshold are "equal," either case may be included.

[0072] If the facial authentication is successful, the facial authentication unit 213 associates an authentication ID indicating a registrant registered in the facial information DB 26 that corresponds to the person (person P1 in this case) indicated by the facial image transmitted from the tracking unit 212 with the tracking ID associated with the facial image, and registers the associated ID in the ID correspondence table 221. In addition to the authentication ID, the facial authentication unit 213 may register an authentication time, which is the time when the facial authentication process was performed, in the ID correspondence table 221. If the facial authentication is unsuccessful, the facial authentication unit 213 may associate information indicating that there is no corresponding person (for example, "N / A (Not Applicable)") with the tracking ID associated with the facial image transmitted from the tracking unit 212, and register the associated information in the ID correspondence table 221.

[0073] The ID correspondence table 221 will be further described with reference to FIG. 8 . As shown in FIG. 8 , in the ID correspondence table 221, a tracking ID, a tracking position (e.g., the position of the feet of person P1), an authentication ID, and an authentication time may be associated with one another. In the ID correspondence table 221, when a tracking ID is associated with a specific authentication ID, this indicates that a registrant corresponding to the person for whom the tracking ID is set is registered in the face information DB 26. When a tracking ID is associated with the character string "N / A", this indicates that a person corresponding to the person for whom the tracking ID is set is not registered in the face information DB 26. When a tracking ID is not associated with an authentication ID (i.e., when the authentication ID field is blank), this indicates that face authentication processing has never been performed.

[0074] The tracking unit 212 may use the image IMG2 and the image IMG1 to identify the person P1 included in the image IMG2. Identifying the person P1 included in the image IMG2 is synonymous with associating the person P1 included in the image IMG1 with the person P1 included in the image IMG2. Therefore, at least one of a matching method and an optical flow method related to association between images can be applied to identifying the person P1 included in the image IMG2. Note that various existing methods can be applied to both the matching method and the optical flow method, and therefore detailed description thereof will be omitted.

[0075] When person P1 included in image IMG2 is identified, the tracking unit 212 detects the head of person P1. Based on the detected head of person P1, the tracking unit 212 sets an area including the head of person P1 as tracking area TA2. Since person P1 included in image IMG1 and person P1 included in image IMG2 are the same person, the tracking ID of person P1 related to tracking area TA2 is the same as the tracking ID of person P1 related to tracking area TA1. The tracking unit 212 calculates the position of person P1 based on tracking area TA2.

[0076] The tracking unit 212 associates the position of the person P1 with the tracking ID associated with the person P1 and registers it in the ID correspondence table 221. In this case, since the position of the person P1 calculated based on the tracking area TA1 is registered in the ID correspondence table 221, the tracking unit 212 registers the position of the person P1 calculated based on the tracking area TA2 in the ID correspondence table 221, thereby updating the position of the person P1.

[0077] When the tracking area TA2 is set, the tracking unit 212 determines whether the face of person P1 is captured in the tracking area TA2. In other words, the tracking unit 212 performs face detection on the tracking area TA2. When it is determined that the face of person P1 is captured in the tracking area TA2, the tracking unit 212 generates a face image including the face area of ​​person P1. The tracking unit 212 associates the generated face image with the tracking ID related to person P1 and transmits the associated face image to the face authentication unit 213. Note that, when it is determined that the face of person P1 is not captured in the tracking area TA2, the tracking unit 212 does not need to generate a face image. Note that, even when it is determined that the face of person P1 is captured in the tracking area TA2, when the tracking ID related to person P1 is associated with the authentication ID in the ID correspondence table 221 (i.e., when face authentication has already been successful), the tracking unit 212 does not need to generate a face image.

[0078] The face authentication unit 213 performs face authentication processing using the face image transmitted from the tracking unit 212. If face authentication is successful, the face authentication unit 213 associates an authentication ID indicating a registrant registered in the face information DB 26 corresponding to the person (here, person P1) indicated by the face image transmitted from the tracking unit 212 with the tracking ID linked to the face image and registers the ID in the ID correspondence table 221. In addition to the authentication ID, the face authentication unit 213 may register an authentication time, which is the time when the face authentication processing was performed, in the ID correspondence table 221. Note that if the authentication time has already been registered in the ID correspondence table 221 (i.e., if face authentication has been successful in the past), the face authentication unit 213 may update the authentication time registered in the ID correspondence table 221. If face authentication is unsuccessful, the face authentication unit 213 may associate information indicating that there is no corresponding person (e.g., “N / A”) with the tracking ID linked to the face image transmitted from the tracking unit 212 and register the information in the ID correspondence table 221.

[0079] The tracking unit 212 may use the images IMG3 and IMG2 to identify the person P1 included in the image IMG3. When the person P1 included in the image IMG3 is identified, the tracking unit 212 detects the head of the person P1. Based on the detected head of the person P1, the tracking unit 212 sets an area including the head of the person P1 as a tracking area TA3. Because the person P1 included in the image IMG2 and the person P1 included in the image IMG3 are the same person, the tracking ID of the person P1 related to the tracking area TA3 is the same as the tracking ID of the person P1 related to the tracking area TA2. The tracking unit 212 calculates the position of the person P1 based on the tracking area TA3.

[0080] The tracking unit 212 associates the position of the person P1 with the tracking ID associated with the person P1 and registers it in the ID correspondence table 221. In this case, since the position of the person P1 calculated based on the tracking area TA2 is registered in the ID correspondence table 221, the tracking unit 212 registers the position of the person P1 calculated based on the tracking area TA3 in the ID correspondence table 221, thereby updating the position of the person P1.

[0081] When the tracking area TA3 is set, the tracking unit 212 determines whether the face of person P1 is captured in the tracking area TA3. In other words, the tracking unit 212 performs face detection on the tracking area TA3. When it is determined that the face of person P1 is captured in the tracking area TA3, the tracking unit 212 generates a face image including the face area of ​​person P1. The tracking unit 212 associates the generated face image with the tracking ID related to person P1 and transmits the associated face image to the face authentication unit 213. Note that, when it is determined that the face of person P1 is not captured in the tracking area TA3, the tracking unit 212 does not need to generate a face image. Note that, even when it is determined that the face of person P1 is captured in the tracking area TA3, when the tracking ID related to person P1 is associated with the authentication ID in the ID correspondence table 221 (i.e., when face authentication has already been successful), the tracking unit 212 does not need to generate a face image.

[0082] The face authentication unit 213 performs face authentication processing using the face image transmitted from the tracking unit 212. If face authentication is successful, the face authentication unit 213 associates an authentication ID indicating a registrant registered in the face information DB 26 corresponding to the person (here, person P1) indicated by the face image transmitted from the tracking unit 212 with the tracking ID linked to the face image and registers the ID in the ID correspondence table 221. In addition to the authentication ID, the face authentication unit 213 may register an authentication time, which is the time when the face authentication processing was performed, in the ID correspondence table 221. Note that if the authentication time has already been registered in the ID correspondence table 221 (i.e., if face authentication has been successful in the past), the face authentication unit 213 may update the authentication time registered in the ID correspondence table 221. If face authentication is unsuccessful, the face authentication unit 213 may associate information indicating that there is no corresponding person (e.g., “N / A”) with the tracking ID linked to the face image transmitted from the tracking unit 212 and register the information in the ID correspondence table 221.

[0083] However, it may become difficult for the tracking unit 212 to continue tracking the person P1, for example, because the head of the person P1 is hidden behind another person. When tracking of the person P1 is interrupted, the tracking unit 212 may perform the following process. After tracking of the person P1 is interrupted, the tracking unit 212 may determine whether a new person has been detected from an image acquired by the image acquisition unit 211 (for example, an image captured by the camera CAM). A "new person" means a person for whom a tracking ID has not been set.

[0084] When a new person is detected, the tracking unit 212 may determine whether the person P1 and the new person are the same person by comparing the feature amount of the tracking area (e.g., at least one of the tracking areas TA1, TA2, and TA3) related to the person P1 with the feature amount of the tracking area related to the new person. If it is determined that the person P1 and the new person are the same person, the tracking unit 212 may set the tracking ID related to the person P1 to the new person. As a result, the tracking unit 212 can track the person P1 again.

[0085] The feature amount may be a feature amount related to the person's head, a feature amount related to the person's upper body, or a feature amount related to the person's entire body. In other words, the tracking area (for example, at least one of tracking areas TA1, TA2, and TA3) may include the person's head, the person's upper body, or the person's entire body. The feature amount may be obtained, for example, by person re-identification technology.

[0086] The tracking unit 212 and the face authentication unit 213 may repeat the above-described process until the person P1 passes through the face authentication area Ar. The tracking unit 212 and the face authentication unit 213 may perform the above-described process for all persons to be authenticated who pass through the face authentication area Ar.

[0087] A first predetermined position and a second predetermined position may be set in the face authentication area Ar. The first predetermined position may be set in the face authentication area Ar outside Station A relative to the second predetermined position. In other words, the second predetermined position may be set in the face authentication area Ar inside Station A relative to the first predetermined position.

[0088] The ID correspondence table 221 may include a history of tracking positions associated with each authentication ID. For example, the information processing unit 214 may determine, based on the history of tracking positions, that a registrant who passed a first predetermined position and then a second predetermined position in the face authentication area Ar is a registrant who entered station A. For example, the information processing unit 214 may determine, based on the history of tracking positions, that a registrant who passed a second predetermined position and then a first predetermined position in the face authentication area Ar is a registrant who exited station A.

[0089] Based on the ID correspondence table 221 and the face information DB 26, the information processing unit 214 acquires ticket information linked to the authentication ID of the registered person corresponding to the person to be authenticated (e.g., person P1) identified by the face authentication unit 213 performing face authentication.

[0090] If the registrant corresponds to an authenticated person who entered station A, the information processing unit 214 may link the ticket information (for example, information relating to at least one of an IC ticket and a mobile ticket) with information indicating entry into station A and transmit the linked information to the station server 41. The station server 41 may transmit the ticket information and information indicating entry into station A transmitted from the information processing unit 214 to the center server 50. As a result, the entry of the registrant into station A may be registered in the entry / exit record 51.

[0091] If the registrant corresponds to an authenticated person who exited at station A, the information processing unit 13 may link the ticket information (for example, information relating to at least one of an IC ticket and a mobile ticket) with information indicating that the person exited at station A and transmit the linked information to the station server 41. The station server 41 may transmit the ticket information and information indicating that the person exited at station A transmitted from the information processing unit 214 to the center server 50. As a result, the fact that the registrant exited at station A may be registered in the entry / exit record 51.

[0092] (Technical effect) According to the information processing device 20, similar to the information processing device 10 according to the first embodiment described above, even if a user enters one railway station using facial recognition and then exits another railway station by passing through an automatic ticket gate, it is possible to appropriately collect railway charges (e.g., fares) from the user.

[0093] <Supplementary Notes> The following supplementary notes are further disclosed regarding the above-described embodiment.

[0094] (Supplementary Note 1) An information processing device comprising: an image acquisition means for acquiring an image including a facial recognition area set in a ticket gate area of ​​a railway station and through which users of the railway station pass; a facial recognition means for performing facial recognition of the user based on the image; and an information processing means for registering, in a system related to the automatic ticket gate, information for using the automatic ticket gate linked to a registered person corresponding to the user identified by the facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station.

[0095] (Supplementary Note 2) The information processing device according to Supplementary Note 1, further comprising a tracking means for tracking the person to be authenticated based on the image, and the facial authentication means for performing facial authentication of the person to be authenticated while the tracking means is tracking the person to be authenticated.

[0096] (Supplementary Note 3) The information processing device according to Supplementary Note 1 or 2, wherein the information for using the automatic ticket gate is information relating to at least one of an IC ticket and a mobile ticket.

[0097] (Supplementary Note 4) An information processing method comprising: acquiring an image including a facial recognition area set in a ticket gate area of ​​a railway station and through which a user of the railway station passes; performing facial recognition of the user based on the image; and registering, in a system related to the automatic ticket gate, information for using the automatic ticket gate linked to a registered person corresponding to the user identified by the facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station.

[0098] (Supplementary Note 5) A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method, the method comprising: acquiring an image including a facial recognition area that is set in a ticket gate area of ​​a railway station and through which users of the railway station pass; performing facial recognition of the user based on the image; and registering, in a system related to the automatic ticket gate, information for using the automatic ticket gate linked to the registered person corresponding to the user identified by the facial recognition, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station.

[0099] This disclosure is not limited to the above-described embodiments, but may be modified as appropriate within the scope of the claims and the gist or idea of ​​the invention as can be read from the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical scope of this disclosure.

[0100] 1 Ticket gate system 10, 20 Information processing device 11, 211 Image acquisition unit 12, 213 Face authentication unit 13, 214 Information processing unit 14, 26, 60 Face information database 212 Tracking unit

Claims

1. an image acquisition means for acquiring an image including a face authentication area set in a ticket gate area of ​​a railway station and through which a user of the railway station passes; a face authentication means for performing face authentication of the user using the image and face information of a registered person who wishes to use the system related to the automatic ticket gate, the face information having been stored in advance in a storage means; an information processing means for registering, in a system related to the automatic ticket gate, information for using the automatic ticket gate that is linked to the registered person corresponding to the user identified by the face authentication, and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station; An information processing device comprising:

2. the information processing device includes a tracking means for tracking the user based on the image; The face authentication means performs face authentication of the user while the tracking means is tracking the user. The information processing device according to claim 1 .

3. The information for using the automatic ticket gate is information about at least one of an IC ticket and a mobile ticket. The information processing device according to claim 1 .

4. Acquire an image including a face recognition area set in a ticket gate area of ​​a railway station and through which a user of the railway station passes; performing face authentication of the user using the image and face information of a registered person who wishes to use the system related to the automatic ticket gate, the face information having been stored in advance in a storage means; Information for using the automatic ticket gate linked to the registered person corresponding to the user identified by the face authentication and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station are registered in a system related to the automatic ticket gate. Information processing methods.

5. On the computer, Acquire an image including a face recognition area set in a ticket gate area of ​​a railway station and through which a user of the railway station passes; performing face authentication of the user using the image and face information of a registered person who wishes to use the system related to the automatic ticket gate, the face information having been stored in advance in a storage means; Registering information for using the automatic ticket gate linked to the registered person corresponding to the user identified by the face authentication and information indicating at least one of the user's entry into the railway station and the user's exit from the railway station in a system related to the automatic ticket gate. A recording medium on which a computer program for executing an information processing method is recorded.