Display control device, display control method, and recording medium

Abstract

This display control device comprises: an image acquisition means that acquires an image including a predetermined area that a person passes through heading for a facial recognition gate device; and a display control means that displays a result of the facial recognition using the acquired image before the person reaches the facial recognition gate device. The display control means changes the display mode according to the result of the facial recognition.

Description

Display control device, display control method, and recording medium

[0001] The present disclosure relates to the technical fields of a display control device, a display control method, and a recording medium.

[0002] For example, a device has been proposed that performs facial recognition on passengers who have completed immigration procedures at airports, and displays a guide when it is determined that the passenger needs to undergo a quarantine inspection based on health information and aircraft seat information associated with the person identified by the facial recognition (see Patent Document 1).Other prior art documents related to this disclosure include Patent Documents 2 to 4.

[0003] International Publication No. 2022 / 074847 International Publication No. 2022 / 044086 International Publication No. 2022 / 003853 Japanese Patent Application Laid-Open No. 2021-163457

[0004] An object of this disclosure is to provide a display control device, a display control method, and a recording medium that aim to improve upon the techniques described in prior art documents.

[0005] One aspect of the display control device comprises an image acquisition means for acquiring an image including a predetermined area through which a person heading towards a facial recognition gate device passes, and a display control means for displaying the results of facial recognition performed using the acquired image before the person reaches the facial recognition gate device, and the display control means changes the display mode based on the results of the facial recognition.

[0006] One aspect of the display control method involves acquiring an image including a predetermined area through which a person heading towards a facial recognition gate device will pass, displaying the results of facial recognition performed using the acquired image before the person reaches the facial recognition gate device, and changing the display mode based on the results of the facial recognition.

[0007] One aspect of the recording medium has recorded thereon a computer program for causing a computer to execute a display control method that acquires an image including a predetermined area through which a person heading towards a facial recognition gate device will pass, displays the results of facial recognition performed using the acquired image before the person reaches the facial recognition gate device, and changes the display mode based on the results of the facial recognition.

[0008] 1 is a block diagram showing an example of the configuration of a display control device. FIG. 2 is a block diagram showing an example of the configuration of an airport system. FIG. 3 is a plan view showing an example of a floor where airport boarding gates are provided. FIG. 4 is a plan view showing an example of a virtual authentication area. FIG. 5 is a block diagram showing another example of the configuration of a display control device. FIG. 6 is a diagram showing an example of a face database provided in a display control device. FIG. 7 is a diagram for explaining passenger tracking processing. FIG. 8 is a diagram showing an example of an ID correspondence table. FIG. 9 is a diagram showing an example of an image. FIG. 10 is a flowchart showing tracking and authentication operations according to a second embodiment. FIG. 11 is a diagram showing an example of a display device. FIG. 12 is a flowchart showing display control operations according to the second embodiment. FIG. 13 is a block diagram showing an example of the configuration of an authentication device. FIG. 14 is a diagram showing another example of a display device. FIG. 15 is a diagram showing an example of a terminal device. FIG. 16 is a plan view showing another example of a virtual authentication area.

[0009] An embodiment relating to a display control device, a display control method, and a recording medium will be described.

[0010] <First embodiment> A first embodiment of a display control device, a display control method, and a recording medium will be described with reference to Fig. 1. Below, the first embodiment of the display control device, the display control method, and the recording medium will be described using a display control device 1. In Fig. 1, the display control device 1 includes an image acquisition unit 11 and a display control unit 12. The image acquisition unit 11 acquires an image including a predetermined area through which a person heading towards the facial recognition gate device passes (in other words, an image in which the predetermined area is captured).

[0011] The facial recognition gate device allows passage of people whose facial authentication is successful, but does not allow passage of people whose facial authentication is unsuccessful. The facial recognition gate device may determine that facial authentication is successful when the face of a person attempting to pass through the facial recognition gate device corresponds to a face shown in a pre-registered facial image. In other words, the facial recognition gate device may determine that facial authentication is successful when there is a person whose pre-registered facial image corresponds to the person attempting to pass through the facial recognition gate device.

[0012] Therefore, if the facial image of a person attempting to pass through the facial recognition gate device is not registered, the facial recognition gate device will not allow that person to pass through (for example, if the facial recognition gate device is a flap-type gate device, the flap will be closed). Therefore, when a person whose facial image is not registered enters the facial recognition gate device, the flow of people passing through the facial recognition gate device is impeded because that person is not allowed to pass through. In other words, the throughput of the facial recognition gate device will decrease.

[0013] The display control unit 12 displays the results of facial authentication performed using the image acquired by the image acquisition unit 11 before a person attempting to pass through the facial recognition gate device reaches the facial recognition gate device. At this time, the display control unit 12 changes the display mode based on the results of facial authentication. The display mode may include the appearance and state of the display. Changing the display mode may mean changing at least one of the appearance and state of the display. In other words, changing the display mode may include changing the appearance of the display, changing the state of the display, and changing the appearance and state of the display.

[0014] The appearance of the display may be represented by at least one of a character string (e.g., a message), a graphic, a color, and a pattern. The state of the display may include at least one of on, off, blinking, and brightness. Changing the appearance of the display may mean changing at least one of a character string, a graphic, a color, and a pattern. Changing the state of the display may include switching the display between on, off, and blinking, and changing the brightness of the display. Note that switching the display between on, off, and blinking may be achieved by changing the brightness of the display. Changing the display mode based on the result of facial authentication may mean making the display mode of one facial authentication result different from the display mode of another facial authentication result.

[0015] The result of the facial authentication may include, for example, a result indicating that the person can pass through the facial authentication gate device and a result indicating that the person cannot pass through the facial authentication gate device. The display control unit 12 may differentiate the display mode when the result of the facial authentication indicates that the person can pass through the facial authentication gate device from the display mode when the result of the facial authentication indicates that the person cannot pass through the facial authentication gate device.

[0016] If the results of facial recognition are displayed in a manner based on the results of facial recognition, a person attempting to pass through the facial recognition gate device can relatively easily determine whether or not they can pass through the facial recognition gate device. In addition, at least one of a guide and a security guard near the facial recognition gate device can relatively easily determine who cannot pass through the facial recognition gate device.

[0017] If it is determined that a person attempting to pass through a predetermined gate device cannot pass through the facial recognition gate device, the person can be prevented from entering the facial recognition gate device. Furthermore, if at least one of the guide and the security guard can determine that a person cannot pass through the facial recognition gate device, the person who cannot pass through the facial recognition gate device can be guided not to enter the facial recognition gate device. Therefore, the display control device 1 can prevent a person who cannot pass through the facial recognition gate device from entering the facial recognition gate device. Therefore, the display control device 1 can prevent a decrease in the throughput of the facial recognition gate device.

[0018] In the display control device 1, the image acquisition unit 11 may acquire an image including a predetermined area through which a person heading toward the facial recognition gate device passes. The display control unit 12 may display the result of facial recognition performed using the image acquired by the image acquisition unit 11 before the person reaches the facial recognition gate device. The display control unit 12 may change the display mode based on the result of facial recognition.

[0019] Such a display control device 1 may be realized, for example, by a computer reading a computer program recorded on a recording medium. In this case, it can be said that the recording medium stores a computer program for causing the computer to execute a display control method that acquires an image including a predetermined area through which a person heading toward the facial recognition gate device will pass, displays the results of facial recognition performed using the acquired image before the person reaches the facial recognition gate device, and changes the display mode based on the results of the facial recognition.

[0020] Second Embodiment A second embodiment of a display control device, a display control method, and a recording medium will be described with reference to Fig. 2 to Fig. 13. In the following, the second embodiment of a display control device, a display control method, and a recording medium will be described using a display control device 3 used in airports.

[0021] First, an airport system 2 at an airport will be described with reference to Fig. 2. In Fig. 2, the airport system 2 includes a management server 21, a check-in terminal 22, and a plurality of gate devices installed at boarding gates. The management server 21, the check-in terminal 22, and the plurality of gate devices are connected to one another via a network NW.

[0022] The airport system 2 may include other check-in terminals in addition to the check-in terminal 22. That is, the airport system 2 may include a plurality of check-in terminals. The network NW may be a wide area network (WAN) such as the Internet, or a local area network (LAN).

[0023] The multiple gate devices have a facial recognition function that performs facial recognition processing, and include a facial recognition gate device 23 that determines whether a passenger can pass through based on the results of the facial recognition processing, and a gate device 24 that determines whether a passenger can pass through based on the airline ticket held by the passenger. Hereinafter, the gate device 24 will be referred to as a "normal gate device 24" as appropriate. As shown in Figure 3, the airport is assumed to be equipped with boarding gates G1, G2, and G3. The facial recognition gate device 23 and the normal gate device 24 are assumed to be installed at boarding gate G1.

[0024] The term "air ticket" may refer to at least one of a paper air ticket and an electronic air ticket. The term "air ticket" may also refer to a concept including, in addition to or instead of at least one of a paper air ticket and an electronic air ticket, personal information associated with an electronic air ticket (for example, a passport, a credit card used to purchase the air ticket, etc.).

[0025] The management server 21 has a face database 211 (hereinafter referred to as "face DB211"), an operation database 212 (hereinafter referred to as "operation DB212"), and a gate database 213 (hereinafter referred to as "gate DB213").

[0026] The face DB 211 stores face images used in face authentication processing. Instead of or in addition to face images, feature quantities related to face images may be stored in the face DB 211. The flight DB 212 stores aircraft flight information (so-called flight information). The flight information may include the flight name of the aircraft, gate identification information for identifying the boarding gate (e.g., boarding gate number), and boarding start time. The flight name, identification information, and boarding start time of the aircraft may be associated with each other.

[0027] The gate DB 213 stores a plurality of pieces of boarding gate information corresponding to a plurality of boarding gates. Note that the entity that manages and / or operates the airport system 2 may be different from the airport company that manages and operates the airport and the airline that operates the aircraft. In this case, the plurality of pieces of boarding gate information stored in the gate DB 213 may be acquired from a system related to at least one of the airport company and the airline. The gate DB 213 may be shared between the airport system 2 and a system related to at least one of the airport company and the airline.

[0028] The boarding gate information may include gate identification information (e.g., a boarding gate number) for identifying the boarding gate and device information related to the installed gate device. The device information may indicate the number of gate devices and at least one of the type, operating status, and installation location of each gate device. The types of gate devices may include a first type capable of facial recognition and a second type not capable of facial recognition. The second type of gate device may include a gate device capable of reading information (e.g., an ID) for identifying passengers (e.g., a two-dimensional barcode). The operating status of the gate device may include an operating state in which the gate device is operating and an inactive state in which the gate device is inactive (i.e., not operating). For example, if two or more gate devices are installed at one boarding gate, the installation locations of the gate devices may be expressed as the relative positions of the two or more gate devices. Alternatively, the installation locations of the gate devices may be expressed as the coordinates of each gate device in a coordinate system related to the floor (the so-called departure floor) where the airport boarding gates are located. If only one gate device is installed at one boarding gate, the device information does not need to include the installation location of the gate device. The inactive state may include a state in which no power is supplied to the gate device (e.g., a power-off state) and a state in which power is supplied to part of the gate device but the gate device cannot perform its function (a so-called standby state). The multiple boarding gate information registered in gate DB 213 may be predetermined according to at least one of the airline and the flight. The multiple boarding gate information registered in gate DB 213 may be automatically updated according to the departure time of the aircraft, etc.

[0029] The check-in terminal 22 is a terminal used for boarding procedures (i.e., check-in). The check-in terminal 22 may be operated by an airport staff member or a passenger (i.e., the check-in terminal 22 may be a so-called automatic check-in machine). Note that the term "airport staff member" is not limited to a person who belongs to an airport company that manages and operates the airport, but also includes a person who works at the airport, such as a person who belongs to an airline that operates aircraft.

[0030] The check-in terminal 22 performs check-in procedures for passengers based on the airline tickets held by the passengers. The check-in terminal 22 may acquire facial images of passengers during check-in procedures. The check-in terminal 22 may acquire facial images of passengers by capturing images of the passengers with a camera, or may acquire facial images of passengers by reading facial photographs printed on the passports held by the passengers. Note that the check-in terminal 22 does not need to acquire facial images of all passengers.

[0031] The check-in terminal 22 transmits passenger information (e.g., name and flight number) of a passenger who has completed check-in procedures and a facial image of the passenger to the management server 21. Note that the check-in terminal 22 may obtain the passenger information from information printed on the airline ticket (in other words, information associated with the airline ticket).

[0032] The management server 21 associates the passenger information and facial image transmitted from the check-in terminal 22 with each other and registers them in the face DB 211. At this time, the management server 21 may assign passenger identification information for identifying the passenger to the passenger information and facial image. As will be described later, the facial image is used in facial authentication processing. For this reason, the passenger identification information assigned to the passenger information and facial image will hereinafter be referred to as an "authentication ID" as appropriate. Note that the passenger's facial image may be acquired by an application (e.g., an application provided by an airline) installed on a terminal device (e.g., a smartphone) carried by the passenger, in addition to or instead of the check-in terminal 22, or may be acquired by a gate device installed in the departure immigration area of ​​the airport.

[0033] Based on the current time, the management server 21 may identify the flight name of an aircraft that is a first predetermined time (e.g., several minutes to several tens of minutes) before the boarding start time from the flight operation information registered in the flight operation DB 212. Based on the gate identification information associated with the identified flight name, the management server 21 may extract gate information corresponding to the gate identification information from multiple pieces of gate information registered in the gate DB 213. Based on the extracted gate information, the management server 21 may determine whether a facial recognition gate device (e.g., facial recognition gate device 23) is installed at the boarding gate corresponding to the extracted gate information.

[0034] If it is determined that a facial recognition gate device (e.g., the facial recognition gate device 23) is installed, the management server 21 may extract, from the face DB 211, a facial image of a passenger associated with passenger information including the specified flight name (in other words, a passenger scheduled to board an aircraft with the specified flight name). The management server 21 may transmit the extracted facial image to a facial recognition gate device (e.g., the facial recognition gate device 23) identified by the extracted gate information. If it is determined that a facial recognition gate device is not installed, the management server 21 may not perform these processes. If it is determined that a facial recognition gate device is installed, the management server 21 may determine whether the installed facial recognition gate device is operating based on the extracted gate information. If feature amounts related to facial images are registered in the face DB 211, the management server 21 may transmit the feature amounts related to the facial image to the facial recognition gate device instead of or in addition to the facial image.

[0035] The facial recognition gate device 23 has a camera 231, a face database 232 (hereinafter referred to as "face DB 232"), and a facial recognition device 233. Facial images transmitted from the management server 21 are registered in the face DB 232. The facial recognition gate device 23 may have a display device 234. The display device 234 may display at least one of an image captured by the camera 231 and information indicating the result of the facial recognition processing.

[0036] The face authentication device 233 performs face authentication processing using an image captured by the camera 231 (for example, an image that includes the passenger's face) and a face image registered in the face DB 232. If face authentication is successful, the face authentication gate device 23 allows the passenger to pass. On the other hand, if face authentication is unsuccessful, the face authentication gate device 23 does not allow the passenger to pass. Note that various existing methods (for example, at least one of a two-dimensional (2D) authentication method and a three-dimensional (3D) authentication method) can be applied to the face authentication processing.

[0037] A specific example of facial recognition processing in the facial recognition gate device 23 will be described. The facial recognition gate device 23 may be a flap-type gate device. The facial recognition device 233 may extract features of an image captured by the camera 231. The facial recognition device 233 may compare the extracted features with features related to facial images registered in the face DB 232. In this case, the facial recognition device 233 may calculate a matching score (or a similarity score) based on the extracted features and the features related to facial images registered in the face DB 232. If the matching score is equal to or greater than a threshold (i.e., if facial recognition is successful), the facial recognition device 233 may allow the passenger to pass. In this case, the facial recognition device 233 may identify an authentication ID related to the passenger. In this case, the facial recognition gate device 23 may open the flap. If the matching score is less than the threshold (i.e., if facial recognition is unsuccessful), the facial recognition device 233 may not allow the passenger to pass. In this case, the face authentication gate device 23 may close the flap. Note that the face DB 232 may store feature amounts related to face images instead of or in addition to face images.

[0038] Note that a device other than the facial recognition gate device 23 (e.g., the management server 21) may perform part of the facial recognition process. In this case, the facial recognition device 233 of the facial recognition gate device 23 may extract features of the image captured by the camera 231. The facial recognition device 233 may transmit the extracted features to the other device. The other device may compare the features transmitted from the facial recognition device 233 with features related to facial images registered in a face database (e.g., the face DB 211). The other device may transmit information indicating the matching result (e.g., information indicating whether the matching score is equal to or greater than a threshold) to the facial recognition gate device 23. Note that if the matching score is equal to or greater than the threshold, the other device may include an authentication ID related to the passenger in the information indicating the matching result. If the information indicating the matching result indicates that the matching score is equal to or greater than the threshold, the facial recognition device 233 may allow the passenger to pass. If the information indicating the matching result indicates that the matching score is less than the threshold, the facial recognition device 233 may not allow the passenger to pass.

[0039] It is expected that passengers will pass through the facial recognition gate device 23 at a faster speed than through the normal gate device 24. In other words, it is expected that the number of passengers passing through the facial recognition gate device 23 in a given period of time will be greater than the number of passengers passing through the normal gate device 24. In other words, the facial recognition gate device 23 can improve throughput at the boarding gate.

[0040] However, as described above, only passengers who have successfully passed face authentication are allowed to pass through the face authentication gate device 23. In order for a passenger to pass face authentication at the face authentication gate device 23, the face image of the passenger needs to be registered in the face DB 232 of the face authentication gate device 23. If a passenger whose face image is not registered in the face DB 232 enters the face authentication gate device 23, the passenger will not be permitted to pass through, which will result in a decrease in throughput at the boarding gate.

[0041] In view of the above-mentioned problems, the display control device 3 determines whether or not a passenger can pass through the facial recognition gate device 23 before the passenger enters the facial recognition gate device 23. In other words, the display control device 3 determines whether or not a passenger can pass through the facial recognition gate device 23, separately from the facial recognition gate device 23.

[0042] 4, a virtual authentication area RA1 is provided around boarding gate G1 where face recognition gate device 23 and normal gate device 24 are installed. Authentication area RA1 is an area through which multiple passengers who are heading to either face recognition gate device 23 or normal gate device 24 pass. Note that authentication area RA1 corresponds to an example of the "predetermined area" in the first embodiment described above.

[0043] Camera CAM1 is installed so as to be able to capture an image of authentication area RA1. Camera CAM1 may be connected to the display control device 3 via a network NW. Camera CAM1 does not have to be installed so that the entire authentication area RA1 is included in its angle of view. In other words, camera CAM1 may be installed so as to be able to capture an image of at least a portion of authentication area RA1. Camera CAM1 may be installed in a position that does not interfere with passenger passage (e.g., higher than the passenger's head). As will be described later, images captured by camera CAM1 may be used for facial recognition processing. For this reason, camera CAM1 may be a high-resolution camera, such as a 4K resolution camera. Camera CAM1 does not have to be connected to the display control device 3 via a network NW. In this case, camera CAM1 may be connected to the display control device 3 via a cable (e.g., a USB (Universal Serial Bus) cable).

[0044] The camera CAM1 may capture an image of the authentication area RA1 to generate a first image that includes the authentication area RA1. The first image may be an image that corresponds to one frame of a video.

[0045] The display control device 3 performs face recognition processing using a first image generated by the camera CAM1 capturing an image of the recognition area RA1, and determines whether or not a passenger included in the first image is permitted to pass through the face recognition gate device 23 before the passenger enters the face recognition gate device 23. The display control device 3 will be described in detail below.

[0046] As shown in FIG. 5 , the display control device 3 includes a calculation device 31, a storage device 32, and a communication device 33. The display control device 3 may include an input device 34 and an output device 35. The display control device 3 may include a face database 36 (hereinafter referred to as "face DB 36"). The display control device 3 does not need to include at least one of the input device 34 and the output device 35. In the display control device 3, the calculation device 31, the storage device 32, the communication device 33, the input device 34, the output device 35, and the face DB 36 may be connected via a data bus 37. The display control device 3 is connected to the management server 21 via the communication device 33 and the network NW. The display control device 3 may constitute a part of the airport system 2. In other words, the airport system 2 may include the display control device 3.

[0047] The arithmetic device 31 may include, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a TPU (Tensor Processing Unit), and a quantum processor.

[0048] The storage device 32 may include, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and an optical disk array. In other words, the storage device 32 may include a non-transitory recording medium. The storage device 32 is capable of storing desired data. For example, the storage device 32 may temporarily store a computer program executed by the arithmetic device 31. The storage device 32 may temporarily store data that is temporarily used by the arithmetic device 31 when the arithmetic device 31 is executing a computer program.

[0049] The communication device 33 is capable of communicating with the management server 21 via the network NW. The communication device 33 may be capable of communicating with devices external to the display control device 3 other than the management server 21 via the network NW. The communication device 33 may perform wired communication or wireless communication.

[0050] The input device 34 is a device capable of accepting information input to the display control device 3 from outside. It may include an operation device (e.g., a keyboard, a mouse, a touch panel, etc.) that can be operated by an operator of the display control device 3. The input device 34 may include a recording medium reading device that can read information recorded on a recording medium that is detachable from the display control device 3, such as a USB memory. Note that when information is input to the display control device 3 via the communication device 33 (in other words, when the display control device 2 acquires information via the communication device 33), the communication device 33 may function as an input device.

[0051] The output device 35 is a device capable of outputting information to the outside of the display control device 3. The output device 35 may output visual information such as characters or images, auditory information such as sound, or tactile information such as vibration, as the information. The output device 35 may include, for example, at least one of a display, a speaker, a printer, and a vibration motor. The output device 35 may be capable of outputting information to a recording medium that is detachable from the display control device 3, such as a USB memory. Note that when the display control device 3 outputs information via the communication device 33, the communication device 33 may function as the output device.

[0052] The arithmetic device 31 may have an image acquisition unit 311, a tracking unit 312, a face authentication unit 313, a determination unit 314, and a display control unit 315 as logically realized functional blocks or as physically realized processing circuits. At least one of the image acquisition unit 311, the tracking unit 312, the face authentication unit 313, the determination unit 314, and the display control unit 315 may be realized in a form in which a logical functional block and a physical processing circuit (i.e., hardware) are mixed. When at least some of the image acquisition unit 311, the tracking unit 312, the face authentication unit 313, the determination unit 314, and the display control unit 315 are functional blocks, at least some of the image acquisition unit 311, the tracking unit 312, the face authentication unit 313, the determination unit 314, and the display control unit 315 may be realized by the arithmetic device 31 executing a predetermined computer program.

[0053] The arithmetic device 31 may acquire (in other words, read) the predetermined computer program from the storage device 32. The arithmetic device 31 may read the predetermined computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (not shown) included in the display control device 3. The arithmetic device 31 may acquire (in other words, download or read) the predetermined computer program from a device (not shown) external to the display control device 3 via the communication device 33. Note that the recording medium for recording the predetermined computer program executed by the arithmetic device 31 may be at least one of an optical disk, a magnetic medium, a magneto-optical disk, a semiconductor memory, and any other medium capable of storing a program.

[0054] The management server 21 transmits at least a portion of the facial images registered in the face DB 211 to the display control device 3 together with the authentication ID (i.e., passenger identification information) assigned to the facial images. In this case, the management server 21 may transmit gate identification information and boarding start time to the display control device 3 in addition to the facial images. The management server 21 may identify the gate identification information and boarding start time based on the flight name of the aircraft included in the passenger information associated with the facial image registered in the face DB 211 and the flight information registered in the flight DB 212. In other words, the management server 21 may identify the gate identification information and boarding start time related to the aircraft on which the passenger represented by the facial image will board. Note that if feature amounts related to the facial images are registered in the face DB 211 instead of or in addition to the facial images, the management server 21 may transmit the feature amounts related to the facial images to the display control device 3 instead of or in addition to the facial images.

[0055] The display control device 3 registers the facial image transmitted from the management server 21 in the face DB 36. The display control device 3 may register gate identification information and boarding start times in addition to facial images in the face DB 36. As shown in FIG. 6 , the face DB 36 may include (i) a plurality of facial images and (ii) a table that associates authentication IDs, gate identification information (e.g., boarding gate numbers), boarding start times, and facial images with each other. The management server 21 may transmit at least some of the facial images registered in the face DB 211 to the display control device 3 at predetermined intervals. The display control device 3 may update the facial images registered in the face DB 36 every time a facial image is transmitted from the management server 21. When the management server 21 transmits feature amounts related to the facial images to the display control device 3 instead of or in addition to the facial images, the feature amounts related to the facial images may be registered in the face DB 36 instead of or in addition to the facial images.

[0056] The image acquisition unit 311 of the calculation device 31 may acquire one or more first images generated by the camera CAM1 via the communication device 33. The image acquisition unit 311 may store the acquired one or more first images in the storage device 32.

[0057] The tracking unit 312 of the calculation device 31 tracks one or more passengers passing through the authentication area RA1 using the multiple first images acquired by the image acquisition unit 311 (i.e., the multiple first images generated by the camera CAM1). The face authentication unit 313 of the calculation device 31 performs face authentication processing on one or more passengers passing through the authentication area RA1 using the multiple first images acquired by the image acquisition unit 311.

[0058] A case where the tracking unit 312 performs tracking processing of a passenger P1 passing through the authentication area RA1 shown in Fig. 4 and the face authentication unit 313 performs face authentication processing of the passenger P1 will be described with reference to Fig. 7. Note that the passenger P1 is also a target of the face authentication processing, and therefore may also be referred to as a person to be authenticated.

[0059] Images IMG1, IMG2, and IMG3 shown in Fig. 7 are images including passenger P1 (in other words, images in which passenger P1 is captured). Image IMG1 is an image generated by camera CAM1 capturing an image of authentication area RA1 at time t1. Image IMG2 is an image generated by camera CAM1 capturing an image of authentication area RA1 at time t2, which is later than time t1. Image IMG3 is an image generated by camera CAM1 capturing an image of authentication area RA1 at time t3, which is later than time t2.

[0060] Each of images IMG1, IMG2, and IMG3 may be an image corresponding to one frame of a video. In this case, image IMG2 does not have to be an image corresponding to the frame immediately following the frame corresponding to image IMG1. In other words, image IMG2 may be an image corresponding to a frame that is two or more frames after the frame corresponding to image IMG1. Similarly, image IMG3 does not have to be an image corresponding to the frame immediately following the frame corresponding to image IMG2. In other words, image IMG3 may be an image corresponding to a frame that is two or more frames after the frame corresponding to image IMG2.

[0061] The tracking unit 312 detects the head of passenger P1 contained in image IMG1 from image IMG1. Note that the method for detecting a person's head from an image is an existing technique, so a detailed description thereof will be omitted. Based on the detected head of passenger P1, the tracking unit 312 sets an area including the head of passenger P1 as a tracking area TA1.

[0062] When the tracking unit 312 sets the tracking area TA1, it sets a tracking ID, which is identification information for identifying the passenger P1 related to the tracking area TA1, to the passenger P1. The tracking unit 312 calculates the position of the passenger P1 based on the tracking area TA1. Note that, since existing technology can be applied to a method for calculating the position of a subject included in an image from the image, a detailed description thereof will be omitted. The tracking unit 312 may associate the tracking ID with the position of the passenger P1 and register them in an ID correspondence table 321 stored in the storage device 32.

[0063] When the tracking area TA1 is set, the tracking unit 312 may determine whether or not the face of passenger P1 is captured in the tracking area TA1. In other words, the tracking unit 312 may perform face detection on the tracking area TA1. When it is determined that the face of passenger P1 is captured in the tracking area TA1, the tracking unit 312 generates a face image including the face area of ​​passenger P1. The tracking unit 312 associates the generated face image with the tracking ID related to passenger P1 and transmits it to the face authentication unit 313. Note that when it is determined that the face of passenger P1 is not captured in the tracking area TA1, the tracking unit 312 does not need to generate a face image.

[0064] The face authentication unit 313 performs face authentication processing using the face image transmitted from the tracking unit 312 and the face images registered in the face DB 36. The face authentication unit 313 may extract features of the face image transmitted from the tracking unit 312. The face authentication unit 313 may compare the features of the face image transmitted from the tracking unit 312 with features related to face images registered in the face DB 36. At this time, the face authentication unit 313 may calculate a matching score (or a similarity score) based on the extracted features and the features related to the face images registered in the face DB 36. The face authentication unit 313 may compare the matching score with a threshold. If the matching score is equal to or greater than the threshold, the face authentication unit 313 may determine that the face indicated by the face image transmitted from the tracking unit 312 corresponds to the face indicated by one of the face images registered in the face DB 36. If the matching score is less than the threshold value, it may be determined that a face image indicating a face corresponding to the face indicated by the face image transmitted from the tracking unit 312 is not registered in the face DB 36 .

[0065] When the face indicated by the facial image transmitted from the tracking unit 312 corresponds to the face indicated by one of the facial images registered in the face DB 36 (in other words, when facial authentication is successful), the facial authentication unit 313 registers the authentication ID associated with the one facial image in the ID correspondence table 321 in association with the tracking ID associated with the facial image transmitted from the tracking unit 312. The facial authentication unit 313 may register, in the ID correspondence table 321, the authentication time, which is the time when the facial authentication process was performed, in addition to the authentication ID.

[0066] If a facial image showing a face corresponding to the face indicated by the facial image transmitted from the tracking unit 312 is not registered in the face DB 36 (in other words, if facial authentication fails), the facial authentication unit 313 may register information indicating that there is no corresponding person (for example, “N / A (Not Applicable)”) in the ID correspondence table 321 in association with the tracking ID associated with the facial image transmitted from the tracking unit 312.

[0067] The tracking unit 312 may use the image IMG2 and the image IMG1 to identify the passenger P1 included in the image IMG2. Identifying the passenger P1 included in the image IMG2 is equivalent to associating the passenger P1 included in the image IMG1 with the passenger P1 included in the image IMG2. Therefore, at least one of a matching method and an optical flow method related to association between images can be applied to identifying the passenger P1 included in the image IMG2. Note that various existing methods can be applied to both the matching method and the optical flow method, and therefore detailed description thereof will be omitted.

[0068] When passenger P1 included in image IMG2 is identified, the tracking unit 312 detects the head of passenger P1. Based on the detected head of passenger P1, the tracking unit 312 sets an area including the head of passenger P1 as tracking area TA2. Because passenger P1 included in image IMG1 and passenger P1 included in image IMG2 are the same passenger, the tracking ID of passenger P1 related to tracking area TA2 is the same as the tracking ID of passenger P1 related to tracking area TA1. The tracking unit 312 calculates the position of passenger P1 based on tracking area TA2.

[0069] The tracking unit 312 associates the position of passenger P1 with the tracking ID related to passenger P1 and registers it in the ID correspondence table 321. In this case, since the position of passenger P1 calculated based on tracking area TA1 is registered in the ID correspondence table 321, the tracking unit 312 registers the position of passenger P1 calculated based on tracking area TA2 in the ID correspondence table 321, thereby updating the position of passenger P1.

[0070] When the tracking area TA2 is set, the tracking unit 312 may determine whether the face of passenger P1 is captured in the tracking area TA2. In other words, the tracking unit 312 may perform face detection on the tracking area TA2. When it is determined that the face of passenger P1 is captured in the tracking area TA2, the tracking unit 312 generates a face image including the face area of ​​passenger P1. The tracking unit 312 associates the generated face image with the tracking ID related to passenger P1 and transmits the image to the face authentication unit 313. Note that, when it is determined that the face of passenger P1 is not captured in the tracking area TA2, the tracking unit 312 does not need to generate a face image. Note that, even when it is determined that the face of passenger P1 is captured in the tracking area TA2, when the tracking ID related to passenger P1 is associated with the authentication ID in the ID correspondence table 321 (i.e., when face authentication has already been successful), the tracking unit 312 does not need to generate a face image.

[0071] The face authentication unit 313 performs face authentication processing using the face image transmitted from the tracking unit 312 and the face images registered in the face DB 36. If the face indicated by the face image transmitted from the tracking unit 312 corresponds to the face indicated by one of the face images registered in the face DB 36 (in other words, if face authentication is successful), the face authentication unit 313 registers the authentication ID associated with the one face image in the ID correspondence table 321 in association with the tracking ID associated with the face image transmitted from the tracking unit 312. The face authentication unit 313 may register, in the ID correspondence table 321, the authentication time, which is the time when the face authentication processing was performed, in addition to the authentication ID. Note that if the authentication time has already been registered in the ID correspondence table 321 (i.e., if face authentication was successful in the past), the face authentication unit 313 may update the authentication time registered in the ID correspondence table 321.

[0072] If a facial image showing a face corresponding to the face indicated by the facial image sent from the tracking unit 312 is not registered in the face DB 36 (in other words, if facial authentication fails), the facial authentication unit 313 may register information indicating that there is no corresponding person in the ID correspondence table 321, by associating it with the tracking ID associated with the facial image sent from the tracking unit 312.

[0073] The tracking unit 312 may use the image IMG3 and the image IMG2 to identify the passenger P1 included in the image IMG3. When the passenger P1 included in the image IMG3 is identified, the tracking unit 312 detects the head of the passenger P1. Based on the detected head of the passenger P1, the tracking unit 312 sets an area including the head of the passenger P1 as a tracking area TA3. Because the passenger P1 included in the image IMG2 and the passenger P1 included in the image IMG3 are the same passenger, the tracking ID of the passenger P1 related to the tracking area TA3 is the same as the tracking ID of the passenger P1 related to the tracking area TA2. The tracking unit 312 calculates the position of the passenger P1 based on the tracking area TA3.

[0074] The tracking unit 312 associates the position of passenger P1 with the tracking ID related to passenger P1 and registers it in the ID correspondence table 321. In this case, since the position of passenger P1 calculated based on tracking area TA2 is registered in the ID correspondence table 321, the tracking unit 312 registers the position of passenger P1 calculated based on tracking area TA3 in the ID correspondence table 321, thereby updating the position of passenger P1.

[0075] When the tracking area TA3 is set, the tracking unit 312 may determine whether the face of passenger P1 is captured in the tracking area TA3. In other words, the tracking unit 312 may perform face detection on the tracking area TA3. When it is determined that the face of passenger P1 is captured in the tracking area TA3, the tracking unit 312 generates a face image including the face area of ​​passenger P1. The tracking unit 312 associates the generated face image with the tracking ID related to passenger P1 and transmits the image to the face authentication unit 313. Note that, when it is determined that the face of passenger P1 is not captured in the tracking area TA3, the tracking unit 312 does not need to generate a face image. Note that, even when it is determined that the face of passenger P1 is captured in the tracking area TA3, if the tracking ID related to passenger P1 is associated with the authentication ID in the ID correspondence table 321 (i.e., if face authentication has already been successful), the tracking unit 312 does not need to generate a face image.

[0076] The face authentication unit 313 performs face authentication processing using the face image transmitted from the tracking unit 312 and the face images registered in the face DB 36. If the face indicated by the face image transmitted from the tracking unit 312 corresponds to the face indicated by one of the face images registered in the face DB 36 (in other words, if face authentication is successful), the face authentication unit 313 registers the authentication ID associated with the one face image in the ID correspondence table 321 in association with the tracking ID associated with the face image transmitted from the tracking unit 312. The face authentication unit 313 may register, in the ID correspondence table 321, the authentication time, which is the time when the face authentication processing was performed, in addition to the authentication ID. Note that if the authentication time has already been registered in the ID correspondence table 321 (i.e., if face authentication was successful in the past), the face authentication unit 313 may update the authentication time registered in the ID correspondence table 321.

[0077] If a facial image showing a face corresponding to the face indicated by the facial image sent from the tracking unit 312 is not registered in the face DB 36 (in other words, if facial authentication fails), the facial authentication unit 313 may register information indicating that there is no corresponding person in the ID correspondence table 321, by associating it with the tracking ID associated with the facial image sent from the tracking unit 312.

[0078] The tracking unit 312 and the face authentication unit 313 may repeat the above-described process until passenger P1 passes through the authentication area RA1. As described above, the tracking unit 312 calculates the position of passenger P1. In other words, it can be said that the tracking unit 312 detects the position of passenger P1. For this reason, the tracking unit 312 may be referred to as a position detection means.

[0079] An example of the ID correspondence table 321 will now be described with reference to FIG. 8 . As shown in FIG. 8 , in the ID correspondence table 321, a tracking ID, a tracking location (e.g., the location of passenger P1), an authentication ID, and an authentication time may be associated with one another. When a tracking ID is associated with a specific authentication ID in the ID correspondence table 321, this indicates that a facial image of a person corresponding to the passenger to whom the tracking ID is set is registered in the face DB 36. When a tracking ID is associated with the character string "N / A", this indicates that a facial image of a person corresponding to the passenger to whom the tracking ID is set is not registered in the face DB 36. When a tracking ID is not associated with an authentication ID (i.e., when the authentication ID field is blank), this indicates that a facial authentication process has never been performed.

[0080] However, it may be difficult for the tracking unit 312 to continue tracking passenger P1, for example, because the head of passenger P1 is hidden behind another person. When tracking of passenger P1 is interrupted, the tracking unit 312 may perform the following process. After tracking of passenger P1 is interrupted, the tracking unit 312 may determine whether a new passenger has been detected from the first image acquired by the image acquisition unit 311 (i.e., the first image generated by camera CAM1). A "new passenger" refers to a passenger for whom a tracking ID has not been set.

[0081] When a new passenger is detected, the tracking unit 312 may determine whether passenger P1 and the new passenger are the same person by comparing the feature amount of the tracking area (e.g., at least one of tracking areas TA1, TA2, and TA3) related to passenger P1 with the feature amount of the tracking area related to the new passenger. If it is determined that passenger P1 and the new passenger are the same person, the tracking unit 312 may assign the tracking ID related to passenger P1 to the new passenger. As a result, the tracking unit 312 can track passenger P1 again. Note that the feature amount may be a feature amount related to the passenger's head, a feature amount related to the passenger's upper body, or a feature amount related to the passenger's entire body. Therefore, the tracking area may include the passenger's head, the passenger's upper body, or the passenger's entire body. Note that the feature amount may be obtained, for example, by person re-identification technology.

[0082] The tracking unit 312 and the face authentication unit 313 may perform the above-described tracking process and face authentication process for each of the multiple passengers included in one first image generated by the camera CAM1. Here, the authentication area RA1 may be, for example, an area of ​​5 meters by 5 meters. A distance of 5 meters is a distance that allows 5 to 6 passengers to pass through in a line. If the authentication area RA1 is an area of ​​5 meters by 5 meters, 20 to 30 passengers may exist within the authentication area RA1. Therefore, as shown in FIG. 9, the display control device 3 can perform tracking process and face authentication process for multiple passengers based on one first image generated by the camera CAM1. Note that the multiple dotted rectangles in FIG. 9 represent the tracking area.

[0083] The tracking area (e.g., at least one of the tracking areas TA1, TA2, and TA3) may include parts of the passenger other than the head (e.g., shoulders) in addition to the head of the passenger (e.g., passenger P1) that is tracked by the tracking unit 312. The tracking area may be set to include the upper body of the passenger, or may be set to include the entire body of the passenger.

[0084] The operations of the tracking unit 312 and the face authentication unit 313 will be described with reference to the flowchart of Fig. 10 . In Fig. 10 , the tracking unit 312 detects one or more passengers (i.e., people) from the first image acquired by the image acquisition unit 311 (i.e., the first image generated by the camera CAM1) (step S101). In the processing of step S101, the tracking unit 312 may detect the head of the passenger. In the processing of step S101, the tracking unit 312 may set a tracking region including the detected head of the passenger based on the detected head of the passenger. In the processing of step S101, the tracking unit 312 may perform face detection of the detected one or more passengers.

[0085] The tracking unit 312 performs a tracking process of at least one of the one or more passengers detected in the process of step S101 using another first image acquired by the image acquisition unit 311 (step S102). In the process of step S102, the tracking unit 312 may perform face detection of the detected one or more passengers. When the face of at least one of the one or more passengers is detected in the process of at least one of steps S101 and S102, the tracking unit 312 transmits a face image including a facial region of at least one passenger to the face authentication unit 313.

[0086] The face authentication unit 313 performs face authentication processing using the face image transmitted from the tracking unit 312 (step S103). The face authentication unit 313 registers the result of the face authentication processing in the ID correspondence table 321, or updates the ID correspondence table 321 based on the result of the face authentication processing (step S104).

[0087] Next, the operation of the determination unit 314 and the display control unit 315 of the calculation device 31 will be described. Here, it is assumed that lanes L1 and L2 shown in FIG. 11 are provided near the facial recognition gate device 23 and the normal gate device 24 of the boarding gate G1 shown in FIG. 4. Lane L1 is a lane through which passengers who wish to pass through the facial recognition gate device 23 pass. Therefore, lane L1 can be said to be a route along which passengers flow toward the facial recognition gate device 23. Lane L2 is a lane through which passengers who wish to pass through the normal gate device 24 pass. Therefore, lane L2 can be said to be a route along which passengers flow toward the normal gate device 24. Note that lanes L1 and L2 may be configured to be distinguishable from each other by, for example, at least one of a floor mat and a partition pole.

[0088] As shown in FIG. 11 , a display device 41 is disposed in lane L1 along the direction in which lane L1 extends. The display device 41 may be disposed along at least a portion of lane L1. As shown in FIG. 4 , the display device 41 may be disposed between the facial recognition gate device 23 and the authentication area RA1. That is, the display device 41 may be disposed along a portion of the route along which passengers flow toward the facial recognition gate device 23, between the facial recognition gate device 23 and the authentication area RA1. The display device 41 may be configured, for example, with at least one of an LED (Light Emitting Diode) light guide plate, a liquid crystal panel, an organic EL (Electro Luminescence) panel, and a surface-emitting LED. The display device 41 may be a single display device or may be configured with multiple display devices.

[0089] The determination unit 314 acquires the position of each passenger from the ID correspondence table 321. For example, the determination unit 314 may acquire the value of the tracking position in the ID correspondence table 321 shown in Fig. 8 as the position of each passenger. The determination unit 314 may identify passengers who are close to the display device 41 based on the position of each passenger.

[0090] The determination unit 314 determines whether one or more passengers approaching the display device 41 can pass through the face recognition gate device 23 based on the tracking IDs of one or more passengers approaching the display device 41 and the ID correspondence table 321. When the tracking ID of a passenger is associated with a specific authentication ID in the ID correspondence table 321, the determination unit 314 may determine that the passenger is a "matched person." When the tracking ID of a passenger is associated with the character string "N / A" in the ID correspondence table 321, the determination unit 314 may determine that the passenger is a "not matched person." When the tracking ID of a passenger is not associated with an authentication ID in the ID correspondence table 321 (i.e., when the authentication ID field is blank), the determination unit 314 may determine that the passenger is "unauthenticated." The determination unit 314 may perform a determination based on the ID correspondence table 321 multiple times for a passenger determined to be unauthenticated. The determination unit 314 may determine that a passenger who has been determined to be unauthenticated a predetermined number of times is "not a match."

[0091] If the passenger is determined to be a "matching person," the determination unit 314 may determine whether or not the passenger is allowed to pass through the facial recognition gate device 23, based on the passenger's authentication ID and the face DB 36. For example, the determination unit 314 may determine whether or not the boarding gate number (see FIG. 6 ) associated with the authentication ID in the face DB 36 matches the boarding gate number at which the facial recognition gate device 23 is installed. If the boarding gate number associated with the authentication ID does not match the boarding gate number at which the facial recognition gate device 23 is installed, the determination unit 314 may determine that the passenger is not allowed to pass through the facial recognition gate device 23. If the boarding gate number associated with the authentication ID is determined to match the boarding gate number at which the facial recognition gate device 23 is installed, the determination unit 314 may determine that the passenger is allowed to pass through the facial recognition gate device 23. Alternatively, if it is determined that the boarding gate number associated with the authentication ID matches the boarding gate number where the facial recognition gate device 23 is installed, the determination unit 314 may compare the boarding start time associated with the authentication ID in the face DB 36 with the current time to determine whether the passenger is allowed to pass through the facial recognition gate device 23.

[0092] As shown in Figure 3, the facial recognition gate device 23 is installed at boarding gate G1. If the boarding gate of the aircraft on which the passenger is boarding is not boarding gate G1, the determination unit 314 may determine that the passenger cannot pass through the facial recognition gate device 23. If the boarding gate of the aircraft on which the passenger is boarding is boarding gate G1 but the current time is more than a second predetermined time (e.g., from several tens of minutes to several tens of minutes) before the boarding start time, the determination unit 314 may determine that the passenger cannot pass through the facial recognition gate device 23. If the boarding gate of the aircraft on which the passenger is boarding is boarding gate G1 and the current time is not more than the second predetermined time before the boarding start time, the determination unit 314 may determine that the passenger can pass through the facial recognition gate device 23.

[0093] The judgment unit 314 may output, as the judgment result for a passenger, any of the following: (i) a first judgment result indicating that a matching person is found and that the person can pass through the facial recognition gate device 23; (ii) a second judgment result indicating that a matching person is found but that the person cannot pass through the facial recognition gate device 23; (iii) a third judgment result indicating that a matching person is not found; and (iv) a fourth judgment result indicating that the person has not been authenticated.

[0094] Here, the first determination result is a determination result indicating that the passenger can pass through the facial recognition gate device 23. The second determination result, the third determination result, and the fourth determination result are determination results indicating that the passenger cannot pass through the facial recognition gate device 23. The determination unit 314 transmits the determination result for the passenger and the position of the passenger to the display control unit 315.

[0095] The display control unit 315 displays the determination result for the passenger on the display device 41. At this time, the display control unit 315 determines the display mode of the determination result based on the determination result for the passenger.

[0096] If the judgment result for the passenger is the first judgment result (i.e., a match is found and the passenger can pass through the facial recognition gate device 23), the display control unit 315 may determine to display a green graphic (e.g., a rectangle) as an indication that the passenger can pass through the facial recognition gate device 23. If the judgment result for the passenger is the second judgment result (i.e., a match is found but the passenger cannot pass through the facial recognition gate device 23), the display control unit 315 may determine to display a yellow graphic (e.g., a rectangle) as an indication that the passenger cannot currently pass through the facial recognition gate device 23 or that the passenger has entered the wrong gate. If the judgment result for the passenger is the third judgment result (i.e., no match is found), the display control unit 315 may determine to display a red graphic (e.g., a rectangle) as an indication that the passenger cannot pass through the facial recognition gate device 23. If the determination result for the passenger is the fourth determination result (i.e., unauthenticated), the display control unit 315 may determine to display a gray shape (e.g., a rectangle) as a sign indicating unauthenticated. In this way, the display mode related to the first determination result, the display mode related to the second determination result, the display mode related to the third determination result, and the display mode related to the fourth determination result are different.

[0097] The determination result by the determination unit 314 is based on the result of the face authentication process performed by the face authentication unit 313. Therefore, the determination result by the determination unit 314 may be rephrased as the result of face authentication. Therefore, when the display control unit 315 determines the display mode based on the determination result by the determination unit 314, it can be said that the display mode is determined based on the result of face authentication.

[0098] The display control unit 315 determines the display position of the determination result for the passenger on the display device 41 based on the position of the passenger. The display control unit 315 may determine the display position of the determination result for the passenger to be a position in front of and near one passenger (for example, a position one step ahead of the passenger). The display control unit 315 controls the display device 41 so that the determination result for the passenger is displayed at the determined display position in the determined display mode. The display control unit 315 changes the display position of the determination result for the passenger as the position of the passenger changes.

[0099] 11 , the display control unit 315 may control the display device 41 so that a sign 411 as a display mode determined based on the determination result for passenger P11 is displayed at a position on the display device 41 determined based on the position of passenger P11. The display control unit 315 may control the display device 41 so that a sign 412 as a display mode determined based on the determination result for passenger P12 is displayed at a position on the display device 41 determined based on the position of passenger P12. The display control unit 315 may control the display device 41 so that a sign 413 as a display mode determined based on the determination result for passenger P13 is displayed at a position on the display device 41 determined based on the position of passenger P13. The display control unit 315 may control the display device 41 so that the display positions of the signs 411, 412, and 413 change in accordance with changes in the positions of passengers P11, P12, and P13.

[0100] The operations of the determination unit 314 and the display control unit 315 will be described with reference to the flowchart of Fig. 12. In Fig. 12, the determination unit 314 acquires the position of a passenger as a target person from the ID correspondence table 321 (step S201). Based on the position of a passenger acquired in the processing of step S201, the determination unit 314 determines whether the passenger is close to the display device 41 (step S202).

[0101] If it is determined in the processing of step S202 that the passenger is not close to the display device 41 (step S202: No), the determination unit 314 changes the target person (e.g., changes from one passenger to another passenger) and performs the processing of step S201. If it is determined in the processing of step S202 that the passenger is close to the display device 41 (step S202: Yes), the determination unit 314 determines the display mode and display position of the determination result for the passenger. The determination unit 314 may generate display information indicating the display mode and display position of the determination result for the passenger (step S203). The determination unit 314 transmits the display information to the display control unit 315. The display control unit 315 controls the display device 41 based on the display information transmitted from the determination unit 314 (step S204).

[0102] The display control device 3 may acquire at least a portion of the multiple gate information registered in the gate DB 213 of the management server 21. The display control device 3 may determine whether or not a facial recognition gate device (e.g., facial recognition gate device 23) is installed at a boarding gate (e.g., boarding gate G1) based on the acquired gate information. If it is determined that a facial recognition gate device is not installed, the display control device 3 may determine not to perform the above-described operation. If it is determined that a facial recognition gate device is installed, the display control device 3 may determine to perform the above-described operation. Alternatively, if it is determined that a facial recognition gate device is installed, the display control device 3 may determine whether or not the installed facial recognition gate device is operating based on the acquired gate information. If it is determined that the installed facial recognition gate device is operating, the display control device 3 may determine to perform the above-described operation. If it is determined that the installed facial recognition gate device is not operating, the display control device 3 may determine not to perform the above-described operation. In other words, the display control device 3 may determine (or decide) whether to perform the above-mentioned operation based on whether a facial recognition gate device is installed and / or whether the facial recognition gate device is in operation.

[0103] The above-described operations of the display control device 3 may be realized by the display control device 3 reading a computer program recorded on a recording medium. In this case, it can be said that the recording medium records a computer program for causing the display control device 3 to execute the above-described operations.

[0104] 11 , the display control device 3 displays a plurality of determination results corresponding to each of a plurality of passengers on the display device 41 in a display format based on the determination result. Therefore, passengers attempting to pass through the facial recognition gate device 23 can determine whether or not they can pass through the facial recognition gate device 23 based on the display on the display device 41. In addition, airport staff (for example, at least one of a guide and a security guard) near the facial recognition gate device 23 can determine who cannot pass through the facial recognition gate device 23.

[0105] If a passenger attempting to pass through the facial recognition gate device 23 determines that he or she is unable to pass through the facial recognition gate device 23, the passenger can be prevented from entering the facial recognition gate device 23. Furthermore, if airport staff (for example, at least one of a guide and a security guard) can identify people who are unable to pass through the facial recognition gate device 23, the staff can guide the people who are unable to pass through the facial recognition gate device 23 not to enter the facial recognition gate device 23. Therefore, the display control device 3 can prevent people who are unable to pass through the facial recognition gate device 23 from entering the facial recognition gate device 23. Therefore, the display control device 3 can prevent a decrease in the throughput of the facial recognition gate device 23.

[0106] In addition to airport boarding gates, the display control device 3 may be applied to, for example, at least one of airport security gates (i.e., gates installed at airport security checkpoints) and immigration gates. In addition to airports, the display control device 3 may be applied to, for example, at least one of offices, train stations, theme parks, and event venues that use facial recognition gate devices for at least one of entry and exit.

[0107] (First Modification) In the second embodiment described above, the display control device 3 (specifically, the face authentication unit 313) performs face authentication processing. However, the display control device 3 does not have to perform face authentication processing. In this case, the display control device 3 does not have to include the face authentication unit 313 and the face DB 36. In this case, the face authentication processing may be performed by an authentication device 5 different from the display control device 3. In this case, as shown in FIG. 13 , the display control device 3 and the authentication device 5 may be connected via a network NW. Note that the display control device 3 and the authentication device 5 may form one system. The one system may be referred to as a display control system or an authentication system.

[0108] The authentication device 5 includes a face authentication unit 51 and a face database 52 (hereinafter referred to as "face DB 52"). The face authentication unit 51 is configured to be able to execute face authentication processing. In other words, the authentication device 5 is an authentication device having a face authentication function. The management server 21 of the airport system 2 may transmit at least a portion of the face images registered in the face DB 211 to the authentication device 5 together with an authentication ID (i.e., passenger identification information) assigned to the face image. In this case, the management server 21 may transmit gate identification information and boarding start time in addition to the face image to the authentication device 5. The authentication device 5 may register the face image transmitted from the management server 21 in the face DB 52. The authentication device 5 may register gate identification information and boarding start time in addition to the face image in the face DB 52.

[0109] The tracking unit 312 of the display control device 3 may detect a passenger (e.g., passenger P1) included in the image acquired by the image acquisition unit 311. The tracking unit 312 may set a tracking area including the head of the detected passenger. The tracking unit 312 may perform face detection of the detected passenger. When the passenger's face is detected, the tracking unit 312 may transmit a face image including the passenger's facial area to the authentication device 5 via the communication device 33.

[0110] The face authentication unit 51 of the authentication device 5 may perform face authentication processing using the face image transmitted from the display control device 3 (specifically, the tracking unit 312) and the face DB 52. The face authentication unit 51 may transmit information indicating the result of the face authentication processing to the display control device 3. The display control device 3 may register the result of the face authentication processing indicated by the information transmitted from the authentication device 5 (specifically, the face authentication unit 51) in the ID correspondence table 321.

[0111] (Second Modification) In the second embodiment described above, the display control unit 315 of the display control device 3 controls the display device 41 arranged along the direction in which the lane L1 extends. In place of the display device 41, a columnar member 42 having a display device may be installed in the lane L1. As shown in Fig. 14, the columnar member 42 may include columnar members 42a, 42b, and 42c. Note that the display device of the columnar member 42 emits light, and therefore may be referred to as a light-emitting unit.

[0112] The display control unit 315 of the display control device 3 may determine a pillar-shaped member (e.g., any one of the pillar-shaped members 42 a, 42 b, and 42 c) that will display the determination result for the passenger based on the passenger's position. The display control unit 315 may control the display device of the determined pillar-shaped member (e.g., any one of the pillar-shaped members 42 a, 42 b, and 42 c) so that the display device of the determined pillar-shaped member displays the determination result in a display mode determined based on the determination result by the determination unit 314.

[0113] As a result, as shown in Fig. 14, the determination result for passenger P11 may be displayed on the display device of pillar member 42a, the determination result for passenger P12 may be displayed on the display device of pillar member 42b, and the determination result for passenger P13 may be displayed on the display device of pillar member 42c.

[0114] (Third Modification) Instead of the display device 41, a projection device capable of projecting an image may be installed in at least one of the lane L1 and its surroundings. The display control unit 315 of the display control device 3 may determine a display position for displaying the determination result for the passenger based on the position of the passenger. The display control unit 315 may control the projection device so that the determination result for the passenger is projected at the determined display position in a display mode determined based on the determination result by the determination unit 314.

[0115] (Fourth Modification) The display control device 3 may be connected to a terminal device 6 shown in Fig. 15 via a network NW. The terminal device 6 may be configured as any one of a tablet terminal, a smartphone, and a notebook PC (Personal Computer). The terminal device 6 may be a terminal device that can be used by airport staff (for example, at least one of a guidance staff member and a security guard).

[0116] The display control unit 315 of the display control device 3 may control the terminal device 6 to display information related to passengers for whom the determination result by the determination unit 314 is the second determination result, the third determination result, or the fourth determination result. In other words, the display control unit 315 may control the terminal device 6 to display information related to passengers who are not permitted to pass through the face authentication gate device 23 installed at the boarding gate G1. As a result, an image such as that shown in FIG. 15 may be displayed on the terminal device 6.

[0117] The display control unit 315 may control the terminal device 6 to display, for example, a message indicating that the passenger will be guided to the correct boarding gate and a facial image for the passenger whose determination result by the determination unit 314 is the second determination result (see display 61 in FIG. 15 ). Note that the facial image may be a facial image registered in the face DB 36, or may be a facial image extracted from the first image acquired by the image acquisition unit 311 (i.e., the first image generated by the camera CAM1).

[0118] The display control unit 315 may control the terminal device 6 to display, for example, a message indicating that a passenger whose determination result by the determination unit 314 is the third determination result will be guided to the normal gate device 24 (see display 62 in FIG. 15 ). In this case, the display control unit 315 may control the terminal device 6 to display a message indicating that there is no matching person ("Unregistered Person" in FIG. 15 ).

[0119] The display control unit 315 may control the terminal device 6 to display, for example, a message indicating that a passenger for whom the determination result by the determination unit 314 is the fourth determination result will be guided to the normal gate device 24 (see display 63 in FIG. 15 ). In this case, the display control unit 315 may control the terminal device 6 to display a message indicating that the passenger has not been authenticated (i.e., face authentication processing has not been performed) ("Unauthenticated Person" in FIG. 15 ).

[0120] The display control unit 315 may determine the display mode of the determination result to be displayed on the display device 41 in parallel with the control of the terminal device 6. In this case, the display control device 315 may blink the determination result displayed on the display device 41 to indicate that guidance is required for the passenger displayed on the terminal device 6.

[0121] If the judgment result for the passenger is the second judgment result, the display control device 315 may decide to flash a yellow rectangle indicating that the passenger cannot currently pass through the facial recognition gate device 23 or that the passenger has entered the wrong gate. If the judgment result for the passenger is the third judgment result, the display control device 315 may decide to flash a red rectangle indicating that the passenger cannot currently pass through the facial recognition gate device 23. If the judgment result for the passenger is the fourth judgment result, the display control device 315 may decide to flash a gray rectangle indicating that the passenger has not been authenticated.

[0122] Based on the position of the passenger, the display control unit 315 may determine a display position of the determination result for that passenger on the display device 41. The display control unit 315 may control the display device 41 so that the determination result for that passenger is displayed at the determined display position in the determined display mode.

[0123] With this configuration, it is possible to highlight passengers who are unable to pass through the facial recognition gate device 23 installed at boarding gate G1. In addition, information according to the determination result regarding passengers who are unable to pass through the facial recognition gate device 23 is displayed on the terminal device 6, allowing airport staff (for example, at least one of a guide and a security guard) to take appropriate action against passengers who are unable to pass through the facial recognition gate device 23.

[0124] <Third Embodiment> A third embodiment of a display control device will be described with reference to Fig. 16 . Below, the third embodiment of a display control device, a display control method, and a recording medium will be described using a display control device 3. In the third embodiment, some of the operations of the image acquisition unit 311 and the tracking unit 312 differ from those of the second embodiment described above. Other aspects of the third embodiment may be the same as those of the second embodiment.

[0125] 16, in addition to a virtual authentication area RA1, a virtual authentication area RA2 is provided around a boarding gate G1 where the facial recognition gate device 23 and the normal gate device 24 are installed. The authentication area RA2 is an area that includes at least a part of the queue (for example, lane L1 shown in FIG. 11) formed by passengers waiting to pass through the facial recognition gate device 23.

[0126] Camera CAM2 is configured to be able to capture an image of the authentication area RA2. Camera CAM2 may be connected to the display control device 3 via the network NW. Camera CAM2 does not have to be installed so that the entire authentication area RA2 is included in its angle of view. In other words, camera CAM2 may be installed so that it can capture an image of at least a portion of the authentication area RA2. Camera CAM2 may be installed in a position that does not interfere with passenger passage (e.g., higher than the passenger's head). As will be described later, images captured by camera CAM2 may be used for facial recognition processing. For this reason, camera CAM2 may be a high-resolution camera, such as a 4K resolution camera. Camera CAM2 does not have to be connected to the display control device 3 via the network NW. In this case, camera CAM2 may be connected to the display control device 3 via a cable (e.g., a USB cable).

[0127] The camera CAM2 may capture an image of the authentication area RA2 to generate a second image that includes the authentication area RA2. The second image may be an image corresponding to one frame of a video. The image acquisition unit 311 may acquire one or more second images generated by the camera CAM2 via the communication device 33. The image acquisition unit 311 may store the acquired one or more second images in the storage device 32.

[0128] The tracking unit 312 performs tracking processing of one or more passengers passing through the authentication area RA2 using one or more second images acquired by the image acquisition unit 311 (i.e., one or more second images generated by the camera CAM2).

[0129] 11 , the lane L1 through which passengers attempting to pass through the facial recognition gate device 23 pass and the lane L2 through which passengers attempting to pass through the normal gate device 24 pass are close to each other. Therefore, the second image may include passengers attempting to pass through the normal gate device 24 in addition to passengers attempting to pass through the facial recognition gate device 23.

[0130] The tracking unit 312 may calculate the positions (e.g., the positions of the feet) of one or more passengers included in the second image. Based on the calculated positions, the tracking unit 312 may identify passengers who are about to pass through the facial recognition gate device 23 (i.e., passengers passing through lane L1) included in the second image.

[0131] When lane L1 is closer to lane L2 as viewed from camera CAM2, the tracking unit 312 may identify passengers attempting to pass through the facial recognition gate device 23 (i.e., passengers passing through lane L1) based on the size of the heads of one or more passengers included in the second image.

[0132] The tracking unit 312 detects the head of a passenger attempting to pass through the facial recognition gate device 23, which is included in the second image. The tracking unit 312 sets a tracking area including the head of the passenger based on the detected head of the passenger. The tracking unit 312 may perform face detection in the set tracking area. When the face of the passenger is detected, the tracking unit 312 may transmit a face image including the facial area of ​​the passenger to the face authentication unit 313. The face authentication unit 313 may perform face authentication processing using the face image transmitted from the tracking unit 312 and face images registered in the face DB 36.

[0133] (Technical Effect) In the third embodiment, in addition to the authentication area RA1, facial recognition processing is also performed on passengers passing through the authentication area RA2 (i.e., passengers waiting in line to pass through the facial recognition gate device 23). This increases the opportunities for facial recognition processing to be performed on passengers attempting to pass through the facial recognition gate device 23. As a result, it is possible to reduce the number of passengers for whom facial recognition processing is never performed (for example, passengers who are determined to be unauthenticated by the determination unit 314).

[0134] <Supplementary Notes> The following supplementary notes are further disclosed regarding the above-described embodiment.

[0135] (Supplementary Note 1) A display control device comprising: an image acquisition means for acquiring an image including a predetermined area through which a person heading towards a facial recognition gate device will pass; and a display control means for displaying the result of facial recognition performed using the acquired image before the person reaches the facial recognition gate device, wherein the display control means changes the display mode based on the result of the facial recognition.

[0136] (Supplementary Note 2) The display control device according to Supplementary Note 1, wherein the result of the facial authentication includes at least one of a first result indicating that a matched person is found and that the person can pass through the facial authentication gate device, a second result indicating that a matched person is found but that the person cannot pass through the facial authentication gate device, and a third result indicating that no matched person is found, and wherein the display mode relating to the first result, the display mode relating to the second result, and the display mode relating to the third result are different.

[0137] (Supplementary Note 3) The display control device according to Supplementary Note 1 or 2, further comprising: a position detection means for detecting a position of the person based on the acquired image; and the display control means for changing a display position based on the position of the person.

[0138] (Supplementary Note 4) The display control device described in Supplementary Note 3, wherein the display control means displays the results of the facial recognition on a display device provided along at least a part of a route along which a flow of people heads toward the facial recognition gate device, the display control means detects a change in the position of the person based on the position of the person, and the display control means changes the display position of the results of the facial recognition on the display device based on the change in the position of the person.

[0139] (Supplementary Note 5) The display control device according to Supplementary Note 4, wherein the display control means causes the display device to display the result of the face authentication in front of and near the person based on the position of the person.

[0140] (Supplementary Note 6) The display control device according to Supplementary Note 4 or 5, wherein the display device is provided along a portion of the path between the face recognition gate device and the predetermined area.

[0141] (Supplementary Note 7) The display control device according to any one of Supplementary Notes 4 to 6, wherein the display device has one or more light-emitting panels.

[0142] (Supplementary Note 8) The display control device according to any one of Supplementary Notes 4 to 6, wherein the display device includes one or more columnar members each having a light-emitting portion.

[0143] (Supplementary Note 9) The display control device according to any one of Supplementary Notes 1 to 3, wherein the display control means displays the result of the face authentication by controlling a projection device to project an image showing the result of the face authentication.

[0144] (Supplementary Note 10) The display control device according to any one of Supplementary Notes 1 to 9, wherein the display mode includes an appearance and a state of the display, and the change in the display mode includes a change in at least one of the appearance and the state of the display.

[0145] (Supplementary Note 11) The display control device according to any one of Supplementary Notes 1 to 10, further comprising a first determination means for determining whether or not to guide a user to another gate device different from the facial recognition gate device based on the result of the facial recognition, and the display control means for changing the display mode based on the determination result by the first determination means and the result of the facial recognition.

[0146] (Supplementary Note 12) The display control device according to any one of Supplementary Notes 1 to 11, further comprising: a face authentication unit configured to perform face authentication using the acquired image.

[0147] (Supplementary Note 13) The display control device described in any one of Supplementary Notes 1 to 12, wherein the acquired image includes a plurality of people, and the display control means displays at least a portion of a plurality of results corresponding to each of the plurality of people of the facial recognition performed using the acquired image before the plurality of people reach the facial recognition gate device.

[0148] (Supplementary Note 14) The display control device according to any one of Supplementary Notes 1 to 13, wherein the predetermined area includes a line formed by people waiting to pass through the face recognition gate device.

[0149] (Supplementary Note 15) The display control device according to Supplementary Note 14, further comprising a second determination means for determining whether or not a person included in the acquired image is included in the column based on a size of the person's head.

[0150] (Supplementary Note 16) The display control device according to Supplementary Note 14, further comprising a second determination means configured to determine whether or not a person included in the acquired image is included in the column, based on a position of the person included in the acquired image.

[0151] (Supplementary Note 17) The display control device according to any one of Supplementary Notes 14 to 16, wherein the display control means displays, from the results of the face authentication, a face authentication result corresponding to a person included in the column.

[0152] (Supplementary Note 18) The display control device according to any one of Supplementary Notes 14 to 17, wherein the display control means displays, for a person included in the column for whom the result of the face authentication is pending, that the result of the face authentication is pending.

[0153] (Supplementary Note 19) A display control method comprising: acquiring an image including a predetermined area through which a person heading towards a facial recognition gate device will pass; displaying a result of facial recognition performed using the acquired image before the person reaches the facial recognition gate device; and changing a display mode based on the result of the facial recognition.

[0154] (Supplementary Note 20) A recording medium having recorded thereon a computer program for causing a computer to execute a display control method, which includes acquiring an image including a predetermined area through which a person heading towards a facial recognition gate device will pass, displaying the result of facial recognition performed using the acquired image before the person reaches the facial recognition gate device, and changing the display mode based on the result of the facial recognition.

[0155] (Supplementary Note 21) A display control system comprising: a facial recognition gate device; image acquisition means for acquiring an image including a predetermined area through which a person heading towards the facial recognition gate device will pass; authentication means for performing facial recognition using the acquired image; and display control means for displaying the result of the facial recognition before the person reaches the facial recognition gate device, wherein the display control means changes the display mode based on the result of the facial recognition.

[0156] This disclosure is not limited to the above-described embodiments, but may be modified as appropriate within the scope of the claims and the gist or idea of ​​the invention as can be read from the entire specification, and display control devices, display control methods, and recording media that involve such modifications are also included in the technical scope of this disclosure.

[0157] REFERENCE SIGNS LIST 1, 3 Display control device 2 Airport system 5 Authentication device 6 Terminal device 11, 311 Image acquisition unit 12, 315 Display control unit 21 Management server 22 Check-in terminal 23 Face recognition gate device 24 Normal gate device 31 Arithmetic unit 32 Storage device 33 Communication device 34 Input device 35 Output device 36, 52, 211, 232 Face database 51, 313 Face recognition unit 312 Tracking unit 314 Determination unit

Claims

1. an image acquisition means for acquiring an image including a predetermined area through which a person approaching the face recognition gate device passes; a display control means for displaying a result of face authentication performed using the acquired image before the person reaches the face authentication gate device; Equipped with The display control means changes a display mode based on a result of the face authentication. Display control device.

2. The result of the face authentication includes at least one of a first result indicating that a match is found and that the match can pass through the face authentication gate device, a second result indicating that a match is found but the match cannot pass through the face authentication gate device, and a third result indicating that a match is not found, The display mode for the first result, the display mode for the second result, and the display mode for the third result are different from each other. The display control device according to claim 1 .

3. The display control device includes a position detection means for detecting a position of the person based on the acquired image, The display control means changes a display position based on a position of the person. The display control device according to claim 1 .

4. the display control means displays the result of the face recognition on a display device provided along at least a part of a route along which people flow toward the face recognition gate device; The display control means detects a change in a position of the person based on a position of the person, The display control means changes a display position of the face authentication result on the display device based on a change in the position of the person. The display control device according to claim 3 .

5. The display control means causes the display device to display the result of the face authentication in front of and near the person based on the position of the person. The display control device according to claim 4.

6. The display device is provided along a portion of the path between the face recognition gate device and the predetermined area. The display control device according to claim 4.

7. The display device has one or more light-emitting panels. The display control device according to claim 4.

8. The predetermined area includes a queue formed by people waiting to pass through the face recognition gate device. The display control device according to claim 1 .

9. Acquire an image including a predetermined area through which a person approaching the face recognition gate device passes, displaying a result of face authentication performed using the acquired image before the person reaches the face authentication gate device; The display mode is changed based on the result of the face authentication. Display control method.

10. On the computer, Acquire an image including a predetermined area through which a person approaching the face recognition gate device passes, displaying a result of face authentication performed using the acquired image before the person reaches the face authentication gate device; The display mode is changed based on the result of the face authentication. A recording medium on which a computer program for executing a display control method is recorded.