Information processing device, information processing method, and program

Abstract

To provide an information processing device for supporting procedures in a boarding gate of a passenger.SOLUTION: An information processing device according to the present invention includes an acquisition part for acquiring biological information of a passenger who boards an aircraft and does not pass through a boarding gate corresponding to the aircraft, from a photographic image obtained by photographing the passenger, an input part for inputting an extraction condition related to boarding reservation information of the passenger, and an output part for outputting information for identifying the passenger whose boarding reservation information identified by using the biological information satisfies the extraction condition.SELECTED DRAWING: Figure 28

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a recording medium.

Background Art

[0002] Patent Document 1 discloses a ticketless boarding system that uses a passenger's biometric information (face image) to perform procedures by face authentication at a plurality of checkpoints (check-in lobby, security inspection area, boarding gate, etc.).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As exemplified in Patent Document 1, it is expected to improve the throughput in an airport by promoting the use of terminals having a face authentication function. However, in a conventional system as exemplified in Patent Document 1, it is not assumed to support the procedures at the boarding gate for passengers waiting around the boarding gate.

[0005] Therefore, in view of the above problems, an object of the present invention is to provide an information processing apparatus, an information processing method, and a recording medium that support a passenger's procedures at a boarding gate.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing device is provided, comprising: an acquisition unit that acquires biometric information of a passenger from a photograph taken of a passenger who is on board an aircraft but has not passed through a boarding gate corresponding to the aircraft; a receiving unit that receives extraction conditions relating to the passenger's boarding reservation information; and an output unit that superimposes information identifying the passenger whose boarding reservation information, identified using the biometric information, satisfies the extraction conditions onto the photograph and outputs it.

[0007] According to another aspect of the present invention, an information processing method is provided in which a computer acquires biometric information of a passenger from a captured image of the passenger who is boarding an aircraft but has not passed through the boarding gate corresponding to the aircraft, receives extraction conditions relating to the passenger's boarding reservation information, and outputs information that identifies the passenger whose boarding reservation information, identified using the biometric information, satisfies the extraction conditions, superimposed on the captured image.

[0008] In yet another aspect of the present invention, a program is provided which causes a computer to perform the steps of: acquiring biometric information of a passenger from a photograph taken of a passenger who is boarding an aircraft but has not passed through a boarding gate corresponding to the aircraft; receiving extraction conditions relating to the passenger's boarding reservation information; and outputting information superimposed on the photographed image that identifies the passenger whose boarding reservation information, identified using the biometric information, satisfies the extraction conditions. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an information processing device, an information processing method, and a recording medium that support procedures at passenger boarding gates. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing an example of the overall configuration of the information processing system in the first embodiment. [Figure 2] This figure shows an example of the arrangement of the boarding gate device, business terminal, and camera in the first embodiment. [Figure 3] This figure shows an example of the information stored in the token ID information database in the first embodiment. [Figure 4] This figure shows an example of the information stored in the passage history information database in the first embodiment. [Figure 5] This figure shows an example of the information stored in the business information database in the first embodiment. [Figure 6] This figure shows an example of the information stored in the reservation information database in the first embodiment. [Figure 7] This is a block diagram showing an example of the hardware configuration of the management server in the first embodiment. [Figure 8] Block diagram showing an example of the hardware configuration of a check-in terminal in the first embodiment. [Figure 9] This is a block diagram showing an example of the hardware configuration of an automated baggage drop-off machine in the first embodiment. [Figure 10] This is a block diagram showing an example of the hardware configuration of a security inspection device in the first embodiment. [Figure 11] This is a block diagram showing an example of the hardware configuration of an automated gate device in the first embodiment. [Figure 12] This is a block diagram showing an example of the hardware configuration of a boarding gate device in the first embodiment. [Figure 13] This is a block diagram showing an example of the hardware configuration of a business terminal in the first embodiment. [Figure 14] This is a sequence diagram showing an example of the processing of the reservation system, check-in terminal, and management server in the first embodiment. [Figure 15] This is a sequence diagram showing an example of the processing of the reservation system, automated baggage drop-off machine, and management server in the first embodiment. [Figure 16] This is a sequence diagram showing an example of the processing of the reservation system, security inspection device, and management server in the first embodiment. [Figure 17]It is a sequence diagram showing an example of the processing of a reservation system, an automated gate device, and a management server in the first embodiment. [Figure 18] It is a sequence diagram showing an example of the processing of a reservation system, a business terminal, a management server, and a camera in the first embodiment. [Figure 19] It is a diagram showing an example of a business screen displayed on the business terminal in the first embodiment. [Figure 20] It is a diagram showing an example of a passenger list screen displayed on the business terminal in the first embodiment. [Figure 21] It is a sequence diagram showing an example of the processing of a reservation system, a business terminal, a management server, and a camera in the first embodiment. [Figure 22] It is a diagram showing an example of a guidance instruction screen for priority boarding displayed on the business terminal in the first embodiment. [Figure 23] It is a sequence diagram showing an example of the processing of a reservation system, a business terminal, a management server, and a camera in the first embodiment. [Figure 24] It is a diagram showing an example of a guidance instruction screen for a waiting lane displayed on the business terminal in the first embodiment. [Figure 25] It is a sequence diagram showing an example of the processing of a reservation system, a business terminal, a management server, and a camera in the first embodiment. [Figure 26] It is a diagram showing an example of a passenger inquiry screen displayed on the business terminal in the first embodiment. [Figure 27] It is a sequence diagram showing an example of the processing of a reservation system, a business terminal, a management server, and a camera in the first embodiment. [Figure 28] It is a diagram showing an example of an extraction condition input screen displayed on the business terminal in the first embodiment. [Figure 29] It is a diagram showing an example of a passenger extraction result screen displayed on the business terminal in the first embodiment. [Figure 30] It is a sequence diagram showing an example of the processing of a reservation system, a boarding gate device, and a management server in the first embodiment. [Figure 31]This flowchart shows an example of the process for verifying the presence of a companion in the second embodiment. [Figure 32] This figure shows an example of a companion confirmation instruction screen displayed on a business terminal in the second embodiment. [Figure 33] This figure shows an example of a companion confirmation instruction screen displayed on a business terminal in the second embodiment. [Figure 34] This is a curved diagram showing an example of the overall configuration of the information processing system in the third embodiment. [Figure 35] This is a sequence diagram showing an example of processing for the business terminal, management server, and camera in the third embodiment. [Figure 36] This figure shows an example of a passenger search results screen displayed on a business terminal in the third embodiment. [Figure 37] This is a block diagram showing an example of the overall configuration of the information processing device in the fourth embodiment. [Modes for carrying out the invention]

[0011] Illustrative embodiments of the present invention will be described below with reference to the drawings. In the drawings, similar or corresponding elements are denoted by the same reference numerals, and their descriptions may be omitted or simplified.

[0012] <First Embodiment> Figure 1 is a block diagram showing an example of the overall configuration of the information processing system 1 in this embodiment. The information processing system 1 is a computer system that manages and supports operations related to the immigration and customs screening procedures for users (hereinafter also referred to as "passengers") U at airport A. The information processing system 1 is operated by a public institution such as an immigration and customs authority or a contractor entrusted with the operations by such an institution. The reservation system 2, unlike the information processing system 1, is a computer system installed by an airline. The reservation system 2 is equipped with a reservation information DB (database) 3 that manages boarding reservation information. Note that in Figure 1, only one reservation system 2 is shown for the sake of explanation, but reservation systems 2 are installed in multiple airlines.

[0013] In the information processing system 1 of this embodiment, the check-in terminal 20, automated baggage drop-off machine 30, security screening device 40, automated gate device 50, and boarding gate device 60 are each connected to a common management server 10 via network NW1. The security screening device 40, automated gate device 50, and boarding gate device 60 are installed in security area SA, indicated by dashed lines. Similarly, each airline's reservation system 2 has the check-in terminal 20, automated baggage drop-off machine 30, security screening device 40, automated gate device 50, and boarding gate device 60 each connected to a server (not shown) via network NW2. The work terminal 70 used by staff S is connected to networks NW1 and NW2 via an access point (not shown).

[0014] Networks NW1 and NW2 consist of a LAN (Local Area Network), WAN (Wide Area Network), mobile communication network, etc., including the airport A's internal communication network. The connection method is not limited to wired; it may also be wireless. Networks NW1 and NW2 are different networks. In other words, in this embodiment, the information processing system 1 is not directly connected to the reservation system 2.

[0015] The management server 10 is located, for example, within the facilities of the airport company that operates airport A. However, the management server 10 may not be a server located within the facility where operations are actually performed, but rather a cloud server. Furthermore, the management server 10 does not need to be a single server; it may be configured as a group of servers including multiple servers.

[0016] As shown in Figure 1, the immigration procedures for departure at Airport A are carried out sequentially at five touchpoints. The relationship between each device and each touchpoint is explained below.

[0017] Check-in terminal 20 is located in the check-in lobby of Airport A (hereinafter referred to as "touchpoint P1"). Check-in terminal 20 is a self-service terminal for passenger U to complete the check-in procedure (boarding procedure) themselves. After completing the procedure at touchpoint P1, passenger U proceeds to the baggage counter or security checkpoint.

[0018] The automated baggage drop-off machine 30 is installed at the baggage counter (hereinafter referred to as "touchpoint P2") within Airport A. The automated baggage drop-off machine 30 is a self-service terminal that allows user U to operate and check in baggage that will not be carried onto the aircraft (baggage check-in procedure). After completing the procedure at touchpoint P2, user U proceeds to the security checkpoint. If user U does not check in baggage, the procedure at touchpoint P2 is omitted.

[0019] The security screening device 40 is installed at the security checkpoint (hereinafter referred to as "touchpoint P3") within Airport A. The security screening device 40 is a device that uses a metal detector to check whether or not user U is wearing any metal objects that could be considered dangerous. In this embodiment, the term "security screening device" is used to include not only metal detectors, but also X-ray inspection devices that use X-rays to check for dangerous objects in carry-on baggage, etc., and terminal devices of the Passenger Reconciliation System (PRS) that determine whether user U is allowed to pass through the entrance of the security checkpoint. User U, who has completed check-in procedures and automated baggage drop-off procedures, undergoes security screening procedures using the security screening device 40 at the security checkpoint. After completing the procedures at touchpoint P3, user U will proceed to the immigration area.

[0020] The automated gate device 50 is installed at the departure immigration area (hereinafter referred to as "touchpoint P4") within Airport A. The automated gate device 50 is a device that automatically performs the departure immigration procedures for user U. After completing the procedures at touchpoint P4, user U moves to the departure area where duty-free shops and boarding gates are located.

[0021] The boarding gate device 60 is a traffic control device installed at each boarding gate in the departure area (hereinafter referred to as "touch point P5"). The boarding gate device 60 confirms that user U is a passenger on an aircraft that can be boarded from the boarding gate. Once user U completes the procedures at touch point P5, they board the aircraft and depart.

[0022] The business terminal 70 is a terminal device used by staff member S for their duties. Examples of business terminals 70 include, but are not limited to, personal computers, tablet terminals, and smartphones. The business terminal 70 receives information transmitted from the management server 10 and displays it on its screen. In this embodiment, the management server 10 transmits information to the business terminal 70 to support passenger boarding procedures at the boarding gate. Details of the information will be described later.

[0023] Camera 80 is a camera device that is positioned in multiple locations in the area surrounding the boarding gate (adjacent area) and each camera captures the area surrounding the boarding gate. Camera 80 is mounted on, for example, the ceiling, wall, pillar, etc., so that it can capture the faces of passengers U waiting to board who are present in the area surrounding the boarding gate. Camera 80 can be either fixed or movable.

[0024] Figure 2 shows an example of the arrangement of the boarding gate device 60, the work terminal 70, and the camera 80. Here, the camera 80 is shown as being installed in multiple locations around the boarding gate to photograph passengers from various angles. It is preferable that each work terminal 70 can arbitrarily switch between images captured by multiple cameras 80 on its work screen. Two types of work terminals 70 used by staff S are shown: a portable type and a stationary type. L1 to L3 illustrate waiting areas (waiting lanes) where users U line up before boarding.

[0025] Generally, airlines pre-assign passengers to several groups based on various criteria such as seat class, membership status, companion information, and seat location, and guide passengers to the boarding gate in a predetermined group order. For example, passengers traveling with infants, passengers in first class, and passengers recognized as elite members by the airline are assigned to groups with relatively higher priority for boarding. These higher priority groups are eligible for priority boarding services.

[0026] Furthermore, as shown in Figure 1, the management server 10 has a token ID information DB 11, a transaction history information DB 12, and a business information DB 13. However, the databases possessed by the management server 10 are not limited to these databases.

[0027] Figure 3 shows an example of the information stored in the Token ID Information DB11. The Token ID Information DB11 includes the following data items: Token ID, Group ID, Feature quantity, Registered facial image, Token issuance time, Token issuing device name, Invalid flag, Invalidation time, and Companion ID. The Token ID is an identifier that uniquely identifies the ID information. In this embodiment, the Token ID is issued temporarily on the condition that the comparison result between the passport facial image read from the passport at touchpoint P1 and the facial image taken of user U, who possesses the passport, matches. The Token ID is then invalidated when user U completes the procedure at touchpoint P5 (boarding gate). In other words, the Token ID is a one-time ID with an expiration period (lifecycle).

[0028] The group ID is an identifier used to group ID information. Feature quantities are values ​​extracted from biometric information. For example, the same group ID is assigned to individuals with companions or to groups. The registered face image is the face image registered for user U. In this embodiment, the term "biometric information" refers to the face image and the feature quantities extracted from the face image. Note that biometric information is not limited to face images and face feature quantities. That is, biometric authentication may be performed using fingerprint images, palm print images, ear images, iris images, etc., as the biometric information of user U.

[0029] The token issuance time is the time when the management server 10 issued the token ID. The token issuance device name is the name of the device from which the registered face image that triggered the issuance of the token ID was obtained. The invalid flag is flag information indicating whether the token ID is currently valid or not. When a token ID is issued, the invalid flag becomes a value of "1", indicating that the token ID is valid. Also, when certain conditions are met, the invalid flag is updated to a value of "0", indicating that the token ID is invalid. The invalidation time is the timestamp when the invalid flag was invalidated.

[0030] A companion ID is a token ID issued for a person who boards an aircraft with the assistance of another passenger, such as an infant or young child (hereinafter referred to as the "supported person"). The person to whom the companion ID is associated is the person providing the support, such as a parent or guardian, or a caregiver (hereinafter referred to as the "supporter"). The companion ID is issued to the supported person simultaneously, for example, when the supporter first completes the procedure. In this embodiment, we will describe the case in which one supported person is associated with one supporter. However, it is also possible for multiple supported people to be associated with one supporter.

[0031] Figure 4 shows an example of the information stored in the passage history information DB12. The passage history information DB12 has the following data items: passage history ID, token ID, passage time, device name, business system type, and passage touchpoint. The passage history ID is an identifier that uniquely identifies the passage history information. The passage time is the timestamp when the passenger passed through touchpoints P1 to P5. The device name is the name of the terminal used for the procedure at touchpoints P1 to P5. The business system type is the type of business system to which the terminal belongs. The passage touchpoint is the name of the touchpoint P1 to P5 that was passed through. The management server 10 can determine which touchpoints the user U has completed the procedure up to by extracting the passage history information on a token ID basis.

[0032] Figure 5 shows an example of the information stored in the Business Information DB13. The Business Information DB13 uses Token ID, Reservation Number, Airline Code, and Business Information as data items. The Reservation Number is an identifier that uniquely identifies the reservation information for boarding an aircraft. The Airline Code is an identifier that uniquely identifies the airline. Business Information is arbitrary information obtained through operations at each touchpoint.

[0033] Figure 6 shows an example of the information stored in the reservation information DB3. The reservation information DB3 includes the following data items: reservation number, airline code, passenger name, departure city, destination, flight number, date of operation, seat number, seat class (e.g., first class / business class / economy class), nationality, passport number, last name, first name, date of birth, gender, membership status, presence or absence of accompanying persons, and accompanying person status.

[0034] In this embodiment, the business information DB13 and the reservation information DB3 are associated by the reservation number and airline code. Specifically, when the terminal device at each touchpoint (check-in terminal 20, etc.) reads the reservation number and airline code from the ticket medium presented by the passenger, it can query the airline's reservation system 2 corresponding to the airline code for boarding reservation information based on the reservation number. However, the method of querying the reservation system 2 for boarding reservation information is not limited to this.

[0035] Next, with reference to Figures 7 to 13, we will describe examples of the hardware configurations of each device that constitutes the information processing system 1. Note that devices with the same name but different reference numerals in Figures 7 to 13 have equivalent functions, and therefore, detailed explanations will be omitted in subsequent drawings.

[0036] Figure 7 is a block diagram showing an example of the hardware configuration of the management server 10. As shown in the figure, the management server 10 has a CPU (Central Processing Unit) 101, RAM (Random Access Memory) 102, storage device 103, and communication interface 104. Each device is connected to bus line 105.

[0037] The CPU 101 is a processor that performs predetermined operations according to a program stored in the storage device 103 and also has the function of controlling various parts of the management server 10. The RAM 102 is composed of a volatile storage medium and provides a temporary memory area necessary for the operation of the CPU 101.

[0038] The storage device 103 is composed of a storage medium such as non-volatile memory or a hard disk drive, and functions as a storage unit. The storage device 103 stores programs executed by the CPU 101, data referenced by the CPU 101 during the execution of those programs, and so on.

[0039] Communication I / F104 is a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), and 4G, and is a module for communicating with check-in terminals 20, etc.

[0040] Figure 8 is a block diagram showing an example of the hardware configuration of the check-in terminal 20. As shown in the figure, the check-in terminal 20 includes a CPU 201, RAM 202, storage device 203, communication interface 204, input device 206, display device 207, media reader 208, and biometric information acquisition device 209. Each device is connected to the bus line 205.

[0041] The input device 206 is, for example, a pointing device such as a touch panel or a keyboard. The display device 207 is a liquid crystal display device, an OLED (Organic Light Emitting Diode) display device, etc., and is used for displaying videos, still images, text, etc. In the check-in terminal 20 of this embodiment, the input device 206 and the display device 207 are integrally formed as a touch panel.

[0042] The media reader 208 is a device that reads the user U's passport or airline ticket media and acquires the information recorded on the passport or airline ticket. Examples of airline ticket media include paper airline tickets and mobile terminals that display e-ticket confirmations. The media reader 208 is composed of, for example, a code reader, an image scanner, a contactless IC (Integrated Circuit) reader, an OCR (Optical Character Reader) device, and acquires information from various media held over its reading unit.

[0043] The biometric information acquisition device 209 is a device that acquires a facial image of user U as biometric information of user U. For example, the biometric information acquisition device 209 is a digital camera that photographs the face of user U standing in front of the check-in terminal 20, and acquires the facial image of user U by photographing it.

[0044] Figure 9 is a block diagram showing an example of the hardware configuration of the automated baggage drop-off machine 30. As shown in the figure, the automated baggage drop-off machine 30 includes a CPU 301, RAM 302, storage device 303, communication interface 304, input device 306, display device 307, media reader 308, biometric information acquisition device 309, baggage transport device 310, and output device 311. Each device is connected to the bus line 305.

[0045] The baggage handling device 310, upon successful verification of user U's identity, transports the baggage to be loaded onto the aircraft on which user U will be boarding. The baggage handling device 310 transports the baggage, which has been placed in the receiving area by user U and tagged, toward baggage handling.

[0046] Output device 311 is a device that prints baggage tags to be attached to checked baggage. Output device 311 also prints baggage claim slips that are necessary for claiming baggage after arriving at the destination. Note that baggage tags and baggage claim slips are associated with at least one of the following: passport information or boarding information.

[0047] Figure 10 is a block diagram showing an example of the hardware configuration of the security inspection device 40. As shown in the figure, the security inspection device 40 includes a CPU 401, RAM 402, storage device 403, communication I / F 404, input device 406, display device 407, media reader 408, biometric information acquisition device 409, and metal detection gate 410. Each device is connected to a bus line 405.

[0048] The metal detection gate 410 is a gate-type metal detector that detects metal objects worn by user U as they pass through the metal detection gate 410.

[0049] Figure 11 is a block diagram showing an example of the hardware configuration of the automated gate device 50. The automated gate device 50 includes a CPU 501, RAM 502, storage device 503, communication I / F 504, input device 506, display device 507, media reader 508, biometric information acquisition device 509, and gate 511. Each device is connected to the bus line 505. The automated gate device 50 installed in the immigration inspection area has the same hardware configuration as the automated gate device 50 installed in the departure inspection area.

[0050] When user U's identity is successfully verified by the automated gate device 50 and user U passes through immigration, gate 511 transitions from a closed state that blocks user U's passage while waiting to an open state that allows user U to pass, in accordance with the control of the CPU 501. The type of gate 511 is not particularly limited and may include, for example, a flapper gate in which flappers provided on one or both sides of the passage open and close, or a turnstile gate in which three bars rotate.

[0051] Figure 12 is a block diagram showing an example of the hardware configuration of the boarding gate device 60. As shown in the figure, the boarding gate device 60 includes a CPU 601, RAM 602, storage device 603, communication I / F 604, input device 606, display device 607, biometric information acquisition device 609, and gate 611. Each device is connected to a bus line 605.

[0052] Figure 13 is a block diagram showing an example of the hardware configuration of a business terminal 70. As shown in the figure, the business terminal 70 has a CPU 701, RAM 702, storage device 703, communication interface 704, input device 706, and display device 707. Each device is connected to a bus line 705.

[0053] Next, the operation of each device in the information processing system 1 in this embodiment is shown in Figures 14 to 14. I will explain based on 30.

[0054] [Check-in procedure] Figure 14 is a sequence diagram showing an example of processing by the reservation system 2, check-in terminal 20, and management server 10. This processing is executed when user U performs the check-in procedure using the check-in terminal 20.

[0055] First, the check-in terminal 20 determines whether or not user U's passport has been held over the reading section (not shown) of the media reader 208 (step S101), and waits until the passport is held over the reader (step S101, NO).

[0056] Next, when the check-in terminal 20 determines that the passport has been held over the reader of the media reader 208 (step S101, YES), it obtains the passport information of user U from the passport that has been held over the reader (step S102). The obtained passport information includes the passport facial image of user U, identity verification information, passport number, information of the passport issuing country, etc.

[0057] Next, the check-in terminal 20 takes a picture of user U's face using the biometric information acquisition device 209. (Step S103) The facial image and passport information are sent to the management server 10 (Step S104). It is preferable to display a screen to obtain the user U's consent before taking the facial image.

[0058] When the management server 10 receives information from the check-in terminal 20, it performs a one-to-one comparison between the facial image recorded on user U's passport (hereinafter referred to as the "passport facial image") and the facial image taken by the check-in terminal 20 (hereinafter referred to as the "target facial image") (step S105).

[0059] Next, if the management server 10 determines that the comparison result between the passport face image and the target face image is a match (step S106, YES), it issues a token ID (step S107). The token ID is set to a unique value based, for example, on the date and time of processing or a sequence number.

[0060] Next, the management server 10 registers the relationship between the token ID and the registered face image as the target face image in the token ID information DB 11 (step S108).

[0061] In this embodiment, the reason for using a facial image taken on-site (target facial image) as the registered facial image is that the validity period (lifecycle) of the token ID is completed within the same day, and the quality (appearance) of the captured image is closer to that of the image taken in the subsequent authentication process than that of the passport facial image. However, instead of the target facial image (captured facial image), the passport facial image may be set as the registered facial image (registered biometric information). For example, if the lifecycle of the token ID is long-term (for example, in the airline industry, if the token ID is only valid for a certain period for members), it is advisable to set the facial image from the passport or driver's license as the registered facial image.

[0062] Next, the management server 10 sends the issued token ID and the matching result to the check-in terminal 20 (step S109).

[0063] On the other hand, if the management server 10 determines that the comparison result between the passport face image and the target face image is a mismatch (step S106, NO), it sends the mismatch result to the check-in terminal 20 (step S110).

[0064] Next, if the check-in terminal 20 determines that it can perform the check-in procedure based on the matching result received from the management server 10 (step S111, YES), the process proceeds to step S112. Conversely, if the check-in terminal 20 determines that it cannot perform the check-in procedure based on the matching result received from the management server 10 (step S111, NO), it notifies user U of an error message (step S113).

[0065] In step S112, the check-in terminal 20 determines whether or not user U's airline ticket medium has been held over the reader of the medium reader 208. The check-in terminal 20 waits until the airline ticket medium is held over the reader (step S112, NO).

[0066] Next, when the check-in terminal 20 determines that the airline ticket medium has been held over the reader of the medium reader 208 (step S112, YES), it obtains recorded data such as the reservation number and airline code from the scanned airline ticket medium (step S114).

[0067] Next, the check-in terminal 20 transmits the recorded data to the airline's reservation system 2 corresponding to the airline code (step S115) and requests verification of the recorded data against the boarding reservation information.

[0068] When the reservation system 2 receives the recorded data from the check-in terminal 20, it compares the recorded data with the boarding reservation information recorded in the reservation information DB3 (step S116), and sends the comparison result to the check-in terminal 20 (step S117).

[0069] Next, when the check-in terminal 20 receives a matching result from the reservation system 2, it performs check-in procedures such as confirming the itinerary and selecting a seat based on the information entered by user U (step S118). If the matching result in the reservation system 2 is not a match, the check-in terminal 20 should not perform the check-in procedure and should notify an error. The check-in terminal 20 then sends the token ID, business information, and passage history information indicating the completion of the procedure at the check-in terminal 20 to the management server 10 (step S119). The business information shall include at least the reservation number and airline code. The passage history information shall include the time of passage at touchpoint P1 and information such as the device name of the terminal used for the procedure.

[0070] Next, when the management server 10 receives information from the check-in terminal 20, it registers passage history information showing the relationship between the token ID and the passage information of touch point P1 in the passage history information DB 12 (step S120). Then, if necessary, the management server 10 registers the business information received from the check-in terminal 20 in the business information DB 13 (step S121).

[0071] In this way, during the check-in procedure, the target facial image (captured facial image) that matches the passport facial image obtained from the passport is registered as the registered facial image in the token ID information DB11, and the registered facial image is associated with the business information in the business information DB13 using the issued token ID. As a result, biometric authentication by facial matching between the captured facial image and the registered facial image becomes possible at each subsequent touchpoint.

[0072] [Baggage check-in procedure] Figure 15 is a sequence diagram showing an example of the processing performed by the reservation system 2, the automated baggage drop-off machine 30, and the management server 10. This process is executed when a user U, who has completed the check-in procedure, proceeds with baggage drop-off as needed.

[0073] The automated baggage drop-off machine 30 continuously or periodically photographs the area in front of the device and determines whether or not the face of user U standing in front of the automated baggage drop-off machine 30 is detected in the captured image (step S201). The automated baggage drop-off machine 30 waits until the biometric information acquisition device 309 detects the face of user U in the image (step S201, NO).

[0074] When the automated baggage drop-off machine 30 determines that the face of user U has been detected by the biometric information acquisition device 309 (step S201, YES), it takes a photograph of user U's face and acquires user U's face image as the target face image (step S202).

[0075] Next, the automated baggage drop-off machine 30 transmits the target face image of user U, captured by the biometric information acquisition device 309, to the management server 10 along with a matching request (step S203). This prompts the automated baggage drop-off machine 30 to request the management server 10 to perform a one-to-many matching between the target face image of user U, captured by the biometric information acquisition device 309, and multiple registered face images registered in the token ID information DB 11 of the management server 10.

[0076] When the management server 10 receives the target face image and matching request from the automated baggage drop-off machine 30, it performs a matching of the user U's face image (step S204). That is, the management server 10 performs a one-to-many matching of the target face image received from the automated baggage drop-off machine 30 with multiple registered face images registered in the token ID information DB 11. Note that the registered face images to be matched are limited to images associated with token IDs whose invalid flag value is "1" (valid).

[0077] If the management server 10 determines that the matching result is a mismatch (step S205, NO), the management server 10 sends the mismatched matching result to the automated baggage drop-off machine 30 (step S208), and the process proceeds to step S209. If the management server 10 determines that the matching result is a match (step S205, YES), the process proceeds to step S206.

[0078] In step S206, the management server 10 retrieves the token ID associated with the matched registered face image in the token ID information DB 11, and retrieves the reservation number and airline code from the business information DB 13 using the token ID as a key. Then, the management server 10 transmits the token ID, reservation number, airline code and matching result to the automated baggage drop-off machine 30 (step S207).

[0079] Next, if the automated baggage drop-off machine 30 determines that it can perform the procedure based on the verification result (step S209, YES), it sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S210) and inquires about the boarding reservation information.

[0080] In response, if the automated baggage drop-off machine 30 determines that it cannot perform the procedure based on the verification result (step S209, NO), it notifies user U of an error message (step S213).

[0081] When reservation system 2 receives a reservation number from automated baggage drop-off machine 30, it transmits the corresponding boarding reservation information to automated baggage drop-off machine 30 (step S211).

[0082] Next, when the automated baggage drop-off machine 30 receives boarding reservation information from the reservation system 2, it performs the baggage drop-off procedure for user U (step S212).

[0083] Next, the automated baggage drop-off machine 30 sends passage history information to the management server 10, indicating that user U's baggage drop-off procedure has been completed, after verifying the token ID, business information, and facial image (step S214). The passage history information includes information such as the time of passage through touchpoint P2 and the device name of the terminal used.

[0084] When the management server 10 receives information from the automated baggage drop-off machine 30, it registers passage history information showing the relationship between the token ID and the passage information of user U's touchpoint P2 in the passage history information DB 12 (step S215). Then, if necessary, the management server 10 registers the business information received from the automated baggage drop-off machine 30 in the business information DB 13 (step S216).

[0085] [Security Inspection Procedures] Figure 16 is a sequence diagram showing an example of the processing performed by the reservation system 2, the security screening device 40, and the management server 10. This processing is performed when a user U, who has completed the check-in procedure or baggage drop-off procedure, proceeds to the security screening procedure.

[0086] The security inspection device 40 continuously or periodically photographs the area in front of the metal detection gate 410 and determines whether or not the face of user U standing in front of the metal detection gate 410 is detected in the captured image (step S301). The security inspection device 40 waits until the face of user U is detected in the image by the biometric information acquisition device 409 (step S301, NO).

[0087] When the security inspection device 40 determines that the face of user U has been detected by the biometric information acquisition device 409 (step S301, YES), it takes a photograph of user U's face and acquires user U's face image as the target face image (step S302).

[0088] Next, the security inspection device 40 transmits the target face image of user U, captured by the biometric information acquisition device 409, to the management server 10 along with a matching request (step S303). This prompts the security inspection device 40 to request the management server 10 to perform a one-to-many matching between the target face image of user U captured by the biometric information acquisition device 409 and multiple registered face images registered in the token ID information DB 11 of the management server 10.

[0089] When the management server 10 receives the target face image and matching request from the security inspection device 40, it performs a matching of the user U's face image (step S304). That is, the management server 10 performs a one-to-many matching of the target face image received from the security inspection device 40 with multiple registered face images registered in the token ID information DB 11. Note that the registered face images to be matched are limited to images associated with token IDs whose invalid flag value is "1" (valid).

[0090] If the management server 10 determines that the matching result is a mismatch (step S305, NO), the management server 10 sends the mismatch result to the security inspection device 40 (step S308), and the process proceeds to step S309. If the management server 10 determines that the matching result is a match (step S305, YES), the process proceeds to step S306.

[0091] In step S306, the management server 10 retrieves the token ID associated with the matched registered face image from the token ID information DB 11, and retrieves the reservation number and airline code from the business information DB 13 using the token ID as a key. The management server 10 then transmits the token ID, reservation number, airline code, and matching result to the security inspection device 40 (step S307). The process then proceeds to step S309.

[0092] Next, if the security screening device 40 determines that it can perform the procedure based on the verification result (step S309, YES), it sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S310) and inquires about the boarding reservation information.

[0093] In response, if the security inspection device 40 determines that it cannot perform the procedure based on the verification result (step S309, NO), it notifies the user U of an error message (step S313).

[0094] When the reservation system 2 receives a reservation number from the security screening device 40, it transmits the corresponding boarding reservation information to the security screening device 40 (step S311).

[0095] Next, when the security screening device 40 receives boarding reservation information from the reservation system 2, it performs the security screening procedure for user U (step S312). During the security screening process, the CPU 401 controls various parts of the security screening device 40. As a result, the security screening device 40 detects any metal objects worn by user U as they pass through the metal detector gate 410. After passing through the metal detector gate 410, user U proceeds to the immigration area.

[0096] Next, the security inspection device 40 sends passage history information to the management server 10, indicating that user U's security inspection procedure has been completed, after verifying the token ID, business information, and facial image (step S314). The passage history information includes information such as the time of passage through touch point P3 and the device name of the terminal used.

[0097] When the management server 10 receives information from the security inspection device 40, it registers passage history information showing the relationship between the token ID and the passage information of user U's touchpoint P3 in the passage history information DB 12 (step S315). Then, if necessary, the management server 10 registers the business information received from the security inspection device 40 in the business information DB 13 (step S316).

[0098] [Departure Immigration Procedures] Figure 17 is a sequence diagram showing an example of the processing performed by the reservation system 2, the automated gate device 50, and the management server 10. This processing is performed when user U undergoes immigration procedures.

[0099] The automated gate device 50 continuously or periodically photographs the area in front of it and determines whether or not the face of user U standing in front of the automated gate device 50 is detected in the captured image (step S401). The automated gate device 50 waits until the face of user U is detected in the image by the biometric information acquisition device 509 (step S401, NO).

[0100] When the automated gate device 50 determines that the face of user U has been detected by the biometric information acquisition device 509 (step S401, YES), it takes a photograph of user U's face and acquires user U's face image as the target face image (step S402).

[0101] Next, the automated gate device 50 transmits the target face image of user U, captured by the biometric information acquisition device 509, to the management server 10 along with a matching request (step S403). This prompts the automated gate device 50 to request the management server 10 to perform a one-to-many matching between the target face image of user U captured by the biometric information acquisition device 509 and multiple registered face images registered in the token ID information DB 11 of the management server 10.

[0102] When the management server 10 receives the target face image and matching request from the automated gate device 50, it performs a matching of the user U's face image (step S404). That is, the management server 10 performs a one-to-many matching of the target face image received from the automated gate device 50 with multiple registered face images registered in the token ID information DB 11. Note that the registered face images to be matched are limited to images associated with token IDs whose invalid flag value is "1" (valid).

[0103] Here, if the management server 10 determines that the matching result is a mismatch (step S4) If step S405, NO, the management server 10 sends the matching result of the mismatch to the automated gate device 50 (step S408), and the process proceeds to step S409. If the management server 10 determines that the matching result is a match (step S405, YES), the process proceeds to step S406.

[0104] In step S406, the management server 10 retrieves the token ID associated with the matched registered face image from the token ID information DB 11, and retrieves the reservation number and airline code from the business information DB 13 using the token ID as a key. The management server 10 then transmits the token ID, reservation number, airline code, and matching result to the automated gate device 50 (step S407). The process then proceeds to step S409.

[0105] Next, if the automated gate device 50 determines that it can perform the procedure based on the matching result (step S409, YES), it sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S410) and inquires about the boarding reservation information.

[0106] In response, if the automated gate device 50 determines that it cannot perform the procedure based on the verification result (step S409, NO), it notifies the user U of an error message (step S413). For example, it displays a notification screen on the display device 507 that includes a message such as, "Please proceed with the departure immigration procedure at the manned counter."

[0107] When the reservation system 2 receives a reservation number from the automated gate device 50, it transmits the corresponding boarding reservation information to the automated gate device 50 (step S411).

[0108] Next, when the automated gate device 50 receives boarding reservation information from the reservation system 2, it performs the departure immigration procedure for passenger U (step S412) and opens gate 511 (step S414). Passenger U, having passed through touchpoint P4, proceeds to the departure area where the boarding gate is located.

[0109] Next, the automated gate device 50 sends passage history information to the management server 10, indicating that user U's departure immigration procedure has been completed, after matching the token ID, business information, and facial image (step S415). The passage history information includes information such as the time of passage through touchpoint P4 and the device name of the terminal used.

[0110] When the management server 10 receives information from the automated gate device 50, it registers passage history information showing the relationship between the token ID and the passage information of user U's touchpoint P4 in the passage history information DB 12 (step S416). Then, if necessary, the management server 10 registers the business information received from the automated gate device 50 in the business information DB 13 (step S417).

[0111] [Procedure support before boarding (1)] Figure 18 is a sequence diagram showing an example of processing by the reservation system 2, the business terminal 70, the management server 10, and the camera 80. This processing is performed, for example, when an employee S refers to passenger information contained in the surveillance image of the area surrounding the boarding gate.

[0112] First, the business terminal 70 sends a request to the management server 10 to display the business screen based on the input information from the staff member S (step S501). When the management server 10 receives the display request from the business terminal 70, it acquires images of the area around the boarding gate from multiple cameras 80 that are distributed around the boarding gate (step S502).

[0113] Next, when the management server 10 acquires the captured image from the camera 80, it sends the business screen containing the captured image to the business terminal 70 (step S503).

[0114] Next, the business terminal 70 displays the business screen received from the management server 10 on the display device 707 (step S504).

[0115] Figure 19 shows an example of a business screen displayed on the business terminal 70. The central area of ​​the screen displays the captured image. The dashed lines within the captured image indicate the detection areas for the individual faces of multiple users U1 to U12. The lower area of ​​the screen displays a camera switching button for switching the displayed image to images captured by cameras 80 with different shooting angles, a passenger list screen button for displaying a list of passengers included in the captured image, and an exit button for closing the screen. It is preferable to distinguish passengers whose token IDs are not registered or who could not be authenticated from registered passengers (authenticated passengers) on the screen using a different display format.

[0116] Next, the business terminal 70 sends a request to the management server 10 to display the passenger list screen based on the input information from the staff member S (step S505). When the management server 10 receives the display request from the business terminal 70, it detects people from the captured image of the area around the boarding gate (step S506). If the captured image contains multiple people, the management server 10 detects the face image of each person as a target face image.

[0117] Next, the management server 10 performs a one-to-many comparison between the target face image and multiple registered face images registered in the token ID information DB 11 (step S507). That is, the management server 10 identifies each registered face image that matches the target face image of the passenger included in the captured image. Note that the registered face images to be matched are limited to images associated with token IDs whose invalid flag value is "1" (valid). Furthermore, by referring to the passage history information DB 12, only images of people who have passed through the departure immigration checkpoint may be used as the match target.

[0118] Next, the management server 10 retrieves the token ID associated with the matched registered face image in the token ID information DB 11, and uses the token ID as a key to retrieve each person's reservation number and airline code from the business information DB 13 (step S508). Then, the management server 10 sends the dataset of token ID, reservation number, and airline code to the business terminal 70 (step S509).

[0119] Next, when the business terminal 70 receives the data set for each person from the management server 10, it sequentially sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S510) and queries the boarding reservation information for each person.

[0120] When the reservation system 2 receives each person's reservation number from the business terminal 70, it transmits the corresponding boarding reservation information to the business terminal 70 (step S511).

[0121] Next, when the business terminal 70 receives the boarding reservation information for each person, it sends a dataset containing the token ID, reservation number, airline code, and boarding reservation information to the management server 10 (step S512).

[0122] Next, when the management server 10 receives the data set for each person from the business terminal 70, it sends a passenger list screen created based on the data set to the business terminal 70 (step S513). The passenger list screen includes boarding reservation information associated with each registered face image identified by biometric authentication.

[0123] Then, the business terminal 70 displays the passenger list screen received from the management server 10 on the display device 707 (step S514).

[0124] Figure 20 shows an example of a passenger list screen displayed on the business terminal 70. Here, the boarding reservation information (name, nationality, flight number, departure city, destination, departure time, boarding gate) and facial images of multiple people detected from the captured image are displayed in a list format. Note that the data items to be displayed are not limited to these. For example, it is preferable to also display information such as membership category, priority during boarding guidance, and the pre-boarding waiting area (waiting lane) corresponding to the priority. In this case, by referring to this passenger list screen, staff member S can easily identify passengers who may be in the wrong boarding gate or waiting lane location and guide them to the correct location.

[0125] [Procedure support before boarding (2)] Figure 21 is a sequence diagram showing an example of processing by the reservation system 2, the business terminal 70, the management server 10, and the camera 80. This processing is performed at each boarding gate to notify the staff member S of the individuals eligible for priority boarding.

[0126] First, the management server 10 determines whether the current time is a predetermined time before the start time for priority boarding announcements (step S601). If the management server 10 determines that the current time is a predetermined time before the start time for announcements (step S601, YES), it acquires an image of the area surrounding the boarding gate from the camera 80 (step S602). After that, the process proceeds to step S603. On the other hand, if the management server 10 determines that the current time is not a predetermined time before the start time for announcements (step S601, NO), it waits until the current time is a predetermined time before the start time for announcements.

[0127] In step S603, the management server 10 detects people from the captured image of the area surrounding the boarding gate. If the captured image contains multiple people, the management server 10 detects the face image of each person as a target face image.

[0128] Next, the management server 10 performs a one-to-many comparison between the target face image and multiple registered face images registered in the token ID information DB 11 (step S604). Note that the registered face images to be compared are limited to those associated with token IDs whose invalid flag value is "1" (valid). Furthermore, by referring to the passage history information DB 12, only images of individuals who have passed through the departure immigration checkpoint may be used as the comparison target.

[0129] Next, the management server 10 retrieves the token ID associated with the matched registered face image in the token ID information DB 11, and uses the token ID as a key to retrieve each person's reservation number and airline code from the business information DB 13 (step S605). Then, the management server 10 sends the dataset of token ID, reservation number, and airline code to the business terminal 70 (step S606).

[0130] Next, when the business terminal 70 receives the data set for each person from the management server 10, it sequentially sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S607) and queries the boarding reservation information for each person.

[0131] When the reservation system 2 receives each person's reservation number from the business terminal 70, it transmits the corresponding boarding reservation information to the business terminal 70 (step S608).

[0132] Next, when the business terminal 70 receives the boarding reservation information for each person, it sends a dataset containing the token ID, reservation number, airline code, and boarding reservation information to the management server 10 (step S609).

[0133] Next, when the management server 10 receives the data set for each person from the business terminal 70, it determines the priority for boarding announcements based on the seat class, membership category, and accompanying person information of each person included in the data set, and extracts passengers with high priority (step S610).

[0134] Next, the management server 10 sends a priority boarding instruction screen for the extracted passengers to the business terminal 70 (step S611).

[0135] Then, the business terminal 70 displays the priority boarding instruction screen received from the management server 10 on the display device 707 (step S612).

[0136] Figure 22 shows an example of a priority boarding guidance screen displayed on the work terminal 70. The center of the screen displays a captured image. The dashed lines on the screen indicate the detection area for the parent and child passengers, who have been determined to have the highest priority for boarding guidance. In addition, the lower part of the screen displays a message instructing staff member S to provide priority boarding guidance ("Priority boarding will begin shortly. Please guide the two passengers to the boarding gate.").

[0137] [Procedure support before boarding (3)] Figure 23 is a sequence diagram showing an example of the processing performed by the reservation system 2, the business terminal 70, the management server 10, and the camera 80. This processing is performed to guide passengers to the appropriate waiting lane according to their priority before initiating priority boarding guidance.

[0138] First, the management server 10 determines whether the current time is a predetermined time before the start time for priority boarding announcements (step S701). If the management server 10 determines that the current time is a predetermined time before the start time for announcements (step S701, YES), it acquires a captured image from the camera 80 that includes the passenger boarding queue (step S702). After that, the process proceeds to step S703. On the other hand, if the management server 10 determines that the current time is not a predetermined time before the start time for announcements (step S701, NO), it waits until the current time is a predetermined time before the start time for announcements.

[0139] In step S703, the management server 10 detects all people from the captured image, including the passenger boarding queue.

[0140] Next, the management server 10 performs a one-to-many comparison of the target face image with multiple registered face images registered in the token ID information DB 11 (step S704).

[0141] Next, the management server 10 retrieves the token ID associated with the matched registered face image in the token ID information DB 11, and uses the token ID as a key to retrieve each person's reservation number and airline code from the business information DB 13 (step S705). Then, the management server 10 sends the dataset of token ID, reservation number, and airline code to the business terminal 70 (step S706).

[0142] Next, when the business terminal 70 receives the data set for each person from the management server 10, it sequentially sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S707) and queries the boarding reservation information for each person.

[0143] When the reservation system 2 receives each person's reservation number from the business terminal 70, it transmits the corresponding boarding reservation information to the business terminal 70 (step S708).

[0144] Next, when the business terminal 70 receives the boarding reservation information for each person, it sends a dataset containing the token ID, reservation number, airline code, and boarding reservation information to the management server 10 (step S709).

[0145] Next, when the management server 10 receives a data set for each person from the business terminal 70, it determines the boarding priority for each person based on the seat class, membership category, and accompanying person information included in the data set, and extracts passengers whose priority does not match the priority for the waiting lane, i.e., passengers who are lined up in different waiting lanes (step S710).

[0146] Next, the management server 10 sends a guidance screen for the waiting lane for the extracted passengers to the business terminal 70 (step S711). That is, if the management server 10 finds that among the passengers waiting at a waiting location there are passengers with a different priority than the one assigned to that waiting location, it outputs information prompting the passengers to be guided to the correct waiting location corresponding to their priority.

[0147] Then, the business terminal 70 displays the guidance and instruction screen received from the management server 10 on the display device 707 (step S712).

[0148] Figure 24 shows an example of a waiting lane guidance screen displayed on the business terminal 70. The central area of ​​the screen displays a photograph showing multiple passengers lined up in a waiting lane before boarding, located around the boarding gate. The dashed lines on the screen indicate the detection area for passenger E, who has lined up in the wrong waiting lane.

[0149] Additionally, the lower part of the screen displays an instruction message to staff member S, guiding passenger E to the correct waiting lane corresponding to their priority ("There is a passenger in the wrong waiting lane (No. 1). Please guide the passenger to the correct waiting lane (No. 3).").

[0150] [Procedure support before boarding (4)] Figure 25 is a sequence diagram showing an example of processing by the reservation system 2, the business terminal 70, the management server 10, and the camera 80. This process is performed when an employee S arbitrarily selects a person included in the captured image and queries information about the selected person.

[0151] First, the business terminal 70 sends a request to the management server 10 to display the business screen based on the input information from the staff member S (step S801). When the management server 10 receives the display request from the business terminal 70, it acquires images of the area around the boarding gate from multiple cameras 80 that are distributed around the boarding gate (step S802).

[0152] Next, when the management server 10 acquires an image from the camera 80, it sends the business screen containing the image to the business terminal 70 (step S803).

[0153] Next, the business terminal 70 displays the business screen received from the management server 10 on the display device 707 (step S804).

[0154] Next, the work terminal 70 transmits the coordinate information of the designated person within the employee S's work screen to the management server 10 (step S805). When the management server 10 receives the coordinate information of the designated person from the work terminal 70, it detects the designated person from the captured image based on the coordinate information (step S806). The facial image of the designated person extracted from the captured image becomes the target facial image.

[0155] Next, the management server 10 performs a one-to-many comparison between the target face image of the specified person and multiple registered face images registered in the token ID information DB 11 (step S807). That is, the management server 10 identifies a registered face image that matches the target face image of the passenger specified by the staff member S on the screen of the business terminal 70.

[0156] Next, the management server 10 retrieves the token ID associated with the matched registered face image in the token ID information DB 11, and uses the token ID as a key to retrieve the reservation number and airline code of the specified person from the business information DB 13 (step S808). Then, the management server 10 sends the dataset of token ID, reservation number, and airline code to the business terminal 70 (step S809).

[0157] Next, when the business terminal 70 receives the data set of the specified person from the management server 10, it sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S810) and queries for boarding reservation information.

[0158] When the reservation system 2 receives the reservation number of a specified person from the business terminal 70, it transmits the corresponding boarding reservation information to the business terminal 70 (step S811).

[0159] Next, when the business terminal 70 receives the boarding reservation information of the designated person, it sends a dataset containing the token ID, reservation number, airline code, and boarding reservation information to the management server 10 (step S812).

[0160] Next, when the management server 10 receives the data set of the specified person from the business terminal 70, it sends the passenger inquiry screen created based on the data set to the business terminal 70 (step S813). In other words, the management server 10 outputs the boarding reservation information associated with the registered facial image of the specified person to the business terminal 70.

[0161] Then, the business terminal 70 displays the passenger inquiry screen received from the management server 10 on the display device 707 (step S814).

[0162] Figure 26 shows an example of a passenger inquiry screen displayed on the business terminal 70. The left side of the screen displays the captured image, the target face image of the specified person extracted from the captured image, and registered face images that match the target face image. The dashed lines in the captured image represent the detection area of ​​the specified person T. The right side of the screen displays the boarding reservation information of the specified person in a list format as an inquiry result.

[0163] For example, when staff member S selects a person on the work screen, the management server 10 identifies the passenger based on the facial image of the selected person and displays their boarding reservation information on the screen. This allows staff member S to check the passenger's boarding reservation information included in the captured image in advance and use it for their work.

[0164] [Procedure support before boarding (5)] Figure 27 is a sequence diagram showing an example of processing by the reservation system 2, the business terminal 70, the management server 10, and the camera 80. This process is performed when an attendant S selects a desired passenger from among the passengers present around the boarding gate.

[0165] First, the business terminal 70 displays the extraction criteria input screen (step S901). Figure 28 shows an example of the extraction criteria input screen displayed on the business terminal 70. Here, the extraction criteria include boarding gate, flight number, priority, membership category, seat class, and presence or absence of token ID. For example, if staff member S wants to extract only the passenger with the highest boarding priority from among multiple passengers included in the captured image, they set the priority checkbox ("1") to ON.

[0166] Next, the business terminal 70 sends the extraction conditions entered by staff member S on the extraction conditions input screen to the management server 10 (step S902).

[0167] Next, when the management server 10 receives extraction conditions from the business terminal 70, it acquires images of the area surrounding the boarding gate from multiple cameras 80 that are distributed around the boarding gate (step S903).

[0168] Next, when the management server 10 acquires the captured image from the camera 80, it detects all people from the captured image (step S904).

[0169] Next, the management server 10 matches the target face image of each detected person with multiple registered face images registered in the token ID information DB 11 in a one-to-many relationship (step S905).

[0170] Next, the management server 10 retrieves the token ID associated with the matched registered face image in the token ID information DB 11, and uses the token ID as a key to retrieve each person's reservation number and airline code from the business information DB 13 (step S906). Then, the management server 10 sends the dataset of token ID, reservation number, and airline code to the business terminal 70 (step S907).

[0171] Next, when the business terminal 70 receives the data set for each person from the management server 10, it sequentially sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S908) and queries the boarding reservation information for each person.

[0172] When the reservation system 2 receives each person's reservation number from the business terminal 70, it transmits the corresponding boarding reservation information to the business terminal 70 (step S909).

[0173] Next, when the business terminal 70 receives the boarding reservation information for each person, it sends a dataset containing the token ID, reservation number, airline code, and boarding reservation information to the management server 10 (step S910).

[0174] Next, when the management server 10 receives the dataset of each person from the business terminal 70, it extracts passengers that match the extraction criteria (step S911).

[0175] Next, the management server 10 sends the passenger extraction result screen for the extracted passengers to the business terminal 70 (step S912).

[0176] Then, the business terminal 70 displays the passenger extraction result screen received from the management server 10 on the display device 7 Display in step S913.

[0177] Figure 29 shows an example of a passenger extraction results screen displayed on the business terminal 70. The upper area of ​​the screen displays the extraction conditions specified by the staff member S. In this case, three conditions are specified: boarding gate, flight number, and membership category. The central area of ​​the screen displays the captured image. Multiple dashed lines in the captured image represent the detection areas for passengers who meet the extraction conditions. In other words, the management server 10 can extract passengers from the captured image whose boarding reservation information corresponds to the extraction conditions entered by the staff member S.

[0178] [Boarding check at the boarding gate] Figure 30 is a sequence diagram showing an example of the processing performed by the reservation system 2, the boarding gate device 60, and the management server 10. This processing is performed when user U passes through the boarding gate.

[0179] The boarding gate device 60 continuously or periodically photographs the area in front of the device and determines whether or not the face of user U standing in front of the boarding gate device 60 is detected in the captured image (step S1001). The boarding gate device 60 waits until the face of user U is detected in the image by the biometric information acquisition device 609 (step S1001, NO).

[0180] When the boarding gate device 60 determines that the face of user U has been detected by the biometric information acquisition device 609 (step S1001, YES), it photographs the face of user U and acquires the face image of user U as the target face image (step S1002).

[0181] Next, the boarding gate device 60 transmits the target face image of user U, captured by the biometric information acquisition device 609, to the management server 10 along with a matching request (step S1003). This prompts the boarding gate device 60 to request the management server 10 to perform a one-to-many matching between the target face image of user U captured by the biometric information acquisition device 609 and multiple registered face images registered in the token ID information DB 11 of the management server 10.

[0182] When the management server 10 receives the target face image and matching request from the boarding gate device 60, it performs a matching of the user U's face image (step S1004). That is, the management server 10 performs a one-to-many matching of the target face image received from the boarding gate device 60 with multiple registered face images registered in the token ID information DB 11. Note that the registered face images to be matched are limited to images associated with token IDs whose invalid flag value is "1" (valid).

[0183] If the management server 10 determines that the matching result is a mismatch (step S1005, NO), the management server 10 sends the mismatch result to the boarding gate device 60 (step S1008), and the process proceeds to step S1009. If the management server 10 determines that the matching result is a match (step S1005, YES), the process proceeds to step S1006.

[0184] In step S1006, the management server 10 retrieves the token ID associated with the matched registered face image from the token ID information DB 11, and retrieves the reservation number and airline code from the business information DB 13 using the token ID as a key. The management server 10 then transmits the token ID, reservation number, airline code, and matching result to the boarding gate device 60 (step S1007). The process then proceeds to step S1009.

[0185] Next, if the boarding gate device 60 determines that it can perform the procedure based on the verification result (step S1009, YES), it sends the reservation number to the airline's reservation system 2 corresponding to the airline code (step S1010) and inquires about boarding reservation information.

[0186] In response, if the boarding gate device 60 determines that it cannot perform the procedure based on the verification result (step S1009, NO), it notifies user U of an error message (step S1012). For example, the boarding gate device 60 displays a notification screen on the display device 607 that includes a message such as, "Please complete the procedure at a manned counter."

[0187] When the reservation system 2 receives a reservation number from the boarding gate device 60, it transmits the corresponding boarding reservation information to the boarding gate device 60 (step S1011).

[0188] Next, when the boarding gate device 60 receives boarding reservation information from the reservation system 2, it performs a check on user U at the boarding gate based on the flight number, gate number, and boarding start time included in the boarding reservation information (step S1013), and opens gate 611 if boarding is permitted (step S1014). User U, having passed through touchpoint P5, boards the aircraft. In step S1013, if the boarding gate device 60 determines that boarding is not permitted, it is preferable to notify user U of an error message without opening gate 611. For example, if user U enters the wrong gate number for gate 611, or if the current time has not yet reached the boarding start time, gate 611 will not be opened.

[0189] Next, the boarding gate device 60 sends passage history information to the management server 10, indicating that user U has completed boarding the aircraft, after verifying the token ID, business information, and facial image (step S1015). The passage history information includes information such as the time of passage through touchpoint P5 and the device name of the terminal used.

[0190] When the management server 10 receives information from the boarding gate device 60, it registers the passage history information of user U's touchpoint P5 in the passage history information DB 12 (step S1016). In addition, the management server 10 registers the business information received from the boarding gate device 60 in the business information DB 13 as needed (step S1017).

[0191] Then, the management server 10 updates the token ID information DB 11 (step S1018). Specifically, the management server 10 updates the invalid flag in the token ID information DB 11 to the invalid value ("0"). This means that the validity period (lifecycle) of the token ID has expired.

[0192] According to this embodiment, the management server 10 stores the boarding reservation information of passengers boarding the aircraft in association with the passengers' registered facial images. The management server 10 also acquires target facial images of passengers who have not yet boarded the aircraft from images taken in the area surrounding the boarding gate, and identifies registered facial images that match the target facial images. Once the management server 10 identifies the boarding reservation information associated with the identified registered facial image, it outputs information to the work terminal 70 used by staff S to support the passenger's boarding gate procedures based on that boarding reservation information. This enables efficient procedures at the boarding gate.

[0193] The management server 10 outputs the priority level for each passenger's boarding gate procedure as information to support passenger boarding procedures at the gate. This allows staff member S to efficiently provide priority boarding services based on the priority level of each passenger obtained from the captured image, without having to check recording media such as passports or boarding passes.

[0194] <Second Embodiment> The information processing system 1 in this embodiment will be described below. Note that the same symbols used in the figures of the first embodiment indicate the same objects. The descriptions of parts common to the first embodiment will be omitted, and the differences will be described in detail.

[0195] This embodiment describes the boarding check at the boarding gate (touch point P5). This differs from the first embodiment in that it further involves verification processing regarding the passenger's companions.

[0196] [Companion check at the boarding gate] Figure 31 is a flowchart showing an example of the process for verifying the presence of a companion. This process is executed on the management server 10, for example, between steps S1011 and S1013 in Figure 30 described above.

[0197] The management server 10 determines, based on the boarding reservation information received from the reservation system 2, whether the person undergoing boarding check at the boarding gate is a passenger with an accompanying person (step S1101). If the management server 10 determines that the person is a passenger with an accompanying person (step S1101, YES), the process proceeds to step S1102.

[0198] In response, if the management server 10 determines that the person in question is a passenger without an accompanying person (step S1101, NO), the process proceeds to step S1013.

[0199] In step S1102, the management server 10 determines whether or not other people are included in the captured image. If the management server 10 determines that other people are included in the captured image (step S1102, YES), the process proceeds to step S1103.

[0200] If the management server 10 determines that no other people are included in the captured image (step S1102, NO), the process proceeds to step S1105.

[0201] In step S1103, the management server 10 determines whether the person included in the captured image is a person registered as a companion. If the management server 10 determines that the person included in the captured image is a person registered as a companion (step S1103, YES), the process proceeds to step S1013.

[0202] In response, if the management server 10 determines that the person included in the captured image is not a person registered as a companion (step S1103, NO), the process proceeds to step S1106.

[0203] In step S1105, the management server 10 outputs a confirmation message screen indicating the absence of a companion to the business terminal 70 and terminates the process.

[0204] Figure 32 shows an example of a companion verification instruction screen displayed on the business terminal 70. The left side of the screen displays a photograph taken at the boarding gate. Person T in the photograph is the passenger whose face recognition was performed. The right side of the screen displays boarding reservation information for person T in a list format. Here, it is indicated that person T's companion is a "female / infant". However, as shown by the dashed line in the photograph, child C, who is registered as person T's companion, is not included in the photograph. Therefore, the lower part of the screen displays a message instructing staff member S to verify the companion ("This is a customer with registered companion information. Please verify the companion.").

[0205] In step S1106, the management server 10 outputs a confirmation instruction screen indicating a mismatch in the accompanying person to the business terminal 70 and terminates the process.

[0206] Figure 33 shows an example of a companion verification instruction screen displayed on the business terminal 70. The left side of the screen displays a photograph taken at the boarding gate. Person T in the photograph is the passenger whose face recognition was performed. The right side of the screen displays a list of boarding reservation information for person T. Here, it is indicated that person T's companion is a "male / infant". However, child C in the photograph is different from the registered companion. Therefore, the lower part of the screen displays a message instructing staff member S to verify the companion ("The companion does not match the registered information. Please verify the companion.").

[0207] Thus, according to this embodiment, the management server 10 can analyze the images taken at the boarding gate and efficiently instruct the staff member S to verify the passenger's companions. Specifically, the management server 10 obtains the passenger's target face image from the images taken of the passenger at the boarding gate and identifies a registered face image that matches the target face image. If the companion associated with the identified registered face image is not included in the images taken at the boarding gate, the management server 10 outputs information prompting the staff member to verify the companions.

[0208] Furthermore, if the management server 10 finds that the companion associated with the identified registered facial image does not match another passenger who is included with the passenger in other captured images, it outputs information prompting the user to verify the companion.

[0209] [Third Embodiment] The information processing system 4 in this embodiment will now be described. Note that the same symbols used in the figures of the first embodiment indicate the same objects. The descriptions of parts common to the first embodiment will be omitted, and the differences will be described in detail.

[0210] Figure 34 is a block diagram showing an example of the overall configuration of the information processing system 4 in this embodiment. The information processing system 4 differs from the information processing system 1 of the first embodiment in that it further includes a function to search for the location of a designated passenger within the airport by utilizing images captured by cameras 80 placed at various locations within the airport, such as touchpoints P1 to P5, lounges (not shown), duty-free shops, and corridors.

[0211] [Searching for designated passengers within the airport] Figure 35 is a sequence diagram showing an example of processing by the business terminal 70, management server 10, and camera 80. This processing is performed when staff member S searches for the current location of a desired passenger from among the passengers present in the airport. The passengers to be searched are those for whom a token ID has already been issued.

[0212] First, the business terminal 70 displays a search condition input screen (step S1201) and then sends the search conditions entered by the staff member S on the condition input screen to the management server 10 (step S1202). On the search condition input screen, the same conditions as those on the extraction condition input screen shown in Figure 28 may be entered, or additional conditions may be entered. Examples of search conditions include reservation number, airline code, seat class, membership category, and passenger name. In this embodiment, it is assumed that this boarding reservation information is recorded as business information in the business information DB 13 when the token ID is issued. This eliminates the need to query the reservation system 2 for boarding reservation information based on the reservation number.

[0213] Next, when the management server 10 receives the search conditions from the business terminal 70, it identifies the token ID of the person to be searched in the business information DB 13 to which business information matching the search conditions is associated (step S1203).

[0214] Next, the management server 10 retrieves the registered face image of the person being searched from the token ID information DB 11 based on the identified token ID (step S1204).

[0215] Next, the management server 10 acquires images from multiple cameras 80 that are distributed throughout airport A (step S1205).

[0216] Next, when the management server 10 acquires images from the camera 80, it detects all people from each captured image (step S1206).

[0217] Next, the management server 10 matches the registered face image of the person being searched with multiple face images detected from the captured image in a one-to-N relationship (step S1207).

[0218] Next, the management server 10 identifies the current location of the person being searched based on the captured image containing the person whose face image matches the registered face image of the person being searched (step S1208), and then sends a passenger search result screen containing information about the current location of the person being searched to the business terminal 70 (step S1209).

[0219] Then, the business terminal 70 displays the passenger search result screen received from the management server 10 on the display device 7 Display in step S1210 (07).

[0220] Figure 36 shows an example of a passenger search results screen displayed on the business terminal 70. The left side of the screen displays a photograph of the location where the searched person was found, the target face image of the searched person extracted from the photograph, and registered face images that matched the target face image. The dashed lines in the photograph indicate the detection area of ​​the designated person T. In addition, information about the searched person's current location ("Terminal 1, 4F, ○○○ Lounge") is displayed above the photograph. The right side of the screen displays a list of boarding reservation information related to person T.

[0221] According to this embodiment, the management server 10 selects an image from among the images captured by multiple cameras 80 (photography devices) placed within the airport that contains a face image matching the registered face image of the passenger to be searched. The management server 10 then outputs the passenger's location information within the airport based on the selected image. In other words, the management server 10 can easily search for passengers not only in the area around the boarding gate but also in various locations within the airport, and can notify the business terminal 70 of the location information (current location) of the searched passenger. This allows staff S to share information about the desired passenger.

[0222] <Fourth Embodiment> Figure 37 is a block diagram showing an example of the overall configuration of the information processing device 100 in this embodiment. The information processing device 100 comprises an acquisition unit 100A, a identification unit 100B, and an output unit 100C. The acquisition unit 100A acquires the passenger's biometric information from a photograph taken of a passenger who is boarding an aircraft but has not yet passed through the boarding gate corresponding to the aircraft. The identification unit 100B identifies boarding reservation information related to boarding using the biometric information acquired by the acquisition unit 100A. The output unit 100C outputs information to support the passenger's procedures at the boarding gate based on the boarding reservation information identified by the identification unit 100B. According to the information processing device 100 in this embodiment, it is possible to support the passenger's procedures at the boarding gate.

[0223] <Modified Embodiment> Although the present invention has been described above with reference to embodiments, the present invention is not limited to the embodiments described above. Various modifications to the configuration and details of the present invention are possible, without departing from the spirit of the invention, as can be understood by those skilled in the art. For example, embodiments to which the present invention can be applied include embodiments to which the configuration of one embodiment is added to another embodiment, or embodiments to which the configuration of another embodiment is replaced.

[0224] Furthermore, the configuration of the present invention can be applied not only to international flights but also to domestic flights. In the case of domestic flights, in addition to the departure immigration procedures, the one-to-one matching process between the passport facial image and the captured facial image can be omitted. In this case, the registered biometric image can be, for example, the facial image captured when purchasing the boarding pass. When purchasing a boarding pass or checking in online using a terminal such as a smartphone or personal computer, if the facial image captured by the terminal is registered, boarding the aircraft at Airport A is possible through facial recognition.

[0225] In the first embodiment described above, the check-in terminal 20 read the passport facial image from the passport and applied to the management server 10 for the issuance of a token ID. However, considering cases where check-in procedures are performed online, it would be preferable for the application to be made at the automated baggage drop-off machine 30 or the security screening device 40. That is, the management server 10 obtains the passport facial image and the target biometric image from any one of the terminal devices that perform operations related to the departure of user U. The application for the issuance of a token ID may be made in the first procedure among the series of procedures performed at the time of departure.

[0226] Furthermore, in the first embodiment described above, the business information DB13 of the management server 10 stores only some items of the boarding reservation information (reservation number and airline code), unlike the reservation information DB3 of the reservation system 2. For this reason, each touchpoint terminal device (check-in terminal 20, etc.) queries the reservation system 2 for boarding reservation information based on the reservation number when executing a procedure. However, it is also possible to configure the system so that all the boarding reservation information obtained from the reservation system 2 is copied to the business information DB13 as business information during the first procedure. In this case, the terminal devices of subsequent touchpoints can obtain boarding reservation information from the management server 10 (business information DB13), and can therefore omit queries to the reservation system (airline system) 2 or change the query method as needed.

[0227] For example, the processing between the boarding gate device 60, the management server 10, and the reservation system 2 at the boarding gate (touch point P5) can be carried out using the following procedure. First, the boarding gate device 60 photographs the passenger's face and transmits the face image to the management server 10. Next, the management server 10 performs a face comparison with the registered face image registered in the token ID information DB 11 and obtains the token ID corresponding to the registered face image that matches. Next, the management server 10 uses the token ID as a key to transmit the boarding reservation information (boarding pass data) obtained from the business information DB 13 to the boarding gate device 60. Next, the boarding gate device 60 transmits the acquired boarding reservation information to the reservation system 2. Next, when the reservation system 2 receives the boarding reservation information, it compares the acquired boarding reservation information (boarding pass data) with the boarding reservation information stored in the reservation information DB 3, determines whether the passenger is allowed to board, and transmits the determination result to the boarding gate device 60. As a result, the boarding gate device 60 can control the opening of gate 611 based on the determination result received from the reservation system 2.

[0228] Furthermore, while the first embodiment described above explains the case where the output destination for information to support passenger procedures at the boarding gate is a business terminal 70, the output destination terminal device is not limited to this. For example, information may be output to a signage terminal located in the area surrounding the boarding gate or in a passageway, or to a mobile device carried by the passenger. If the contact information, such as the email address of the passenger's mobile device, is registered in a database (not shown), when it is detected that the passenger is standing in the wrong waiting lane, guidance information indicating the correct waiting lane can be output to that contact information.

[0229] The processing method of recording a program that operates the configuration of the embodiment to realize the functions of the embodiment described above on a recording medium, reading the program recorded on the recording medium as code, and executing it on a computer is also included in the scope of each embodiment. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0230] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, and non-volatile memory cards. Furthermore, the scope of each embodiment is not limited to programs that perform processing on the recording media alone, but also includes programs that operate on the OS in cooperation with other software and the functions of expansion boards to perform processing.

[0231] Some or all of the embodiments described above may also be described as follows, but are not limited to the following.

[0232] (Note 1) An acquisition unit that acquires biometric information of a passenger from a captured image of a passenger who is boarding an aircraft but has not passed through the boarding gate corresponding to the aircraft, An identification unit that identifies boarding reservation information related to the boarding using the acquired biometric information, An output unit that outputs information to support the passenger's procedures at the boarding gate based on the identified boarding reservation information, An information processing device equipped with the following features.

[0233] (Note 2) The system further includes a control unit that issues a token ID corresponding to the passenger's registered biometric information for each passenger, and that pre-associates the registered biometric information and the boarding reservation information via the token ID. The identification unit identifies the passenger's boarding reservation information based on the token ID corresponding to the registered biometric information that matches the biometric information. The information processing device described in Appendix 1.

[0234] (Note 3) The aforementioned information includes the priority of the passenger in the procedures performed at the boarding gate. The information processing device described in Appendix 1 or 2.

[0235] (Note 4) The aforementioned boarding reservation information includes the class of seat on the aircraft or the passenger classification set by the airline. The output unit outputs the priority based on the grade or classification. The information processing device described in Appendix 3.

[0236] (Note 5) The output unit outputs the waiting location before boarding corresponding to the priority. The information processing device described in Appendix 4.

[0237] (Note 6) The output unit outputs information prompting the passenger to be guided to another waiting location corresponding to their priority if the passenger waiting at the waiting location has a priority different from that of the waiting location. The information processing device described in Appendix 5.

[0238] (Note 7) The aforementioned boarding reservation information further includes registration information regarding a designated accompanying person, The output unit outputs the priority based on the presence or absence of the accompanying person. An information processing device as described in any of the appendices 3 to 6.

[0239] (Note 8) The acquisition unit acquires other biometric information of the passenger from other images taken of the passenger at the boarding gate. The identification unit identifies the passenger's boarding reservation information using the other biometric information, The output unit outputs the information prompting the verification of the companion if the companion recorded in the identified boarding reservation information is not included in the other captured images. The information processing device described in Appendix 7.

[0240] (Note 9) The acquisition unit acquires other biometric information of the passenger from other images taken of the passenger at the boarding gate. The identification unit identifies the passenger's boarding reservation information using the other biometric information, The output unit outputs information prompting the passenger to verify the identity of the companion if the companion recorded in the identified boarding reservation information does not match the person detected together with the passenger in the other captured image. The information processing device described in Appendix 7.

[0241] (Note 10) The identification unit identifies the passenger's boarding reservation information using all of the passenger's biometric information included in the captured image. The output unit outputs a list screen containing the identified boarding reservation information. An information processing device as described in any of the appendices 1 to 9.

[0242] (Note 11) The identification unit identifies the boarding reservation information using the passenger's biometric information specified on the screen displaying the captured image. The output unit outputs the identified boarding reservation information to the screen. An information processing device as described in any of the appendices 1 to 10.

[0243] (Note 12) An input unit for inputting the passenger extraction criteria, An extraction unit that extracts passengers from among the passengers included in the captured image that correspond to the boarding reservation information that satisfies the extraction conditions, An information processing device as described in any of Appendix 1 to 11, further comprising:

[0244] (Note 13) The system further includes a selection unit that selects from images taken by a camera positioned within the airport that contain the biometric information of the passenger being searched. The output unit outputs the passenger's location information within the airport based on the selected image. An information processing device as described in any of the appendices 1 to 12.

[0245] (Appendix 14) A step of obtaining biometric information of a passenger boarding an aircraft from a captured image of the passenger who has not passed through the boarding gate corresponding to the aircraft, A step of identifying boarding reservation information related to the boarding using the obtained biometric information, and , A step of outputting information for assisting the procedures of the passenger at the boarding gate based on the identified boarding reservation information, and An information processing method comprising the above steps.

[0246] (Appendix 15) A recording medium on which a program for causing a computer to perform a step of obtaining biometric information of a passenger boarding an aircraft from a captured image of the passenger who has not passed through the boarding gate corresponding to the aircraft, a step of identifying boarding reservation information related to the boarding using the obtained biometric information, and , a step of outputting information for assisting the procedures of the passenger at the boarding gate based on the identified boarding reservation information, and is recorded.

Explanation of Signs

[0247] NW1, NW2... Networks 1, 4... Information processing systems 2... Reservation systems 3... Reservation information DB 10... Management servers 11... Token ID information DB 12... Passage history information DB 13... Business information DB 20... Check-in terminals 30... Automated baggage deposit machines 40... Security inspection devices 50... Automated gate devices 60... Boarding gate devices 70... Business terminals 80... Camera 100... Information Processing Device 100A... Acquisition section 100B...Specific part 100C...Output section

Claims

1. An acquisition unit that detects a passenger from a captured image of a passenger boarding an aircraft who has not passed through the boarding gate corresponding to the aircraft, and acquires the biometric information of the detected passenger, A receiving unit that receives extraction conditions related to the passenger's boarding reservation information, An extraction unit identifies the passenger's boarding reservation information by biometric authentication using the aforementioned biometric information and pre-registered registered biometric information, and extracts the passenger whose identified boarding reservation information matches the extraction conditions. An output unit that outputs information highlighting the detected area of ​​the extracted passenger in the captured image, Equipped with, Information processing device.

2. A control unit issues a token ID corresponding to the passenger's registered biometric information for each passenger, and pre-associates the registered biometric information and the boarding reservation information via the token ID. A identification unit identifies the passenger's boarding reservation information based on the token ID corresponding to the registered biometric information that matches the biometric information, The information processing apparatus according to claim 1, further comprising:

3. The extraction criteria include priority in the procedures performed for the passenger at the boarding gate. The information processing apparatus according to claim 2.

4. The aforementioned boarding reservation information includes the class of seat on the aircraft or the passenger classification set by the airline. The aforementioned priority is determined based on the aforementioned grade or classification. The information processing apparatus according to claim 3.

5. The output unit outputs the waiting location before boarding corresponding to the priority. The information processing apparatus according to claim 4.

6. The output unit outputs information prompting the passenger to be directed to another waiting location corresponding to their priority if the passenger waiting at the waiting location has a priority different from that of the waiting location. The information processing apparatus according to claim 5.

7. The aforementioned boarding reservation information further includes registration information regarding a designated accompanying person, The aforementioned priority is determined based on the presence or absence of the accompanying person. The information processing apparatus according to any one of claims 3 to 6.

8. The acquisition unit acquires other biometric information of the passenger from other images taken of the passenger at the boarding gate. The identification unit identifies the passenger's boarding reservation information using the other biometric information, The output unit outputs information prompting the verification of the companion if the companion recorded in the identified boarding reservation information is not included in the other captured images. The information processing apparatus according to claim 7.

9. The acquisition unit acquires other biometric information of the passenger from other images taken of the passenger at the boarding gate. The identification unit identifies the passenger's boarding reservation information using the other biometric information, The output unit outputs information prompting the passenger to verify the identity of the companion if the companion recorded in the identified boarding reservation information does not match the person detected together with the passenger in the other captured image. The information processing apparatus according to claim 7.

10. The identification unit identifies the passenger's boarding reservation information using all of the passenger's biometric information included in the captured image. The output unit outputs a list screen containing the identified boarding reservation information. The information processing apparatus according to any one of claims 2 to 9.

11. The system further includes a selection unit that selects from images taken by a camera positioned within the airport that contain the biometric information of the passenger being searched. The output unit outputs the passenger's location information within the airport based on the selected image. The information processing apparatus according to any one of claims 1 to 10.

12. Computers The system detects passengers from captured images of passengers boarding an aircraft who have not yet passed through the boarding gate corresponding to the aircraft, and acquires the biometric information of the detected passengers. Upon receiving the extraction criteria for the passenger's boarding reservation information, The passenger's boarding reservation information is identified by biometric authentication using the aforementioned biometric information and pre-registered registered biometric information, and the passenger whose identified boarding reservation information matches the extraction conditions is extracted. The system outputs information that highlights the detected area of ​​the extracted passenger in the captured image. Information processing methods.

13. On the computer, The steps include: detecting a passenger from a photograph taken of a passenger boarding an aircraft who has not passed through the boarding gate corresponding to the aircraft, and obtaining the biometric information of the detected passenger; The steps include receiving extraction criteria related to the passenger's boarding reservation information, The steps include: identifying the passenger's boarding reservation information by biometric authentication using the aforementioned biometric information and pre-registered biometric information, and extracting the passenger whose identified boarding reservation information matches the extraction criteria; The steps include outputting information that highlights the detected area of ​​the extracted passenger in the captured image, A program that executes something.

Images