Gate apparatus, control method of gate apparatus, and recording medium

Abstract

To provide a gate device capable of shortening the time required for a user to pass, a control method for the gate device, and a recording medium.SOLUTION: The gate apparatus includes an exit gate door, a first biometric information acquisition unit configured to acquire first target biometric information to be compared with preregistered registered biometric information from a user moving toward the exit gate door in a closed state, a second biometric information acquisition unit configured to acquire second target biometric information to be compared with registered biometric information from a user staying in front of the exit gate door when the first target biometric information and the registered biometric information do not match each other or cannot be compared with each other, and a control unit configured to: And a door control part for opening the exit gate door in a closed state.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to a gate device, a control method for the gate device, and a recording medium. [Background technology]

[0002] Patent Document 1 describes a gate device that is applied to a gate system that automatically performs face recognition work carried out during immigration inspections. The gate device described in Patent Document 1 has a face recognition device that compares and collates a face image acquired from a camera with a face image acquired from a passport reader. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 061812 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the gate device described in Patent Document 1, the camera captures the face of a person who places their passport on a passport reader in front of a facial authentication device, making it difficult for the person to pass through the gate device in a short time.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a gate device, a control method for the gate device, and a recording medium that can reduce the time required for a user to pass through the gate device. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided a gate device comprising: an exit gate door; a first biometric information acquisition unit that acquires first target biometric information from a user moving toward the exit gate door in a closed state, the first target biometric information being compared with pre-registered registered biometric information; a second biometric information acquisition unit that acquires second target biometric information from the user remaining in front of the exit gate door in the event that the first target biometric information does not match the registered biometric information or the matching is not possible; and a door control unit that opens the exit gate door in the closed state depending on the result of the matching between the first target biometric information or the second target biometric information and the registered biometric information.

[0007] According to another aspect of the present invention, there is provided a gate device comprising: an exit gate door; a first biometric information acquisition unit that acquires first target biometric information from a user moving toward the exit gate door in a closed state, the first target biometric information being compared with pre-registered registered biometric information; a second biometric information acquisition unit that acquires second target biometric information from a user remaining in front of the exit gate door in a case where the face of the user moving toward the exit gate door in the closed state is not facing the exit gate door or the user is wearing an attachment on their face, the second target biometric information being compared with the registered biometric information; and a door control unit that opens the exit gate door in the closed state depending on the result of comparing the first target biometric information or the second target biometric information with the registered biometric information.

[0008] According to yet another aspect of the present invention, there is provided a gate device having an exit gate door, a biometric information acquisition unit that acquires target biometric information from a user who moves toward or remains in front of the exit gate door in a closed state, and compares the target biometric information with pre-registered registered biometric information, and a door control unit that opens the exit gate door in the closed state depending on the determination result of declared information associated with the registered biometric information that matches the target biometric information.

[0009] According to yet another aspect of the present invention, there is provided a control method for a gate device having an exit gate door, the control method comprising: acquiring first target biometric information from a user moving toward the exit gate door in a closed state, which is to be matched with pre-registered registered biometric information; if there is a mismatch between the first target biometric information and the registered biometric information or if the match is not possible, acquiring second target biometric information from the user remaining in front of the exit gate door, which is to be matched with the registered biometric information; and opening the exit gate door in the closed state depending on the result of matching the first target biometric information or the second target biometric information with the registered biometric information.

[0010] According to yet another aspect of the present invention, there is provided a program that causes a gate device having an exit gate door to acquire first target biometric information from a user moving toward the exit gate door in a closed state, to be matched with pre-registered registered biometric information, and if there is a mismatch between the first target biometric information and the registered biometric information or if the match is not possible, to acquire second target biometric information from the user remaining in front of the exit gate door, to be matched with the registered biometric information, and to open the exit gate door in the closed state depending on the result of matching the first target biometric information or the second target biometric information with the registered biometric information. [Effects of the Invention]

[0011] According to the present invention, the time required for a user to pass through a gate device can be reduced. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram showing the overall configuration of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing a customs inspection station where a kiosk terminal and an electronic gate according to an embodiment of the present invention are installed. [Figure 3] FIG. 3 is a block diagram showing an example of the hardware configuration of a mobile terminal according to an embodiment of the present invention. [Figure 4A] FIG. 4A is a schematic diagram showing the appearance of a kiosk terminal according to one embodiment of the present invention. [Figure 4B] FIG. 4B is a schematic diagram showing the appearance of a kiosk terminal according to one embodiment of the present invention. [Figure 5] FIG. 5 is a block diagram showing an example of the hardware configuration of a kiosk terminal according to an embodiment of the present invention. [Figure 6A] FIG. 6A is a schematic diagram showing a configuration in which one kiosk terminal according to an embodiment of the present invention is installed. [Figure 6B] FIG. 6B is a schematic diagram showing a configuration in which one kiosk terminal according to an embodiment of the present invention is installed. [Figure 7A] FIG. 7A is a schematic diagram showing a configuration in which two kiosk terminals according to an embodiment of the present invention are installed. [Figure 7B] FIG. 7B is a schematic diagram showing a configuration in which two kiosk terminals according to an embodiment of the present invention are installed. [Figure 8A] FIG. 8A is a schematic diagram showing a configuration in which three kiosk terminals according to an embodiment of the present invention are installed. [Figure 8B] FIG. 8B is a schematic diagram showing a configuration in which three kiosk terminals according to an embodiment of the present invention are installed. [Figure 9] FIG. 9 is a schematic diagram illustrating prevention of peeping in a kiosk terminal according to an embodiment of the present invention. [Figure 10] FIG. 10 is a schematic diagram showing a configuration in which kiosk terminals according to an embodiment of the present invention are installed at different heights. [Figure 11] FIG. 11 is a schematic diagram showing the appearance of an entrance gate terminal and an exit gate terminal that constitute an electronic gate according to one embodiment of the present invention. [Figure 12] FIG. 12 is a block diagram showing an example of the hardware configuration of an entrance gate terminal that constitutes an electronic gate according to one embodiment of the present invention. [Figure 13]FIG. 13 is a block diagram showing an example of the hardware configuration of an exit gate terminal that constitutes an electronic gate according to one embodiment of the present invention. [Figure 14] FIG. 14 is a block diagram showing an example of the hardware configuration of a management server according to an embodiment of the present invention. [Figure 15A] FIG. 15A is a schematic diagram showing an example of a declaration information input screen on a mobile terminal according to an embodiment of the present invention. [Figure 15B] FIG. 15B is a schematic diagram showing an example of a reporting code display screen on the mobile terminal according to one embodiment of the present invention. [Figure 16] FIG. 16 is a sequence diagram showing the operations of the kiosk terminal and the management server in the information processing system according to one embodiment of the present invention. [Figure 17] FIG. 17 is a sequence diagram showing the language setting operation of the kiosk terminal in the information processing system according to one embodiment of the present invention. [Figure 18A] FIG. 18A is a schematic diagram showing an example of a reception screen in a kiosk terminal according to an embodiment of the present invention. [Figure 18B] FIG. 18B is a schematic diagram showing an example of a declaration content confirmation screen on a kiosk terminal according to an embodiment of the present invention. [Figure 18C] FIG. 18C is a schematic diagram illustrating an example of a guidance screen in a kiosk terminal according to an embodiment of the present invention. [Figure 19] FIG. 19 is a sequence diagram showing the operations of the entrance gate terminal, the exit gate terminal, and the management server in the information processing system according to one embodiment of the present invention. [Figure 20] FIG. 20 is a schematic diagram showing an example of a notification screen at the exit gate terminal according to one embodiment of the present invention. [Figure 21] FIG. 21 is a block diagram showing the configuration of an information processing device according to another embodiment of the present invention. [Figure 22] FIG. 22 is a block diagram showing the configuration of an information processing device according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] [One embodiment] An information processing device, a gate device, an information processing method, and a gate device control method according to an embodiment of the present invention will be described with reference to FIGS.

[0014] First, the overall configuration of the information processing system according to this embodiment will be described with reference to Figures 1 and 2. Figure 1 is a schematic diagram showing the overall configuration of the information processing system according to this embodiment. Figure 2 is a schematic diagram showing a customs inspection site where the kiosk terminal and electronic gates according to this embodiment are installed.

[0015] 1, an information processing system 1 according to this embodiment includes a kiosk terminal 10, an electronic gate 20, a management server 50, and a mobile terminal 60. The electronic gate 20 has an entrance gate terminal 30 and an exit gate terminal 40. The information processing system 1 is a system that accepts declarations for customs procedures at the time of entry of users U, such as passengers and crew members entering from a foreign country at an airport, seaport, etc., and controls the electronic gate 20 through which specific users U pass after completing the declaration.

[0016] For example, if the information processing system 1 is installed at an airport in Japan, the user U is a passenger or crew member who arrives at the airport by airplane from a foreign country, a Japanese national returning to Japan from a foreign country from which they departed, a foreign national entering Japan from a foreign country, etc. A customs declaration, which is a declaration in customs procedures accepted by the information processing system 1, is, more specifically, a declaration in baggage clearance, for example, and in the case of Japan, is a declaration with the same content as a declaration using Customs Form C-5360, Declaration of Personal Baggage and Unaccompanied Baggage.

[0017] The management server 50 is installed, for example, in a customs facility. The kiosk terminal 10 and the electronic gate 20 are installed, for example, at a customs inspection station C where customs inspection is carried out at an airport, a seaport, or the like.

[0018] As shown in FIG. 2, a kiosk terminal 10 is installed near the entrance to a customs inspection area C where a user U enters. A digital signage terminal 70 is installed near the kiosk terminal 10, providing guidance on how to proceed at the kiosk terminal 10 by displaying moving or still images. Furthermore, an electronic gate 20 and a manned booth M are installed at the exit of the customs inspection area C. A digital signage terminal 80 is installed near the entrance gate terminal 30 of the electronic gate 20, providing guidance by displaying moving or still images about things to note when passing through the electronic gate 20. At the manned booth M, a customs officer S performs face-to-face customs inspections, such as accepting declarations for customs procedures and inspecting baggage.

[0019] The mobile terminal 60 is a terminal that is owned or carried by the user U. A customs declaration application for making a customs declaration at the kiosk terminal 10 is installed in the mobile terminal 60. As will be described later, the user U can make a customs declaration at the kiosk terminal 10 by using a declaration code generated by the customs declaration application on the mobile terminal 60.

[0020] The kiosk terminal 10, the entrance gate terminal 30, the exit gate terminal 40, and the management server 50 are connected to a network NW. The network NW is composed of a WAN (Wide Area Network), a LAN (Local Area Network), a mobile communication network, etc. The kiosk terminal 10 and the management server 50 are capable of communicating via the network NW. The entrance gate terminal 30 and the exit gate terminal 40 are capable of communicating via the network NW or without the network NW. The exit gate terminal 40 and the management server 50 are capable of communicating via the network NW. The mobile terminal 60 is capable of communicating with servers such as the management server 50 via the network NW.

[0021] Next, each component of the information processing system 1 according to this embodiment will be further described with reference to FIGS.

[0022] First, the configuration of the mobile terminal 60 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the hardware configuration of the mobile terminal 60.

[0023] The mobile terminal 60 is a terminal that is owned or carried by a user U and is, for example, a mobile information device such as a smartphone, a tablet terminal, or a mobile phone. As shown in Fig. 3 , the mobile terminal 60 has a CPU (Central Processing Unit) 602, a RAM (Random Access Memory) 604, a storage device 606, an input device 608, a display 610, a mobile wireless communication unit 612, and a short-range wireless communication unit 614. The CPU 602, RAM 604, storage device 606, input device 608, display 610, mobile wireless communication unit 612, and short-range wireless communication unit 614 are connected to a bus line 616.

[0024] The CPU 602 operates by executing a program stored in the storage device 606, and functions as a control unit that controls the overall operation of the mobile terminal 60. The CPU 602 also executes an application program stored in the storage device 606 to perform various processes as the mobile terminal 60. The RAM 604 provides a memory area necessary for the operation of the CPU 602.

[0025] For example, a customs declaration application for making a customs declaration at the kiosk terminal 10 is installed in the storage device 606. By executing the customs declaration application, the CPU 602 can perform various processes for the user U to make a customs declaration.

[0026] More specifically, the CPU 602 can accept, for example, input by the user U of declaration information required for customs declaration. The declaration information is, for example, information that is the same as the items to be declared in a personal effects and unaccompanied baggage declaration form. That is, the declaration information includes, for example, information required for customs declaration, such as the user U's name, current address or place of stay in the country of entry, occupation, date of birth, passport number, flight number, place of departure, date of entry, and items to be declared in customs declaration.

[0027] Furthermore, the CPU 602 can generate a reporting code, which is a code such as a two-dimensional code including the reporting information input by the user U, as will be described later, and display the code on the display 610. The mobile terminal 60 can function as a medium for displaying the reporting code.

[0028] The declaration code is not particularly limited as long as it can include declaration information, and is, for example, a two-dimensional code such as a QR code (registered trademark). The declaration code may also be a one-dimensional code such as a barcode. The CPU 602 can also generate a declaration code with an expiration date set. The CPU 602 can display the set expiration date on the display 610 along with the declaration code. After being generated, a declaration code with an expiration date becomes invalid when the date and time set as the expiration date has passed. By setting an expiration date in the declaration code, it is possible to encourage the user U to make a customs declaration promptly after generating the declaration code.

[0029] If user U is accompanied by a family member, user U and the family member can each make a customs declaration using their own mobile terminal 60, or user U can make a customs declaration on behalf of the user. When user U makes a customs declaration on behalf of the user, the CPU 602 can generate a declaration code for the accompanying family member to use at the kiosk terminal 10, in addition to the declaration code for user U himself / herself to use at the kiosk terminal 10, and display these on the display 610. The accompanying family member can use the kiosk terminal 10 in the same way as user U, using the declaration code for the accompanying family member displayed on user U's mobile terminal 60. After using the kiosk terminal 10, the accompanying family member is guided to the electronic gate 20 or manned booth M in the same way as user U.

[0030] Furthermore, the code display medium that displays the reporting code does not necessarily have to be the mobile terminal 60, but may be other media. For example, the reporting code may be printed on paper as the code display medium.

[0031] The customs declaration application also supports multiple languages, such as Japanese, English, Chinese, and Korean. This allows the customs declaration application to be displayed and input in a language that is selected and set from among multiple languages ​​in the language settings of the mobile terminal 60, for example. The customs declaration application may also be displayed and input in a language that is selected and set from among multiple languages ​​in settings within the application, for example, separate from the language settings of the mobile terminal 60. The customs declaration application can be used in the language that is set in this way.

[0032] The CPU 602 can further include language information, which is information indicating the language used in the customs declaration application, in the declaration code. The language information is, for example, information indicating the language set in the language settings of the mobile terminal 60, or information about the language if a language is set in the settings of the customs declaration application.

[0033] The storage device 606 is configured with a storage medium such as a nonvolatile memory, a hard disk drive, etc., and functions as a storage unit. The storage device 606 stores programs executed by the CPU 602, data referenced by the CPU 602 when executing the programs, etc. The storage device 606 stores a customs declaration application as an application program executed by the CPU 602.

[0034] The storage device 606 can store information such as declaration information entered in the customs declaration application, and can also store a declaration code generated by the customs declaration application.

[0035] The input device 608 is, for example, a touch panel incorporated in the display 610. The input device 608 functions as an input unit that accepts input from the user U. The user U can input various information and instructions to execute processing to the mobile terminal 60 via the input device 608. For example, the user U can input declaration information and instructions to generate a declaration code to the mobile terminal 60 that runs a customs declaration application via the input device 608, which is a touch panel.

[0036] The display 610 functions as a display unit that displays various screens to the user U. For example, in the mobile terminal 60 that executes a customs declaration application, the display 610 displays a declaration information input screen that accepts input of declaration information, a declaration code display screen that displays a declaration code, and the like, as will be described later.

[0037] The mobile wireless communication unit 612 connects to the network NW via a mobile communication network under the control of the CPU 602. The communication method of the mobile wireless communication unit 612 is not particularly limited, but examples thereof include a third generation mobile communication method, an LTE (Long Term Evolution) method, and a fourth generation mobile communication method.

[0038] The short-range wireless communication unit 614 performs wireless communication with access points, external devices, etc. under the control of the CPU 602, and connects to the network NW via an access point, for example, on the premises of an airport, etc. The communication method of the short-range wireless communication unit 614 is not particularly limited, and examples thereof include a wireless LAN method such as Wi-Fi (registered trademark), Bluetooth (registered trademark) communication method, NFC (Near Field Communication) method, and infrared communication method.

[0039] In this way, the mobile terminal 60 is configured.

[0040] Next, the configuration of the kiosk terminal 10 will be described with reference to FIGS. 4A to 10. FIGS. 4A and 4B are schematic diagrams showing the exterior of the kiosk terminal 10. FIG. 4A is a perspective view showing the entire kiosk terminal 10 installed on a pole 100, and FIG. 4B is an enlarged perspective view of the kiosk terminal 10 installed on the pole 100. FIG. 5 is a block diagram showing an example of the hardware configuration of the kiosk terminal 10. FIGS. 6A and 6B are schematic diagrams showing the configuration when one kiosk terminal 10 is installed. FIGS. 7A and 7B are schematic diagrams showing the configuration when two kiosks 10 are installed. FIGS. 8A and 8B are schematic diagrams showing the configuration when three kiosks 10 are installed. FIG. 9 is a schematic diagram explaining how the kiosk terminal 10 prevents peeping. FIG. 10 is a schematic diagram showing the configuration when the kiosk terminals 10 are installed at different heights.

[0041] The kiosk terminal 10 is an information processing device that accepts a customs declaration using a declaration code displayed on the mobile terminal 60 of the user U, and guides the user U to the electronic gate 20 or a manned booth M depending on the content of the customs declaration. As shown in FIGS. 4A and 4B , the kiosk terminal 10 is installed by being attached to the side of a pole 100, which is a support body installed with its longitudinal direction vertical. Note that one or more kiosk terminals 10 can be installed on one pole 100, as will be described later.

[0042] 5, the kiosk terminal 10 includes a CPU 102, a RAM 104, a storage device 106, an input device 108, and a display 110. The kiosk terminal 10 also includes a passport reader 112, a code reader 114, a camera 116, a depth camera 118, a communication unit 120, and a status lamp 122. The CPU 102, RAM 104, storage device 106, input device 108, display 110, passport reader 112, code reader 114, camera 116, depth camera 118, communication unit 120, and status lamp 122 are connected to a bus line 124.

[0043] 4A and 4B, the kiosk terminal 10 has a housing 126 attached to the pole 100. The housing 126 has a first housing section 126a in which the display 110 and the camera 116 are provided, and a second housing section 126b in which the passport reader 112 and the code reader 114 are provided. The housing 126 also houses the CPU 102, RAM 104, storage device 106, and communication section 120.

[0044] The first housing unit 126a has a display surface on which the display 110 is provided. The second housing unit 126b has a reading surface on which the passport reader 112 and the code reader 114 are provided. The second housing unit 126b is integrally connected to the lower part of the first housing unit 126a so that the display surface and the reading surface form an obtuse angle with each other. The first housing unit 126a and the second housing unit 126b are configured so that when the kiosk terminal 10 is installed on the pole 100, the display surface of the first housing unit 126a faces diagonally upward and forward, opposite to the rear on the pole 100 side, and the reading surface of the second housing unit 126b is approximately horizontal.

[0045] The kiosk terminal 10 is installed on a pole 100 at a height where a user U who is taller than a certain height can look down on the display 110, passport reader 112, and code reader 114.

[0046] A display 110 is provided on the display surface of the first housing 126a with its screen facing diagonally upward and forward. The display 110 incorporates an input device 108 as a touch panel.

[0047] Furthermore, a camera 116 is provided in the center of the portion between the display 110 and the second housing 126b below the display surface of the first housing 126a. The camera 116 is provided between the display 110, which is a display unit provided in the first housing 126a, and the passport reader 112 and the code reader 114, which are information acquisition units provided in the second housing 126b. Note that one camera 116 may be provided, or two cameras 116 may be provided above and below the display 110. In this case, the lower camera 116 is provided, for example, in the center of the portion between the display 110 and the second housing 126b below the display surface of the first housing 126a, as described above. Meanwhile, the upper camera 116 is provided, for example, in the center of the portion above the display 110 above the display surface of the first housing 126a.

[0048] The camera 116 is installed so that its shooting direction faces diagonally upward and forward. A user U in front of the kiosk terminal 10 will operate the kiosk terminal 10 while directing his or her gaze diagonally downward and looking down at the display 110, passport reader 112, or code reader 114. The camera 116 has an angle of view that allows it to capture an image of the face of the user U operating the kiosk terminal 10 while looking down at the display 110, passport reader 112, or code reader 114.

[0049] Furthermore, a depth camera 118 is provided at the center of the upper part of the display surface of the first housing unit 126a. The depth camera 118 is provided so as to be able to acquire three-dimensional information of an object photographed by the camera 116. Note that the depth camera 118 may be provided adjacent to one or two cameras 116 provided as described above.

[0050] Furthermore, plate-shaped visors 128 are provided on both sides of the display surface of the first housing portion 126a so as to protrude from the display surface along the display 110. The visors 128 function as shields that block the view of the display 110 from the side, diagonally behind, and diagonally in front of the kiosk terminal 10, making it difficult for anyone other than the user U using the kiosk terminal 10 to peek at the display 110.

[0051] Furthermore, on both sides of the display surface of the first housing 126a, there are provided lights 130, which are lighting units for illuminating the face of a user U operating the kiosk terminal 10. The lights 130 may be configured, for example, as a light source that illuminates the display 110 from below to above, or as a rod-shaped light source provided along the display 110. The light source is not particularly limited, but may be an LED (Light Emitting Diode), for example. The lights 130 provided on both sides of the display 110 ensure sufficient brightness when the camera 116 captures the face of a user U operating the kiosk terminal 10. The lights 130 may be constantly on while the kiosk terminal 10 is in operation. Furthermore, the lights 130 do not necessarily need to be constantly on while the kiosk terminal 10 is in operation, and may be configured to be turned on when the camera 116 captures the face of the user U. In this case, the light 130 can be configured to turn on, for example, when the passport is read by the passport reader 112 or the declaration code is read by the code reader 114. The light 130 can also be configured to turn on, for example, when the user U is detected by a human presence sensor (not shown) provided in the kiosk terminal 10. The light 130 that turns on at these predetermined times can also be configured to turn off, for example, after the camera 116 has finished photographing the face of the user U or after a predetermined time has passed.

[0052] On the reading surface of the second housing unit 126b, a passport reader 112 is provided on the left side as one faces the display 110, and a code reader 114 is provided on the right side as one faces the display 110. In other kiosk terminals installed at airports and the like, a reader for reading codes such as QR codes is generally provided on the right side as one faces the terminal. In the kiosk terminal 10 as well, the code reader 114 is provided on the right side as one faces the display 110, so that the user U can smoothly hold the declaration code displayed on the portable terminal 60 over the code reader 114 to have it read.

[0053] The passport reader 112 is configured, for example, with the reader unit for reading the passport facing upward, and has a slot above the reader unit into which the passport can be inserted. In this case, the user U can have the passport read by the passport reader 112 by inserting the passport with the personal information page open into the slot with the page facing downwards and holding it over the reader unit of the passport reader 112. With the method of inserting the passport into the slot, once the passport is inserted into the slot, the user U does not need to continue holding the passport to keep the pages open, and the passport can be accurately positioned with respect to the reader unit of the passport reader 112. This allows for smooth passport reading.

[0054] The configuration of the passport reading device 112 is not limited to the above configuration, and various configurations can be adopted. For example, the passport reading device 112 can be configured so that the user U holds the passport with the personal information page open over the reading unit of the passport reading device 112. In this case, the user U can have the passport read by the passport reading device 112 by holding the passport with the personal information page open over the reading unit of the passport reading device 112 with the page facing downwards.

[0055] Also, the code reader 114 is provided with, for example, a reading unit for reading the declaration code facing upward. In this case, the user U can have the code reader 114 read the declaration code by holding the display 610 of the mobile terminal 60, on which the declaration code is displayed, facing downward over the reading unit of the code reader 114.

[0056] The configuration of the code reader 114 is not limited to the above configuration, and various configurations can be adopted. For example, the code reader 114 can be configured so that its reading unit reads the declaration code from above. In this case, the user U can have the code reader 114 read the declaration code by holding the display 610 of the mobile terminal 60, on which the declaration code is displayed, facing upward over the reading unit of the code reader 114.

[0057] A status lamp 122 is provided on the top of the pole 100 so as to be positioned above the first housing 126a including the display 110. The status lamp 122 is provided at the tip of an arm 132 attached to the top of the pole 100 so as to be positioned corresponding to above the kiosk terminal 10 whose status the status lamp 122 indicates. The arm 132 protrudes from the pole 100 above the kiosk terminal 10.

[0058] The arm 132 is also provided with a surveillance camera 134 that monitors the status of the kiosk terminal 10. More specifically, the surveillance camera 134, which functions as a monitoring unit, captures and monitors the kiosk terminal 10 and its surroundings, the status of the user U operating the kiosk terminal 10, etc. The surveillance camera 134 is connected to the corresponding kiosk terminal 10 or to the network NW. The surveillance camera 134 can transmit the captured video of the status of the user U to the management server 50 via the network NW, either via the kiosk terminal 10 or without via the kiosk terminal 10.

[0059] The CPU 102 operates by executing a program stored in the storage device 106, and functions as a control unit that controls the overall operation of the kiosk terminal 10. The CPU 102 also executes application programs stored in the storage device 106 to perform various processes as the kiosk terminal 10. The RAM 104 provides a memory area necessary for the operation of the CPU 102.

[0060] More specifically, the CPU 102 functions as a matching request unit that requests the management server 50 to match the face of user U who is making a customs declaration using the kiosk terminal 10. As a matching request unit, the CPU 102 requests a match between the captured face image of user U and the face image on user U's passport to match the face of user U. The captured face image is a face image of user U captured by the camera 116 when using the kiosk terminal 10. The passport face image is a face image acquired from user U's passport by the passport reader 112. To request the match, the CPU 102 transmits the captured face image and the passport face image or facial features extracted from them to the management server 50 along with a matching request. The CPU 102 reads at least one of the passport and the declaration code to obtain the captured face image of user U, and can also read the other of the passport and the declaration code to request the management server 50 to match the face of user U. The timing at which the CPU 102 acquires a photographed face image and the timing at which the CPU 102 requests face matching are not limited to those described above, and various variations are possible for each.

[0061] The CPU 102 can estimate the attachments in the captured face image while the camera 116 is capturing an image. The CPU 102 can detect face attachments such as masks, sunglasses, glasses, and hats in the captured face image through attachment estimation. When the CPU 102 detects a face attachment worn on the face of the user U, it can warn the user U to remove the face attachment that is interfering with face matching. The CPU 102 can issue a warning by, for example, displaying a message on the display 110 instructing the user U to remove the face attachment, or by emitting a sound from a speaker or the like instructing the user U to remove the face attachment.

[0062] The CPU 102 also functions as a matching information acquisition unit that receives and acquires matching information, which is information indicating the result of face matching requested to the management server 50, from the management server 50. The matching information indicates that the passport face image and the photographed face image match as a result of face matching, thereby successfully verifying the identity of the user U making a customs declaration at the kiosk terminal 10, or that they do not match, thereby failing to verify the identity of the user U. Note that a match between the passport face image and the photographed face image includes a case where the matching score, which indicates the similarity between the two face images, exceeds a predetermined threshold, indicating a high similarity between the two face images.

[0063] Note that, instead of functioning as a matching request unit and a matching information acquisition unit, the CPU 102 can also function as a matching unit that matches the passport face image with the photographed face image to acquire matching information for face matching of the user U. When matching the passport face image with the photographed face image, the CPU 102 can match both face images by matching the facial feature amount extracted from the passport face image with the facial feature amount extracted from the photographed face image.

[0064] The CPU 102 also functions as a user information transmission unit that transmits the user information of the user U to the management server 50. The user information of the user U includes the user U's personal information, face information, and declaration information, which are associated with each other.

[0065] The personal information includes information about the user U, such as the user U's name, date of birth, etc. The personal information also includes information about the user U's passport, such as the user U's passport number and expiration date. The personal information is obtained from the user U's passport by the passport reading device 112.

[0066] The facial information is a photographed facial image and a passport facial image of the user U. Furthermore, the facial information does not have to be the photographed facial image and the passport facial image themselves, but may be facial feature amounts extracted from the photographed facial image and the passport facial image, respectively. In this case, the CPU 102 extracts facial feature amounts from the photographed facial image and the passport facial image, respectively.

[0067] The declaration information includes information that user U must declare in a customs declaration when entering the country. The declaration information is obtained by the code reading device 114 from the declaration code displayed on user U's mobile terminal 60. As described above, the declaration information includes information necessary for a customs declaration, such as user U's name, current address or place of stay in the country of entry, occupation, date of birth, passport number, flight number, place of departure, date of entry, and items to be declared in a customs declaration.

[0068] The user information of user U transmitted to the management server 50 is registered in a user information database (DB) 506a in the management server 50, as will be described later. The photographed face image or passport face image included in the user information is registered in the user information DB 506a as a registered face image, which is pre-registered registered biometric information.

[0069] The CPU 102 also functions as a determination request unit that requests the management server 50 to determine the declaration contents included in the declaration information of a user U who makes a customs declaration using the kiosk terminal 10. The CPU 102 as the determination request unit requests the management server 50 to determine the declaration contents included in the declaration information transmitted to the management server 50. The CPU 102 receives determination information from the management server 50, which is information indicating the determination result of the declaration contents included in the declaration information.

[0070] The CPU 102 also functions as a determination information acquisition unit that receives and acquires, from the management server 50, determination information that indicates the determination result of the declaration content requested from the management server 50. The determination information indicates, as a result of the determination, whether or not the user U is subject to taxation, and whether or not the user U is at risk. Cases in which the user U is subject to taxation include, for example, when the declaration includes a declaration that the user U is carrying items that exceed the tax-free range, or when the declaration includes a declaration that the user U is carrying merchandise cargo or merchandise samples. Cases in which the user U is at risk include, for example, when the declaration includes a declaration that the user U is carrying items that are prohibited or restricted from being brought into the country, or when the declaration includes a declaration that the user U is carrying items entrusted to them by another person.

[0071] Note that instead of functioning as a determination request unit and a determination information acquisition unit, the CPU 102 can also function as a determination unit that determines the declaration content included in the declaration information of the user U. In this case, the CPU 102 can determine the declaration content included in the declaration information in the same way as the management server 50.

[0072] Furthermore, the CPU 102 functions as a determination unit that determines whether or not the user U can use the electronic gate 20 based on the matching information and determination information received from the management server 50. As a determination unit, the CPU 102 determines that the user U can use the electronic gate 20 if the face image on the user U's passport matches the photographed face image, the user U is not subject to taxation, and there is no risk to the user U. On the other hand, the CPU 102 determines that the user U cannot use the electronic gate 20 if the face image on the user U's passport does not match the photographed face image, the user U is subject to taxation, or there is a risk to the user U.

[0073] Furthermore, the CPU 102 can transmit gate availability information, which is information indicating the result of a determination as to whether the electronic gate 20 can be used, to the management server 50. In this case, the gate availability information transmitted to the management server 50 is associated with the user information and registered in the user information DB 506a in the management server 50, as will be described later. If the passport facial image of the user U matches the photographed facial image, and the user U is not subject to taxation and there is no risk to the user U, the gate availability information indicates that the user U can use the electronic gate 20.

[0074] The CPU 102 also functions as a display control unit that causes the display 110 to display screens for guidance, notifications, etc. for the user U who is making a customs declaration at the kiosk terminal 10. For example, the CPU 102 as a display control unit causes the display 110 to display a guidance screen that provides guidance on how to use the kiosk terminal 10. The CPU 102 also causes the display 110 to display a declaration content confirmation screen that allows the user U to check the declaration content included in the declaration information acquired from the mobile terminal 60 of the user U and to correct it as necessary.

[0075] Furthermore, the CPU 102 functions as a processing unit that performs processing for the user U who has made a customs declaration at the kiosk terminal 10, depending on the determination result as to whether the electronic gate 20 can be used. More specifically, the CPU 102 displays a guidance screen on the display 110 that guides the user U to the electronic gate 20 or the manned booth M, depending on the determination result as to whether the electronic gate 20 can be used, and guides the user U to the electronic gate 20 or the manned booth M.

[0076] The CPU 102 also functions as a language setting unit that sets the language used by the kiosk terminal 10 based on language information read from the declaration code displayed on the mobile terminal 60 of the user U who is making a customs declaration at the kiosk terminal 10. When the user U inputs a change input to change the language used by the kiosk terminal 10 to another language via the input device 108, the CPU 102 changes the language used in accordance with the change input. The language used by the kiosk terminal 10 is, for example, a language used to display various information on the display 110 and to accept input from the input device 108. Like the customs declaration application on the mobile terminal 60, the kiosk terminal 10 supports multiple languages, such as Japanese, English, Chinese, and Korean. This allows the kiosk terminal 10 to display and input in a language selected from multiple languages.

[0077] The storage device 106 is configured with a storage medium such as a nonvolatile memory, a hard disk drive, etc., and functions as a storage unit. The storage device 106 stores programs executed by the CPU 102, data referenced by the CPU 102 when the programs are executed, etc.

[0078] The input device 108 is, for example, a touch panel incorporated in the display 110. The input device 108 functions as an input unit that accepts input of instructions from the user U. The user U can input various information and instructions to execute processing to the kiosk terminal 10 via the input device 108.

[0079] The display 110 functions as a display unit that displays various screens to the user U who uses the kiosk terminal 10. For example, the display 110 displays a guide screen that provides instructions on how to use the kiosk terminal 10, a declaration content confirmation screen for the user U, a guidance screen for the user U, etc. The display 110 can also display matching information that indicates the result of matching the photographed facial image with the passport facial image, and determination information that indicates the determination result of the declaration content.

[0080] The display 110 can be installed, for example, vertically. The vertically oriented display 110 allows the kiosk terminal 10 to be installed in a small space while displaying the same amount of information as a horizontally oriented display of the same size. Furthermore, the vertically oriented display 110 prevents the user U standing in front of the display 110 from peeking at the display 110 from behind the user U.

[0081] The passport reading device 112 functions as a reading unit that reads the passport of the user U and acquires information recorded on the passport. The passport reading device 112 is configured, for example, with an image scanner, a contactless IC (Integrated Circuit) reader, an OCR (Optical Character Reader) device, etc. The passport reading device 112 reads the passport, which is a medium held over the reading unit, and acquires information from the passport.

[0082] For example, the passport reading device 112 reads and acquires the identity information of user U displayed on the face of the passport using an OCR device. Also, for example, the passport reading device 112 reads and acquires the passport face image of user U displayed on the face of the passport using an image scanner. Also, in the case of an ePassport, the passport reading device 112 reads and acquires the identity information, passport face image, etc. of user U recorded in the IC chip of the ePassport using a contactless IC reader. Note that the biometric information of user U recorded and included in the passport is not limited to a facial image, and may be other biometric information such as an iris image. The passport reading device 112 can acquire the biometric information of user U included in the passport.

[0083] The code reading device 114 functions as a reading unit that reads the declaration code displayed on the mobile terminal 60 of the user U and acquires the declaration information and language information contained in the declaration code. The code reading device 114 is a code reader that corresponds to the type of declaration code to be read, such as a QR code reader or a barcode reader. The code reading device 114 reads the declaration code displayed on the mobile terminal 60, which is a code display medium, held over the reading unit, and acquires information from the declaration code.

[0084] The camera 116 functions as a biometric information acquisition unit that acquires a facial image of the user U as biometric information of the user U who uses the kiosk terminal 10 to make a customs declaration. The camera 116 is, for example, a photographing device such as a digital camera that captures video or still images of the face of the user U in front of the kiosk terminal 10, and acquires a photographed facial image, which is a facial image of the user U, from the captured video or still image. The camera 116, under the control of the CPU 102, captures a facial image of the user U who has the passport reader 112 read their passport or the code reader 114 read their declaration code in front of the kiosk terminal 10, as described below, to acquire the photographed facial image. In this way, the camera 116 can acquire a photographed facial image by capturing a facial image of the user U when it reads at least one of the passport and the declaration code. Note that instead of the camera 116, a biometric information acquisition unit that acquires from the user U the same type of biometric information as that acquired from the passport by the passport reader 112 may be provided.

[0085] CPU 102 can set the number of pixels between the eyes in a captured face image captured by camera 116 to a predetermined range. In this case, CPU 102 can, for example, exclude from the target face images captured by camera 116 any face images in which the number of pixels between the eyes is less than the predetermined number of pixels. This allows camera 116 to reliably capture the image of user U operating kiosk terminal 10 without capturing an image of a person far away behind user U operating kiosk terminal 10.

[0086] Furthermore, CPU 102 can mask predetermined ranges on both sides of the image capturing range of camera 116. This allows camera 116 to reliably capture an image of user U operating kiosk terminal 10 while preventing people around user U operating kiosk terminal 10 from being captured in the image.

[0087] The kiosk terminal 10 can be configured so that the facial image captured by the camera 116 is not displayed on the display 110, etc. In this case, the user U will not be aware that his or her face has been photographed by the kiosk terminal 10. Therefore, the user U will not feel any resistance to being photographed, and will not be subject to psychological pressure due to being photographed, allowing the user U to use the kiosk terminal 10 comfortably.

[0088] The depth camera 118 acquires three-dimensional information of an object photographed by the camera 116. Based on the three-dimensional information acquired by the depth camera 118, the CPU 102 can determine whether the facial image of the user U photographed by the camera 116 was acquired by photographing a living person or by photographing a two-dimensional image such as a photo. This makes it possible to prevent fraudulent acts such as impersonating another person.

[0089] The communication unit 120 is connected to the network NW and transmits and receives data via the network NW. The communication unit 120 communicates with the management server 50 and the like under the control of the CPU 102.

[0090] The status lamp 122 functions as a status display unit that indicates the status of the kiosk terminal 10. More specifically, the status lamp 122 can indicate the status of the kiosk terminal 10, such as the progress of the customs declaration by the user U at the kiosk terminal 10, whether there is an abnormality in the kiosk terminal 10, and whether the kiosk terminal 10 is available for use, using different lamp colors. The status lamp 122 can light up in a lamp color that corresponds to the status of the kiosk terminal 10 under the control of the CPU 102. A surveillance camera for detecting suspicious behavior may be provided between the status lamp 122 and the pole 100. This surveillance camera can be configured to start capturing images, for example, when a passport is read by the passport reader 112 or a declaration code is read by the code reader 114.

[0091] The status lamp 122 may be illuminated in different colors depending on the situation, such as green, yellow, red, flashing green, flashing yellow, or flashing red. In this case, each lamp color indicates the following situation: green indicates that the facial matching is consistent and the customs information is OK; yellow indicates that the facial matching has failed; red indicates that there is a problem with the customs information; flashing green indicates that the customs declaration status is slow; flashing yellow indicates that there is a problem with the system including the kiosk terminal 10; flashing red indicates that the person has an illegal passport or is determined to be blacklisted based on facial matching. These are merely examples, and other combinations of lamp colors and situations may be used. Thus, the status lamp 122 may be illuminated in different colors based on at least one of the information read by the passport reader 112 or the code reader 114, the results of the facial matching, and the status of the procedure. The above-mentioned light 130 provided on the side of the display 110 may also be illuminated in the same color as the status lamp 122. Like the status lamp 122, the light 130 can be illuminated differently based on at least one of the information read by the passport reader 112 or the code reader 114, the results of facial matching, and the status of the procedure. In this case, the light 130 can be provided, for example, on the top surface or upper end of the visor 128. While the contents of the screen of the display 110 are hidden from the surroundings by the visor 128, relevant personnel such as staff who know the relationship between the lamp color and situations such as errors can understand the status of the kiosk terminal 10 based on the illumination status of the status lamp 122 and the light 130. This allows relevant personnel to take appropriate action, such as taking immediate action when the display 110 or the light 130 flashes red.

[0092] In this way, the kiosk terminal 10 is configured.

[0093] The kiosk terminal 10 configured as described above can be installed one unit per pole 100, or multiple units per pole 100. Fig. 6B is a perspective view showing the configuration when one kiosk terminal 10 is installed, and Fig. 6A is a top view of Fig. 6B. Fig. 7B is a perspective view showing the configuration when two kiosk terminals 10 are installed, and Fig. 7A is a top view of Fig. 7B. Fig. 8B is a perspective view showing the configuration when three kiosk terminals 10 are installed, and Fig. 8A is a top view of Fig. 8B.

[0094] In a configuration in which one kiosk terminal 10 is installed on one pole 100, the kiosk terminal 10 is installed on the side of the pole 100 facing in a predetermined direction, as shown in Figures 6A and 6B.

[0095] Furthermore, by installing a plurality of kiosk terminals 10 on one pole 100, a group of kiosk terminals, which is a group of information processing devices, can be configured.

[0096] In a configuration in which two kiosks 10 are installed on one pole 100, the two kiosks 10 are installed on the sides of the pole 100 at a predetermined angular interval radially from the center of the pole 100, as shown in Figures 7A and 7B. The angular interval between two adjacent kiosks 10 is, for example, 120°. In this way, a kiosk group including two kiosks 10 is configured.

[0097] In a configuration in which three kiosks 10 are installed on one pole 100, the three kiosks 10 are installed on the sides of the pole 100 at predetermined angular intervals in a radial pattern around the pole 100 as shown in Figures 8A and 8B. The angular intervals between two adjacent kiosks 10 of the three kiosks 10 are equal, for example, 120°. In this way, a kiosk group including three kiosks 10 is configured.

[0098] In this way, when multiple kiosk terminals 10 are installed, by installing them radially from the pole 100 as the center, it is possible to ensure a larger gap between users U using adjacent kiosk terminals 10 than if they were installed in a horizontal line. This makes it difficult for users U in front of adjacent kiosk terminals 10 to peek at the displays 110 of the kiosk terminals 10.

[0099] When multiple kiosk terminals 10 are installed radially, the angular interval between two adjacent kiosk terminals 10 is not limited to the above 120° and can be set as appropriate. Furthermore, the number of kiosk terminals 10 installed on one pole 100 is not limited to the above number and may be four or more.

[0100] Furthermore, the kiosk terminal 10 is provided with visors 128 on both sides of the display 110 along the display 110. The visors 128 also make it difficult to peek at the display 110 in the kiosk terminal 10.

[0101] Figure 9 shows how the visor 128 blocks the line of sight in a configuration where three kiosks 10 are installed as shown in Figures 8A and 8B. As shown in Figure 9, the line of sight of a user U in front of a kiosk 10 toward an adjacent kiosk 10 is blocked by the visor 128 provided on the adjacent kiosk 10, as indicated by the dashed arrow in the figure. In this way, the visor 128 also makes it difficult to peek at the display 110.

[0102] Furthermore, when multiple kiosk terminals 10 are installed on one pole 100, the multiple kiosk terminals 10 can be installed at different heights. For example, as shown in FIG. 10 , in a configuration where three kiosk terminals 10 are installed, one kiosk terminal 10 can be installed at a lower position than the other two kiosk terminals 10. By installing the multiple kiosk terminals 10 at different heights in this way, users U of a wide range of heights can use the kiosk terminals 10. Furthermore, a kiosk terminal 10 installed at a lower position than the other kiosk terminals 10 can be used by, for example, a wheelchair-using user U while remaining in his or her wheelchair.

[0103] The means for attaching and fixing the kiosk terminal 10 to the pole 100 is not particularly limited, and various means can be used. For example, the kiosk terminal 10 can be fixed and installed with screws at one of multiple mounting positions at different heights provided on the pole 100. Furthermore, for example, the kiosk terminal 10 can be installed on the pole 100 so that it can slide up and down by the user U who uses the kiosk terminal 10, or by a customs officer or the like who manages the kiosk terminal 10. In this case, the user U or the like can adjust the height of the kiosk terminal 10 by sliding the kiosk terminal 10 installed on the pole 100 up and down.

[0104] Furthermore, cords or cables such as a power cord and communication cable (not shown) connected to the kiosk terminal 10 are stored and hidden inside the pole 100. Excess cords or cables can be stored inside the housing 126 of the kiosk terminal 10.

[0105] The kiosk terminal 10 can be installed on various supports such as a support stand, a counter, etc., in addition to the pole 100 described above.

[0106] Next, the configuration of the electronic gate 20 will be described with reference to Figs. 11 to 13. Fig. 11 is a schematic diagram showing the appearance of the entrance gate terminal 30 and the exit gate terminal 40 that make up the electronic gate 20. Fig. 12 is a block diagram showing an example of the hardware configuration of the entrance gate terminal 30. Fig. 13 is a block diagram showing an example of the hardware configuration of the exit gate terminal 40.

[0107] The electronic gate 20 is a gate device that allows or denies passage of a user U who has been guided to the electronic gate 20 by the kiosk terminal 10, based on the results of facial matching at the electronic gate 20, etc. A user U who is permitted to pass through the electronic gate 20 can exit the customs inspection area C. A user U who is not permitted to pass through the electronic gate 20 will be dealt with individually, for example, by being guided by a customs official to a manned booth M and undergoing a face-to-face customs inspection by a customs official.

[0108] 11, the entrance gate terminal 30 and the exit gate terminal 40 that make up the electronic gate 20 are installed at the entrance and exit of a gate passage P through which a user U must pass after being guided to the electronic gate 20 and deemed to be able to use the electronic gate 20, respectively. The gate passage P is restricted from allowing a user U who has entered the gate passage P to exit from any location other than the exit gate terminal 40 by, for example, partitions, walls, fences, inspection tables, etc., installed on both sides along the gate passage P.

[0109] First, the entrance gate terminal 30 will be described with reference to Figures 11 and 12. As shown in Figure 11, the entrance gate terminal 30 is a gate device installed at the entrance of a gate passage P through which a user U guided to the electronic gate 20 passes. As shown in Figure 12, the entrance gate terminal 30 has a CPU 302, RAM 304, a storage device 306, an entrance gate door 308, a passage detection sensor 310, a communication unit 312, and a guidance display 314. The CPU 302, RAM 304, storage device 306, entrance gate door 308, passage detection sensor 310, communication unit 312, and guidance display 314 are connected to a bus line 316.

[0110] 11 , the entrance gate terminal 30 has a pair of housings 318 that are placed opposite each other with the entrance to the gate passage P in between. The pair of housings 318 are provided with an entrance gate door 308 that can close the space between the pair of housings 318. The pair of housings 318 are also provided with a plurality of passage detection sensors 310 that are aligned in the direction of entry into the gate passage P. The pair of housings 318 are also provided with guide displays 314 that face the front of the entrance gate terminal 30, which is outside the gate passage P. The CPU 302, RAM 304, storage device 306, and communication unit 312 may be housed in the housing 318, or may be housed in a housing separate from the housing 318.

[0111] CPU 302 operates by executing programs stored in storage device 306, and functions as a control unit that controls the overall operation of entrance gate terminal 30. CPU 302 also executes application programs stored in storage device 306 to perform various processes as entrance gate terminal 30. RAM 304 provides the memory area necessary for the operation of CPU 302.

[0112] More specifically, the CPU 302 functions as a determination unit that determines whether or not the user U has passed through the entrance gate terminal 30 and entered the gate passage P based on the output signals of the multiple passage detection sensors 310 and their output order.

[0113] The CPU 302 also functions as a door control unit that controls the opening and closing operations of the entrance gate door 308. When the user U has not entered the gate passage P, the CPU 302 maintains the entrance gate door 308 in an open state. When the CPU 302 determines, based on the output signal of the passage detection sensor 310, that the user U has passed through the entrance gate terminal 30 and entered the gate passage P, the CPU 302 closes the entrance gate door 308. The CPU 302 maintains the entrance gate door 308 in a closed state until it determines that the user U who entered the gate passage P has passed through the exit gate terminal 40 and exited from the gate passage P. The CPU 302 determines, based on exit passage information transmitted from the exit gate terminal 40, that the user U has exited from the gate passage P. When the CPU 302 determines that the user U has exited from the gate passage P, the CPU 302 opens the entrance gate door 308.

[0114] In addition, when the CPU 302 determines that the user U has passed through the entrance gate terminal 30 and entered the gate passage P as described above, it transmits entrance passage information indicating that the user U has passed through the entrance gate terminal 30 and entered the gate passage P to the exit gate terminal 40.

[0115] The CPU 302 also functions as a display control unit that controls the display on the guidance display 314 depending on the open / closed state of the entrance gate door 308. When the entrance gate door 308 is in an open state, the CPU 302 causes the guidance display 314 to display a display indicating that entry into the gate passage P is permitted. When the entrance gate door 308 is in a closed state, the CPU 302 causes the guidance display 314 to display a display indicating that entry into the gate passage P is prohibited.

[0116] The storage device 306 is configured with a storage medium such as a nonvolatile memory, a hard disk drive, etc., and functions as a storage unit. The storage device 306 stores programs executed by the CPU 302, data referenced by the CPU 302 when executing the programs, etc.

[0117] The entrance gate door 308 is an opening / closing door that opens and closes under the control of the CPU 302, and transitions between an open state that allows the user U to pass through and a closed state that blocks the user U. The opening / closing method of the entrance gate door 308 is not particularly limited, but may be, for example, a flapper type, a sliding type, a rotating type, or the like.

[0118] When the passage detection sensor 310 detects the passage of user U, it outputs an output signal indicating the passage of user U. Based on the output signals of multiple passage detection sensors 310 provided in a pair of housings 318 aligned in the direction of entry into gate passage P and the order in which they are output, the CPU 302 can determine whether or not user U has passed through the entrance gate terminal 30 and entered gate passage P. The passage detection sensor 310 is not particularly limited, but may be, for example, a transmission type photoelectric sensor or a reflection type photoelectric sensor. A transmission type photoelectric sensor has an irradiation unit that irradiates light such as infrared light and a light receiving unit that receives the light irradiated by the irradiation unit, and outputs an output signal indicating the passage of an object based on a change in the light received by the light receiving unit.

[0119] The communication unit 312 is connected to the network NW and transmits and receives data via the network NW. The communication unit 312 communicates with the exit gate terminal 40 and the like in accordance with the control of the CPU 302. Note that the communication unit 312 may be connected to the exit gate terminal 40 so as to be able to communicate with the exit gate terminal 40 without going through the network NW.

[0120] The guidance display 314 displays a display indicating whether entry into the gate passage P is permitted or not, in accordance with control by the CPU 302. When the entrance gate door 308 is in an open state, the guidance display 314 displays a display indicating that entry into the gate passage P is permitted. Furthermore, when the entrance gate door 308 is in a closed state, the guidance display 314 displays a display indicating that entry into the gate passage P is prohibited. The guidance display 314 can display a display indicating whether entry into the gate passage P is permitted or not, for example, by using a color display, a symbol display, a character display, or the like.

[0121] In this way, the entrance gate terminal 30 is constructed.

[0122] Next, the configuration of the exit gate terminal 40 will be described with reference to Figures 11 and 13. As shown in Figure 11, the exit gate terminal 40 is a gate device installed at the exit of gate passage P, which is a passage through which a user U guided to the electronic gate 20 passes. As shown in Figure 13, the exit gate terminal 40 has a CPU 402, a RAM 404, a storage device 406, and a display 408. The exit gate terminal 40 also has a first camera 410, a second camera 412, an exit gate door 414, a passage detection sensor 416, a communication unit 418, and a guidance display 420. The CPU 402, RAM 404, storage device 406, display 408, first camera 410, second camera 412, exit gate door 414, passage detection sensor 416, communication unit 418, and guidance display 420 are connected to a bus line 422.

[0123] 11 , the exit gate terminal 40 has a pair of housings 424 that are installed facing each other across the exit of the gate passage P. The pair of housings 424 are provided with an exit gate door 414 that can close the space between the pair of housings 424. The pair of housings 424 are also provided with a plurality of passage detection sensors 416 that are aligned in the direction of exiting from the gate passage P. The pair of housings 424 are also provided with a guide display 420 that faces the front of the exit gate terminal 40, which is inside the gate passage P. The CPU 402, RAM 404, storage device 406, and communication unit 418 may be housed in the housing 424, or may be housed in a housing separate from the housing 424.

[0124] A display 408 is installed on one of the pair of housings 424. The display 408 is installed, for example, so that its screen faces the center of the gate passage P in front of the exit gate terminal 40. A first camera 410 and a second camera 412 are installed on the bottom and top of the screen side of the display 408, respectively. The installation position of the display 408 is not particularly limited, and it may be located in any position in the gate passage P where the user U can see the display 408.

[0125] The first camera 410 is a long-range camera whose image capturing range covers at least the inside of the gate passage P between the entrance gate terminal 30 and the exit gate terminal 40, and can capture images at a longer distance than the second camera 412. The second camera 412 is a short-range camera whose image capturing range covers at least the area in front of the exit gate terminal 40. The positions at which the first camera 410 and the second camera 412 are installed are not particularly limited, and may be any positions that allow the realization of their respective image capturing ranges.

[0126] The first camera 410 photographs the user U moving through the gate passage P from the entrance gate terminal 30 toward the exit gate terminal 40, and acquires the facial image as a first target facial image. The second camera 412 photographs the user U moving through the gate passage P, arriving at the exit gate terminal 40, and standing in front of the exit gate door 414, as necessary, and acquires the facial image as a second target facial image. The first camera 410 photographs the user U without the user U's awareness. On the other hand, the second camera 412 photographs the user U after the user U receives a notification of the photographing, for example, via a display on the display 408. The exit gate terminal 40 can perform passive authentication to verify the identity of the user U by face matching using the first target facial image photographed by the first camera 410 without the user U's awareness. Furthermore, if passive authentication fails, the exit gate terminal 40 can perform active authentication to verify the identity of the user U by face matching using the second target facial image photographed by the second camera 412 with the user U's awareness.

[0127] The CPU 402 operates by executing programs stored in the storage device 406, and functions as a control unit that controls the overall operation of the exit gate terminal 40. The CPU 402 also executes application programs stored in the storage device 406 to perform various processes as the exit gate terminal 40. The RAM 404 provides a memory area necessary for the operation of the CPU 402.

[0128] More specifically, the CPU 402 functions as a verification request unit that requests the management server 50 to verify the face of a user U who has passed through the entrance gate terminal 30 and entered the gate passage P.

[0129] The CPU 402 as a matching request unit requests the management server 50 to perform a 1:N matching between a first target face image, which is a face image of user U captured by the first camera 410, and a plurality of registered face images registered in the user information DB 506a of the management server 50. For this purpose, the CPU 402 transmits the facial feature amounts extracted from the first target face image or the first target face image itself together with the matching request to the management server 50.

[0130] The CPU 402 can function as a quality determination unit that calculates a quality value indicating the quality of the first target face image and determines whether the calculated quality value is equal to or greater than a predetermined threshold. The quality value can be set to a larger value as the quality of the first target face image becomes higher. The CPU 402 as a matching request unit can request the management server 50 to match a high-quality first target face image whose quality value is equal to or greater than a predetermined threshold with a registered face image. The quality value can also be set to a smaller value as the quality of the first target face image becomes higher. In this case, the CPU 402 as a quality determination unit can determine whether the calculated quality value is equal to or less than a predetermined threshold. In this case, the CPU 402 as a matching request unit can request the management server 50 to match a high-quality first target face image whose quality value is equal to or less than a predetermined threshold with a registered face image. In this way, the CPU 402 can determine the quality of the first target face image and request that a high-quality first target face image whose quality value is equal to or greater than a certain threshold be matched with a registered face image.

[0131] The first target face image is captured by the first camera 410 without the user U being aware of it. For this reason, there are cases where a first target face image with a certain quality value or higher cannot be obtained, or where face matching using the first target face image does not match due to face accessories worn on the face of the user U, such as a mask, sunglasses, glasses, or a hat. In these cases, the CPU 402, which serves as a matching request unit, requests the management server 50 to perform a 1:N matching between the second target face image, which is a face image of the user U captured by the second camera 412, and multiple registered face images registered in the user information DB 506a. For this reason, the CPU 402 transmits facial feature amounts extracted from the second target face image or the second target face image itself to the management server 50 together with a matching request.

[0132] Furthermore, CPU 402 functions as a warning unit that issues a warning to user U depending on the state of user U's face while capturing an image by first camera 410. More specifically, CPU 402 can issue a warning, for example, as follows.

[0133] First, the CPU 402 can perform gaze detection on the first target face image while capturing an image with the first camera 410. The CPU 402 can issue a warning depending on the state of the user U's gaze detected from the first target face image by gaze detection. That is, when the CPU 402 determines that the user U is looking down or to the side as a result of gaze detection, it can issue a warning to the user U to face forward, which is the side of the exit gate terminal 40. The CPU 402 can issue a warning, for example, by displaying on the display 408 a message instructing the user U to change the direction of their face so that they face forward, or by emitting a sound from a speaker or the like instructing the user U to change the direction of their face so that they face forward. In this case, the warning can be displayed on the display 408 before face matching is performed in order to prevent unnecessary face matching failures.

[0134] Furthermore, the CPU 402 can perform accessory estimation on the captured first target face image while capturing an image with the first camera 410. The CPU 402 can detect face accessories such as masks, sunglasses, glasses, and hats in the first target face image through accessory estimation. When the CPU 402 detects a face accessory, it can warn the user U to remove the face accessory that is interfering with face matching. The CPU 402 can issue a warning by, for example, displaying a message on the display 408 instructing the user U to remove the face accessory, or by emitting a sound from a speaker or the like instructing the user U to remove the face accessory. In this case, the warning on the display 408 can be displayed before face matching is performed to prevent unnecessary face matching failures.

[0135] The CPU 402 also functions as a matching information acquisition unit that acquires, from the management server 50, matching information indicating the result of face matching of the first target face image or the second target face image requested from the management server 50. The matching information indicates that, as a result of matching, a registered face image matching the first target face image or the second target face image has been found and the identity of the user U has been successfully verified, or that a matching registered face image has not been found and the identity of the user U has been unsuccessful. Note that when the first target face image or the second target face image matches the registered face image, this includes a case where the matching score indicating the similarity between both face images exceeds a predetermined threshold and both face images show a high similarity.

[0136] Note that, instead of functioning as a matching request unit and a matching information acquisition unit, the CPU 402 can also be configured to function as a matching unit that matches the first target face image or the second target face image with the registered face images in the user information DB 506a in a 1:N ratio and outputs matching information. In this case, the CPU 402 can perform facial image matching by referring to the user information DB 506a in the management server 50 or by referring to a user information DB that is synchronized with the user information DB 506a and stored in the storage device 406.

[0137] Furthermore, when the CPU 402 has successfully verified the identity of the user U through facial matching, it can function as an information acquisition unit that acquires gate availability information associated with the user information of the user U from the management server 50. As described above, the gate availability information is information that indicates the result of a determination as to whether the user U can use the electronic gate 20.

[0138] The CPU 402 also functions as a determination unit that determines whether a user U in the gate passage P can pass through the electronic gate 20 based on the result of face matching of the first target face image or the second target face image and the determination result of whether the user U can use the electronic gate 20. The CPU 402 as a determination unit determines that the user U can pass through the electronic gate 20 when the face matching of the first target face image or the second target face image matches, the identity of the user U is successful, and the user U is allowed to use the electronic gate 20. A successful identity verification occurs when the first target face image or the second target face image matches a registered face image, which is a photographed face image acquired by the kiosk terminal 10 or a passport face image. A user U is allowed to use the electronic gate 20 when there is gate availability information associated with user information including the matching registered face image and indicating that the user U can use the electronic gate 20.

[0139] The CPU 402 also functions as a door control unit that controls the opening and closing operations of the exit gate door 414. When the CPU 402 determines that the user U is able to pass through the electronic gate 20, it opens the exit gate door 414, which is normally in a closed state. When the CPU 402 determines, based on the output signal of the passage detection sensor 416, that the user U has passed through the exit gate terminal 40 and exited from the gate passage P, it closes the exit gate door 414.

[0140] The CPU 402 also functions as a determination unit that determines whether the user U has passed through the exit gate terminal 40 and exited from the gate passage P based on the output signals of the plurality of passage detection sensors 416 and the order in which they are output.

[0141] In addition, when the CPU 402 determines that the user U has passed through the exit gate terminal 40 and exited from the gate passage P as described above, it transmits exit passage information indicating that the user U has passed through the exit gate terminal 40 and exited from the gate passage P to the entrance gate terminal 30.

[0142] The CPU 402 also functions as a display control unit that controls the display on the display 408. The CPU 402 can display various notifications on the display 408 for the user U in the gate passage P. For example, the CPU 402 can display a warning on the display 408 instructing the user U in the gate passage P to remove any face accessories, such as a mask, sunglasses, eyeglasses, or a hat, that may hinder face matching. Furthermore, for example, the CPU 402 can display on the display 408 a notification to the user U who remains in front of the exit gate door 414, as necessary, that a face image will be taken by the second camera 412. If the first target face image does not match the registered face image or if the matching is not possible, the CPU 402 can display on the display 408 a message urging the user U to remain in front of the exit gate door 414. In this case, the CPU 402 can further display on the display 408 a message urging the user U to remain in front of the exit gate door 414 and look at the display 408. This allows the user U to be guided so that the second camera 412 can capture an image of the user U's face.

[0143] The CPU 402 also functions as a display control unit that controls the display on the guidance display 420 depending on the open / closed state of the exit gate door 414. When the exit gate door 414 is in an open state, the CPU 402 causes the guidance display 420 to display a display indicating that exit from the gate passage P is permitted. When the exit gate door 414 is in a closed state, the CPU 402 causes the guidance display 420 to display a display indicating that exit from the gate passage P is prohibited.

[0144] The storage device 406 is configured with a storage medium such as a nonvolatile memory, a hard disk drive, etc., and functions as a storage unit. The storage device 406 stores programs executed by the CPU 402, data referenced by the CPU 402 when executing the programs, etc.

[0145] The display 408 functions as a display unit that displays various screens to the user U in the gate passage P. For example, the display 408 displays a warning instructing the user U to remove any face accessories that may hinder face matching, such as a mask, sunglasses, glasses, or a hat. Furthermore, for example, the display 408 displays a notification that the face will be photographed by the second camera 412, as necessary. Specifically, if matching cannot be performed using the first target face image photographed by the first camera 410, the display 408 displays a message instructing the user to stand in front of the display 408 and look in the direction of the second camera 412 so that active authentication can be performed. For example, the display 408 can display a message such as, "Facial authentication will be performed, so please stand in front of the display 408 and look in the direction of the second camera 412." The display 408 can also display the matching results of passive authentication using the first target face image, which is a face image of the user U in motion, and the matching results of active authentication using the second target face image, which is a face image of the user U at rest.

[0146] The first camera 410 functions as a biometric information acquisition unit that captures an image of a user U entering the gate passage P and moving through the gate passage P toward the exit gate terminal 40, and acquires a first target facial image, which is a facial image of the user U, as biometric information of the user U. The first target facial image captured by the first camera 410 is a facial image of the moving user U. The first camera 410 is, for example, a photographing device such as a digital camera that captures video or still images of the user U moving through the gate passage P, and acquires the first target facial image of the user U from the captured video or still image. As described above, the first camera 410 is a long-distance camera that can capture at least the interior of the gate passage P between the entrance gate terminal 30 and the exit gate terminal 40 and can capture images at a longer distance than the second camera 412. The first camera 410 captures the user U without the user U being aware that he or she is being photographed. Instead of the first camera 410, a biometric information acquisition unit that acquires from the user U the same type of biometric information as the registered biometric information registered in the user information DB 506a of the management server 50 can be provided.

[0147] The second camera 412 functions as a biometric information acquisition unit that, as necessary, photographs the user U who has traveled through the gate passage P and arrived at and stopped at the exit gate door 414 of the exit gate terminal 40, and acquires a second target facial image, which is a facial image of the user U, as biometric information of the user U. The second target facial image captured by the second camera 412 is a facial image of the user U who is stationary. The second camera 412 is, for example, a photographing device such as a digital camera that captures video or still images of the user U in front of the exit gate door 414, and acquires the second target facial image of the user U from the captured video or still image. As described above, the second camera 412 is a close-range camera whose captureable range is at least in front of the exit gate terminal 40. The second camera 412 photographs the user U upon receiving a notification of the photographing, for example, via a display on the display 408, an audio guide, or the like. The second camera 412 acquires the second target facial image from the user U who is standing in front of the exit gate door 414 in a predetermined case, to be compared with the registered facial image. The "predetermined case" referred to here is a case where the first target face image does not match the registered face image or where the matching is not possible, as will be described later. Specifically, the "predetermined case" refers to a case where the face of the user U moving toward the exit gate door 414 in the closed state is not facing the exit gate door 414, or a case where the user U is wearing an attachment on his / her face, etc. Note that instead of the second camera 412, a biometric information acquisition unit that acquires from the user U the same type of biometric information as the registered biometric information registered in the user information DB 506a of the management server 50 can be provided.

[0148] The exit gate door 414 is an opening and closing door that opens and closes under the control of the CPU 402, and transitions between a closed state that blocks the passage of the user U and an open state that allows the passage of the user U. The opening and closing method of the exit gate door 414 is not particularly limited, but may be, for example, a flapper type, a sliding type, a rotating type, or the like.

[0149] When the passage detection sensor 416 detects the passage of the user U, it outputs an output signal indicating the passage of the user U. The CPU 402 can determine whether or not the user U has passed through the exit gate terminal 40 and exited from the gate passage P based on the output signals of the multiple passage detection sensors 416 provided in a pair of housings 424 so as to be aligned in the direction of exit to the gate passage P and the order in which they are output. The passage detection sensor 416 is not particularly limited, but, like the passage detection sensor 310 of the entrance gate terminal 30, it may be, for example, a transmission type photoelectric sensor, a reflective type photoelectric sensor, or the like.

[0150] The communication unit 418 is connected to the network NW and transmits and receives data via the network NW. The communication unit 418 communicates with the management server 50, entrance gate terminal 30, etc. under the control of the CPU 402. Note that the communication unit 418 may be connected to the entrance gate terminal 30 so as to be able to communicate with the entrance gate terminal 30 without going through the network NW.

[0151] The guidance display 420 displays a display indicating whether or not exit from the gate passage P is permitted, in accordance with control by the CPU 402. When the exit gate door 414 is closed, the guidance display 420 displays a display indicating that exit from the gate passage P is prohibited. Furthermore, when the exit gate door 414 is open, the guidance display 420 displays a display indicating that exit from the gate passage P is permitted. The guidance display 420 can display a display indicating whether or not exit from the gate passage P is permitted, for example, by using a color display, a symbol display, a character display, or the like.

[0152] In this way, the exit gate terminal 40 is constructed.

[0153] Next, the configuration of the management server 50 will be described with reference to Fig. 14. Fig. 14 is a block diagram showing an example of the hardware configuration of the management server 50.

[0154] The management server 50 is an information processing device that manages the information processing system 1, compares the first target face image or the second target face image with a plurality of registered face images in the user information DB 506a, determines the declaration content included in the declaration information, etc. The functions of the management server 50 may be realized by a single server or by a plurality of servers.

[0155] 14, the management server 50 includes a CPU 502, a RAM 504, a storage device 506, and a communication unit 508. The CPU 502, the RAM 504, the storage device 506, and the communication unit 508 are connected to a bus line 510.

[0156] The CPU 502 operates by executing programs stored in the storage device 506, and functions as a control unit that controls the overall operation of the management server 50. The CPU 502 also executes application programs stored in the storage device 506 to perform various processes as the management server 50. The RAM 504 provides a memory area necessary for the operation of the CPU 502.

[0157] More specifically, when the CPU 502 receives a matching request from the kiosk terminal 10 to match the face of the user U, the CPU 502 functions as a matching unit that matches the face of the user U. The CPU 502 as a matching unit matches the photographed face image included in the user information with the face image on the passport on a one-to-one basis. When matching the photographed face image with the face image on the passport, the CPU 502 can match both face images by matching the facial feature amount extracted from the photographed face image with the facial feature amount extracted from the face image on the passport.

[0158] The CPU 502 also functions as a transmitter that transmits matching information indicating the result of facial matching of the user U to the kiosk terminal 10 that sent the matching request. The matching information indicates whether the matching is a match or a mismatch. That is, the matching information indicates that the photographed facial image matches the facial image on the passport, and the identity of the user U making the customs declaration has been successfully verified, or that the photographed facial image does not match the facial image on the passport, and the identity of the user U has not been verified.

[0159] Furthermore, when the CPU 502 receives the user information of the user U from the kiosk terminal 10, the CPU 502 functions as a determination unit that determines the declaration content included in the declaration information of the user U. As a determination unit, the CPU 502 determines whether the user U is subject to taxation and whether the user U is at risk, based on the declaration content included in the declaration information of the user U. The cases where the user U is subject to taxation and the cases where the user U is at risk are as described above.

[0160] The CPU 502 also functions as a transmission unit that transmits, to the kiosk terminal 10 that transmitted the user information, determination information indicating the determination result of the declaration content included in the declaration information of the user U. As described above, the determination information indicates whether the user U is subject to taxation or whether the user U is at risk as a result of the determination.

[0161] The CPU 502 functions as an information management unit that stores and manages the user information of the user U received from the kiosk terminal 10 in the storage device 506. The CPU 502 as an information management unit registers and manages the user information received from the kiosk terminal 10 in a user information DB 506a stored in the storage device 506. Every time the CPU 502 receives user information from the kiosk terminal 10, it registers the received user information in the user information DB 506a.

[0162] As described above, the user information of user U includes the user U's personal information, facial information, and declared information, which are associated with each other. The facial information is a photographed facial image and a passport facial image acquired by the kiosk terminal 10, or facial features extracted from these. The photographed facial image or passport facial image registered in the user information DB 506a is used as a registered facial image, which is pre-registered registered biometric information. Either a photographed facial image or a passport facial image can be used as the registered facial image, but a photographed facial image newer than the passport facial image can be used as the registered facial image. The registered facial image, which is a photographed facial image or a passport facial image registered in the user information DB 506a, or its facial features is used for face matching to verify the identity of user U at the electronic gate 20.

[0163] Furthermore, when the CPU 502 receives gate availability information from the kiosk terminal 10, it registers the gate availability information in association with the user information in the user information DB 506a. As described above, the gate availability information is information that indicates the determination result as to whether or not the electronic gate 20 can be used for the user U who makes a customs declaration.

[0164] The CPU 502 can also register, in the user information DB 506a, the user information of a user U who is determined to be able to use the electronic gate 20 based on the gate availability information. In other words, the CPU 502 can also not register, in the user information DB 506a, the user information of a user U who is determined to be unable to use the electronic gate 20.

[0165] In addition, instead of receiving gate availability information from the kiosk terminal 10, the CPU 502 can also determine whether the user U can use the electronic gate 20 in the same way as the kiosk terminal 10 and obtain the gate availability information.

[0166] Furthermore, the CPU 502 can delete the user information from the user information DB 506a after a certain time has elapsed since its registration. For example, the CPU 502 can delete the user information after the time normally required from the completion of customs declaration at the kiosk terminal 10 to passing through the electronic gate 20 at the customs inspection site C has elapsed.

[0167] Furthermore, when the CPU 502 receives a matching request requesting face matching of user U from the exit gate terminal 40, the CPU 502 functions as a matching unit that performs face matching of user U. As a matching unit, the CPU 502 matches the first target face image or the second target face image or their facial feature amounts with a plurality of registered face images or their facial feature amounts registered in the user information DB 506a in a 1:N ratio. When matching the first target face image or the second target face image with the registered face image, the CPU 502 can match both face images by matching the facial feature amounts extracted from the first target face image or the second target face image with the facial feature amounts extracted from the registered face image.

[0168] The CPU 502 as a matching unit matches the first or second target face image with the multiple registered face images to attempt to find a registered face image that matches the first or second target face image from among the multiple registered face images. The CPU 502 also matches the facial feature amounts of the first or second target face image with the facial feature amounts of the multiple registered face images to attempt to find a registered face image that matches the facial feature amounts of the first or second target face image from among the multiple registered face images. A match between the first or second target face image and a registered face image includes a case where a matching score indicating the similarity between the two facial images exceeds a predetermined threshold, indicating a high similarity between the two facial images. A match between the facial feature amounts of the first or second target face image and a registered face image also includes a case where a matching score indicating the similarity between the two facial features exceeds a predetermined threshold, indicating a high similarity between the two facial features.

[0169] The CPU 502 functions as a transmitter that transmits matching information indicating the result of the matching to the exit gate terminal 40 that transmitted the matching request. The matching information indicates a match or a mismatch. That is, the matching information indicates that a registered face image matching the first target face image or the second target face image was found as a result of the matching, thereby successfully verifying the identity of the user U, or that a matching registered face image was not found and thus the identity of the user U was not verified. The matching information also indicates that a facial feature of a registered face image matching the facial feature of the first target face image or the second target face image was found as a result of the matching, thereby successfully verifying the identity of the user U, or that a matching facial feature of a registered face image was not found and thus the identity of the user U was not verified. The CPU 502 may perform a match between the first target face image or the second target face image or its facial feature and one registered face image or its facial feature.

[0170] The CPU 502 can delete a registered face image that matches the first target face image or the second target face image or user information including the registered face image from the user information DB 506a. The CPU 502 can also delete facial feature amounts of a registered face image that matches the facial feature amounts of the first target face image or the second target face image or user information including the facial feature amounts of the registered face image from the user information DB 506a.

[0171] Furthermore, the CPU 502 functions as a transmitting unit that transmits gate availability information to the exit gate terminal 40 as needed.

[0172] The storage device 506 is configured with a storage medium such as a nonvolatile memory, a hard disk drive, etc., and functions as a storage unit. The storage device 506 stores programs executed by the CPU 502, data referenced by the CPU 502 when executing the programs, etc.

[0173] The storage device 506 also stores a user information DB 506a. The user information DB 506a stores user information of multiple users U who have made customs declarations using the kiosk terminal 10. The user information DB 506a also stores gate availability information for each user U, associated with the user information.

[0174] The communication unit 508 is connected to the network NW and transmits and receives data via the network NW. The communication unit 508 communicates with the kiosk terminal 10, the exit gate terminal 40, etc. under the control of the CPU 502.

[0175] In this way, the management server 50 is configured.

[0176] The kiosk terminal 10 according to this embodiment photographs a user U operating the kiosk terminal 10 using a camera 116 provided between the display 110 and the passport reader 112 and code reader 114. The user U faces the camera 116 when looking at the display 110, holding a passport over the passport reader 112, or holding a portable terminal 60 over the code reader 114. As a result, the kiosk terminal 10 captures a photograph of the user U's face with the camera 116 and acquires a photographed facial image while the passport reader 112 is reading the passport or the code reader 114 is reading the declaration code. Because the user U necessarily faces the camera 116 provided between the display 110 and the passport reader 112 and code reader 114, there is no need for the user U to interrupt his or her operation of the kiosk terminal 10 to face the camera 116. Furthermore, the kiosk terminal 10 can read a passport using the passport reader 112 and read a declaration code using the code reader 114 simultaneously or in any order.

[0177] Therefore, with the kiosk terminal 10 according to this embodiment, procedures at the kiosk terminal 10, including reading the passport, reading the declaration code, and acquiring a photographed face image to be used for face matching, can be completed in a short time. Furthermore, since the kiosk terminal 10 can photograph the face of the user U while reading the passport or declaration code, the user U is not made aware that the photograph is being taken, and therefore the psychological burden on the user U can be reduced.

[0178] Furthermore, the exit gate terminal 40 of the electronic gate 20 according to this embodiment compares a first target face image obtained by photographing a user U moving toward the exit gate terminal 40 with a registered face image registered in the user information DB 506a of the management server 50. The first target face image can be obtained without the user U stopping. Therefore, the electronic gate 20 according to this embodiment can efficiently perform face matching of the user U using the first target face image. This reduces the time required for the user U to pass through the electronic gate 20. Furthermore, if the first target face image does not match or cannot be matched, the exit gate terminal 40 compares a second target face image obtained by photographing the user U standing in front of the exit gate door 414 with the registered face image. Therefore, the electronic gate 20 according to this embodiment can reliably verify the identity of the user U through face matching.

[0179] Furthermore, the exit gate terminal 40 of the electronic gate 20 according to this embodiment opens the exit gate door 414 in accordance with a predetermined determination result of whether the gate is available for use. Therefore, according to the electronic gate 20 according to this embodiment, the time required for the user U to pass through the electronic gate 20 can be shortened.

[0180] Hereinafter, the operation of each component in the information processing system 1 according to this embodiment and the control method for realizing these operations will be further described with reference to FIGS. 15A to 20. FIG.

[0181] First, the operation of the kiosk terminal 10 will be described together with the operation of the management server 50 using Figs. 15A to 18C. Fig. 15A is a schematic diagram showing an example of a declaration information input screen on the mobile terminal 60. Fig. 15B is a schematic diagram showing an example of a declaration code display screen on the mobile terminal 60. Fig. 16 is a sequence diagram showing the operation of the kiosk terminal 10 and the management server 50. Fig. 17 is a sequence diagram showing the language setting operation of the kiosk terminal 10. Fig. 18A is a schematic diagram showing an example of a reception screen on the kiosk terminal 10. Fig. 18B is a schematic diagram showing an example of a declaration content confirmation screen on the kiosk terminal 10. Fig. 18C is a schematic diagram showing an example of a guidance screen on the kiosk terminal 10.

[0182] For example, a user U who arrives at an airport from a foreign country by airplane goes through immigration at the immigration inspection area, collects his / her checked baggage at the baggage claim area, and then heads to customs inspection area C to make a customs declaration. The user U inputs declaration information into a customs declaration application executed on the mobile terminal 60, and a declaration code including the declaration information is generated and displayed. The user U only needs to generate and display the declaration code before using the kiosk terminal 10.

[0183] FIG. 15A shows a declaration information input screen SHE of the customs declaration application on the mobile terminal 60. The declaration information input screen SHE has a declaration information input field SHE10 for inputting information to be declared in a declaration for baggage customs clearance, for example. For example, in the case of Japan, the declaration information input field SHE10 allows input of information to be declared in a declaration using Customs Form C-5360, a personal effects and unaccompanied baggage declaration form. That is, the declaration information input field SHE10 allows input of boarding information, such as flight number, departure point, and date of entry, related to user U's boarding, as well as answers to questions. In the customs declaration application, basic information of user U, such as user U's name, current address, telephone number, date of birth, and passport number, which does not change with each customs declaration, can be entered and registered on a user registration screen, setting screen, etc. (not shown).

[0184] 15B shows the declaration code display screen SHC of the customs declaration application on the mobile terminal 60. The declaration code display screen SHC has a declaration code display field SHC10 in which a declaration code including the declaration information entered on the declaration information input screen SHE is generated and displayed. The declaration code displayed in the declaration code display field SHC10 includes not only the declaration information but also language information related to the language used in the customs declaration application. In addition, the declaration code display field SHC10 can display the expiration date and time of the declaration code, how to use the declaration code, etc.

[0185] Upon entering the customs inspection area C, user U makes a customs declaration at a kiosk terminal 10 installed near the entrance to the customs inspection area C. User U can generate and display a declaration code on his / her mobile terminal 60 before arriving at the kiosk terminal 10. At that time, user U can refer to instructions on how to use the declaration code displayed on the customs declaration application on the mobile terminal 60, and instructions on how to proceed at the kiosk terminal 10 displayed on a digital signage terminal 70 installed near the kiosk terminal 10.

[0186] 16, the CPU 102 of the kiosk terminal 10 continuously determines whether or not a passport has been held over the passport reading device 112 and whether or not a declaration code has been held over the code reading device 114 (step S102). In this way, the CPU 102 waits until at least one of the passport and the declaration code is held over (step S102, NO).

[0187] 18A shows an example of a reception screen SKR displayed on the display 110 of the kiosk terminal 10 waiting for the user U to hold up at least one of their passport and declaration code. The reception screen SKR displays, for example, that a declaration code is required to use the kiosk terminal 10 and how to hold up the passport and declaration code.

[0188] 16, when CPU 102 determines that a passport has been swiped (step S102, YES), it reads and acquires passport information such as user U's personal information and passport face image from the passport using passport reading device 112 (step S104). Also, when CPU 102 determines that a declaration code has been swiped (step S102, YES), it reads and acquires declaration information and language information from the declaration code using code reading device 114 (step S106).

[0189] User U may hold his / her passport over passport reading device 112 to have it read before the declaration code, or may hold his / her passport over code reading device 114 to have it read before the passport. User U may also hold his / her passport and declaration code over passport reading device 112 and code reading device 114 at the same time to have them read. CPU 102 can cause passport reading device 112 to read the passport and code reading device 114 to read the declaration code in any order or simultaneously, depending on the timing at which user U's passport and declaration code are read.

[0190] Furthermore, if the CPU 102 detects that a certain time has passed after reading one of the passport and the declaration code without the other being held over the kiosk terminal 10, or that the user U has left the kiosk terminal 10, it can suspend the process. In this case, the CPU 102 can, for example, display an error message on the display 110 or perform a reset process to invalidate the processes up to that point. Thereafter, the CPU 102 can proceed to step S102. The CPU 102 can detect that the user U has left the kiosk terminal 10, for example, based on moving images captured by the camera 116 or still images continuously captured.

[0191] Furthermore, when CPU 102 determines that the passport or declaration code has been held up (step S102, YES), it causes camera 116 to photograph the face of user U and acquire a photographed facial image of user U (step S108). CPU 102 can photograph the face of user U with camera 116 while the passport or declaration code is being read, in parallel with the passport reading device 112 reading the passport or the declaration code reading device 114. Camera 116 can start photographing when the passport or declaration code is held up.

[0192] In this way, the CPU 102 uses the camera 116 to capture a facial image of the user U who is standing in front of the kiosk terminal 10 and has the passport reader 112 read the passport or the code reader 114 read the declaration code, thereby obtaining a captured facial image.

[0193] As described above, camera 116 is installed so that its shooting direction faces diagonally upward and forward, and has an angle of view that allows it to capture the face of user U who operates kiosk terminal 10 while looking down at display 110, passport reader 112, or code reader 114. Therefore, camera 116 can capture the face of user U who operates kiosk terminal 10 while looking down at display 110, passport reader 112, or code reader 114 with his or her line of sight facing diagonally downward and forward, without user U being aware that he or she is being photographed. For example, when camera 116 is photographing, there is no need to request user U to face camera 116.

[0194] Furthermore, camera 116 can capture multiple facial images of user U operating kiosk terminal 10 by capturing video at a predetermined frame rate or capturing still images at predetermined time intervals. In this case, CPU 102 can select and capture the facial image with the highest quality from the multiple facial images of user U captured by camera 116 as the captured facial image of user U. Camera 116 can start capturing images when triggered by the holding of a passport or declaration code. The quality of the facial image can be evaluated based on, for example, the orientation of the face relative to camera 116, and when the orientation of the face relative to camera 116 is within a predetermined range, the facial image can be evaluated as having a certain level of quality or higher.

[0195] The CPU 102 acquires the passport face image of the user U from the passport and acquires the photographed face image of the user U by the camera 116, and then transmits a matching request to the management server 50 via the network NW to request face matching of the user U (step S110). The CPU 102 transmits the photographed face image and the passport face image or their facial features together with the matching request to the management server 50.

[0196] When the CPU 502 of the management server 50 receives the matching request from the kiosk terminal 10, it performs face matching of the user U (step S112). The CPU 502 matches the photographed face image with the face image on the passport on a one-to-one basis.

[0197] Next, the CPU 502 transmits collation information indicating the result of collation between the photographed face image and the passport face image to the kiosk terminal 10 via the network NW (step S114).

[0198] In this way, while the user U is operating the kiosk terminal 10, face verification of the user U is performed by verifying the photographed face image with the face image on the passport.

[0199] If the comparison between the photographed facial image and the passport facial image does not match, the kiosk terminal 10 can re-take the photographed facial image by executing step S108 again to photograph the face of the user U using the camera 116 and obtain the photographed facial image of the user U. The kiosk terminal 10 can re-take the photographed facial image only once or multiple times.

[0200] Furthermore, the CPU 102 sets the language used in the kiosk terminal 10 so that the user U can check the declaration content and correct it as necessary (step S116). In setting the language used, the CPU 102 first obtains the language information read from the declaration code, as shown in Fig. 17 (step S202).

[0201] Next, CPU 102 sets the language of the customs declaration application indicated in the acquired language information as the language of kiosk terminal 10 (step S204). Here, the language of the customs declaration application set as the language of kiosk terminal 10 is the language set in the language settings of mobile terminal 60, or if a language is set in the settings within the customs declaration application, that language. This allows CPU 102 to display various information on display 110 in the same language as the language of the customs declaration application of mobile terminal 60 used by user U. Furthermore, CPU 102 can accept input by user U from input device 108, such as a touch panel, incorporated in display 110 in the same language as the language of the customs declaration application of mobile terminal 60 used by user U.

[0202] Furthermore, CPU 102 determines whether or not a change instruction to change the used language to another language has been input by user U via input device 108 (step S206). If a change instruction has not been input (step S206, NO), CPU 102 maintains the used language set in step S204.

[0203] If a change instruction is input (step S206, YES), the CPU 102 changes the language used in the kiosk terminal 10 to another language instructed by the change instruction (step S208).

[0204] In this way, even if the user U is a foreigner, he or she can check the displayed content and input information in the language he or she uses or the language he or she has instructed to change to on the kiosk terminal 10. This allows the user U to use the kiosk terminal 10 smoothly without language being an obstacle.

[0205] After setting the language used in the kiosk terminal 10, the CPU 102 displays the declaration details read from the declaration code on the display 110 as shown in Fig. 16 for the user U to confirm, and accepts corrections from the user U as necessary (step S118). The user U confirms the declaration details displayed on the display 110, and if any corrections are necessary, corrects the declaration details via the input device 108. After confirming the declaration details, the user U inputs a confirmation instruction via the input device 108 to confirm and confirm the declaration details. This finalizes the declaration details to be declared by the user U in the customs declaration.

[0206] 18B shows an example of a declaration content confirmation screen SKC displayed on the display 110 of the kiosk terminal 10, on which the user U confirms the declaration content and corrects it as necessary. The declaration content confirmation screen SKC displays the declaration content included in the declaration information. For example, when the user U touches a correction button on the declaration content confirmation screen SKC, the screen can transition to a declaration content correction screen on which the declaration content can be corrected. Furthermore, when the user U touches a confirmation button on the declaration content confirmation screen SKC, a confirmation instruction is input to the kiosk terminal 10. This finalizes the declaration content declared by the user U in the customs declaration.

[0207] 15A, which is displayed on the display 610 of the mobile terminal 60. The layout of the two screens is unified, making it possible to realize a screen display that makes it easy to operate the kiosk terminal 10.

[0208] 16, the CPU 102 transmits the user information of the user U to the management server 50 via the network NW (step S120). The user information includes the user U's personal information, facial information, and declaration information, which are associated with each other. Note that the CPU 102 transmits the photographed facial image and the passport facial image together with the matching request in step S110, so there is no need to retransmit the facial information included in the user information.

[0209] Next, the CPU 502 determines the declaration content included in the declaration information of the user information of the user U (step S122). The CPU 502 determines, from the declaration content included in the declaration information of the user U, whether the user U is subject to taxation and whether the user U is at risk.

[0210] Furthermore, in addition to determining the declared contents, the CPU 502 can also check the passport number. In this case, the CPU 502 checks the passport number included in the identity information of the user information against the passport number included in the declared information of the user information.

[0211] Next, the CPU 502 transmits determination information indicating the determination result of the declaration content included in the declaration information of the user U to the kiosk terminal 10 via the network NW (step S124). The CPU 502 can also include the passport number matching result in the determination information.

[0212] When the CPU 102 of the kiosk terminal 10 receives the determination information from the management server 50, it determines whether or not the user U can use the electronic gate 20 based on the received determination information and the previously received matching information (step S126). The previously received matching information was sent from the management server 50 in step S114. The CPU 102 can display the matching information indicating the result of matching the photographed facial image with the passport facial image, and the determination information indicating the result of the determination of the declared contents, on the display 110 to show the user U.

[0213] If the face image on user U's passport matches the photographed face image, user U is not subject to taxation, and user U poses no risk, CPU 102 determines that user U can use electronic gate 20. If the determination information includes a passport number matching result, CPU 102 determines that user U can use electronic gate 20 if the face image on user U's passport matches the photographed face image, user U is not subject to taxation, user U poses no risk, and the passport numbers match.

[0214] On the other hand, if the passport facial image of user U does not match the photographed facial image, if user U is subject to taxation, or if user U is at risk, CPU 102 determines that user U cannot use the electronic gate 20. Furthermore, if the determination information includes the passport number collation result, CPU 102 also determines that user U cannot use the electronic gate 20 if the passport numbers do not match.

[0215] The CPU 102 transmits gate availability information indicating the determination result of whether the electronic gate 20 can be used to the management server 50 (step S128). The gate availability information indicates that the user U can use the electronic gate 20 if the face image on the user U's passport matches the photographed face image, the user U is not subject to taxation, and there is no risk to the user U. Furthermore, if the determination information includes the passport number collation result, the gate availability information indicates that the user U can use the electronic gate 20 if the passport numbers also match.

[0216] When the CPU 502 of the management server 50 receives the gate availability information from the kiosk terminal 10, it registers the user information of the user U in the user information DB 506a in association with the gate availability information (step S130). Note that instead of receiving the gate availability information from the kiosk terminal 10, the CPU 502 can also determine whether the user U can use the electronic gate 20 based on the collation information and determination information, similar to the CPU 102 of the kiosk terminal 10.

[0217] Furthermore, instead of registering user information in the user information DB 506a in association with gate availability information, the CPU 502 can also register the user information of a specific user U in the user information DB 506a. That is, the CPU 502 can also register the user information of a user U who is determined to be able to use the electronic gate 20 based on the gate availability information in the user information DB 506a. That is, the CPU 502 can also not register the user information of users U other than the specific user U in the user information DB 506a.

[0218] Next, when the CPU 102 of the kiosk terminal 10 determines that the user U can use the electronic gate 20 (step S132, YES), it executes a process to guide the user U to the electronic gate 20 (step S134). The CPU 102, for example, displays a guidance screen on the display 110 to guide the user U to the electronic gate 20.

[0219] 18C shows an example of a guidance screen SKG displayed on the display 110 of the kiosk terminal 10 to guide the user U to the electronic gate 20. The guidance screen SKG shows, for example, words, symbols, pictures, etc. that guide the user U to the electronic gate 20, as well as words, symbols, pictures, etc. that indicate that the customs declaration has been completed.

[0220] On the other hand, when the CPU 102 determines that the user U cannot use the electronic gate 20 (step S132, NO), it executes a response process for the user U who cannot use the electronic gate 20 (step S136). As the response process, the CPU 102, for example, displays a guidance screen on the display 110 that guides the user U to the manned booth M, thereby guiding the user U to the manned booth M.

[0221] In this way, the user U makes a customs declaration at the kiosk terminal 10.

[0222] Next, the operation of the electronic gate 20 to which the user U is guided on the kiosk terminal 10 will be described together with the operation of the management server 50 using Figures 19 and 20. Figure 19 is a sequence diagram showing the operation of the entrance gate terminal 30 and exit gate terminal 40 that make up the electronic gate 20, and the management server 50.

[0223] In the standby state of the electronic gate 20 where no user U has entered the gate passage P, the entrance gate door 308 of the entrance gate terminal 30 is in an open state, and the exit gate door 414 of the exit gate terminal 40 is in a closed state. Furthermore, in the standby state, the information display 314 of the entrance gate terminal 30 displays a message indicating that entry into the gate passage P is permitted, and the information display 420 of the exit gate terminal 40 displays a message indicating that exit from the gate passage P is prohibited.

[0224] After completing the customs declaration at the kiosk terminal 10, the user U is guided to the electronic gate 20 and enters the gate passage P through the entrance gate terminal 30 whose entrance gate door 308 is in an open state.

[0225] 19, in the standby state, the CPU 302 of the entrance gate terminal 30 determines whether or not the user U has passed through the entrance gate terminal 30 and entered the gate passage P (step S302), and waits for the user U to pass through (step S302, NO). The CPU 302 can determine whether or not the user U has passed through the entrance gate terminal 30 and entered the gate passage P based on the output signals of the multiple passage detection sensors 310 and the order in which they are output.

[0226] When the CPU 302 determines that the user U has passed through the entrance gate terminal 30 and entered the gate passage P (step S302, YES), it closes the entrance gate door 308 (step S304). In addition, the CPU 302 changes the display on the guidance display 314 from a display indicating that entry into the gate passage P is permitted to a display indicating that entry into the gate passage P is prohibited.

[0227] As will be described later, the entrance gate door 308 is controlled so as not to open until a user U who has entered the gate passage P passes through the exit gate terminal 40 and exits from the gate passage P. Therefore, only one user U can enter the gate passage P. This prevents multiple users U from being photographed by the first camera 410, and ensures that the first target face image can be matched reliably.

[0228] Furthermore, the CPU 302 transmits entrance passage information indicating that the user U has passed through the entrance gate terminal 30 and entered the gate passage P to the exit gate terminal 40 (step S306). The CPU 302 can execute either step S304 or step S306 in the order that comes first, or can execute both steps S304 and S306 simultaneously.

[0229] The CPU 402 of the exit gate terminal 40, which has received the entrance passage information from the entrance gate terminal 30, uses the first camera 410 to capture an image of the user U moving through the gate passage P toward the exit gate terminal 40, thereby acquiring a first target face image of the user U (step S308). The first camera 410 captures moving images at a predetermined frame rate or still images at predetermined time intervals, thereby acquiring a plurality of first target face images of the user U moving through the gate passage P. In this way, a first target face image is acquired, which is a facial image of the user U moving through the gate passage P. While the first camera 410 is capturing images, the user U can move toward the exit gate terminal 40 without stopping in the gate passage P.

[0230] While the first camera 410 is capturing an image, the CPU 402 performs gaze detection on the first target face image, and if it determines as a result of the gaze detection that the user U is looking down or to the side, it can warn the user U to face forward, which is the side of the exit gate terminal 40. The CPU 402 can warn the user U to face forward, for example, by displaying the warning on the display 408 or by sound from a speaker or the like. This makes it possible to more reliably acquire a high-quality first target face image.

[0231] If the user U does not face forward despite the above warning, the CPU 402 may not perform the quality determination in the following step S310 or the matching request in step S312, and may not perform matching of the first target face image.

[0232] Furthermore, while the first camera 410 is capturing an image, the CPU 402 performs accessory estimation on the captured first target face image and attempts to detect face accessories such as sunglasses, glasses, and hats in the first target face image. If the CPU 402 detects face accessories, it can issue a warning to the user U, for example, by displaying a message on the display 408 or by issuing a voice message from a speaker, instructing the user U to remove the face accessories. This can eliminate factors that hinder matching of the first target face image.

[0233] If the face attachment is not removed from the face of the user U despite the above warning, the CPU 402 may not perform the quality judgment in the following step S310 or the matching request in step S312, and may not perform matching of the first target face image.

[0234] The CPU 402 calculates a quality value for each of the acquired first target face images, and determines the quality of the first target face image based on the calculated quality value (step S310).

[0235] As a result of the quality determination, the CPU 402 sequentially transmits a matching request to the management server 50 via the network NW for first target face images having a quality equal to or higher than a certain level (step S312). As a result, the CPU 402 requests the management server 50 to perform a 1:N matching of the first target face images having a quality equal to or higher than a certain level with a plurality of registered face images registered in the user information DB 506a of the management server 50. The CPU 402 transmits the facial feature amounts extracted from the first target face images or the first target face images themselves to the management server 50 together with the matching request.

[0236] The CPU 402 may transmit all of the acquired first target face images to the management server 50 regardless of their quality. In this case, the CPU 502 of the management server 50 may similarly determine the quality of the first target face images and compare the first target face images of a certain quality or higher with the registered face images. The CPU 502 may also compare the first target face images with the registered face images regardless of their quality.

[0237] When the CPU 502 of the management server 50 receives a request to compare the first target face image from the exit gate terminal 40, it compares the first target face image (step S314). The CPU 502 compares the facial feature amounts of the first target face image received from the exit gate terminal 40 with the facial feature amounts of multiple registered face images registered in the user information DB 506a in a 1:N ratio. The facial feature amounts of the first target face image are those received from the exit gate terminal 40 or those extracted from the first target face image received from the exit gate terminal 40.

[0238] Next, CPU 502 transmits matching information indicating the result of matching the first target face image to exit gate terminal 40 via network NW (step S316).

[0239] The CPU 402 of the exit gate terminal 40 and the CPU 502 of the management server 50 can repeatedly execute the above steps S312, S314, and S316 for a plurality of first target face images of a certain quality or higher.

[0240] The CPU 402 of the exit gate terminal 40 receives the matching information from the management server 50. When the matching information for any of the first target face images indicates that a matching registered face image has been found and the identity of the user U moving through the gate passage P has been successfully verified (step S318, YES), the CPU 402 ends the photographing by the first camera 410. Then, the CPU 402 proceeds to step S336.

[0241] On the other hand, if the matching information for any of the first target face images indicates that a matching registered face image has not been found and identity verification of user U has failed by the predetermined time point (step S318, NO), CPU 402 determines that identity verification by matching the first target face image has failed. Note that the predetermined time point is, for example, the time point when user U reaches exit gate door 414 of exit gate terminal 40. CPU 402 can determine whether user U has reached exit gate door 414 from the image captured by first camera 410.

[0242] Furthermore, the CPU 402 can also determine that identity verification by matching the first target face image has failed if matching information is not received by the same predetermined time point as above. In this case, a first target face image of a certain quality or higher cannot be acquired, and the first target face image cannot be matched with a registered face image.

[0243] If the CPU 402 determines that identity verification by matching the first target face image has failed, it notifies the user U that an image will be taken with the second camera 412 (step S320). That is, if the first target face image does not match or cannot be matched with the registered face image, the CPU 402 notifies the user U that an image will be taken with the second camera 412. The CPU 402 can notify the user U that an image will be taken with the second camera 412, for example, by displaying on the display 408, providing audio guidance, or the like.

[0244] Next, the CPU 402 receives the notification and causes the second camera 412 to photograph the user U who remains in front of the exit gate door 414 (step S322). As a result, the CPU 402 acquires a second target face image of the user U who has arrived in front of the exit gate door 414 of the exit gate terminal 40 and remains in front of the exit gate door 414. In this way, the second target face image, which is a face image of the user U who remains stationary in front of the exit gate door 414, is acquired.

[0245] The second camera 412 can capture a plurality of facial images of the user U in front of the exit gate door 414 by taking a video at a predetermined frame rate or taking a still image at a predetermined time interval. In this case, the CPU 402 can select and acquire the facial image with the highest quality from the plurality of facial images of the user U taken by the second camera 412 as the second target facial image of the user U.

[0246] When capturing an image using the second camera 412, the CPU 402 can provide guidance to the user U, for example, by displaying on the display 408 or by audio from a speaker, etc. For example, if a face attachment is detected in the second target face image, the CPU 402 can guide the user U to remove the face attachment, to point the second camera 412 in the direction of the display 408, etc.

[0247] 20 shows an example of a guidance display displayed on display 408 when capturing an image with second camera 412. As shown in the figure, the display on display 408 can guide the user to remove a mask, which is a face attachment.

[0248] Next, the CPU 402 transmits a matching request for the second target face image to the management server 50 via the network NW (step S324). As a result, the CPU 402 requests the management server 50 to match the second target face image with a plurality of registered face images registered in the user information DB 506a of the management server 50 in a 1:N ratio. The CPU 402 transmits the facial feature amounts extracted from the second target face image or the second target face image itself to the management server 50 together with the matching request.

[0249] When the CPU 502 of the management server 50 receives the request to compare the second target face image from the exit gate terminal 40, it compares the second target face image (step S326). The CPU 502 compares the facial feature amounts of the second target face image received from the exit gate terminal 40 with the feature amounts of multiple registered face images registered in the user information DB 506a in a 1:N ratio. The facial feature amounts of the second target face image are those received from the exit gate terminal 40 or those extracted from the second target face image received from the exit gate terminal 40.

[0250] Next, CPU 502 transmits matching information indicating the result of matching the second target face image to exit gate terminal 40 via network NW (step S328).

[0251] The CPU 402 of the exit gate terminal 40 receives the matching information from the management server 50. If the matching information for the second target face image indicates that a matching registered face image has been found and the identity of the user U in front of the exit gate door 414 has been successfully verified (step S330, YES), the CPU 402 proceeds to step S336.

[0252] If the second target face image does not match the registered face image, the exit gate terminal 40 can re-capture the second target face image by executing step S322 again to photograph the user U with the second camera 412 and obtain the second target face image of the user U. The exit gate terminal 40 can re-capture the second target face image only once or multiple times.

[0253] Furthermore, when the CPU 502 of the management server 50 has successfully identified the user U as a result of the matching of the first target face image or the second target face image (step S332, YES), it transmits gate availability information (step S334). That is, the CPU 502 obtains gate availability information associated with the user information of the user U whose identity has been successfully identified from the user information DB 506a, and transmits the information to the exit gate terminal 40 via the network NW.

[0254] If the matching information for the second target face image indicates that a matching registered face image was not found and identity verification of user U failed (step S330, NO), the CPU 402 of the exit gate terminal 40 executes a response process (step S338). In this case, identity verification of user U has failed both by matching the first target face image and the second target face image. As a response process, the CPU 402 can sound an alarm or send a notification to the customs officer's terminal to call a customs officer to the gate passage P. In this case, the customs officer can, for example, guide user U to a manned booth M or a separate room. Furthermore, after conducting a face-to-face customs inspection at an inspection table installed to the side of the electronic gate 20, the customs officer can allow user U to exit the gate passage P through the manually opened exit gate door 414.

[0255] On the other hand, when the CPU 402 has successfully verified the identity of the user U by matching the first target face image or the second target face image and has proceeded to step S336, it determines whether the user U whose identity has been successfully verified can use the electronic gate 20 (step S336). The CPU 402 determines whether the electronic gate 20 can be used based on the gate availability information received from the management server 50.

[0256] When the CPU 402 determines that the electronic gate 20 cannot be used (step S336, NO), it executes a response process (step S338). In this case, although the identity of the user U has been successfully verified by matching the first target face image or the second target face image, the user U is determined to be a person who is subject to taxation, a person who is determined to be at risk, or the like. As a response process, the CPU 402 can sound an alarm or send a notification to the customs officer's terminal to call a customs officer to the gate passage P. In this case, the customs officer can also respond to the user U in the same manner as described above.

[0257] On the other hand, when the CPU 402 determines that the electronic gate 20 can be used (step S336, YES), it opens the exit gate door 414 (step S340). In addition, the CPU 402 changes the display on the guidance display 420 from a display indicating that exit from the gate passage P is prohibited to a display indicating that exit from the gate passage P is permitted.

[0258] Next, the CPU 402 determines whether the user U has passed through the exit gate terminal 40 and exited from the gate passage P (step S342), and waits for the user U to pass through (step S342, NO). The CPU 402 can determine whether the user U has passed through the exit gate terminal 40 and exited from the gate passage P based on the output signals of the multiple passage detection sensors 416 and the order in which they are output.

[0259] When the CPU 402 determines that the user U has passed through the exit gate terminal 40 and exited through the gate passage P (step S342, YES), it closes the exit gate door 414 (step S344). In addition, the CPU 402 changes the display on the guidance display 420 from a display indicating that exit from the gate passage P is permitted to a display indicating that exit from the gate passage P is prohibited.

[0260] The CPU 402 also transmits exit passage information to the entrance gate terminal 30, indicating that the user U has passed through the exit gate terminal 40 and exited through the gate passage P (step S346). The CPU 402 can execute either step S344 or step S346 in any order, or can execute both steps S344 and S346 simultaneously.

[0261] Upon receiving the exit passage information from the exit gate terminal 40, the CPU 302 of the entrance gate terminal 30 opens the entrance gate door 308 (step S348). The CPU 302 also changes the display on the guidance display 314 from a display indicating that entry into the gate passage P is prohibited to a display indicating that entry into the gate passage P is permitted.

[0262] In this way, the electronic gate 20 transitions to a standby state in which it waits for the user U to enter the gate passage P.

[0263] As described above, according to this embodiment, a photographed face image to be used for face matching, which is a biometric information match, is acquired at the kiosk terminal 10 while the passport or declaration code is being read. Therefore, according to this embodiment, the procedure at the kiosk terminal 10, which includes acquiring information by reading the passport and the declaration code, and face matching using the photographed face image, can be completed in a short time.

[0264] Furthermore, according to this embodiment, at the exit gate terminal 40 of the electronic gate 20, a first camera 410 photographs a user U moving toward the exit gate terminal 40, and a first target face image is acquired to be used for face matching, which is a matching of biometric information. Furthermore, according to this embodiment, the exit gate door 414 is opened according to a predetermined determination result of whether the gate can be used. Therefore, according to this embodiment, the time required for the user U to pass through the electronic gate 20 can be shortened.

[0265] [Other embodiments] According to another embodiment, the gate device described in the above embodiment can also be configured as shown in Figures 21 and 22. Figures 21 and 22 are schematic diagrams showing the configuration of an information processing device according to another embodiment.

[0266] As shown in FIG. 21 , a gate apparatus 1000 according to another embodiment includes an exit gate door 1002, a first biometric information acquisition unit 1004, a second biometric information acquisition unit 1006, and a door control unit 1008. The first biometric information acquisition unit 1004 acquires first target biometric information from a user moving toward the closed exit gate door 1002, the first target biometric information being compared with pre-registered biometric information. If the first target biometric information does not match the registered biometric information or if matching is not possible, the second biometric information acquisition unit 1006 acquires second target biometric information from a user remaining in front of the exit gate door 1002, the second target biometric information being compared with the registered biometric information. The door control unit 1008 opens the closed exit gate door 1002 in accordance with the result of comparing the first target biometric information or the second target biometric information with the registered biometric information.

[0267] In addition to the above, the second biometric information acquisition unit 1006 can also be configured to acquire, in a predetermined case, second target biometric information to be compared with the registered biometric information from a user who remains in front of the exit gate door 1002. The predetermined case here refers to a case where the face of a user moving toward the exit gate door 1002 in a closed state is not facing the exit gate door 1002, or a case where the user is wearing an attachment on their face.

[0268] According to another embodiment of the gate device 1000, first target biometric information is obtained from a user moving toward the exit gate door 1002 in a closed state, and is compared with pre-registered registered biometric information, thereby reducing the time required for the user to pass through.

[0269] 22, a gate device 2000 according to another embodiment includes an exit gate door 2002, a biometric information acquisition unit 2004, and a door control unit 2006. The biometric information acquisition unit 2004 acquires target biometric information from a user moving toward or remaining in front of a closed exit gate door, to be compared with pre-registered registered biometric information. The door control unit 2006 opens the closed exit gate door 2002 in accordance with the determination result of the declared information associated with the registered biometric information that matches the target biometric information.

[0270] According to another embodiment of the gate device 2000, the exit gate door 2002 that is in a closed state is opened depending on the judgment result of the declared information associated with the registered biometric information that matches the target biometric information, thereby reducing the time required for the user to pass through.

[0271] [Modified embodiment] The present invention is not limited to the above-described embodiment, and various modifications are possible.

[0272] For example, in the above embodiment, an example has been described in which a customs declaration is made at the kiosk terminal 10, but the present invention is not limited to this. The kiosk terminal 10 can be configured to accept various declarations from users.

[0273] In the above embodiment, the case where camera 116, first camera 410, and second camera 412 capture and acquire a facial image of user U as biometric information of user U has been described as an example, but the present invention is not limited to this. Camera 116, first camera 410, and second camera 412 may acquire an iris image or the like as biometric information of user U in addition to a facial image, which is facial information.

[0274] Furthermore, the kiosk terminal 10 according to the above embodiment can be configured as a system consisting of one or more devices. The entrance gate terminal 30 according to the above embodiment can be configured as a system consisting of one or more devices. The exit gate terminal 40 according to the above embodiment can be configured as a system consisting of one or more devices. The management server 50 according to the above embodiment can be configured as a system consisting of one or more devices.

[0275] Furthermore, the scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described computer program is recorded, but also the computer program itself.

[0276] Examples of the recording medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM (Compact Disc-Read Only Memory), a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment is not limited to programs that execute processing by themselves recorded on the recording medium, but also includes programs that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0277] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0278] (Appendix 1) The exit gate and a first biometric information acquisition unit that acquires, from a user moving toward the exit gate door in a closed state, first target biometric information to be compared with pre-registered registered biometric information; a second biometric information acquisition unit that acquires second target biometric information to be matched with the registered biometric information from the user who remains in front of the exit gate door when the first target biometric information does not match the registered biometric information or when the matching is not possible; a door control unit that opens the exit gate door that is in the closed state in accordance with a result of matching the first target biometric information or the second target biometric information with the registered biometric information; A gate device having:

[0279] (Appendix 2) The gate device described in Appendix 1, wherein the door control unit opens the exit gate door in the closed state depending on the judgment result of the declared information associated with the registered biometric information that matches the first target biometric information or the second target biometric information.

[0280] (Appendix 3) 3. The gate device according to claim 2, wherein the declaration information is information required for a customs declaration.

[0281] (Appendix 4) The exit gate door is installed at the exit of the passageway, 4. A gate device according to any one of appendices 1 to 3, comprising an entrance gate door that is installed at the entrance of the passage and that closes when the user enters the passage, and that opens when the user exits the passage, with the exit gate door being opened.

[0282] (Appendix 5) the first biometric information acquisition unit acquires, as the first target biometric information, first target face information to be matched with registered face information as the registered biometric information from the user; 5. A gate device according to any one of claims 1 to 4, comprising a display unit that displays information according to the facial state of the user moving toward the exit gate door in the closed state.

[0283] (Appendix 6) 5. A gate device according to any one of appendices 1 to 4, comprising a display unit that displays a message to the user to stay in front of the exit gate door when there is a mismatch between the first target biometric information and the registered biometric information or when the matching is not possible.

[0284] (Appendix 7) The gate device according to claim 6, wherein the display unit displays a message to the user to stay in front of the exit gate door and look at the display unit when there is a mismatch between the first target biometric information and the registered biometric information or when the matching is not possible.

[0285] (Appendix 8) 6. The gate device according to claim 5, wherein the display unit performs the display in accordance with a line of sight of the user detected from the first target face information.

[0286] (Appendix 9) 9. The gate device according to claim 8, wherein the display unit displays the display instructing the user to change the direction of the face.

[0287] (Appendix 10) 10. The gate device according to any one of claims 5, 8 and 9, wherein the display unit performs the display when an attachment worn on the face of the user is detected.

[0288] (Appendix 11) The gate device according to any one of appendices 5, 8 and 9, wherein the display unit, when detecting an attachment worn on the face of the user, displays a message to advise the user to remove the attachment.

[0289] (Appendix 12) the first biological information acquisition unit acquires a plurality of pieces of first target biological information; 12. A gate device according to any one of appendices 1 to 11, comprising a control unit that performs or requests matching of the first target biometric information of a certain quality or higher among the plurality of first target biometric information with the registered biometric information.

[0290] (Appendix 13) The exit gate and a first biometric information acquisition unit that acquires, from a user moving toward the exit gate door in a closed state, first target biometric information to be compared with pre-registered registered biometric information; a second biometric information acquisition unit that acquires second target biometric information to be compared with the registered biometric information from the user who remains in front of the exit gate door when the face of the user who is moving toward the exit gate door in the closed state is not facing the exit gate door or when the user is wearing an attachment on the face; a door control unit that opens the exit gate door that is in the closed state in accordance with a result of matching the first target biometric information or the second target biometric information with the registered biometric information; A gate device having:

[0291] (Appendix 14) The exit gate and a biometric information acquisition unit that acquires target biometric information to be compared with pre-registered registered biometric information from a user who moves toward the exit gate door in a closed state or who stays in front of the exit gate door; a door control unit that opens the exit gate door in the closed state in accordance with a determination result of declared information associated with the registered biometric information that matches the target biometric information; A gate device having:

[0292] (Appendix 15) A method for controlling a gate device having an exit gate door, comprising: acquiring first target biometric information from a user moving toward the exit gate door in a closed state, the first target biometric information being compared with pre-registered registered biometric information; If the first target biometric information does not match the registered biometric information or if the matching is not possible, obtain second target biometric information to be matched with the registered biometric information from the user who remains in front of the exit gate door; A gate device control method for opening the exit gate door in the closed state in accordance with the result of matching the first target biometric information or the second target biometric information with the registered biometric information.

[0293] (Appendix 16) A gate device having an exit gate door, acquiring first target biometric information from a user moving toward the exit gate door in a closed state, the first target biometric information being compared with pre-registered registered biometric information; If the first target biometric information does not match the registered biometric information or if the matching is not possible, obtain second target biometric information to be matched with the registered biometric information from the user who remains in front of the exit gate door; Opening the exit gate door in the closed state according to a result of matching the first target biometric information or the second target biometric information with the registered biometric information. A recording medium on which a program to perform the following is recorded. [Explanation of symbols]

[0294] 1. Information processing system 10...Kiosk terminal 20...Electronic gate 30...Entrance gate terminal 40...Exit gate terminal 50...Administrative server 60...Mobile devices

Claims

1. a first reading unit that reads a first medium containing biometric information and acquires the biometric information of a user from the first medium; a second reading unit that reads a code displayed on a second medium that is held or carried by the user and that acquires reporting information from the code; an output unit that outputs information that prompts the user to confirm or correct the acquired declaration information; a control unit that determines whether the user is permitted to use the gate device based on a comparison result between the biometric information and photographed biometric information of the user and a determination result regarding the declared information; and A registration device comprising:

2. A registration device, a first reading unit that reads a first medium containing biometric information and acquires the biometric information of a user from the first medium; a second reading unit that reads a code displayed on a second medium and obtains, from the code, declaration information and language information indicating a language used by an application included in the second medium; a control unit that sets the language used by the application indicated by the language information as the language used by the registration device, and determines whether the user is permitted to use the gate device based on a comparison result between the biometric information and photographed biometric information of the user and a determination result regarding the declared information; A registration device comprising:

3. the control unit determines that the user is permitted to use the gate device when the biometric information matches the captured biometric information and the declaration content included in the declaration information satisfies a predetermined condition. The registration device according to claim 1 or 2.

4. the control unit causes the biometric information, the declaration information, and a determination result of whether or not the gate device is available to be associated with each other and registered in the server device. The registration device according to any one of claims 1 to 3.

5. a first imaging unit that captures an image of the user and acquires the captured biometric information; a second imaging unit that acquires three-dimensional information of the user, the control unit determines, based on the three-dimensional information, whether the photographed biometric information has been acquired by photographing a living body. The registration device according to any one of claims 1 to 4.

6. a status display unit that displays the status of the registration device; the status display unit performs different lighting based on at least one of the progress of the determination regarding the declared information of the user, the presence or absence of an abnormality in the registered device, and the availability or unavailability of the registered device. The registration device according to any one of claims 1 to 5.

7. 1. A computer-implemented method comprising: reading a first medium containing biometric information and acquiring the biometric information of the user from the first medium; reading a code displayed on a second medium that is held or carried by the user and used, and acquiring declaration information from the code; outputting information prompting confirmation or correction of the acquired declaration information; A method for determining whether the user is permitted to use the gate device based on the result of matching the biometric information with photographed biometric information of the user and the result of determination regarding the declared information.

8. 1. A computer-implemented method comprising: reading a first medium containing biometric information and acquiring the biometric information of the user from the first medium; reading a code displayed on a second medium, and obtaining, from the code, declaration information and language information indicating a language used by an application included in the second medium; A method for determining whether the user can use the gate device based on the result of comparing the biometric information with the biometric information captured by photographing the user and the result of determining the declared information, by setting the language used by the application indicated by the language information as the language used by the computer.

9. On the computer, reading a first medium containing biometric information and acquiring the biometric information of the user from the first medium; reading a code displayed on a second medium that is held or carried by the user and used, and acquiring declaration information from the code; outputting information prompting confirmation or correction of the acquired declaration information; A program that executes a process of determining whether the user is permitted to use the gate device based on the result of comparing the biometric information with photographed biometric information of the user and the result of a determination regarding the declared information.

10. On the computer, reading a first medium containing biometric information and acquiring the biometric information of the user from the first medium; reading a code displayed on a second medium, and obtaining, from the code, declaration information and language information indicating a language used by an application included in the second medium; A program that sets the language used by the application indicated by the language information as the language used by the computer, and determines whether the user can use the gate device based on the results of comparing the biometric information with the photographed biometric information of the user, and the determination result regarding the declared information.

Images