Information processing device, information processing method, and program

Abstract

To provide an information processing device which can easily register and manage a relation between baggage and an owner.SOLUTION: An information processing device comprises: a first acquisition unit which acquires a person's biological information; a second acquisition unit which acquires identification information of baggage possessed by the person; a creation unit which creates link information associating the biological information with the identification information; and a control unit which acquires result of baggage inspection obtained by determining whether or not a target article in which bringing-in to a restricted zone is restricted is included in the baggage, and the result of physical examination obtained by determining whether or not the person is carrying the target article. The control unit determines the transport destination of the baggage on the basis of the inspection result of the baggage inspection, controls the transport of the baggage to the transport destination, and displays information on the transport destination of the baggage on a display terminal, on the basis of collation result of second biological information on the person acquired in the display terminal and the biological information included in the link information.SELECTED DRAWING: Figure 19

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent document 1 describes a baggage management system in which a passenger planning to board an aircraft attaches an RFID tag recording boarding reservation information and baggage information to their baggage, and then entrusts the baggage to a transport robot. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-23363 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the baggage management system exemplified in Patent Document 1, data must be written to and read from a recording medium during the baggage check-in procedure and transportation. For this reason, there has been a demand for a method for easily registering and managing the relationship between baggage and its owner without using a recording medium such as an RFID tag.

[0005] In view of the above-mentioned problems with the conventional technology, an object of the present invention is to provide an information processing device, an information processing method, and a program that can easily register and manage the relationship between baggage and its owner. [Means for solving the problem]

[0006] According to one aspect of the present invention, a first acquisition unit acquires biometric information of a person, a second acquisition unit acquires identification information of baggage carried by the person, a creation unit creates link information that associates the biometric information with the identification information, and a baggage inspection unit determines whether the baggage contains any object that is restricted from being brought into a restricted area.and a physical examination result to determine whether the person is carrying the object item. and a control unit for controlling the baggage inspection. and determining a destination of the baggage based on the result of the comparison between second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information, and displaying information on the destination of the baggage on the signage terminal based on a comparison result between the second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information. An information processing device is provided.

[0007] According to another aspect of the present invention, a method for detecting a person's biometric information by a computer includes: The computer, A step of acquiring identification information of baggage carried by the person, a step of the computer generating link information associating the biometric information with the identification information, and a baggage inspection step of the computer determining whether the baggage contains any object that is restricted from being brought into a restricted area. and a physical examination result to determine whether the person is carrying the object item. and Obtain the results of the baggage inspection and the physical inspection In the step, the computer and determining a destination of the baggage based on the result of the comparison between second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information, and displaying information on the destination of the baggage on the signage terminal based on a comparison result between the second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information. An information processing method is provided.

[0008] According to yet another aspect of the present invention, a computer includes the steps of acquiring biometric information of a person, acquiring identification information of baggage carried by the person, creating link information that associates the biometric information with the identification information, and a baggage inspection that determines whether the baggage contains an object that is restricted from being brought into a restricted area. and a physical examination result to determine whether the person is carrying the object item. and executing the steps of: Obtain the results of the baggage inspection and the physical inspection In the step of and determining a destination of the baggage based on the result of the comparison between second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information, and displaying information on the destination of the baggage on the signage terminal based on a comparison result between the second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information. A program will be provided. Effect of the Invention

[0009] According to the present invention, it is possible to provide an information processing device, an information processing method, and a program that can easily register and manage the relationship between baggage and its owner. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of an overall configuration of an information processing system according to a first embodiment. [Diagram 2]FIG. 4 is a diagram illustrating an example of information stored in a token ID information database in the first embodiment. [Diagram 3] 5 is a diagram showing an example of information stored in a passage history information database in the first embodiment. FIG. [Figure 4] FIG. 4 is a diagram illustrating an example of information stored in a business information database in the first embodiment. [Diagram 5] 1 is a block diagram showing an example of the overall configuration of a security inspection system in a first embodiment. [Figure 6] FIG. 2 is a schematic diagram showing an example of device layout within a security checkpoint in the first embodiment. [Figure 7] FIG. 4 is a diagram showing an example of information stored in a security inspection information database in the first embodiment. [Figure 8] FIG. 2 is a block diagram showing an example of a hardware configuration of an edge server in the first embodiment. [Figure 9] 6 is a flowchart showing an example of a process of an edge server in the first embodiment. [Figure 10] FIG. 2 is a diagram showing an example of an image captured at the start of security inspection in the first embodiment. [Figure 11] 13 is a flowchart showing an example of a process performed by the edge server during baggage inspection in the first embodiment. [Figure 12] FIG. 11 is a diagram showing an example of a screen displayed on a business terminal on the baggage inspection area side in the first embodiment. [Figure 13] FIG. 2 is a diagram showing an example of a screen displayed on a service terminal in a physical examination area in the first embodiment. [Figure 14] 6 is a flowchart showing an example of a process of an edge server in the first embodiment. [Figure 15] 13 is a flowchart showing an example of a process performed by the edge server during a physical examination in the first embodiment. [Figure 16] FIG. 2 is a diagram showing an example of a screen displayed on a service terminal in a physical examination area in the first embodiment. [Figure 17]FIG. 11 is a diagram showing an example of a screen displayed on a business terminal on the baggage inspection area side in the first embodiment. [Figure 18] 6 is a flowchart showing an example of a process of an edge server in the first embodiment. [Figure 19] FIG. 4 is a sequence diagram showing an example of processing between an edge server and a signage terminal in the first embodiment. [Figure 20] FIG. 4 is a diagram showing an example of a screen displayed on a signage terminal in the first embodiment. [Figure 21] FIG. 4 is a diagram showing an example of a screen displayed on a signage terminal in the first embodiment. [Figure 22] FIG. 11 is a sequence diagram showing an example of a process between an edge server and a storage box in the first embodiment. [Figure 23] FIG. 2 is a front view showing an example of the configuration of the storage box in the first embodiment. [Figure 24] FIG. 11 is a block diagram showing an example of an overall configuration of an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, exemplary embodiments of the present invention will be described with reference to the drawings. In the drawings, similar or corresponding elements are designated by the same reference numerals, and descriptions thereof may be omitted or simplified.

[0012] [First embodiment] 1 is a block diagram showing an example of the overall configuration of an information processing system 1 in this embodiment. The information processing system 1 is a computer system that supports operations related to a series of screening procedures performed on a user (traveler) U who uses airport A. The information processing system 1 is operated, for example, by a public institution such as an immigration control bureau or a contractor entrusted with the operations by the public institution.

[0013] The information processing system 1 of this embodiment includes a management server 10, a check-in terminal 20, an automated baggage drop machine 30, a security inspection system 40, an automated gate device 50, and a boarding gate device 60. The management server 10 and the other devices are connected to each other via a network NW. The network NW is composed of a LAN (Local Area Network) including an in-house communication network of airport A, a WAN (Wide Area Network), a mobile communication network, etc. The connection method is not limited to a wired method, and may be a wireless method.

[0014] For ease of explanation, Fig. 1 only illustrates a terminal device (business terminal) used for the procedures at the time of departure from airport A. The screening procedures at the time of departure at airport A are carried out sequentially at five touch points P1 to P5 using face recognition technology. Details of each device and touch points P1 to P5 will be described later.

[0015] The management server 10 is a server computer that manages operations related to the screening procedures at the time of entry and departure of the user U. The management server 10 is installed, for example, in the facility of an airport company that operates airport A, an airline, or the like. The management server 10 may be a cloud server rather than a server installed in the facility where the actual operations are performed. The management server 10 does not need to be a single server, and may be configured as a server group including multiple servers.

[0016] 1, the management server 10 includes a token ID information DB 11, a passage history information DB 12, and a business information DB 13. However, the databases included in the management server 10 are not limited to these databases. Furthermore, the data items stored in each database are not limited to those exemplified below.

[0017] 2 is a diagram showing an example of information stored in the token ID information DB 11. The token ID information DB 11 has the following data items: token ID, group ID, registered face image, feature amount, token issuance time, token issuing device name, invalid flag, and invalidation time. The token ID is an identifier that uniquely identifies ID information.

[0018] The token ID in this embodiment is issued temporarily on the condition that a match is found between a facial image of a user U holding a passport taken at touch point P1 and a passport facial image read from the passport. Then, when the user U completes the procedure at the last touch point (boarding gate), the token ID is invalidated. In other words, the token ID is not a permanently used identifier, but a one-time ID with a validity period (life cycle).

[0019] The group ID is an identifier for grouping ID information. The registered face image is a face image registered for the user U. The feature amount is a value extracted from the biometric information (registered face image). Note that the term "biometric information" in this embodiment means a face image and a feature amount extracted from the face image, but the biometric information is not limited to a face image and a face feature amount. In other words, biometric authentication may be performed using an iris image, a fingerprint image, a palm print image, an ear image, etc. as the biometric information of the user U.

[0020] The token issue time is the time when the management server 10 issues the token ID. The token issuing device name is the name of the device from which the registered face image that triggered the issuance of the token ID was acquired. The invalid flag is flag information indicating whether the token ID is currently valid or not. When a token ID is issued, the invalid flag in this embodiment becomes a value indicating that the token ID is valid. Furthermore, when a predetermined condition is satisfied, the invalid flag is updated to a value indicating that the token ID is invalid. The invalid time is a timestamp when the invalid flag is invalidated.

[0021] FIG. 3 is a diagram showing an example of information stored in the passage history information DB12. The passage history information DB12 has the following data items: passage history ID, token ID, passage time, device name, business system type, and passed touch point. The passage history ID is an identifier that uniquely identifies the passage history information. The passage time is a timestamp when the touch point was passed. The device name is the name of the terminal device used for the procedure at the touch point. The business system type is the type of the business system to which the terminal device belongs. The passed touch point is a touch point that user U has passed through. By extracting the passage history information for each token ID, the management server 10 can determine up to which touch point user U has completed the procedure.

[0022] 4 is a diagram showing an example of information stored in the business information DB 13. The business information DB 13 has the following data items: token ID, passenger name, reservation number, departure point, destination, airline code, flight number, operation date, seat number, nationality, passport number, surname, first name, date of birth, sex, membership category, presence or absence of accompanying person, travel history, and security inspection history.

[0023] The reservation number is an identifier that uniquely identifies the boarding reservation information. The airline code is an identifier that uniquely identifies the airline. The boarding reservation information included in the business information includes the passenger's name, reservation number, departure point, destination, airline code, flight number, operation date, seat number, nationality, passport number, surname, first name, date of birth, and gender. The boarding reservation information can be obtained from recording media such as a passport and boarding pass. The boarding reservation information can also be obtained from the airline's reservation system using the passport number or reservation number as a key. The boarding reservation information obtained from the recording media or reservation system is stored as business information in the business information DB13.

[0024] The membership category is the category of airline member. The presence or absence of an escort is the presence or absence of a person (supportee) such as an infant or small child who is boarding an aircraft with the support of user U. The persons for whom the presence or absence of an escort is recorded are guardians such as parents and others, and people who provide support such as caregivers (hereinafter referred to as "supporters"). The travel history is historical information including user U's past travel destinations (destinations), stopovers, travel dates and times, and the number of travels within a specified period. The security inspection history is historical information including the results and dates and times of security inspections that user U underwent during past travels.

[0025] The check-in terminal 20 is installed in a check-in lobby (hereinafter referred to as "touch point P1") in airport A. The check-in terminal 20 is a self-service terminal that allows a user U to carry out check-in procedures (boarding procedures) by operating it himself / herself. After completing the check-in procedures at touch point P1, the user U moves to the baggage drop-off area or security checkpoint.

[0026] The automated baggage check-in machine 30 is installed in an area adjacent to the baggage counter (manned counter) in airport A or in a nearby area of ​​the check-in terminal 20 (hereinafter referred to as "touch point P2"). The automated baggage check-in machine 30 is a self-service terminal that a user U operates to carry on the procedure for checking in baggage that will not be carried on board the aircraft. After completing the baggage check-in procedure, the user U moves to the security inspection area. If the user U does not have baggage to check in, the procedure at touch point P2 is omitted. In the following explanation, the term "baggage" refers to baggage that the user U wishes to carry on board the aircraft, excluding baggage that the user U has checked in at touch point P2.

[0027] The security inspection system 40 is a computer system installed at a security inspection area (hereinafter referred to as "touch point P3") in airport A, and is composed of multiple devices. Fig. 5 is a block diagram showing an example of the overall configuration of the security inspection system 40. Fig. 6 is a schematic diagram showing an example of the arrangement of devices in the security inspection area.

[0028] 5, the security inspection system 40 in this embodiment includes an edge server 401, a passage control device 402, a baggage transport device 403, a camera 404, a baggage inspection device 405, a physical inspection device 406, a business terminal 408, a signage terminal 409, and a storage box 410. The edge server 401 is connected to other devices via a network NW1.

[0029] The edge server 401 is a server computer that manages the execution of security checks, and is connected to the management server 10 via a network NW (see FIG. 1). The edge server 401 may be a cloud server instead of a server installed within an airport facility. The edge server 401 does not have to be a single server, and may be configured as a server group including multiple servers. The edge server 401 may be installed in a section of the security checkpoint.

[0030] As shown in FIG. 5, the edge server 401 includes a security inspection information DB 401a, a link information creation unit 401b, a control unit 401c, an attribute information acquisition unit 401d, a baggage inspection unit 401e, a physical inspection unit 401f, and a matching unit 401g.

[0031] 7 is a diagram showing an example of information stored in the security inspection information DB 401a. The security inspection information DB 401a has data items including a target person ID, a photographed face image, a token ID, a tray ID, a photographing time, a baggage inspection status (not yet performed / in progress / completed), a baggage inspection result (normal / error), a physical inspection status (not yet performed / in progress / completed), a physical inspection result (normal / error), a baggage storage location, a delivery status (before storage / in storage / delivery completed), a passport number, and person attribute information. However, the database held by the edge server 401 is not limited to the security inspection information DB 401a. Furthermore, the data items stored in the security inspection information DB 401a are not limited to those described above.

[0032] Specific examples of the personal attribute information of user U include the security inspection history (inspection result / inspection date and time, etc.) conducted during past travels, travel history (destination / stopover / travel date and time / number of travels, etc.), the gender, age, nationality, whether or not the user U is accompanied, airline membership category, etc. The personal attribute information can be acquired from business information (boarding reservation information) included in the business information DB 13 of the management server 10, for example.

[0033] The link information creation unit 401b creates link information that associates the biometric information of the user U who has requested entry into the departure area (restricted area) with the identification information of the baggage carried by the user U. In this embodiment, the security inspection information registered in the security inspection information DB 401a for each target person corresponds to the link information.

[0034] The control unit 401c executes, in parallel based on link information, a baggage inspection to determine whether or not the baggage contains any items that are restricted or prohibited from being brought into the restricted area (hereinafter referred to as "target items"), and a physical inspection to determine whether or not the user U is carrying the target items. That is, in this embodiment, the baggage inspection and the physical inspection are executed separately, while being related to each other.

[0035] The attribute information acquisition unit 401d acquires the attribute information of user U from the management server 10 (business information DB13) by requesting the management server 10 to perform a matching process between registered biometric information (registered facial image) that has been pre-associated with the attribute information of user U in the token ID information DB11 and the biometric information (facial image) of user U obtained from an image captured by a camera 404 within the security checkpoint.

[0036] The baggage inspection unit 401e performs baggage inspection based on a learning model that has previously learned the relationship between an image (for example, an X-ray image) of baggage captured using a baggage inspection device 405 (described later) and feature information on the target article.

[0037] The physical examination unit 401f performs a physical examination based on a learning model that has previously learned the relationship between an image (or measured value) of the user U's body taken using a physical examination device 406 described below and characteristic information regarding the target item.

[0038] The matching unit 401g performs a matching process between the biometric information of the user U newly acquired from the storage box 410, the signage terminal 409, etc. and the biometric information registered in the security inspection information DB 401a, and outputs the matching result to the control unit 401c.

[0039] The passage control device 402 is installed at the entrance and exit of the security checkpoint, and is a device that determines whether or not a user U can pass through. The passage control device 402 includes a biometric information acquisition device 402a, a medium reading device 402b, and a gate 402c. The biometric information acquisition device 402a is, for example, a digital camera that captures an image in front of the passage control device 402. When the biometric information acquisition device 402a detects the face of a user U standing in front of the passage control device 402 in an image that is constantly or periodically captured, it captures the face of the user U and acquires the facial image.

[0040] Next, the traffic control device 402 transmits the captured face image to the management server 10 via the edge server 401, and requests authentication processing to determine whether or not the user U is a registrant of the token ID. Then, if the management server 10 authenticates the user U as a "registrant of the token ID," the traffic control device 402 opens the gate 402c and permits the user U to enter the security inspection area.

[0041] Furthermore, in the case of user U for whom a token ID has not been issued during check-in procedures or the like, the passage control device 402 reads the recorded information from the recording medium using the medium reading device 402b and determines whether user U is allowed to pass.

[0042] Examples of the medium reading device 402b include a code reader, an image scanner, a non-contact IC (Integrated Circuit) reader, an OCR (Optical Character Reader), etc. Examples of the recording medium include a paper airline ticket, a mobile terminal that displays a copy of an e-ticket, etc.

[0043] When the identity verification of the user U by the traffic control device 402 is successful, the gate 402c transitions from a closed standby state that blocks the passage of the user U to an open state that allows the passage of the user U. The type of the gate 402c is not particularly limited, and may be, for example, a flapper gate in which a flapper provided on one or both sides of the passage opens and closes, or a turnstile gate in which three bars rotate.

[0044] The baggage transport device 403 is a transport device that automatically transports baggage on a belt conveyor moving in a predetermined direction to a baggage inspection device 405 and a storage destination. The edge server 401 controls the driving of the baggage transport device 403 and switches the destination (storage destination) of the baggage.

[0045] The camera 404 is, for example, a digital camera equipped with a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. The camera 404 transmits captured images to the edge server 401 one by one.

[0046] The baggage inspection device 405 is a device that performs a non-contact inspection to determine whether or not the baggage entrusted by the user U contains any target items (such as dangerous goods). Examples of the baggage inspection device 405 include an X-ray scanner that uses X-rays to photograph carry-on baggage and outputs the photographed image to the edge server 401, and a CT scanner. In order to improve detection accuracy, it is preferable to use a CT scanner that can capture three-dimensional X-ray images as the baggage inspection device 405. Upon receiving the photographed image from the baggage inspection device 405, the edge server 401 determines whether or not the baggage contains any target items that are restricted from being brought onto the plane, based on a predetermined learning model.

[0047] The physical inspection device 406 is a device that performs a non-contact inspection to determine whether or not the user U is carrying a target item. Examples of the physical inspection device 406 include a metal detector, a millimeter wave scanner, and a terahertz wave scanner. The metal detector is a device that checks whether or not the user U is carrying any metal object that may be dangerous. In airport facilities, gate-type metal detectors are widely used.

[0048] The millimeter wave scanner is a device that irradiates, for example, millimeter waves onto the body of the user U and measures the energy of the waves reflected from the body of the user U to image the body of the user U. The millimeter wave is, for example, an electromagnetic wave with a wavelength of 1 mm or more and 10 mm or less and a frequency band of about 30 GHz to 300 GHz.

[0049] Similarly, a terahertz wave scanner is a device that uses terahertz waves to image the body of a user U. Terahertz waves are electromagnetic waves with a wavelength of, for example, 100 micrometers or more and 1 millimeter or less, and a frequency band of about 300 GHz to 3 THz. There are two types of millimeter wave scanners and terahertz wave scanners: a passive type that measures electromagnetic waves naturally emitted from an object, and an active type that measures reflected waves when electromagnetic waves are irradiated onto an object.

[0050] The following describes the processing between the edge server 401 (physical examination section 401f) and the active type millimeter wave scanner (physical examination device 406).

[0051] When the edge server 401 receives a measurement value (signal) obtained by measuring the body of the user U from the millimeter wave scanner, it determines whether or not a target object is included in the body of the user U based on a predetermined learning model. If a target object is included in the body of the user U, the edge server 401 causes the business terminal 408 or the like to display a screen in which, for example, a schematic diagram of the human body is superimposed with information indicating the location of the target object. It is preferable that the data measured by the millimeter wave scanner is deleted after a predetermined time has elapsed.

[0052] The edge server 401 may also create a transmission image based on the signal of the reflected wave received by the millimeter wave scanner. Then, the edge server 401 may determine whether or not a pre-specified target item is in possession based on the shape that appears in the transmission image.

[0053] As an example of a determination method based on a shape appearing in a see-through image, the use of machine learning as described above may be considered. For example, a see-through image (teaching data) of a pre-specified target object may be prepared, and a learning model for estimating the target object appearing in the see-through image may be created by machine learning based on the teaching data. Then, the edge server 401 may determine whether or not the pre-specified target object appears in the see-through image based on the created see-through image and the learning model.

[0054] As another example, the edge server 401 may search for a pre-specified target object in the transparent image by template matching using a template that indicates the shape of the pre-specified target object.

[0055] The edge server 401 may also determine whether the user U is carrying a pre-specified target item based on a feature (transmission feature) that appears in the signal of the reflected wave received from the millimeter wave scanner. That is, the edge server 401 may perform the above determination process by directly using the signal of the reflected wave without creating a transmission image.

[0056] In this case, too, the above-mentioned learning model based on machine learning or template matching may be used. The edge server 401 can also learn a learning model used for baggage inspection of X-ray images using a similar method.

[0057] The business terminal 408 is a terminal device used by the staff S for business purposes. Examples of the business terminal 408 include, but are not limited to, a personal computer, a tablet terminal, and a smartphone. In Fig. 6, two types of business terminals 408 are shown as used by the staff S1 to S5: a portable type and a stationary type. The business terminal 408 displays information received from the edge server 401 on the screen.

[0058] The signage terminal 409 is a display terminal for presenting various kinds of guidance information received from the edge server 401 to the user U. In this embodiment, the signage terminal 409 presents the user U with the storage location of the baggage (storage box 410) and the status of the baggage inspection based on face authentication.

[0059] The storage box 410 is a device that stores the baggage of the user U transported by the baggage transport device 403. In this embodiment, when the edge server 401 authenticates that the user U is the owner of the baggage, the edge server 401 unlocks the electronic lock of the storage box 410 and hands over the baggage to the user U.

[0060] In FIG. 6, multiple users U1 to U11 and multiple staff members S1 to S5 are present at a security checkpoint. The area on the left side of the figure is a baggage inspection area and a baggage storage area. The five users U1 to U5 place their baggage on a predetermined transport tray T on the belt conveyor of the baggage transport device 403. As a result, the transport tray T moves in the direction of arrow D. The transport tray T with the baggage stored therein is inserted into the baggage inspection device 405. Then, the baggage inspection of the baggage in the transport tray T is performed in cooperation with the baggage inspection device 405 and the edge server 401 (not shown).

[0061] In the figure, staff members S1 and S2 are located on the right side of the baggage transport device 403. Staff member S1 judges the state of the transport tray T on which the baggage is stored, and if necessary, separates the baggage into multiple trays and sends them through the baggage inspection device 405. At this time, it is preferable for staff member S1 to use a dedicated tray.

[0062] Specifically, it is advisable to print another code on the surface of the special tray, for example, to indicate that the special tray is storing baggage separated from the previous transport tray T. This makes it easy to associate the baggage with the user U even after the user U and the baggage are separated.

[0063] Further, downstream in the conveying direction of the baggage conveying device 403, a total of seven lanes are branched out, including an NG lane L1, OK lanes L2 to L6, and an unknown lane L7. The NG lane L1 is a re-inspection lane to which baggage in which a target article has been detected is conveyed, and an officer S2 is stationed there. The OK lanes L2 to L6 are lanes to which baggage in which a target article has not been detected is conveyed, and a storage box 410 is provided at the end of each of the OK lanes L2 to L6. Baggage is automatically stored from the back side of the storage box 410. The unknown lane L7 is a lane to which a conveyance tray T with an unknown owner is conveyed.

[0064] Meanwhile, the area on the right side of the figure is a physical inspection area, and two inspection lanes are illustrated. Each inspection lane is provided with a physical inspection device 406 and a business terminal 408. The physical inspection consists of two types of inspection, for example, an inspection using a metal detector and an inspection using a millimeter wave scanner. FIG. 6 shows users U7 and U9 undergoing a physical inspection using the physical inspection device 406.

[0065] User U10 has completed a physical examination and is standing in front of the signage terminal 409. The signage terminal 409 takes a photograph of the face of user U10 and transmits the photographed face image to the edge server 401. Depending on the authentication result in the edge server 401, the signage terminal 409 displays information on the storage location of the baggage or the status of the baggage inspection.

[0066] Also, it is preferable that the signage terminal 409 displays, in a list format, images of baggage that has been normally inspected (or a facial image of the owner of the baggage) and the number of the storage box 410 where the baggage is to be stored.

[0067] The storage box 410 captures a facial image of each user U and transmits the captured facial image to the edge server 401. The storage box 410 releases the electronic lock when authentication by the edge server 401 is successful. For example, user U11 is successfully authenticated and is able to remove his / her baggage from the storage box 410. In this way, after completing the security inspection procedure by the security inspection system 40 at touch point P3, users U1 to U11 move to the departure inspection area.

[0068] The automated gate device 50 is installed at the immigration area (hereinafter referred to as "touch point P4") in airport A. The automated gate device 50 is a device that automatically performs the immigration procedure for user U. After completing the immigration procedure at touch point P4, user U moves to the departure area where duty-free shops and boarding gates are located.

[0069] The boarding gate device 60 is a gate device installed at each boarding gate (hereinafter referred to as "touch point P5") in the departure area. The boarding gate device 60 confirms that the user U is a passenger of an aircraft that can be boarded through the boarding gate. After completing the procedures at the touch point P5, the user U boards the aircraft and departs for the second country.

[0070] Fig. 8 is a block diagram showing an example of a hardware configuration of the edge server 401 in this embodiment. As shown in Fig. 8, the edge server 401 is a computer that performs calculations, control, and storage, and includes a CPU (Central Processing Unit) 411, a RAM (Random Access Memory) 412, a ROM (Read Only Memory) 413, a storage 414, and a communication I / F (Interface) 415. Each device is connected to each other via a bus, wiring, a drive device, etc.

[0071] The CPU 411 is a processor that performs a predetermined operation according to a program stored in the ROM 413 or the storage 414, and has a function of controlling each unit of the edge server 401. The RAM 412 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of the CPU 411.

[0072] The CPU 411 loads a program stored in the ROM 413 or the like into the RAM 412 and executes it, thereby realizing the functions of each part, such as the link information creation part 401b, the control part 401c, the attribute information acquisition part 401d, the baggage inspection part 401e, the physical inspection part 401f and the matching part 401g, described above.

[0073] The ROM 413 is composed of a non-volatile storage medium, and stores necessary information such as programs used for the operation of the edge server 401. The storage 414 is composed of a non-volatile storage medium, and stores programs and data for the operation of the edge server 401. The storage 414 is composed of, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0074] The communication I / F 415 is a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), and 4G, and is a module for communicating with other devices.

[0075] Note that the hardware configuration shown in Fig. 8 is an example, and other devices may be added, or some devices may not be provided. Also, some devices may be replaced with other devices having similar functions. Also, some functions of this embodiment may be provided by other devices (e.g., management server 10, etc.) via networks NW and NW1, or the functions of this embodiment may be distributed and realized among multiple devices. In this way, the hardware configuration shown in Fig. 8 can be changed as appropriate.

[0076] Next, the operation of the security inspection system 40 in this embodiment will be described with reference to FIGS.

[0077] <Associating users with baggage> 9 is a flowchart showing an example of the processing of the edge server 401. This processing is executed to associate the user U, who is the owner of the baggage, with the baggage when the user U undergoes baggage inspection at the security checkpoint.

[0078] First, the edge server 401 acquires an image captured by the camera 404 (step S101). Next, when the edge server 401 determines that a user U who is a subject of security inspection is detected in the acquired captured image (step S102, YES), the process proceeds to step S103.

[0079] On the other hand, if the edge server 401 determines that the user U is not detected in the acquired photographed image (step S102, NO), the process returns to step S101.

[0080] Next, when the edge server 401 determines that a predetermined barcode affixed to the surface of the transport tray has been detected from the captured image (step S103, YES), the process proceeds to step S104.

[0081] On the other hand, if the edge server 401 determines that the predetermined barcode on the surface of the transport tray is not detected in the captured image (step S103, NO), the process returns to step S101. That is, the processes of steps S101 to S103 are repeated until a captured image showing both the target person and the barcode on the transport tray is acquired.

[0082] FIG. 10 is a diagram showing an example of a captured image taken at the start of security inspection. Here, a captured image IMG01 is shown, which was taken when a user U placed baggage B in a transport tray T. A barcode C indicating a tray ID is printed on the surface of the transport tray T. In this way, a facial image (biometric information) of the user U and identification information of the transport tray T are simultaneously obtained from one captured image IMG01, and link information is created. Note that if a user U places baggage B separately on multiple transport trays T, multiple pieces of link information are created.

[0083] Next, the edge server 401 acquires a face image of the user U from the captured image (step S104). When multiple people are included in the same captured image, it is preferable to determine the positional relationship between the people based on the eye distance or the like and acquire the face image of the person located in front.

[0084] Next, the edge server 401 issues a unique target person ID to the user U (step S105). The target person ID is set separately from the above-mentioned token ID. The target person ID is, for example, a sequence number that does not overlap between people.

[0085] Next, the edge server 401 decodes the barcode detected in step S103 to obtain a tray ID for identifying the transport tray (step S106). Next, the edge server 401 makes an authentication request to the management server 10 based on the face image obtained in step S104 (step S107). Upon receiving the authentication request from the edge server 401, the management server 10 compares the face image of the user U included in the authentication request with each of the multiple registered face images stored in the token ID information DB11.

[0086] Next, if the edge server 401 determines based on the authentication result in the management server 10 that the user U is a registrant of the token ID (step S108, YES), it obtains the registration information (person attribute information) of the user U from the management server 10 using the token ID as a key (step S109).

[0087] Then, the edge server 401 registers the security inspection information in which the target person ID, the tray ID, and the registration information (person attribute information) are associated with each other in the security inspection information DB 401a (step S110), and ends the process.

[0088] In response to this, if the edge server 401 determines based on the authentication result in the management server 10 that the user U is not a registered user of the token ID (step S108, NO), it registers the security inspection information associated with the target person ID and tray ID in the security inspection information DB 401a (step S111) and terminates the processing.

[0089] <Baggage Inspection> 11 is a flowchart showing an example of processing during baggage inspection by the edge server 401. This processing is executed, for example, when baggage stored on a transport tray is transported by the baggage transport device 403 to the position of the baggage inspection device 405 (i.e., the location where the baggage inspection is performed).

[0090] First, when the edge server 401 determines that a transport tray moving into the baggage inspection device 405 is detected from an image captured by the camera 404 installed near the baggage inspection device 405 (step S201, YES), the process proceeds to step S202. On the other hand, when the edge server 401 determines that a transport tray is not detected (step S201, NO), the standby state is maintained.

[0091] Next, the edge server 401 acquires the tray ID of the transport tray by image analysis of the captured image, and identifies the target person ID by referring to the security inspection information DB 401a using the tray ID as a key (step S202).

[0092] Next, the edge server 401 sets a threshold value, which is a detection condition for the target item in the baggage inspection, based on various information associated with the target person ID (step S203). Details of this process will be described later. The threshold value is, for example, a determination threshold value for the similarity score with the target item.

[0093] Next, the edge server 401 cooperates with the baggage inspection device 405 to perform baggage inspection (step S204). Specifically, the edge server 401 performs image analysis processing on the captured image captured by the baggage inspection device 405.

[0094] Next, when the edge server 401 determines that a target item is detected in the baggage by the image analysis process (step S205, YES), it outputs the detection information of the target item to a predetermined business terminal 408 (step S206). Next, the edge server 401 determines the destination of the baggage in which the target item is detected to be the NG lane (step S207). After that, the process proceeds to step S208.

[0095] On the other hand, if the edge server 401 determines that the target article is not detected in the baggage through the image analysis process (step S205, NO), the process proceeds to step S211.

[0096] Next, the edge server 401 refers to the security inspection information DB 401a using the target person ID as a key, and determines whether or not the owner of the baggage has completed a physical inspection (step S208).

[0097] If the edge server 401 determines that the physical examination of the owner of the baggage has been completed (step S208, YES), the process proceeds to step S209. On the other hand, if the edge server 401 determines that the physical examination of the owner has not been completed (step S208, NO), the process proceeds to step S210.

[0098] In step S209, the edge server 401 refers to the security inspection information DB 401a using the target person ID as a key, and determines whether or not the user U is a person for whom a target article was detected during a physical inspection.

[0099] Here, if the edge server 401 determines that the user U is a person for whom a target article was detected in the physical examination (step S209, YES), the process proceeds to step S213.

[0100] On the other hand, if the edge server 401 determines that the user U is not a person for whom the target article was detected in the physical examination (step S209, NO), the process proceeds to step S210.

[0101] In step S210, the edge server 401 outputs a service instruction to a predetermined service terminal 408. For example, if a target object is detected in a baggage inspection by a user U, but if the target object is not detected in a physical inspection, the user U is assumed to wait after leaving the physical inspection area until the baggage inspection is completed. Similarly, if a target object is detected in a baggage inspection by a user U, but if the physical inspection is not completed, the user U is assumed to be waiting in line in front of the physical inspection area or to be undergoing physical inspection. In these cases, the edge server 401 may inform the staff S of the current location of the customer U and issue a service instruction to guide the user U to another room or the like.

[0102] In step S211, the edge server 401 outputs non-detection information of the target item to a predetermined business terminal 408. Then, the edge server 401 determines a storage box 410 in which the baggage will be stored (step S212). After that, the process proceeds to step S213. Note that, when there are multiple pieces of baggage belonging to the same person, it is preferable to set the storage locations to adjacent lanes or the same lane.

[0103] In step S213, the edge server 401 outputs drive control information to the baggage conveying device 403, and causes the baggage conveying device 403 to convey the baggage to the destination (storage destination) determined in step S207 or step S212. Then, the edge server 401 updates the security inspection information in the security inspection information DB 401a based on the target person ID of the user U and the inspection result of the baggage inspection (step S214), and ends the process.

[0104] 12 is a diagram showing an example of a screen displayed on the business terminal 408 on the baggage inspection area side. An error message ("A target item has been detected in the user's baggage. The transport tray has been moved to the re-inspection lane. Please check your baggage.") is displayed in the upper area of ​​the screen, and user information such as the name and sex of user U, the baggage inspection results, and an X-ray image IMG02 are displayed in the lower area of ​​the screen. The X-ray image IMG02 shows that a handgun, which is a target item I, has been detected in baggage B.

[0105] Meanwhile, Fig. 13 is a diagram showing an example of a screen displayed on the service terminal 408 on the physical inspection area side. Here, an error message ("A target item has been detected in the user's baggage. Please guide the user to another room immediately.") is displayed in the upper region of the screen, instructing the staff S on the physical inspection area side to guide the user U. Also, the lower region of the screen displays the baggage inspection results, information indicating the current location of the user U, the user information of the user U, the photographed face image IMG03, and the X-ray photographed image IMG02. This enables the staff S to search for the user U based on the information on the service instruction screen and guide him / her promptly.

[0106] Next, a method for setting detection conditions for target articles during baggage inspection will be described. Fig. 14 is a flowchart showing an example of the processing performed by the edge server 401. This processing corresponds to the processing in step S203 in Fig. 11 described above.

[0107] In step S301, the edge server 401 judges whether or not the user U is a person for whom a target object has been detected in a physical inspection. If the edge server 401 judges that the user U is a person for whom a target object has been detected in a physical inspection (step S301, YES), the edge server 401 sets the baggage inspection threshold to a value higher than normal (step S302), and the process proceeds to step S204. In other words, the detection conditions for the target object are made stricter than the currently set conditions.

[0108] In contrast, if the edge server 401 determines that the user U is not a person for whom a target item was detected during a physical inspection (step S301, NO), it sets the baggage inspection threshold to the normal value (step S303), and the process proceeds to step S204.

[0109] <Physical Examination> 15 is a flowchart showing an example of processing during a physical inspection by the edge server 401. This processing is executed for a user U after he or she has checked in their baggage in a baggage inspection area, for example.

[0110] First, the edge server 401 analyzes the captured image received from the camera 404 installed in the physical examination area and determines whether the face of the user U is detected (step S401). Here, the edge server 401 waits until the face of the user U is detected (step S401, NO).

[0111] Next, when the edge server 401 determines that the face of the user U has been detected (step S401, YES), it acquires a facial image of the user U from the captured image (step S402). Next, the edge server 401 compares the acquired facial image with multiple captured facial images stored in the security inspection information DB 401a in a 1:N manner (step S403).

[0112] Next, the edge server 401 identifies a target person ID associated with a captured face image that matches the face image in the security inspection information DB 401a (step S404).

[0113] Next, the edge server 401 sets a threshold value, which is a detection condition for the target article in the physical examination, based on various information associated with the target person ID (step S405). The details of this process will be described later.

[0114] Next, the edge server 401 executes a physical examination in cooperation with the physical examination device 406 (step S406). Specifically, the edge server 401 performs an analysis process of the measurement signal measured by the physical examination device 406.

[0115] When the edge server 401 determines that a target object is detected in the body of the user U through the analysis process (step S407, YES), it outputs detection information of the target object to a predetermined business terminal 408 (step S408). After that, the process proceeds to step S409.

[0116] On the other hand, if the edge server 401 determines that the target article is not detected on the body of the user U (step S407, NO), it outputs information indicating that the target article is not detected to the predetermined business terminal 408 (step S412). After that, the process proceeds to step S413.

[0117] Next, the edge server 401 refers to the security inspection information DB 401a using the target person ID as a key, and if it is determined that the baggage inspection has been completed (step S409, YES), the process proceeds to step S410. On the other hand, if it is determined that the baggage inspection has not been completed (step S409, NO), the process proceeds to step S413.

[0118] In step S410, the edge server 401 refers to the security inspection information DB 401a using the target person ID as a key, and determines whether the user U is a person for whom a target article has been detected during baggage inspection.

[0119] Here, if the edge server 401 determines that the user U is a person whose object was detected during baggage inspection (step S410, YES), the process proceeds to step S413.

[0120] In response to this, if the edge server 401 determines that the user U is not the person whose target item was detected during baggage inspection (step S410, NO), it causes the business terminal 408 in the baggage inspection area to display a screen instructing the user to re-inspect the baggage (step S411).

[0121] Then, the edge server 401 updates the security inspection information in the security inspection information DB 401a based on the target person ID of the user U and the inspection result of the baggage inspection (step S413), and ends the process.

[0122] FIG. 16 is a diagram showing an example of a screen displayed on the service terminal 408 on the physical examination area side. An error message indicating that a target object has been detected inside the body of user U ("Target object has been detected. Please check the detected location.") is displayed in the upper region of the screen. Additionally, user information about user U and the physical examination results are displayed in the lower region of the screen. The physical examination results include image IMG04, which is a schematic diagram of the front side of the human body, and image IMG05, which is a schematic diagram of the back side. Here, a knife, which is target object I, is displayed at the position of user U's right thigh in image IMG05.

[0123] On the other hand, Fig. 17 is a diagram showing an example of a screen displayed on the business terminal 408 on the baggage inspection area side. An error message ("Target item was detected during physical inspection. Please reinspect the baggage stored in Box No. 2.") is displayed in the upper region of the screen to instruct the staff member S on the baggage inspection area side to reinspect the baggage. The screen in Fig. 17 may be displayed to have the baggage inspected again depending on the results of the physical inspection when a target item was not detected in the baggage inspection that has already been conducted on user U.

[0124] Next, a method for setting detection conditions for a target article during a physical examination will be described. Fig. 18 is a flowchart showing an example of the processing performed by the edge server 401. This processing corresponds to the processing performed in step S405 in Fig. 15 described above.

[0125] In step S501, the edge server 401 judges whether the user U is a person whose target object has been detected during baggage inspection. If the edge server 401 judges that the user U is a person whose target object has been detected during baggage inspection (step S501, YES), the edge server 401 sets the threshold value for the physical inspection to a value higher than normal (step S502), and the process proceeds to step S406. In other words, the detection conditions for the target object during the physical inspection are made stricter than the currently set conditions.

[0126] In contrast, if the edge server 401 determines that the user U is not the person whose target item was detected during baggage inspection (step S501, NO), it sets the physical inspection threshold to the normal value (step S503), and processing proceeds to step S406.

[0127] <Information on storage location> 19 is a sequence diagram showing an example of processing between the edge server 401 and the signage terminal 409. This processing is executed for a user U who has completed a physical examination.

[0128] The signage terminal 409 constantly or periodically captures an image in front of the terminal, and determines whether the face of the user U standing in front of the signage terminal 409 is detected in the captured image (step S601). If the signage terminal 409 determines that the face of the user U is not detected in the image (step S601, NO), it waits until the face of the user U is detected in the image.

[0129] When signage terminal 409 determines that the face of user U has been detected (step S601, YES), it acquires a facial image of user U from the captured image as a target facial image (step S602).

[0130] Next, the signage terminal 409 transmits the face image (target face image) of the user U together with a matching request to the edge server 401 (step S603). As a result, the signage terminal 409 requests the edge server 401 to match the target face image of the user U with multiple registered face images registered in the security inspection information DB 401a on a 1:N basis.

[0131] When the edge server 401 receives the target face image and the matching request from the signage terminal 409, it matches the face image of the user U (step S604). That is, the edge server 401 matches the target face image received from the signage terminal 409 with multiple registered face images registered in the security inspection information DB 401a on a 1:N basis.

[0132] Here, if the edge server 401 determines that the matching result is a mismatch (step S605, NO), the edge server 401 transmits the mismatch matching result to the signage terminal 409 (step S608), and the process proceeds to step S609. On the other hand, if the edge server 401 determines that the matching result is a match (step S605, YES), the process proceeds to step S606.

[0133] In step S606, the edge server 401 acquires the baggage inspection result and the baggage storage location information associated with the registered face image that matches the match in the security inspection information DB 401a. Then, the edge server 401 transmits the matching result, the baggage inspection result, and the baggage storage location information to the signage terminal 409 (step S607). After that, the process proceeds to step S609.

[0134] Next, if signage terminal 409 determines, based on the information received from edge server 401, that the baggage of user U is stored in storage box 410 (step S609, YES), the process proceeds to step S610.

[0135] On the other hand, if the signage terminal 409 determines that the user U's baggage is not stored in the storage box 410 (step S609, NO), it notifies the user U of an error message (step S611). For example, a notification screen including an error message such as "Your baggage is not being stored" is displayed.

[0136] In step 610, signage terminal 409 displays a screen including information about the storage location of user U's baggage, and ends the process.

[0137] 20 and 21 are diagrams showing examples of screens displayed on signage terminal 409. In Fig. 20, signage terminal 409 is provided with camera 409a as a biometric information acquisition device. When a match is found between a facial image of user U captured by camera 409a and a captured facial image stored in security inspection information DB 401a, a facial image of user U and a guidance message indicating where the baggage is to be stored ("Your baggage is being stored in locker no. 1") are displayed on the screen as shown in Fig. 20.

[0138] 21, the result of matching between the face image of user U and the photographed face image stored in the security inspection information DB 401a is a match, but a guidance message indicating that the baggage inspection status of user U is "not yet completed" or "in progress" ("The inspection of your baggage has not yet been completed. Please wait until the inspection is completed.") is displayed. Note that if a target article is detected during the baggage inspection of user U, a guidance screen like that of FIG. 21 may be displayed to prompt user U to wait.

[0139] <Baggage collection> 22 is a sequence diagram showing an example of a process between the edge server 401 and the storage box 410. This process is executed for a user U who has completed both a physical inspection and baggage inspection.

[0140] The storage box 410 constantly or periodically captures an image in front of the device, and determines whether the face of the user U standing in front of the storage box 410 is detected in the captured image (step S701). The storage box 410 waits until the face of the user U is detected in the image (step S701, NO).

[0141] When the storage box 410 determines that the face of the user U has been detected (step S701, YES), it photographs the face of the user U and acquires the facial image of the user U as a target facial image (step S702).

[0142] Next, the storage box 410 transmits the target face image of the user U together with a matching request to the edge server 401 (step S703). As a result, the storage box 410 requests the edge server 401 to match the target face image of the user U with multiple registered face images registered in the security inspection information DB 401a on a 1:N basis.

[0143] When the edge server 401 receives the target face image and the matching request from the storage box 410, it matches the face image of the user U (step S704). That is, the edge server 401 matches the target face image received from the storage box 410 with multiple registered face images registered in the security inspection information DB 401a on a 1:N basis.

[0144] Here, if the edge server 401 determines that the matching result is a mismatch (step S705, NO), the edge server 401 transmits the mismatch matching result to the storage box 410 (step S708), and the process proceeds to step S709. On the other hand, if the edge server 401 determines that the matching result is a match (step S705, YES), the process proceeds to step S706.

[0145] In step S706, the edge server 401 acquires the baggage inspection result and the baggage storage location information associated with the matching registered face image in the security inspection information DB 401a. Then, the edge server 401 transmits the matching result, the baggage inspection result, the baggage storage location information, the unlocking control information, and the like to the storage box 410 (step S707). After that, the process proceeds to step S709.

[0146] Next, if the storage box 410 determines based on the received information that the user U's luggage is being stored inside (step S709, YES), it unlocks the door based on the unlocking control information (step S710), and the process proceeds to step S712.

[0147] On the other hand, if the storage box 410 determines that the user U's baggage is not stored inside (step S709, NO), it notifies the user U of an error (step S711). For example, it displays a notification screen including an error message such as "Your baggage is not stored in this box" or "Your baggage is stored in box number 3."

[0148] Next, if the storage box 410 determines that the delivery of the stored baggage is complete (step S712, YES), the process proceeds to step S713.

[0149] On the other hand, if the storage box 410 determines that the handover of the baggage has not been completed (step S712, NO), the storage box 410 waits until the handover of the baggage is completed.

[0150] In step S713, the storage box 410 transmits status information indicating that the baggage delivery has been completed to the edge server 401. Then, upon receiving the status information, the edge server 401 updates the delivery status of the user U in the security inspection information DB 401a to “delivery completed” (step S714), and ends the process.

[0151] 23 is a front view showing an example of the configuration of the storage box 410. Here, the storage box 410 includes a space 410a capable of storing baggage B and a transport tray T, an openable and closable door 410b, and a camera 410c as a biometric information acquisition device. When a match is found between a facial image of the user U captured by the camera 410c and a captured facial image stored in the security inspection information DB 401a, the edge server 401 outputs control information for unlocking an electronic lock (not shown) to the storage box 410. This allows the user U to collect the baggage that was deposited while undergoing baggage inspection and physical inspection.

[0152] According to this embodiment, when a baggage inspection is carried out on a user U, the biometric information of the user U is associated with the identification information of the baggage. This allows the user U to hand over his / her baggage for inspection at any time, and to move to the physical inspection area to undergo a physical inspection. In other words, by associating the user U with the baggage, physical inspection and baggage inspection can be carried out in parallel and in no particular order for the user U (owner). As a result, the throughput of security inspection can be improved.

[0153] Furthermore, according to this embodiment, the biometric information of the user U and the identification information of the baggage are automatically acquired from the same captured image taken at the moment when the user U checks in the baggage. When associating the biometric information with the identification information of the baggage, the user U does not need to present a recording medium such as a passport or boarding pass, which improves convenience for the user U.

[0154] Furthermore, according to this embodiment, when a target article is detected in either the baggage inspection or the body inspection, the detection conditions for the target article in the other inspection are changed. The results of the two types of inspections are shared, further improving the detection accuracy of the target article.

[0155] Furthermore, according to this embodiment, the edge server 401 determines the storage box 410 in which the baggage is to be stored based on the results of the baggage inspection, and controls the transport of the baggage to the storage destination by the baggage transport device 403. Since the baggage is automatically stored in the designated storage box 410, the workload of the staff S can be significantly reduced.

[0156] Furthermore, according to this embodiment, if no target article is detected during baggage inspection and physical inspection, the edge server 401 displays information about where the baggage is stored on the signage terminal 409. For this reason, for example, if the signage terminal 409 is placed in the direction of travel of a user U who has just finished a physical inspection, the user U can immediately know where his or her baggage is being stored.

[0157] Furthermore, according to this embodiment, the edge server 401 displays information on the storage location on the signage terminal 409 based on the result of matching the biometric information of the user U acquired by the signage terminal 409 with the biometric information included in the link information (security inspection information). This allows the user U to know where his / her baggage is being stored at any time of his / her choice.

[0158] Furthermore, according to this embodiment, the edge server 401 controls the removal of baggage from the storage box 410 based on the result of matching the biometric information of the user U acquired in the storage box 410 with the biometric information included in the link information (security inspection information). This ensures that the baggage is handed over only to its owner, thereby reliably preventing the baggage from being mixed up.

[0159] Furthermore, according to this embodiment, the edge server 401 uses a learning model obtained by machine learning for each of the baggage inspection and body inspection. Therefore, it is possible to flexibly and quickly respond to not only already known target objects (dangerous objects, suspicious objects), but also various types of target objects. As a result, it is possible to improve the detection accuracy of the target objects.

[0160] [Second embodiment] FIG. 24 is a block diagram showing an example of the overall configuration of the information processing device 100 in this embodiment. The information processing device 100 includes a first acquisition unit 100A, a second acquisition unit 100B, and a creation unit 100C. The first acquisition unit 100A acquires biometric information of a person. The second acquisition unit 100B acquires identification information of baggage carried by the person. The creation unit 100C creates link information that associates the biometric information with the identification information of the baggage. According to this embodiment, the relationship between baggage and its owner can be easily registered and managed.

[0161] [Modified embodiment] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention without departing from the gist of the present invention. For example, it should be understood that an embodiment in which a part of the configuration of any embodiment is added to another embodiment, or an embodiment in which a part of the configuration of another embodiment is replaced with a part of the configuration of another embodiment, is also an embodiment to which the present invention can be applied.

[0162] In the above embodiment, a case where a face image is used as biometric information has been described, but the biometric information applicable to the present invention is not limited to a face image. For example, the management server 10 and the edge server 401 may perform matching processing using other biometric information such as a fingerprint image, an iris image, or an ear image instead of a face image. In addition, the management server 10 and the edge server 401 may perform matching processing based on a combination of different types of biometric information.

[0163] In the above embodiment, the detection conditions for the target object in the baggage inspection or the body inspection are changed according to the result of the other inspection, but the detection conditions for the target object may be changed based on other information. For example, personal attribute information of the user U may be used. As the personal attribute information, at least one of gender, age, nationality, security inspection history information, travel destination history information, etc. may be used.

[0164] Specifically, for a person whose object was detected in a security check conducted during a past trip, the detection threshold may be set higher than the normal level. Similarly, when the destination or transit point of the user U's past trip is a country or region that requires attention and is designated in advance, the detection threshold may be set higher.

[0165] Furthermore, when the edge server 401 uses travel history information (travel destination / stopover / date / number of times within a specified period, etc.) or security inspection history information (inspection result / inspection date / time) as attribute information, the edge server 401 may input the history information into a trained model and calculate the trustworthiness and risk of each person. In this case, the edge server 401 (control unit) may increase the detection threshold for target items for persons determined to have low trustworthiness (high risk) from the normal threshold. This allows, for example, unnatural points in the travel history of user U to be automatically detected, further improving the accuracy of security inspections.

[0166] Conversely, for good travelers with no problems in their past travel history, it is possible to set the detection threshold for security checks lower than the normal level. In particular, in the case of a system configuration in which personal attribute information can be easily acquired based on the biometric information of user U, there is an advantage in that the most suitable security check can be performed for each user U. Note that, if user U is registered on a blacklist, an error determination may be made without performing the check, and a detailed check may be performed at another location.

[0167] In the above embodiment, the determination threshold for the captured image or the measurement signal is changed, but the image capturing conditions or the signal measurement conditions themselves may be changed. For example, if the baggage inspection device 405 is a 3D CT scanner, the angle of capture, the interval between capture positions, etc. may be changed to perform a more detailed inspection than usual.

[0168] In the above embodiment, the case where one baggage inspection device 405 is provided is exemplified, but the number of baggage inspection devices 405 is arbitrary. For example, the baggage inspection devices 405 may be arranged side by side to inspect a plurality of pieces of baggage at the same time. The same applies to the body inspection device 406 (for example, a metal detector or a millimeter wave scanner).

[0169] In the above embodiment, the case where the baggage of user U is stored in the storage box 410 on an individual basis has been described, but the storage method is not limited to this. For example, the baggage of multiple users U may be stored in each lane. In this case, it is preferable to provide other monitoring means to prevent the mix-up of baggage. Furthermore, when the baggage of multiple users U is stored in each lane, it is preferable to display in which lane the baggage is stored on the signage terminal 409 or the like. This has the advantage of making effective use of the limited space at the security inspection site, thereby improving throughput at the storage inspection site.

[0170] In the above embodiment, the case where the user U's baggage is stored within the security checkpoint has been described, but the storage location does not necessarily have to be the security checkpoint. For example, the baggage may be stored in a dedicated locker provided in the departure area.

[0171] In the above embodiment, the case where a barcode obtained from an image area of ​​the transport tray storing the baggage in the captured image is used as the identification information of the baggage has been described, but the method of obtaining the identification information of the baggage is not limited to reading the barcode. For example, characters or figures printed or formed on the surface of the transport tray may be read and used as the identification information of the baggage. Also, the characteristic amount of the baggage itself may be calculated and used as the identification information of the baggage. Furthermore, the characteristic amount may be read from a recording medium built into the transport tray. Similarly, as the identification information of the user U, information read from a recording medium such as a passport, a boarding pass, or various identification cards may be used instead of the biometric information of the user U.

[0172] In addition, in the above embodiment, the security checkpoint of an airport facility has been described as an example, but the facilities to which the present invention can be applied are not limited to this. For example, the present invention can be similarly applied to security check procedures before entry into concert venues, theme parks, high-speed trains, ships, etc.

[0173] The scope of each embodiment also includes a processing method in which a program that operates the configuration of the embodiment to realize the functions of the above-mentioned embodiment is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. That is, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0174] Examples of the recording medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a non-volatile memory card, etc. Furthermore, the scope of each embodiment is not limited to those that execute processing by a single program recorded on the recording medium, but also includes those that execute processing by operating on an OS in cooperation with other software and functions of an expansion board.

[0175] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0176] (Appendix 1) A first acquisition unit that acquires biometric information of a person; A second acquisition unit that acquires identification information of baggage carried by the person; a creating unit that creates link information that associates the biometric information with the identification information; An information processing device comprising:

[0177] (Appendix 2) a control unit that executes, in parallel, a baggage inspection to determine whether or not the baggage contains an object restricted from being brought into a restricted area, and a physical inspection to determine whether or not the person is carrying the object object, based on the link information; 2. The information processing device according to claim 1, further comprising:

[0178] (Appendix 3) the control unit changes a detection condition for the target article in one of the baggage inspection and the physical inspection based on an inspection result in the other inspection. 3. An information processing device according to claim 2.

[0179] (Appendix 4) when the target article is detected in one of the baggage inspection and the physical inspection, the control unit sets stricter detection conditions for the target article in the other inspection than currently set conditions. 4. The information processing device according to claim 3.

[0180] (Appendix 5) An acquisition unit acquires the attribute information by a process of matching registered biometric information associated in advance with the attribute information of the person with the biometric information, the control unit changes detection conditions for the target article in each of the baggage inspection and the physical inspection based on the attribute information. 5. An information processing device according to any one of claims 2 to 4.

[0181] (Appendix 6) The attribute information includes at least one of the person's gender, age, and nationality. 6. The information processing device according to claim 5.

[0182] (Appendix 7) The attribute information includes inspection history information of the physical inspection and the baggage inspection that have been conducted. 7. The information processing device according to claim 5 or 6.

[0183] (Appendix 8) the control unit sets the detection conditions for the target object to be stricter than the currently set conditions when the inspection history information indicating that the target object has been detected in the past is acquired. 8. The information processing device according to claim 7.

[0184] (Appendix 9) The attribute information includes travel history information of the person. An information processing device according to any one of appendix 5 to 8.

[0185] (Appendix 10) The control unit determines a storage location of the baggage based on the inspection result of the baggage inspection, and controls the transport of the baggage to the storage location. 10. An information processing device according to any one of claims 2 to 9.

[0186] (Appendix 11) the control unit causes a display terminal to display information about the storage location of the baggage when the target article is not detected in the baggage inspection and the body inspection. 11. The information processing device according to claim 10.

[0187] (Appendix 12) the control unit causes the display terminal to display the information on the storage location based on a comparison result between the second biometric information of the person acquired on the display terminal and the biometric information included in the link information. 12. The information processing device according to claim 11.

[0188] (Appendix 13) the control unit controls removal of the baggage from the storage device based on a comparison result between the third biometric information of the person acquired in the storage device that is the storage destination and the biometric information included in the link information. 13. An information processing device according to any one of appendixes 10 to 12.

[0189] (Appendix 14) When the target object is detected during the physical inspection or the baggage inspection, the control unit outputs detection information of the target object to a predetermined business terminal. An information processing device according to any one of appendix 2 to 13.

[0190] (Appendix 15) a baggage inspection unit that performs the baggage inspection based on a learning model that has previously learned a relationship between an image of the baggage and feature information related to the target article; 15. The information processing device according to any one of claims 2 to 14, further comprising:

[0191] (Appendix 16) a physical examination unit that executes the physical examination based on a learning model that has previously learned a relationship between an image of the person's body and feature information related to the target object; 15. The information processing device according to any one of claims 2 to 14, further comprising:

[0192] (Appendix 17) the biometric information and the identification information are obtained from a same captured image including the person and the baggage; 17. An information processing device according to any one of claims 1 to 16.

[0193] (Appendix 18) The identification information is any one of characters, figures, and barcodes obtained from an image area of ​​a container that contains the baggage in the captured image. 18. The information processing device according to claim 17.

[0194] (Appendix 19) the identification information is a feature amount obtained from an image area of ​​the baggage in the captured image; 18. The information processing device according to claim 17.

[0195] (Appendix 20) acquiring biometric information of a person; obtaining identification information for baggage carried by said person; creating link information that associates the biometric information with the identification information; An information processing method comprising:

[0196] (Appendix 21) On the computer, acquiring biometric information of a person; obtaining identification information for baggage carried by said person; creating link information that associates the biometric information with the identification information; A recording medium on which a program for executing the above is recorded. [Explanation of symbols]

[0197] NW, NW1...Network U, U1~U11...User S, S1 to S5: Staff 1. Information Processing System 10. Management Server 11 Token ID information DB 12 Passing history information DB 13...Business information DB 20. Check-in terminal 30. Self-service baggage drop machine 40. Security Inspection System 50. Automated gate device 60. Boarding gate equipment 100.... Information processing device 100A···First acquisition section 100B...Second acquisition part 100C···Creation Department 401... Edge Server 401a···Security inspection information DB 401b Link Information Creation Department 401c...Control section 401d...Attribute information acquisition section 401e Baggage Inspection Section 401f Physical Examination Department 401g... Verification section 402 Traffic control device 402a Biometric information acquisition device 402b Media reader 402c···Gate 403 Baggage handling equipment 404...Camera 405 Baggage Inspection Equipment 406 Physical examination equipment 408...Business terminal 409 Signage terminal 410···Storage box 411··CPU 412...RAM 413···ROM 414...Storage 415···Communication I / F

Claims

1. A first acquisition unit that acquires biometric information of a person; A second acquisition unit that acquires identification information of baggage carried by the person; a creating unit that creates link information that associates the biometric information with the identification information; a control unit that acquires the result of a baggage inspection that determines whether the baggage contains an object that is restricted from being brought into a restricted area, and the result of a physical inspection that determines whether the person is carrying the object; Equipped with the control unit determines a destination of the baggage based on a result of the baggage inspection, controls the transportation of the baggage to the destination, and causes the signage terminal to display information on the destination of the baggage based on a comparison result between second biometric information of the person acquired by imaging the person at the signage terminal and the biometric information included in the link information. Information processing device.

2. the first acquisition unit and the second acquisition unit acquire the biometric information and the identification information of the person, respectively, from images of the person and the baggage carried by the person. The information processing device according to claim 1 .

3. the control unit changes a detection condition for the target article in one of the baggage inspection and the physical inspection based on a result of the other inspection.

3. The information processing device according to claim 1 or 2.

4. An acquisition unit acquires the attribute information by a process of matching registered biometric information associated in advance with the attribute information of the person with the biometric information, the control unit changes detection conditions for the target article in each of the baggage inspection and the physical inspection based on the attribute information. The information processing device according to claim 1 .

5. the control unit controls removal of the baggage from the storage device based on a comparison result between the third biometric information of the person acquired in the storage device that is the destination and the biometric information included in the link information. The information processing device according to claim 1 .

6. A computer acquires biometric information of a person; said computer acquiring identification information of baggage carried by said person; creating link information by the computer that associates the biometric information with the identification information; obtaining, by the computer, results of a baggage inspection to determine whether the baggage contains an object that is restricted from being brought into a restricted area, and results of a physical inspection to determine whether the person is carrying the object; Equipped with In the step of acquiring the results of the baggage inspection and the physical inspection, the computer determines a destination of the baggage based on the results of the baggage inspection, controls the transportation of the baggage to the destination, and displays information on the destination of the baggage on the signage terminal based on a comparison result between second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information. Information processing methods.

7. On the computer, acquiring biometric information of a person; obtaining identification information for baggage carried by said person; creating link information that associates the biometric information with the identification information; obtaining results of a baggage screening to determine whether the baggage contains an object that is restricted from being brought into a restricted area, and a physical screening to determine whether the person is carrying the object; Run the command, In the step of acquiring the results of the baggage inspection and the physical inspection, a destination of the baggage is determined based on the results of the baggage inspection, and the transportation of the baggage to the destination is controlled; and information on the destination of the baggage is displayed on the signage terminal based on a comparison result between second biometric information of the person acquired by imaging the person at a signage terminal and the biometric information included in the link information. program.

Images