Unauthorized User Identification Device, Unauthorized User Identification Method, and Unauthorized User Identification System

Abstract

To enable an illegal user of a facility to be recognized before a user utilizes the facility without registering a face image of the user in a registration server in advance.SOLUTION: Provided is an illegal user recognition device 30 including: a first detection unit 31 that collects use ticket information indicating a use ticket of a facility from a portable terminal 50 possessed by a user present at an entrance of the facility, and detects an illegal user of the facility based on the use ticket information; and a first video acquisition unit 32 that, when the illegal user is detected by the first detection unit 31, acquires video data on the entrance as video data indicating video in which the illegal user is reflected from a first camera 11-n having captured an image of the entrance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an unauthorized user identification device, an unauthorized user identification method, and an unauthorized user identification system.

Background Art

[0002] There is an unauthorized user identification device for identifying unauthorized users of a facility. Examples of the facility include an event venue, an art museum, or a transportation facility. For example, Non-Patent Document 1 discloses an unauthorized user identification device for identifying unauthorized users of a transportation facility. The unauthorized user identification device disclosed in Non-Patent Document 1 includes a registration server in which face images of users who wish to use the transportation facility are registered, and a camera that captures the face of a user present at the ticket gate of the station and outputs video data showing the face image of the user. Further, the unauthorized user identification device includes a face authentication device that determines that the user is an unauthorized user if the face image of the user shown in the video data output from the camera is not recorded in the registration server.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional unauthorized user identification devices for identifying unauthorized users of a facility have a problem in that face images of users must be registered in a registration server in advance. For example, when the face images of legitimate users of a facility are recorded in a registration server, users whose face images are not recorded in the registration server are recognized as unauthorized users of the facility. In this case, the face images of all users who wish to use the facility must be recorded in the registration server in advance. For example, in the unauthorized user recognition device disclosed in Patent Document 1, the face images of all users who wish to use a transportation facility must be recorded in the registration server in advance. Since the number of users who wish to use a transportation facility is enormous, the number of face images recorded in the registration server is also enormous. Therefore, since the face recognition device needs to compare the face image shown in the video data output from the camera with the enormous number of face images recorded in the registration server, there may be a delay in the face image comparison process. Due to the delay in the face image comparison process by the face recognition device, users may be detained at the ticket gate. On the other hand, in the unauthorized user recognition device, a mode in which the face images of unauthorized users of the facility are registered in the registration server in advance and users whose face images are registered in the registration server are recognized as unauthorized users of the facility can also be considered. For example, in the case of a transportation facility, generally, the number of unauthorized users is small compared to the number of legitimate users, so the number of face images to be registered in the registration server is also relatively small. However, it is difficult to identify and register unauthorized users of the facility in the registration server in advance.

[0005] The present disclosure has been made to solve the above problems, and an object thereof is to obtain an unauthorized user recognition device and an unauthorized user recognition method capable of recognizing unauthorized users of a facility without registering the face images of users in a registration server in advance before the users use the facility.

Means for Solving the Problems

[0006] The unauthorized user identification device according to the present disclosure collects ticket information indicating a facility use ticket from a mobile terminal possessed by a user present at the entrance of a facility, and based on the ticket information, a first detection unit that detects an unauthorized user of the facility, and when an unauthorized user is detected by the first detection unit, a first video acquisition unit that acquires video data of the entrance as video data showing a video in which the unauthorized user appears from a first camera that has photographed the entrance.

Effect of the Invention

[0007] According to the present disclosure, before a user uses a facility, it is possible to identify an unauthorized user of the facility without previously registering the user's face image in a registration server.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0009] Hereinafter, in order to explain the present disclosure in more detail, modes for carrying out the present disclosure will be described with reference to the accompanying drawings.

[0010] Embodiment 1. FIG. 1 is a configuration diagram showing an unauthorized user authentication system including an unauthorized user authentication device 30 according to Embodiment 1. FIG. 2 is a configuration diagram showing the unauthorized user authentication device 30 according to Embodiment 1. FIG. 3 is a hardware configuration diagram showing the hardware of the unauthorized user authentication device 30 according to Embodiment 1. In the unauthorized user authentication system shown in FIG. 1, the facility is a transportation facility, and the unauthorized user authentication device 30 authenticates unauthorized users of the transportation facility. The boarding station 1 is a part of the facility included in the transportation facility and has N entrance gates. N is an integer of 1 or more. The alighting station 2 is a part of the facility included in the transportation facility and has M exit gates. M is an integer of 1 or more. In the unauthorized user authentication system shown in FIG. 1, the facility is a transportation facility. However, this is only an example, and the facility may be, for example, an event venue or an art museum.

[0011] At the boarding station 1, N first cameras 11-1 to 11-N and an unauthorized user image receiving device 12 are installed. At the alighting station 2, M second cameras 21-1 to 21-M and an unauthorized user image receiving device 22 are installed. The unauthorized user identification device 30 includes a first detection unit 31, a first video acquisition unit 32, a second detection unit 33, a second video acquisition unit 34, and an unauthorized user identification unit 35. The unauthorized user identification device 30 is a device that can identify unauthorized users of the facility without the user registering their face image in advance with a registration server before using the facility. Since the unauthorized user identification device 30 does not require the user to register their face image in advance with the registration server, it is possible to simplify the configuration and reduce the processing load. The first cameras 11-1 to 11-N, the unauthorized user image receiving device 12, the second cameras 21-1 to 21-M, the unauthorized user image receiving device 22, and the unauthorized user identification device 30 are connected to each other via a network 3. Examples of the network 3 include the Internet or a LAN (Local Area Network).

[0012] The first detection unit 31 is realized, for example, by the first detection circuit 41 shown in FIG. 3. When a user present at the entrance of the boarding station 1 is detected, the first detection unit 31 transmits a request to transmit ticket information indicating the facility use ticket to the mobile terminal 50 held by the user. The first detection unit 31 collects the ticket information transmitted from the mobile terminal 50 and detects unauthorized users of the facility based on the ticket information. When the first detection unit 31 detects an unauthorized user of the facility, it indicates the detection result R D1 to the first camera 11-n among the first cameras 11-1 to 11-N that is photographing the entrance where the user is present. The detection result R indicating that D1 includes the detection time t, which is the time when the unauthorized user of the facility was detected by the first detection unit 31. D1 is included. Also, the first detection unit 31 outputs a detection result R indicating that fact D1 to the first video acquisition unit 32.

[0013] The first camera 11-n (n = 1, ···, N) photographs any entrance at the boarding station 1. The first camera 11-n obtains a detection result R indicating that an unauthorized user of the facility has been detected from the first detection unit 31. D1 The first camera 11-n transmits, to the first video acquisition unit 32, the video data P IN of the entrance, D1 including the detection time t D1 included in the detection result R. IN,tD1 The video shown by the video data P of the entrance IN may be a moving image or a still image.

[0014] The first video acquisition unit 32 is realized, for example, by the first video acquisition circuit 42 shown in FIG. 3. After obtaining the detection result R from the first detection unit 31, the first video acquisition unit 32 obtains, as video data showing an image of an unauthorized user, the video data P D1 at the detection time t D1 from the first camera 11-n that has photographed the entrance where the user is present. IN,tD1 The first video acquisition unit 32 outputs the video data P D1 at the detection time t IN,tD1 to the unauthorized user determination unit 35.

[0015] The second detection unit 33 is realized, for example, by the second detection circuit 43 shown in FIG. 3. When a user present at the exit of the alighting station 2 is detected, the second detection unit 33 transmits a transmission request for ticket information indicating a facility use ticket to the mobile terminal 50 held by the user. The second detection unit 33 collects the ticket information transmitted from the mobile terminal 50 and detects an unauthorized user of the facility based on the ticket information. ​​​When the second detection unit 33 detects an unauthorized user of the facility, it indicates the detection result R to that effect. D2 The detection result R to that effect is transmitted to the second camera 21-m among the second cameras 21-1 to 21-M that is capturing the exit where the user is present. The detection result R to that effect D2 includes the detection time t, which is the time when the unauthorized user of the facility was detected by the second detection unit 33. D2 is included. In addition, the second detection unit 33 outputs the detection result R to that effect D2 to the second video acquisition unit 34.

[0016] The second camera 21-m (m = 1, ···, M) captures any one of the exits at the disembarkation station 2. The second camera 21-m obtains the detection result R indicating that an unauthorized user of the facility has been detected from the second detection unit 33. D2 is obtained. The second camera 21-m transmits the video data P OUT of the exit, among which the video data P D2 corresponding to the detection time t D2 included in the detection result R OUT,tD2 to the second video acquisition unit 34. The video shown by the video data P OUT of the exit may be a moving image or a still image.

[0017] The second video acquisition unit 34 is realized, for example, by the second video acquisition circuit 44 shown in FIG. 3. After obtaining the detection result R from the second detection unit 33, the second video acquisition unit 34 obtains the video data P D2 corresponding to the detection time t as video data showing the video in which the unauthorized user appears from the second camera 21-m that captures the exit where the user is present. D2 of the exit OUT,tD2 is obtained. The second video acquisition unit 34 outputs the video data P D2 corresponding to the detection time t OUT,tD2 to the unauthorized user determination unit 35.

[0018] The unauthorized user identification unit 35 is realized by, for example, the unauthorized user identification circuit 45 shown in FIG. 3. The unauthorized user identification unit 35 includes a recording medium 35a. The recording medium 35a corresponds to, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory) or a flash memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, or a mini disk. The unauthorized user identification unit 35 obtains the video data P D1 at the detection time t IN,tD1 from the first video acquisition unit 32. The unauthorized user identification unit 35 obtains the video data P D2 at the detection time t OUT,tD2 from the second video acquisition unit 34. In the unauthorized user identification system shown in FIG. 1, the images respectively shown by the video data P D1 at the detection time t IN,tD1 and the video data P D2 at the detection time t OUT,tD2 are still images. However, this is only an example, and the images shown by the respective image data are not limited to still images and may be moving images. However, when the images shown by the respective image data are moving images, the video data P D1 at the detection time t IN,tD1 is the video data in the time range including the detection time t D1 , and the video data P D2 at the detection time t OUT,tD2 is the video data in the time range including the detection time t D2 .

[0019] The unauthorized user identification unit 35 compares the video shown by the video data P D1 at the detection time t IN,tD1 with the video shown by the video data P D2 at the detection time t OUT,tD2 and searches for the user shown in the videos of both video data. The unauthorized user identification unit 35 determines that the user shown in the videos of both video data is an unauthorized user of the facility. The unauthorized user identification unit 35 causes the video of the user identified as an unauthorized user of the facility to be recorded on the recording medium 35a. When the unauthorized user identification unit 35 identifies an unauthorized user of the facility, it transmits the video of the user to each of the unauthorized user image receiving devices 12 and 22. In the unauthorized user identification system shown in FIG. 1, the unauthorized user identification unit 35 includes the recording medium 35a. However, this is only an example, and the recording medium 35a may be provided outside the unauthorized user identification device 30.

[0020] The unauthorized user image receiving device 12 acquires the video of the user from the unauthorized user identification unit 35. The unauthorized user image receiving device 12 causes the video of the user to be displayed on a display (not shown), and also causes a message indicating that an unauthorized user of the facility has been detected to be displayed on the display. The unauthorized user image receiving device 22 acquires the video of the user from the unauthorized user identification unit 35. The unauthorized user image receiving device 22 causes the video of the user to be displayed on a display (not shown), and also causes a message indicating that an unauthorized user of the facility has been detected to be displayed on the display. In the unauthorized user identification system shown in FIG. 1, each of the unauthorized user image receiving devices 12 and 22 causes a message indicating that an unauthorized user of the facility has been detected to be displayed on the display. However, this is only an example, and a voice output may be made from a speaker (not shown) to indicate that an unauthorized user of the facility has been detected.

[0021] The mobile terminal 50 is realized by, for example, a smartphone or an IC (Integrated Circuit) card. In the internal memory of the mobile terminal 50, if the user who holds the mobile terminal 50 is a legitimate user who can use the facility, ticket information indicating a legitimate use ticket for the facility is recorded. When the mobile terminal 50 receives a transmission request for the ticket information from the first detection unit 31, it transmits the ticket information recorded in the internal memory to the first detection unit 31. Also, when the mobile terminal 50 receives a transmission request for the ticket information from the second detection unit 33, it transmits the ticket information recorded in the internal memory to the second detection unit 33.

[0022] In FIG. 2, it is assumed that each of the first detection unit 31, the first video acquisition unit 32, the second detection unit 33, the second video acquisition unit 34, and the unauthorized user identification unit 35, which are components of the unauthorized user identification device 30, is realized by dedicated hardware as shown in FIG. 3. That is, it is assumed that the unauthorized user identification device 30 is realized by the first detection circuit 41, the first video acquisition circuit 42, the second detection circuit 43, the second video acquisition circuit 44, and the unauthorized user identification circuit 45. Each of the first detection circuit 41, the first video acquisition circuit 42, the second detection circuit 43, the second video acquisition circuit 44, and the unauthorized user identification circuit 45 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0023] The components of the unauthorized user identification device 30 are not limited to those realized by dedicated hardware, and the unauthorized user identification device 30 may be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored in the memory of the computer as a program. The computer means the hardware that executes the program, and for example, a CPU (Central Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor) corresponds to it.

[0024] FIG. 4 is a hardware configuration diagram of a computer when the unauthorized user identification device 30 is implemented by software, firmware, or the like. When the unauthorized user identification device 30 is implemented by software, firmware, or the like, a program for causing a computer to execute respective processing procedures in the first detection unit 31, the first video acquisition unit 32, the second detection unit 33, the second video acquisition unit 34, and the unauthorized user identification unit 35 is stored in the memory 81. Then, the processor 82 of the computer executes the program stored in the memory 81.

[0025] In addition, FIG. 3 shows an example in which each component of the unauthorized user identification device 30 is implemented by dedicated hardware, and FIG. 4 shows an example in which the unauthorized user identification device 30 is implemented by software, firmware, or the like. However, this is merely an example, and some components of the unauthorized user identification device 30 may be implemented by dedicated hardware and the remaining components may be implemented by software, firmware, or the like.

[0026] FIG. 5 is a configuration diagram showing the first cameras 11-n (n = 1, ···, N). The first camera 11-n shown in FIG. 5 includes a photographing unit 61, a face extraction unit 62, a recording unit 63, an encoding unit 64, and a communication unit 65. The photographing unit 61 is realized by, for example, a CCD (Charge Coupled Device). The photographing unit 61 photographs any entrance at the boarding station 1 and outputs the photographed video data to the face extraction unit 62.

[0027] The face extraction unit 62 extracts the face image of the user from the video shown by the video data output from the photographing unit 61. If there are a plurality of users at the photographed entrance, the face extraction unit 62 extracts the face images of the respective users. The face extraction unit 62 outputs video data P showing a video including the extracted face image of the user IN to the recording unit 63.

[0028] The recording unit 63 is implemented by, for example, a non-volatile or volatile semiconductor memory such as a RAM or a flash memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, or a mini disk. The recording unit 63 records the video data P IN output from the face extraction unit 62.

[0029] When receiving a notification indicating that the detection result R D1 has been received from the communication unit 65, the encoding unit 64 acquires the video data P IN recorded in the recording unit 63 and included in the detection result R D1 at the detection time t D1 contained in the detection result R. IN,tD1 The encoding unit 64 encodes the video data P D1 at the detection time t IN,tD1 in an arbitrary video compression format, and outputs the encoded video data P IN,tD1 to the communication unit 65. Examples of the arbitrary video compression format include, for example, the H.264 encoding method or the JPEG (Joint Photographic Experts Group) encoding method. The encoded video data P IN,tD1 is, for example, IP (Internet Protocol) data.

[0030] When the detection result R D1 is transmitted from the first detection unit 31, the communication unit 65 receives the detection result R D1 and outputs a notification indicating that the detection result R D1 has been received to the encoding unit 64. The communication unit 65 transmits the encoded video data P IN,tD1 output from the encoding unit 64 to the first video acquisition unit 32.

[0031] FIG. 6 is a configuration diagram showing the second camera 21-m (m = 1, ···, M). The second camera 21-m shown in FIG. 6 includes a photographing unit 71, a face extraction unit 72, a recording unit 73, an encoding unit 74, and a communication unit 75. The imaging unit 71 is realized by, for example, a CCD. The imaging unit 71 captures any exit at the getting-off station 2, and outputs the captured video data to the face extraction unit 72.

[0032] The face extraction unit 72 extracts the face image of the user from the video indicated by the video data output from the imaging unit 71. If there are a plurality of users at the captured exit, the face extraction unit 72 extracts the face images of the respective users. The face extraction unit 72 outputs video data P indicating a video including the extracted face image of the user OUT to the recording unit 73.

[0033] The recording unit 73 is realized by, for example, a non-volatile or volatile semiconductor memory such as a RAM, ROM, flash memory, magnetic disk, flexible disk, optical disk, compact disk, or mini disk. The recording unit 73 records the video data P OUT output from the face extraction unit 72.

[0034] When the encoding unit 74 receives a notification indicating that it has received the detection result R D2 from the communication unit 75, it acquires the video data P OUT in the video data P D2 recorded in the recording unit 73 and including the detection time t D2 contained in the detection result R. OUT,tD2 The encoding unit 74 encodes the video data P D2 at the detection time t OUT,tD2 in an arbitrary video compression format, and outputs the encoded video data P OUT,tD2 to the communication unit 75. The encoded video data P OUT,tD2 is, for example, IP data.

[0035] When the communication unit 75 receives the detection result R D2 sent from the second detection unit 33, it receives the detection result R D2 and outputs a notification indicating that it has received the detection result R D2 to the encoding unit 74.​ The communication unit 75 transmits the encoded video data P output from the encoding unit 74 OUT,tD2 to the second video acquisition unit 34.

[0036] Next, the operation of the unauthorized user identification system shown in FIG. 1 will be described. Each of FIGS. 7 and 8 is a flowchart showing an unauthorized user identification method which is a processing procedure of the unauthorized user identification device 30 according to Embodiment 1. When a certain user goes to any entrance at the boarding station 1, for example, a motion sensor (not shown) installed at the entrance detects the user. The motion sensor transmits a detection result indicating that the user has been detected to the first detection unit 31 of the unauthorized user identification device 30.

[0037] When the first detection unit 31 receives a detection result indicating that the user has been detected from the motion sensor, it transmits a request for transmitting ticket information indicating a facility use ticket to the mobile terminal 50 held by the user (step ST1 in FIG. 7). In the unauthorized user identification system shown in FIG. 1, the facility use ticket is a transportation ticket. When the mobile terminal 50 receives a request for transmitting ticket information from the first detection unit 31, it transmits the ticket information recorded in the internal memory to the first detection unit 31. In the unauthorized user identification system shown in FIG. 1, the mobile terminal 50 has received a request for transmitting ticket information from the first detection unit 31. However, this is only an example. For example, when the user brings the mobile terminal 50 close to the touch sensor at the ticket gate, the mobile terminal 50 may receive a request for transmitting ticket information from the touch sensor.

[0038] After transmitting the request for transmitting ticket information to the mobile terminal 50, the first detection unit 31 receives the ticket information transmitted from the mobile terminal 50. The first detection unit 31 detects an unauthorized user of the facility based on the ticket information. That is, if the ticket indicated by the ticket information satisfies the facility usage conditions, the first detection unit 31 determines that the user is a legitimate user of the facility; if it does not satisfy the facility usage conditions, the first detection unit 31 detects an unauthorized user of the facility. Examples of the mode in which the facility usage conditions are not satisfied include cases where the available period of the facility has already passed. Also, if the facility is a transportation facility, it corresponds to a case where the boarding station 1 is not included in the available boarding section. In the unauthorized user identification system shown in FIG. 1, the first detection unit 31 detects an unauthorized user of the facility based on the ticket information. However, this is only an example. For example, even if a certain period of time has elapsed after the first detection unit 31 transmits a transmission request to the mobile terminal 50 and the ticket information has not been transmitted from the mobile terminal 50, the first detection unit 31 may be configured to detect an unauthorized user of the facility. For example, depending on the transmission status of the network 3, there is a possibility that incorrect ticket information is transmitted to the first detection unit 31. Therefore, there is a possibility that the detection of unauthorized users based on the ticket information is a false detection.

[0039] If the first detection unit 31 detects an unauthorized user of the facility (in the case of YES in step ST2 of FIG. 7), the detection result R indicating that D1 is transmitted to the first camera 11-n that has photographed the entrance where the user is present among the first cameras 11-1 to 11-N (step ST3 of FIG. 7). The detection result R indicating that D1 includes the detection time t, which is the time when the unauthorized user was detected by the first detection unit 31. D1 Also, the first detection unit 31 outputs the detection result R indicating that D1 to the first video acquisition unit 32. If the first detection unit 31 does not detect an unauthorized user of the facility (in the case of NO in step ST2 of FIG. 7), the detection result R indicating that D1 is not transmitted to the first camera 11-n that has photographed the entrance where the user is present. Also, the first detection unit 31 does not output the detection result R indicating that D1 to the first video acquisition unit 32.

[0040] When an unauthorized user is detected by the first detection unit 31, generally, the gate at the entrance is closed, so a user present at the entrance cannot pass through the gate at the entrance. However, if the user is walking in a line following another user walking in front of the user, the other user may pass through the gate, and the user may pass through the gate before the gate closes. Also, when a malfunction occurs at the gate at the entrance and the gate remains open, the user may pass through the gate. Note that if the unauthorized user identification system shown in FIG. 1 is, for example, a system applied to a gateless ticket barrier, even if an unauthorized user is detected by the first detection unit 31, the unauthorized user can enter the facility. A system applied to a gateless ticket barrier is, for example, intended to collect a regular usage fee from an unauthorized user.

[0041] Among the first cameras 11-1 to 11-N, the imaging unit 61 of the first camera 11-n installed at the same entrance as the human sensor that detected the user captures the entrance and outputs video data showing an image as shown in FIG. 9 to the face extraction unit 62. FIG. 9 is an explanatory diagram showing an example of an image shown by the video data captured by the imaging unit 61. The face extraction unit 62 extracts the user's face image by cutting out the portion surrounded by the dashed line shown in FIG. 10 from the image shown by the video data output from the imaging unit 61. If there are a plurality of users at the captured entrance, the face extraction unit 62 extracts the face images of each user. FIG. 10 is an explanatory diagram showing the cutout range of the video by the face extraction unit 62. FIG. 11 is an explanatory diagram showing an example of the face image of the user cut out by the face extraction unit 62. The face extraction unit 62 causes the recording unit 63 to record video data P showing a video including the cut-out face image of the user. IN to be recorded in the recording unit 63.

[0042] The communication unit 65 receives the detection result R D1 when it is transmitted from the first detection unit 31, and outputs a notification indicating that the detection result R D1 has been received to the encoding unit 64. D1 When the encoding unit 64 receives a notification indicating that the detection result R has been received from the communication unit 65, it acquires the video data P D1 recorded in the recording unit 63, IN and from among the video data P D1 the video data P D1 corresponding to the detection time t IN,tD1 included in the detection result R. The encoding unit 64 encodes the video data P D1 corresponding to the detection time t IN,tD1 and outputs the encoded video data P IN,tD1 to the communication unit 65. The communication unit 65 transmits the encoded video data P IN,tD1 output from the encoding unit 64 to the first video acquisition unit 32.

[0043] The first video acquisition unit 32 of the unauthorized user identification device 30 acquires the detection result R D1 from the first detection unit 31. After the first video acquisition unit 32 acquires the detection result R D1 from the first detection unit 31, it acquires the encoded video data P IN,tD1 transmitted from the communication unit 65 as video data showing the video of the unauthorized user at the entrance (step ST4 in FIG. 7). The first video acquisition unit 32 outputs the video data P D1 corresponding to the detection time t IN,tD1 to the unauthorized user identification unit 35.

[0044] The unauthorized user identification unit 35 acquires the video data P D1 corresponding to the detection time t IN,tD1 from the first video acquisition unit 32. The unauthorized user identification unit 35 decodes the video data P IN,tD1 and the decoded video data P IN,tD1If the face image of the user included in the video shown is already recorded on the recording medium 35a, it is determined that the user is an unauthorized user of the facility. If the unauthorized user determination unit 35 determines that the user is an unauthorized user of the facility, the video data P IN,tD1 after decoding, the face image of the user included in the video shown is transmitted to the unauthorized user image receiving device 12. The unauthorized user determination unit 35 IN,tD1 If the face image of the user included in the video shown is not yet recorded on the recording medium 35a, at this stage, it is not determined that the user is an unauthorized user of the facility.

[0045] When a certain user goes to any exit at the alighting station 2, for example, a human presence sensor (not shown) installed at the exit detects the user. The human presence sensor transmits a detection result indicating that the user has been detected to the second detection unit 33 of the unauthorized user determination device 30.

[0046] When the second detection unit 33 receives a detection result indicating that the user has been detected from the human presence sensor, it transmits a request to transmit ticket information indicating the facility use ticket to the mobile terminal 50 held by the user (step ST5 in FIG. 8). When the mobile terminal 50 receives a request to transmit ticket information from the second detection unit 33, it transmits the ticket information recorded in the internal memory to the second detection unit 33. In the unauthorized user determination system shown in FIG. 1, the mobile terminal 50 has received a request to transmit ticket information from the second detection unit 33. However, this is just an example, and when the user brings the mobile terminal 50 close to the touch sensor of the ticket gate, the mobile terminal 50 may receive a request to transmit ticket information from the touch sensor.

[0047] After transmitting the request to transmit ticket information to the mobile terminal 50, the second detection unit 33 receives the ticket information transmitted from the mobile terminal 50. The second detection unit 33 detects unauthorized users of the facility based on the ticket information. That is, if the ticket indicated by the ticket information satisfies the facility usage conditions, the second detection unit 33 determines that the user is a legitimate user of the facility, and if the facility usage conditions are not satisfied, the second detection unit 33 detects an unauthorized user of the facility. In the unauthorized user identification system shown in FIG. 1, the second detection unit 33 detects an unauthorized user of the facility based on the ticket information. However, this is only an example. For example, if the second detection unit 33 transmits a transmission request to the mobile terminal 50 and the ticket information is not transmitted from the mobile terminal 50 even after a certain period of time has elapsed, the second detection unit 33 may detect an unauthorized user of the facility.

[0048] If the second detection unit 33 detects an unauthorized user of the facility (in the case of YES in step ST6 of FIG. 8), the detection result R indicating this D2 is transmitted to the second camera 21-m that has captured the exit where the user is present among the second cameras 21-1 to 21-M (step ST7 of FIG. 8). The detection result R indicating this D2 includes the detection time t, which is the time when the unauthorized user was detected by the second detection unit 33. D2 is included. In addition, the second detection unit 33 outputs the detection result R indicating this D2 to the second video acquisition unit 34. If the second detection unit 33 does not detect an unauthorized user of the facility (in the case of NO in step ST6 of FIG. 8), the detection result R indicating this D2 is not transmitted to the second camera 21-n that has captured the exit where the user is present. Also, the second detection unit 33 does not output the detection result R indicating this D2 to the second video acquisition unit 34.

[0049] When an unauthorized user is detected by the second detection unit 33, generally, the gate at the exit is closed, so the user present at the exit cannot pass through the gate at the exit. However, if the user is walking in a line with another user walking in front of the user, the user may pass through the gate before the gate closes when the other user passes through the gate. In addition, even when a malfunction occurs at the gate of the exit and the gate remains open, the user may pass through the gate. If the unauthorized user identification system shown in FIG. 1 is, for example, a system applied to a gate-less ticket barrier, even when an unauthorized user is detected by the second detection unit 33, the unauthorized user can leave the facility.

[0050] Of the second cameras 21-1 to 21-M, the imaging unit 71 of the second camera installed at the same exit as the human sensor that detected the user captures the exit and outputs the captured video data to the face extraction unit 72. The face extraction unit 72 extracts the face image of the user from the video shown by the video data output from the imaging unit 71. If there are a plurality of users at the captured exit, the face extraction unit 72 extracts the face images of the respective users. The face image extraction process by the face extraction unit 72 is the same as the face image extraction process by the face extraction unit 72. The face extraction unit 72 outputs video data P showing a video including the face image of the extracted user OUT to the recording unit 73.

[0051] The communication unit 75 receives the detection result R D2 when it is transmitted from the second detection unit 33, and outputs a notification indicating that the detection result R D2 has been received to the encoding unit 74. D2 When the encoding unit 74 receives a notification that the detection result R has been received from the communication unit 75, it acquires the video data P D2 recorded in the recording unit 73 and including the detection time t OUT contained in the detection result R D2 from the video data P D2 at the detection time t. OUT,tD2 The encoding unit 74 encodes the video data P D2 at the detection time t OUT,tD2 and outputs the encoded video data P OUT,tD2 to the communication unit 75. ​The communication unit 75 transmits the encoded video data P output from the encoding unit 74 OUT,tD2 to the second video acquisition unit 34.

[0052] The second video acquisition unit 34 of the unauthorized user identification device 30 obtains the detection result R D2 from the second detection unit 33. After the second video acquisition unit 34 obtains the detection result R D2 from the second detection unit 33, it obtains the encoded video data P transmitted from the communication unit 75 as video data showing a video in which an unauthorized user at the exit is shown (step ST8 in FIG. 8). OUT,tD2 The second video acquisition unit 34 outputs the video data P D2 at the detection time t OUT,tD2 to the unauthorized user identification unit 35.

[0053] The unauthorized user identification unit 35 obtains the video data P D2 at the detection time t OUT,tD2 from the second video acquisition unit 34. The unauthorized user identification unit 35 decodes the video data P OUT,tD2 . The unauthorized user identification unit 35 collates the face image of the user included in the video shown by the decoded video data P IN,tD1 at the entrance with the face image of the user included in the video shown by the decoded video data P OUT,tD2 at the exit. Then, the unauthorized user identification unit 35 searches for a user whose face image is included in both the video shown by the video data P IN,tD1 and the video shown by the video data P OUT,tD2 (step ST9 in FIG. 8). If there is a user whose face image is included in the videos shown by both pieces of video data, the unauthorized user identification unit 35 determines that the user is an unauthorized user of the facility (step ST10 in FIG. 8). The unauthorized user identification unit 35 records the face image of the user determined to be an unauthorized user of the facility on the recording medium 35a. ​In addition, the unauthorized user identification unit 35 transmits the face image of the user identified as an unauthorized user of the facility to each of the unauthorized user image receiving devices 12 and 22. If there is no user whose face image is included in the video shown by the video data of both, the unauthorized user identification unit 35 does not make a determination that the user is an unauthorized user of the facility.

[0054] When the face image of the user is transmitted from the unauthorized user identification unit 35, the unauthorized user image receiving device 12 receives the face image of the user. The unauthorized user image receiving device 12 causes the received face image of the user to be displayed on a display (not shown) as the face image of the unauthorized user of the facility, and causes the display to show that the unauthorized user of the facility has been detected. When the face image of the user is transmitted from the unauthorized user identification unit 35, the unauthorized user image receiving device 22 receives the face image of the user. The unauthorized user image receiving device 22 causes the received face image of the user to be displayed on a display (not shown) as the face image of the unauthorized user of the facility, and causes the display to show that the unauthorized user of the facility has been detected.

[0055] In the above-described first embodiment, the first detection unit 31 that collects the ticket information indicating the facility use ticket from the mobile terminal 50 possessed by the user present at the entrance of the facility, and detects an unauthorized user of the facility based on the ticket information, and when an unauthorized user is detected by the first detection unit 31, the first video acquisition unit 32 that acquires the video data of the entrance as the video data showing the video in which the unauthorized user appears from the first camera 11-n that photographed the entrance, are provided to configure the unauthorized user identification device 30. Therefore, the unauthorized user identification device 30 can identify an unauthorized user of the facility without previously registering the face image of the user in the registration server before the user uses the facility.

[0056] In the unauthorized user authentication system shown in FIG. 1, the face extraction unit 62 of the first camera 11-n (n = 1, ···, N) extracts the face image of the user from the video shown in the video data output from the imaging unit 61, and the communication unit 65 transmits the encoded video data indicating the video including the extracted face image of the user to the first video acquisition unit 32. Also, the face extraction unit 72 of the second camera 21-m (m = 1, ···, M) extracts the face image of the user from the video shown in the video data output from the imaging unit 71, and the communication unit 75 transmits the encoded video data indicating the video including the extracted face image of the user to the second video acquisition unit 34. However, this is only an example, and the encoding unit 64 may encode the video data output from the imaging unit 61, the communication unit 65 may transmit the encoded video data to the first video acquisition unit 32, the encoding unit 74 may encode the video data output from the imaging unit 71, and the communication unit 75 may transmit the encoded video data to the second video acquisition unit 34. In this case, the unauthorized user authentication unit 35 of the unauthorized user authentication device 30 extracts the face image of the user from the video shown by the decoded video data P IN,tD1 and extracts the face image of the user from the video shown by the decoded video data P OUT,tD2 and extracts the face image of the user from the video shown by the decoded video data P. In this case, the first camera 11-n does not need to implement the face extraction unit 62, and the second camera 21-m does not need to implement the face extraction unit 72.

[0057] In the unauthorized user authentication system shown in FIG. 1, the unauthorized user authentication device 30 includes the first detection unit 31 and the second detection unit 33. However, this is only an example, and as shown in FIG. 12, the first detection unit 31 may be installed at the boarding station 1 and the second detection unit 33 may be installed at the alighting station 2. In this case, the unauthorized user authentication device 30 does not need to include the first detection unit 31 and the second detection unit 33 as shown in FIG. 13. FIG. 12 is a configuration diagram showing an unauthorized user authentication system including another unauthorized user authentication device 30 according to Embodiment 1. FIG. 13 is a configuration diagram showing another unauthorized user identification device 30 according to Embodiment 1.

[0058] In the unauthorized user identification system shown in FIG. 1, the unauthorized user identification unit 35 of the unauthorized user identification device 30 unconditionally collates the face image of the user included in the video shown by the decoded video data P at the entrance IN,tD1 with the face image of the user included in the video shown by the decoded video data P at the exit. OUT,tD2 However, this is only an example, and the unauthorized user identification unit 35, as shown in the following formula (1), is the detection time t which is the time when an unauthorized user is detected by the first detection unit 31 and the detection time t which is the time when an unauthorized user is detected by the second detection unit 33 D1 to calculate the time difference Δt therebetween. D2 Δt = |t D1 - t D2 | (1) Further, the unauthorized user identification unit 35 acquires time information indicating the required time TR, for example, from a server operated by a transportation agency. The required time TR indicated by the time information is the travel time from boarding station 1 to alighting station 2. The unauthorized user identification unit 35 collates the face image of the user included in the video shown by the decoded video data P at the entrance IN,tD1 with the face image of the user included in the video shown by the decoded video data P at the exit OUT,tD2 only when the difference between the time difference Δt and the required time TR is equal to or less than a threshold value. The threshold value may be stored in the internal memory of the unauthorized user identification unit 35 or may be given from outside the unauthorized user identification device 30. If the difference between the time difference Δt and the required time TR is equal to or less than the threshold value, the probability that the user present at the entrance of boarding station 1 and the user present at the exit of alighting station 2 are the same user is high. On the other hand, if the difference between the time difference Δt and the required time TR is greater than the threshold value, the probability that the user present at the entrance of boarding station 1 and the user present at the exit of alighting station 2 are different users is high. ​If the difference between the time difference Δt and the required time TR is equal to or less than the threshold value, the unauthorized user determination unit 35 performs the face image matching process only when the probability that the users are different is high, thereby omitting unnecessary matching processes.

[0059] Embodiment 2. In the unauthorized user determination system shown in FIG. 1, the face extraction unit 62 of the first camera 11-n (n = 1, ···, N) extracts the face image of the user from the video shown by the video data output from the imaging unit 61. Further, the face extraction unit 72 of the second camera 21-m (m = 1, ···, M) extracts the face image of the user from the video shown by the video data output from the imaging unit 71.

[0060] However, this is only an example, and instead of the face extraction unit 62 extracting the face image of the user, the face extraction unit 62 may extract the skeleton image of the user. Further, instead of the face extraction unit 72 extracting the face image of the user, the face extraction unit 72 may extract the skeleton image of the user. Since the skeleton image extraction process itself is a known technique, a detailed description thereof is omitted. The video shown by the video data at this time is, for example, an X-ray video. In this case, the unauthorized user determination unit 35 of the unauthorized user determination device 30 compares the skeleton image of the user included in the video shown by the decrypted video data P IN,tD1 at the entrance with the skeleton image of the user included in the video shown by the decrypted video data P OUT,tD2 at the exit, and searches for a user whose skeleton image is included in the videos shown by both pieces of video data. If there is a user whose skeleton image is included in the videos shown by both pieces of video data, the unauthorized user determination unit 35 determines that the user is an unauthorized user of the facility. The unauthorized user determination unit 35 records the skeleton image of the user determined to be an unauthorized user of the facility on the recording medium 35a. In addition, the unauthorized user identification unit 35 transmits the skeletal image of the user identified as an unauthorized user of the facility to each of the unauthorized user image receiving devices 12 and 22. If there is no user whose skeletal image is included in the video shown by the video data of both, the unauthorized user identification unit 35 does not make a determination that the user is an unauthorized user of the facility.

[0061] As described above, similar to the unauthorized user identification device 30 according to the first embodiment, the unauthorized user identification device 30 according to the second embodiment can identify an unauthorized user of the facility without registering the face image of the user in advance in the registration server before the user uses the facility.

[0062] Note that in the present disclosure, a free combination of each embodiment, a modification of any component of each embodiment, or an omission of any component in each embodiment is possible.

Description of Reference Numerals

[0063] 1 boarding station, 2 alighting station, 3 network, 11-1 to 11-N first cameras, 12 unauthorized user image receiving device, 21-1 to 21-M second cameras, 22 unauthorized user image receiving device, 30 unauthorized user identification device, 31 first detection unit, 32 first video acquisition unit, 33 second detection unit, 34 second video acquisition unit, 35 unauthorized user identification unit, 35a recording medium, 41 first detection circuit, 42 first video acquisition circuit, 43 second detection circuit, 44 second video acquisition circuit, 45 unauthorized user identification circuit, 50 mobile terminal, 61 imaging unit, 62 face extraction unit, 63 recording unit, 64 encoding unit, 65 communication unit, 71 imaging unit, 72 face extraction unit, 73 recording unit, 74 encoding unit, 75 communication unit, 81 memory, 82 processor.

Claims

Claim 1. A first detection unit that collects ticket information indicating a facility use ticket from a mobile terminal held by a user present at an entrance of a facility, and detects an unauthorized user of the facility based on the ticket information; A first video acquisition unit that, when an unauthorized user is detected by the first detection unit, acquires video data of the entrance as video data indicating a video in which the unauthorized user appears, from a first camera that has photographed the entrance; A second detection unit that collects the ticket information from a mobile terminal held by a user present at an exit of the facility, and detects an unauthorized user of the facility based on the ticket information; A second video acquisition unit that, when an unauthorized user is detected by the second detection unit, acquires video data of the exit as video data indicating a video in which the unauthorized user appears, from a second camera that has photographed the exit; An unauthorized user determination unit that determines that a user appearing in both the video indicated by the video data of the entrance acquired by the first video acquisition unit and the video indicated by the video data of the exit acquired by the second video acquisition unit is an unauthorized user of the facility; An unauthorized user determination device comprising the above.

2. The video indicated by the video data of the entrance acquired from the first camera is a video including a face image of a user present at the entrance; The video indicated by the video data of the exit acquired from the second camera is a video including a face image of a user present at the exit; The unauthorized user determination unit determines that a user whose face image is included in both the video indicated by the video data of the entrance and the video indicated by the video data of the exit is an unauthorized user of the facility. The unauthorized user determination device according to Claim 1.

3. The unauthorized user determination unit is provided with a recording medium for recording a face image of a user determined to be an unauthorized user of the facility. The unauthorized user determination device according to Claim 2.

4. The unauthorized user determination unit determines that a user is an unauthorized user of the facility if a face image of the user included in the video indicated by the video data of the entrance acquired by the first video acquisition unit has already been recorded on the recording medium. The unauthorized user determination device according to Claim 3.

5. The video shown by the video data of the entrance obtained from the first camera is a video including a skeletal image of a user present at the entrance. The video shown by the video data of the exit obtained from the second camera is a video including a skeletal image of a user present at the exit. The unauthorized user determination unit determines that a user whose skeletal image is included in both the video shown by the video data of the entrance and the video shown by the video data of the exit is an unauthorized user of the facility. The unauthorized user determination device according to claim 1, characterized in that.

6. The unauthorized user determination unit The unauthorized user determination device according to claim 5, characterized in that it includes a recording medium for recording the skeletal image of the user determined to be an unauthorized user of the facility.

7. The unauthorized user determination unit If the skeletal image of the user included in the video shown by the video data of the entrance obtained by the first video acquisition unit has already been recorded on the recording medium, the unauthorized user determination device according to claim 6, characterized in that it determines that the user is an unauthorized user of the facility.

8. The facility is a transportation facility, the entrance of the facility is the entrance of a railway station, the exit of the facility is the exit of a railway station, The unauthorized user determination unit Calculates the time difference between the time when an unauthorized user is detected by the first detection unit and the time when an unauthorized user is detected by the second detection unit, and only when the difference between the required time from the boarding station to the alighting station and the time difference is equal to or less than a threshold value, the video shown by the video data of the entrance obtained by the first video acquisition unit and the video shown by the video data of the exit obtained by the second video acquisition unit The unauthorized user determination device according to claim 1, characterized in that it determines that the user shown in both is an unauthorized user of the facility.

9. The first detection unit collects ticket information indicating a ticket for using the facility from a mobile terminal possessed by a user present at the entrance of the facility, and based on the ticket information, detects an unauthorized user of the facility. When an unauthorized user is detected by the first detection unit, the first video acquisition unit acquires the video data of the entrance as video data showing a video in which the unauthorized user appears from the first camera that has photographed the entrance. The second detection unit collects the ticket information indicating the facility ticket from the mobile terminal possessed by the user present at the exit of the facility, and detects an unauthorized user of the facility based on the ticket information. When an unauthorized user is detected by the second detection unit, the second video acquisition unit acquires the video data of the exit as video data showing a video in which the unauthorized user appears, from the second camera that has photographed the exit. The unauthorized user determination unit determines that a user who appears in both the video shown by the entrance video data acquired by the first video acquisition unit and the video shown by the exit video data acquired by the second video acquisition unit is an unauthorized user of the facility. Unauthorized user determination method.

10. A first detection unit that collects ticket information indicating a ticket for the facility from a mobile terminal possessed by a user present at an entrance of the facility, and detects an unauthorized user of the facility based on the ticket information; A first camera that photographs the entrance when an unauthorized user is detected by the first detection unit; A first video acquisition unit that acquires the entrance video data as video data showing a video in which an unauthorized user appears, from the first camera; A second detection unit that collects the ticket information from a mobile terminal possessed by a user present at an exit of the facility, and detects an unauthorized user of the facility based on the ticket information; A second camera that photographs the exit when an unauthorized user is detected by the second detection unit; A second video acquisition unit that acquires the exit video data as video data showing a video in which an unauthorized user appears, from the second camera; An unauthorized user determination unit that determines that a user who appears in both the video shown by the entrance video data acquired by the first video acquisition unit and the video shown by the exit video data acquired by the second video acquisition unit is an unauthorized user of the facility An unauthorized user determination system comprising the same.

Images