Information processing device, information processing method, and program

Abstract

To improve the accuracy of determining whether or not a person has passed through a particular passageway, such as a passageway at a gate.SOLUTION: An information processing device 20 includes: a detection unit 203 that detects a foreign object not included in a steady-state image for each of a plurality of zones set for a passageway in an image taken of the passage from above; and a determination unit 205 that determines whether or not a person corresponding to the foreign object has passed through the passageway based on changes over time in the zones where the foreign object is detected.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is a technology for managing the entry and exit of people passing through gates installed at stations, airports, etc. For example, Patent Document 1 describes a device that, when a person obtains permission to pass through a gate using a wireless card and enters the gate from the entrance, tracks whether the person has passed through the gate (whether the person has returned to the entrance of the gate) based on changes in the position of the wireless card. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 09-330440 Summary of the Invention [Problem to be solved by the invention]

[0004] In a gate system (e.g., a facial recognition gate system) in which a person is permitted to pass through the gate without using a wireless card that the person carries, there is room for consideration as to how to reliably determine whether or not a person has passed through the passage at the gate.

[0005] Non-limiting examples of the present disclosure contribute to providing an information processing device, an information processing method, and a program that can improve the accuracy of determining whether a person has passed through a specific passage such as a passage in a gate. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present disclosure includes a detection unit that detects foreign objects not included in a steady-state image for each of a plurality of zones set for the corridor in an image taken from above the corridor, and a determination unit that determines whether a person corresponding to the foreign object has passed through the corridor based on changes over time in the zone in which the foreign object is detected.

[0007] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0008] Non-limiting examples of the present disclosure can improve the accuracy of determining whether a person has passed through a particular passageway, such as a passageway at a gate.

[0009] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a face authentication ticket gate passage management system 1 according to an embodiment. [Figure 2] A perspective view showing an example of the configuration of a facial recognition ticket gate [Figure 3] A block diagram showing an example of the configuration of the terminal PC shown in Figure 1. [Figure 4] FIG. 2 is a diagram showing an example of an image of a passageway captured by the surveillance camera illustrated in FIG. 1; [Figure 5] FIG. 5 is a diagram showing an example in which a plurality of areas are set for the passage image shown in FIG. 4. [Figure 6] A diagram showing an example in which the area in which a person is detected changes sequentially over time. [Figure 7] FIG. 7 is a diagram showing an example of the relationship between the passage of time and the area in which a person is detected, as illustrated in FIG. 6 . [Figure 8] A diagram showing an example in which the order of areas in which people are detected is reversed over time. [Figure 9] FIG. 9 is a diagram showing an example of the relationship between the passage of time and the area in which a person is detected, as illustrated in FIG. 8 . [Figure 10] A diagram showing an example in which the area in which a person is detected does not change over time. [Figure 11] FIG. 11 is a diagram showing an example of the relationship between the passage of time and the area in which a person is detected, as illustrated in FIG. 10 . [Figure 12] A diagram showing an example in which the areas in which two people are detected change in sequence over time. [Figure 13] FIG. 13 is a diagram showing an example of the relationship between the passage of time and the area in which a person is detected, as illustrated in FIG. 12 . [Figure 14] A diagram showing an example in which the areas detected for two of three people change in sequence over time, and the order of the areas detected for the remaining person is reversed midway through. [Figure 15] FIG. 15 is a diagram showing an example of the relationship between the passage of time and the area in which a person is detected, as illustrated in FIG. 14 . [Figure 16] A flowchart showing an example of operations related to presetting of the terminal PC illustrated in FIGS. 1 and 3. [Figure 17] A flowchart showing an example of the operation of the terminal PC illustrated in FIGS. 1 and 3 during operation. [Figure 18] A flowchart showing an example of an operation related to the passage determination of the terminal PC illustrated in FIGS. 1 and 3. [Figure 19] FIG. 1 is a diagram illustrating a first variation of an embodiment. [Figure 20] FIG. 1 is a diagram illustrating a first variation of an embodiment. [Figure 21] FIG. 1 is a diagram illustrating a first variation of an embodiment. [Figure 22] FIG. 10 is a diagram illustrating a variation 2 of an embodiment. [Figure 23]FIG. 10 is a diagram illustrating a variation 2 of an embodiment. [Figure 24] FIG. 10 is a diagram illustrating a fourth variation of an embodiment. [Figure 25] FIG. 10 is a diagram illustrating a variation 5 of an embodiment. [Figure 26] FIG. 6 is a diagram illustrating a variation 6 of an embodiment. [Figure 27] FIG. 7 is a diagram illustrating a variation 7 of an embodiment. [Figure 28] FIG. 7 is a diagram illustrating a variation 7 of an embodiment. [Figure 29] FIG. 8 is a diagram illustrating variation 8 of an embodiment. [Figure 30] FIG. 9 is a diagram illustrating a variation 9 of an embodiment. [Figure 31] FIG. 9 is a diagram illustrating a variation 9 of an embodiment. [Figure 32] A diagram showing an example of a computer hardware configuration DETAILED DESCRIPTION OF THE INVENTION

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions are designated by the same reference numerals, and redundant description will be omitted.

[0012] [One embodiment] Transportation (e.g., train) ticket gates using facial recognition are being considered as the next generation of ticket gates. For example, a person is photographed using a facial recognition camera attached to the ticket gate. The photographed image information is sent to an authentication device (e.g., a server) in which facial image information for facial recognition is registered. The server performs authentication based on the received image information and the registered facial image information, and returns an authentication result (e.g., authentication success (OK) or authentication failure (NG)) to the ticket gate. The ticket gate then controls, for example, opening and closing the gate depending on the authentication result.

[0013] Unlike systems that use touch-type IC (Integrated Circuit) cards or magnetic tickets inserted into the ticket gate, such ticket gates that use facial recognition can have difficulty determining whether a person intends to pass through the gate. For example, if a person simply looks through the ticket gate, facial recognition can be established using image information captured by a camera, and the person may be determined to have entered the gate and be charged.

[0014] This situation can be avoided by providing an operation unit at the ticket gate that triggers the start of facial recognition, such as a "face recognition start button." However, this would require a person trying to pass through the ticket gate to stop and operate the operation unit, which could undermine the significance or convenience of using facial recognition at ticket gates where many people are expected to pass through in a short period of time.

[0015] Therefore, it is important to determine whether a person who has been successfully authenticated has passed through the ticket gate. For example, a method is being considered in which a camera is installed above the ticket gate, and the person is identified and tracked based on an image taken by the camera from above (for example, from above).

[0016] However, due to differences in hair color or whether or not a person is wearing accessories (e.g., a hat, helmet, etc.), it can be difficult to identify a person from image information taken from above, and the accuracy of judgment or tracking may be reduced.

[0017] Furthermore, depending on the environment around the ticket gate (for example, the color of the floor at the ticket gate or the external light), it may be difficult to distinguish between people. In addition to changes over time, it may also be difficult to distinguish between people due to the influence of lighting or shadows installed around the ticket gate.

[0018] Furthermore, since the ceiling height around the ticket gates may differ depending on the station (in other words, the shooting environment for the camera may differ), it may be necessary to set or learn the system according to the shooting environment (for example, to prepare a library that has been trained according to the shooting environment for each station). Setting or learning the system separately for different shooting environments takes time and is not realistic.

[0019] In one non-limiting example of the present disclosure, for example, a camera (hereinafter sometimes referred to as a surveillance camera) is installed above the ticket gate to monitor the passageway within the ticket gate, and the passageway image captured by the surveillance camera is divided (or segmented) into a plurality of areas (or zones). Then, for each divided area (hereinafter sometimes abbreviated as a "divided area"), it is detected whether or not a person has entered that area, and based on the time-series change in the detection result (in other words, the change over time), it is determined whether or not the person has passed through the ticket gate.

[0020] The detection of a person's entry into each divided area may be performed using, for example, differential detection or foreign object detection using deep learning, which regards a person as a foreign object. For example, a divided area into which a person has entered may be detected by comparing an aisle image captured by a surveillance camera in a state where no person is present inside the ticket gate (conveniently referred to as a "steady state") with an aisle image captured by the surveillance camera in a non-steady state (for example, while a facial recognition ticket gate is in operation). For example, if a part of the aisle image in the non-steady state that differs from the aisle image in the steady state is detected, it can be determined that a person has entered the divided area corresponding to that part.

[0021] Therefore, for example, if the divided areas where a person's entry is detected (hereinafter referred to as "person detection areas" or "entry detection areas") change sequentially over time, it can be determined that the person is moving through the ticket gate. Also, for example, if entry is detected in a divided area corresponding to the exit of the ticket gate, it can be determined that the person has passed through the ticket gate.

[0022] On the other hand, for example, if the foreign object detection areas do not change in sequence over time, it can be determined that the person has not passed through the ticket gate.

[0023] Ticket gates are usually wide enough for one person to pass through at a time (in other words, they are not wide enough for two or more people to pass side by side). For this reason, it is difficult to imagine a situation in which a person entering a ticket gate overtakes another person who entered before them, or is overtaken by another person who entered after them (in other words, a situation in which people switch places).

[0024] Therefore, even if there are two or more people inside the ticket gate, the passing status of the ticket gate can be determined for each of the multiple people by tracking the time series changes in the entrance detection for each divided area.

[0025] Furthermore, the detection of entry into each divided area does not need to be performed all the time. For example, the detection of entry into each divided area may be performed depending on the timing of face authentication. Furthermore, the steady-state passage image may be acquired, for example, when the passage management system 1 including the ticket gate 40 and the surveillance camera 12 is installed, during maintenance, or at any time (or timing) such as morning, afternoon, or night.

[0026] To reliably detect people, it is advisable to use images of the passageway in a steady state that do not contain people or foreign objects. Since there is a high probability that people or foreign objects will not be present in the passageway during installation and maintenance, it is highly likely that a good steady-state image will be obtained. It is also possible to allow a user to capture a steady-state passageway image. In this case, the user may visually check for people or foreign objects in the passageway. Furthermore, if the ticket gate 40 is installed outdoors, shadows or the like that fall on the passageway change depending on the time of day, and there is a risk that these shadows or the like will be erroneously detected as people. To prevent this problem from occurring, it is advisable to acquire steady-state passageway images at multiple time periods and use the passageway image from the same time period as the current time period for passage management.

[0027] For example, a steady-state passage image may be acquired at the timing when it is detected that no person is present in the passage of the ticket gate 40, based on the state of face authentication using the face authentication camera 11 installed at the ticket gate 40. Furthermore, if a photoelectric sensor is installed at the ticket gate 40, the photoelectric sensor may be used to identify the timing when it is detected that no person is present in the passage of the ticket gate 40, and a steady-state passage image may be acquired.

[0028] [System configuration example] FIG. 1 is a block diagram showing an example of the configuration of a face authentication ticket gate passage management system 1 according to an embodiment.

[0029] As illustrated in FIG. 1, the passage management system 1 includes, for example, a face authentication camera 11, a monitoring camera 12, a terminal PC (personal computer) 20, a server 30, and a ticket gate 40.

[0030] The facial authentication camera 11, for example, photographs an area including the face of a person passing through the ticket gate 40, and outputs the photographed image to the terminal PC 20. The facial authentication camera 11 may be provided on one or both of the opposing side walls 41A and 41B of the ticket gate 40, as shown in Fig. 2, for example. The space between the side walls 41A and 41B forms a passageway within the ticket gate 40.

[0031] For example, as shown in Fig. 2, a camera 11A may be installed at the top of one side wall 41A to capture an image of a person entering ticket gate 40 from the direction indicated by arrow 2A in Fig. 2. Also, a camera 11B may be installed at the top of the other side wall 41B to capture an image of a person entering ticket gate 40 from the opposite direction (for example, the direction indicated by arrow 2B in Fig. 2).

[0032] In other words, a person using ticket gate 40 can pass through ticket gate 40 in either the direction indicated by arrow 2A in Fig. 2 or the direction indicated by arrow 2B in Fig. 2 through facial recognition. Ticket gate 40 may be configured or set to allow a person to pass through in only one of the directions indicated by arrow 2A and arrow 2B. In this case, facial recognition camera 11 may be provided at ticket gate 40 for only one of the two directions.

[0033] The surveillance camera 12 is installed, for example, above the ticket gate 40, and captures an image (passage image) including the passage inside the ticket gate 40 from above the ticket gate 40. The passage image captured by the surveillance camera 12 is output to, for example, the terminal PC 20. The surveillance camera 12 may be any camera that can capture the entire passage inside the ticket gate 40 from above. For example, a PTZ camera or an omnidirectional camera may be used as the surveillance camera 12. Furthermore, the surveillance camera 12 does not necessarily have to be configured as an integral part of the ticket gate 40 or positioned directly above the passage. The placement of the surveillance camera 12 is not important as long as the entire passage inside the ticket gate 40 is included in the capture range. For example, a camera installed on the ceiling of the station near the ticket gate 40 may be used as the surveillance camera 12.

[0034] One non-limiting example of the installation form of the surveillance camera 12 is, for example, as shown in Fig. 2, an embodiment in which a pole-shaped support 42 extending above the ticket gate 40 is installed at the ticket gate 40, and the surveillance camera 12 is installed on top of this support 42. The shape of the support 42 is not limited to a pole shape and may be other shapes such as an arch shape. Furthermore, the support 42 does not need to be a support 42 dedicated to installing the surveillance camera 12; for example, existing equipment around the ticket gate 40 may be used for both purposes.

[0035] The terminal PC 20, for example, transmits information on a face image input from the face authentication camera 11 to the server 30 and requests face authentication from the server 30. In addition, the terminal PC 20 controls the opening and closing of a ticket barrier (gate) 40 according to the face authentication result from the server 30.

[0036] For example, if the facial authentication result indicates successful authentication (OK), the terminal PC20 controls the ticket gate 40 to an open state to allow the person to pass through. On the other hand, if the facial authentication result indicates unsuccessful authentication (NG), the terminal PC20 controls the ticket gate 40 to a closed state to restrict the person from passing through. Note that, in addition to controlling the opening and closing of the ticket gate 40, the terminal PC20 may also control the ticket gate 40 to output or display audio in accordance with the facial authentication result.

[0037] Furthermore, as described above, the terminal PC 20 detects whether a person has entered each divided area of ​​the passage image of the ticket gate 40 captured by the monitoring camera 12, and determines whether the person has passed through the ticket gate 40 based on time-series changes in the detection results. The result of this determination (hereinafter sometimes referred to as "passing determination") may be managed, for example, in association with the face authentication result of the person related to the passing determination. This management makes it possible to manage, for example, whether a person whose face has been successfully authenticated has reliably passed through the ticket gate 40.

[0038] For example, when the server 30 receives a request for face authentication from the terminal PC 20, the server 30 determines whether a face image matching the face image received from the terminal PC 20 is included in a database (DB) that stores face images of multiple people that have been registered in advance, and returns the determination result (in other words, the face authentication result) to the terminal PC 20.

[0039] The facial image may be pre-registered in the server 30 via a facial image registration terminal as shown in FIG. 1, for example, or may be registered directly in the server 30.

[0040] The ticket gate 40, for example, receives opening / closing control from the terminal PC 20 based on the face authentication result, thereby permitting or restricting the entry of people into and exit from the station premises.

[0041] [Configuration example of terminal PC20] Next, a configuration example of terminal PC 20 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing a configuration example of terminal PC 20. As illustrated in Fig. 3, terminal PC 20 may include, for example, an authentication processing unit 201, a setting unit 202, a detection unit 203, a storage unit 204, a determination unit 205, and a passage management unit 206.

[0042] The authentication processing unit 201, for example, detects (or extracts) an area corresponding to a person's face (hereinafter sometimes referred to as a "face area") in an image received from the face authentication camera 11 by image processing, and transmits information about the detected face area to the server 30 to request face authentication. Note that the information about the face area may be, for example, an image of the face area, or information indicating feature points extracted from the image of the face area.

[0043] Furthermore, for example, when the authentication processing unit 201 receives a face authentication result from the server 30, it controls the opening and closing of the ticket gate 40 based on the received face authentication result. Furthermore, the authentication processing unit 201 may output the face authentication result received from the server 30 to the passage management unit 206 in order to manage whether or not a person whose face has been successfully authenticated has passed through the ticket gate 40.

[0044] The setting unit 202 sets, for example, divided areas for dividing the passage of a ticket gate captured by the monitoring camera 12 into a plurality of passages.

[0045] For example, the detection unit 203 detects whether a person has entered each divided area of ​​the passage image of the ticket gate 40 received from the surveillance camera 12 by image processing. For example, the detection unit 203 may detect which divided area the person has entered by comparing the passage image captured by the surveillance camera 12 with a passage image in a steady state.

[0046] The detection operation by the detection unit 203 may be started in response to the start of authentication processing by the authentication processing unit 201. In this case, the operation time of the detection unit 203 can be reduced, which contributes to power saving of the terminal PC20.

[0047] The passage image in a steady state may be set in the detection unit 203 by the setting unit 202, for example. The divided areas in which the entry of a person is detected (person detection areas) may be stored in the storage unit 204 in chronological order, for example.

[0048] The determination unit 205 determines whether or not a person has passed through the ticket gate 40, for example, based on information about the person detection area stored in chronological order in the storage unit 204. For example, the determination unit 205 may determine that a person has passed through the ticket gate 40 when the person detection area changes over time in sequence from an area corresponding to the entrance of the ticket gate 40 to an area corresponding to the exit of the ticket gate 40.

[0049] On the other hand, for example, if the person detection areas do not change in sequence over time, the determination unit 205 may determine that the person has not passed through the ticket gate 40. For example, as will be described later, it may be determined that the person has not passed through the ticket gate 40 in cases where a person who entered through the entrance of the ticket gate 40 remains inside the ticket gate 40, or in cases where a person entered the ticket gate 40 through the entrance but turned back and exited through the entrance.

[0050] The passage management unit 206 manages (for example, stores) information about a person for whom the authentication processing result has been obtained in the authentication processing unit 201, in association with the result of the passage determination by the determination unit 205. This allows management of, for example, whether or not a person who has been successfully authenticated has passed through the ticket gate 40.

[0051] It should be noted that some or all of the functional units 201 to 206 of the terminal PC 20 may be provided in, for example, the server 30. Furthermore, the face authentication process by the server 30 may be performed in the terminal PC 20.

[0052] In other words, the processing related to the face authentication and passage determination (or passage management) of the person described above may be performed in a distributed manner by the terminal PC 20 and the server 30 operating in cooperation or collaboration, or may be performed in a centralized manner in the terminal PC 20 or the server 30. Furthermore, the terminal PC 20 and the server 30 may be integrated into one information processing device.

[0053] Furthermore, the persons who are the subject of facial recognition and passage determination are not limited to persons who move on their own, but may also include, for example, persons who move in a vehicle such as a wheelchair (whether manual or automatic (e.g., electric)).

[0054] [Example of operation] Next, an example of the operation of the passage management system 1 will be described with reference to FIGS.

[0055] Fig. 4 is a diagram showing an example of a passage image captured by the monitoring camera 12. Fig. 5 is a diagram showing an example in which a plurality of areas (or zones) A ​​to G are set for the passage image shown in Fig. 4. The terminal PC20 (for example, the detection unit 203) detects the entry of a person into each of the areas A to G.

[0056] 6, if a person moves from area G toward area A, the person's entry into each area is detected in the order of area G → F → E → D → C → B → A as time passes from time t1 to t7. In other words, the person detection area changes (or transitions) in the order of area G → F → E → D → C → B → A as time passes from time t1 to t7.

[0057] In this case, the relationship between the passage of time (t1 to t7) and the person detection area can be expressed with the horizontal axis representing time and the vertical axis representing the position of the divided area, as shown in Figure 7. As shown in Figure 7, when the person detection area changes linearly with the passage of time from t1 to t7, it can be seen that a person who has entered ticket gate 40 is moving in one direction within ticket gate 40 from time t1 to t7.

[0058] Here, for example, as shown in Fig. 7, if the passage determination area for determining that a person has passed through the ticket gate 40 is set to area B, the terminal PC20 (for example, the determination unit 205) determines that the person has passed through the ticket gate 40 normally at time t6. The passage determination area may be set by, for example, the setting unit 202. Depending on the determination of normal passage, for example, a charging process may be executed for the person.

[0059] The billing process may be performed, for example, in the terminal PC 20, or may be performed in a device (for example, the server 30) different from the terminal PC 20 in response to a request from the terminal PC 20. Note that the result of the passage determination by the terminal PC 20 (for example, the determination unit 205) may be used for recording or managing passage without involving billing process.

[0060] In contrast to this, for example, Figure 8 shows a case where a person who entered ticket gate 40 turns back halfway through ticket gate 40. Figure 8 shows an example in which a person who entered ticket gate 40 from area G at time t1 reverses their direction of travel between times t4 and t5 and returns to area G at time t8.

[0061] The relationship between the passage of time (t1 to t8) and the person detection area in this case is shown, for example, in Figure 9. As shown in Figure 9, if the person detection area changes nonlinearly (in a mountain shape) as time passes from t1 to t8, it can be seen that a person who entered ticket gate 40 turned back midway through ticket gate 40 (for example, area D) and left ticket gate 40 (i.e., did not pass through ticket gate 40).

[0062] 10 shows a case where a person who enters ticket gate 40 stops midway inside ticket gate 40. In FIG. 10, an example is shown in which a person who enters ticket gate 40 from area G at time t1 reaches area E over the course of time from time t2 to t3, but remains in area E from time t3 to t8.

[0063] The relationship between the passage of time (t1 to t8) and the person detection area in this case is shown, for example, in Figure 11. As shown in Figure 11, the time when the person detection area does not change (for example, t3 to t8) indicates that a person who has entered ticket gate 40 is staying somewhere within ticket gate 40 (for example, area E) (that is, has not passed through ticket gate 40).

[0064] [Multiple people passing through] Next, Fig. 12 to Fig. 15 show cases where multiple people enter the ticket gate 40. Fig. 12 shows a case where two people P1 and P2 pass through the ticket gate 40 in turn, and Fig. 13 is a diagram showing an example of the relationship between the passage of time (t1 to t8) and the person detection area in the case of Fig. 12.

[0065] 14 shows a case where three people P1 to P3 enter a ticket gate 40, and two people P1 and P2 pass through the ticket gate 40 normally, but person P3 turns back inside the ticket gate 40. FIG. 15 is a diagram showing an example of the relationship between the passage of time (t1 to t8) and the person detection area in the case of FIG. 14.

[0066] In the case of Figure 12, as illustrated in Figure 13, the person detection area changes in the order of areas G → F → E → D → C → B → A from time t1 to t7, and from time t2 to t8 after time t1, the person detection area changes in the order of areas G → F → E → D → C → B → A.

[0067] In this embodiment, the width of the passageway within the ticket gate 40 is sufficiently narrow, so it can be assumed that no switching of people occurs in the passageway, and therefore it can be determined that the linear change in the person detection area starting from time t1 corresponds to person P1, and the linear change in the person detection area starting from time t2 corresponds to person P2.

[0068] Furthermore, in the case of Figure 14, as illustrated in Figure 15, a linear change in the human detection area equivalent to the example of Figure 13 occurs from time t1 to t7 and from time t2 to t8, and a mountain-shaped change occurs as the human detection area changes in the order of area G → F → E → E → F from time t4 to t8.

[0069] Therefore, it can be determined that the two linear changes in the person detection area starting from time t1 and time t2 correspond to people P1 and P2 who passed through the ticket gate 40 normally, respectively, and the mountain-shaped change in the person detection area starting from time t4 corresponds to person P3 who turned back halfway through the ticket gate 40.

[0070] In this way, even when multiple people enter the ticket gate 40, the terminal PC20 can distinguish the passing state of each of the multiple people by tracking the time-series changes in the person detection area.

[0071] [Example of operation flow] Next, an example of the operation of the terminal PC20 described above will be described with reference to the flowcharts shown in FIGS.

[0072] [Preset] 16, the terminal PC20, for example, uses the setting unit 202 to set an area (monitoring area or photographing area) to be monitored (in other words, photographed) by the monitoring camera 12 (S101). For example, the setting unit 202 sets the monitoring area so that the entire area including the passage of the ticket gate 40 is included in the photographing range.

[0073] Furthermore, the setting unit 202 sets (or registers) in the detection unit 203, for example, an image captured by the monitoring camera 12 in a steady state (S102).

[0074] Furthermore, the setting unit 202 sets divided areas (for example, areas A to G) for the passage portion of the captured image of the monitoring area (S103).

[0075] The setting of each divided area (for example, the width in the direction along which people pass through the ticket gate 40) may be fixed or variable. If it is variable, the setting unit 202 may, for example, provide the detection unit 203 with a setting related to updating the divided areas (S104). If it is variable, the setting unit 202 may set the divided areas in response to an instruction from the user. In this case, various user operations are conceivable. For example, the user may manually specify each divided area, or the setting unit 202 may set the divided areas by dividing the passageway portion at equal intervals when the user specifies the number of divisions. The setting of the passageway portion may also be fixed or variable. If it is variable, the setting unit 202 may set the passageway portion in response to an instruction from the user. The setting unit 202 may also detect information that can identify the passageway portion from the captured image (for example, a marker or a specific background color) and automatically identify the passageway portion.

[0076] [Operation] FIG. 17 is a flowchart showing an example of the operation of the terminal PC20 when the currency management system 1 is in operation after the above-described presetting is completed.

[0077] 17, during operation of the passage management system 1, the terminal PC20, for example, in the authentication processing unit 201, detects a person's face area in an image received from the face authentication camera 11 by image processing (S201). Information on the detected face area is transmitted to the server 30, for example, as a face authentication request.

[0078] In addition, the terminal PC20, for example, in the detection unit 203, detects whether a person has entered each divided area by image processing (e.g., foreign object detection) based on the passage image of the ticket gate 40 received from the surveillance camera 12 and the passage image in a steady state (S202).

[0079] The detection unit 203 records (or registers) the detection results (e.g., person detection areas) in S202 in time series in the storage unit 204 (S203), as shown in, for example, Figures 7, 9, 11, 13, and 15. Note that the information or data recorded in the storage unit 204 in this manner may be referred to as "time series data" for convenience.

[0080] Thereafter, the terminal PC20 determines, for example, in the determination unit 205, based on the time-series data recorded in the storage unit 204, whether or not the person who entered the ticket gate 40 passed through the ticket gate 40 (S204). An example of this determination (passing determination) will be described later with reference to FIG.

[0081] After the passage determination, the terminal PC20, for example, checks in the authentication processing unit 201 whether the face area of ​​the person who entered the ticket gate 40 has been detected (S205). If the face area of ​​the person has been detected (S205; Yes), the authentication processing unit 201, for example, checks whether the face authentication has been successful (S206).

[0082] For example, when the authentication processing unit 201 receives information indicating successful face authentication from the server 30, the authentication processing unit 201 may determine that face authentication has been successful for the person whose face area was detected in S201 (S206; Yes).

[0083] In this case, the authentication processing unit 201 may determine, for example, that the person whose face area has been detected has successfully entered the ticket gate 40 (S207). In response to this successful determination, the authentication processing unit 201 may, for example, transmit a signal to the ticket gate 40 to control the gate to an open state, thereby allowing the person who has been successfully authenticated to pass through the ticket gate 40.

[0084] On the other hand, if the authentication processing unit 201 receives information indicating a failure of face authentication from the server 30, the authentication processing unit 201 may determine that face authentication has failed for the person whose face area was detected in S201 (S206; No).

[0085] In this case, the authentication processing unit 201 may determine, for example, that the person from whom the face area was detected has entered the ticket gate 40 illegally (S209). In response to this determination of illegality, the authentication processing unit 201, for example, transmits a signal to the ticket gate 40 to control the gate to a closed state, thereby restricting the passage of the person who failed authentication through the ticket gate 40.

[0086] If the face area of ​​a person has not been detected in S205 (S205; No), the authentication processing unit 201 may determine, for example, that a foreign object other than a person is present in the image captured by the face authentication camera 11, or that an unauthorized entry has occurred (S208). It is possible that the detection of the person's face area fails, resulting in an undetected face area, such as when a person enters the ticket gate 40 with their face covered. In this case, for example, foreign object detection may be performed using an image near the entrance to the ticket gate 40 as the target area, and the foreign object detection may be used as a trigger to switch the target area for foreign object detection to an image including the entire passage, thereby tracking the person whose face area has not been detected and making a determination such as in S208. Furthermore, foreign object detection may be performed at all times, for example, regardless of whether a face area has been detected (in other words, without using the detection of a face area as a trigger).

[0087] [Passage Judgment] Next, an example of the passage determination operation in S204 will be described with reference to FIG. As illustrated in FIG. 18, for example, in the judgment unit 205, if the time series data recorded in the memory unit 204 indicates that the person detection area changes sequentially until it reaches the passage judgment area (e.g., area B) (S2041), the terminal PC20 judges that the person has passed through the ticket gate 40 normally (passage completed) (S2042).

[0088] Furthermore, for example, when the time-series data recorded in the storage unit 204 indicates that the person detection area does not change (or update) in an intermediate area (e.g., area E) before reaching the passage determination area (e.g., area B) (S2043), the determination unit 205 determines that the person is staying inside the ticket gate 40 (stay determination) (S2044). If the stay determination continues for a certain period of time (threshold time), the determination unit 205 may, for example, send a signal to issue an alert to the ticket gate 40.

[0089] In addition, for example, if the time series data recorded in the memory unit 204 changes in reverse order in an intermediate area (e.g., area D) before the person detection area reaches the passage judgment area (e.g., area B) (S2045), the judgment unit 205 judges that the person has turned back in the opposite direction within the ticket gate 40 (reverse passage judgment) (S2046).

[0090] There may be cases where the time series data recorded in the storage unit 204 is for multiple people (S2047). For example, people may enter the ticket gate 40 from both directions during the same time period. In this case, the time series data for the forward direction and the time series data for the reverse direction of the change in the person detection area are recorded in the storage unit 204.

[0091] For example, when passage in one direction at the ticket gate 40 is set or assumed as passage in the forward direction, the determination unit 205 may, for example, use the time series data in that direction in preference to the time series data in the reverse direction to make the determination shown in S2041 to S2046 (S2048).

[0092] For example, if the direction from area G to area A is the forward direction, the determination unit 205 may perform passage determination by using time series data in which the person detection area changes from area G → F → E → D → C → B → A in preference to time series data in which the area changes in the order of area A → B → C → B → A.

[0093] As described above, according to the above-described embodiment, for example, it is determined whether a person has passed through the passage of the ticket gate 40 based on a change over time in the (divided) area where the person's entry has been detected in the passage image of the ticket gate 40. Therefore, for example, it is possible to determine whether a person whose face has been successfully authenticated has reliably passed through the ticket gate 40, thereby improving the accuracy of the determination.

[0094] In addition, by using foreign object detection (which may also be called "difference detection") based on passageway images and steady-state passageway images respectively captured by surveillance camera 12 to detect people in divided areas, it is possible to avoid a decrease in detection accuracy due to changes or diversity in the environment around ticket gate 40 (for example, the color of the floor at the ticket gate or external light).

[0095] Furthermore, even if the environment around each ticket gate 40 is different for each ticket gate 40, detecting differences in images captured by the surveillance camera 12 eliminates the need for special preparations such as strict adjustment of the imaging environment for each ticket gate 40 or preparing a learned library for person detection.

[0096] Furthermore, in the above-described embodiment, foreign object detection using background subtraction is used to detect objects believed to be people, and passage determination is performed based on time-series movement. This allows for detection with fewer omissions than simply detecting the person themselves. In particular, people wearing hats, which are difficult to detect with a library trained to detect "people," can be more reliably detected as "foreign objects." Therefore, the above-described embodiment makes it possible to achieve passage control for people dressed in a variety of outfits.

[0097] A similar system exists in which a large number of photoelectric sensors are provided at the ticket gate 40 to directly detect the passage of an object believed to be a person through the passage. However, this system cannot detect how far a person has passed unless a large number of photoelectric sensors are provided at the ticket gate 40. For this reason, the configuration of the above-described embodiment, which can achieve passage management if there is a surveillance camera 12 that can capture an image of the entire passage, can reduce the number of sensors required and is therefore advantageous in terms of cost.

[0098] Therefore, detecting differences in images captured by the surveillance camera 12 contributes to, for example, simplifying the passage management system 1.

[0099] [Variation 1] In detecting intrusions (e.g., detecting foreign objects) using images captured by a surveillance camera 12 (in other words, images captured from above a person's head), if the color of the passageway floor at the ticket gate 40 is uniform throughout, it is possible that the success rate of detection will decrease due to differences in, for example, the person's hair color, clothing, or headwear such as a hat or helmet.

[0100] Therefore, for example, the color of the passage floor at the ticket gate 40 may be partially different. For example, as shown in Fig. 19, the zones (for convenience, referred to as "floor zones") of the passage floor corresponding to each of the above-mentioned multiple divided areas (for example, areas A to G) may be color-coded according to the floor zone.

[0101] The color coding (or "color scheme") may be, for example, a color coding according to a pattern that changes gradually for each of floor zones A to G (in other words, a gradation), as illustrated in Fig. 19. Fig. 19 shows an example in which the color changes gradually from warm to cool from floor zone A to floor zone G. Conversely, color coding in which the color changes gradually from warm to cool from floor zone G to floor zone A may also be applied to floor zones A to G.

[0102] By color-coding the passage floor in this way, compared to when the passage floor is a single uniform color, for example, it is possible to create a situation in which, in detecting the entrance of a person in divided areas using an image taken from above the person, even if the person is likely to fail to be detected in a floor zone of a specific color, the person is likely to be detected successfully in floor zones of other colors. Therefore, it is possible to reduce the probability of failing to detect the entrance of a person passing through the passage of ticket gate 40 (in other words, to improve the detection success rate).

[0103] Alternatively, the color coding may be such that, for example, as shown in Figure 20, when the aisle floor is divided in the aisle width direction (divided into two in the example of Figure 20), the gradual color change differs for each divided area in the aisle width direction.

[0104] For example, Fig. 20 shows an example in which a gradual color change in the reverse order to the gradual color change applied to one of the areas divided in the aisle width direction (e.g., the left side) is applied to the other of the areas divided in the aisle width direction (e.g., the right side). For example, similar to Fig. 19, a color coding that gradually changes from warm to cool colors from floor zone A to floor zone G may be applied to the left side, and a color coding that gradually changes from warm to cool colors from floor zone G to floor zone A may be applied to the right side.

[0105] With this type of color coding, for example, in a case where a person passes through the ticket gate 40 in the forward direction and then another person passes through the ticket gate 40 in the reverse direction, it is possible to create a situation in which detection is likely to be successful for both the forward and reverse directions using floor zones of different colors. Therefore, it is possible to reduce the probability of failing to detect the entry of a person passing through the passage of the ticket gate 40 in either the forward or reverse direction (in other words, to improve the detection success rate).

[0106] The color coding of the passage floor does not have to be gradational (or gradient), and for example, different colors may be applied to different floor zones. Not applying color to some of the multiple floor zones may also be included in "color coding."

[0107] Also, instead of or in addition to color (for example, hue), for example, at least one of brightness, saturation, and contrast may be different for some or all of the multiple floor areas.

[0108] In this disclosure, the term "color coding" may be conveniently understood to include aspects in which at least one of hue, brightness, saturation, and contrast (which may be conveniently referred to as a "color parameter" relating to at least one of them) is different.

[0109] Furthermore, the color coding of the passage floor is not limited to physical methods such as applying paint or installing floor materials of different colors, but may be virtually realized using a technology such as projection mapping. For example, arrow-like marks indicating the direction of traffic may be set on the passage floor by projection mapping, and the marks may be displayed in different colors according to the divided areas.

[0110] Furthermore, instead of applying the markings to the passage floor, an effect equivalent to the color coding described above may be achieved by, for example, image processing. For example, in the image processing, a specific color may be emphasized (or amplified) or attenuated for each divided area, or at least one of brightness, saturation, and contrast may be emphasized (or amplified) or attenuated for each divided area.

[0111] FIG. 21 shows, as a non-limiting example, an example in which purple is emphasized in divided area A among divided areas A to G, and red is attenuated in divided area G.

[0112] The processing related to color classification using such image processing may be set in the detection unit 203 by the setting unit 202, for example.

[0113] By realizing processing equivalent to color coding using image processing (or projection mapping), there is no need to perform special processing on the passage floor surface, making it easy to introduce the passage management system 1.

[0114] Furthermore, by color-coding the floor of the divided areas or by color-coding using image processing, the success rate of detecting a person in any divided area can be improved even when there are differences in the person's hair color or whether or not they are wearing something. Therefore, the accuracy of determining whether a person has passed through or is being tracked at the ticket gate 40 can be improved compared to when color-coding is not applied.

[0115] [Variation 2] The color of the passage floor may be, for example, striped with two colors (e.g., blue and white) alternating, as shown in Fig. 22. In other words, the color of the passage floor may be changed periodically for each floor area. The color of the passage floor may be changed periodically using three or more colors. The periodic color combination applied to the passage floor may be any combination of different colors.

[0116] With this type of periodic color coding, floor areas of the same color appear periodically in image processing as people pass by. Therefore, even for people who are likely to be undetectable in floor areas of a specific color, the change in the person's position at the ticket gate 40 can be estimated by tracking the change in the entry detection area over time.

[0117] For example, in Figure 22, if a person is detected in the order of white floor zone F → white floor zone D, and then no person is detected in white floor zone B, it is possible to infer that "a person who is likely to fail to be detected on a blue floor is likely to be located in zone C of ticket gate 40."

[0118] The periodic color coding described above may also be achieved by periodically changing color parameters that are periodically emphasized or attenuated in image processing, as in Variation 1. As a non-limiting example, as shown in Fig. 23, a zone that emphasizes blue and a zone that emphasizes white may be set alternately.

[0119] Furthermore, when using three or more colors, the order in which the colors are used within each cycle can be arbitrary, as long as the set of colors used appears periodically. For example, it could be configured as follows: first cycle (blue, red, white), second cycle (red, white, blue), etc. Even with this configuration, it is expected that a person will be detected at least once within each cycle, making it possible to track changes in the intrusion detection area over time. In the above example, for example, a person who can only be detected in white will be detected at the end of the first cycle and in the center of the second cycle. As a result, it can be seen that the person has moved from the area in the first cycle to the area in the second cycle over time.

[0120] By using this type of periodic color coding through image processing, just like the color coding of the passageway floor, it is possible to estimate the change in the position of a person at the ticket gate 40 by tracking the changes in the entry detection area over time, even for people who are likely to fail to be detected in floor areas of a specific color.

[0121] [Variation 3] In the above-described embodiment, the passage image in the steady state may be acquired in a state where no person is present in the passage of the ticket gate 40. However, in a non-steady state, for example, when the monitoring camera 12 is fixed, a passage image taken in a state where a person is present (for example, passing through) in the passage of the ticket gate 40 (for example, after the start of operation) may be set as the passage image in the steady state.

[0122] For example, images (for example, frames) taken at multiple times by the surveillance camera 12 may be superimposed and the difference may be calculated, so that an image from which moving objects have been removed may be set as a steady-state corridor image.

[0123] [Variation 4] 2 illustrates an example in which the facial authentication camera 11 is provided on one or both of the side walls 41A and 41B of the ticket gate 40, but the facial authentication camera 11 may be installed in any position that allows it to capture an image of an area including the face of a person entering the ticket gate 40. For example, as shown in FIG. 24, the facial authentication camera 11 may be provided on a support part 42 on which the surveillance camera 12 is provided.

[0124] [Variation 5] The size of the divided areas (for example, the width in the direction in which people pass through the ticket gate 40), in other words, the number of divided areas in the direction in which the passage of the ticket gate 40 is passed, may be any size. For example, as shown in FIG. 25, the size of the divided areas may be any of the following: a size larger than the average size of a person (hereinafter sometimes referred to as "person size"), a size similar to the average size of a person, and a size smaller than the average size of a person.

[0125] If the size of the divided area is larger than the average size of a person, for example, the position of the person within the ticket gate 40 can be estimated based on the coordinates of the detected person on the image. If the size of the divided area is smaller than the average size of a person, for example, a person may be detected across multiple divided areas, and the multiple divided areas may be combined into a single entry detection area.

[0126] If the divided areas are set to a size corresponding to the average size of a person (hereinafter sometimes referred to as "person size"), it becomes easy to grasp the individual passage status without using additional processes such as tracking coordinates and combining divided areas. Therefore, for example, as illustrated in Figures 12 to 15, it becomes easy to grasp the individual passage status when multiple people enter ticket gate 40.

[0127] [Variation 6] Furthermore, even if an unintended object 261 such as dust appears in the passage image of the ticket gate 40, as shown in FIG. 26, the unintended object 261 can be easily excluded from the entry detection targets by setting the divided area to an appropriate size (for example, a size appropriate for the size of a person). For example, the determination unit 205 may determine whether or not the foreign object is a person based on the size of the foreign object in the passage image. That is, it is possible to determine that an object that is too small or too large compared to the divided area is an object 261 that is not a person. The determination unit 205 may not use the foreign object determined not to be a person in determining whether or not a person has passed through the passage.

[0128] Alternatively or additionally, the object 261 may be excluded depending on whether or not face recognition processing has been performed. For example, the object 261 detected when face recognition processing has not been performed may be determined not to be a person and may be excluded from the intrusion detection targets.

[0129] [Variation 7] The size of the divided areas (e.g., the width along the passageway direction) may not be uniform for all or some of the divided areas. For example, the size of some or all of the divided areas may be set to different sizes depending on the position of a person in the passageway image.

[0130] For example, as shown in Fig. 27, the size of the area occupied by a person in an image captured by surveillance camera 12 may vary depending on the position of the person relative to surveillance camera 12. For example, as shown in the upper part of Fig. 27, the closer a person is to surveillance camera 12, the smaller the size of the person in the image captured by surveillance camera 12 may become, as shown in the lower part of Fig. 27.

[0131] Therefore, for example, as shown in FIG. 28, the size of the divided areas may be set smaller (or narrower) as the area is closer to the installation position of the monitoring camera 12 (for example, the center of the area photographed by the monitoring camera 12).

[0132] For example, the size of the divided areas may be set non-uniformly so that when the position of a person relative to the surveillance camera 12 changes, "the person fits into one divided area" or "the person spans a certain number of divided areas."

[0133] 28 shows, as a non-limiting example, an example in which the size of the divided areas is set so that "the person straddles three divided areas" in the passage image at each different position of the person relative to the surveillance camera 12. Note that the number of divided areas straddled by the person in the passage image may be four or more.

[0134] With this setting, the number of entry detection areas per person can be kept constant in relation to the size of the person in the passage image, regardless of changes in the person's position relative to the surveillance camera 12. Therefore, compared to, for example, setting the sizes of multiple divided areas to a uniform constant, it is possible to obtain the same effects as variation 5 with simpler image processing.

[0135] Alternatively, the size of the divided areas may not be changed (or adjusted), but the number of divided areas that the person spans, or the proportion of the person's size in each divided area, may be changed (or adjusted), for example, depending on the position where the person appears in the corridor image.

[0136] For example, the closer to the center of the area captured by the surveillance camera 12, the smaller the number of divided areas each person spans, or the smaller the proportion of person size in each divided area of ​​the same size may be set.

[0137] [Variation 8] The lines dividing the divided areas in the direction of travel of people may be straight or curved. For example, as shown in Figure 29 (left side), a portion of each of multiple concentric circles (in other words, arcs) radiating from the center of the passage of ticket gate 40 may correspond to the lines dividing the divided areas in the direction of travel of people.

[0138] As illustrated by the thick frame 291 in Figure 29 (right side), by setting the areas bounded by individual arcs within the range where a person passes through the ticket gate 40 as the divided areas for person detection, areas bounded by arcs located outside the ticket gate 40 can be excluded from the areas for person detection.

[0139] [Variation 9] In the embodiments including the above-described variations 1 to 8, the passageway for which passage management is performed is not limited to the passageway of ticket gate 40 installed in a station. For example, the embodiments including the above-described variations 1 to 8 may be applied to the passageway of gates installed in various facilities or venues where entrance and exit management may be implemented, such as airports, ports, commercial facilities, public facilities, amusement facilities, theme parks, parks, and event venues.

[0140] Furthermore, the above-described embodiments including variations 1 to 8 may be applied to passage management at gates that do not have the function of controlling opening and closing.

[0141] [Overall supplement] In the above-described embodiments including variations 1 to 8, the passageway of ticket gate 40 (or gate) is formed by side walls 41A and 41B (FIG. 2) facing each other, but this is not limiting. For example, the passageway may be set as continuous or discontinuous areas that are distinguished from other areas by members such as ropes, tapes, or poles that guide people through the passageway. Furthermore, instead of or in addition to such physical members, a "passageway" that guides people may be set as distinct from other areas by using virtual means such as projection mapping.

[0142] The length of the passageway to be judged as passing may be any length. For example, the length of the passageway may be longer than the length generally expected for the ticket barrier 40 or gate. For example, the face authentication camera 11 may be installed in a location where the area including the face of a person entering the passageway falls within the photographing range, and the above-mentioned passageway judgment area may be set in the area where the person exits from the passageway.

[0143] Furthermore, the passageway to be judged for passage is not limited to being straight, but may have a curved portion or a difference in elevation (for example, a step such as a staircase) as shown in Fig. 30. The passageway may be a stationary passageway, or may be a moving passageway such as a moving walkway or escalator. Fig. 31 shows a spiral escalator having a spiral moving passageway as a non-limiting example of a passageway with a curved portion and a difference in elevation.

[0144] Regardless of the length of the passage, or whether or not there are curved sections or elevation differences in part or all of the passage, by including the entire passage within the shooting range from above of the surveillance camera 12, it is possible to reliably determine whether or not a person has passed through the passage without, for example, installing multiple physical sensors such as motion sensors along the passage's direction. In this case, the divided areas may be set according to the shape of the passage as seen by the surveillance camera 12. For example, for curved sections, divided areas are set in a shape that matches the curvature, and for sections with elevation differences, wider divided areas are set according to the law of perspective the closer to the surveillance camera 12 the section is.

[0145] The number of surveillance cameras 12 is not limited to one. For example, the entire passage may be photographed by multiple surveillance cameras 12. Separate areas may be set for each partial passage image photographed by the multiple surveillance cameras 12, and the passing status of people may be determined and managed.

[0146] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the disclosure.

[0147] Furthermore, the specific examples of the present disclosure are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above.

[0148] Furthermore, the notation "... section" in the above-described embodiments may be interchangeably replaced with other notations such as "... circuitry," "... device," "... unit," or "... module."

[0149] Furthermore, the present disclosure can be realized by software, hardware, or software in cooperation with hardware. For example, the functions of the above-described system can be realized by a computer program.

[0150] Fig. 32 is a diagram showing the hardware configuration of a computer (or information processing device) that uses a program to realize the functions of each device that makes up the above-mentioned currency management system 1. In Fig. 32, a computer 1100 includes an input device 1101 such as a keyboard, mouse, or touchpad, an output device 1102 such as a display or speaker, a CPU (Central Processing Unit) 1103, a GPU (Graphics Processing Unit) 1104, a ROM (Read Only Memory) 1105, a RAM (Random Access Memory) 1106, a storage device 1107 such as a hard disk device or SSD (Solid State Drive), a reading device 1108 that reads information from a recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or USB (Universal Serial Bus) memory, and a transmitting / receiving device 1109 that communicates via a network, and each unit is connected by a bus 1110.

[0151] The reading device 1108 reads a program for realizing the functions of the terminal PC20 from a recording medium on which the program is recorded, and stores the program in the storage device 1107. Alternatively, the transmitting / receiving device 1109 communicates with a server device (which may be the server 30 or a server device different from the server 30) connected to the network, and stores the program for realizing the functions of each of the above devices downloaded from the server device in the storage device 1107.

[0152] The CPU 1103 copies a program stored in the storage device 1107 to the RAM 1106, and sequentially reads and executes the instructions contained in the program from the RAM 1106, thereby realizing the functions of the terminal PC20 according to the above-described embodiment.

[0153] Each functional block used in the description of the above-mentioned embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above-mentioned embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0154] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0155] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility.

[0156] The present disclosure may be implemented in any type of apparatus, device, or system with communications capabilities (collectively referred to as a communications device). A communications device may include a radio transceiver and processing / control circuitry. The radio transceiver may include a receiver and / or a transmitter. The radio transceiver (transmitter and receiver) may include a radio frequency (RF) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of communications devices include telephones (e.g., mobile phones, smartphones), tablets, PCs (e.g., laptops, desktops, notebooks), cameras (e.g., digital still / video cameras), digital players (e.g., digital audio / video players), wearable devices (e.g., wearable cameras, smartwatches, tracking devices), game consoles, digital book readers, telehealth / telemedicine devices, communications-enabled vehicles or mobile transportation (e.g., automobiles, airplanes, ships), and combinations of the above devices.

[0157] Communications equipment is not limited to portable or mobile equipment, but also includes non-portable or fixed equipment, devices, and systems of any kind, such as smart home devices (such as appliances, lighting equipment, smart meters or metering devices, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0158] Furthermore, in recent years, in the field of IoT (Internet of Things) technology, CPS (Cyber ​​Physical Systems) has been attracting attention as a new concept that creates new added value by linking information between physical space and cyberspace. This CPS concept can also be adopted in the above-mentioned embodiments.

[0159] That is, as a basic configuration of a CPS, for example, an edge server located in physical space and a cloud server located in cyberspace can be connected via a network, and processing can be distributed and performed by processors installed on both servers. Here, it is preferable that each piece of processing data generated on the edge server or cloud server is generated on a standardized platform, and the use of such a standardized platform can improve the efficiency of building a system that includes a variety of sensor groups and IoT application software.

[0160] Communications include data communications via cellular systems, wireless LAN systems, communications satellite systems, etc., as well as data communications via combinations of these.

[0161] A communications apparatus also includes devices such as controllers and sensors connected or coupled to a communications device that performs the communications functions described in this disclosure, such as controllers and sensors that generate control and data signals used by the communications device to perform the communications functions of the communications apparatus.

[0162] The communication apparatus also includes infrastructure facilities, such as base stations, access points, and any other apparatus, device, or system that communicates with or controls the various apparatuses listed above, but are not limited to these. [Industrial Applicability]

[0163] An embodiment of the present disclosure is suitable for managing passage through a gate such as a ticket gate. [Explanation of symbols]

[0164] 1. Currency Management System 11 Facial recognition camera 12. Surveillance Cameras 20 Terminal PC 30 servers 40 Ticket Gate (Gate) 41A,41B side wall 42 Support part 201 Authentication processing section 202 Settings 203 Detection unit 204 Storage section 205 Judgment section 206 Passage Management Department 1100 Computer 1101 Input Device 1102 Output device 1103 CPU 1104 GPU(Graphics Processing Unit) 1105 ROM (Read Only Memory) 1106 RAM (Random Access Memory) 1107 Storage device 1108 Reading device 1109 Transmitting and receiving equipment 1110 Bus

Claims

1. a detection unit that detects foreign objects not included in a steady-state image for each of a plurality of zones set for the passage in an image captured from above the passage; a determination unit that determines whether a person corresponding to the foreign object has passed through the passageway based on a change over time in the zone in which the foreign object is detected; An information processing device comprising:

2. the determination unit determines that the person has passed from the entrance to the exit of the passage when the change over time indicates that the zone in which the foreign object was detected has changed in sequence from the zone corresponding to the entrance of the passage to the zone corresponding to the exit of the passage among the plurality of zones. The information processing device according to claim 1 .

3. the determination unit determines that the person has turned back in the passage when the change over time indicates that the order of the zones in which the foreign object was detected has been reversed. The information processing device according to claim 1 .

4. the determination unit determines whether the foreign object is the person based on the size of the foreign object, and does not use the foreign object determined to be not the person in determining whether the person has passed through the passage. The information processing device according to claim 1 .

5. the detection unit starts the detection in response to the start of authentication processing using a face image of the person. The information processing device according to claim 1 .

6. The floor of the passage is color-coded according to the zone. The information processing device according to claim 1 .

7. The color coding is a pattern that changes gradually for each of the zones. The information processing device according to claim 6 .

8. The color coding is a pattern in which a specific color appears periodically for each of the zones. The information processing device according to claim 6 .

9. the detection unit varies at least one of hue, brightness, saturation, and contrast for each of the zones in detecting the foreign matter; The information processing device according to claim 1 .

10. a width of one of the zones along the passageway direction that allows only one person to pass through; When foreign objects corresponding to a plurality of people are detected in the passage, the determination unit determines whether the people have passed through the passage, assuming that the order of the people along the passage's traveling direction will not change until the people pass through the passage. The information processing device according to claim 1 .

11. the determination unit sets a width of a part or all of the plurality of zones along a passage direction of the passage to a different width depending on a position of the person in the image. The information processing device according to claim 1 .

12. the determination unit sets a width of the zone closer to the center of the passage to be narrower as the position of the person is closer to the center of the image; The information processing device according to claim 11.

13. The passage is a passage at a gate where facial authentication of the person is performed. The information processing device according to claim 1 .

14. By one or more information processing devices, In an image of the passage photographed from above, foreign objects not included in a steady-state image are detected for each of a plurality of zones set for the passage; determining whether or not a person corresponding to the foreign object has passed through the passageway based on a change over time in the zone in which the foreign object is detected; Information processing methods.

15. One or more information processing devices, A process of detecting foreign objects not included in a steady-state image for each of a plurality of zones set for the passage in an image taken from above the passage; a process of determining whether or not a person corresponding to the foreign object has passed through the passageway based on a change over time in the zone in which the foreign object is detected; Execute program.

Images