Position / people flow management device, position / people flow management method, and program

Abstract

A position and people flow management device according to an embodiment of the present invention comprises a determination unit, an acquisition unit, and an association unit. The determination unit acquires a determination result of entry of a user into a target area determined on the basis of: an external transmission / reception state of an external transmission / reception wave between a user terminal and an external transceiver outside the target area; and an internal transmission / reception state of an internal transmission / reception wave between the user terminal and an internal transceiver. The acquisition unit acquires an image including a person in a specific area through which the person passes when entering the target area. The association unit associates person-in-image identification information assigned to the person included in the image with user identification information relating to the user. The association unit associates the person-in-image identification information with the user identification information on the basis of the image at a time when the user determined to have made entry into the target area has passed through the specific area.

Description

Location pedestrian flow management apparatus, location pedestrian flow management method, and program

[0001] Embodiments of the present invention relate to a location pedestrian flow management apparatus, a location pedestrian flow management method, and a program. The present application claims priority based on Japanese Patent Application No. 2025-029315 filed on February 26, 2025, the content of which is incorporated herein by reference.

[0002] In commercial facilities and the like, there are attempts to identify customers, adjust the content of services and advertisements provided to customers, and provide services and advertisements suitable for customers. Identification of customers is performed, for example, by collecting face data relating to feature amounts of customers' faces in advance, and using face authentication technology that compares images of customers captured at the time of entry with the pre-collected face data. However, face data is personal information, and the need for privacy protection is increasing, so it is difficult to collect and store face data. Therefore, a technology for accurately identifying customers without using face authentication is desired.

[0003] Japanese Unexamined Patent Application Publication No. 2013-109051 Japanese Unexamined Patent Application Publication No. 2015-055924 Japanese Unexamined Patent Application Publication No. 2006-113711

[0004] The problem to be solved by the present invention is to provide a location pedestrian flow management apparatus, a location pedestrian flow management method, and a program that can accurately identify customers.

[0005] The location-based pedestrian flow management device of this embodiment includes a determination unit, an acquisition unit, and a linking unit. The determination unit acquires a determination result of whether the user has entered the target area, which is determined based on the external transmission and reception state of the external transmission and reception waves between the user terminal held by the user and an external transceiver outside the target area that transmits and receives external transmission and reception waves, and the internal transmission and reception state of the internal transmission and reception waves between the user terminal and an internal transceiver inside the target area that transmits and receives internal transmission and reception waves. The acquisition unit acquires an image taken by an imaging device, which includes a person in a specific area that the user passes through when entering the target area. The linking unit links the image person identification information assigned to the person in the image with user identification information relating to the user. The linking unit links the image person identification information with the user identification information based on the image at the time the user, whose entry into the target area has been determined, passed through the specific area.

[0006] A diagram showing an example of the location-based pedestrian flow management system 1 of the first embodiment. A diagram schematically showing an example of a situation in which user P enters commercial facility M. A diagram schematically showing an example of a situation in which multiple people enter commercial facility M at the same time. A diagram showing an example of the configuration of user terminal 20. A diagram showing an example of the contents of user attribute information 25. A diagram showing an example of the configuration of location-based pedestrian flow management device 100. A diagram showing an example of the contents of image person linking information 151. A flowchart showing an example of processing of user terminal 20. A flowchart showing an example of processing of location-based pedestrian flow management device 100. A diagram showing an example of the location-based pedestrian flow management system 2 of the second embodiment. A diagram schematically showing an example of a situation in which multiple people board an elevator at the same time. A diagram schematically showing an example of a situation in which one of the multiple people disembarks from an elevator. A flowchart showing an example of processing of user terminal 20. A flowchart showing an example of processing of location-based pedestrian flow management device 100. A diagram showing an example of the configuration of location-based pedestrian flow management device 300 of the third embodiment. A flowchart showing an example of the processing of the location-based pedestrian flow management device 300. A diagram showing an example of the contents of user-specific attribute information 152. A diagram showing an example of the contents of store attribute information 154. A diagram showing an example of the contents of product attribute information 156. A diagram showing an example of the contents of product placement information 158. A flowchart showing an example of the processing of the location-based pedestrian flow management system of the fourth embodiment.

[0007] The following describes the location-based pedestrian flow management device, location-based pedestrian flow management method, and program of the embodiment with reference to the drawings.

[0008] (First Embodiment) The overall configuration of the location-based pedestrian flow management system 1 of the embodiment will be described. Figure 1 is a diagram showing an example of the location-based pedestrian flow management system 1 of the first embodiment. The location-based pedestrian flow management system 1 includes, for example, an in-store camera 11, an in-store beacon 12, a user terminal 20, and a location-based pedestrian flow management device 100. The in-store camera 11, the in-store beacon 12, and the location-based pedestrian flow management device 100 are installed, for example, in a commercial facility M which is a building. The user terminal 20 is, for example, held by a user P who is a customer of the commercial facility M. The location-based pedestrian flow management device 100 may be, for example, a cloud server located outside the commercial facility M.

[0009] The user terminal 20 is equipped with, for example, a GNSS (Global Navigation Satellite System) or GPS (Global Positioning System) application, and receives GPS signals transmitted by GPS satellites to detect its own position. The in-store camera 11, in-store beacon 12, user terminal 20, and location-based pedestrian flow management device 100 are each connected to each other via a network NW in a manner that enables communication between them. The network NW can be an intranet, a local area network (LAN), or a wireless LAN.

[0010] Figure 2 schematically illustrates an example of a situation in which user P enters commercial facility M. User P enters commercial facility M through entrance ME, for example, carrying user terminal 20, or placing it in their pocket or bag. User P may enter commercial facility M alone, or multiple users may enter the commercial facility at almost the same time.

[0011] Figure 3 schematically illustrates an example of a situation where multiple people enter commercial facility M simultaneously. As shown in the left diagram of Figure 3, the entrance ME is equipped with, for example, an automatic door MD. The automatic door MD is in a closed state, and its outer surface MDO is visible from outside commercial facility M.

[0012] Standing in front of the automatic door MD are the first user P1, the second user P2, and the third user P3, each possessing a user terminal 20. Subsequently, as shown in the middle diagram of Figure 3, the automatic door MD opens and the entrance ME is opened, allowing the first user P1, the second user P2, and the third user P3 to enter. Inside the entrance ME, an in-store camera 11 and an in-store beacon 12 are installed.

[0013] Next, as shown in the right-hand diagram of Figure 3, when the first user P1, the second user P2, and the third user P3 enter the commercial facility M, the automatic door MD closes. The right-hand diagram of Figure 3 shows the automatic door MD viewed from inside the commercial facility M, and the inner surface MDI of the automatic door MD is visible. The first user P1, the second user P2, and the third user P3 who enter the commercial facility M are imaged by the in-store camera 11, and each person's user terminal 20 receives the in-store beacon signal transmitted by the in-store beacon 12. Users entering the commercial facility M usually move forward as they enter. Since the in-store camera 11 is installed inside the commercial facility M, it is easy for the in-store camera 11 to capture the faces of the first user P1, the second user P2, and the third user P3 from the front.

[0014] Returning to Figure 2, the in-store camera 11 is installed, for example, on the ceiling of the commercial facility M. The in-store camera 11 may also be installed in places other than the ceiling, for example, on the walls, display shelves, cash registers, etc. of the commercial facility M. The in-store camera 11 is installed in multiple locations within the commercial facility M, and each in-store camera 11 is capable of capturing images of any location within the commercial facility M.

[0015] The in-store camera 11 is installed, for example, near the entrance ME, and captures images of users passing through the entrance ME. The in-store camera 11 transmits the captured images to the location-based pedestrian flow management device 100. When the in-store camera 11 transmits the captured images to the location-based pedestrian flow management device 100, it may also transmit information about the time the images were captured.

[0016] The in-store camera 11 is, for example, a digital camera that captures digital images. The in-store camera 11 may also be an analog camera other than a digital camera. If the in-store camera 11 is an analog camera, it may be equipped with an A / D (Analog to Digital) converter to digitize the analog images.

[0017] The in-store beacon 12 is installed, for example, on the ceiling of a commercial facility M, attached to a lighting fixture, for example. The in-store beacon 12 may be installed adjacent to an in-store camera 11, for example. The in-store beacon 12 may be installed in a location other than the ceiling, or in a location unrelated to the location of the lighting fixture and the in-store camera 11. The in-store beacon 12 may be installed, for example, on the wall, display shelves, cash register, etc. of the commercial facility M.

[0018] The in-store beacon 12 transmits and receives electrical signals (hereinafter referred to as beacon signals) using information communication utilizing telecommunications technology. Examples of information communication utilizing telecommunications technology include UWB (Ultra Wide Band), Bluetooth®, Wi-Fi, and NFC (Near Field Communication).

[0019] Figure 4 shows an example of the configuration of a user terminal 20. The user terminal 20 is, for example, a smartphone. The user terminal 20 includes, for example, a communication unit 21, a touch panel 22, a beacon receiver 23, a memory 24, and a processing unit 30. The memory 24 stores, for example, user attribute information 25, a GPS application 26, and a location and human flow management application 27. The processing unit 30 includes, for example, a GPS processing unit 31, a location and human flow management processing unit 32, and a Wi-Fi processing unit 33.

[0020] User attribute information 25 is information relating to the attributes of the owner of the user terminal 20, and is stored in memory 24 in advance through user P's input operations, etc. Figure 5 is a diagram showing an example of the contents of user attribute information 25. User attribute information 25 includes, for example, user ID and information such as age, gender, height, build, posture, appearance preferences, VIP status, and foreign language speaker status. User attribute information may also include other information, such as purchase history, visit history, occupation, hobbies, their history, family structure, place of origin, place of residence, and other information related to matters that the user may be interested in.

[0021] The user ID is a unique ID assigned to each user. The information regarding age, gender, height, body type, posture, and appearance preferences refers to user P's age, gender, height, body type, posture, and appearance preferences, respectively. Body type refers to information such as thin, average build, slightly overweight, or overweight, while posture refers to information such as whether the user takes long strides, short strides, or leans forward when walking.

[0022] Appearance preferences refer to, for example, the user's preferred clothing style, such as whether they like wearing heavy clothes, light clothes, street style, hats, or glasses. VIP information refers to whether user P is a VIP customer at commercial facility M. If user P is a VIP customer, a VIP flag is added to user attribute information 25. The conditions for being a VIP customer may be determined, for example, by the operator of commercial facility M.

[0023] The conditions for being designated a VIP customer may include, for example, that the annual payment amount for the entire business operated by the commercial facility M or the business operator is a predetermined amount, for example, 3 million yen or more, or that the number of times the customer has received services or purchased goods related to the business is a predetermined number or more, for example, 50 times or more. Foreign language speaker information is information on whether or not user P communicates in a foreign language, and if the user is a foreign language speaker, a foreign language flag is added to user attribute information 25. Foreign language information may also be information that identifies the type of foreign language, for example, English, Chinese, etc.

[0024] The GPS application 26 is an application program that activates the GPS processing unit 31 included in the control unit 140. The GPS processing unit 31 becomes functional when the GPS application 26 is started. The location and human flow management application 27 is an application program that collects information to be provided to the location and human flow management device 100 and generates information to be provided to the location and human flow management device 100. The location and human flow management application 27 is started, for example, by the wake-up function, and the location and human flow management processing unit 32 becomes functional when the location and human flow management application 27 is started.

[0025] The communication unit 21 of the user terminal 20 is, for example, an interface circuit for performing communication via a network NW with an external device such as a location and human flow management device 100. The communication interface 110 is implemented by, for example, a network card, network adapter, NIC (Network Interface Card), etc.

[0026] The touch panel 22 functions, for example, as an input interface for the user to input information and as an output interface to display information corresponding to the information input by the user and the processing results of the processing unit 30. The touch panel 22 displays, for example, a GUI (Graphical User Interface) for the user to perform various input operations. The touch panel 22 outputs information corresponding to the user's input operations to the processing unit 30.

[0027] The beacon receiver 23 receives beacon signals transmitted by beacon transmitters such as in-store beacons 12 and external beacons installed outside the commercial facility M. When the beacon receiver 23 receives a beacon signal such as an in-store beacon signal, it outputs the beacon reception signal, such as an in-store beacon reception signal, to the processing unit 30.

[0028] In the processing unit 30, the GPS processing unit 31 detects its own position based on the GPS signal received by an antenna (not shown) provided on the user terminal 20 when the GPS application stored in the memory 24 is launched. While the location flow management application 27 is running, the GPS processing unit 31 detects the signal strength of the GPS signal received by the antenna and records it in the memory 24.

[0029] The location and pedestrian flow management processing unit 32 executes processing according to instructions from the location and pedestrian flow management application 27. For example, when the location and pedestrian flow management application 27 is not running, the location and pedestrian flow management processing unit 32 receives an in-store beacon signal and activates the wake-up function to activate the location and pedestrian flow management application 27. The location and pedestrian flow management processing unit 32 records the time when the beacon receiver 23 received the in-store beacon signal that activated the wake-up function as the reception time in the memory 24.

[0030] The location flow management processing unit 32 determines whether the signal strength of the GPS signal recorded by the GPS processing unit 31 falls below a threshold while the location flow management application 27 is running. The signal strength threshold is an arbitrary signal strength, and may be, for example, a signal strength at which the GPS processing unit 31 of the user terminal 20 can no longer detect the location. Depending on the determination result that the signal strength is below the threshold, the location flow management processing unit 32 causes the communication unit 21 to transmit entry information and user attribute information 25 to the location flow management device 100.

[0031] The entry information is the result of determining whether or not user P has entered commercial facility M, and indicates that user P has entered commercial facility M. The entry information includes information on the time of reception of the in-store beacon signal. The user terminal 20 functions as part of the location flow management system 1, for example, when the location flow management application 27 is launched.

[0032] The Wi-Fi processing unit 33 acquires and processes various information corresponding to the Wi-Fi signal received by an antenna (not shown) provided on the user terminal 20. While the location flow management application 27 is running, the Wi-Fi processing unit 33 detects the signal strength of the Wi-Fi signal received by the antenna and records it in the memory 24.

[0033] Figure 6 shows an example of the configuration of the location-based human flow management device 100. The location-based human flow management device 100 includes, for example, a communication interface 110, an input interface 120, an output interface 130, a control unit 140, and a storage unit 150. The control unit 140 includes, for example, a determination unit 141, an acquisition unit 142, an image processing unit 143, a linking unit 144, and a tracking unit 145.

[0034] The communication interface 110 is an interface circuit for performing communication via a network NW with external devices such as the in-store camera 11 and the user terminal 20. The communication interface 110 is implemented by, for example, a network card, network adapter, NIC, etc. The communication interface 110 receives user attribute information 25 transmitted by the user terminal 20.

[0035] The input interface 120 accepts various instructions and input operations for various information from the operator. The input interface 120 can be implemented using GUI images, or by means of, for example, a mouse, keyboard, touch panel, trackball, switch, button, joystick, camera, infrared sensor, microphone, etc.

[0036] In this specification, the input interface 120 is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit is also included as an example of the input interface 120.

[0037] The output interface 130 is an interface for displaying various information or outputting it as sound. The output interface 130 may be, for example, a display or a speaker. The output interface 130 may also be a touch panel integrated with the input interface 120.

[0038] The determination unit 141 in the control unit 140 acquires the determination result of whether the user has entered the target area, which is determined based on the external transmission and reception status of the external transmission and reception waves between the user terminal and the external transceiver and the internal transmission and reception status of the internal transmission and reception waves between the user terminal and the internal transceiver. The determination unit 141 acquires, for example, the determination result of whether user P has entered the commercial facility M, which is determined based on the reception by the user terminal 20 of the in-store beacon signal transmitted by the in-store beacon 12 and the radio wave intensity attenuation state of the GPS signal transmitted by the GPS satellite AS received by the user terminal 20, as entry information. The determination unit 141 acquires the reception time included in the entry information and further acquires user attribute information 25 transmitted together with the entry information.

[0039] The determination of whether user P has entered commercial facility M is performed by the user terminal 20, and the determination unit 141 acquires the determination result transmitted by the user terminal 20. The determination unit 141 may also acquire the determination result by determining whether user P has entered commercial facility M on behalf of the user terminal 20. In this case, the user terminal 20 may transmit information necessary for determining whether user P has entered commercial facility M, such as a signal indicating GPS strength, to the location and pedestrian flow management device 100.

[0040] The acquisition unit 142 acquires images including people in a specific area that pass through when entering a target area captured by the imaging device. For example, the acquisition unit 142 acquires users P of entrance ME that pass through when entering a commercial facility M, captured by the in-store camera 11. The images may also be included in the video. Images transmitted by the in-store camera 11 may also be included in the video.

[0041] The image processing unit 143 processes the image acquired by the acquisition unit 142 to extract people contained in the image (hereinafter referred to as "image people"). The image processing unit 143 identifies the visible characteristics of the extracted people and assigns an image person ID to each extracted person. If multiple people are extracted by image processing, the image processing unit 143 identifies the visible characteristics of each person and assigns an image person ID to each person.

[0042] Visible features include, for example, the color, brightness distribution, and shape of clothing. Visible features may also include other features, such as the color, brightness distribution, and shape of parts of the human body such as hair (hairstyle), skin, hats, shoes, canes, belts, ties, and other equipment, as well as the color, brightness distribution, and shape of accessories such as necklaces, rings, pendants, and hair accessories.

[0043] The processing by the image processing unit 143 may be performed, for example, on an appropriate image processing target person at an appropriate image processing imaging position. The image processing target person may be identified when user P enters the commercial facility M or after the image person ID is linked to the user ID, and may be, for example, a VIP customer or a person requiring surveillance, such as a habitual shoplifter. The image processing imaging position is the user's position when image processing is performed, and is, for example, a position calculated based on location information calculated based on the transmission and reception of in-store beacon signals between the in-store beacon 12 and the user terminal 20. The image processing imaging position may be set for each image processing target person, or it may be a specific location regardless of the image processing target person, for example, near a specific store. By specifying the image processing target person and the image processing imaging position, the burden of image processing can be reduced.

[0044] The linking unit 144 links image person identification information assigned to an image person with user identification information related to a user. The linking unit 144 links image person identification information, for example, an image person ID, with user identification information, for example, a user ID, based on an image captured at the time when a user determined to have entered a target area passes through a specific area. The user identification information may be information that identifies the user P themself, or may be information assigned to an article related to the user P. The information that identifies the user P themself may be, for example, the user P's email address, UDID (Unique Device Identifier), MAC (Media Access Control) address, IDFV (Identifier for Vendor), device ID, serial number, ANDROID_ID (ANDROID: registered trademark), ADID (Advertising ID), or the like. The information assigned to an article related to the user P may be, for example, an ID (smartphone ID, wearable terminal ID) of a user terminal (e.g., a smartphone or a wearable terminal) assigned to the user terminal 20 possessed by the user P. When there are a plurality of persons in the specific area, the linking unit 144 matches attributes of the user with attributes of the persons, and links the image person identification information and the user identification information based on a result of the matching.

[0045] The linking unit 144 links, for example, the image person ID assigned by the image processing unit 143 with the user ID included in the user attribute information 25 acquired by the acquisition unit 142. The linking unit 144 identifies an image captured by the in-store camera 11 at the time when the user P passes through the entrance ME, for example, based on the reception time of an in-store beacon signal included in entry information transmitted together with the user attribute information 25 and imaging time information added to an image by the in-store camera 11. The linking unit 144 links the image person ID of the person extracted from the identified image with the user ID included in the user attribute information 25.

[0046] When a plurality of persons are included in the identified image, the associating unit 144 collates the visual features of the image person identified by the image processing unit 143 with the user attribute information 25 acquired by the determining unit 141. The associating unit 144 associates the user ID of the user whose visual features match most closely the visual features of the image person included in the user attribute information 25 with the image person ID. The associating unit 144 generates image-person association information 151 based on a result of associating the image person ID with the user ID, and records the image-person association information 151 in the storage unit 150.

[0047] FIG. 7 is a diagram showing an example of the content of the image-person association information 151. The image-person association information 151 includes, for example, a user ID associated with an image person ID, further each piece of information included in the user attribute information 25 assigned with the user ID, and information on visual features based on image processing of the person assigned with the image person ID. For each piece of information included in the user attribute information 25 and the information on visual features based on image processing, when the same item has different contents, the visual features based on image processing are recorded. By recording the visual features based on image processing, it is possible to improve the accuracy when tracking the user P who entered the commercial facility M on the same day.

[0048] The tracking unit 145 tracks users in a target area. For example, the tracking unit 145 identifies visual features included in captured images transmitted by in-store cameras 11 installed at various locations in the commercial facility M. The tracking unit 145 compares the identified visual features with the image-person association information 151 recorded in the storage unit 150, and tracks the position of the user P assigned with the user ID in the commercial facility M.

[0049] Next, processing in the location pedestrian flow management system 1 will be described. The location pedestrian flow management system 1 manages the position of the user P in the commercial facility M and the pedestrian flow accompanying the movement of the user P starting from when the user P enters the commercial facility M. Here, first, processing of the user terminal 20 will be described, and subsequently, processing of the location pedestrian flow management apparatus 100 will be described.

[0050] Figure 8 is a flowchart showing an example of processing by the user terminal 20. When user P, who possesses the user terminal 20, enters the commercial facility M, the user terminal 20 receives an in-store beacon signal. The user terminal 20 then first determines whether the beacon receiving unit 23 has received the in-store beacon signal transmitted by the in-store beacon 12 (step S101).

[0051] If the beacon receiver 23 determines that it has not received an in-store beacon signal, the user terminal 20 repeats the process in step S101. If the beacon receiver 23 determines that it has received an in-store beacon signal, the location flow management application 27 wakes itself up using its wake-up function (step S103) and activates the location flow management processing unit 32. Subsequently, the location flow management processing unit 32 checks the reception time when the beacon receiver 23 received the in-store beacon signal and records it in the memory 24 (step S105).

[0052] Even after user P, who possesses user terminal 20, enters commercial facility M, user terminal 20 continues to receive GPS signals transmitted by GPS satellite AS until user P moves a certain distance away from the entrance ME. However, as user P moves further into commercial facility M, the signal strength of the received GPS signal decreases. After the location flow management application 27 is launched, the GPS processing unit 31 detects the signal strength of the GPS signal received by the antenna and records it in memory 24 (step S107).

[0053] Next, the location flow management processing unit 32 determines whether the signal strength of the GPS signal recorded in the memory 24 is below a threshold (step S109). If the signal strength is not below the threshold (i.e., it is above the threshold), it is considered that it cannot be confirmed that user P has entered the commercial facility M.

[0054] If the location flow management processing unit 32 determines that the signal strength of the GPS signal recorded in the memory 24 is not below a threshold (i.e., it is above a threshold), it determines whether a certain amount of time has elapsed since the beacon receiver 23 received the in-store beacon signal (step S111). If it determines that a certain amount of time has not elapsed, the location flow management processing unit 32 determines that it is unclear whether user P has entered the commercial facility M, so it returns to step S107 and repeats the processing from steps S107 to S109. If it determines that a certain amount of time has elapsed, it is considered that user P entered the commercial facility M but left after a short time, so the location flow management processing unit 32 returns to step S101.

[0055] In step S109, if the radio wave strength is determined to be below a threshold, the location flow management processing unit 32 determines that user P has entered the commercial facility M and generates entry information (step S113). Subsequently, the location flow management processing unit 32 transmits the generated entry information and user attribute information 25 stored in memory 24 to the location flow management device 100 using the communication unit 21 (step S115). In this way, the user terminal 20 completes the process shown in Figure 8.

[0056] Next, the processing of the location-based pedestrian flow management device 100 will be explained. Figure 9 is a flowchart showing an example of the processing of the location-based pedestrian flow management device 100. First, the location-based pedestrian flow management device 100 determines in the determination unit 141 whether or not the communication interface 110 has received the entry information transmitted by the user terminal 20 (step S201).

[0057] If the communication interface 110 determines that it has not received entry information, the determination unit 141 determines that user P has not entered the commercial facility M and repeats the process in step S201. If the determination unit 141 determines that the communication interface 110 has received entry information, the acquisition unit 142 uses the reception time included in the entry information as the entry time when user P entered the commercial facility M, and acquires images (video) of the area around the entrance ME captured by the in-store camera 11 before and after the entry time (step S203).

[0058] Next, the image processing unit 143 extracts the individuals who entered the commercial facility M at the time of entry by processing the images acquired by the acquisition unit 142 (step S205). The individuals extracted by the image processing may be one or multiple. The image processing unit 143 identifies the visible characteristics of all individuals through image processing and assigns an image person ID to each individual as an image person (step S207).

[0059] Next, the linking unit 144 determines whether or not the person entering the system, as extracted by the image processing unit 143, is a group of people (step S209). If the image processing unit 143 determines that there are multiple people entering the system, the linking unit 144 performs a matching process (step S211). As part of the matching process, the linking unit 144 matches the visible features identified by the image processing unit 143 with the visible features included in the user attribute information 25 acquired by the determination unit 141.

[0060] Next, the linking unit 144 links the user ID of the user corresponding to the user attribute information 25 with the image person ID assigned to the person extracted by the image processing unit 143, based on the result of matching the visible features (step S213). The linking unit 144 links the user ID of the user with the image person ID of the person who entered whose matched visible features match or are the most similar. After linking the image person ID and the user ID, the linking unit 144 generates image person linking information 151 and records it in the storage unit 150 (step S215).

[0061] Next, the tracking unit 145 performs tracking processing (step S217). For example, the tracking unit 145 tracks the person in the image based on visible features captured by multiple in-store cameras 11 installed in the commercial facility M, and tracks the user P from the user ID associated with the person in the image ID assigned to the person in the image. In this way, the location-based pedestrian flow management device 100 completes the processing shown in Figure 9.

[0062] In the first embodiment, the location-based pedestrian flow management system 1 links the user ID of a user P entering a commercial facility M with the image ID of a person tracked by an in-store camera 11. Therefore, user P can be tracked within the commercial facility M using images captured by the in-store camera 11 without using facial data. The location-based pedestrian flow management system 1 links the user ID with the image person ID using the image of user P entering the commercial facility. Therefore, user P, who is a customer, can be identified with high accuracy.

[0063] (Second Embodiment) Next, a location-based pedestrian flow management system of the second embodiment will be described. In the following description, elements that have the same function or use as the first embodiment may be denoted by the same reference numerals and their descriptions may be omitted. In the location-based pedestrian flow management system of the second embodiment, the user ID of a user getting on or off the elevator is linked to the person ID of a person extracted from an image captured by an in-flight camera installed inside the elevator.

[0064] Figure 10 shows an example of a location-based pedestrian flow management system 2 according to the second embodiment. The location-based pedestrian flow management system 2 according to the second embodiment includes, for example, an in-store camera 11, an in-flight camera 13, an in-flight beacon 14, a Wi-Fi router 15, a user terminal 20, and a location-based pedestrian flow management device 100. The in-store camera 11, Wi-Fi router 15, and location-based pedestrian flow management device 100 are installed, for example, in a commercial facility M. An elevator EV is installed in the commercial facility M, and the in-flight camera 13 and in-flight beacon 14 are installed inside the elevator EV.

[0065] Figure 11 is a schematic diagram illustrating an example of a situation where multiple people are riding an elevator simultaneously. The left and middle diagrams of Figure 11 show, for example, the entrance to an elevator on the first floor of a commercial facility M. As shown in the left diagram of Figure 11, the entrance to the elevator EV is equipped with, for example, an automatic door ED. The automatic door ED is in a closed state, and its outer surface EDO is visible from the outside of the elevator EV.

[0066] Standing in front of the automatic door ED are the first user P1, the second user P2, the third user P3, and the fourth user P4, each possessing a user terminal 20. Subsequently, when the elevator EV arrives on the first floor, as shown in the middle diagram of Figure 11, the automatic door ED opens and the elevator entrance is opened, allowing the first user P1, the second user P2, the third user P3, and the fourth user to board. Inside the elevator EV, an in-flight camera 13 and an in-flight beacon 14 are installed.

[0067] Next, as shown in the right-hand diagram of Figure 11, when the first user P1, second user P2, third user P3, and fourth user P4 board the elevator EV, the automatic door ED closes. The right-hand diagram of Figure 11 shows the automatic door ED viewed from inside the elevator EV, and the inner surface EDI of the automatic door ED is visible.

[0068] The first user P1, second user P2, third user P3, and fourth user P4 who board the elevator EV are photographed by the in-flight camera 13, and each person's user terminal 20 receives the in-flight beacon signal transmitted by the in-flight beacon 14. Users boarding the elevator EV usually enter the elevator EV while moving forward. Since the in-flight camera 13 is installed inside the elevator EV, it is easy for the in-flight camera 13 to photograph the faces of the first user P1, second user P2, third user P3, and fourth user P4 from the front. The image of user P captured by the in-flight camera 13 is an image of user P taken from the front, but it may also be an image of user P taken from the back.

[0069] Users who board an elevator (EV) disembark at their respective floors. Figure 12 schematically shows an example of one person disembarking from an elevator among several people. The middle and right diagrams of Figure 12 show, for example, the elevator entrance on the 10th floor of a commercial facility M.

[0070] As shown in the left diagram of Figure 12, the elevator EV is in motion (up and down). The automatic door ED is closed, and the first user P1, second user P2, third user P3, and fourth user P4 are inside the elevator EV, standing in front of the automatic door ED. From the outside of the elevator EV, the outer surface EDO of the automatic door ED is visible.

[0071] Next, when the elevator EV arrives on the 10th floor, as shown in the middle diagram of Figure 12, the automatic door ED opens and the elevator entrance is opened, allowing the first user P1, second user P2, third user P3, and fourth user P4 to disembark. Here, the first user P1 disembarks from the elevator EV, while the second user P2, third user P3, and fourth user remain inside. Subsequently, as shown in the right diagram of Figure 12, the automatic door ED closes.

[0072] Returning to Figure 10, the in-flight camera 13 is a digital camera similar to the in-store camera 11. The in-flight camera 13 is installed, for example, on the ceiling of the elevator. The in-flight camera 13 may also be installed on the walls of the elevator, etc., instead of the ceiling. The in-flight camera 13 is installed in one location inside the elevator, but it may be installed in multiple locations. The in-flight camera 13 is installed, for example, near the boarding gate of the elevator and captures images of users passing through the boarding gate. The in-flight camera 13 transmits the captured images to the location and flow management device 100.

[0073] The in-flight beacon 14, like the in-store beacon 12, transmits and receives beacon signals (in-flight beacon signals) using information communication utilizing telecommunication technologies such as UWB, Bluetooth®, Wi-Fi, and NFC. The in-flight beacon 14 is installed, for example, on the ceiling of an elevator, attached to a lighting fixture.

[0074] The in-flight beacon 14 is installed, for example, adjacent to the in-flight camera 13. The in-flight beacon 14 may be installed in a location other than the ceiling of the elevator EV, or in a location unrelated to the lighting fixtures and the location of the in-flight camera 13. The in-flight beacon 14 may be installed, for example, on the wall of the elevator EV.

[0075] The Wi-Fi router 15 is installed on each floor of a building where an elevator is installed. For example, it is installed on each floor of commercial facility M where an elevator EV is installed. The Wi-Fi router 15 sends and receives Wi-Fi signals to and from the user terminal 20, for example, to connect the user terminal 20 to the internet wirelessly. The Wi-Fi signal transmitted by the Wi-Fi router 15 includes installation floor information, which is information about the floor on which the Wi-Fi router 15 is installed.

[0076] In the second embodiment, the location flow management processing unit 32 (Figure 5) in the user terminal 20 determines whether the Wi-Fi signal strength recorded by the Wi-Fi processing unit 33 falls below a threshold while the location flow management application 27 is running. The Wi-Fi signal strength threshold is an arbitrary signal strength, for example, it may be a signal strength at which the Wi-Fi processing unit 33 of the user terminal 20 loses internet connectivity.

[0077] The location-based passenger flow management processing unit 32 causes the communication unit 21 to transmit boarding information and user attribute information 25 to the location-based passenger flow management device 100, according to the determination result that the Wi-Fi signal strength is below a threshold. The boarding information is information indicating that user P has boarded the elevator EV. The boarding information includes information on the time of reception of the in-flight beacon signal.

[0078] The location and passenger flow management processing unit 32 further determines whether the strength of the in-flight beacon signal falls below a threshold while the location and passenger flow management application 27 is running. The threshold for radio wave strength can be any radio wave strength. Depending on the determination result that the radio wave strength of the in-flight beacon signal is below the threshold, the location and passenger flow management processing unit 32 causes the communication unit 21 to transmit disembarkation information to the location and passenger flow management device 100.

[0079] The location and passenger flow management processing unit 32 may, instead of determining whether the in-flight beacon signal falls below a threshold, determine whether the Wi-Fi signal strength at the user's disembarking floor exceeds a threshold. In this case, the location and passenger flow management processing unit 32 may also determine whether a certain amount of time has elapsed since the Wi-Fi signal strength exceeded the threshold.

[0080] Next, we will describe the processing of the location-based passenger flow management system 2 in the second embodiment. We will describe the processing of the user terminal 20 and the location-based passenger flow management device 100 for both when user P boards the elevator EV and when disembarks from the elevator EV. First, we will describe the processing of the user terminal 20 when user P boards the elevator EV.

[0081] Figure 13 is a flowchart illustrating an example of the processing performed by the user terminal 20. Figure 13 shows the processing performed by the user terminal 20 when user P boards an elevator. Prior to the start of the flowchart shown in Figure 13, user P stays on one of the floors of the commercial facility M, and the user terminal 20 carried by user P receives a Wi-Fi signal transmitted by the Wi-Fi router 15 installed on that floor. The user terminal 20 stores the installation floor information contained in the received Wi-Fi signal in its memory.

[0082] When user P, who possesses user terminal 20, boards elevator EV, user terminal 20 receives an in-flight beacon signal. First, user terminal 20 determines whether the beacon receiving unit 23 has received the in-flight beacon signal transmitted by the in-flight beacon 14 (step S301).

[0083] If the beacon receiver 23 determines that it has not received an in-flight beacon signal, the user terminal 20 repeats the process in step S301. If the beacon receiver 23 determines that it has received an in-flight beacon signal, the location flow management application 27 wakes itself up using its wake-up function (step S303) and activates the location flow management processing unit 32.

[0084] If the location flow management app 27 is already running when entering commercial facility M, the process in step S303 may be omitted. Next, the location flow management processing unit 32 checks the reception time when the beacon receiver 23 receives the in-flight beacon signal and records it in the memory 24 (step S305).

[0085] After user P, who possesses user terminal 20, boards the elevator EV, user terminal 20 receives Wi-Fi signals transmitted by the Wi-Fi router 15 installed on the floor where user P boarded the elevator EV, for example, until the elevator EV's automatic door ED closes. Eventually, when the elevator EV's automatic door ED closes, the signal strength of the Wi-Fi signal received by user terminal 20 is attenuated due to the radio wave shielding effect corresponding to the closing of the automatic door ED. After the location flow management application 27 is launched, the Wi-Fi processing unit 33 detects the signal strength of the Wi-Fi signal received by the antenna and records it in memory 24 (step S307).

[0086] Next, the location and flow management processing unit 32 determines whether the signal strength of the Wi-Fi signal recorded in the memory 24 is below a threshold (step S309). If the signal strength is not below the threshold (i.e., above the threshold), it is considered that it cannot be confirmed that user P has boarded the elevator EV.

[0087] If the location-based passenger flow management processing unit 32 determines that the radio wave intensity recorded in the memory 24 is not below a threshold (i.e., it is above a threshold), it determines whether a certain amount of time has elapsed since the beacon receiver 23 received the in-flight beacon signal (step S311). If it determines that a certain amount of time has not elapsed, the location-based passenger flow management processing unit 32 determines that it is unclear whether user P boarded the elevator EV, so it returns to step S307 and repeats the processing from steps S307 to S309. If it determines that a certain amount of time has elapsed, it is considered that user P boarded the elevator EV but exited after a short time, so the location-based passenger flow management processing unit 32 returns to step S301.

[0088] In step S309, if the radio wave intensity is determined to be below a threshold, the location and passenger flow management processing unit 32 determines that user P has boarded the elevator EV and generates boarding information (step S313). The boarding information is information indicating that user P has boarded the elevator EV, as a result of the determination of whether or not user P has boarded the elevator EV. The boarding information includes information on the reception time when the beacon receiver 23 received the in-cabin beacon signal. The location and passenger flow management processing unit 32, having generated the boarding information, records the installation floor information recorded in the memory 24 as boarding floor information indicating the floor on which user P boarded the elevator EV.

[0089] The location-based passenger flow management processing unit 32 may include the recorded boarding floor information in the boarding information. Subsequently, the location-based passenger flow management processing unit 32 transmits the generated boarding information and the user attribute information 25 stored in the memory 24 to the location-based passenger flow management device 100 using the communication unit 21 (step S315). In this way, the user terminal 20 completes the process shown in Figure 13.

[0090] Next, we will explain the processing of the position-based passenger flow management device 100 when user P boards the elevator EV. Figure 14 is a flowchart showing an example of the processing of the position-based passenger flow management device 100. Figure 14 shows the processing of the position-based passenger flow management device 100 when user P boards the elevator.

[0091] First, the location-based passenger flow management device 100 determines in the determination unit 141 whether or not the communication interface 110 has received the passenger boarding information transmitted by the user terminal 20 (step S401). If the determination unit 141 determines that the communication interface 110 has not received the passenger boarding information, it determines that user P has not boarded the elevator EV and repeats the process in step S201.

[0092] If the determination unit 141 determines that the communication interface 110 has received boarding information, the acquisition unit 142 takes the reception time included in the boarding information as the boarding time when user P boarded the elevator EV, and acquires images (video) of the area around the elevator EV's boarding gate captured by the in-flight camera 13 before and after the boarding time (step S403).

[0093] Next, the image processing unit 143 extracts the passengers who boarded the elevator at the time of boarding by processing the image acquired by the acquisition unit 142 (step S405). The passengers extracted by image processing may be one or multiple. The image processing unit 143 identifies the visible characteristics of all passengers by image processing and assigns an image person ID to each passenger as an image person (step S407).

[0094] Next, the linking unit 144 determines whether or not the person on board the aircraft extracted by the image processing unit 143 is multiple people (step S409). If it is determined that the person on board the aircraft extracted by the image processing unit 143 is multiple people, the linking unit 144 performs a matching process (step S411). As part of the matching process, the linking unit 144 matches the visible features identified by the image processing unit 143 with the visible features included in the user attribute information 25 acquired by the determination unit 141.

[0095] Next, the linking unit 144 provisionally links the user ID of the user corresponding to the user attribute information 25 with the image person ID assigned to the person extracted by the image processing unit 143, based on the result of matching the visible features (step S413). For example, the linking unit 144 provisionally links the image person ID of the person riding the vehicle whose matched visible features match or are the most similar with the user ID of the user. After provisionally linking the image person ID and the user ID, the linking unit 144 generates image person linking information 151 and records it in the storage unit 150 (step S415). In this way, the location-based passenger flow management device 100 completes the process shown in Figure 14.

[0096] Next, we will explain the processing of the user terminal 20 when user P disembarks from the elevator EV. Figure 15 is a flowchart showing an example of the processing of the user terminal 20. Figure 15 shows the processing of the user terminal 20 when user P disembarks from the elevator. The user terminal 20, which is carried by a user who boards the elevator EV, has the boarding floor information recorded on it. Therefore, the user terminal 20 first determines whether or not the boarding floor information is recorded in the memory 24 (step S501).

[0097] If the user terminal 20 determines that the boarding floor information is not recorded in memory 24, it repeats the process in step S501. If the user terminal 20 determines that the boarding floor information is recorded in memory 24, it determines whether the location flow management processing unit 32 has received a Wi-Fi signal transmitted by a Wi-Fi router 15 installed on any floor of the commercial facility M (step S503).

[0098] If the Wi-Fi signal is not received, the location and pedestrian flow management application 27 repeats the process in step S503. If the Wi-Fi signal is received, the location and pedestrian flow management application 27 stores the installation floor information included in the Wi-Fi signal in memory 24 separately from the boarding floor information, and also checks the reception time of the Wi-Fi signal and stores it in memory 24 (step S505).

[0099] After a user P, who has boarded the elevator EV and is carrying a user terminal 20, disembarks from the elevator EV, the user terminal 20 receives an in-flight beacon signal transmitted by an in-flight beacon 14 installed in the elevator EV until the elevator EV's automatic door ED closes. When the elevator EV's automatic door ED closes, the radio wave intensity of the in-flight beacon signal received by the user terminal 20 is attenuated due to the radio wave shielding effect of the automatic door ED. After the location flow management processing unit 32 starts the location flow management application 27, it detects the radio wave intensity of the in-flight beacon signal received by the beacon receiver 23 and records it in the memory 24 (step S507).

[0100] Next, the location and passenger flow management processing unit 32 determines whether the radio wave strength of the in-flight beacon signal recorded in the memory 24 is below a threshold (step S509). If the radio wave strength of the in-flight beacon signal is not below the threshold (i.e., it is above the threshold), it is considered that it cannot be confirmed that user P has disembarked into the elevator EV.

[0101] If the location flow management processing unit 32 determines that the radio wave strength of the in-flight beacon signal recorded in memory 24 is not below a threshold (i.e., above a threshold), it determines whether a certain amount of time has elapsed since the Wi-Fi signal was received (step S511). If it determines that a certain amount of time has not elapsed, the location flow management processing unit 32 determines that user P may still be in the elevator, so it returns to step S507 and repeats the processing from steps S507 to S509. If it determines that a certain amount of time has elapsed, it is considered that user P is still in the elevator, so the location flow management processing unit 32 returns to step S501.

[0102] If the system determines in step S509 that the radio wave strength of the in-flight beacon signal is below a threshold, the location and passenger flow management processing unit 32 determines that user P has disembarked from the elevator EV and generates disembarkation information (step S513). Disembarkation information is information indicating that user P has disembarked from the elevator EV, as a result of determining whether or not user P has disembarked from the elevator EV.

[0103] The disembarkation information includes information about the time the location-based passenger flow management processing unit 32 received the Wi-Fi signal transmitted by the Wi-Fi router 15 installed on the disembarkation floor of user P. The location-based passenger flow management processing unit 32 that generated the disembarkation information records the installation floor information stored in the memory 24 as disembarkation floor information in the memory 24.

[0104] The location-based passenger flow management processing unit 32 may include the recorded disembarkation floor information in the disembarkation information. Subsequently, the location-based passenger flow management processing unit 32 transmits the generated disembarkation information and the user attribute information 25 stored in the memory 24 to the location-based passenger flow management device 100 using the communication unit 21 (step S517). In this way, the user terminal 20 completes the process shown in Figure 15.

[0105] Next, we will explain the processing of the position-based passenger flow management device 100 when user P disembarks from the elevator EV. Figure 16 is a flowchart showing an example of the processing of the position-based passenger flow management device 100. Figure 16 shows the processing of the position-based passenger flow management device 100 when user P disembarks from the elevator.

[0106] First, the location-based passenger flow management device 100 determines in the determination unit 141 whether or not the communication interface 110 has received disembarking information transmitted by the user terminal 20 (step S601). If the communication interface 110 determines that it has not received disembarking information, the determination unit 141 determines that the user P who boarded the elevator EV has not disembarked from the elevator EV, and repeats the process in step S601.

[0107] If the determination unit 141 determines that the communication interface 110 has received disembarkation information, the acquisition unit 142 takes the reception time included in the disembarkation information as the disembarkation time when user P disembarked from the elevator EV, and acquires images (video) of the area around the elevator EV boarding gate captured by the in-flight camera 13 before and after the disembarkation time (step S603). The acquisition unit 142 may also acquire images (video) of the area around the elevator EV boarding gate captured by the in-store camera 11 near the elevator EV before and after the disembarkation time.

[0108] Next, the image processing unit 143 extracts the people who boarded the elevator EV at the boarding time by processing the image acquired by the acquisition unit 142 (step S605). The extracted disembarking people may be one or multiple. The image processing unit 143 identifies the visible characteristics of all disembarking people through image processing and assigns an image person ID to the boarding people as image people (step S607).

[0109] Next, the linking unit 144 determines whether or not there are multiple people who have disembarked, as extracted by the image processing unit 143 (step S609). If it is determined that there are multiple people who have disembarked, the linking unit 144 performs a matching process (step S611). As part of the matching process, the linking unit 144 matches the visible features identified by the image processing unit 143 with the visible features included in the user attribute information 25 acquired by the determination unit 141.

[0110] Next, the linking unit 144 confirms and links the user ID of the user corresponding to the user attribute information 25 with the image person ID assigned to the person extracted by the image processing unit 143, based on the result of matching the visible features (step S613). For example, the linking unit 144 confirms and links the image person ID of the person riding the aircraft whose matched visible features match or are the most similar with the user ID of the user.

[0111] Next, the linking unit 144 reads the image person linking information 151 that was temporarily linked during the ride and recorded in the storage unit 150 in step S415 of Figure 14. The linking unit 144 compares the relationship between the temporarily linked user ID and image person ID that was read with the relationship between the user ID and image person ID that was confirmed to be linked in step S613, and determines whether the two match or not (step S615).

[0112] The linking unit 144 determines, for example, that the relationship between a user ID and an image person ID that has been confirmed to be linked during the ride matches the relationship between a user ID and an image person ID that has been provisionally linked, if the visible characteristics of the relationship match completely or to a considerable extent. If it determines that the two match, the linking unit 144 confirms the relationship between the user ID and the image person ID (step S617).

[0113] If it is determined that the two do not match, the linking unit 144 provisionally sets the relationship between the user ID and the image person ID (step S619). The image person linking information 151 may be stored separately in the storage unit 150 depending on whether the two match or not. This distinction allows the various information obtained using the image person linking information 151 to be appropriately utilized according to the purpose and application of the analysis. For example, the various information obtained based on the image person linking information 151 when the two match can be used for more accurate marketing analysis, while the various information obtained based on the image person linking information 151 when the two do not match can be used in cases where it is more important to increase the sample size than the individual accuracy of the analysis, such as when conducting comparative analysis between customer segments and wanting to increase the sample size for each customer segment.

[0114] Next, the tracking unit 145 performs tracking processing in the same manner as described in step S217 (Figure 9) of the first embodiment (step S419). In this way, the location-based pedestrian flow management device 100 completes the processing shown in Figure 16.

[0115] The location-based pedestrian flow management system 2 of the second embodiment provides the same effects and advantages as the location-based pedestrian flow management system 1 of the first embodiment. Furthermore, the location-based pedestrian flow management system 2 of the second embodiment uses images acquired when user P boards and alights from the elevator EV to verify the accuracy of the linking between user ID and image person ID. This further improves the accuracy of the linking between user ID and image person ID.

[0116] In the second embodiment described above, the linking unit 144 temporarily links the image person ID and the user ID when user P boards the elevator EV, and then determines the relationship between the user ID and the image person ID after user P disembarks from the elevator EV. However, the linking unit 144 may perform the linking in other ways. For example, the linking unit 144 may link the user ID and the image person ID based on information when user P boards the elevator EV, or it may link the user ID and the image person ID based on information when user P disembarks from the elevator EV.

[0117] (Third Embodiment) Next, a third embodiment will be described. Figure 17 is a diagram showing an example of the configuration of the positional human flow management device 300 of the third embodiment. The positional human flow management device 300 of the third embodiment differs from the positional human flow management device 100 of the first embodiment mainly in that the control unit 340 is provided with a supply unit 341.

[0118] The third embodiment of the location-based pedestrian flow management system differs primarily in that, for example, when VIP customers or foreign language speakers enter a commercial facility, it provides information on the visible characteristics of VIP customers or foreign language speakers to a designated person. The location-based pedestrian flow management device 300 of the third embodiment will be described below, focusing on the differences from the first embodiment.

[0119] In the third embodiment of the location-based pedestrian flow management device 300, the providing unit 341 provides linking information relating to the linking of image person identification information and user identification information. Based on the user identification information, the providing unit 341 determines whether or not a user is a specific user. If the providing unit 341 determines that the user is a specific user, for example, a VIP customer or a foreign language speaker, it provides the linking information to the VIP customer representative or the foreign language representative, respectively.

[0120] The provisioning unit 341 determines, for example, whether user P is a VIP customer or a foreign language speaker based on user attribute information 25 transmitted by the user terminal 20. If user P is a VIP customer, the provisioning unit 341 transmits user P's entry and image person linking information 151 to VIP-friendly tenants or VIP service staff such as their employees or related parties.

[0121] If user P is a foreigner or other foreign language speaker, the service unit 341 transmits user P's entry and image person linking information 151 to foreign language-speaking staff such as employees who can interpret. In the following explanation, among the VIP service staff, those who can interpret are referred to as special VIP service staff, and those who cannot interpret are referred to as regular VIP service staff.

[0122] Next, the processing of the location-based pedestrian flow management device 300 will be described. Figure 18 is a flowchart showing an example of the processing of the location-based pedestrian flow management device 300. In the location-based pedestrian flow management device 300 of the third embodiment, processing common to that in the location-based pedestrian flow management device 100 of the first and second embodiments shown in Figure 9 is performed.

[0123] Furthermore, the flowchart shown in Figure 18 is started, for example, after the flowchart shown in Figure 9, described in the first embodiment, has finished and the image-person linking information has been recorded (step S215). In the location-based human flow management device 300 of the third embodiment, first, the provisioning unit 341 determines whether or not a VIP flag has been added to the user attribute information 25 transmitted by the user terminal 20 (step S701).

[0124] If the service provider determines that the VIP flag is attached to the user attribute information 25, the service provider 341 determines whether or not the foreign language flag is attached to the user attribute information 25 (step S703). If the service provider determines that the foreign language flag is attached to the user attribute information 25, the service provider 341 secures a special VIP service staff member to stay inside or outside the commercial facility M (step S705).

[0125] Next, the tracking process shown in Figure 9 (step S217) is executed, and the processes shown in Figures 14 and 16 in the second embodiment are executed, and the tracking process continues (step S621). Subsequently, the providing unit 341 identifies the disembarking floor to which user P disembarked based on the disembarking floor information included in the disembarking floor information transmitted by the user terminal 20 (step S707).

[0126] Next, the service unit 341 determines whether or not there is a VIP-friendly tenant on the floor where user P disembarks from the elevator (step S709). VIP-friendly tenants are, for example, precious metals stores, luxury brand stores, art stores, and other tenants or retailers that sell relatively expensive goods or services.

[0127] If the service unit 341 determines that there are no VIP-compatible tenants on the disembarking floor, the location-based passenger flow management device 300 terminates the process shown in Figure 18. If it determines that there are VIP-compatible tenants on the disembarking floor, the service unit 341 prompts the VIP-compatible tenant to assign a special VIP service staff member (step S711). Furthermore, the service unit 341 provides the image-based person-linking information 151 to the special VIP service staff member (step S713). In this way, the location-based passenger flow management device 300 terminates the process shown in Figure 18.

[0128] On the other hand, if in step S703 it is determined that the foreign language flag is not attached to the user attribute information 25, the service unit 341 secures, for example, a regular VIP service staff member staying in the commercial facility M (step S715). Subsequently, the service unit 341 identifies the disembarking floor to which user P disembarked based on the disembarking floor information included in the disembarking information transmitted by the user terminal 20 (step S717).

[0129] Next, the provisioning unit 341 prompts the system to station a regular VIP service staff member on the disembarking floor where user P disembarked (step S711). Furthermore, the provisioning unit 341 provides the image person linking information 151 to the special VIP service staff member (step S713). In this way, the location-based passenger flow management device 300 completes the process shown in Figure 18.

[0130] In step S701, if it is determined that the VIP flag is not attached to the user attribute information 25, the provision unit 341 determines whether or not the foreign language flag is attached to the user attribute information 25 (step S719). If the provision unit 341 determines that the foreign language flag is not attached to the user attribute information 25, the location-based human flow management device 300 terminates the process shown in Figure 18.

[0131] If the system determines that a foreign language flag has been added to the user attribute information 25, the service unit 341 secures a foreign language-speaking staff member who is present in the commercial facility M (step S721). Subsequently, the service unit 341 identifies the disembarking floor to which user P disembarked, based on the disembarking floor information included in the disembarking information transmitted by the user terminal 20 (step S723).

[0132] Next, the provisioning unit 341 prompts the foreign language-speaking staff member to be stationed on the disembarking floor where user P disembarked (step S725). Furthermore, the provisioning unit 341 provides the foreign language-speaking staff member with the image person linking information 151 (step S727). In this way, the location-based passenger flow management device 300 completes the process shown in Figure 18.

[0133] The location-based pedestrian flow management system of the third embodiment provides the same effects and advantages as the location-based pedestrian flow management system 1 of the first embodiment. Furthermore, the location-based pedestrian flow management system of the third embodiment can assign appropriate staff according to the characteristics of user P, even if user P is an important person such as a VIP customer or manager of a commercial facility M, or a foreigner who only speaks a foreign language. Therefore, it is possible to provide customers with highly satisfying service.

[0134] Traditionally, the technology used to notify VIP staff of VIP customer arrivals involved creating and registering pre-captured facial data of VIP customers, and comparing customer facial images captured by cameras installed within the commercial facility with the registered facial data. When a VIP customer arrived, the VIP staff were notified of the customer's arrival by attaching the remembered facial data. In this conventional method, for example, if a long period of time had passed since the facial data was captured, the VIP customer's current face (including hairstyle, etc.) and clothing would often have changed from the facial data. If the VIP staff member was unfamiliar with the VIP customer, they may not be able to recognize the VIP staff member's current face. Furthermore, VIP customers sometimes disliked the registration of their facial data from the perspective of protecting personal information.

[0135] In this regard, in the third embodiment of the location-based customer flow management system, after the commercial facility recognizes the arrival of a VIP customer based on the VIP flag included in the user attribute information 25, it links the image person ID of the image captured by the in-store camera 11, etc., with the user ID. As a result, VIP service staff can be notified of images of VIP customers that were captured on the same day, so even VIP service staff who have not met the VIP customer before can recognize the VIP customer's face and clothing without hesitation. Since there is no need to register the VIP customer's facial data, the VIP customer's anxiety about data leakage can also be reduced.

[0136] In the third embodiment described above, when a user entering the commercial facility M is a VIP customer or a foreign language speaker, information on the visible characteristics of the VIP customer or foreign language speaker is provided to a designated person in charge. However, the response when a VIP customer or other user P enters the commercial facility M may differ. For example, when a VIP customer or other user P enters the commercial facility M, the advertising signage on the floor where user P disembarks may display advertisements with content targeted at VIP customers.

[0137] In the embodiments described above, the external transceiver was a GPS satellite AS or a Wi-Fi router, but the external transceiver may be other signal transmitters, such as external beacons or UWB transmitters installed outside the commercial facility M or elevators. The external transceiver may not be a transmitter that sends signals to the user terminal 20, but rather a signal receiver that receives signals transmitted from the user terminal 20, such as a beacon receiver, Wi-Fi receiver, or UWB receiver. In these cases, for example, the external transceiver waves used to determine entry into the commercial facility may be transmitted to the location and pedestrian flow management device 100 by each receiver instead of the user terminal 20.

[0138] The data obtained through tracking in each of the above embodiments may be used for various marketing purposes, event invitation services, coupon distribution services, etc. For example, when providing an event invitation service, if an event is canceled, a coupon to compensate for the loss, tailored to the event cancellation, may be distributed to the user P. Furthermore, weather information may be used to estimate the cancellation of an event.

[0139] (Fourth Embodiment) Next, a fourth embodiment will be described. The location-based pedestrian flow management system of the fourth embodiment differs from the first embodiment mainly in that the storage unit 150 of the location-based pedestrian flow management device 100 stores preference-related data regarding the user's preferences, and in the processing performed in the tracking unit 145 when tracking is performed. The image processing unit 143 performs an action detection process to detect the user's actions (hereinafter referred to as user actions), including the direction the user's face is facing and actions such as picking up products. The tracking unit 145 analyzes the user actions detected by the image processing unit 143 (action analysis) and generates action analysis data. The action analysis process includes, for example, image processing (video processing) of images captured by the in-store camera 11 and processing of information obtained as a result of the image processing. Based on the preference-related data and the action analysis data, the tracking unit 145 determines the information to be provided to the user P, for example, the content of advertisements and coupons that are tailored to the user P's preferences. The image processing unit 143 and the tracking unit 145 may be controlled to process only images containing the user P after identifying the user P to be analyzed, meaning that some or all of the image processing, including user behavior detection and behavior analysis processing for images that do not contain the user P, may not be performed. Continuing to track and analyze a person based on images can result in enormous computational costs due to the analysis process. In the configuration of this embodiment, it is possible to focus the analysis processing on images containing the person or images that are highly likely to contain the person, thereby reducing various computational costs in image processing.

[0140] The preference-related data includes, for example, user attribute information 152, store attribute information 154, product attribute information 156, and product placement information 158 stored in the storage unit 150 of the location-based pedestrian flow management device 100 shown in Figures 19 to 21. In the fourth embodiment, the tracking process in the tracking unit 145 using this preference-related data will be described, followed by a description of the processing of the location-based pedestrian flow system.

[0141] Figure 19 shows an example of the contents of user attribute information 152. User attribute information 152 is information that associates user ID with favorite stores, visit history, products handled, level of interest, and purchase history. Favorite stores are stores that the user frequents from among the stores shown in the store classification defined in store attribute information 154.

[0142] "Visit history" is information that shows the history of a user's visits to their favorite store. Visit history is shown, for example, by visit frequency or number of visits. "Products (services) handled" is information about the products or services (hereinafter referred to as "products, etc.") that are normally handled at the favorite store. "Products (services) handled" are defined in store attribute information 154.

[0143] "Favorite Products" are information about products (services) that are of interest to the user, among the "Products (Services) Handled" included in the store attribute information 154. "Level of Interest" indicates the degree of interest the user has in the favorite products (services). "Level of Interest" is calculated, for example, by the user's usage history (frequency of use) of the product (service) in question, and the number and frequency of times the user has paid attention to the product (for example, looked at it, picked it up, or spoken about it aloud). "Purchase History" is information that shows the history of purchasing the product in question. Purchase history is shown by purchase frequency or number of purchases.

[0144] Figure 20 shows an example of the contents of store attribute information 154. Store attribute information 154 is information that associates store ID with store classification, store location, products handled, etc. "Store classification" is information that indicates the type of store. "Store location" is information about the store's location (floor, location within the floor) within the commercial facility M. "Store classification" indicates the type of store that handles related products, etc., such as "household goods," "restaurants," "sports goods stores," "playground equipment stores," etc. "Products handled (services)" is information about the products, etc. that are mainly handled by the store indicated in the store classification.

[0145] Figure 21 shows an example of the contents of product attribute information 156. Product attribute information 156 is information that specifically indicates individual products, etc. (including the service itself or products provided by the service) included in the products (services) handled. For example, "men's products" may include "clothing," "bags," "wallets," "key cases," and "umbrellas" as individual products.

[0146] Figure 22 shows an example of the contents of product placement information 158. Product placement information 158 is information that indicates the placement of products within a store. Product placement information 158 is, for example, information that is provided for each store and associates the product name and the product's placement location (hereinafter referred to as product placement location) with a product ID assigned to each product.

[0147] The tracking unit 145 analyzes the user P's actions (hereinafter referred to as "user actions") detected by the action detection processing of the image processing unit 143 and generates action analysis data. The action analysis data shows what kind of actions user P is taking. The action analysis data includes data showing user P's movement path, the direction the user is looking (face direction), and whether the user is holding the product.

[0148] Next, the processing in the fourth embodiment of the location-based pedestrian flow management system will be described. Here, the same processing as in the first embodiment of the location-based pedestrian flow management system 1 is performed, and the tracking unit 145 performs information provision processing simultaneously with the tracking processing in step S217 shown in Figure 9. Here, the processing of the location-based pedestrian flow management system when the tracking unit 145 performs the tracking processing will be described.

[0149] After user P enters commercial facility M, their image person ID is linked to their user ID, and they then wander around commercial facility M, entering stores within the facility to shop or receive services. User terminal 20 owned by user P receives in-store beacon signals transmitted by in-store beacons 12 installed at multiple locations within commercial facility M.

[0150] The user terminal 20 calculates the relative distance to the in-store beacon 12 based on the strength of the received in-store beacon signal, generates relative distance information, and transmits it to the location and pedestrian flow management device 100. The location and pedestrian flow management device 100 uses the received relative distance information to perform information provision processing, including tracking processing. The processing of the location and pedestrian flow management device 100 is described below.

[0151] Figure 23 is a flowchart showing an example of the processing of the location-based pedestrian flow management system according to the fourth embodiment. The location-based pedestrian flow system determines whether user P has not entered the store based on the calculated user location and store location in the tracking unit 145 (step S801). If it is determined that user P has not entered the store, the tracking unit 145 acquires non-entry behavior analysis data (step S803). Non-entry behavior analysis data is analysis data from before user P entered the store. Non-entry behavior analysis data includes information on the relative distance between the user location and the store location.

[0152] The tracking unit 145 determines whether user P is approaching the store based on non-store entry behavior analysis data, such as user P's movement path (step S805). For example, the tracking unit 145 determines that user P is approaching the store when user P is heading in the direction of the store and the relative distance between user P and the store is below a predetermined threshold, or when the relative distance between user P and the store is gradually decreasing.

[0153] If the tracking unit 145 determines that user P is not approaching a store, it proceeds to step S813. If the tracking unit 145 determines that user P is approaching a store, it determines whether the store being tracked is a favorite store of user P (step S807). For example, the tracking unit 145 obtains the store ID of the store the user is approaching and obtains the store classification by referring the obtained store ID to the store attribute information 154 shown in Figure 20. The tracking unit 145 refers the obtained store classification to the user attribute information 152 shown in Figure 19 and determines whether the store being tracked is a favorite store of user P based on whether the obtained store classification is a favorite store of user P.

[0154] If the tracking unit 145 determines that the store under evaluation is not a favorite store of user P, it proceeds to step S813. If the tracking unit 145 determines that the store under evaluation is a favorite store of user P, it determines whether user P is paying attention to the store under evaluation (step S809). The tracking unit 145 determines whether the store under evaluation is a favorite store based, for example, on behavioral analysis data, for example, the direction in which user P is paying attention.

[0155] If the tracking unit 145 determines that user P is not paying attention to the target store, it proceeds to step S813. If the tracking unit 145 determines that user P is paying attention to the target store, it transmits and provides advertising information corresponding to the target store to the user terminal 20 (step S811). The advertising information corresponding to the target store includes, for example, information that would be useful to user P when they enter the target store, such as information on products sold at the target store and special offers.

[0156] Next, the tracking unit 145 determines, for example, whether user P has left the commercial facility M based on a signal indicating GPS strength transmitted by the user terminal 20 (step S813). If it determines that user P has not left the commercial facility M, the tracking unit 145 returns to step S801. If the tracking unit 145 determines that user P has left the commercial facility M, the location flow management device terminates the process shown in Figure 23.

[0157] In step S801, if it is determined that the user has entered the store (i.e., has already entered the store), the tracking unit 145 determines whether or not the store that user P entered (hereinafter referred to as the entered store) has user P's favorite products (step S815). If it is determined that the entered store does not have user P's favorite products, the tracking unit 145 proceeds to step S813.

[0158] If the tracking unit 145 determines that a store P has entered has a favorite product of user P, it acquires store entry behavior analysis data (step S817). Store entry behavior analysis data is behavior analysis data after the user enters the store. Store entry behavior analysis data includes, for example, information on the direction in which user P is looking and whether user P has picked up a product, etc.

[0159] Next, the tracking unit 145 determines whether user P is looking at their favorite products based on the direction user P is looking and the location of the products (including the height of the products displayed on the shelves) included in the store entry behavior analysis data (step S819). For example, the tracking unit 145 identifies the location in front of user P's face and refers the identified location to the placement location in the product placement information 158 to identify the product that user P is looking at. The tracking unit 145 then refers to the product that user P is looking at to the user attribute information 152 shown in Figure 19 to determine whether user P is looking at their favorite products.

[0160] If the tracking unit 145 determines that user P is not paying attention to the favorite product, it proceeds to step S813. If the tracking unit 145 determines that user P is paying attention to the favorite product, it transmits and provides advertising information for the favorite product to the user terminal 20 (step S821). The advertising information for the favorite product includes, for example, information that would encourage user P to purchase the favorite product, such as information about the features and usage of the favorite product.

[0161] Next, the tracking unit 145 determines whether user P is holding a favorite product based on the store entry behavior analysis data (step S823). If it determines that user P is not holding a favorite product, the tracking unit 145 proceeds to step S813. If it determines that user P is holding a favorite product, the tracking unit 145 transmits coupon information for the favorite product to the user terminal 20 (step S825).

[0162] The coupon information is, for example, information about coupons that can be used to get a discount on a favorite product. By providing coupon information to user P, it is possible to encourage user P to purchase their favorite products. After that, the tracking unit 145 proceeds to step S813 to determine whether or not the user has left the commercial facility and then proceeds with the processing after that. In this way, the location and flow management device completes the processing shown in Figure 23.

[0163] The location-based pedestrian flow management system of the fourth embodiment provides the same effects and advantages as the location-based pedestrian flow management system 1 of the first embodiment. Furthermore, the location-based pedestrian flow management system of the fourth embodiment provides advertisements and coupons to users after linking image person IDs with user IDs. Therefore, it is possible to provide users with appropriate advertisements and coupons.

[0164] In the fourth embodiment, the user attribute information 152 may include information that identifies the interests of user P, in place of or in addition to visit history and purchase history. This information may include, for example, that the first user P's interests are men's shoes, and the second user P's interests are jewelry. When classifying these interests, they may be identified, for example, by product classification used in a POS system.

[0165] The user behavior detected by the image processing unit 143 may be the movement path of user P as they move or attempt to move, or it may be the user's actions such as approaching a product or reaching for a product that is in a high place. This user behavior may be referenced when providing advertising information or coupon information to user P who has entered the store.

[0166] Instead of determining whether user P approaches a store and approaches or looks at the store under evaluation, or in addition to determining whether user P approaches or looks at store-related information corresponding to the store under evaluation, the system may also determine whether user P approaches or looks at store-related information corresponding to the store under evaluation. Store-related information may include, for example, advertisements for the store under evaluation within a commercial facility M, or facilities corresponding to the store under evaluation, such as facilities that issue coupons usable at the store under evaluation. Advertisements for the store under evaluation may include banners, signs, or digital signage.

[0167] The tracking unit 145 may comprehensively utilize preference-related data and behavioral analysis data to identify the user's real-time needs and provide advertisements and coupons. For example, if user P approaches a store that sells their favorite products, the tracking unit 145 may determine that user P is considering purchasing those favorite products as a real-time need. Conversely, if user P approaches a store that does not sell their favorite products, the tracking unit 145 may determine that user P has no real-time needs.

[0168] Furthermore, the tracking unit 145 may identify the level of interest to provide based on the level of interest identified for each of the user P's favorite products or based on the level of interest derived from behavioral analysis. For example, the tracking unit 145 may identify the level of interest in products that user P is paying attention to based on the user attribute information 152 shown in Figure 19, and adjust the content of the advertisements and coupons provided according to the level of interest. In addition, the tracking unit 145 may estimate the level of interest in products based on the results of behavioral analysis of user P. For example, the tracking unit 145 may estimate that user P is more interested in a product when they pick it up and hold it than when they are merely looking at it.

[0169] In the fourth embodiment, the tracking unit 145 identifies users whose user behavior is to be analyzed (hereinafter referred to as "users to be analyzed"), and the image processing unit 143 may reduce the burden of image processing by limiting the subjects of image processing to users to be analyzed. Users to be analyzed are basically users whose image person ID and user ID are linked by the linking unit 144, but for such users, image person linking information 151 and conditions for users to be targeted may be set in advance in the storage unit 150, or they may be set when they are users of interest after behavioral analysis has been performed. For example, VIP customers or foreigners may be set in advance as users to be analyzed, or as a result of behavioral analysis, users who repeatedly engage in shoplifting-like behavior or users who grasp or stare at specific products for a long time may be set as users to be analyzed. Alternatively, among users whose image person ID and user ID are linked by the linking unit 144, VIP customers, foreigners, and those who repeatedly engage in shoplifting-like behavior may be set in advance as users to be analyzed, and other users may be excluded from users to be analyzed. By excluding other individuals from the users being analyzed, the processing burden on those not targeted for user behavior analysis can be reduced.

[0170] Furthermore, the tracking unit 145 may, for example, reduce the processing load by only analyzing images that include the target user, or by only analyzing the target user even if the captured image includes multiple customers including the target user, thereby avoiding behavioral analysis for other users P. The image processing unit 143 may also reduce the processing load by not processing images of users other than the target user (users for whom behavioral analysis is not performed).

[0171] The location-based pedestrian flow management device of this embodiment includes: a determination unit that acquires a determination result of whether the user has entered the target area, determined based on the external transmission and reception state of the external transmission and reception waves between the user terminal held by the user and an external transceiver outside the target area that transmits and receives external transmission and reception waves, and the internal transmission and reception state of the internal transmission and reception waves between the user terminal and an internal transceiver inside the target area that transmits and receives internal transmission and reception waves; an acquisition unit that acquires an image of a person in a specific area that the user passes through when entering the target area, captured by an imaging device; and a linking unit that links image person identification information assigned to the person in the image with user identification information relating to the user. The linking unit can accurately identify customers by linking the image person identification information with the user identification information based on the image at the time the user, whose entry into the target area has been determined, passed through the specific area.

[0172] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0173] 1, 2...Location-based pedestrian flow management system, 11...In-store camera, 12...In-store beacon, 13...In-flight camera, 14...In-flight beacon, 15...Wi-Fi router, 20...User terminal, 21...Communication unit, 22...Touch panel, 23...Beacon receiver, 24...Memory, 25...User attribute information, 26...GPS application, 27...Location-based pedestrian flow management application, 30...Processing unit, 31...GPS processing unit, 32...Location-based pedestrian flow management processing unit, 33...Wi-Fi processing unit, 100, 300 ...Location-based pedestrian flow management device, 110...Communication interface, 120...Input interface, 130...Output interface, 140, 340...Control unit, 141...Determination unit, 142...Acquisition unit, 143...Image processing unit, 144...Linking unit, 145...Tracking unit, 150...Storage unit, 151...Image person linking information, 341...Providing unit, AS...GPS satellite, ED, MD...Automatic door, EV...Elevator, ME...Entrance, NW...Network, P...User.

Claims

1. A location-based human flow management device comprising: a determination unit that acquires a determination result of whether the user has entered the target area, determined based on the external transmission and reception state of the external transmission and reception waves between the user terminal possessed by the user and an external transceiver outside the target area that transmits and receives external transmission and reception waves, and the internal transmission and reception state of the internal transmission and reception waves between the user terminal and an internal transceiver inside the target area that transmits and receives internal transmission and reception waves; an acquisition unit that acquires an image taken by an imaging device that includes a person in a specific area that the user passes through when entering the target area; and a linking unit that links image person identification information assigned to a person in the image with user identification information relating to the user, wherein the linking unit links the image person identification information with the user identification information based on the image at the time the user, whose entry into the target area has been determined, passed through the specific area.

2. The location-based pedestrian flow management device according to claim 1, wherein the target area is the interior of a building.

3. The location-based human flow management device according to claim 2, wherein the external transceiver is a GPS satellite.

4. The location-based human flow management device according to claim 3, wherein the external transmission / reception state includes attenuation of the intensity of the external transmission / reception wave within a certain period of time.

5. The location-based pedestrian flow management device according to claim 1, wherein the target area is an elevator.

6. The location-based human flow management device according to claim 5, wherein the external transceiver is a Wi-Fi router.

7. The location-based pedestrian flow management device according to claim 6, wherein the Wi-Fi router is installed on each floor of the building where the elevator is installed, and the external transmitting and receiving waves include information regarding the number of floors in the target area.

8. The location-based pedestrian flow management device according to claim 5, wherein the external transmission / reception state includes attenuation of the Wi-Fi signal strength in response to the closing of the elevator door.

9. The location-based pedestrian flow management device according to claim 5, wherein the acquisition unit acquires disembarking floor information relating to the floor on which a user who has boarded the elevator and whose user identification information is linked to the image person identification information disembarks from the elevator.

10. The location-based pedestrian flow management device according to claim 1, wherein the linking unit, when there are multiple people in the specific area, matches the user's attributes with the attributes of the people, and links the image person identification information with the user identification information based on the result of the matching.

11. The location-based pedestrian flow management device according to claim 10, wherein the attribute includes at least one of age, gender, height, hairstyle, body shape, posture, or appearance preference.

12. The location-based human flow management device according to claim 1, further comprising a tracking unit that tracks the user whose image person identification information and user identification information are linked by the linking unit and analyzes their behavior.

13. The location-based human flow management device according to claim 12, wherein the tracking unit determines information to be provided to the user based on preference-related data relating to the user's preferences and behavioral analysis data analyzing the user's behavior.

14. The location-based pedestrian flow management device according to claim 13, wherein the information provided to the user includes at least one of an advertisement or a coupon corresponding to the preference-related data.

15. The location-based human flow management device according to claim 13, wherein the tracking unit determines the user's level of interest based on the preference-related data and the behavioral analysis data, and determines the information to be provided to the user according to the level of interest.

16. The location-based pedestrian flow management device according to claim 1, further comprising a providing unit that provides linking information relating to the linking of image person identification information and user identification information.

17. The location-based human flow management device according to claim 16, wherein the linking unit determines whether the user is a specific user based on the user identification information, and the providing unit provides the linking information if the user is a specific user.

18. The location-based human flow management device according to claim 12, wherein the tracking unit tracks users who are subject to analysis and have been set in advance or based on behavioral analysis data.

19. A location-based human flow management method comprising: a computer obtaining a determination result of whether the user has entered the target area, determined based on the external transmission and reception state of the external transmission and reception waves between the user terminal possessed by the user and an external transceiver outside the target area that transmits and receives external transmission and reception waves, and the internal transmission and reception state of the internal transmission and reception waves between the user terminal and an internal transceiver inside the target area that transmits and receives internal transmission and reception waves; obtaining an image of the specific area through which the user passes when entering the target area, captured by an imaging device; linking image person identification information assigned to the person in the image with user identification information relating to the user; and the computer linking the image person identification information with the user identification information based on the image at the time the user, whose entry into the target area has been determined, passed through the specific area.

20. A program that causes a computer to obtain a determination result of whether the user has entered the target area, determined based on the external transmission and reception state of the external transmission and reception waves between the user terminal possessed by the user and an external transceiver outside the target area that transmits and receives external transmission and reception waves, and the internal transmission and reception state of the internal transmission and reception waves between the user terminal and an internal transceiver inside the target area that transmits and receives internal transmission and reception waves; obtain an image of the specific area that the user passes through when entering the target area, captured by an imaging device; link the image person identification information assigned to the person in the image with user identification information relating to the user; and cause the computer to link the image person identification information with the user identification information based on the image at the time the user, whose entry into the target area has been determined, passed through the specific area.

Images