Human flow management system, human flow management program, and human flow management method

Abstract

We provide a human flow management system, a human flow management method, and a human flow management program that accurately manage human flow data. [Solution] A crowd flow management system for managing information on the flow of people at an exhibition, the crowd flow management device comprising an image acquisition unit, a registration unit, an identification unit, and a calculation unit, the image acquisition unit acquires a first image of an area accessible only to non-visitors and a second image of an area to be analyzed, the registration unit registers the appearance images and / or characteristics of non-visitors in a database based on the first image, the identification unit identifies visitors based on the appearance images and / or characteristics of non-visitors and the second image, and the calculation unit calculates crowd flow information at the exhibition based on the identified visitors.

Description

Technical Field

[0001] The present invention relates to a crowd flow management system, a crowd flow management program, and a crowd flow management method.

Background Art

[0002] Conventionally, by measuring the crowd flow, the behavior of people has been analyzed. In particular, an example of a system for measuring the crowd flow at an event has been proposed, for example, in Patent Document 1.

[0003] Patent Document 1 discloses that arrival person information is read from a storage in order from the earliest shooting time, and each time arrival person information is read, it is determined whether the arrival person indicated by the arrival person information is an arrival person detected in a previous process. If there is no read arrival person information with a matching collation result, it is determined that the arrival person is a newly detected arrival person. In the case of a newly detected arrival person, the attributes or status of the arrival person to be identified can be set. The feature amounts F1 to FN included in the arrival person information of the newly detected arrival person are collated with the feature amounts indicating the attributes or status set as the target to be counted, and based on the collation result, it is determined whether the person from whom the arrival person information was extracted should be counted as the target. If the collation result matches, a predetermined counter is incremented and the arrival person information is stored in a memory as read arrival person information.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In Patent Document 1, there is a problem that it is impossible to distinguish between arrival persons and event-related persons (non-arrival persons) and manage the crowd flow.

[0006] In view of the above circumstances, the problem that the present invention aims to solve is to provide a novel technology that enables accurate management of pedestrian flow data. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides a human flow management system for managing information on the flow of people at an exhibition, the human flow management system comprising an image acquisition unit, a registration unit, an identification unit, and a calculation unit, wherein the image acquisition unit acquires a first image of an area accessible only to non-visitors and a second image of an area to be analyzed, the registration unit registers the appearance images and / or characteristics of the non-visitors in a database based on the first image, the identification unit identifies visitors based on the appearance images and / or characteristics of the non-visitors and the second image, and the calculation unit calculates human flow information at the exhibition based on the identified visitors.

[0008] Furthermore, in order to solve the above problems, the present invention provides a crowd management program for managing information on the flow of people at an exhibition, wherein the crowd management program uses a computer as an image acquisition unit, a registration unit, an identification unit, and a calculation unit, the image acquisition unit acquires a first image of an area accessible only to non-visitors and a second image of an area to be analyzed, the registration unit registers the appearance images and / or characteristics of the non-visitors in a database based on the first image, the identification unit identifies visitors based on the appearance images and / or characteristics of the non-visitors and the second image, and the calculation unit calculates crowd flow information at the exhibition based on the identified visitors.

[0009] Furthermore, in order to solve the above problems, the present invention provides a method for managing information on the flow of people at an exhibition, wherein a computer performs the following processes: acquiring a first video showing an area accessible only to non-visitors and a second video showing an area to be analyzed; registering the appearance images and / or characteristics of the non-visitors in a database based on the first video; identifying visitors based on the appearance images and / or characteristics of the non-visitors and the second video; and calculating the flow of people at the exhibition based on the identified visitors.

[0010] This configuration allows for the reliable identification of non-visitors, precise identification of visitors, and accurate calculation of pedestrian flow information at the exhibition.

[0011] In a preferred embodiment of the present invention, the registration unit registers, based on the acquired first video, a plurality of person appearance images and / or person feature quantities captured in the first video as non-visitor feature information, and updates non-visitor reference information for identifying the non-visitor on a daily basis based on the non-visitor feature information.

[0012] Depending on the exhibition, the individuals treated as non-visitors may differ from day to day. This configuration allows us to handle such cases where non-visitors change from day to day, enabling us to calculate accurate visitor flow information.

[0013] In a preferred embodiment of the present invention, each time the non-visitor is captured in the first video, the registration unit registers non-visitor characteristic information, including the person appearance image of the non-visitor and / or the characteristic quantities of the non-visitor, based on the first video in which the non-visitor is captured.

[0014] In a preferred embodiment of the present invention, the registration unit registers the non-visitor characteristic information based on a newly captured full-body image of the same non-visitor if the non-visitor characteristic information based on a newly captured full-body image of the same non-visitor differs from the non-visitor characteristic information based on a previously registered full-body image of the same non-visitor.

[0015] This configuration allows for reliable identification of a non-visitor even if the same person's clothing or hairstyle changes.

[0016] In a preferred embodiment of the present invention, the second image is an image of an area to be analyzed at an exhibition, the area to be analyzed includes the exhibition area and a predetermined front area in front of the exhibition area, the identification unit identifies potential visitors entering the front area based on the second image, and the calculation unit calculates the number of visitors as the flow information based on the number of potential visitors.

[0017] In a preferred embodiment of the present invention, the second image is an image of an area to be analyzed at an exhibition, the area to be analyzed includes an exhibition area and a front area, the identification unit identifies visitors who are people entering the exhibition area from the front area and potential visitors who are people entering the front area based on the second image, and the calculation unit calculates the attraction rate as the pedestrian flow information based on the number of visitors and potential visitors.

[0018] By using this configuration, it is possible to improve the accuracy of analyzing pedestrian traffic in the exhibition area by identifying people moving not only within the exhibition area but also in the front area.

[0019] In a preferred embodiment of the present invention, the calculation unit calculates net stay-related information for the area to be analyzed as the pedestrian flow information, based on a predetermined threshold for the number of people staying in the area to be analyzed at the exhibition.

[0020] In a preferred embodiment of the present invention, the calculation unit calculates the net stay-related information based on the time when the number of people staying in the analysis target area exceeds the stay number threshold value and the time when the number of people staying in the analysis target area is below the stay number threshold value.

[0021] With such a configuration, even when there are people who cannot be identified as non-visitors in the analysis target area, it is possible to prevent inaccurate stay-related information from being registered by such people.

[0022] In a preferred embodiment of the present invention, the crowd management system further includes an attraction identification unit. The attraction identification unit identifies the face angle or line of sight of the imaged person and the trajectory information of the imaged person based on the second video, and identifies the attraction information to the analysis target area in the exhibition based on the face angle or line of sight of the person and the trajectory information of the person. The registration unit registers the attraction information as the crowd flow information.

[0023] With such a configuration, it is possible to grasp how a person is attracted to the analysis target area based on the face angle and trajectory of the imaged person.

[0024] In a preferred embodiment of the present invention, the crowd management system further includes a map generation unit. The video acquisition unit acquires video data for each of a plurality of imaging devices. The map generation unit generates, for each imaging device, an individual determination map in which the determination rate of the imaged person and the position where the person is imaged are associated based on each of the video data, and generates an imaging allocation map for specifying the imaging area of each imaging device based on the individual determination maps of the plurality of imaging devices.

[0025] With such a configuration, it is possible to optimize the identification of people using a plurality of imaging devices, realize accurate person identification, and calculate accurate crowd flow information.

Effects of the Invention

[0026] The present invention can provide a novel technology that enables accurate management of human flow data.

Brief Description of the Drawings

[0027] [Figure 1] It is a block diagram showing the configuration of the system in one embodiment of the present invention. [Figure 2] It is a block diagram of the hardware configuration of the system in the present invention. [Figure 3] It is a block diagram of the functional configuration in one embodiment of the present invention. [Figure 4] It is an image diagram of human flow measurement in one embodiment of the present invention. [Figure 5] It is an example of a processing flowchart in one embodiment of the present invention. [Figure 6] It is a diagram showing an example of an imaging allocation map in the present invention. [Figure 7] It is an example of a screen displayed on the user terminal device in the present invention.

Modes for Carrying Out the Invention

[0028] (Embodiment 1) Hereinafter, it will be described in more detail with reference to the attached screens. Preferred embodiments are shown in the drawings. However, it can be implemented in many different forms and is not limited to the embodiments described in this specification.

[0029] For example, in this embodiment, the configuration, operation, etc. of the human flow management system will be described. However, the same effects can also be achieved by the method, device, computer program, etc. executed. Further, the program may be stored in a recording medium. By using this recording medium, for example, a program can be installed in a computer, and thereby a human flow management device and a human flow management system can be configured. Here, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

[0030] <1. Overview of Embodiment 1> The present invention relates to a system for managing information on pedestrian flow at an event. In this invention, an imaging device (hereinafter referred to as the second imaging device) is installed at the event venue to image an area to be analyzed for pedestrian flow and the area surrounding the area to be analyzed, and an imaging device (hereinafter referred to as the first imaging device) is installed to image a non-visitor recording area that is accessible only to persons who are not subject to pedestrian flow analysis (for example, event personnel and staff, hereinafter referred to as non-visitors). In this embodiment, based on the image captured by the first imaging device, which includes at least non-visitors (hereinafter referred to as the first image), and the image captured by the second imaging device, which includes the area to be analyzed and the area surrounding the area to be analyzed (hereinafter referred to as the second image), persons who enter the area to be analyzed and who are subject to pedestrian flow analysis (hereinafter referred to as visitors) are identified, and pedestrian flow information for the area to be analyzed is calculated.

[0031] In this embodiment, the areas to be analyzed are the exhibition booth area of ​​the event (hereinafter referred to as the exhibition area), a predetermined area in front of the exhibition area (hereinafter referred to as the front area), and the flow of people to the exhibition area. Based on the first and second video, visitors who have entered the exhibition area and people who are leaving the exhibition area (hereinafter referred to as past visitors), and people who have entered the front area and are likely to enter the exhibition area (hereinafter referred to as potential visitors) are identified, and pedestrian flow information related to the area to be analyzed is calculated.

[0032] In this embodiment, video is used as the image, but still images may also be used. Furthermore, the area to be analyzed can be any area with visitor traffic, and is not limited to the exhibition area. Also, while the non-visitor recording area is described using a stockroom attached to each exhibition area, it can be any area where only non-visitors enter and exit, for example, an area used by non-visitors from multiple exhibition areas. Furthermore, the front area is not limited to the area in front of the exhibition area in one direction, but includes the area in front of the exhibition area in all directions. Also, while the event is described as a trade show, it can be a short-term event or a permanent event, and is not limited to that.

[0033] <1.1. System Configuration of Embodiment 1> Figure 1 is a block diagram showing the configuration of the system of Embodiment 1. As shown in Figure 1, the pedestrian flow management system 0 comprises a pedestrian flow management device 1, a first imaging device 3, a second imaging device 4, and a user terminal device 5, and is configured to communicate via a communication network NW. The communication network NW in the present invention is an IP (Internet Protocol) network, but there are no restrictions on the type of communication protocol, and furthermore, there are no restrictions on the type or size of the network.

[0034] The pedestrian flow management device 1 can utilize a general-purpose server computer or personal computer. The first imaging device 3 and the second imaging device 4 can utilize general-purpose video cameras such as network cameras capable of recording video. The user terminal device 5 can utilize a smartphone, tablet, personal computer, wearable device, etc. Furthermore, the pedestrian flow management device 1 may be composed of multiple computers capable of sending and receiving information via a communication network NW or another network.

[0035] <1.2. Hardware Configuration of the Invention> Figure 2 is a block diagram of the hardware configuration of the pedestrian flow management system 0. As shown in Figure 2(a), the server 10 (pedestrian flow management device 1) comprises a processing unit 101, a storage unit 102, and a communication unit 103.

[0036] The processing unit 101 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation process of the human flow management device 1 by executing the human flow management program, OS, and other applications according to the present invention. The storage unit 102 includes a volatile memory such as RAM capable of storing instruction sets, and a non-volatile recording medium such as an HDD or SSD capable of recording the OS and the human flow management program according to the present invention. The communication unit 103 has a communication interface device with the communication network NW, and performs communication control with the communication network NW to input and output information.

[0037] As shown in Figure 2(b), the imaging device 8 (first imaging device 3 and second imaging device 4) comprises a processing unit 81 that controls the operation of the imaging device 8, a storage unit 82 that stores the captured images, a communication unit 83 that communicates with the server 10, and an imaging unit 84 that captures images.

[0038] As shown in Figure 2(c), the terminal device 9 (user terminal device 5) comprises a processing unit 91, a storage unit 92, a communication unit 93, an input unit 94, and an output unit 95.

[0039] The processing unit 91 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation process of the terminal device 9 by executing the OS and other applications. The memory unit 92 includes volatile memory such as RAM capable of storing instruction sets, and non-volatile recording media such as HDDs or SSDs capable of storing the OS, etc. The communication unit 93 has a communication interface device for connecting to a network and performs communication control with the communication network NW to input and output information. The input unit 94 has an input device capable of input processing, such as a keyboard or a touch panel. The output unit 95 has a display device capable of display processing, such as a display.

[0040] <1.3. System Functional Configuration> Figure 3 is a block diagram of the functional configuration of the pedestrian flow management device 1. As shown in Figure 3, the pedestrian flow management device 1 comprises an image acquisition unit 11, a registration unit 12, a identification unit 13, a calculation unit 14, an attraction identification unit 15, a map generation unit 16, a display processing unit 17, and a database 2. This represents the concrete realization of information processing by software (stored in the storage unit 102) by hardware (processing unit 101).

[0041] In this embodiment, the system configuration is a so-called server-client type, where the user terminal device 5 (client) receives the processing results performed by the pedestrian flow management device 1 (server) in response to the user terminal device's request. Alternatively, it may be a so-called standalone type, where the pedestrian flow management program is launched on the client terminal. In this case, the user terminal device 5 may include some or all of the functional components (parts) of the pedestrian flow management device 1. For example, the user terminal device 5 may include an image acquisition unit 11, a registration unit 12, a specification unit 13, a calculation unit 14, an attraction specification unit 15, a map generation unit 16, and a display processing unit 17, and the pedestrian flow management device 1 may be a cloud storage that stores image data captured by the imaging device 8.

[0042] <1.3.1. Database 2> Database 2 stores the video information to be analyzed, non-visitor reference information, and area pedestrian flow count information.

[0043] <1.3.1.1. Video Information to be Analyzed> The video information to be analyzed is information about the first and second videos to be analyzed, received from the imaging device 8. The video information to be analyzed is assigned a video ID, imaging device ID, and date to uniquely identify the video, and is stored for each event identification information (event name, etc.) and exhibition area identification information (exhibiting company name, etc.).

[0044] <1.3.1.2. Information for Non-Visitors> Non-visitor reference information is information referenced to identify individuals who should be classified as non-visitors. Non-visitor characteristic information is set daily as non-visitor reference information. Non-visitor characteristic information includes a non-visitor ID to uniquely identify the non-visitor, an image of the non-visitor's appearance, and the non-visitor's feature quantities (multidimensional vector representation). In this embodiment, non-visitor characteristic information is set by assigning one non-visitor ID to each non-visitor's appearance image and / or feature quantity.

[0045] <1.3.1.3. Area Foot Traffic Count Information> Area pedestrian flow count information is information used to calculate pedestrian flow information in the area being analyzed. Area pedestrian flow count information includes a time-series identifier, exhibition area count information related to pedestrian flow in the exhibition area, movement path count information related to pedestrian flow along the movement paths to the exhibition area, and front area count information related to pedestrian flow in the front area. In this embodiment, the time-series identifier is the video frame, but the imaging time may also be used.

[0046] The exhibition area count information includes the total number of visitors who entered the exhibition area, and the provisional number of visitors who were currently staying in the exhibition area.

[0047] The movement count information includes the cumulative number of people who entered the movement path to the exhibition area (cumulative number of people entering the path) and the cumulative number of people who exited the movement path to the exhibition area (cumulative number of people exiting the path).

[0048] Front area count information includes the cumulative number of existing visitors who have entered the front area from the exhibition area, the provisional number of potential visitors currently staying in the front area, the cumulative number of visitors who have entered the exhibition area from the front area, and the cumulative number of potential visitors who have entered the front area from outside the exhibition area.

[0049] Figure 4 is an illustrative diagram of the counting process for front area count information. The cumulative number of existing visitors is the cumulative number of people moving in the in(1) direction relative to the front area in Figure 4. The cumulative number of attracted visitors is the cumulative number of people moving out of the front area in Figure 4. The cumulative number of potential visitors is the cumulative number of people moving in the in(1), in(2), and in(3) directions relative to the front area in Figure 4.

[0050] In a preferred embodiment of the present invention, individual analysis target areas are set within the exhibition area, and individual count information for the individual analysis target areas is set as area pedestrian flow count information. Specifically, the individual count information includes the cumulative number of individual visitors, which is the total number of visitors who entered the individual analysis target area (for example, a predetermined area in front of the exhibition boards), and the provisional number of individual visitors staying in the individual analysis target area.

[0051] <1.3.2. Video Acquisition Unit 11> The video acquisition unit 11 acquires the first video and the second video. The video acquisition unit 11 acquires the first video, which captures only non-visitors, and the second video, which captures at least the exhibition area and the front area, as well as people moving around the event venue, and registers them in the database 2 as video information to be analyzed.

[0052] <1.3.3. Registration Section 12> The registration unit 12 registers non-visitor characteristic information. Based on the first video acquired by the video acquisition unit 11, the registration unit 12 registers non-visitor characteristic information, including the appearance image of the non-visitor and / or the person's characteristic quantities, into the database 2.

[0053] The registration unit 12 also updates the non-visitor reference information. Based on the first video acquired in a day, the registration unit 12 registers non-visitor characteristic information for each of the multiple non-visitors captured in the first video, and updates the non-visitor reference information on a daily basis.

[0054] <1.3.4. Specific part 13> The identification unit 13 identifies individuals entering and exiting the exhibition area and the front area. Based on non-visitor reference information and the second video, the identification unit 13 identifies visitors, potential visitors, and past visitors.

[0055] <1.3.5. Calculation Section 14> The calculation unit 14 calculates pedestrian flow information related to the exhibition area. The calculation unit 14 includes a stay-related data calculation unit 141 that calculates net stay-related information regarding the length of time visitors spend in the area under analysis as pedestrian flow information, a visitor turnover calculation unit 142 that calculates the number of people who visit the exhibition area as pedestrian flow information, and an attraction rate calculation unit 143 that calculates the attraction rate to the exhibition area as pedestrian flow information. Details of the processing of each calculation unit will be described later.

[0056] <1.3.6. Attractant Specification Section 15> The attraction identification unit 15 identifies the angle of the person's face and the person's trajectory information based on the second video. The attraction identification unit 15 identifies the angle of the person's face and the person's trajectory information based on the second image of a series of frames of the second video.

[0057] Furthermore, the attraction identification unit 15 registers information on how to attract visitors to the exhibition area. Based on the facial angle of the identified person and the trajectory information of the person, the attraction identification unit 15 registers attraction information as pedestrian flow information indicating that the person was attracted to the exhibition area.

[0058] <1.3.7. Map Generation Unit 16> The map generation unit 16 generates an imaging allocation map to identify the imaging areas of multiple second imaging devices 4. Based on the video data acquired for each of the multiple second imaging devices 4, the map generation unit 16 generates an individual determination map for each second imaging device 4 that associates the determination rate of the person being imaged and the position in which the person was imaged. Based on the individual determination map for each second imaging device 4 and the acquired imaging time, the map generation unit 16 generates an imaging allocation map to identify the imaging area of ​​each camera.

[0059] <1.3.8. Display Processing Unit 17> The display processing unit 17 overlays various information onto the video footage of the area to be analyzed and displays the result on the user terminal device 5. An example of the video display will be described later.

[0060] <1.4. Processing Flowchart> Referring to Figure 5, the method of managing human flow using the human flow management system 0 will be explained. Figure 5 is a flowchart showing the process from when the human flow management device 1 optimizes the imaging areas of multiple second imaging devices 4, calculates human flow information using video data (first video and second video) acquired from the imaging device 8 over one day, and updates the non-visitor reference information when the date is updated. In the flowchart shown in Figure 5, the process is completed when the non-visitor reference information is updated, but processes S4 to S7 are executed for the human flow information of the next day using the updated non-visitor reference information.

[0061] <1.4.1. Generating the imaging assignment map> First, in step S1 (hereinafter, "step SX" will be simply abbreviated as "SX"), the map generation unit 16 generates an imaging assignment map. In this embodiment, the video acquisition unit 11 acquires a second video for each of the multiple second imaging devices 4 arranged for each event, which includes the imaging time and a combination of second images including at least the exhibition area and the front area.

[0062] The map generation unit 16 generates an individual judgment map for each second imaging device 4, which associates the detection rate of the person being captured and the position where the person was captured, based on the second image captured by each second imaging device 4 and pre-registered person features. The map generation unit 16 then synchronizes the position and detection rate of the captured person using the acquired imaging time for each individual judgment map generated for each second imaging device 4, and for a given position, identifies the second imaging device 4 among the multiple second imaging devices 4 that has the highest detection rate of the person at that position, and assigns that position as the imaging area of ​​that second imaging device 4. The map generation unit 16 then repeats the above process for multiple positions in the captured area to generate an imaging assignment map.

[0063] Figure 6 shows an example of a method for creating an imaging assignment map (a) and an example of a created imaging assignment map (b).

[0064] Figure 6(a) shows an example of an individual judgment map generated based on a second image captured by the second imaging device 4(B). In the illustrated example, a person whose personal features have been pre-registered walks comprehensively through the analysis area, causing the second imaging device 4(B) to capture an image of the person. Based on the captured second image, the person's judgment rate and position are associated, generating an individual judgment map (the darker the black, the lower the judgment rate). Then, by combining the individual judgment maps of each second imaging device 4(A, B, C), an imaging assignment map is generated that optimizes the person's judgment rate (Figure 6(b)).

[0065] Once the process in S1 is complete, the calculation of pedestrian flow information for the specific analysis area described below will be executed.

[0066] <1.4.2. Acquisition of video data> In S2, the video acquisition unit 11 acquires the first video and the second video. In this embodiment, the video acquisition unit 11 receives the designation of the event and exhibition area for which pedestrian flow information is to be calculated, and acquires the first video and the second video corresponding to the event and exhibition area from the database 2. Alternatively, the video acquisition unit 11 may directly acquire the first video from the first imaging device 3 and the second video from the second imaging device 4.

[0067] <1.4.3. Registration of reference information for non-visitors> In S3, the registration unit 12 registers non-visitor reference information based on the first video. In this embodiment, the registration unit 12 inputs each frame (first image) of the first video into a well-known image recognition model to calculate the feature quantities of the whole body image of the non-visitor captured in the first image. The whole body image of the non-visitor is used as the person appearance image, and the registration unit 12 registers visitor feature information including the person appearance image and the feature quantities of the whole body image, as well as non-visitor reference information including the date it was captured. Note that features based on a face image may be used instead of a whole body image as the feature quantities of the non-visitor.

[0068] In this embodiment, each time a non-visitor is captured in the first video, the registration unit 12 registers non-visitor characteristic information based on the first video in which the non-visitor was captured. Specifically, if the visitor characteristic information (personal appearance image of the whole body image, and / or feature quantities of the whole body image) based on a newly captured full-body image of a non-visitor differs from the visitor characteristic information of the same non-visitor that has already been registered, the registration unit 12 registers the non-visitor characteristic information based on the newly captured full-body image of the same non-visitor.

[0069] In a preferred embodiment of the present invention, if the uniform worn by a non-visitor has a distinctive feature (such as a logo or company name), the registration unit 12 calculates a feature quantity by assigning weights to the distinctive feature on the entire body and registers the non-visitor reference information.

[0070] <1.4.4. Identification of Visitors> In S4, the identification unit 13 identifies visitors, potential visitors, and past visitors based on non-visitor reference information and the second video. In this embodiment, the identification unit 13 identifies people captured in the second video by inputting the second video acquired in S2 into a well-known image recognition model. The identification unit 13 then compares the person appearance image or feature quantity of the identified person with the person appearance image or feature quantity of the non-visitor reference information registered in S3, and identifies people whose similarity between the person appearance image or feature quantity of the identified person and the person appearance image or feature quantity of the non-visitor reference information is below a predetermined threshold as visitors, potential visitors, and past visitors. Specifically, the location of the front area is set for each exhibition area, and the identification unit 13 identifies visitors, potential visitors, and past visitors based on the person appearance image or feature quantity of people entering and exiting the exhibition area and the front area associated with that exhibition area.

[0071] The identification unit 13 then registers area pedestrian flow count information in the database 2 based on the identified visitors, potential visitors, and past visitors, and the frames (time) of the second video in which the visitors, potential visitors, and past visitors were identified. Specifically, the identification unit 13 counts up the cumulative number of visitors, the number of visitors with a provisional stay, and the cumulative number of visitors attracted by identifying visitors. The identification unit 13 also counts up the cumulative number of people entering the flow path, the cumulative number of people exiting the flow path, the number of potential visitors with a provisional stay, or the cumulative number of potential visitors by identifying potential visitors. The identification unit 13 also counts up the cumulative number of past visitors by identifying past visitors.

[0072] In a preferred embodiment of the present invention, a predetermined threshold for the similarity of feature quantities is varied for each event. Specifically, the setting unit of the pedestrian flow management device 1 (not shown) varies and sets the predetermined threshold according to the determination rate at each position in the analysis target area of ​​the video allocation map generated in S1.

[0073] <1.4.5. Calculation of Human Flow Information> In S5, the calculation unit 14 calculates pedestrian flow information. In this embodiment, the calculation unit 14 calculates pedestrian flow information based on area pedestrian flow count information. Each piece of pedestrian flow information calculated by the calculation unit 14 is described below.

[0074] <1.4.5.1. Calculation of Net Stay Related Information> The stay-related data calculation unit 141 calculates net stay-related information when the stay of visitors (including potential visitors) in the area under analysis meets predetermined conditions. In this embodiment, the stay-related data calculation unit 141 calculates net stay-related information using a predetermined threshold for the number of people staying in the area as a predetermined condition. Specifically, the stay-related data calculation unit 141 calculates net stay-related information based on the time when the number of people staying in the area under analysis exceeds the threshold for the number of people staying, and the time when the number of people staying in the area under analysis falls below the threshold for the number of people staying.

[0075] For example, the stay-related data calculation unit 141 calculates the net stay time as net stay-related information, starting from the time when the number of visitors staying in the exhibition area exceeds a threshold (e.g., 1 person) and ending when the number of visitors staying in the exhibition area falls below the threshold. The stay-related data calculation unit 141 also calculates the number of times the net stay time occurs (cumulative net stay) as net stay-related information. Furthermore, the stay-related data calculation unit 141 calculates the average net stay time per person, which is the net stay time per person, by dividing the net stay time by the number of visitors who stayed during the net stay time. The net stay time, cumulative net stay time, and average net stay time per person for the front area are calculated in the same manner.

[0076] For example, the stay-related data calculation unit 141 calculates the number of times (turnover) when the number of visitors staying in the exhibition area falls below the stay threshold after the number of visitors staying in the exhibition area exceeds the stay threshold, as net stay-related information.

[0077] <1.4.5.2. Calculation of the number of players> The visitor count calculation unit 142 calculates the visitor count for the exhibition area based on the number of people who entered the front area and the number of people who entered the front area from the exhibition booths. In this embodiment, the visitor count calculation unit 142 calculates the visitor count as the cumulative number of potential visitors from the area's pedestrian flow count information.

[0078] <1.4.5.3. Calculation of Attraction Rate> The attraction rate calculation unit 143 calculates the attraction rate for the exhibition area based on the number of people who entered the exhibition area, the number of people who entered the front area, and the number of people who entered the front area from the exhibition booth. In this embodiment, the attraction rate calculation unit 143 calculates the attraction rate based on the number of visitors and the number of potential visitors. Specifically, the attraction rate calculation unit 143 calculates the attraction rate by dividing the cumulative number of attracted visitors from the area pedestrian flow count information by the cumulative number of potential visitors (number of visitors).

[0079] In this embodiment, the cumulative number of visitors is stored in the area pedestrian flow count information. Alternatively, the area pedestrian flow count information may store the increment in the number of visitors for each time-series identifier, and the calculation unit 14 may calculate the cumulative number of visitors as pedestrian flow information by accumulating the number of visitors.

[0080] <1.4.6. Identification of Human Flow Information> Furthermore, in S5, the attraction identification unit 15 identifies attraction information as pedestrian flow information. In this embodiment, the attraction identification unit 15 identifies the face angle or gaze direction of the same person in each frame based on the second image of each frame in a series of frames of the second video acquired from the second imaging device 4, and identifies the trajectory information of the same person based on multiple second images of the series of frames. Here, the face angle used is the angle with the x-axis as the axis of rotation in three-dimensional space (so-called pitch angle), the angle with the y-axis as the axis of rotation (so-called roll angle), and the angle with the z-axis as the axis of rotation (so-called yaw angle).

[0081] The attraction identification unit 15 then registers the attraction target that attracted the person to the exhibition area as attraction information, based on the face angle or gaze of the identified person and the person's trajectory information. In this embodiment, the attraction identification unit 15 determines the object of the person's view based on the face angle or gaze of the identified person, and registers the object of view as an attraction target if the trajectory information from the frame (second image) onward in which the face angle or gaze is directed towards the object of view overlaps with the exhibition area. Preferably, the attraction identification unit 15 targets potential visitors as the person for whom attraction information is registered.

[0082] Furthermore, in a preferred embodiment of the present invention, the attraction identification unit 15 registers a target for attraction based on the angle of a person's face or gaze as such when the person views the target for a predetermined period of time or longer and the trajectory information overlaps with the exhibition area.

[0083] Figure 7 is an example of a single frame display of a video shown on the user terminal device 5, which displays various information on the second video captured by the second imaging device 4. In Figure 7, the exhibition area VA, venue visitors VVI, and non-visitors NVI captured in the second video are displayed, and various information such as the front area FA and movement path FL, and the trajectory information TI of identified persons are displayed based on pre-set coordinate positions. In this embodiment, the display processing unit 17 displays the cumulative number of people who have passed through the front area FA and movement path FL according to the direction of their passage (for example, when a person passes into the front area FA, the number in parentheses of "IN(3)" in the illustrated example is counted up). The display processing unit 17 also assigns identifiers to and displays persons other than non-visitors identified by the identification unit 13.

[0084] <1.4.7. Updating Reference Information for Non-Visitors> In S6, the registration unit 12 determines whether the date of the acquired first video is different from the date of the first video acquired in S2. If it determines that the dates are the same (NO in S6), the process ends. On the other hand, if it determines that the dates are different (YES in S6), the registration unit 12 updates the non-visitor reference information (S7). In this embodiment, the registration unit 12 registers non-visitor characteristic information based on the first image of the non-visitor captured in the first video, based on the first video whose date is different from the first video acquired in S2, as non-visitor reference information.

[0085] In this embodiment, the registration unit 12 updates the non-visitor reference information by registering non-visitor characteristic information for each date of the first video, and the identification unit 13 identifies visitors by referring to the non-visitor reference information for each date. On the other hand, the registration unit 12 does not have to register non-visitor reference information for each date. In this case, the registration unit 12 may update the non-visitor reference information by deleting all past non-visitor reference information and registering only newly acquired non-visitor characteristic information. Alternatively, instead of registering non-visitor reference information for each date, the registration unit 12 may update the non-visitor reference information by setting a flag indicating that the person should not be identified as a visitor along with the registered non-visitor characteristic information.

[0086] As described above, by executing the processes S1 to S7 in Embodiment 1, it is possible to reliably manage event staff (non-attendees) whose appearance, clothing, and hairstyles change depending on the day and time, and to accurately grasp the number of attendees visiting the exhibition area. Furthermore, by imaging the front area, it is possible to grasp not only the number of attendees visiting the exhibition area, but also the number of potential attendees who may lead to those attendees.

[0087] In this embodiment, visitors are identified by using the first imaging device 3 and the second imaging device 4 to image the stockroom and the area to be analyzed, respectively. Alternatively, visitors may be identified by registering non-visitor reference information based on a second image in which decorative items worn by non-visitors to identify them are captured using only the second imaging device 4. Furthermore, visitors may be identified by registering a predetermined time when the event is held using only the second imaging device 4, and then registering non-visitor reference information based on a second image in which a person is captured before that predetermined time, thereby identifying the person as a non-visitor.

[0088] Furthermore, in this embodiment, based on the first image captured by the first imaging device 3 installed in a stockroom adjacent to each exhibition area, reference information for non-visitors is registered for each exhibition area of ​​a company or organization (hereinafter referred to as "company, etc."). On the other hand, if non-visitors from multiple exhibition areas share one area, and the first imaging device 3 is installed in that shared area, for example, the registration unit 12 may register reference information for each company, etc. based on a second image in which decorative items, etc., used to identify non-visitors from each company, etc., are captured.

[0089] As another example, database 2 may store company reference information that indicates which companies are using each section within the shared area, including images of the sections in the shared area and combinations of companies, and registration unit 12 may register non-visitor reference information for each company based on the first video, non-visitor trajectory information, and company reference information. Specifically, if the trajectory information of a non-visitor overlaps with a section of the shared area captured in the first video corresponding to the company reference information, registration unit 12 registers the non-visitor as a non-visitor of the company corresponding to that section and registers the non-visitor reference information for that company.

[0090] As another example, the registration unit 12 may register non-visitor reference information for each company, etc., corresponding to the company identification information, based on the company identification information (such as company name and logo) provided in the shared area section captured in the first video, and the trajectory information of non-visitors. Specifically, the registration unit 12 identifies the company identification information by performing image recognition on the first video, and when the trajectory information of a non-visitor overlaps with the section where the company identification information is provided, it registers the non-visitor as a non-visitor of the company, etc., corresponding to the company identification information, and registers the non-visitor reference information of the company, etc.

[0091] In this embodiment, the display process refers to the process by which the display processing unit 17 executes a process to generate information necessary for display, transmits the generated information to the terminal device 9, and the terminal device 9 displays the generated information. Alternatively, the display process may also refer to the process by which the display processing unit 17 transmits a processing command to the terminal device 9 to generate information necessary for display, causing the terminal device 9 to generate the information necessary for display and display the generated information. Furthermore, if the display processing unit 17 is provided in the terminal device 9 (in the case of a standalone type), the display process may refer to the process by which the display processing unit 17 executes a process to generate the necessary information, transmits the generated information to the output unit 95 of the terminal device 9, and the output unit 95 displays the generated information. [Explanation of symbols]

[0092] 0: Human flow management system 1:Person flow management device 2: Database 3: First imaging device 4: Second imaging device 5: User terminal device 100: Server 101: Processing Unit 102: Storage section 103: Communications Department 8: Imaging device 81: Processing Unit 82: Storage section 83: Communications Department 84: Imaging Department 9: Terminal device 91: Processing Unit 92: Storage section 93: Communications Department 94: Input section 95: Output section 11: Video Acquisition Unit 12: Registration Department 13: Specific part 14: Calculation Unit 141: Stay-related data calculation unit 142: Circulation Count Calculation Unit 143: Attraction Rate Calculation Unit 15: Attractant Identification Section 16: Map generation section 17: Display Processing Unit NW: Communication Network

Claims

1. A crowd flow management system that manages information regarding the flow of people at an exhibition, The aforementioned pedestrian flow management system comprises a video acquisition unit, a registration unit, a identification unit, and a calculation unit. The aforementioned video acquisition unit acquires a first video showing an area accessible only to non-visitors, and a second video showing the area to be analyzed. The registration unit registers the appearance image and / or characteristic features of the non-visitor in the database based on the first video. The identification unit identifies the visitor based on the visual appearance image and / or personal features of the non-visitor and the second video. The calculation unit calculates pedestrian flow information at the exhibition based on the identified visitors. A human flow management system.

2. The registration unit registers non-visitor characteristic information, including multiple images of the appearance of each non-visitor captured in the first video and / or individual feature quantities, based on the acquired first video, and updates non-visitor reference information for identifying the non-visitor on a daily basis based on the non-visitor characteristic information. The pedestrian flow management system according to claim 1.

3. The registration unit registers, each time the non-visitor is captured in the first video, the non-visitor characteristic information, including the non-visitor's physical appearance image and / or the non-visitor's characteristic quantities, based on the first video in which the non-visitor is captured. The pedestrian flow management system according to claim 1.

4. The registration unit registers the non-visitor characteristic information based on a newly captured full-body image of the same non-visitor if the non-visitor characteristic information based on a newly captured full-body image of the same non-visitor differs from the non-visitor characteristic information already registered based on a full-body image of the same non-visitor. The pedestrian flow management system according to claim 3.

5. The second video above is a video of the area to be analyzed at the exhibition, The area to be analyzed includes the exhibition area and a predetermined front area in front of the exhibition area. The aforementioned identification unit identifies potential visitors entering the front area based on the second video, The calculation unit calculates the number of visitors as the flow of people information based on the number of potential visitors. The pedestrian flow management system according to claim 1.

6. The second video above is a video of the area to be analyzed at the exhibition, The aforementioned analysis target area includes the exhibition area and the front area. The identifying unit identifies, based on the second video, visitors who are people entering the exhibition area from the front area, and potential visitors who are people entering the front area. The calculation unit calculates the attraction rate as the pedestrian flow information based on the number of visitors and potential visitors. The pedestrian flow management system according to claim 1.

7. The calculation unit calculates net stay-related information for the area under analysis as pedestrian flow information, based on a predetermined threshold for the number of people staying in the area under analysis at the exhibition. The pedestrian flow management system according to claim 1.

8. The calculation unit calculates the net stay-related information based on the time when the number of people in the area under analysis exceeds the threshold for the number of people, and the time when the number of people in the area under analysis falls below the threshold for the number of people. The pedestrian flow management system according to claim 7.

9. The aforementioned pedestrian flow management system further comprises an attraction and identification unit, The attraction identification unit identifies the facial angle or gaze of the person captured and the trajectory information of the person captured based on the second video, and identifies attraction information to the area to be analyzed at the exhibition based on the facial angle or gaze of the person and the trajectory information of the person. The registration unit registers the attraction information as the pedestrian flow information. The pedestrian flow management system according to claim 1.

10. The aforementioned pedestrian flow management system further comprises a map generation unit, The aforementioned video acquisition unit acquires video data for each of the multiple imaging devices. The map generation unit generates an individual determination map for each imaging device based on the respective video data, which associates the detection rate of the person being imaged with the location where the person was imaged. Based on the individual determination maps of the multiple imaging devices, it generates an imaging assignment map to identify the imaging area of ​​each imaging device. The pedestrian flow management system according to claim 1.

11. A crowd flow management program that manages information regarding the flow of people at an exhibition, The aforementioned pedestrian flow management program uses a computer to function as an image acquisition unit, a registration unit, an identification unit, and a calculation unit. The aforementioned video acquisition unit acquires a first video showing an area accessible only to non-visitors, and a second video showing the area to be analyzed. The registration unit registers the appearance image and / or characteristic features of the non-visitor in the database based on the first video. The identification unit identifies the visitor based on the visual appearance image and / or personal features of the non-visitor and the second video. The calculation unit calculates pedestrian flow information at the exhibition based on the identified visitors. A human flow management program.

12. A method for managing information regarding the flow of people at an exhibition, Computers The process involves acquiring a first video image capturing an area accessible only to non-visitors, and a second video image capturing the area to be analyzed. Based on the first video, the process involves registering the appearance image and / or characteristic features of the non-visitor in a database. A process to identify visitors based on the aforementioned visual appearance image and / or personal features of the non-visitor and the aforementioned second video, A process to calculate pedestrian flow information at the exhibition based on the identified visitors, A method for managing human traffic that implements this.

Images