People flow management system, people flow management program, and people flow management method

Abstract

To provide a people flow management system, a people flow management method, and a people flow management program that accurately manage people flow data. [Solution] A people flow management system that manages information regarding people flow at an exhibition, the people flow management device comprises an image acquisition unit, a registration unit, an identification unit, and a calculation unit, the image acquisition unit acquires a first image that captures an area that only non-visitors can enter and exit, and a second image that captures an area to be analyzed, the registration unit registers the appearance images and / or person features of non-visitors in a database based on the first image, the identification unit identifies visitors based on the appearance images and / or person features of non-visitors and the second image, and the calculation unit calculates people flow information at the exhibition based on the identified visitors.

Description

[Technical Field]

[0001] The present invention relates to a people flow management system, a people flow management program, and a people flow management method. [Background technology]

[0002] Conventionally, people's behavior has been analyzed by measuring people flow. In particular, an example of a system for measuring people flow at an event has been proposed in Patent Document 1, for example.

[0003] Patent document 1 discloses that visitor information is read from storage in order of earliest photographed time, and each time a piece of visitor information is read, it is identified whether the visitor indicated by that visitor information is a visitor who has already been detected in a previous process, and if there is no previously read visitor information that matches the comparison result, it is determined that the visitor is a first-time detected visitor, and in the case of a first-time detected visitor, the attributes or status of the visitor to be identified can be set, and the features F1 to FN contained in the visitor information of the first-time detected visitor are compared with the features indicating the attributes or status set as the target to be counted, and based on the comparison result, it is identified whether the person from whom the visitor information was extracted is a target to be counted, and if the comparison result matches, a specified counter is counted up and the visitor information is stored in memory as previously read visitor information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-102342 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem with Patent Document 1 is that it is not possible to distinguish between visitors and event-related personnel (non-visitors) and manage the flow of people.

[0006] In view of the above circumstances, an object of the present invention is to provide a new technology that enables accurate management of people flow data. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a people flow management system that manages information regarding people flow at an exhibition, the people flow management system comprising an image acquisition unit, a registration unit, an identification unit, and a calculation unit, the image acquisition unit acquires a first image that captures an area that only non-visitors can enter and exit, and a second image that captures an area to be analyzed, the registration unit registers the appearance images and / or person features of the non-visitors in a database based on the first image, the identification unit identifies visitors based on the appearance images and / or person features of the non-visitors and the second image, and the calculation unit calculates people flow information at the exhibition based on the identified visitors.

[0008] In addition, in order to solve the above-mentioned problems, the present invention provides a people flow management program that manages information related to people flow at an exhibition, wherein the people flow management program causes a computer to function as an image acquisition unit, a registration unit, an identification unit, and a calculation unit, wherein the image acquisition unit acquires a first image that captures an area that only non-visitors can enter and exit, and a second image that captures an area to be analyzed, the registration unit registers the appearance images and / or person features of the non-visitors in a database based on the first image, the identification unit identifies visitors based on the appearance images and / or person features of the non-visitors and the second image, and the calculation unit calculates people flow information at the exhibition based on the identified visitors.

[0009] In addition, in order to solve the above-mentioned problems, the present invention is a people flow management method for managing information regarding people flow at an exhibition, in which a computer performs the following processes: acquiring a first video of an area that only non-visitors can enter and exit, and a second video of an area to be analyzed; registering appearance images and / or person features of the non-visitors in a database based on the first video; identifying visitors based on the appearance images and / or person features of the non-visitors and the second video; and calculating people flow information at the exhibition based on the identified visitors.

[0010] With this configuration, non-attendees can be reliably identified, attendees can be reliably identified, and people flow information at the exhibition can be accurately calculated.

[0011] In a preferred embodiment of the present invention, the registration unit registers, based on the acquired first video, the person appearance images and / or person characteristics of each of the multiple non-visitors captured in the first video as non-visitor characteristic information, and updates non-visitor reference information for identifying the non-visitors on a daily basis based on the non-visitor characteristic information.

[0012] Depending on the exhibition, the people who are considered non-visitors may differ depending on the day. By configuring in this way, it is possible to deal with cases where the non-visitors change depending on the day, and accurate people flow information can be calculated.

[0013] In a preferred embodiment of the present invention, each time a non-attendee is captured in the first video, the registration unit registers an image of the non-attendee's appearance and / or non-attendee characteristic information including the non-attendee's characteristic amounts based on the first video in which the non-attendee is captured.

[0014] In a preferred form of the present invention, if the non-visitor characteristic information based on a newly captured full-body image of a non-visitor differs from non-visitor characteristic information based on an already registered full-body image of the same non-visitor, the registration unit registers the non-visitor characteristic information based on the newly captured full-body image of the same non-visitor.

[0015] With this configuration, even if the same person changes their clothes or hairstyle, they can be reliably identified as a non-attendee.

[0016] In a preferred embodiment of the present invention, the second image is an image further capturing an area to be analyzed at an exhibition, the area to be analyzed including the exhibit area and a predetermined front area in front of the exhibit area, the identification unit identifies potential visitors entering the front area based on the second image, and the calculation unit calculates the number of people circulating as the people flow information based on the number of potential visitors.

[0017] In a preferred form of the present invention, the second image is an image further capturing an area to be analyzed at an exhibition, the area to be analyzed including an exhibition area and a front area, and the identification unit identifies visitors who are people entering the exhibition area from the front area and potential visitors who are people entering the front area based on the second image, and the calculation unit calculates an attraction rate as the people flow information based on the number of visitors and potential visitors.

[0018] With this configuration, it is possible to identify people passing through not only the exhibition area but also the front area, thereby improving the accuracy of analyzing people flow in the exhibition area.

[0019] In a preferred embodiment of the present invention, the calculation unit calculates net stay-related information for the analysis target area as the people flow information based on a predetermined threshold value for the number of visitors in the analysis target area at the exhibition.

[0020] In a preferred embodiment of the present invention, the calculation unit calculates the net stay-related information based on the time when the number of visitors in the analysis target area exceeded the visitor number threshold and the time when the number of visitors in the analysis target area fell below the visitor number threshold.

[0021] By adopting this configuration, even if there is a person in the area being analyzed who cannot be identified as a non-visitor, it is possible to prevent inaccurate stay-related information from being registered by that person.

[0022] In a preferred embodiment of the present invention, the people flow management system further includes an attraction identification unit that identifies the facial angle or line of sight of the person captured in the image and trajectory information of the person captured in the image based on the second video, and identifies attraction information to an area of the exhibition to be analyzed based on the facial angle or line of sight of the person and the trajectory information of the person, and the registration unit registers the attraction information as the people flow information.

[0023] With this configuration, it is possible to understand how a person was attracted to the area to be analyzed based on the facial angle and trajectory of the captured image of the person.

[0024] In a preferred form of the present invention, the people flow management system further includes a map generation unit, wherein the image acquisition unit acquires image data for each of a plurality of imaging devices, and the map generation unit generates, for each imaging device based on the image data, an individual judgment map that corresponds the judgment rate of the person being imaged with the position at which the person was imaged, and generates an imaging allocation map for identifying the imaging area of each imaging device based on the individual judgment maps of the plurality of imaging devices.

[0025] With this configuration, it is possible to optimize the identification of people using a plurality of imaging devices, thereby achieving accurate person identification and calculating accurate people flow information. [Effects of the Invention]

[0026] The present invention can provide a novel technique that enables accurate management of people flow data. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a block diagram showing a configuration of a system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram of a hardware configuration of a system according to the present invention. [Figure 3] FIG. 2 is a block diagram of a functional configuration according to an embodiment of the present invention. [Figure 4] FIG. 1 is an image diagram of people flow measurement in one embodiment of the present invention. [Figure 5] 1 is an example of a processing flowchart according to an embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing an example of an imaging allocation map in the present invention. [Figure 7] 10 is an example of a screen displayed on a user terminal device according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0028] (Embodiment 1) The present invention will be described in more detail below with reference to the accompanying drawings, in which preferred embodiments are shown, but which may be embodied in many different forms and are not limited to the embodiments set forth herein.

[0029] For example, while the configuration, operation, etc. of a people flow management system are described in this embodiment, similar effects can be achieved by the executed method, device, computer program, etc. Furthermore, the program may be stored on a recording medium. Using this recording medium, the program can be installed on a computer, for example, thereby configuring a people flow management device and a people flow management system. Here, the recording medium storing the program may be a non-transitory recording medium, such as a CD-ROM.

[0030] <1. Overview of First Embodiment> The present invention relates to a system for managing information about people flow at an event. In this invention, an image capturing device (hereinafter referred to as a second image capturing device) is installed at an event venue to capture images of an analysis target area and the surrounding area of the analysis target area, which are targets for people flow analysis, and an image capturing device (hereinafter referred to as a first image capturing device) to capture images of a non-attendee recording area, which is accessible only to people not targeted for people flow analysis (e.g., event personnel and staff, hereinafter referred to as non-attendees). In this embodiment, based on an image captured by the first image capturing device and including at least non-attendees (hereinafter referred to as a first image) and an image captured by the second image capturing device and including the analysis target area and the surrounding area of the analysis target area (hereinafter referred to as a second image), people who enter the analysis target area and are targets for people flow analysis (hereinafter referred to as attendees) are identified, and people flow information about the analysis target area is calculated.

[0031] In this embodiment, the area of the event's exhibition booths (hereinafter referred to as the exhibition area), a predetermined area in front of the exhibition area (hereinafter referred to as the front area), and the flow lines to the exhibition area are used as the areas to be analyzed. Based on the first and second images, visitors who have entered the exhibition area, people who are leaving the exhibition area (hereinafter referred to as existing visitors), and people who have entered the front area and are likely to enter the exhibition area (hereinafter referred to as potential visitors) are identified, and people flow information for the area to be analyzed is calculated.

[0032] In this embodiment, video is used as the video, but still images may also be used. The area to be analyzed is not limited to the exhibit area, and may be any area where visitors come and go. While the non-visitor recording area will be described using a stockroom attached to each exhibit area, it may be any area where only non-visitors enter and exit, and may be, for example, an area used by non-visitors of multiple exhibit areas. The front area is not limited to the area in front of the exhibit area in one direction, but includes areas in front of the exhibit area in all directions. Although the event will be described as an exhibition, it may also be a short-term event or a permanent event, and is not limited to this.

[0033] <1.1. System configuration of embodiment 1> Fig. 1 is a block diagram showing the configuration of a system according to embodiment 1. As shown in Fig. 1, a people flow management system 0 includes a people flow management device 1, a first imaging device 3, a second imaging device 4, and a user terminal device 5, and is configured to be able to communicate via a communication network NW. The communication network NW in the present invention is an IP (Internet Protocol) network, but there are no restrictions on the type of communication protocol, and there are also no restrictions on the type or scale of the network.

[0034] A general-purpose server computer or a personal computer can be used as the people flow management device 1. Furthermore, a general-purpose video camera such as a network camera capable of recording video can be used as the first imaging device 3 and the second imaging device 4. Furthermore, a smartphone, tablet terminal, personal computer, wearable device, etc. can be used as the user terminal device 5. Furthermore, the people flow management device 1 may be configured with multiple computers that are capable of sending and receiving information via a communication network NW or another network.

[0035] <1.2. Hardware configuration of the present invention> 2 is a block diagram of the hardware configuration of the people flow management system 0. As shown in FIG. 2(a), the server 10 (people flow management device 1) includes a processing unit 101, a storage unit 102, and a communication unit 103.

[0036] The processing unit 101 has one or more processors, such as a CPU, that can execute an instruction set, and controls the overall operation and processing of the people flow management device 1 by executing the people flow management program, OS, and other applications related to the present invention. The storage unit 102 has a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording an OS and the people flow management program according to the present invention. The communication unit 103 has a communication interface device with the communication network NW, and controls communication with the communication network NW to input and output information.

[0037] As shown in FIG. 2(b), the imaging device 8 (first imaging device 3 and second imaging device 4) includes a processing unit 81 that controls the operation of the imaging device 8, a memory unit 82 that stores the captured image, a communication unit 83 that communicates with the server 10, and an imaging unit 84 that captures the image.

[0038] As shown in FIG. 2( c ), the terminal device 9 (user terminal device 5 ) includes a processing unit 91 , a storage unit 92 , a communication unit 93 , an input unit 94 , and an output unit 95 .

[0039] The processing unit 91 has one or more processors such as a CPU that can execute an instruction set, and controls the overall operation and processing of the terminal device 9 by executing an OS and other applications. The storage unit 92 has a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording an OS and the like. The communication unit 93 has a communication interface device for connecting to a network, and controls communication with the communication network NW to input and output information. The input unit 94 has an input device capable of input processing, such as a keyboard or a touch panel. The output unit 95 has a display device capable of display processing, such as a display.

[0040] <1.3. System Functional Configuration> Fig. 3 is a block diagram of the functional configuration of the people flow management device 1. As shown in Fig. 3, the people flow management device 1 includes an image acquisition unit 11, a registration unit 12, an identification unit 13, a calculation unit 14, an attraction identification unit 15, a map generation unit 16, a display processing unit 17, and a database 2. This is information processing by software (stored in a storage unit 102) specifically realized by hardware (processing unit 101).

[0041] The system configuration in this embodiment is a so-called server-client type, in which the user terminal device 5 (client) receives the processing results performed by the people flow management device 1 (server) in response to a request from the client. Alternatively, it may be a so-called standalone type, in which a people flow management program is launched on the client terminal. In this case, the user terminal device 5 may include some or all of the functional components (units) of the people flow management device 1. For example, the user terminal device 5 may include an image acquisition unit 11, a registration unit 12, an identification unit 13, a calculation unit 14, an attraction identification unit 15, a map generation unit 16, and a display processing unit 17, and the people flow management device 1 may be a cloud storage that stores video data captured by the imaging device 8.

[0042] <1.3.1.Database 2> The database 2 stores analysis target video information, non-visitor reference information, and area people flow count information.

[0043] <1.3.1.1. Video information to be analyzed> The video information to be analyzed is information relating to the first video and the second video to be analyzed received from the imaging device 8. The video information to be analyzed is set with a video ID, imaging device ID, and date to uniquely identify the video, and is stored for each event identification information (event name, etc.) and exhibition area identification information (exhibiting company name, etc.).

[0044] <1.3.1.2. Reference Information for Non-Visitors> The non-visitor reference information is information that is referenced to identify people who should be considered non-visitors. As the non-visitor reference information, non-visitor characteristic information is set for each day. As the non-visitor characteristic information, a non-visitor ID for uniquely identifying a non-visitor, an external image of the non-visitor, and a feature amount (multidimensional vector representation) of the non-visitor are registered. In this embodiment, the non-visitor characteristic information is set by assigning one non-visitor ID to each external image and / or feature amount of a non-visitor.

[0045] <1.3.1.3. Area People Flow Count Information> The area people flow count information is information used to calculate people flow information in the area to be analyzed. The area people flow count information includes a time series identifier, exhibition area count information related to people flow in the exhibition area, flow line count information related to people flow along the flow line to the exhibition area, and front area count information related to people flow in the front area. In this embodiment, video frames are used as the time series identifier, but the shooting time may also be used.

[0046] The exhibition area count information includes the cumulative number of visitors who have entered the exhibition area, and the provisional number of visitors who are staying in the exhibition area.

[0047] The flow line count information includes the total number of people who entered the flow line to the exhibition area, that is, the total number of people who left the flow line to the exhibition area, and the total number of people who left the flow line to the exhibition area.

[0048] The front area count information includes the cumulative number of existing visitors, which is the cumulative number of existing visitors who have entered the front area from the exhibition area; the provisional number of potential visitors, which is the provisional number of potential visitors staying in the front area; the cumulative number of attracted visitors, which is the cumulative number of visitors entering the exhibition area from the front area; and the cumulative number of potential visitors, which is the cumulative number of potential visitors who have entered the front area from other than the exhibition area.

[0049] FIG. 4 is an image diagram related to counting of front area count information. The cumulative number of existing visitors is the cumulative number of people moving in the in(1) direction relative to the front area in FIG. 4. The cumulative number of attracting visitors is the cumulative number of people moving in the out direction relative to the front area in FIG. 4. The cumulative number of potential visitors is the cumulative number of people moving in the in(1), in(2), and in(3) directions relative to the front area in FIG. 4.

[0050] In a preferred embodiment of the present invention, an individual analysis target area is set within the exhibition area, and individual count information for the individual analysis target area is set as the area people flow count information. Specifically, the individual count information includes the cumulative individual visitor count, which is the cumulative number of visitors who entered the individual analysis target area (for example, an area within a predetermined range in front of the exhibition board), and the provisional individual visitor count, which is the provisional number of visitors staying in the individual analysis target area.

[0051] <1.3.2. Video Acquisition Unit 11> The video acquisition unit 11 acquires the first video and the second video. The video acquisition unit 11 acquires the first video, which captures only non-attendees, and the second video, which captures at least the exhibit area and the front area and captures people coming and going through the event venue, and registers them in the database 2 as video information to be analyzed.

[0052] <1.3.3. Registration Section 12> The registration unit 12 registers the non-attendee characteristic information. The registration unit 12 registers the non-attendee characteristic information including the appearance images and / or person characteristic amounts of the non-attendees in the database 2 based on the first video acquired by the video acquisition unit 11.

[0053] The registration unit 12 also updates the non-attendee reference information. Based on the first video captured in a day, the registration unit 12 registers non-attendee characteristic information for each of the multiple non-attendees captured in the first video, and updates the non-attendee reference information daily.

[0054] <1.3.4. Specific part 13> The identification unit 13 identifies people entering and leaving the exhibition area and the front area. The identification unit 13 identifies visitors, potential visitors, and existing visitors based on the non-visitor reference information and the second video.

[0055] <1.3.5. Calculation Unit 14> The calculation unit 14 calculates people flow information related to the exhibition area. The calculation unit 14 includes a stay-related data calculation unit 141 that calculates net stay-related information related to the stay of visitors in the area to be analyzed as people flow information, a migration number calculation unit 142 that calculates the number of migrations to the exhibition area as people flow information, and an attraction rate calculation unit 143 that calculates the attraction rate to the exhibition area as people flow information. Details of the processing of each calculation unit will be described later.

[0056] <1.3.6. Incentive Identification Unit 15> The attraction identification unit 15 identifies the facial angle of the person and the trajectory information of the person based on the second video. The attraction identification unit 15 identifies the facial angle of the captured person and the trajectory information of the person based on the second image of the series of frames of the second video.

[0057] The attraction identification unit 15 also registers attraction information for visitors to the exhibition area. The attraction identification unit 15 registers, as people flow information, attraction information indicating that a person has been attracted to the exhibition area based on the face angle of the identified person and the trajectory information of the person.

[0058] <1.3.7. Map Generation Unit 16> The map generation unit 16 generates an imaging allocation map for specifying the imaging areas of the multiple second imaging devices 4. Based on the video data acquired for each of the multiple second imaging devices 4, the map generation unit 16 generates an individual judgment map for each second imaging device 4 in which the judgment rate of the person being imaged and the position where the person was imaged correspond to each other, and generates an imaging allocation map for specifying the imaging area of each camera based on the individual judgment map for each second imaging device 4 and the acquired imaging time.

[0059] <1.3.8. Display Processing Unit 17> The display processing unit 17 performs display processing of an image in which various pieces of information are superimposed on an image of the area to be analyzed, and causes the display processing result to be displayed on the user terminal device 5. An example of the display of the image will be described later.

[0060] <1.4. Processing Flowchart> A people flow management method using the people flow management system 0 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the process in which the people flow management device 1 optimizes the imaging areas of the multiple second imaging devices 4, calculates people flow information using video data (first video and second video) acquired from the imaging devices 8 for one day, and updates the non-visitor reference information as the date is updated. In the flowchart shown in Fig. 5, the process ends with the non-visitor reference information being updated, but the processes of S4 to S7 are executed for the next day's people flow information using the updated non-visitor reference information.

[0061] <1.4.1. Generation of imaging allocation map> First, in step S1 (hereinafter, "step SX" will be abbreviated simply as "SX"), the map generation unit 16 generates an imaging allocation map. In this embodiment, the video acquisition unit 11 acquires second video, which includes a combination of imaging times and second images including at least the exhibit area and the front area, for each of the second imaging devices 4 arranged for each event.

[0062] The map generation unit 16 generates an individual determination map for each second imaging device 4, in which the determination rate of the imaged person corresponds to the position where the person was imaged, based on the second video captured by each second imaging device 4 and the person feature amounts registered in advance. Then, the map generation unit 16 synchronizes the position and determination rate of the imaged person using the imaging time acquired for the generated individual determination map for each second imaging device 4, and for one position, identifies the second imaging device 4 among the multiple second imaging devices 4 that has the highest determination rate for the person at that position, and assigns that position as the imaging area of that second imaging device 4. Then, the map generation unit 16 repeats the above process for multiple positions in the imaged area, and generates an imaging allocation map.

[0063] FIG. 6 is a diagram showing an example (a) of a method for creating an imaging allocation map, and an example (b) of the created imaging allocation map.

[0064] 6(a) is an example of an individual determination map generated based on the second video captured by the second imaging device 4(B). In the illustrated example, a person whose person features have been registered in advance walks comprehensively through the analysis target area, and the second imaging device 4(B) captures the person's image. The individual determination map is generated by associating the person's determination rate and position based on the captured second video (the darker the black, the lower the determination rate). Then, the individual determination maps of the second imaging devices 4(A, B, C) are combined to generate an imaging allocation map that optimizes the person's determination rate (FIG. 6(b)).

[0065] When the processing in S1 is completed, the following specific calculation processing of people flow information for the area to be analyzed is executed.

[0066] <1.4.2. Acquiring video data> In S2, the video acquisition unit 11 acquires the first video and the second video. In this embodiment, the video acquisition unit 11 receives a specification of the event and exhibition area for which people flow information is to be calculated, and acquires the first video and the second video corresponding to the event and exhibition area from the database 2. Note that the video acquisition unit 11 may directly acquire the first video from the first imaging device 3 and the second video from the second imaging device 4.

[0067] <1.4.3. Registration of non-visitor reference information> In S3, the registration unit 12 registers non-attendee reference information based on the first video. In this embodiment, the registration unit 12 inputs each frame of the first video (first image) into a known image recognition model to calculate feature amounts of the full-body image of the non-attendee captured in the first image, and registers the full-body image of the non-attendee as a human appearance image, and non-attendee reference information including the feature amounts of the human appearance image and the full-body image, as well as the date the image was captured. Note that features based on a facial image may be used as the feature amounts of the non-attendee instead of a full-body image.

[0068] Furthermore, in this embodiment, each time a non-visitor is captured in the first video, the registration unit 12 registers non-visitor characteristic information based on the first video in which the non-visitor is captured. Specifically, if the visitor characteristic information (the person's appearance image of the full-body image and / or the feature amount of the full-body image) based on a newly captured full-body image of a non-visitor differs from visitor characteristic information of the same non-visitor that has already been registered, the registration unit 12 registers the non-visitor characteristic information based on the newly captured full-body image of the same non-visitor.

[0069] In a preferred embodiment of the present invention, if the uniform worn by a non-attendee has a distinctive feature (such as a logo or company name), the registration unit 12 calculates the feature amount by weighting the distinctive feature of the entire body, and registers non-attendee reference information.

[0070] <1.4.4. Identification of Visitors> In S4, the identification unit 13 identifies visitors, potential visitors, and past visitors based on the non-visitor reference information and the second video. In this embodiment, the identification unit 13 inputs the second video acquired in S2 into a known image recognition model to identify people captured in the second video. The identification unit 13 then compares the appearance image or feature values of the identified people with the appearance image or feature values of the non-visitor reference information registered in S3, and identifies people whose appearance image or feature values of the identified people and the appearance image or feature values of the non-visitor reference information have a similarity below a predetermined threshold as visitors, potential visitors, and past visitors. Specifically, the position of a front area is set for each exhibition area, and the identification unit 13 identifies visitors, potential visitors, and past visitors based on the appearance image or feature values of people entering and leaving the exhibition area and the front area associated with that exhibition area.

[0071] The identification unit 13 then registers area people flow count information in the database 2 based on the identified visitors, potential visitors, and existing visitors and the frames (times) of the second video in which the visitors, potential visitors, and existing visitors were identified. Specifically, the identification unit 13 counts up the cumulative number of visitors, the number of visitors temporarily staying, and the cumulative number of attracted visitors by identifying visitors. The identification unit 13 also counts up the cumulative number of people entering the flow line, the cumulative number of people leaving the flow line, the number of potential visitors temporarily staying, or the cumulative number of potential visitors by identifying potential visitors. The identification unit 13 also counts up the cumulative number of existing visitors by identifying existing visitors.

[0072] In a preferred embodiment of the present invention, a predetermined threshold value related to the similarity of feature values is varied for each event. Specifically, a setting unit of the people flow management device 1 (not shown) varies and sets the predetermined threshold value according to the determination rate at each position in the analysis target area of the video allocation map generated in S1.

[0073] <1.4.5. Calculating people flow information> In S5, the calculation unit 14 calculates people flow information. In this embodiment, the calculation unit 14 calculates people flow information based on area people flow count information. Each piece of people flow information calculated by the calculation unit 14 will be described below.

[0074] <1.4.5.1. Calculation of Net Stay-Related Information> The stay related data calculation unit 141 calculates net stay related information when the stay of visitors (including potential visitors) in the analysis target area satisfies a predetermined condition. In this embodiment, the stay related data calculation unit 141 calculates the net stay related information using a predetermined visitor number threshold related to the number of visitors in the target area as the predetermined condition. Specifically, the stay related data calculation unit 141 calculates the net stay related information based on the time when the number of visitors in the analysis target area exceeded the visitor number threshold and the time when the number of visitors in the analysis target area fell below the visitor number threshold.

[0075] For example, the stay-related data calculation unit 141 calculates the period (net stay time) from the time when the number of visitors staying in the exhibition area exceeded a stay number threshold (for example, 1 person) to the time when the number of visitors staying in the exhibition area fell below the stay number threshold as net stay-related information. The stay-related data calculation unit 141 also calculates the number of times the net stay time occurs (cumulative net stays) as net stay-related information. The stay-related data calculation unit 141 also calculates the average net stay time per person, which is the net stay time per person, by dividing the net stay time by the number of visitors who stayed within the net stay time. The net stay time, cumulative net stays, and average net stay time per person for the front area are also calculated in a similar manner.

[0076] For example, the stay-related data calculation unit 141 calculates as net stay-related information the number of times (turnover) that the number of visitors staying in the exhibition area exceeds the visitor number threshold and then falls below the visitor number threshold.

[0077] <1.4.5.2. Calculating the number of visits> The migration number calculation unit 142 calculates the number of migrations for the exhibition area based on the number of people who entered the front area and the number of people who entered the front area from the exhibition booths. In this embodiment, the migration number calculation unit 142 calculates the cumulative number of potential visitors from the area people flow count information as the migration number.

[0078] <1.4.5.3. Calculation of attraction rate> The attraction rate calculation unit 143 calculates an attraction rate for the exhibition area based on the number of people who entered the exhibition area, the number of people who entered the front area, and the number of people who entered the front area from the exhibition booth. In this embodiment, the attraction rate calculation unit 143 calculates the attraction rate based on the number of visitors and the number of potential visitors. Specifically, the attraction rate calculation unit 143 calculates the attraction rate by dividing the cumulative number of attracted visitors in the area people flow count information by the cumulative number of potential visitors (number of visits).

[0079] In this embodiment, the cumulative number of visitors is stored in the area people flow count information, but the area people flow count information may also store the increment in the number of visitors for each time series identifier, and the calculation unit 14 may add up the number of visitors to calculate the cumulative number of visitors as people flow information.

[0080] <1.4.6. Identifying People Flow Information> In addition, in S5, the attraction identification unit 15 identifies attraction information as people flow information. In this embodiment, the attraction identification unit 15 identifies the facial angle or line of sight of the same person in each frame based on the second image of each frame in the series of frames of the second video acquired from the second imaging device 4, and identifies trajectory information of the same person based on the second images of the series of frames. Here, the facial angles used are the angle around the x-axis of the three-dimensional space as the rotation axis (so-called pitch angle), the angle around the y-axis as the rotation axis (so-called roll angle), and the angle around the z-axis as the rotation axis (so-called yaw angle).

[0081] Then, the attraction identification unit 15 registers the attraction target that attracted the person to the exhibition area as attraction information based on the face angle or line of sight of the identified person and the person's trajectory information. In this embodiment, the attraction identification unit 15 determines the gaze target of the identified person based on the face angle or line of sight of the person, and registers the gaze target as an attraction target when trajectory information from the frame (second image) onwards in which the face angle or line of sight is directed towards the gaze target overlaps with the exhibition area. Preferably, the attraction identification unit 15 targets potential visitors as people for whom attraction information is to be registered.

[0082] In addition, in a preferred embodiment of the present invention, the attraction identification unit 15 registers a visual target based on a person's facial angle or line of sight as an attraction target when the person looks at the visual target for a predetermined period of time or longer and the trajectory information overlaps with the exhibition area.

[0083] FIG. 7 is a display example of one frame of a video displayed on the user terminal device 5, displaying various information in the second video captured by the second imaging device 4. In FIG. 7, the exhibition area VA, venue visitors VVI, and non-visitors NVI captured in the second video are displayed, and various information, such as the front area FA, flow lines FL, and trajectory information TI of identified people, are displayed based on preset coordinate positions. In this embodiment, the display processing unit 17 displays the cumulative number of people who have passed through the front area FA and flow lines FL according to the direction of their passage each time a person passes through them (for example, when a person passes so as to enter the front area FA, the number in parentheses of "IN(3)" in the illustrated example is counted up). The display processing unit 17 also assigns identifiers to people other than non-visitors identified by the identification unit 13 and displays them.

[0084] <1.4.7. Updating non-visitor reference information> In S6, the registration unit 12 determines whether the date of the acquired first video is different from the date of the first video acquired in S2. If it is determined that the dates are the same (NO in S6), the processing ends. On the other hand, if it is determined that the dates are different (YES in S6), the registration unit 12 updates the non-attendee reference information (S7). In this embodiment, the registration unit 12 registers non-attendee characteristic information based on the first image of the non-attendee captured in the first video as non-attendee reference information, based on the first video whose date is different from the first video acquired in S2.

[0085] In this embodiment, the registration unit 12 registers non-visitor characteristic information for each date in the first video and updates the non-visitor reference information, allowing the identification unit 13 to identify visitors, etc. by referring to the non-visitor reference information for each date. On the other hand, the registration unit 12 does not have to register non-visitor reference information for each date. In this case, the registration unit 12 may update the non-visitor reference information by deleting all past non-visitor reference information and registering only newly acquired non-visitor characteristic information. Furthermore, instead of registering non-visitor reference information for each date, the registration unit 12 may update the non-visitor reference information by setting a flag indicating that the non-visitor is not identified as a visitor along with the registered non-visitor characteristic information.

[0086] As described above, by executing the processes of S1 to S7 in the first embodiment, it is possible to reliably manage event staff (non-attendees) whose appearance, clothing, and hairstyle change depending on the day and time, and to accurately grasp the number of visitors who will be visiting the exhibition area. Furthermore, by capturing images of the front area, it is possible to grasp not only the number of visitors who will be visiting the exhibition area, but also the number of potential visitors who will be connected to the visitors.

[0087] In this embodiment, attendees are identified by using the first imaging device 3 and the second imaging device 4 to capture images of the stockroom and the area to be analyzed, respectively. Alternatively, attendees may be identified by registering non-attendee reference information based on a second image captured by the second imaging device 4 alone, in which accessories worn by non-attendees for identifying non-attendees are captured. Alternatively, attendees may be identified by registering a predetermined time when an event is held using the second imaging device 4 alone, and registering non-attendee reference information for a person as a non-attendee based on a second image captured of the person before the predetermined time.

[0088] Furthermore, in this embodiment, non-visitor reference information of non-visitors is registered for each exhibition area of a company or organization (hereinafter referred to as a company, etc.) based on a first image captured by a first imaging device 3 installed in a stockroom attached to each exhibition area. On the other hand, if non-visitors from multiple exhibition areas share one area and a first imaging device 3 is installed in the shared area, for example, the registration unit 12 may register non-visitor reference information for each company, etc. based on a second image captured of accessories, etc. for identifying non-visitors for each company, etc.

[0089] As another example, database 2 may store company reference information indicating which companies etc. are using each section in the shared area, including a combination of images of the sections of the shared area and companies etc., and registration unit 12 may register non-visitor reference information for each company etc. based on the first video, non-visitor trajectory information, and company reference information. Specifically, when the trajectory information of a non-visitor overlaps with a section of the shared area captured in the first video corresponding to the company reference information, registration unit 12 registers the non-visitor reference information of the company etc. as a non-visitor of the company etc. corresponding to the section.

[0090] As another example, the registration unit 12 may register non-visitor reference information for each company, etc. corresponding to the company identification information (company name, logo, etc.) provided in a section of the common area captured in the first video, based on the company identification information and the trajectory information of the non-visitor. Specifically, the registration unit 12 identifies the company identification information by performing image recognition on the first video, and when the trajectory information of the non-visitor overlaps with the section in which the company identification information is provided, the registration unit 12 registers the non-visitor reference information of the company, etc. as a non-visitor of the company, etc. corresponding to the company identification information.

[0091] Furthermore, in this embodiment, the display processing refers to a process in which the display processing unit 17 executes a process of generating information necessary for display, and transmits the generated information to the terminal device 9, thereby causing the terminal device 9 to display the generated information. On the other hand, the display processing may also be a process in which the display processing unit 17 transmits a processing command to the terminal device 9 to generate information necessary for display, thereby causing the terminal device 9 to generate information necessary for display, and display the generated information. Furthermore, in the case where the display processing unit 17 is provided in the terminal device 9 (in the case of a stand-alone type), the display processing may also be a process in which the display processing unit 17 executes a process of generating necessary information, and transmits the generated information to the output unit 95 of the terminal device 9, thereby causing the output unit 95 to display the generated information. [Explanation of symbols]

[0092] 0: People flow management system 1:Person flow management device 2: Database 3: First imaging device 4: Second imaging device 5: User terminal device 100: Server 101: Processing section 102: Storage section 103: Communications Department 8: Imaging device 81: Processing section 82: Storage section 83: Communications Department 84: Imaging unit 9: Terminal device 91: Processing section 92: Storage section 93: Communications Department 94: Input section 95: Output section 11: Video acquisition unit 12: Registration section 13: Specific part 14: Calculation section 141: Stay-related data calculation unit 142: Number of visits calculation unit 143: Attraction rate calculation unit 15: Attractive Identification Section 16: Map generation section 17: Display processing section NW: Communication network

Claims

1. A people flow management system for managing information about people flow at an exhibition, The people flow management system includes an image acquisition unit, a registration unit, an identification unit, and a calculation unit, The video acquisition unit acquires a first video of an area that only non-attendees can enter and exit, and a second video of an area to be analyzed, the registration unit registers, in a database, a person appearance image and / or a person feature amount of the non-attendee based on the first video; the identification unit identifies the attendee based on the person appearance images and / or person feature amounts of the non-attendees and the second video; The calculation unit calculates people flow information at the exhibition based on the identified visitors. People flow management system.

2. The registration unit registers non-visitor characteristic information including a person appearance image and / or a person characteristic amount for each of the plurality of non-visitors captured in the first video based on the acquired first video, and updates non-visitor reference information for identifying the non-visitors daily based on the non-visitor characteristic information. The people flow management system according to claim 1 .

3. The registration unit registers, every time the non-attendee is captured in the first video, an appearance image of the non-attendee and / or non-attendee characteristic information including a characteristic amount of the non-attendee based on the first video in which the non-attendee is captured. The people flow management system according to claim 1 .

4. When the non-visitor characteristic information based on a newly captured full-body image of a non-visitor differs from non-visitor characteristic information based on a previously registered full-body image of the same non-visitor, the registration unit registers the non-visitor characteristic information based on the newly captured full-body image of the same non-visitor. The people flow management system according to claim 3.

5. The second image is an image further capturing an area to be analyzed in the exhibition, The analysis target area includes an exhibition area and a predetermined front area in front of the exhibition area, the identification unit identifies a potential visitor entering the front area based on the second video; The calculation unit calculates a number of people moving around as the people flow information based on the number of potential visitors. The people flow management system according to claim 1 .

6. The second image is an image further capturing an area to be analyzed in the exhibition, The analysis target area includes an exhibition area and a front area, the identification unit identifies, based on the second video, a visitor who is a person entering the exhibition area from the front area and a potential visitor who is a person entering the front area; The calculation unit calculates an attraction rate as the people flow information based on the number of visitors and the number of potential visitors. The people flow management system according to claim 1 .

7. The calculation unit calculates net stay-related information of the analysis target area as the people flow information based on a predetermined visitor number threshold related to the number of visitors in the analysis target area at the exhibition. The people flow management system according to claim 1 .

8. The calculation unit calculates the net stay-related information based on a time when the number of visitors in the analysis target area exceeded the visitor number threshold and a time when the number of visitors in the analysis target area fell below the visitor number threshold. The people flow management system according to claim 7.

9. The people flow management system further includes an attraction identification unit, the attraction identification unit identifies a facial angle or a line of sight of the captured person and trajectory information of the captured person based on the second video, and identifies attraction information to an area to be analyzed in the exhibition based on the facial angle or the line of sight of the person and the trajectory information of the person; The registration unit registers the attraction information as the people flow information. The people flow management system according to claim 1 .

10. The people flow management system further includes a map generation unit, the image acquisition unit acquires image data from each of a plurality of image capture devices; The map generation unit generates, for each imaging device, an individual determination map in which a determination rate of a person to be imaged and a position where the person is imaged correspond to each other based on the video data of each imaging device, and generates an imaging allocation map for specifying an imaging area of each imaging device based on the individual determination maps of the plurality of imaging devices. The people flow management system according to claim 1 .

11. A people flow management program for managing information about people flow at an exhibition, The people flow management program causes a computer to function as an image acquisition unit, a registration unit, an identification unit, and a calculation unit. the video acquisition unit acquires a first video of an area accessible only to non-attendees and a second video of an area to be analyzed; the registration unit registers, in a database, a person appearance image and / or a person feature amount of the non-attendee based on the first video; the identification unit identifies the attendee based on the person appearance images and / or person feature amounts of the non-attendees and the second video; the calculation unit calculates people flow information at the exhibition based on the identified visitors. People flow management program.

12. A people flow management method for managing information about people flow at an exhibition, comprising: The computer A process of acquiring a first video of an area accessible only to non-attendees and a second video of an area to be analyzed; a process of registering a person appearance image and / or person feature amount of the non-attendee in a database based on the first video; a process of identifying attendees based on the appearance images and / or person feature amounts of the non-attendees and the second video; A process of calculating people flow information at the exhibition based on the identified visitors; A people flow management method that implements the following.

Images