Congestion information display system and congestion information display method

Abstract

To provide a congestion information display system and a congestion information display method that can grasp congestion caused by a moving object moving within a facility.SOLUTION: An acquisition unit 101 acquires position information of a moving object 4 and image information of a surrounding area of the moving object 4 including the moving object 4 imaged by a camera 70. A generation unit 102 analyzes the image information and generates a congestion degree map indicating a distribution of degrees of congestion caused by people in the surrounding area. A determining unit 103 determines, on the basis of the position information and the congestion degree map, whether or not the surrounding area of the moving object 4 is in a congestion state caused by the moving object 4. A display unit 35 displays the congestion degree map and the determination result of the determination unit 103 as congestion information.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to a congestion information display system and a congestion information display method that display congestion information around a mobile object moving within a facility. [Background technology]

[0002] In commercial facilities and other such facilities, costumed mascot characters or children's trains for children to ride on entertain their customers. The facility managers ensure the safety of customers and guide their passage so that the mascot characters and other characters do not collide with customers or cause inconvenience to other customers due to overcrowding.

[0003] However, if more people than expected gather around a mascot character, the facility manager may be unable to ensure the safety and smooth flow of people due to a lack of staff. To prevent this, the facility manager needs to understand the congestion situation within the facility.

[0004] One method for automatically calculating and displaying the congestion level is to calculate the number of users, etc. within a certain area and then calculate the density of users, etc. based on this. For example, Japanese Patent Application Laid-Open No. 2017-152964 (Patent Document 1) discloses a monitoring device that calculates the congestion level of each area to be monitored. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-152964 Summary of the Invention [Problem to be solved by the invention]

[0006] However, while the above-mentioned methods can grasp the degree of congestion within a facility, they cannot identify the cause of the congestion. Therefore, since it is not clear whether the congestion around a moving object, such as a mascot character that entertains users, is the cause, it is not possible for facility managers to determine whether they need to secure or increase personnel to ensure the safety and smooth passage of users.

[0007] The present disclosure has been made to solve the above-mentioned problems, and its purpose is to provide a congestion information display system and a congestion information display method that can determine whether congestion is occurring due to moving objects moving within a facility. [Means for solving the problem]

[0008] A congestion information display system according to the present disclosure is a system that displays congestion information around a mobile object moving within a facility. The congestion information display system includes an acquisition unit, a generation unit, a determination unit, and a display unit. The acquisition unit acquires location information of the mobile object and image information of the area around the mobile object, including the mobile object, captured by a camera. The generation unit analyzes the image information and generates a congestion map that indicates the distribution of congestion levels caused by people in the area around the mobile object. The determination unit determines, based on the location information and the congestion map, whether the area around the mobile object is in a congested state due to the mobile object. The display unit displays the congestion map and the determination result of the determination unit as congestion information.

[0009] A congestion information display method according to the present disclosure is a method for displaying congestion information about the area surrounding a mobile object moving within a facility. The congestion information display method includes the steps of acquiring position information about the mobile object and image information about the area surrounding the mobile object, captured by a camera, analyzing the image information to generate a congestion map indicating the degree of congestion caused by people in the area surrounding the mobile object, determining whether the area surrounding the mobile object is in a congested state due to the mobile object based on the position information and the congestion map, and displaying the congestion map and the determination result of the determining step as congestion information. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to grasp whether congestion is occurring due to moving objects moving within a facility, thereby enabling facility managers to be appropriately assigned to ensure the safety and smooth passage of facility users. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is an overall configuration diagram of a congestion information display system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating a hardware configuration of a congestion information display system. [Figure 3] FIG. 2 is a diagram schematically showing a plan view of a floor in a facility as seen from the ceiling. [Figure 4] FIG. 1 is a diagram for explaining an image capturing situation of a moving object by a camera. [Figure 5] FIG. 1 is a diagram showing an example of congestion information displayed by the congestion information display system. [Figure 6] FIG. 2 is a functional block diagram of the congestion information display system. [Figure 7] 10 is a flowchart of a main process. [Figure 8] 10 is a flowchart of a congestion cause determination process. [Figure 9] FIG. 1 is a diagram for explaining a crowded state caused by moving objects. [Figure 10] FIG. 10 is a diagram for explaining a state where congestion is not caused by moving objects. [Figure 11] FIG. 10 is a diagram for explaining a congestion degree map when there is a fence around the moving object. [Figure 12] FIG. 10 is a diagram for explaining a congestion degree map when a moving object is close to a wall. [Figure 13] FIG. 10 is a diagram for explaining a congestion degree map when guidance is provided by a guide. [Figure 14] 10 is a flowchart of a congestion level determination process. [Figure 15] FIG. 10 is a diagram for explaining changes in congestion level. [Figure 16] 10 is a flowchart of a congestion map generation process. [Figure 17] FIG. 10 is a diagram illustrating an image synthesized based on images captured by a plurality of cameras. [Figure 18] FIG. 1 is a diagram for explaining an image estimated based on time-series images captured by a plurality of cameras. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of these components are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0013] 1 is a diagram illustrating an overall configuration of a congestion information display system 100 according to an embodiment of the present disclosure. As shown in FIG. 1, the congestion information display system 100 includes a server 10, a location monitoring device 20, a terminal 30, a plurality of wireless communication devices 72, and a plurality of cameras 70.

[0014] The congestion information display system 100 is a system that displays congestion information around a mobile object 4 moving within a facility. In this embodiment, a case where congestion information within a shopping center is displayed will be described as an example of a facility. A facility manager who manages the facility is stationed within this facility (shopping center).

[0015] The moving object 4 may be, for example, a mascot character wearing a costume. Such a mascot character entertains facility users (people 5), and crowds gather around the mascot character. The facility manager ensures the safety of users and guides their passage so that the mascot character does not collide with users or cause inconvenience to other users due to crowding.

[0016] However, if the number of users is greater than expected, the facility manager may be short of staff, making it impossible to ensure the safety and smooth passage of users. For this reason, this embodiment is configured to enable the facility manager to accurately grasp the congestion state within the facility.

[0017] The moving object 4 that entertains facility users is not limited to a mascot character, but may also be a street performer or the like. The moving object 4 may also be a children's train that carries children. In this case, the children's parents or other children who are not riding the children's train surround the moving children's train and move together. The moving object 4 may be any movable object that attracts the attention of facility users.

[0018] The customers are people 5 shown in Fig. 1. People 5 include people pushing strollers or shopping carts. In this embodiment, the "moving object 4" refers to a mascot character or the like that entertains customers of the facility, and customers of the facility are not included in the moving object 4.

[0019] The server 10, the location monitoring device 20, and the terminal 30 are configured to be able to communicate with each other via a communication network NW (typically, the Internet). Within the facility, multiple wireless communication devices 72 and multiple cameras 70 are installed, and these are configured to be able to measure the current locations of mobile objects 4 and users (people 5) within the facility.

[0020] The mobile object 4 includes a wireless communication device 62. The wireless communication device 62 transmits a signal for detecting the position of the mobile object 4 using a communication method conforming to the BLE (Bluetooth Low Energy, "Bluetooth" is a registered trademark) communication standard. Instead of the BLE communication standard, a communication method conforming to the UWB (Ultra Wide Band) communication standard or the like may be used. Alternatively, the wireless communication device 62 transmits a signal indicating an ID (Identification) or the like for identifying the mobile object 4 to the position monitoring device 20 using a communication method conforming to a wireless communication standard such as LTE (Long Term Evolution).

[0021] The multiple wireless communication devices 72 and the multiple cameras 70 are installed, for example, at an appropriate distance from each other on the ceiling 45 of the facility. The multiple wireless communication devices 72 receive signals emitted from the wireless communication device 62 of the mobile object 4 using a communication method that complies with the same communication standard as the wireless communication device 62 of the mobile object 4, and detect the reception strength of the signals. The position of the mobile object 4 within the facility can be determined from the reception strength at the wireless communication device 72. The wireless communication device 72 outputs the reception strength of the signal received from the wireless communication device 62 of the mobile object 4 to the position monitoring device 20.

[0022] The camera 70 captures images of the facility and outputs the captured image information to the location monitoring device 20. The captured image information includes images of the moving object 4 and the user (person 5). The wireless communication device 72 and the camera 70 may be installed on a wall.

[0023] The position monitoring device 20 is communicatively connected to a plurality of wireless communication devices 72 and cameras 70 installed on the ceiling 45. The position monitoring device 20 may be configured to include an indoor position information device and a monitoring camera device. In this case, the indoor position information device manages the position information of the mobile object 4 based on information from the plurality of wireless communication devices 72. The monitoring camera device manages image information of images captured by the plurality of cameras 70.

[0024] The position monitoring device 20 may be configured to include an indoor position information device and a monitoring camera device, or may be configured to include either an indoor position information device or a monitoring camera device.

[0025] The position monitoring device 20 (indoor position information device) receives the reception strength of the signal received by the wireless communication devices 72 from the wireless communication devices 72, and measures the position of the mobile object 4 within the facility from the reception strength at each wireless communication device 72.

[0026] Furthermore, the position monitoring device 20 (monitoring camera device) stores image information of images captured by the camera 70. The position monitoring device 20 (monitoring camera device) acquires position information of the moving object 4 from the position monitoring device 20 (indoor position information device), and identifies the camera 70 capturing the moving object 4 or the cameras 70 located around the moving object 4 based on the position information.

[0027] The location monitoring device 20 transmits the location information of the moving object 4 and the image information of the image captured by the identified camera 70 to the server 10 via the communication network NW. Note that the location monitoring device 20 may be configured to identify the location of the moving object 4 using the image information of the camera 70 without using the information from the wireless communication device 72.

[0028] The server 10 generates congestion information about the area around the mobile object 4. The facility manager can check the generated congestion information on the display unit 35 of the terminal 30.

[0029] The terminal 30 is communicably connected to the server 10 via the communication network NW. The terminal 30 is an information processing device having communication and display functions, such as a smartphone or tablet. The terminal 30 can be used by a facility manager or the like. Note that the terminal 30 is not limited to a mobile terminal such as a smartphone or tablet, and may be a PC (Personal Computer) or the like installed in the facility.

[0030] The terminal 30 typically has a web browser. The terminal 30 accesses the server 10 and displays screen data (web screen) on the terminal 30 itself. Alternatively, the terminal 30 may access the server 10 using dedicated application software installed thereon.

[0031] 2 is a diagram showing the hardware configuration of the congestion information display system 100. As described above, the congestion information display system 100 includes the server 10, the location monitoring device 20, the terminal 30, a plurality of wireless communication devices 72, and a plurality of cameras 70.

[0032] Server 10 includes, as its main components, a processor 11 that executes a program, a ROM (Read Only Memory) 12 that stores data in a non-volatile manner, a RAM (Random Access Memory) 13 that volatilely stores data generated by the execution of the program by processor 11 or data input via an input device, an HDD (Hard Disk Drive) 14 that non-volatilely stores data, and a communication IF (Interface) 15. The components are interconnected by a data bus 16.

[0033] The communication IF 15 is an interface for communication between the server 10 and the location monitoring device 20 and the terminal 30. The HDD 14 stores various data. Note that the server 10 may include another non-volatile storage device instead of or in addition to the HDD 14.

[0034] The location monitoring device 20 mainly includes a processor 21, a ROM 22, a RAM 23, a HDD 24, and a communication IF 25. The components are interconnected via a data bus 26. The communication IF 25 is an interface for communicating with the server 10, the wireless communication device 72, and the camera 70. The HDD 24 stores location information of the moving object 4, facility information, image information of images captured by the camera 70, etc.

[0035] Note that, while FIG. 2 shows an example of a configuration in which the necessary processing is provided by each of the processors 11, 21, and 31 executing a program, some or all of the processing provided may be implemented using dedicated hardware circuits (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array)).

[0036] The terminal 30 includes, as its main components, a processor 31, a ROM 32, a RAM 33, an input unit 34, a display unit 35, a communication IF 36, and a HDD 38. The components are interconnected via a data bus 37. The input unit 34 accepts input from a user (e.g., a facility manager). The display unit 35 displays the processing results of the processor 11. A touch panel display that integrates the input unit 34 and the display unit 35 may be employed. The communication IF 36 is an interface for communicating with the server 10. The HDD 38 stores installed application software, etc.

[0037] The location monitoring device 20 monitors the moving object 4 and the person 5. The location monitoring device 20 is communicatively connected to a plurality of wireless communication devices 72 and cameras 70 installed on the ceiling 45. The plurality of wireless communication devices 72 and cameras 70 are installed at appropriate distances on the ceiling 45 and are used to detect the positions of the moving object 4 and the person 5 present within the facility. The wireless communication device 62 carried by the moving object 4 transmits a signal for detecting the position of the moving object 4 using a communication method conforming to the BLE communication standard or the UWB communication standard, similar to the wireless communication device 72.

[0038] The wireless communication device 72 receives a signal emitted from the wireless communication device 62 using a communication method that complies with the same communication standard as the wireless communication device 62 carried by the mobile object 4, and detects the reception strength of the signal. The wireless communication device 72 outputs the reception strength of the signal received from the wireless communication device 62 to the position monitoring device 20. The camera 70 captures images of the inside of the facility and outputs the captured images (moving images) to the position monitoring device 20. The captured images include images of the mobile object 4 and people 5 in the facility.

[0039] The position monitoring device 20 receives the reception strength of the signal received by the wireless communication devices 72 from the wireless communication devices 72 and determines the position of the mobile object 4 within the facility from the reception strength at each wireless communication device 72.

[0040] Based on the position information of the mobile object 4, the position monitoring device 20 identifies the camera 70 capturing an image of the mobile object 4 or the camera 70 located in the vicinity of the mobile object 4. The position monitoring device 20 transmits the position information of the mobile object 4 and image information of the image captured by the identified camera 70 to the server 10 via the communication network NW.

[0041] Fig. 3 is a diagram showing a schematic plan view of a floor in a facility as seen from the ceiling. A screen 91 in Fig. 3 shows a schematic plan view (floor map) of a floor in a facility (in this example, the third floor of a shopping center) as seen from the ceiling.

[0042] As shown in Figure 3, the facility has multiple entrances and exits, multiple stores, a store warehouse, a shopping aisle for shoppers (customers), an employee aisle for employees, a machine room, a break room, restrooms (toilet), elevators EV1 to EV4, a parking lot, and an aisle to the parking lot.

[0043] The facility manager can view this floor map by displaying it on the display unit 35 of the terminal 30. Screen 91 shows that there are three moving objects 4a to 4c on the shopping passage (hereinafter also simply referred to as "passage"). In this embodiment, moving objects 4a to 4c are collectively referred to as "moving object 4."

[0044] The floor map is stored in the HDD 14 of the server 10. The server 10 acquires the position information of the moving objects 4a to 4c from the position monitoring device 20. The server 10 generates, as display information, an image in which the position information of the moving objects 4a to 4c is superimposed (plotted) on the floor map read from the HDD 14. The generated display information can be displayed on the display unit 35 of the terminal 30.

[0045] FIG. 4 is a diagram for explaining how a moving object 4 (moving objects 4a to 4c) is imaged by a camera 70. A plurality of cameras 70 (cameras 70a to 70e, etc.) are installed on the ceiling or walls of the facility. In this embodiment, the cameras 70a to 70e are collectively referred to as "cameras 70." FIG. 4 shows some of the plurality of cameras 70 installed in the facility.

[0046] As shown on screen 92 in Fig. 4, moving object 4a is present within the range that can be captured by camera 70a. Moving object 4b is present within the range that can be captured by camera 70b. Moving object 4c is present within the range that can be captured by camera 70c. Moving object 4 is not present within the range that can be captured by cameras 70d and 70e.

[0047] The position monitoring device 20 identifies camera 70a as the camera capturing an image of moving object 4a, camera 70b as the camera capturing an image of moving object 4b, and camera 70c as the camera capturing an image of moving object 4c.

[0048] The location monitoring device 20 transmits the location information of the moving objects 4a to 4c and the image information of the images captured by the cameras 70a to 70c to the server 10. As will be described later with reference to FIGS. 16 to 18, when one moving object 4 is captured by multiple cameras 70, the images captured by the multiple cameras are transmitted to the server 10. Even when the moving object 4 is not captured by a camera 70, if the moving object 4 is located near the camera 70, the image captured by the camera may be transmitted to the server 10.

[0049] Fig. 5 is a diagram showing an example of congestion information displayed by the congestion information display system 100. The screen 93 in Fig. 5 shows the location information of the moving object 4, a congestion map, and a determination result determined by the determination unit 103 (described later). Details of the congestion map and the determination result of the determination unit 103 will be described later with reference to Fig. 6 and subsequent figures.

[0050] The camera 70a captures images of the moving object 4a and a plurality of people 5 around the moving object 4a. The congestion map G1 is generated based on image information captured by the camera 70a. The determination unit 103 determines that congestion caused by the moving object 4a has occurred due to the plurality of people 5, and determines that the congestion level is congestion level A. As will be described in detail later, the congestion levels C, B, and A indicate increasing congestion levels in this order.

[0051] A confirmation message is displayed on the screen indicating that congestion has occurred due to a moving object and that the congestion level is congestion level A. This case will be described in detail later with reference to FIG. 9.

[0052] The camera 70b captures images of the moving object 4b and a plurality of people 5 around the moving object 4b. The congestion degree map G2 is generated based on image information captured by the camera 70b. The determination unit 103 determines that the moving object 4b is not causing congestion due to the plurality of people 5.

[0053] A message is displayed on the screen indicating that there is no congestion caused by moving objects. This case will be explained in detail later with reference to Figure 10.

[0054] The camera 70c captures an image of the moving object 4c and a plurality of people 5 around the moving object 4c. The congestion map G3 is generated based on image information captured by the camera 70c. The determination unit 103 determines that congestion caused by the moving object 4c has occurred due to the plurality of people 5, and determines that the congestion level is congestion level B. Furthermore, the determination unit 103 predicts that the congestion level will change from congestion level B to congestion level C.

[0055] A confirmation message is displayed on the screen indicating that congestion has occurred due to a moving object, and predicting that the congestion level is congestion level B and will later change to congestion level C. This case will be described in detail later with reference to FIG. 15.

[0056] 6 is a functional block diagram of the congestion information display system 100. The congestion information display system 100 includes at least a camera identification unit 110, an acquisition unit 101, a generation unit 102, a determination unit 103, a display information generation unit 104, and a display unit 35.

[0057] Each function will be explained below using a flowchart. FIG. 7 is a flowchart of the main processing. The main processing is executed by the congestion information display system 100. The main processing may be started at predetermined intervals (for example, every 10 msec). Hereinafter, "step" may also be simply referred to as "S".

[0058] When the main processing starts, in S101, the camera identification unit 110 identifies the camera 70 that captured the image of the moving object 4 from the position information of the moving object 4 identified by the position monitoring device 20. The HDD 24 stores a camera range map that can identify the shooting range of each camera 70 on a floor map. The camera identification unit 110 collates the position information of the moving object 4 with the camera range map to identify the camera 70 that is capturing the image of the moving object 4. Note that even if the moving object 4 is not captured, the camera 70 in the vicinity of the moving object 4 may be identified.

[0059] In S102, the acquisition unit 101 acquires the position information of the moving object 4 and the image information of the identified camera 70. Image information of the surrounding area of ​​the moving object 4, including the moving object 4, is acquired from the identified camera 70. If the moving object 4 is imaged by multiple cameras 70, image information from the multiple cameras is acquired.

[0060] In S103, the generation unit 102 executes a congestion map generation process. Here, the generation unit 102 analyzes the acquired image information and generates a congestion map. The congestion map is a map that shows the distribution of the congestion degree of people in the surrounding area. Details will be described later using the flowchart in FIG. 16, etc.

[0061] In S104, the determination unit 103 executes congestion cause determination processing. Here, the determination unit 103 determines whether or not the area around the moving object 4 is in a congestion state where congestion is caused by the moving object 4, based on the location information of the moving object 4 and the congestion degree map. Details will be described later using the flowchart in FIG. 8 etc.

[0062] In S105, the determination unit 103 executes a congestion level determination process. Here, the determination unit 103 determines the congestion level as one of three levels, A to C, based on a congestion map, and also predicts the future congestion level. Details will be described later using the flowchart in FIG. 14 etc.

[0063] In S106, the display information generation unit 104 generates display information to be displayed on the display unit 35 of the terminal 30. Specifically, the display information generation unit 104 acquires, as congestion information, the congestion degree map generated by the generation unit 102 in S103, the determination results of the determination unit 103 in S104 and S105, etc. The display information generation unit 104 also acquires information such as the floor map of the facility stored in the HDD 14. The display information generation unit 104 generates display information to be displayed on the display unit 35 based on the acquired information.

[0064] In S107, the server 10 outputs the display information to the display unit 35 of the terminal 30 via the communication network NW, and ends the main processing. The display unit 35 displays the display information generated by the display information generation unit 104. An example of the display on the display unit 35 is as described with reference to FIG. 5. In the example of FIG. 5, moving objects 4 (4a to 4c), congestion degree maps G1 to G3, and the determination result of the determination unit 103 are displayed on a floor map.

[0065] With the above configuration, the facility manager can grasp whether congestion is occurring due to moving objects 4 moving within the facility. In addition, while checking the congestion degree map on the floor map, the manager can determine whether it is necessary to guide users (people 5) around the moving objects 4. This makes it possible to appropriately assign facility managers to ensure the safety and smooth passage of facility users.

[0066] In this embodiment, the location monitoring device 20 and the server 10 are configured as separate entities, but this is not limiting and they may be configured as a single server device. Also, the processing of the server 10 may be performed by the above-mentioned monitoring camera device. In this case, the congestion information display system 100 includes an indoor location information device connected to multiple wireless communication devices 72, a monitoring camera device connected to multiple cameras 70, and a terminal 30 equipped with a display unit 35.

[0067] In this case, the indoor position information device identifies the position information of the moving object 4. The surveillance camera device identifies the camera 70 capturing the image of the moving object 4 based on the position information of the moving object 4. The surveillance camera device records a camera range map that can identify the capturing range of the camera 70. The surveillance camera device compares the position information of the moving object 4 with the camera range map to identify the camera 70 capturing the image of the moving object 4. The surveillance camera device generates a congestion map from the position information of the moving object 4 and the image information of the image captured by the identified camera 70, and determines whether the area around the moving object 4 is in a congested state due to congestion caused by the moving object 4. This information can be confirmed on the display unit 35.

[0068] Next, the congestion cause estimation process will be described using the flowchart of Fig. 8 and specific examples shown in Fig. 9 and Fig. 10. Fig. 8 is a flowchart of the congestion cause estimation process.

[0069] When the congestion map has a first area, a congested area, and a second area as shown in FIG. 9, the determination unit 103 determines that the area around the moving body 4 is in a congested state where congestion is caused by the moving body 4.

[0070] The first region is a region in the congestion map where the congestion level is less than the congestion level Ct and includes the moving body 4. The congested region is a region where the congestion level is equal to or greater than the congestion level Ct, is farther from the moving body 4 than the first region, and surrounds the moving body 4. The second region is a region where the congestion level is less than the congestion level Ct, is farther from the moving body 4 than the congested region, and surrounds the moving body 4.

[0071] 8, when the congestion cause estimation process starts, the determination unit 103 advances the process to S302 if the peak congestion level is equal to or greater than the congestion level Ct in S301 (determination of YES). On the other hand, the determination unit 103 advances the process to S302 if the peak congestion level is not equal to or greater than the congestion level Ct in S301 (determination of NO).

[0072] If the judgment unit 103 determines in S302 that the first area, the congested area, and the second area exist in the congestion map (determination of YES), it proceeds to S303, and if it does not determine that the first area, the congested area, and the second area exist (determination of NO), it proceeds to S304.

[0073] In S303, the determination unit 103 determines that the congestion state is due to the moving object 4, and ends the congestion cause estimation process. In S304, the determination unit 103 determines that the congestion state is not due to the moving object 4, and ends the congestion cause estimation process.

[0074] A detailed description will be given below with reference to Figures 9 and 10. Figure 9 is a diagram for explaining a congestion state caused by a moving object 4. The congestion map G1 shown on the right side of Figure 9 is the same as the congestion map generated based on image information of the image captured by the camera 70a in Figure 5. The graph on the left side of Figure 9 is a graph that schematically shows the relationship between the distance from the moving object 4 and the congestion degree.

[0075] The image acquired from camera 70a is an image capturing a scene in which multiple people 5 are present around moving object 4. Server 10 uses known image analysis technology to extract from the image an area in which each of the multiple people 5 is present. Then, server 10 identifies the position of each of the multiple people 5 using the above-mentioned camera range map that can identify the shooting range of each camera 70 on the floor map.

[0076] The generation unit 102 sets a peripheral area of ​​the moving object 4. The peripheral area of ​​the moving object 4 is an area centered on the moving object 4, and is an area for which a congestion map is to be generated. For example, an area with a radius of 10 m centered on the moving object 4 may be set as the peripheral area, or a range of a predetermined size may be set as the peripheral area. The generation unit 102 acquires position information of people 5 present in the peripheral area.

[0077] The generation unit 102 divides the surrounding area into multiple blocks and calculates the congestion level of people 5 present in each block. Here, the "congestion level" refers to the number of people 5 present per square meter in each block. Alternatively, the "congestion level" may be defined as the ratio of the area of ​​an area where multiple people 5 exist to the area of ​​the block.

[0078] The congestion map shows five levels of congestion, which are represented by five different colors: the higher the congestion level, the darker the block, and the lower the congestion level, the lighter the block.

[0079] In this example, a moving object 4 is present in the passage between wall W1 and wall W2, and multiple people 5 are present surrounding the moving object 4. The congestion map G1 shows that the people 5 are surrounding the moving object 4 in a donut shape with the moving object 4 at the center. In the congestion map G1, the congestion level gradually increases with increasing distance from the moving object 4, and reaches its peak when the distance from the moving object 4 is distance D1. With further increasing distance, the congestion level gradually decreases.

[0080] A graph showing the relationship between the distance from the moving object 4 and the congestion level shows that the congestion level increases as the distance from the moving object 4 increases, reaching congestion level Ct. After that, the congestion level increases further as the distance from the moving object 4 increases, reaching a peak congestion level C1 at distance D1. After that, the congestion level decreases as the distance from the moving object 4 increases, reaching congestion level Ct. After that, the congestion level decreases as the distance from the moving object 4 increases.

[0081] Here, the congestion level Ct is a boundary value that indicates whether or not people can pass through. In an area where the congestion level is equal to or greater than Ct, people cannot pass through the area due to the congestion. In an area where the congestion level is less than Ct, people can pass through the area.

[0082] In this graph, an area where the congestion level is equal to or greater than congestion level Ct is defined as a "congested area." An area closer to the moving object 4 than the congested area is defined as a first area, and an area farther from the moving object 4 than the congested area is defined as a second area. In this example, the width of the area where the congestion level is equal to or greater than congestion level Ct is congestion width H1. In this example, the congestion level is determined to be congestion level A (details will be described later).

[0083] When this is applied to the congestion map G1, the circular area including the moving object 4 becomes the first area, the donut-shaped area outside of that becomes the congested area, and the area outside of that within the surrounding area becomes the second area.

[0084] The determination unit 103 determines that the area around the moving body 4 is in a congested state due to congestion caused by the moving body 4 when the congestion map contains a first area where the congestion level is less than the congestion level Ct, a congested area where the congestion level is equal to or greater than the congestion level Ct, and a second area where the congestion level is less than the congestion level Ct.

[0085] Here, the congested area does not necessarily have to be a circular doughnut shape as in the example of FIG. 9, but may be an oval shape or a shape like the letter "C" with a missing part. The congested area may have any shape as long as it is outside the first area and inside the second area and surrounds the moving object 4. Furthermore, as shown in FIG. 9, in reality, the congestion degree in the congestion degree map varies, so that the boundaries between the first area, the congested area, and the second area can be determined after smoothing the congestion degree, as in the graph.

[0086] With the configuration of this embodiment, it is possible to understand that people 5 (customers) are gathering around the moving object 4 at a short distance to view the moving object 4 (e.g., a mascot character), i.e., it is possible to understand the causal relationship that the moving object 4 is causing the people 5 to gather. If the congestion level caused by the people 5 surrounding the moving object 4 is equal to or greater than a predetermined level (congestion level Ct), it can be determined that the congestion is caused by the people 5 gathering because of the moving object 4. In this way, if it can be determined whether congestion is caused by the moving object 4 moving within the facility, the facility manager can determine whether or not action is required to address the congestion caused by the moving object 4. This makes it possible to appropriately assign facility managers to ensure the safety and smooth passage of facility users.

[0087] Fig. 10 is a diagram for explaining a state in which congestion is not caused by a moving object 4. The congestion map G2 shown on the right side of Fig. 10 is the same as the congestion map generated based on image information of the image captured by the camera 70b in Fig. 5. The graph on the left side of Fig. 10 is a graph that schematically shows the relationship between the distance from the moving object 4 and the congestion degree.

[0088] The image acquired by the camera 70b is an image capturing a scene in which a plurality of people 5 are present around the moving object 4. The generation unit 102 identifies the position of each of the plurality of people 5 by the method described in FIG.

[0089] The generation unit 102 sets a surrounding area of ​​the moving object 4. Here, as in FIG. 9, an area with a radius of 10 m centered on the moving object 4 is set as the surrounding area. The generation unit 102 acquires position information of people 5 present in the surrounding area. The generation unit 102 divides the surrounding area into a plurality of blocks and calculates the degree of congestion of people 5 present in each block.

[0090] In this example, a moving object 4 is present in the passage between wall W1 and wall W2, and multiple people 5 are present around the moving object 4. However, unlike the example in FIG. 9, as shown in the graph that schematically illustrates the relationship between the distance from the moving object 4 and the congestion level, there is no correlation between the congestion level around the moving object 4 and the distance from the moving object 4. As shown in this graph, the congestion level is less than the congestion level Ct regardless of the distance from the moving object 4.

[0091] If the determination unit 103 does not determine that there is a first area, a congested area, and a second area in the congestion degree map, it determines that the area around the moving body 4 is not in a congested state where congestion is caused by the moving body 4.

[0092] In the example of Figure 10, there are multiple people 5 around the moving object 4. However, unlike the example of Figure 9, the people 5 are not gathered around the moving object 4 (there is no doughnut-shaped peak). In this case, even if the surrounding area is somewhat crowded (such as simply being crowded with shoppers), it is not determined that the area around the moving object 4 is in a crowded state due to the moving object 4, since this is not related to the presence of the moving object 4.

[0093] If the congestion is caused by the moving object 4, it is possible that people 5 will gather around the moving object 4, people 5 will move along with the moving object 4, or the number of people 5 will gradually increase, creating a dangerous situation. However, if the congestion is not caused by the moving object 4 (for example, the facility is simply crowded), people can pass through it if the congestion level is below a predetermined level (congestion level Ct), and there is no need to take measures such as guiding people 5 who have gathered to see the moving object 4 (such as a mascot character) so that they do not get in the way of passersby.

[0094] For example, if the number of people 5 in the surrounding area is the same in the example of FIG. 9 and the example of FIG. 10, the conventional technology would determine that both are "crowded." On the other hand, in the present embodiment, it is determined that congestion caused by the moving object 4 has occurred in the former case, and that congestion caused by the moving object 4 has not occurred in the latter case. By configuring as in the present embodiment, it becomes possible to more accurately assign facility managers to ensure the safety and smooth passage of facility users.

[0095] Fig. 11 is a diagram illustrating a congestion map when there is a fence around the moving object 4. Similar to the example in Fig. 9, multiple people 5 are gathered around the moving object 4, but unlike the case in Fig. 9, a fence L1 is installed around the moving object 4.

[0096] In this case, people 5 cannot enter the area from the moving object to the fence L1. People 5 gather close to the fence L1. In this way, when the fence L1 is installed around the moving object 4, the fence L1 becomes the boundary between the first area and the crowded area.

[0097] As shown in the congestion map G4, the area inside the fence L1 is the first area, the area outside the fence L1 is the congested area, and the area further outside that is the second area. As shown in the graph, the congestion level peaks at the location where the fence L1 is installed, a distance Da away from the moving object 4. As the distance increases, the congestion level gradually decreases and becomes less than the congestion level Ct.

[0098] Even in such a case, since the first area, the congested area, and the second area exist in the congestion degree map G4, it is determined that the area around the moving object 4 is in a congested state where congestion is caused by the moving object 4.

[0099] Fig. 12 is a diagram for explaining a congestion map when the moving object 4 is close to a wall. The congestion map G3 shown on the left side of Fig. 12 is the same as the congestion map generated based on the image information of the image captured by the camera 70c in Fig. 5.

[0100] Assume that moving object 4 is moving through the passage between wall W1 and wall W2 as shown in the diagram on the left. Then, as shown in the diagram on the right, moving object 4 moves close to wall W3. In this case, the congestion map will look like G5, and the congestion area will not be doughnut-shaped.

[0101] However, even in this case, there is a first area including the moving body, a congested area outside that, and a second area outside that, so it is determined that the area around moving body 4 is in a congested state due to moving body 4.

[0102] Fig. 13 is a diagram for explaining a congestion level map when guidance is provided by a guide (facility manager). For example, as in the example of Fig. 10, if the congestion level in the entire surrounding area is less than Ct, people can pass through this area. Therefore, there is no need for the facility manager to guide person 5.

[0103] However, when a congested area exists, as in the example of Figure 9, people cannot pass through this area. In particular, when the congested area spreads over the entire aisle, people cannot pass through the aisle.

[0104] In such a case, the facility manager will guide person 5, who is between line L2 and wall W2, toward wall W1 from line L2. By guiding people in this way, other users can pass through. By displaying a congestion map on the layout map as shown in Figure 5, the facility manager can view the map and consider how to guide people and how many people are needed for guidance.

[0105] Next, the congestion level determination process will be described with reference to Figures 14 and 15. Figure 14 is a flowchart of the congestion level determination process. Figure 15 is a diagram for explaining changes in the congestion level.

[0106] The determination unit 103 determines the congestion level based on the congestion map. As shown in Fig. 14, the congestion levels include congestion level A (low congestion), congestion level B (medium congestion), and congestion level C (high congestion). Congestion level B indicates that the surrounding area is more congested than congestion level A. Congestion level C indicates that the surrounding area is more congested than congestion level B.

[0107] The congestion level is determined based on the congestion level determination table. When the congestion level determination process starts, if the congestion width is less than the congestion width HA1 and the peak position is equal to or greater than the peak position DA1 (YES in S401), the determination unit 103 sets the congestion level to congestion level A (S402), and the process proceeds to S406.

[0108] 9, the congestion width is H1, and the peak position is D1. The congestion width H1 is less than the congestion width HA1, and the peak position D1 is equal to or greater than the peak position DA1. Therefore, the congestion level is set to A in the congestion map G1.

[0109] On the other hand, if the congestion width is less than the congestion width HA1 and the peak position is not greater than or equal to the peak position DA1 (NO in S401), and if the congestion width is less than the congestion width HA1 and the peak position is less than the peak position DA1 (YES in S403), the judgment unit 103 sets the congestion level to congestion level B (S404) and proceeds to S406.

[0110] If the congestion width is less than congestion width HA1 and the peak position is not less than peak position DA1 (NO in S403: in this case, the congestion width is equal to or greater than HA1), the judgment unit 103 sets the congestion level to congestion level C (S405) and proceeds to S406.

[0111] The congestion map G3 shown on the right side of Fig. 15 is the same as the congestion map generated based on the image information of the image captured by the camera 70c in Fig. 5. The congestion map G3 is in a more congested state than the congestion map G1 (congestion level A) shown in Fig. 9.

[0112] The congestion map G3 has a narrower first region than the congestion map G1. Furthermore, the peak position D2 of the congestion map G3 is smaller than the peak position D1 of the congestion map G1. The congestion width in the congestion map G3 is congestion width H2.

[0113] The congestion width H2 is less than the congestion width HA1, and the peak position D2 is less than the peak position DA1. Therefore, the congestion level is set to congestion level B in the congestion map G3.

[0114] If the congestion becomes even more severe, the congestion width will widen. Suppose the congestion width widens to congestion width H3, which is wider than congestion width H2. Congestion width H3 is equal to or greater than congestion width HA1. In this case, the congestion level is set to congestion level C. Congestion level B has a narrower first area than congestion level A, and congestion level C has a wider congestion area than congestion level B.

[0115] Returning to Fig. 14, the determination unit 103 predicts a future congestion level. If it is determined in S406 that the congestion level has changed from congestion level A to congestion level B (determination of YES), the determination unit 103 predicts that the future congestion level will be congestion level C, and ends the congestion level determination process.

[0116] On the other hand, if the determination unit 103 does not determine in S406 that the congestion level has changed from congestion level A to congestion level B (determination of NO), the congestion level determination process ends. As shown in Fig. 5, the display unit 35 displays the determined congestion level and the predicted future congestion level.

[0117] The reason why the peak position is closer to the moving object 4 (the first area is narrower) at congestion level B than at congestion level A is as follows: Initially, people 5 surround the moving object 4 (mascot character, street performer, etc.) from a distance, forming a doughnut-shaped crowded area. Then, as the number of people gradually increases, the space becomes narrower, and the doughnut-shaped crowded area naturally moves closer to the moving object 4. Alternatively, as the area becomes crowded, the street performer (moving object 4) or facility manager, etc., instructs people 5 (passengers) to move closer to the moving object 4 so as not to obstruct the passage of other passersby.

[0118] In this way, when the congestion level changes from congestion level A to congestion level B by an instruction (when the first area narrows), further congestion is expected, so the determination unit 103 predicts that the congestion level will change from congestion level B to congestion level C in the future. As the number of people 5 increases, the congestion width will be wider at congestion level C than at congestion level B.

[0119] As described above, by classifying and displaying congestion levels according to the degree of congestion, facility managers can intuitively grasp the congestion situation and take prompt action according to the congestion level. In addition, by displaying predicted future congestion levels, an appropriate number of facility managers can be assigned in preparation for major congestion.

[0120] The method of determining whether or not a congestion state is occurring due to moving objects, the method of determining the congestion level, and the method of predicting the future congestion level are not limited to those described above.

[0121] In this embodiment, when the first area, the congested area, and the second area are present, it is determined that the congestion state is due to the moving object 4. However, it may be determined that the congestion state is present when the relationship between the congestion degree and the distance to the moving object 4 has a distribution shape as shown in Fig. 9. Alternatively, a distribution for determining that the congestion state is present (a congestion degree distribution serving as a determination criterion) may be specified in advance, and if the shape resembles that congestion degree distribution, it may be determined that the congestion state is present.

[0122] Furthermore, a reference congestion distribution may be defined for each congestion level, and if the shape of the congestion distribution is similar to that corresponding to congestion level A, the congestion level may be determined to be A. The same applies to congestion levels B and C. Furthermore, the determination is not limited to the method shown in the congestion level determination table, and may also be based on criteria such as the number of people 5 present in the surrounding area and the size of the congestion peak.

[0123] Furthermore, future congestion levels may be predicted based on time-series information on the number of people 5 present in the surrounding area (for example, if the number of people 5 present in the surrounding area is on the rise, it may be determined that the number of people 5 will further increase in the future). The congestion levels are not limited to three levels, A to C, and may be classified into four or more levels.

[0124] 14, the congestion level determination table may extract the block with the highest congestion level in the congestion map (multiple blocks may be extracted on a doughnut), and define the average value of the distance between the extracted block and the moving object 4 as the peak position. The congestion width may also be determined based on the distance from the boundary line between the first area and the congested area to the boundary line between the congested area and the second area.

[0125] Furthermore, in the congestion level determination table of FIG. 14, the "area of ​​the congested region" may be used instead of the "congestion width" and the "area of ​​the first region" may be used instead of the "peak position." In this case, if the area of ​​the congested region is less than X1 and the area of ​​the first region is greater than or equal to Y1, the congestion level is set to congestion level A. If the area of ​​the congested region is less than X1 and the area of ​​the first region is less than Y1, the congestion level is set to congestion level B. Otherwise (if the area of ​​the congested region is greater than or equal to X1), the congestion level is set to congestion level C. In other words, when the area of ​​the first region becomes smaller than Y1, the congestion level changes from A to B, and then, when the area of ​​the congested region becomes greater than or equal to X1, the congestion level changes from B to C.

[0126] Next, the congestion map generation process will be described with reference to Fig. 16 to Fig. 18. Fig. 16 is a flowchart of the congestion map generation process. Fig. 17 is a diagram for explaining an image synthesized based on images captured by a plurality of cameras 70. Fig. 18 is a diagram for explaining an image estimated based on time-series images captured by a plurality of cameras 70.

[0127] The cameras 70 include a camera A and a camera B. In this example, a scene in which the moving object 4 is imaged by one or both of the cameras A and B, or a scene in which the moving object 4 is not imaged by either the camera A or the camera B, will be described.

[0128] When a moving object 4 is captured in both image information A captured by camera A and image information B captured by camera B, generation unit 102 analyzes either image information A or image information B or image information combining both to generate a congestion map. When a moving object 4 is not captured in either image information A or image information B, generation unit 102 analyzes image information estimated based on time series information of image information A captured in the past and time series information of image information B captured in the past to generate a congestion map. This will be described in detail below.

[0129] 16, when the congestion map generation process starts, if there is complete image information in which the entire surrounding area is captured (YES in S201), the generation unit 102 sets the complete image information as target image information (S202) and proceeds to S209. In S209, the generation unit 102 analyzes the target image information to generate a congestion map, and then ends the congestion map generation process.

[0130] For example, as shown in Figure 9, if the entire surrounding area (excluding areas that cannot be imaged due to walls) is captured by camera A, the image obtained from camera A is analyzed to generate a congestion map G1 as shown in Figure 9.

[0131] Returning to FIG. 16, if there is no complete image information in which the entire surrounding area is captured (NO in S201), or if there is multiple pieces of image information that can be combined (YES in S203), the generation unit 102 proceeds to S204. In S204, the generation unit 102 combines the multiple pieces of image information to generate combined image information, and proceeds to S205. In S205, the generation unit 102 sets the combined image information as target image information, and in S209, analyzes the target image information and generates a congestion degree map.

[0132] A specific explanation will be given using Fig. 17. The position monitoring device 20 identifies the camera 70 that captures images of the moving object 4, and generates a congestion map based on images acquired from the identified camera 70. Fig. 17 explains a case where there are multiple cameras 70 that capture images of the moving object 4. Here, there are two cameras 70 that capture images of the moving object 4, and these are referred to as camera A and camera B.

[0133] 9, a congestion map can be generated using the image acquired from one camera 70. However, as shown in FIG. 17, there are cases where camera A does not capture part of person 5 on the right side of moving object 4, and camera B does not capture part of person 5 on the left side of moving object 4.

[0134] If a congestion map is generated in this state, the congestion map Ga generated based on the image information obtained from camera A will be missing on the right side, and the congestion map Gb generated based on the image information obtained from camera B will be missing on the left side.

[0135] In this example, using a known technique, image information acquired from camera A and image information acquired from camera B are combined so that the entire surrounding area is included. As a result, a congestion map Gc that includes the entire surrounding area is generated.

[0136] Returning to FIG. 16, if there are no multiple pieces of image information that can be combined (NO in S203), the generation unit 102 acquires past image information (S206) and proceeds to the process of S207. In S207, the generation unit 102 estimates image information and proceeds to the process of S208. In S208, the generation unit 102 sets the estimated image information as target image information, and in S209, analyzes the target image information and generates a congestion map.

[0137] This will be explained in detail using Fig. 18. In the example of Fig. 17, by combining the image from camera A and the image from camera B, it is possible to synthesize an image that includes the entire surrounding area. However, because the distance between camera A and camera B is large, there are cases where combining the images from both cameras does not result in an image that includes the entire surrounding area. Such a case will be explained below using Fig. 18.

[0138] Fig. 18 is a diagram for explaining an image estimated based on a time series of images captured by a plurality of cameras 70. Fig. 18 shows a diagram illustrating the change over time of a congestion map generated from images acquired from cameras A and B.

[0139] 18, at time T1, moving object 4 is at position P1, and an image including the entire surrounding area is acquired from camera A. Camera B does not capture an image of moving object 4. In this case, a congestion map can be generated based on the image acquired from camera A.

[0140] Moving object 4 moves to the right. At time T2, moving object 4 is at position P2, and an image that includes the entire surrounding area cannot be acquired from camera A. Furthermore, even if the images acquired from cameras A and B are combined, a congestion map that includes the entire surrounding area cannot be generated.

[0141] At time T3, moving object 4 is at position P3 and is not captured by either camera A or B. At time T4, moving object 4 is at position P4 and is captured by camera B, but even if the images acquired from cameras A and B are combined, a congestion map that includes all of the surrounding area cannot be generated.

[0142] At time T5, moving object 4 is at position P5, and an image including the entire surrounding area is acquired from camera B. Moving object 4 is not captured by camera A. In this case, a congestion map can be generated based on the image acquired from camera B.

[0143] In the embodiment, if it is not possible to generate a congestion map that includes all of the surrounding area by combining images acquired from cameras A and B, an image that includes all of the surrounding area that was previously acquired or generated is used.

[0144] At time T1, the congestion degree map Ga1 generated from the image acquired by camera A becomes the congestion degree map Ga1 at time T1.

[0145] At time T2, no congestion map is generated from cameras A and B, so the image acquired from camera A at time T1 is used as the image acquired at time T2. At that time, the position information of person 5 is corrected by moving it by a distance of P12 = P2 - P1. As a result, the congestion map Gc2 at time T2 is the congestion map obtained by moving the congestion map Gc1 at time T1 by P12.

[0146] The same is true for times T3 and T4. The congestion map Gc3 at time T3 is the congestion map obtained by shifting the congestion map Gc2 at time T2 by P23 (=P3-P2). The congestion map Gc4 at time T4 is the congestion map obtained by shifting the congestion map Gc3 at time T3 by P34 (=P4-P3).

[0147] At time T5, a congestion degree map Gb5 generated from an image acquired by camera B becomes a congestion degree map Gc5 at time T5.

[0148] In this way, if a complete congestion map (a congestion map that includes the entire surrounding area) can be generated using either camera A or B, the congestion map is generated using images from either camera. If a complete congestion map can be generated by combining images from cameras A and B, the congestion map is generated using images from both cameras.

[0149] If a complete congestion map cannot be generated by combining the images from cameras A and B, a congestion map is generated based on time-series information of images captured in the past by cameras A and B. In this case, future location information of a moving body 4 may be predicted from the transition of the location information of the moving body 4 based on the time-series information of images captured in the past by cameras A and B, and future location information of multiple people 5 may be predicted from the transition of the location information of multiple people 5.

[0150] In this way, for example, if the congestion level of the congestion map Gb5 (Gc5) at time T5 is higher than that of the congestion level of the congestion map Ga1 (Gc1) at time T1, it is possible to predict a congestion level map that is even more congested than the congestion level map Gb5 (Gc5) at a time beyond time T5.

[0151] In this embodiment, images are synthesized or predicted from images acquired from cameras A and B, and a complete congestion map is generated based on these images. However, the present invention is not limited to this, and the following method may be used: First, a congestion map A is generated based on an image acquired from camera A, and a congestion map B is generated based on an image acquired from camera B. Then, the congestion maps A and B may be synthesized or predicted to generate a complete congestion map.

[0152] As described above, by combining image information captured by multiple cameras to generate image information that includes the entire surrounding area, or by predicting image information that includes the entire surrounding area based on time-series information of image information captured by multiple cameras, it is possible to generate a congestion map with high accuracy even when image information that includes the entire surrounding area is not captured by a single camera. This makes it possible to accurately determine whether congestion is occurring due to moving objects 4 moving within the facility.

[0153] [Note] The above-described embodiment is a specific example of the following additional notes.

[0154] (Appendix 1) A congestion information display system that displays congestion information around a mobile object moving within a facility, an acquisition unit that acquires position information of the moving object and image information of the surrounding area of ​​the moving object including the moving object captured by a camera; a generation unit that analyzes the image information and generates a congestion map that indicates a distribution of congestion levels of people in the surrounding area; a determination unit that determines whether the area around the moving object is in a congested state where congestion is caused by the moving object, based on the location information and the congestion degree map; A congestion information display system comprising: a display unit that displays the congestion degree map and the determination result of the determination unit as the congestion information.

[0155] (Appendix 2) the cameras include a first camera and a second camera; The generation unit When the moving object is captured in both the first image information captured by the first camera and the second image information captured by the second camera, an image obtained by combining the first image information and the second image information is analyzed to generate the congestion degree map; A congestion information display system as described in Appendix 1, which generates the congestion map by analyzing an estimated image based on time series information of the first image information captured in the past and time series information of the second image information captured in the past when the moving object is not captured in either the first image information or the second image information.

[0156] (Appendix 3) the determination unit determines that the state is congested when the congestion map includes a first region where the congestion degree is less than a predetermined congestion degree, a congested region where the congestion degree is equal to or greater than the predetermined congestion degree, and a second region where the congestion degree is less than the predetermined congestion degree; the first area is an area including the moving object, the congested area is an area that is farther from the moving object than the first area and surrounds the moving object, The congestion information display system according to claim 1 or 2, wherein the second area is an area that is farther from the moving object than the congested area and surrounds the moving object.

[0157] (Appendix 4) 4. The congestion information display system according to any one of appendices 1 to 3, wherein, when a fence is provided around the moving object, the fence serves as a boundary between the first area and the congested area.

[0158] (Appendix 5) The determination unit determines a congestion level based on the congestion map, the congestion level includes a first congestion level, a second congestion level indicating that the surrounding area is more congested than the first congestion level, and a third congestion level indicating that the surrounding area is more congested than the second congestion level, 5. The congestion information display system according to any one of appendices 1 to 4, wherein the display unit further displays the determined congestion level.

[0159] (Appendix 6) The determination unit predicts the future congestion level, The display unit further displays the predicted future congestion level, The second congestion level has a narrower first area than the first congestion level, The third congestion level has a wider congestion area than the second congestion level, A congestion information display system described in any of Appendices 1 to 5, wherein the determination unit predicts that the future congestion level will become the third congestion level when the congestion level changes from the first congestion level to the second congestion level.

[0160] (Appendix 7) A congestion information display method for displaying congestion information around a mobile object moving within a facility, comprising: acquiring position information of the moving object and image information of the surrounding area of ​​the moving object including the moving object captured by a camera; analyzing the image information to generate a crowding map indicating a crowding level of people in the surrounding area; determining whether the area around the moving object is in a congested state where congestion is caused by the moving object, based on the location information and the congestion degree map; A congestion information display method comprising the step of displaying the congestion degree map and the determination result of the determining step as the congestion information.

[0161] The embodiments disclosed herein are merely examples and are not limited to the above. The scope of the present invention is defined by the claims, and it is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0162] 4,4a-c Mobile object, 5 Person, 10 Server, 11,21,31 Processor, 12,22,32 ROM, 13,23,33 RAM, 14,24,38 HDD, 15,25,36 Communication IF, 16,26,37 Data bus, 20 Location monitoring device, 30 Terminal device, 34 Input unit, 35 Display unit, 45 Ceiling, 72 Wireless communication device, 70 Camera, 91 Screen, 100 Congestion information display system, 101 Acquisition unit, 102 Generation unit, 103 Determination unit, 104 Display information generation unit, NW Communication network.

Claims

1. A congestion information display system that displays congestion information around a mobile object moving within a facility, an acquisition unit that acquires position information of the moving object and image information of the surrounding area of ​​the moving object including the moving object captured by a camera; a generation unit that analyzes the image information and generates a congestion map that indicates a distribution of congestion levels of people in the surrounding area; a determination unit that determines whether the area around the moving object is in a congested state due to the moving object; a display unit that displays the congestion degree map and the determination result of the determination unit as the congestion information, the determination unit determines that the state is congested when the congestion degree map includes a first region where the congestion degree is less than a predetermined congestion degree, a congested region where the congestion degree is equal to or greater than the predetermined congestion degree, and a second region where the congestion degree is less than the predetermined congestion degree; the first area is an area including the moving object, the congested area is an area that is farther from the moving object than the first area and surrounds the moving object, A congestion information display system, wherein the second area is an area farther from the mobile body than the congested area and surrounds the mobile body.

2. the cameras include a first camera and a second camera; The generation unit When the moving object is captured in both the first image information captured by the first camera and the second image information captured by the second camera, an image obtained by combining the first image information and the second image information is analyzed to generate the congestion degree map; 2. The congestion information display system of claim 1, wherein if the moving object is not captured in either the first image information or the second image information, the congestion map is generated by analyzing an estimated image based on time series information of the first image information captured in the past and time series information of the second image information captured in the past.

3. The congestion information display system according to claim 1 or 2, wherein, when a fence is provided around the moving object, the fence serves as a boundary between the first area and the congested area.

4. The determination unit determines a congestion level based on the congestion map, the congestion level includes a first congestion level, a second congestion level indicating that the surrounding area is more congested than the first congestion level, and a third congestion level indicating that the surrounding area is more congested than the second congestion level, The congestion information display system according to claim 1 or 2, wherein the display unit further displays the determined congestion level.

5. The determination unit predicts the future congestion level, The display unit further displays the predicted future congestion level, The second congestion level has a narrower first area than the first congestion level, The third congestion level has a wider congestion area than the second congestion level, 5. The congestion information display system according to claim 4, wherein the determination unit predicts that the future congestion level will become the third congestion level when the congestion level changes from the first congestion level to the second congestion level.

6. A congestion information display method executed by a computer for displaying congestion information around a mobile object moving within a facility, comprising: acquiring position information of the moving object and image information of the surrounding area of ​​the moving object including the moving object captured by a camera; analyzing the image information to generate a crowding map indicating a crowding level of people in the surrounding area; determining whether the area around the moving object is in a congested state due to the moving object; a step of displaying the congestion degree map and the determination result of the determining step as the congestion information, the determining step includes a step of determining that the congestion state exists when the congestion map includes a first region where the congestion degree is less than a predetermined congestion degree, a congested region where the congestion degree is equal to or greater than the predetermined congestion degree, and a second region where the congestion degree is less than the predetermined congestion degree, the first area is an area including the moving object, the congested area is an area that is farther from the moving object than the first area and surrounds the moving object, A congestion information display method, wherein the second area is an area farther from the moving object than the congested area and surrounds the moving object.

Images