Facility status display device, program, and facility status display method

Abstract

We provide a facility status display device that allows users to understand the current congestion level within the facility. [Solution] The system comprises a map display processing unit 11 that displays map information including one or more facility icons indicating the location of facility 2a on a map; a virtual space generation unit 12 that generates a virtual space image that reproduces the current state of facility 2a, including the internal space of facility 2a indicated by each of the one or more facility icons, people currently present in facility 2a, and the current internal environment of facility 2a; a virtual space display processing unit 13 that displays a first virtual space image corresponding to the first facility icon from a free viewpoint in response to an operation to specify a first facility icon from among the one or more facility icons; and an operation reception unit 14 that receives an operation to specify the position of the viewpoint from which to display the first virtual space image. The virtual space display processing unit 13 displays the virtual space image from the first viewpoint when the operation reception unit 14 receives an operation to specify the position of the first viewpoint.

Description

Technical Field

[0001] The present disclosure relates to a facility status presentation device, a program, and a facility status presentation method that present the current status of a facility.

Background Art

[0002] For example, Patent Document 1 discloses a congestion status presentation device that allows a user to grasp the congestion status of people in a predetermined space by presenting the user with a congestion status diagram in which person data indicating detected people is drawn.

[0003] Further, for example, Patent Document 2 discloses a live video providing system capable of providing real-time live video to the user side.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problem to be Solved by the Invention

[0005] However, in the technique disclosed in Patent Document 1, the congestion status that can be confirmed from the congestion status diagram drawn by the congestion status presentation device may not match the congestion status that the user perceives when actually visiting the facility.

[0006] Furthermore, the technology disclosed in Patent Document 2 generates real-time live video footage using multiple video cameras that capture images in different directions, but it can only generate live video footage within the field of view of each individual video camera. In other words, users can only perceive the congestion situation within the field of view of the video cameras, which is narrower than the field of view they actually see. Consequently, the congestion situation that can be grasped from the live video footage within the field of view of each video camera may not match the congestion situation that users actually experience when they visit the facility.

[0007] In one aspect, this disclosure aims to provide a facility status display device that allows users to understand the current congestion level within the facility. [Means for solving the problem]

[0008] (1) The facility status display device of the present disclosure comprises: a map display processing unit that displays map information including one or more facility icons indicating the location of a facility on a map; a virtual space generation unit that generates a virtual space image that reproduces the current status of the facility, including the interior space of the facility indicated by each of the one or more facility icons, people currently present in the facility, and the current interior environment of the facility; a virtual space display processing unit that displays a first virtual space image corresponding to the first facility icon from a free viewpoint in response to an operation to specify a first facility icon from the one or more facility icons; and an operation receiving unit that receives an operation to specify the position of the viewpoint from which to display the first virtual space image. The virtual space display processing unit is characterized in that, when the operation receiving unit receives an operation to specify the position of the first viewpoint, it displays the virtual space image from the first viewpoint.

[0009] A "virtual space image" is an image that reproduces the real world, and may include, for example, a digital twin image or an image that constitutes the metaverse.

[0010] The "internal environment" refers to the internal environment of a facility, such as brightness, temperature, humidity, and sound, which changes over time and contribute to the realism of the virtual space image.

[0011] "Free viewpoint" refers to a viewpoint that the user can freely choose.

[0012] (2) In such a facility status display device, the internal environment may be at least one of the following: brightness inside the facility, temperature at one or more locations inside the facility, and sound at one or more locations inside the facility.

[0013] (3) In addition, such a facility status display device may further include a congestion determination unit that determines the current congestion status within the facility based on the people included in the generated virtual space image, and the map display processing unit may display an icon indicating the determined congestion status at a surrounding location including the location of the corresponding facility in the map information.

[0014] (4) In addition, in such a facility status display device, the virtual space generation unit may generate a virtual space image in which each person currently present in the facility is reproduced as an avatar.

[0015] An "avatar" is a character used as a representation of a person; it can be a character that resembles the person themselves, or it can be a character that does not resemble the person at all.

[0016] (5) The program of the present disclosure causes a computer to execute: a map display processing step that displays map information including one or more facility icons indicating the location of a facility on a map; a virtual space generation step that generates a virtual space image that reproduces the current state of the facility, including the interior space of the facility indicated by each of the one or more facility icons, people currently present in the facility, and the current interior environment of the facility; a virtual space display processing step that displays the virtual space image corresponding to the first facility icon from a free viewpoint in response to an operation to specify a first facility icon from the one or more facility icons; and an operation reception step that receives an operation to specify the position of the viewpoint from which to display the first virtual space image, wherein the virtual space display processing step displays the virtual space image from the first viewpoint when an operation to specify the position of the first viewpoint is received in the operation reception step.

[0017] (6) The facility status presentation method of the present disclosure includes: a map display processing step of displaying map information including one or more facility icons indicating locations of facilities on a map; a virtual space generation step of generating a virtual space image that reproduces the current status of the facility including the internal space of the facility indicated by each of the one or more facility icons, persons currently present in the facility, and the current internal environment of the facility; a virtual space display processing step of displaying, in response to an operation of designating a first facility icon among the one or more facility icons, the virtual space image corresponding to the first facility icon from a free viewpoint; and an operation reception step of receiving an operation of designating the position of a viewpoint for displaying the first virtual space image, wherein in the virtual space display processing step, when the operation of designating the position of the first viewpoint is received in the operation reception step, the virtual space image at the first viewpoint is displayed. [Advantageous Effects of Invention]

[0018] The facility status presentation device and the like of the present disclosure enable a user to grasp the current congestion status inside a facility. [Brief Description of Drawings]

[0019] [Figure 1] This is a schematic diagram illustrating an overview of a facility status presentation system according to an embodiment. [Figure 2] This is a functional block diagram illustrating the facility status presentation device of the embodiment. [Figure 3] This is a schematic diagram illustrating an example of map information according to the embodiment. [Figure 4] This is a schematic diagram illustrating an example of map information including a congestion status icon. [Figure 5] This is a schematic diagram illustrating an example of a virtual space image according to the embodiment. [Figure 6] This is a functional block diagram illustrating a facility terminal of the embodiment. [Figure 7] This is a functional block diagram illustrating a viewer terminal of the embodiment. [Figure 8]is a schematic diagram illustrating an example of a hardware configuration of a facility status presentation apparatus. [Figure 9] is a schematic diagram illustrating an example of a hardware configuration of a facility terminal. [Figure 10] is a schematic flow diagram illustrating an example of an operation of the facility status presentation apparatus according to the embodiment. [Mode for Carrying Out the Invention]

[0020] Hereinafter, a facility status presentation apparatus and the like according to one aspect of the present disclosure will be specifically described with reference to the drawings. Note that all the embodiments described below show one specific example of the present disclosure. Numerical values, shapes, constituent elements, arrangement positions of constituent elements, and the like shown in the following embodiments are mere examples, and are not intended to limit the present disclosure. Further, among the constituent elements in the following embodiments, constituent elements not described in the independent claims representing the highest concept will be described as optional constituent elements. Further, the contents of all the embodiments may be combined with each other.

[0021] Further, the following drawings are schematic, and part of the configurations and the like may be omitted for convenience of description. Further, portions common to one or a plurality of embodiments are denoted by the same reference numerals, and description thereof may be omitted.

[0022] [1. General Description] (Facility Status Presentation System 100) FIG. 1 is a schematic diagram illustrating an overview of the facility status presentation system 100 according to the embodiment.

[0023] First, the outline of the facility status display system 100 of this embodiment will be explained using Figure 1. The facility status display system 100 shown in Figure 1 provides a web map to the viewer terminal 3 used by the viewer 51. When a viewer 51 selects a facility on the web map, the facility status display system 100 provides a free-viewpoint virtual space simulating the interior space of the selected facility 2a, the interior environment of facility 2a, and the people currently present in facility 2a. By viewing the free-viewpoint virtual space, the viewer 51 can grasp the congestion level they would experience if they actually visited the facility.

[0024] The facility status display system 100 shown in Figure 1 comprises a facility status display device 1, one or more facilities 2a, and multiple viewer terminals 3. The facility status display device 1 and the multiple viewer terminals 3 are connected via a communication network 4. In addition, one or more facility terminals 2 can connect to the facility status display device 1 via the communication network 4 using the communication equipment of facility 2a.

[0025] (Facility status display device 1) The facility status display device 1 can acquire and display map information including one or more facility icons. Furthermore, the facility status display device 1 can acquire information about the interior space of facility 2a, information about people currently present in facility 2a, and information about the interior environment, and generate a virtual space that simulates the people currently present in facility 2a, the actual interior space of facility 2a, and the actual interior environment of facility 2a.

[0026] When the facility status display device 1 receives a designation of facility 2a on the displayed map information from, for example, the viewer terminal 3, it can display the current virtual space image of facility 2a on the viewer terminal 3 from a free viewpoint. Here, display from a free viewpoint means that, for example, when the position of a first viewpoint is specified in the virtual space image of facility 2a, the current virtual space image from the first viewpoint can be displayed.

[0027] (Virtual space image) A virtual space image is an image that reproduces the real world, and could be, for example, a digital twin image or an image that constitutes the metaverse. In other words, a virtual space image is an image of the real world or an image that simulates the real world, and is a virtual space image consisting of 2D or 3D spatial data.

[0028] (internal environment) The internal environment represents the environment and conditions inside the facility that change over time, such as brightness, temperature, humidity, sound, and the degree of sunlight, and gives a sense of realism to the virtual space image. In this embodiment, the internal environment is described as at least one of the following: brightness inside facility 2a, temperature at one or more locations inside facility 2a, and sound at one or more locations inside facility 2a.

[0029] (Facility 2a) Facility 2a includes, but is not limited to, cafeterias and dining halls. Any indoor facility that does not have a predetermined capacity and whose crowding levels fluctuate can be used, such as department stores, museums, aquariums, art galleries, science museums, pavilions, roadside rest areas, and indoor event venues.

[0030] (Facility Terminal 2) Facility terminal 2 can communicate with facility status display device 1. Facility terminal 2 uploads (transmits) information acquired by cameras, microphones, etc., installed within facility 2a to facility status display device 1. Facility terminal 2 may also acquire temperature and humidity within facility 2a and transmit them to facility status display device 1.

[0031] (Viewers: 51) Viewer 51 is a user who uses the facility status display device 1 via viewer terminal 3.

[0032] (Viewer terminal 3) Viewer terminal 3 is a terminal used by viewer 51 and is capable of communicating with facility status display device 1. Viewer terminal 3 can display map information, including one or more facility icons provided by facility status display device 1, on its display unit 33.

[0033] Furthermore, when facility 2a is selected on the displayed map information, the viewer terminal 3 can display a virtual space image of facility 2a from a free viewpoint. In this way, by displaying the virtual space image, the viewer terminal 3 can allow the viewer 51 to understand the current congestion status of facility 2a.

[0034] The viewer terminal 3 may be connected to the HMD (Head Mounted Display) 3a by wire or wireless connection. In this case, the viewer terminal 3 can allow viewer 51 to experience a virtual reality tour of facility 2a. Note that the HMD 3a is not limited to a head-mounted display; it may also be VR glasses.

[0035] (Communication Network 4) Communication network 4 is, for example, the Internet. Communication network 4 may also be an intranet, or a wired or wireless communication network. Wireless communication may include, for example, Bluetooth®, Wi-Fi®, or LPWA (Low Power Wide Area). As for LPWA, LTE® or frequency bands that do not require a license may be used.

[0036] [2. Detailed explanation] Figure 2 is a functional block diagram showing the facility status display device 1 of this embodiment. Figure 3 is a functional block diagram showing the facility terminal 2 of this embodiment.

[0037] (Facility status display device 1) The facility status display device 1 is a device capable of communicating with multiple facility terminals 2, and is, for example, a personal computer (PC). The facility status display device 1 may also be a server such as a web server. In this embodiment, as shown in Figure 2, the facility status display device 1 comprises a map display processing unit 11, a virtual space generation unit 12, a virtual space display processing unit 13, an operation reception unit 14, a congestion determination unit 15, and a storage unit 16. Note that the facility status display device 1 does not necessarily have to include the storage unit 16.

[0038] (Map display processing unit 11) The map display processing unit 11 displays map information including one or more facility icons indicating the location of facilities on the map. This map information includes buildings and roads, such as a web map, and shows a map of an area specified by the viewer 51. The map display processing unit 11 may also obtain and display map information including the area specified by the viewer 51 from an external server via a communication network 4 such as the internet. Furthermore, the map display processing unit 11 can display icons indicating the congestion status determined by the congestion determination unit 15 (described later) at a location in the vicinity of the map information, including the location of the corresponding facility 2a.

[0039] (Facility icon) Facility icons are icons that represent facilities displayed on a map, and are created using pictograms, icon graphics, or other methods.

[0040] The map display processing unit 11 may also provide the displayed map information to the viewer terminal 3 via the communication network 4. In this case, the map information will be displayed on the display unit 33 of the viewer terminal 3. The map display processing unit 11 may also display information about the facilities 2a included in the displayed map information. This information about facilities 2a may include, for example, the name of facility 2a, or a description of facility 2a.

[0041] Figure 3 is a schematic diagram showing an example of map information in the embodiment, and Figure 4 is a schematic diagram showing an example of map information including congestion status icons. Figures 3 and 4 show map information 61 displayed on the display unit 33 of the viewer terminal 3, and facility icons 611, 612, and 613 are shown. Also, Figures 3 and 4 show spot icons 611a, 612a, and 613a and a search window 614. The map information 61 includes a map of an area specified by the viewer 51, such as an area containing facilities searched by the search window 614, and shows the locations of buildings and roads. The spot icons 611a, 612a, and 613a indicate the facilities searched by the search window 614 and the spots around them.

[0042] Furthermore, Figure 4 shows congestion status icons 611b, 612b, and 613b around facility icons 611, 612, and 613, indicating the congestion level of facility 2a represented by facility icons 611, 612, and 613. Congestion status icons 611b, 612b, and 613b indicate the level of congestion in three stages. Other aspects are the same as in Figure 3, so the explanation is omitted.

[0043] (Virtual space generation unit 12) The virtual space generation unit 12 generates a virtual space image that reproduces the current state of facility 2a, including the interior space of facility 2a shown in one or more facility icons, the people currently present in facility 2a, and the current interior environment of facility 2a. When reproducing the people currently present in facility 2a, the virtual space generation unit 12 may generate a virtual space image in which each of the people currently present in facility 2a is reproduced as an avatar, or it may generate a virtual space image in which the people currently present are reproduced after privacy processing has been applied so that their faces are not recognizable.

[0044] More specifically, the virtual space generation unit 12 generates a virtual space image that mimics the interior space of facility 2a, the people currently present in facility 2a, and the current interior environment of facility 2a, based on 3D model data, which is data that 3D models the interior space of facility 2a. The virtual space generation unit 12 may store the generated virtual space image (or 3D model data) in the storage unit 16. In this case, the virtual space generation unit 12 can reproduce (reflect) the current interior environment of facility 2a and the people currently present in facility 2a, corresponding to a specified facility icon, in the generated virtual space image.

[0045] The virtual space generation unit 12 may acquire parts (components) for generating a virtual space image of facility 2a, such as stored captured images 22a and internal environment information 23a, from the storage unit 16, and generate a virtual space image that reproduces the internal space of facility 2a. The virtual space generation unit 12 may then use the captured images 22a and internal environment information 23a transmitted from the facility terminal 2 to reflect the people currently present in facility 2a and the current internal environment of facility 2a in the generated virtual space image. In this way, the virtual space generation unit 12 can generate a current virtual space image that reflects the people currently present in facility 2a and the current internal environment of facility 2a.

[0046] Alternatively, the virtual space generation unit 12 may reduce the load of real-time virtual space image generation by acquiring the previously generated virtual space image (or 3D model data) of facility 2a stored in the memory unit 16, and generating a current virtual space image that reflects the people currently present in facility 2a and the current internal environment of facility 2a.

[0047] (Virtual space display processing unit 13) The virtual space display processing unit 13 displays the virtual space image generated by the virtual space generation unit 12. More specifically, in response to an operation to specify a first facility icon from among one or more facility icons included in the map information, the virtual space display processing unit 13 displays the first virtual space image corresponding to the first facility icon from a free viewpoint.

[0048] The virtual space display processing unit 13 may also provide the displayed virtual space image to the viewer terminal 3 via the communication network 4. In this case, the display unit 33 of the viewer terminal 3 will display the first virtual space image from a free viewpoint.

[0049] Furthermore, the virtual space display processing unit 13 may display information related to facility 2a. This information could include, for example, the name of facility 2a, or a description of facility 2a. It may also display a congestion status icon.

[0050] Furthermore, when the operation reception unit 14 receives an operation to specify the position of the first viewpoint, the virtual space display processing unit 13 displays the virtual space image from the first viewpoint.

[0051] Figure 5 is a schematic diagram showing an example of a virtual space image of the embodiment. Figure 5 shows a virtual space image 62 of the interior of facility 2a displayed on the display unit 33 of the viewer terminal 3. As an example, Figure 5 shows a virtual space image 62 of a cafeteria attached to an indoor or outdoor sports facility, where people can freely enter and exit. The people present in the virtual space image 62 are reproduced as avatars 621, for example, so that individuals cannot be identified in consideration of privacy. In addition, an information window 622 showing information about facility 2a is displayed in this virtual space image 622. The information window 622 shows a congestion status icon 622b indicating a high level of congestion and facility information 622c indicating the name of the facility. This virtual space image 62 can be displayed from any viewpoint. Therefore, the viewer 51 can grasp the approximate level of congestion from the congestion status icon 622b and then freely change the viewpoint position to grasp the facility situation, including the actual congestion level, in real time.

[0052] The information window 622 may also display the current temperature and humidity of facility 2a. Furthermore, the virtual space image of facility 2a may include sounds such as the current noise levels of facility 2a, and these sounds may be played through user interaction.

[0053] (Operation reception unit 14) The operation reception unit 14 receives operations on the virtual space image of facility 2a displayed by the virtual space display processing unit 13. In this embodiment, the operation reception unit 14 receives an operation to specify the viewpoint position for displaying the virtual space image. This allows the current virtual space image to be displayed from various viewpoint positions, so that the viewer 51 can grasp the congestion status of facility 2a as if they were actually there.

[0054] (Congestion detection unit 15) The congestion determination unit 15 determines the current congestion level within facility 2a based on the people included in the current virtual space image. For example, the congestion determination unit 15 may determine the congestion level by comparing the number of people included in a virtual space image of facility 2a when it is quiet with the number of people included in the current virtual space image, or it may determine the congestion level based on whether the number of people included in the current virtual space image exceeds several threshold levels. Alternatively, the congestion determination unit 15 may determine the congestion level based on the number of people relative to the volume (or floor area) of facility 2a. The congestion determination unit 15 may also use AI trained on training data to acquire the number of people included in the virtual space image, the volume (or floor area) of facility 2a, etc.

[0055] In this embodiment, the congestion determination unit 15 determines the congestion level as, for example, three levels. The congestion determination unit 15 may then display congestion status icons corresponding to the determined congestion level, for example, by overlaying them on the map information as shown in Figure 4, or by displaying them in the information window 622 as shown in Figure 5.

[0056] (Storage unit 16) The storage unit 16 is composed of an HDD (Hard Disk Drive), an SDD (Solid State Drive), or memory. In this embodiment, the storage unit 16 stores, for example, an image 22a of the facility 2a and internal environment information 23a acquired from the facility terminal 2, or a virtual space image generated by the virtual space generation unit 12. The storage unit 16 also stores, for example, the number of people included in the virtual space image acquired by the congestion determination unit 15 and the volume (or floor area) of the facility 2a, or the congestion status determined by the congestion determination unit 15. The storage unit 16 may also store map information acquired by the map display processing unit 11.

[0057] The storage unit 16 may also store an image database (hereinafter referred to as the image database) 1714 in which captured images 22a of the interior space of the facility are stored, and an environment information database (hereinafter referred to as the environment information database) 1715 in which internal environment information 23a indicating the actual interior environment of the facility 2a is stored. The storage unit 16 may also store a virtual space image database (hereinafter referred to as the virtual space image database) 1716 in which virtual space images generated by the virtual space generation unit 12 are stored. Furthermore, the storage unit 16 may also store a congestion information database (hereinafter referred to as the congestion information database) 1717 in which the number of people included in the virtual space images acquired by the congestion determination unit 15, the volume (or floor area) of the facility 2a, and the congestion status determined by the congestion determination unit 15 are stored.

[0058] (Facility Terminal 2) Figure 6 is a functional block diagram showing the facility terminal 2 of this embodiment. Facility terminal 2 is, for example, a personal computer. Alternatively, facility terminal 2 may be a smartphone or tablet device. In this embodiment, as shown in Figure 3, facility terminal 2 comprises a transmission unit 21, an image acquisition unit 22, an environmental information acquisition unit 23, a display unit 24, and a storage unit 25. Facility terminal 2 is connected to the camera 2b and microphone 2c wirelessly or via a wired connection. Note that facility terminal 2 does not necessarily need to have a display unit 24.

[0059] (Transmitter 21) The transmitting unit 21 transmits information to the facility status display device 1 via the communication network 4. For example, the transmitting unit 21 transmits the captured image 22a acquired by the image acquisition unit 22 and the internal environmental information 23a acquired by the environmental information acquisition unit 23 to the facility status display device 1. The timing at which the transmitting unit 21 transmits the captured image 22a and the internal environmental information 23a to the facility status display device 1 may be each time, or it may be at predetermined intervals.

[0060] (Image acquisition unit 22) The image acquisition unit 22 acquires digital image data continuously captured by one or more cameras 2b installed in the facility 2a as captured image 22a.

[0061] (Environmental Information Acquisition Department 23) The environmental information acquisition unit 23 acquires audio data continuously picked up by one or more microphones 2c installed in facility 2a as internal environmental information 23a. The environmental information acquisition unit 23 also extracts brightness and other information inside facility 2a from digital image data from one or more cameras 2b installed in facility 2a or from one or more illuminometers installed in facility 2a, and acquires this as internal environmental information 23a. The environmental information acquisition unit 23 may also use AI trained on training data to extract brightness inside facility 2a from digital image data. The environmental information acquisition unit 23 may also acquire temperature and humidity inside facility 2a from one or more thermometers and hygrometers installed in facility 2a as internal environmental information 23a.

[0062] (Display section 24) The display unit 24 may be an image display device such as a liquid crystal display (LCD), plasma display panel (PDP), or organic electroluminescence (EL). Preferably, the display device has a touch panel function.

[0063] The display unit 24 can also display the captured image 22a acquired by the image acquisition unit 22 and the internal environmental information 23a acquired by the environmental information acquisition unit 23.

[0064] (Storage unit 25) The storage unit 25 is composed of an SSD or memory, etc. In this embodiment, the storage unit 25 stores, for example, the captured image 22a acquired by the image acquisition unit 22 and the internal environmental information 23a acquired by the environmental information acquisition unit 23.

[0065] (Camera 2b) Camera 2b consists of one or more cameras, is installed in facility 2a, and continuously captures images of the interior of facility 2a. Camera 2b outputs the captured images as digital image data. Camera 2b may be a wide-angle camera, such as a surveillance camera. The digital image data captured by camera 2b includes attribute information such as facility ID, camera identifier, date of capture, and time of capture. The facility ID is used to identify the captured image 22a received from facility 2a. The camera identifier indicates the identifier of the camera 2b that captured the image, and is used to identify which of the one or more cameras 2b installed in facility 2a captured the image. The date of capture and time of capture indicate the date and time the image was taken.

[0066] (Mic 2c) Microphone 2c consists of one or more microphones and is installed in facility 2a. Microphone 2c picks up the voices of one or more people present in facility 2a, as well as sounds other than human voices, such as background music playing in facility 2a, and outputs them as audio data.

[0067] (Viewer terminal 3) Figure 7 is a functional block diagram showing the viewer terminal 3 of the embodiment.

[0068] The viewer terminal 3 is, for example, a smartphone. Alternatively, the viewer terminal 3 may be a wearable computer, a mobile phone, a tablet device, or a personal computer. In this embodiment, as shown in Figure 7, the viewer terminal 3 comprises a transmitting / receiving unit 31, an operating unit 32, a display unit 33, and a storage unit 34. The viewer terminal 3 is connected to the HMD 3a, but it does not necessarily have to be connected.

[0069] (Transmitting / receiving unit 31) The transmitting / receiving unit 31 transmits and receives information with the facility status display device 1 via the communication network 4. For example, the transmitting / receiving unit 31 receives map information and virtual space images displayed by the facility status display device 1 and displays them on the display unit 33. The transmitting / receiving unit 31 also transmits operation information for the map information and virtual space images displayed on the display unit 33 to the facility status display device 1.

[0070] (Operation unit 32) The operation unit 32 can perform operations on the map information and virtual space images displayed on the display unit 33 in response to operations by the viewer 51. Specifically, the operation unit 32 performs operations on the virtual space images by transmitting operation information for the map information and virtual space images displayed on the display unit 33 to the facility status display device 1 via the transmission / reception unit 31. Furthermore, if the HMD 3a is connected to the viewer terminal 3, the operation unit 32 can receive operations on the HMD and perform operations on the virtual space images and other content displayed on the HMD 3a.

[0071] (Display section 33) The display unit 33 may be an image display device such as a liquid crystal display (LCD), plasma display (PDP), or organic light-emitting diode (EL). Preferably, the display device has a touch panel function.

[0072] The display unit 33 displays map information, for example, as shown in Figure 3 or Figure 4, or virtual space images, for example, as shown in Figure 5. The display unit 33 also displays operations for the displayed map information or virtual space images.

[0073] (Storage unit 34) The storage unit 34 is composed of an SSD or memory, etc. In this embodiment, the storage unit 34 stores, for example, map information and virtual space images received by the transmitting / receiving unit 31. The storage unit 34 also stores operation information for the operation unit 32, such as operation information for the map information and virtual space images.

[0074] (HMD3a) The HMD3a is worn by the viewer 51. The HMD3a is equipped with speakers and may output sounds from within the facility 2a, such as the noise of people, as internal environmental information for the virtual space image. Note that the HMD3a is not limited to a head-mounted display; it may also be VR glasses or the like.

[0075] [3. Hardware Configuration] Figure 8 is a schematic diagram showing an example of the hardware configuration of the facility status display device 1. Figure 9 is a schematic diagram showing an example of the hardware configuration of the facility terminal 2.

[0076] Next, the hardware configurations of the facility status display device 1 and the facility terminal 2 will be explained using Figures 8 and 9.

[0077] (Hardware configuration of facility status display device 1) The hardware configuration of the facility status display device 1 is described below. As shown in Figure 8, the facility status display device 1 of this embodiment uses, for example, a computer. The facility status display device 1 is equipped with a CPU 170. The CPU 170 is connected to a memory 171, a connection port 173 for connecting / reading storage devices 172, etc., and a communication circuit 174 for communicating with the outside via a network, all via a bus line 175. The memory 171 stores a program 1713 for processing the facility status display device 1. It may also store a browser program 1712 and an OS 1711 (operating system). Furthermore, the memory 171 may store an image database 1714, an environmental information database 1715, a virtual space image database 1716, and a congestion information database 1717. Note that the memory 171 may be a storage unit 16, or it may be something different from the storage unit 16. If memory 171 and storage unit 16 are different, the captured image DB 1714, environmental information DB 1715, virtual space image DB 1716, and congestion information DB 1717 are stored in storage unit 16.

[0078] (Hardware configuration of facility terminal 2) The hardware configuration of facility terminal 2 is almost the same as that of facility status display device 1 described above, so the same parts are denoted by the same reference numerals, and their explanation is omitted. In this embodiment, facility terminal 2 uses, for example, a personal computer.

[0079] The computer's memory 171 stores a program 1718 for processing the facility terminal 2. It may also store a browser program 1712 and an OS 1711 (operating system).

[0080] In this embodiment, programs 1713 and 1718 work together by utilizing the functions of OS 1711 and browser program 1712, respectively. Alternatively, programs 1713 and 1718 may operate independently without utilizing browser program 1712 and OS 1711, respectively.

[0081] In the hardware configuration of the facility status display device 1 and facility terminal 2 described above, the functions shown in Figures 2 and 3 are implemented using, for example, a CPU 170, program 1713, and program 1718. However, some or all of these functions may be sequentially controlled using logic circuits such as a microcontroller or a PLC (programmable logic controller).

[0082] (Image DB1714) As described above, the image database 1714 stores the captured images 22a acquired from the facility terminal 2. The image database 1714 stores the facility ID, camera identifier, date of capture, and time of capture for each captured image 22a. The multiple captured images 22a stored in the image database 1714 are components (parts) for generating a virtual space image of facility 2a.

[0083] (Environmental information DB1715) As described above, the environmental information DB 1715 stores internal environmental information 23a acquired from the facility terminal 2, linked to the captured image 22a. For each piece of internal environmental information 23a, the environmental information DB 1715 stores the facility ID from which the internal environmental information was acquired, the identifier of the associated captured image 22a, and the date and time the internal environmental information was acquired. The multiple pieces of internal environmental information 23a stored in the environmental information DB 1715 are parts (components) for generating a virtual space image of facility 2a.

[0084] (Virtual Space Image DB1716) The virtual space image DB1716 stores virtual space images generated by the virtual space generation unit 12. The virtual space image DB1716 may also store 3D model data, which is data that is a three-dimensional model of the interior space and the internal environment of facility 2a, instead of virtual space images. This makes it possible to generate the current virtual space image from 3D model data rather than from parts (components) used to generate the virtual space image of facility 2a, thereby reducing the load on virtual space image generation.

[0085] (Congestion Information DB1717) The congestion information DB1717 stores congestion information such as the number of people included in the virtual space image acquired by the congestion determination unit 15, the volume (or floor area) of the facility 2a, and the congestion status determined by the congestion determination unit 15.

[0086] The hardware configuration of the viewer terminal 3 in this embodiment is the same as that of the facility terminal 2 described above, so its explanation will be omitted. In this embodiment, the viewer terminal 3 uses, for example, a smartphone.

[0087] [4. Operation of Facility Status Display Device 1] (Operation of facility status display device 1) Next, the operation of the facility status display device 1 will be explained using Figure 10. Figure 10 is a schematic flow showing an example of the operation of the facility status display device 1 in this embodiment. Figure 12 shows a flowchart showing one embodiment of the processing of program 1713 used in the facility status display device 1.

[0088] (S1) The CPU 170 of the facility status display device 1 displays map information including one or more facility icons indicating the location of facility 2a on the map, as shown in Figure 3, for example. In this embodiment, the facility status display device 1 acquires and displays map information from an external server via a communication network 4 such as the Internet. The facility status display device 1 may also display congestion status icons, as shown in Figure 4, at a location in the vicinity of the corresponding facility 2a on the map information. The facility status display device 1 may also provide map information to the viewer terminal 3 and have it displayed on the display unit 33 of the viewer terminal 3.

[0089] (S2) The CPU 170 of the facility status display device 1 generates a virtual space image that reproduces the current status of facility 2a, including the internal space of facility 2a shown in each of the one or more facility icons, the people currently present in facility 2a, and the current internal environment of facility 2a.

[0090] (S3) In response to an operation to select a first facility icon from among one or more facility icons, the CPU 170 of the facility status display device 1 displays a virtual space image corresponding to the first facility icon from a free viewpoint, as shown in Figure 5, for example. The facility status display device 1 may also provide the generated virtual space image to the viewer terminal 3 and have it displayed on the display unit 33 of the viewer terminal 3.

[0091] (S4) The CPU 170 of the facility status display device 1 accepts an operation to specify the position of the viewpoint for displaying the first virtual space image.

[0092] (S5) If the CPU 170 of the facility status display device 1 receives an operation in step S4 to specify the position of the first viewpoint, it displays a virtual space image from the first viewpoint.

[0093] [5. Others] The above embodiments can be used in combination as appropriate. For example, the virtual space image generated by the facility status display device 1 is not limited to a still image; it may also be a video.

[0094] Furthermore, the facility status display device 1 may generate not only current virtual space images but also past virtual space images. In this case, it is possible to understand not only the current congestion status but also past congestion status.

[0095] Furthermore, for example, the facility status display device 1 may operate a reservation site for facilities including facility 2a on a website, and make seats in the displayed virtual space image available for reservation. Alternatively, for example, the facility status display device 1 may overlay information indicating whether a seat is available for reservation (i.e., there are seats available for reservation) on the seats in the virtual space image showing the interior space of facility 2a. When a seat in the virtual space image is selected, the reservation process may be performed.

[0096] Furthermore, although the congestion determination unit 15 was described as determining the congestion level as a three-level congestion level, it may also have four or more levels. In addition, the congestion status icon corresponding to the determined congestion level does not have to be displayed in the information window 622 as shown in Figure 5, but may be displayed in any part of the display screen.

[0097] [6. Summary] (1, 5, 6) The facility status display device 1 comprises: a map display processing unit 11 that displays map information including one or more facility icons indicating the location of a facility on a map; a virtual space generation unit 12 that generates a virtual space image that reproduces the current status of facility 2a, including the internal space of facility 2a indicated by each of the one or more facility icons, people currently present in facility 2a, and the current internal environment of facility 2a; a virtual space display processing unit 13 that displays a first virtual space image corresponding to the first facility icon from a free viewpoint in response to an operation to specify a first facility icon from among the one or more facility icons; and an operation reception unit 14 that receives an operation to specify the position of the viewpoint from which to display the first virtual space image. The virtual space display processing unit 13 is characterized in that, when the operation reception unit 14 receives an operation to specify the position of the first viewpoint, it displays the virtual space image from the first viewpoint.

[0098] In this way, the facility status display device 1 can, for example, display map information from a web map, and when a facility is selected on the map information, it can display the interior space of the selected facility 2a, the interior environment of facility 2a, and a virtual space image simulating people currently present in facility 2a, which can be viewed from any viewpoint.

[0099] Furthermore, because the facility status display device 1 can display virtual space images from any viewpoint, viewers 51 of the virtual space images can see the density of people present in facility 2a from various angles, thus understanding the level of congestion they would experience if they actually visited the facility.

[0100] More specifically, when viewers 51 perceive congestion levels through congestion maps with human data or real-time live video feeds, the perceived congestion levels sometimes differ from those they would experience when actually visiting the facility. In contrast, the free-viewpoint virtual space images presented by the facility status display device 1 allow viewers 51 to grasp the current congestion levels, which match those they would experience when actually visiting the facility. This allows viewers 51 to understand the current congestion levels based on their own perception and decide whether or not to visit the facility 2a.

[0101] Furthermore, the displayed virtual space image reproduces (reflects) the internal environment of facility 2a, allowing viewers 51 to experience the atmosphere, such as the hustle and bustle inside facility 2a, through a realistic virtual space image. This allows viewers 51 to grasp the congestion situation more intuitively and decide whether or not to visit facility 2a.

[0102] (2) In such a facility status display device 1, the internal environment may be at least one of the following: brightness inside facility 2a, temperature at one or more locations inside facility 2a, and sound at one or more locations inside facility 2a. This allows viewers 51 to check the congestion status of facility 2a, including brightness, temperature, noise, etc., in a realistic virtual space image. This allows viewers 51 to understand whether the congestion status of facility 2a, including the temperature and noise level, is acceptable according to their own senses, and to decide whether or not to visit facility 2a.

[0103] (3) Furthermore, such a facility status display device 1 may also include a congestion determination unit that determines the current congestion level within facility 2a based on the people included in the generated virtual space image, and the map display processing unit 11 may display an icon indicating the determined congestion level at a location in the vicinity of the corresponding facility 2a in the map information. This allows the viewer 51 to get an idea of ​​the congestion level of facility 2a by looking at the congestion status icon displayed around the facility icon of facility 2a in the map information, before grasping the actual level of congestion from the virtual space image. In addition, the viewer 51 can get an idea of ​​the level of congestion by looking at the congestion status icon and then decide whether or not to visit facility 2a by grasping the actual level of congestion from the virtual space image.

[0104] (4) In addition, in such a facility status display device 1, the virtual space generation unit 12 may generate a virtual space image in which each person currently present in facility 2a is reproduced as an avatar. This allows the facility status display device 1 to generate a virtual space image in which the people currently present in facility 2a are reproduced after privacy processing has been applied so that people's faces are not recognizable. [Explanation of Symbols]

[0105] 1. Facility status display device 2 Facility terminals 2a Facility 2b camera 2c Microphone 3 Viewer terminal 3a HMD 4. Communication Network 11 Map display processing unit 12. Virtual Space Generation Unit 13. Virtual Space Display Processing Unit 14 Operation Reception Section 15. Congestion detection unit 16, 25, 34 storage section 21 Transmitter 31 Transmitter / Receiver 22 Image acquisition unit 22a Acquired image 23 Environmental Information Acquisition Department 23a Internal environment information 24, 33 Display section 32 Operation section 51 viewers 61 Map Information 62 Virtual Space Images 100 Facility Status Display System 170 CPU 171 memory 172 Storage Devices 173 connection ports 174 Communication Circuits 175 Bus Line 611, 612, 613 Facility Icons 611a, 612a, 613a Spot Icons 611b, 612b, 613b, 622b Congestion status icons 614 Search Window 621 Avatars 622 Information Window 622c Facility Information 1711 OS 1712 Browser Program Programs 1713 and 1718 1714 Image Database 1715 Environmental information DB 1716 Virtual Space Image Database 1717 Congestion Information Database

Claims

1. A map display processing unit that displays map information including one or more facility icons indicating the location of facilities on a map, A virtual space generation unit generates a virtual space image that reproduces the current state of the facility, including the interior space of the facility shown in each of the one or more facility icons, the people currently present in the facility, and the current interior environment of the facility. A virtual space display processing unit that, in response to an operation to select a first facility icon from the one or more facility icons mentioned above, displays a first virtual space image corresponding to the first facility icon from a free viewpoint, It includes an operation receiving unit that accepts an operation to specify the position of the viewpoint for displaying the first virtual space image, When the operation receiving unit receives an operation to specify the position of the first viewpoint, the virtual space display processing unit displays the virtual space image at the first viewpoint. Facility status display device.

2. The aforementioned internal environment is at least one of the following: brightness within the facility, temperature at one or more locations within the facility, and sound at one or more locations within the facility. The facility status display device according to claim 1.

3. Furthermore, the facility includes a congestion determination unit that determines the current congestion status within the facility based on the people included in the generated virtual space image. The map display processing unit displays an icon indicating the determined congestion status at a surrounding location including the location of the corresponding facility in the map information. The facility status display device according to claim 1 or 2.

4. The virtual space generation unit generates virtual space images that reproduce each person currently present in the facility as an avatar. The facility status display device according to claim 1 or 2.

5. A map display processing step that displays map information including one or more facility icons indicating the location of facilities on a map, A virtual space generation step generates a virtual space image that reproduces the current state of the facility, including the interior space of the facility shown in each of the one or more facility icons, the people currently present in the facility, and the current interior environment of the facility. A virtual space display processing step in which, in response to an operation to specify a first facility icon from among the one or more facility icons, a virtual space image corresponding to the first facility icon is displayed from a free viewpoint, An operation acceptance step is performed on the computer to accept an operation to specify the position of the viewpoint for displaying the first virtual space image. In the virtual space display processing step, if an operation to specify the position of the first viewpoint is received in the operation reception step, the virtual space image at the first viewpoint is displayed. program.

6. A map display processing step that displays map information including one or more facility icons indicating the location of facilities on a map, A virtual space generation step generates a virtual space image that reproduces the current state of the facility, including the interior space of the facility shown in each of the one or more facility icons, the people currently present in the facility, and the current interior environment of the facility. A virtual space display processing step in which, in response to an operation to specify a first facility icon from among the one or more facility icons, a virtual space image corresponding to the first facility icon is displayed from a free viewpoint, Includes an operation acceptance step that accepts an operation to specify the position of the viewpoint for displaying the first virtual space image, In the virtual space display processing step, if an operation to specify the position of the first viewpoint is received in the operation reception step, the virtual space image at the first viewpoint is displayed. Method for presenting facility status.

Images