Disaster information display device, disaster information display system, and control method and program for disaster information display device

Abstract

To provide a disaster information display device, a disaster information display system, and a control method of the disaster information display device, and a program which can effectively transmit disaster information to a user while reducing a processing load.SOLUTION: A user terminal 20 as a disaster information display device comprises: a position acquisition unit 41 which acquires position information; a data acquisition unit 42 which transmits the position information acquired by the position acquisition unit 41 to an external server 10 in which 3D model data related information in which 3D model data which is configured such that height information on a position is included in map information indicating a warning area of a predetermined range where disaster is forecasted, 3D model data position information indicating a position of the 3D model data, and azimuth information are associated with each other, is stored, and acquires 3D model related information; and a display processing unit 43 which determines a direction of the 3D model data on the basis of the azimuth and an elevation angle of an imaging device 2, superimposes the data on a real-time image being captured by an imaging device 28, and displays the image on a display device 27.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a disaster information display device, a disaster information display system, a control method for a disaster information display device, and a program. [Background technology]

[0002] Conventionally, a technique for superimposing information about disasters and the like on an actually captured image and displaying it is known. Patent Document 1 describes this type of technique.

[0003] Patent Document 1 relates to a synthetic image display device that displays a virtual space image superimposed on a real-life image captured of a real space. The synthetic image display device disclosed in Patent Document 1 includes an image capture means for capturing a real-life image, a position measurement means for measuring the planar position of the image capture means, an orientation measurement means for measuring the orientation of the image capture means, a storage means for storing a space model in which a predetermined area is planarly divided into multiple meshes and in which flood depths are assigned, an extraction means for extracting a space model of the mesh corresponding to the planar position measured by the position measurement means, an image creation means for creating a virtual space image based on the space model extracted by the extraction means, and a display means for displaying a superimposed image of the real-life image and the virtual space image. Patent Document 1 also discloses that the image creation means performs arithmetic processing based on the imaging height of the image capture means and the orientation of the image capture means measured by the orientation measurement means, and creates a virtual space image on the display means so that it appears as a perspective view. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5862865 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, it was necessary to store in advance on the display device a program for extracting a spatial model of a mesh corresponding to the planar position measured by the position measurement means and executing the process of creating a virtual spatial image based on the spatial model, and there was room for improvement in terms of reducing the processing burden on the disaster information display device that displays disaster information.

[0006] The present invention aims to provide a disaster information display device, a disaster information display system, a control method for a disaster information display device, and a program that can effectively convey disaster information to users while reducing the processing load. [Means for solving the problem]

[0007] The present invention relates to a disaster information display device that includes an imaging device that captures images and a display device that displays various information, and that includes: a position acquisition unit that acquires position information; a data acquisition unit that transmits the position information acquired by the position acquisition unit to an external server that stores 3D model data configured such that position elevation information is included in map information that indicates a warning zone within a predetermined range where a disaster is expected; 3D model data position information that indicates the position of the 3D model data; and 3D model data-related information that is associated with orientation information, and acquires the 3D model data-related information extracted by the server based on the position information and the 3D model data position information; and a display processing unit that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on the display device.

[0008] The display processing unit may perform a process of superimposing information about the disaster together with the 3D model data on a real-time image being captured by the imaging device.

[0009] When the display processing unit receives a user operation to change height information of the position, the display processing unit may perform a process to change the size of the 3D model data based on the received operation.

[0010] When the display processing unit receives a user operation to change the elevation angle, the display processing unit may perform processing to change the position where the 3D model data is displayed based on the received operation.

[0011] The display processing unit may perform processing to display an image in which 3D model data is superimposed on a real-time image captured by the imaging device, together with a two-dimensional map image that includes the same location as the 3D model data.

[0012] The present invention also relates to a disaster information display system comprising: 3D model data configured so that position elevation information is included in map information indicating a warning zone within a predetermined range where a disaster is expected; a server in which 3D model data-related information associated with 3D model data position information indicating the position of the 3D model data and orientation information is stored; and a disaster information display device having an imaging device that captures images and a display device that displays various information, wherein the disaster information display device comprises: a position acquisition unit that acquires position information; a data acquisition unit that transmits the position information acquired by the position acquisition unit to the server and acquires the 3D model data-related information extracted by the server based on the position information and the 3D model data position information; and a display processing unit that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image with the 3D model data superimposed on the display device.

[0013] The present invention also relates to a control method for a disaster information display device that is equipped with an imaging device that captures images and a display device that displays various information, the control method including: a position information acquisition step that acquires position information; a data transmission step that transmits the position information acquired in the position information acquisition step to an external server that stores 3D model data configured such that position elevation information is included in map information that indicates a warning zone within a predetermined range where a disaster is expected, 3D model data position information that indicates the position of the 3D model data, and 3D model data-related information that is associated with orientation information; a data acquisition step that acquires the 3D model data-related information extracted by the server based on the position information and the 3D model data position information; and a display processing step that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on the display device.

[0014] The present invention also relates to a program that causes a computer to execute: a location information acquisition function that acquires location information; a data transmission function that transmits the location information acquired by the location information acquisition function to an external server that stores 3D model data configured such that location height information is included in map information that indicates a predetermined range of alert zones where disasters are expected; 3D model data location information that indicates the position of the 3D model data; and 3D model data-related information that is associated with orientation information; a data acquisition function that acquires the 3D model data-related information extracted by the server based on the location information and the 3D model data location information; and a display processing function that determines the orientation of the 3D model data based on the orientation and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image with the 3D model data superimposed on it on a display device. [Effects of the Invention]

[0015] According to the present invention, it is possible to provide a disaster information display device, a disaster information display system, a control method for a disaster information display device, and a program that can effectively convey disaster information to users while reducing the processing load. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a schematic diagram of a disaster information display system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server according to the present embodiment. [Figure 3] FIG. 2 is a block diagram showing an example of a hardware configuration of a user terminal according to the present embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of a functional configuration for displaying disaster information of the disaster information display system of the present embodiment. [Figure 5] 3A and 3B are schematic diagrams illustrating the relationship between a model search range and 3D model data in the search processing function of this embodiment. [Figure 6] 10A and 10B are schematic diagrams illustrating the relationship between a model search range, which is wider than the model normal display range, of the search processing function of this embodiment and 3D model data. [Figure 7] 10 is a flowchart showing an example of the flow of a disaster display process according to the present embodiment. [Figure 8] FIG. 2 is a schematic diagram of an AR image displayed on a user terminal according to the present embodiment. [Figure 9] FIG. 10 is a schematic diagram illustrating an AR image according to the present embodiment in which area information is added. [Figure 10] FIG. 10 is a schematic diagram illustrating an AR image in which a map image is added to the AR image of the present embodiment. [Figure 11] 10A and 10B are schematic diagrams for explaining the position adjustment of an AR image according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0018] 1 is a schematic diagram of a disaster information display system 1 according to one embodiment of the present invention. As shown in FIG. 1, the disaster information display system 1 of this embodiment includes a GIS (Geographic Information System) 3, a server 10, and a user terminal 20.

[0019] GIS3 is a geographic information system that creates map information and related information on a computer. Using GIS3 and 3D CAD software4, 3D model data of designated warning zones where disasters such as landslides, tornadoes, heavy rain, and tsunamis are predicted, as well as attribute information corresponding to the 3D model data, are generated.

[0020] The GIS 3 of this embodiment generates 3D model data in a format such as glb based on map information, which is vector data (two-dimensional data) such as SHAPE format for the restricted area and special restricted area, and elevation data, which is a digital elevation model such as DEM (digital elevation model). That is, the 3D model data is configured by including location elevation information in map information indicating the restricted area. The GIS 3 also generates attribute information, which is disaster-related information corresponding to the 3D model data, such as latitude, longitude, elevation, and area name, in text format such as CSV (comma-separated values), along with the 3D model data. The GIS 3 also generates 3D model data position information and orientation information indicating the position of the 3D model data. The generated 3D model data and attribute information are sent to the server 10. At this time, the 3D model data, along with the 3D model data position information and orientation information, are sent to the server 10 as 3D model data-related information that is associated with each other. The 3D model data position information may be, for example, information indicating the position of a reference point within the 3D model data. The reference point may be, for example, the center of the 3D model data, or a point other than the center. The GIS 3 may be installed as software on the server 10, or may be installed as software on an external computer, or may function as a server. The map information and elevation data may be obtained, for example, by downloading open data such as government statistics from a website or by using a medium.

[0021] Next, the server 10 will be described. The server 10 is a computer that performs processing to transmit information for displaying disaster information in AR (Augmented Reality) to a user terminal 20 based on 3D model data and attribute information. The server 10 and the user terminal 20 are connected to each other so as to be able to communicate with each other via a network 2 such as the Internet.

[0022] Fig. 2 is a block diagram showing an example of the hardware configuration of the server 10 of this embodiment. As shown in Fig. 2, the server 10 includes a processor 11, a read-only memory (ROM) 12, a random-access memory (RAM) 13, an auxiliary storage device 14, and a communication interface (I / F) 15, and each part is connected by a bus or the like.

[0023] The processor 11 is a central part of a computer that performs various calculations, controls, and other processes required for the operation of the server 10. The processor 11 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 11 may be a combination of two or more of these. The processor 11 may also be a combination of these with a hardware accelerator or the like.

[0024] The processor 11 controls each unit to realize various functions based on programs such as firmware, system software, and application software stored in the ROM 12 or the auxiliary storage device 14. The processor 11 also executes the processes described below based on the programs. Note that some or all of the programs may be incorporated into the circuitry of the processor 11.

[0025] ROM 12 and RAM 13 are main storage devices of a computer centered around processor 11. ROM 12 is a non-volatile memory used exclusively for reading data. ROM 12 stores, for example, firmware among the above programs. ROM 12 also stores data used by processor 11 when performing various processes. RAM 13 is a memory used for reading and writing data. RAM 13 is used as a work area or the like for storing data temporarily used by processor 11 when performing various processes. RAM 13 is typically a volatile memory.

[0026] The auxiliary storage device 14 is, for example, an EEPROM (electric erasable programmable read-only memory), an HDD (hard disk drive), or a flash memory. The auxiliary storage device 14 stores, for example, system software and application software among the above programs. The auxiliary storage device 14 also stores data used by the processor 11 when performing various processes, data generated by the processes in the processor 11, various setting values, and the like.

[0027] The communication I / F 15 is an interface for communicating with an external device such as a user terminal 20.

[0028] Next, we will explain the user terminal 20. The user terminal 20 is a disaster information display device that displays disaster information superimposed on an actually captured image. Note that the images referred to here are images that are continuous in time, and include moving images.

[0029] The following describes the hardware configuration of the user terminal 20. Fig. 3 is a block diagram showing an example of the hardware configuration of the user terminal 20 of this embodiment.

[0030] As shown in Figure 3, the user terminal 20 includes a processor 21, a ROM 22, a RAM 23, an auxiliary storage device 24, a communication I / F 25, an input device 26, a display device 27, an imaging device 28, a GNSS (global navigation satellite system) device 29, and a sensor 30, and each part is connected by a bus or the like.

[0031] Of the components of the user terminal 20, the processor 21, ROM 22, RAM 23, auxiliary storage device 24, and communication I / F (interface) 25 are configured similarly to the processor 11, ROM 12, RAM 13, auxiliary storage device 14, and communication I / F 15 of the server 10, respectively.

[0032] The input device 26 is a means for accepting user operations, and the display device 27 is a means for displaying various types of information to the user. In this embodiment, the input device 26 and the display device 27 are configured by touch panel displays. The input device 26 may be a type that inputs voice using a microphone. Note that the various types of information displayed by the display device 27 include, for example, images captured by the imaging device 28, 3D model data, attribute information, icons for accepting user operations, and the like.

[0033] The imaging device 28 is a camera configured with an optical lens, an image sensor, etc. The GNSS device 29 is means for locating the position of the user terminal 20 using GNSS. Examples of GNSS that are used include GPS, GLONASS, Galileo, and quasi-zenith satellites. The GNSS device 29 of this embodiment includes an antenna and receives positioning satellite signals from multiple positioning satellites to acquire position information of the user terminal 20. Note that the GNSS device 29 may be configured to be external to the user terminal 20 and acquire the position information of the user terminal 20 by wireless or wired communication with a GNSS terminal that receives the positioning satellite signals.

[0034] The sensor 30 is composed of various sensors (such as an acceleration sensor and an angular velocity sensor) and a magnetic sensor for measuring the three-dimensional movement of the user terminal 20 itself. The sensor 30 detects the acceleration and angular velocity applied to the user terminal 20 in response to the movement, and outputs the detected acceleration and angular velocity as sensor information. The orientation of the user terminal 20 is identified based on this sensor information. The sensor 30 also functions as a compass for identifying the direction.

[0035] Next, a process for displaying disaster information will be described. Fig. 4 is a block diagram showing an example of a functional configuration for displaying disaster information in the disaster information display system 1 of this embodiment.

[0036] As shown in FIG. 4, in the server 10, a model information database 35 and a 3D model database 36 are constructed by the auxiliary storage device 14, an external storage device, or the like.

[0037] The model information database 35 is a database that stores attribute information generated by the GIS 3. In this embodiment, the latitude, longitude, and search range of the current location of the user terminal 20 are set as parameters for calling attribute information as return values ​​from the model information database 35. In addition, for example, the model type, model data proxy parameters, model latitude, model longitude, model elevation, model color, natural phenomenon name, area number, area name, county, city, ward, town, village, and district are set as return values.

[0038] The 3D model database 36 stores 3D model data-related information that associates 3D model data to be superimposed on an image with 3D model data position information and orientation information. The 3D model data is created based on a specific orientation, such as north. In this embodiment, 3D model data in a format such as glb generated by the GIS 3 is converted into 3D model data in a format such as glTF, and the converted data is stored in the 3D model database 36.

[0039] The server 10 also includes, as functional units executed by the processor 11, a location acceptance unit (location acceptance function) 31 and a data processing unit (data processing function) 32.

[0040] The position receiving unit 31 executes a process of acquiring position information of the user terminal 20 that displays the AR image. The data processing unit 32 executes a process of transmitting 3D model data related information and attribute information to the user terminal 20 for displaying the AR image on the user terminal 20.

[0041] The data processing unit 32 of this embodiment includes a search processing unit 33 and a proxy unit 34. The search processing unit 33 is a search API (Application Programming Interface) that executes processing to identify 3D model data within a search range set based on the current location of the user terminal 20 and transmit attribute information corresponding to the identified 3D model data to the user terminal 20. The search processing unit 33 identifies 3D model data-related information by, for example, comparing the location information of the user terminal 20 with the 3D model data location information.

[0042] An example of a method for searching for 3D model data by the search processing unit 33 will be described with reference to Fig. 5. Fig. 5 is a schematic diagram illustrating the relationship between the model search range and 3D model data in the search processing function of this embodiment. Fig. 5 shows a search range centered on the latitude and longitude of the current location, and a model normal display range in which a 3D data model can be displayed correctly in an AR image. In Fig. 5, the search range is shown as a solid rectangle, and its center is set at the current location. The model normal display range is shown as a dashed circle, and its center is set at the current location. In the example of Fig. 5, the model search range is set so as to be inside the model normal display range.

[0043] In the 3D model data, a restricted area and a special restricted area are set. The restricted area is an area where the user should be alerted, and a model origin is set as a reference. The special restricted area is an area where the user should be even more alert than the restricted area, and a model origin separate from the restricted area is set. The special restricted area is an area inside the restricted area.

[0044] If either the model origin of the restricted area or the special restricted area is within the search range, it will be the 3D model data to be displayed in the AR image. In Figure 5, a total of four 3D model data items, 3D model A, 3D model B, 3D model C, and 3D model D, are shown inside the model normal display range. 3D model A is included in the display range because both the model origin of the restricted area and the model origin of the special restricted area are inside the model search range. 3D model B is included in the display range because the model origin of the restricted area is outside the search range, but the model origin of the special restricted area is inside the model search range. 3D model C is included in the display range because the model origin of the special restricted area is outside the search range, but the model origin of the restricted area is inside the model search range. On the other hand, 3D model D is not included in the display range because both the model origin of the restricted area and the model origin of the special restricted area are outside the model search range.

[0045] In the example of Fig. 5, the model search range is set so as to be included in the model normal display range, but the model search range may also be set so as to include outside the model normal display range. Fig. 6 is a schematic diagram illustrating the relationship between the model search range, which is wider than the model normal display range of the search processing function of this embodiment, and the 3D model data.

[0046] In the example of Figure 6, a total of three pieces of 3D model data are displayed inside the model search range: 3D model E, 3D model F, and 3D model G. 3D model E and 3D model F are both inside the model normal display range, and both the model origin of the restricted area and the model origin of the special restricted area are inside the model search range, so they are the 3D model data to be displayed correctly.

[0047] Although the model origin of the restricted area is outside the model search range, 3D model G is included in the display target because the model origin of the restricted area is inside the model search range. However, because both the model origin of the restricted area and the model origin of the special restricted area are outside the model normal display range, the 3D model data will not be displayed correctly on the user terminal 20. For example, it will be displayed as a distorted shape on the AR image.

[0048] Next, a description will be given of the proxy unit 34. In response to a data request from the user terminal 20, the proxy unit 34 transmits corresponding 3D model data to the user terminal 20.

[0049] If a cache file of the corresponding 3D model data does not exist, the proxy unit 34 of this embodiment acquires the 3D data model from the 3D model database 36, transmits the 3D model data to the user terminal 20, and saves the 3D model data as a cache file. If a cache file of the corresponding 3D model data exists, the proxy unit 34 transmits the 3D model data from the cache file to the user terminal 20.

[0050] Next, a description will be given of the functions of the user terminal 20. The user terminal 20 includes, as functional units executed by the processor 21, a position acquisition unit (position acquisition function) 41, a data acquisition unit (data acquisition function) 42, and a display processing unit (display processing function) 43.

[0051] The position acquisition unit 41 executes a process of determining the current position of the user terminal 20 based on positioning information from satellites (for example, GPS satellites) received by the GNSS device 29. Information indicating the current position includes latitude, longitude, altitude (elevation), etc.

[0052] The data acquisition unit 42 executes a process of acquiring data necessary for superimposing an image on a real-life image from the server 10. The data acquired by the user terminal 20 from the server 10 includes, for example, 3D model data related information, attribute information linked to the 3D model data included in the 3D model data related information, and a program for displaying the 3D model data on a web browser.

[0053] In order to reproduce a 3D scene on the user terminal 20, the data acquisition unit 42 downloads from the server 10 a plurality of 3D model data that exist within a predetermined range around the user terminal 20 (a model search range or a model normal display range).

[0054] The display processing unit 43 executes processing to display real-time images captured by the imaging device 28, as well as 3D model data and attribute information superimposed on the images, on the display device 27. The display processing unit 43 of this embodiment calls the imaging device 28 on a web browser and generates an AR image by superimposing an image based on the 3D model data on a real-life image.

[0055] The superimposition process by the display processing unit 43 will now be described. The display processing unit 43 calculates the horizontal direction of the viewpoint from the horizontal azimuth value acquired from output information from a magnetic sensor or the like of the sensors 30 of the user terminal 20, and calculates the vertical direction of the viewpoint based on the elevation and depression angles acquired from output information from an acceleration sensor or the like. The display processing unit 43 then calculates the horizontal position at which to display the 3D model data from the latitude and longitude of the current position of the GNSS device 29, and calculates the vertical position at which to display the 3D model data from the altitude (altitude) of the current position of the GNSS device 29. That is, the display processing unit 43 determines the orientation of the 3D model data based on the azimuth and elevation angle of the imaging device 28.

[0056] In this embodiment, the display processing unit 43 displays an AR image on the display device 27 using a web browser on the user terminal 20. Specifically, the display processing unit 43 superimposes the orientation-determined 3D model data on a real-time image captured by the imaging device 28, and displays the real-time image on which the 3D model data is superimposed in the web browser displayed on the display device 27. Web AR can be realized by downloading a program for displaying a 3D model on the web browser together with the 3D model data from the server 10. In this embodiment, the display processing unit 43 draws an AR image when the server 10 is accessed and the 3D model data is downloaded, such as when a web page is opened or reloaded, but the drawing timing is not limited to this timing.

[0057] When the imaging direction of the imaging device 28 changes, the display processing unit 43 performs viewpoint manipulation to change the superimposed image. The viewpoint manipulation is performed based on the newly acquired horizontal and vertical orientations. The superimposed image changes based on this viewpoint manipulation.

[0058] The display processing unit 43 also performs processing to display information related to disaster information in an AR image, and processing to display setting images necessary to correct the current position of the user terminal 20 based on satellites and the orientation based on the compass function.

[0059] The display process of an AR image is started, for example, by operating an AR button on a website that displays disaster information. Next, the flow of the process of displaying an AR image will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of the disaster display process of this embodiment.

[0060] When the disaster display process is started, the location acquisition unit 41 of the user terminal 20 acquires terminal location information indicating the current location of the user terminal 20, determines the terminal location, and transmits it to the server 10 (step S101). The search processing unit 33 of the server 10 determines whether or not it has received terminal location information indicating the current location of the user terminal 20, and if it determines that it has received the terminal location information, it proceeds to step S103 (step S102; Yes). The search processing unit 33 of the server 10 continues the determination process until it receives the terminal location information (step S102; No).

[0061] The search processing unit 33 of the server 10 extracts 3D model data related information based on the received terminal location information and the stored 3D model data location information. Then, the search processing unit 33 acquires attribute information for the 3D model data included in the identified 3D model data related information from the model information database 35 (step S103), and transmits the acquired attribute information to the user terminal 20 (step S104).

[0062] The user terminal 20 transmits a data request to the server 10 to acquire 3D model data based on the received attribute information (step S105).

[0063] The proxy unit 34 of the server 10 determines whether or not a data request has been received, and if it determines that a data request has been received, proceeds to step S107 (step S106; Yes). The proxy unit 34 of the server 10 continues the determination process until a data request is received (step S106; No).

[0064] In step S107, the proxy unit 34 of the server 10 transmits the 3D model data acquired in response to the data request to the user terminal 20 (step S107). The user terminal 20 acquires the 3D model data from the server 10 (step S108), and executes a process of displaying disaster information on the display device 27 using the acquired 3D model data (step S109).

[0065] Next, a process for displaying an AR image on the display device 27 of the user terminal 20 will be described. Fig. 8 is a schematic diagram of an AR image displayed on the user terminal 20 of this embodiment. In Fig. 8, a warning area image 110 indicating a warning area and a special warning area image 111 indicating a special warning area are superimposed by the display processing unit 43 on an actual image captured by the imaging device 28 of the user terminal 20.

[0066] The display processing unit 43 also executes processing to display compass information 101, text message 102, GPS reception strength information 103, sensor information display switch 104, and various setting display buttons 105.

[0067] The compass information 101 is an image that indicates the direction of the compass function. The text message 102 is an image that displays information about a disaster at the current location in text. In this example, the text message 102 displays "May be in a landslide warning area" because the user terminal 20 is located in a warning area. That is, as shown in FIG. 8, the display processing unit 43 may perform processing to superimpose information about the disaster along with 3D model data on a real-time image captured by the imaging device 28. The GPS reception strength information 103 is an image that visually displays the reception strength of a satellite received by the user terminal 20. For example, the display processing unit 43 performs visual changes, such as reducing the number of arcs representing radio waves in the image when the reception strength decreases.

[0068] The sensor information display switch 104 is a tap operation part for determining whether or not to display accuracy information such as latitude, longitude, altitude, and direction, which will be described later (see FIG. 11, which will be described later). In FIG. 9, the sensor information display switch 104 is OFF, and the accuracy information is not displayed. The various settings display button 105 is a tap operation part for displaying a screen on which the user can make various settings.

[0069] Next, information added to an AR image will be described. Fig. 9 is a schematic diagram in which area information 120 is added to an AR image of this embodiment. The area information 120 is information acquired by the user terminal 20 from the server 10 as attribute information. In this embodiment, when a tap operation is performed on the center portion of the AR image shown in Fig. 8, the area information 120 shown in Fig. 9 is added.

[0070] The area information 120 includes a selection image 121 for selecting an area, a disaster information image 122 showing disaster-related information, and a file link display image 123 for displaying disaster information files in, for example, PDF (portable document format) format.

[0071] The selection image 121 is an image displayed in an operation portion for the user to select an area when multiple areas overlap around the current location of the user terminal 20. The selection image 121 displays multiple areas, and when the user taps on any of the areas, the disaster information image 122 is changed to one corresponding to the selected area.

[0072] The disaster information image 122 is an image that displays information about the disaster set for the area and information about the area. In this example, a debris flow is set as the natural phenomenon for this area, and a landslide warning area is set as the designated type. Information about the location of the area, such as the area number, is also displayed.

[0073] The file link display image 123 is an image for displaying the contents of a notice document regarding a disaster in an area based on an administrative agency, etc. In this example, when the user taps the file link display image 123, data in PDF format is displayed on the display device 27.

[0074] Next, a process for displaying a two-dimensional map image 130 together with an AR image will be described. FIG. 10 is a schematic diagram in which a map image 130 is added to an AR image of this embodiment. In FIG. 10, the map image 130 is displayed below the AR image. In this embodiment, the user taps the various settings display button 105 to open a settings screen, and the map image 130 and the AR image are displayed side by side in the vertical direction by performing a tap operation to set the display of the map image. That is, the display processing unit 43 may perform a process for displaying both an image in which 3D model data is superimposed on a real-time image captured by the imaging device 28, and a two-dimensional map image 130 that includes the same location as the 3D model data. Note that the map image 130 and the AR image may be displayed side by side in the horizontal direction on the screen of the display device 27.

[0075] The display processing unit 43 also executes processing to display an arrow image 131 indicating the current position of the user terminal 20 and the orientation of the user terminal 20. The orientation of the user terminal 20 indicates the imaging direction of the imaging device 28 (the orientation of the user terminal 20) acquired from the output information from the sensor 30.

[0076] Next, a process for adjusting the position of an AR image will be described. Fig. 11 is a schematic diagram for explaining the position adjustment of an AR image according to this embodiment. In Fig. 11, a position adjustment controller 150 is displayed together with a compass information image 140.

[0077] The compass information image 140 is accuracy information that indicates, for example, the accuracy of the compass (latitude, longitude, altitude, and direction). The compass information image 140 also displays an XY error as an error in planar view, a Z error as an altitude error, and a C error as an orientation error. The compass information image 140 is displayed by turning on the sensor information display switch 104.

[0078] The position adjustment controller 150 is an image displayed on an operation portion for adjusting the position. The position adjustment controller 150 is composed of a height adjustment unit 151 for adjusting the height and an orientation adjustment unit 152 for adjusting the orientation.

[0079] The height adjustment section 151 is an image displayed on an operation section for increasing or decreasing the height, and the user can manually adjust the altitude (height) by tapping this section. This also makes it possible to correct errors based on the output information of the GNSS device 29.

[0080] The orientation adjustment unit 152 is an image displayed on an operation section for adjusting the orientation of the user terminal 20 + or -, and the user can manually adjust the orientation by tapping this section. Errors based on the output information of the sensor 30 can also be corrected. That is, when the display processing unit 43 receives a user operation to change the elevation angle, it performs processing to change the position where the 3D model data is displayed based on the received operation. Furthermore, by tapping the height adjustment unit 151 and setting the height much higher than the ground, the image capture device 28 can be pointed downward, allowing for a bird's-eye view of the current location and its surroundings. That is, when the display processing unit 43 receives a user operation to change the altitude, it performs processing to change the position where the 3D model data is displayed based on the received operation.

[0081] As described above, the user terminal 20 includes an imaging device 28 that captures images and a display device 27 that displays various information. The user terminal 20 also includes a position acquisition unit 41 that acquires position information indicating the current position of the user terminal 20, a data acquisition unit 42 that transmits the position information acquired by the position acquisition unit 41 to an external server 10 that stores 3D model data configured such that position elevation information is included in map information indicating a warning zone within a predetermined range where a disaster is expected, and 3D model data-related information in which the 3D model data position information indicating the position of the 3D model data and orientation information are associated, and acquires 3D model data-related information extracted by the server 10 based on the position information and the 3D model data position information, and a display processing unit 43 that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposes the determined 3D model data on a real-time image being captured by the imaging device 28, and displays the real-time image on which the 3D model data is superimposed on the display device 27.

[0082] As a result, the generation of 3D model data is not performed on the server 10 or user terminal 20 side, effectively reducing the load on the server 10 or user terminal 20 side, while allowing the user to intuitively recognize the risk of a disaster by using AR images on the display device 27 of the user terminal 20. Furthermore, since AR images can be displayed on a web browser that is less device-dependent, it is possible to effectively make a wide range of users aware of the risk of a disaster.

[0083] In addition, in this embodiment, the display processing unit 43 performs a process of superimposing information about the disaster together with the 3D model data on a real-time image being captured by the imaging device 28.

[0084] This allows the user to be more aware of the risk of disaster and the location information for each area, such as a warning area or special warning area, in more detail.

[0085] Furthermore, in this embodiment, when the display processing unit 43 receives a user operation to change the height information of a position, it performs processing to change the size of the 3D model data based on the received operation.

[0086] This means that even if the orientation cannot be accurately obtained due to a malfunction of the sensor 30 of the user terminal 20, causing the superimposition position of the 3D model data to shift horizontally, the superimposition position can be returned to the appropriate position by adjustment.

[0087] Furthermore, in this embodiment, when the display processing unit 43 receives a user operation to change the angle of elevation, it performs processing to change the position where the 3D model data is displayed based on the received operation.

[0088] This means that even if the altitude cannot be accurately obtained due to a malfunction of the GNSS device 29 of the user terminal 20, causing the superimposition position of the 3D model data to shift vertically, the superimposition position can be returned to the appropriate position by adjustment.

[0089] Furthermore, in this embodiment, the display processing unit 43 performs processing to display an image in which 3D model data is superimposed on a real-time image captured by the imaging device 28, together with a two-dimensional map image 130 that includes the same location as the 3D model data.

[0090] This allows the user to view the current location from above through the map image 130. Also, by checking the map image 130, the user can determine whether or not there is an error in the direction relative to the azimuth.

[0091] The disaster information display system 1 of this embodiment also includes: 3D model data configured such that position elevation information is included in map information indicating a warning zone within a predetermined range where a disaster is expected; a server storing 3D model data-related information in which 3D model data position information indicating the position of the 3D model data and orientation information are associated; and a user terminal 20 including an imaging device 28 that captures images and a display device 27 that displays various information. The user terminal 20 also includes: a position acquisition unit 41 that acquires position information; a data acquisition unit 42 that transmits the position information acquired by the position acquisition unit 41 to the server 10 and acquires the 3D model data-related information extracted by the server 10 based on the position information and the 3D model data position information; and a display processing unit 43 that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposes the determined 3D model data on a real-time image being captured by the imaging device 28, and displays the real-time image with the 3D model data superimposed on the display device 27.

[0092] Furthermore, the control method of the user terminal 20 as a disaster information display device includes: a location information acquisition step of acquiring location information; a data transmission step of transmitting the location information acquired by the location acquisition unit 41 to an external server 10 that stores 3D model data configured such that location height information is included in map information indicating a warning zone within a predetermined range where a disaster is expected, 3D model data location information indicating the position of the 3D model data, and 3D model data-related information that is associated with orientation information; a data acquisition step of acquiring 3D model data-related information extracted by the server 10 based on the location information and the 3D model data location information; and a display processing step of determining the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposing the determined 3D model data on a real-time image being captured by the imaging device 28, and displaying the real-time image on which the 3D model data is superimposed on the display device 27.

[0093] The program of this embodiment also causes the computer to execute a location information acquisition function that acquires location information, a data transmission function that transmits the location information acquired by the location acquisition unit 41 to an external server 10 that stores 3D model data configured such that location height information is included in map information that indicates a warning zone within a predetermined range where a disaster is expected, 3D model data location information that indicates the position of the 3D model data, and 3D model data-related information that is associated with orientation information, a data acquisition function that acquires 3D model data-related information extracted by the server 10 based on the location information and the 3D model data location information, and a display processing function that determines the orientation of the 3D model data based on the orientation and elevation angle of the imaging device 28, superimposes the determined 3D model data on a real-time image being captured by the imaging device 28, and displays the real-time image on which the 3D model data is superimposed on the display device 27.

[0094] The control method and program for the disaster information display system 1 and the user terminal 20 also do not generate 3D model data on the server 10 or user terminal 20 side, effectively reducing the load on the server 10 or user terminal 20, while allowing the user to intuitively recognize the risk of a disaster by using AR images on the display device 27 of the user terminal 20. Furthermore, since AR images can be displayed on a web browser that is less device-dependent, a wide range of users can be made aware of the risk of a disaster effectively.

[0095] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and can be modified as appropriate.

[0096] For example, the display processing unit 43 can add evacuation information related to evacuation to the AR image of the above embodiment. More specifically, the server 10 may store evacuation image information and evacuation text information indicating the location of evacuation shelters, evacuation routes, evacuation directions, etc., and the user terminal 20 may download the evacuation image information and evacuation text information from the server 10 and display them on the AR image.

[0097] In addition, the display processing unit 43 may obtain information on disasters that have actually occurred in the past, real-time weather information, the probability and risk of disasters occurring based on the weather information, etc. from an external website and display it on the AR image of the above embodiment.

[0098] Furthermore, the display processing unit 43 may be configured to additionally display a guide image that serves as a reference for performing position adjustment while the above-described position adjustment controller 150 is displayed.

[0099] Furthermore, the display processing unit 43 may additionally display information other than disaster information, such as information for travelers, on the AR image.

[0100] Furthermore, the series of processes in the above-described embodiments and modifications can be executed by hardware or software. When the series of processes are executed by software, the programs constituting the software are installed into a computer or the like from a network or a recording medium. The computer may be a computer incorporated into dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs thereon, such as a general-purpose personal computer. [Explanation of symbols]

[0101] 1. Disaster Information Display System 10 Servers 20 User terminal (disaster information display device) 27 Display device 28 Imaging device 41 Position acquisition part 42 Data Acquisition Section 43 Display processing section

Claims

1. A disaster information display device including an imaging device that captures images and a display device that displays various information, a location acquisition unit that acquires location information; a data acquisition unit that transmits the location information acquired by the location acquisition unit to an external server that stores 3D model data configured such that location height information is included in map information that indicates a warning zone within a predetermined range where a disaster is expected, 3D model data location information that indicates the location of the 3D model data, and 3D model data-related information that is associated with direction information, and acquires the 3D model data-related information extracted by the server based on the location information and the 3D model data location information; a display processing unit that determines the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on a display device; and A disaster information display device comprising:

2. The display processing unit The disaster information display device according to claim 1 , wherein the disaster information display device performs a process of superimposing the disaster information together with the 3D model data on a real-time image being captured by the imaging device.

3. The display processing unit The disaster information display device according to claim 1 or 2, wherein when a user's operation to change the height information of the position is accepted, a process of changing the size of the 3D model data is performed based on the accepted operation.

4. The display processing unit The disaster information display device according to claim 1 or 2, wherein, when a user's operation to change the elevation angle is accepted, a process of changing the position where the 3D model data is displayed is performed based on the accepted operation.

5. The display processing unit 3. The disaster information display device according to claim 1, wherein the disaster information display device performs processing to display an image in which 3D model data is superimposed on a real-time image captured by the imaging device, and a two-dimensional map image including the same location as the 3D model data.

6. a server storing 3D model data configured by including height information of a position in map information indicating a warning zone within a predetermined range where a disaster is expected, 3D model data position information indicating the position of the 3D model data, and 3D model data related information in which direction information is associated; a disaster information display device including an imaging device for capturing images and a display device for displaying various types of information; The disaster information display device includes: a location acquisition unit that acquires location information; a data acquisition unit that transmits the location information acquired by the location acquisition unit to the server, and acquires the 3D model data related information extracted by the server based on the location information and 3D model data location information; a display processing unit that determines the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on a display device; and A disaster information display system comprising:

7. A method for controlling a disaster information display device that includes an imaging device that captures images and a display device that displays various information, a location information acquisition step of acquiring location information; a data transmission step of transmitting the location information acquired in the location information acquisition step to an external server storing 3D model data configured such that location height information is included in map information indicating a warning zone within a predetermined range where a disaster is expected, 3D model data location information indicating the location of the 3D model data, and 3D model data related information in which direction information is associated; and a data acquisition step of acquiring the 3D model data related information extracted by the server based on the location information and the 3D model data location information; a display processing step of determining the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposing the determined 3D model data on a real-time image being captured by the imaging device, and displaying the real-time image on which the 3D model data is superimposed on a display device; and A method for controlling a disaster information display device, comprising:

8. On the computer, A location information acquisition function that acquires location information; a data transmission function that transmits the location information acquired by the location information acquisition function to an external server that stores 3D model data configured such that location height information is included in map information that indicates a warning zone within a predetermined range where a disaster is expected, 3D model data location information that indicates the location of the 3D model data, and 3D model data related information that is associated with direction information; and a data acquisition function for acquiring the 3D model data related information extracted by the server based on the location information and the 3D model data location information; a display processing function that determines the orientation of the 3D model data based on the azimuth and elevation angle of an imaging device, superimposes the determined 3D model data on a real-time image being captured by the imaging device, and displays the real-time image on which the 3D model data is superimposed on a display device; and A program that executes the following.

Images