Information display apparatus, information display method, and program

Abstract

To enable the rapid grasping and identification of disaster situations.SOLUTION: An information display device 100 is equipped with a matching unit 202 for identifying the position of an object in an overhead image based on a result of matching an overhead image in which an area including the object before disaster is shot with a target image including a situation of the object after the disaster that satisfies the criteria for determining that the disaster is less severe, and a display unit 301 for displaying an image including the situation of the object and information in which the position of the object in the overhead image can be identified.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a display device and a display method for displaying information indicating a disaster situation, etc., and further to a program for implementing these. [Background technology]

[0002] In order to prevent damage from spreading when large-scale natural disasters such as earthquakes, floods, and volcanic eruptions occur, appropriate initial responses based on a rapid understanding of the damage and extent of the damage are necessary. In particular, when natural disasters such as typhoons, heavy rains, and earthquakes occur, which have become increasingly severe in recent years, it is important to quickly understand the location, extent, and extent of the damage so that initial responses such as evacuating victims and carrying out rescue activities can be carried out quickly.

[0003] Previously, the information available immediately after a disaster was limited to global information (such as seismic intensity distribution, power outages, and precipitation) obtained from the Japan Meteorological Agency and public institutions, and victims could only roughly grasp the extent of the damage. In order to grasp the damage situation in more detail, first responders (hereafter referred to as users) had to conduct on-site surveys, which took time. Therefore, in order to efficiently grasp the damage situation, it has been considered to provide users with a large number of on-site images showing the disaster situation that were taken or collected at the time of the disaster.

[0004] In order to efficiently grasp the detailed damage situation, it is necessary to select and organize images that are useful for grasping the detailed damage situation from the large number of site images. For this purpose, there is a method for selecting and organizing useful images by classifying a large number of site images into specific classes (e.g., buildings, cars, roads, etc.) that are defined in advance. For example, Non-Patent Document 1 discloses an image classification method for classifying images into specific classes that are defined in advance. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). Summary of the Invention [Problem to be solved by the invention]

[0006] However, the image classification method disclosed in the non-patent document merely classifies images based on the subject, and therefore, it is difficult to quickly identify and grasp the damage situation and location in the event of a disaster by using only this image classification method.

[0007] One of the objectives of the present disclosure is to solve the above problems and enable rapid understanding and identification of the damage situation. [Means for solving the problem]

[0008] In order to achieve the above object, an information display device according to one aspect of the present disclosure includes: A matching unit that identifies a position of the object in the overhead image based on a result of matching between an overhead image of an area including the object before the disaster and a portion of a target image including the state of the object after the disaster that satisfies a criterion for determining that the object has been less damaged; a display unit that displays an image including a state of the object and information that allows a position of the object in the overhead image to be identified; The present invention is characterized by comprising:

[0009] In order to achieve the above object, an information display method according to one aspect of the present disclosure includes: A matching step of identifying a position of the object in the overhead image based on a result of matching between an overhead image of an area including the object before the disaster and a portion of a target image including the state of the object after the disaster that satisfies a criterion for determining that the object has been less damaged; a display step of displaying an image including a situation of the object and information that allows a position of the object in the overhead image to be specified; The present invention is characterized in that it has the following features:

[0010] Furthermore, in order to achieve the above object, a program according to one aspect of the present disclosure includes: On the computer, A matching step of identifying a position of the object in the overhead image based on a result of matching between an overhead image of an area including the object before the disaster and a portion of a target image including the state of the object after the disaster that satisfies a criterion for determining that the object has been less damaged; a display step of displaying an image including a situation of the object and information that allows a position of the object in the overhead image to be specified; The present invention is characterized in that: Effect of the Invention

[0011] As described above, according to the present disclosure, it is possible to quickly grasp and identify the damage situation. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an example of an information display device. [Diagram 2] FIG. 2 is a diagram specifically showing the configuration of the information display device shown in FIG. [Diagram 3] FIG. 3 is a diagram showing an example of a screen displayed by the information display device. [Figure 4] FIG. 4 is a diagram showing an example of a screen displayed by the information display device. [Diagram 5] FIG. 5 is a diagram showing an example of a screen displayed by the information display device. [Figure 6]FIG. 6 is a diagram showing an example of a screen displayed by the information display device. [Figure 7] FIG. 7 is a diagram showing an example of a screen displayed by the information display device. [Figure 8] FIG. 8 is a diagram showing an example of a screen displayed by the information display device. [Figure 9] FIG. 9 is a diagram showing an example of a screen displayed by the information display device. [Figure 10] FIG. 10 is a flow chart showing an example of the operation of the information display device. [Figure 11] FIG. 11 is a block diagram showing an example of a computer that realizes an information display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Embodiment) Hereinafter, in the embodiments, an example of an information display device, an information display method, and a program will be described with reference to FIGS.

[0014] [Device configuration] First, a schematic configuration of an example of an information display device will be described with reference to Fig. 1. Fig. 1 is a diagram showing a schematic configuration of an example of an information display device.

[0015] 1 is a device for presenting information showing a disaster situation, etc. As shown in FIG.

[0016] The matching unit 202 identifies the position of the object in the overhead image based on the result of matching between the overhead image of the area including the object before the disaster and a portion of the target image including the state of the object after the disaster that satisfies a criterion for determining that the damage is small. The display unit 301 displays an image including the state of the object and information that allows the position of the object in the overhead image to be identified.

[0017] In this way, the information display device 100 can display information that allows identification of the status and location of objects in a disaster-stricken area on an image. Therefore, the information display device 100 makes it possible to quickly grasp and identify the disaster status.

[0018] 2 to 9, the configuration and functions of the information display device 100 will be specifically described. Fig. 2 is a configuration diagram specifically showing the configuration of the information display device shown in Fig. 1.

[0019] As shown in Fig. 2, information display device 100 includes image acquisition unit 101, geographic information acquisition unit 102, overhead image acquisition unit 103, reference image acquisition unit 104, language acquisition unit 105, and superimposition parameter acquisition unit 106 in addition to the above-mentioned comparison unit 202 and display unit 301. Furthermore, information display device 100 also includes image selection unit 201, information superimposition unit 203, selection update unit 204, and superimposition update unit 205. Also, as shown in Fig. 2, display device 200 is connected to information display device 100. Each unit will be described below.

[0020] Image acquisition unit 101: The image acquisition unit 101 acquires one or more images as an image group. Examples of the image group include video images continuously captured by a video camera, a sequence of still images captured at regular intervals, and the like. Examples of the images include images collected from SNS. Furthermore, examples of the images include images captured by street cameras, cameras mounted on flying objects such as drones, surveillance cameras, in-vehicle cameras, or drive recorders. The image acquisition unit 101 may acquire images obtained by capturing landscapes.

[0021] The image acquisition unit 101 can also acquire images other than visible images, for example, images acquired from sensors other than cameras. Specifically, examples of images other than visible images include temperature images and depth images. The images may also be intermediate processing results of deep learning. In this case, the image acquisition unit 101 acquires multi-channel images. Furthermore, the image acquiring unit 101 can acquire numerical values ​​such as measured values ​​in addition to the above-mentioned images. The numerical values ​​include vector data (velocity field, density field, etc.) calculated by a numerical simulation or the like. The images and numerical values ​​acquired by the image acquiring unit 101 are recorded in a storage device such as a memory (not shown).

[0022] Moreover, the images acquired by the image acquisition unit 101 do not need to be taken by a single camera. The image acquisition unit 101 can also acquire images of multiple modalities, for example, images including a visible image and a far-infrared image taken by two cameras, a visible camera and an infrared camera. In that case, the positions of these images may be aligned using, for example, the method disclosed in Reference 1 below. Furthermore, a single image may be synthesized from these images using, for example, the method disclosed in Reference 2 below.

[0023] (Reference 1) Shibata, Takashi, Masayuki Tanaka, and Masatoshi Okutomi. "Accurate joint geometric camera calibration of visible and far-infrared cameras." Electronic Imaging 2017.11 (2017): 7-13. (Reference 2) Shibata, Takashi, Masayuki Tanaka, and Masatoshi Okutomi. "Unified Image Fusion Framework With Learning-Based Application-Adaptive Importance Measure." IEEE Transactions on Computational Imaging 5.1 (2018): 82-96.

[0024] The image is an image showing the state of an object (hereinafter, referred to as a "symmetric image"). The object is, for example, an object having a structure such as a building, a bridge, a house, a school, a hospital, a government office, other buildings, a traffic light, a road, a sidewalk, a road marking, a curb, a guardrail, a utility pole, a steel tower, etc. The image may be, for example, an image before a disaster occurs, or an image after a disaster occurs. For example, the image includes the state of the object after a disaster occurs. The image may also include a state showing an object that was not damaged (not damaged) by the disaster, or an object that was lightly damaged (lightly damaged) by the disaster. In addition, the image may be an image of one object, or an image including a part that was lightly damaged (or was not damaged, was less damaged, or was not damaged) and a damaged part. The object that was not damaged (or was not damaged) by the disaster may be, for example, a natural object such as a mountain, a river, an ocean, or a forest.

[0025] Disasters include, for example, at least one of natural disasters and man-made disasters, such as heavy rain, torrential rain, typhoons, earthquakes, tsunamis, floods, high tides, heavy snowfall, tornadoes, volcanic eruptions, landslides, subsidence, flooding, inundation, fires, forest fires, explosions, and damage.

[0026] Geographical information acquisition unit 102: The geographic information acquisition unit 102 acquires geographic information configured by superimposing information on a two-dimensional plane or three-dimensional space. Examples of the geographic information configured by superimposing information on a two-dimensional plane or three-dimensional space include maps. Examples of the geographic information configured by superimposing information on a three-dimensional space include map data generated by Building Information Modeling, Computer Aided Design, and the like. Note that the geographic information in the embodiment is not limited to the above. Other examples of the geographic information include three-dimensional point cloud data constructed from a large number of two-dimensional images by a three-dimensional restoration technique such as SfM (Structure from Motion). Then, the geographic information acquisition unit 102 records the acquired geographic information in a memory (not shown) or the like.

[0027] Bird's-eye view image acquisition unit 103: The overhead image acquisition unit 103 acquires one or more images obtained by photographing a target area from the sky as an overhead image. Examples of the one or more images obtained by photographing from the sky include an aerial photograph and a satellite image. The overhead image may be an image photographed from the sky by an unmanned aerial vehicle such as a drone. Furthermore, the overhead image is not limited to a still image, and may be, for example, a moving image photographed by a video camera mounted on an aircraft or an unmanned aerial vehicle, or a sequence of still images photographed at regular intervals. Then, the overhead image acquisition unit 103 records the acquired overhead image in a memory (not shown) or the like.

[0028] The overhead image is used as a base image (hereinafter referred to as a "reference image") when a matching unit 202 (described later) performs matching with a target image. Examples of the reference image include an image of an area including an object before a disaster, an image of an area including an object after a disaster, etc.

[0029] Reference image acquisition unit 104: The reference image acquisition unit 104 acquires one or more reference images to be used in selecting or updating the image group acquired by the image acquisition unit 101. The selection of the image group is performed by an image selection unit 201, which will be described later, and the updating of the image group is performed by a selection update unit 204, which will be described later.

[0030] Language acquisition part 105: The language acquisition unit 105 acquires reference language information that specifies one or more languages ​​used in selecting or updating the image group acquired by the image acquisition unit 101. Note that even in this case, the image group is selected by the image selection unit 201, which will be described later, and the image group is updated by the selection update unit 204, which will be described later.

[0031] The reference language information includes information including words and sentences that serve as clues for selecting or updating the image group. The reference language information is input by a user through an external device. Specifically, the reference language information may include a phrase indicating a specific area, such as "XX City" or "Find an image of XX City". Furthermore, the reference language information may include a word or sentence that indicates a clue for selecting or updating the image group, such as "Find an area where a building has collapsed", "Collapsed", or "Collapsed building". Furthermore, the reference language information may include a word or sentence that indicates a sequence that identifies images that have already been selected, such as "Images 1 and 4 are the target images", "Images 3, 6, and 7 are not the target images", or "Image 1".

[0032] Superposition parameter acquisition unit 106: The superimposition parameter acquisition unit 106 acquires superimposition parameters required for the process of superimposing characters on an image, which will be described later. The superimposition parameters determine at least one of the strength, attribute, range, character, etc. for superimposition. The superimposition process is performed by an information superimposition unit 203 and a superimposition update unit 205, which will be described later.

[0033] Specifically, the superimposition parameters are set according to the importance (specifically, the scale of damage, etc.) calculated for each image by, for example, image recognition technology or text recognition technology (more specifically, image captioning technology). The greater the importance, the greater the strength for superimposition, the range of superimposition, etc. are set. The superimposition parameters may be, for example, image attributes calculated by image recognition, or characters (text information) calculated by text recognition (more specifically, image captioning technology). The superimposition parameter acquisition unit 106 records the acquired superimposition parameters in a memory (not shown), etc.

[0034] Image selection unit 201: The image selection unit 201 selects one or more images from the image group acquired by the image acquisition unit 101 based on at least one of the reference language information and the reference image group, and sets the selected images as a selected image group. Specifically, as shown in Figs. 3 to 5, the image selection unit 201 selects images related to the language information from the image group acquired by the image acquisition unit 101 in response to an instruction from a user, using the reference language information acquired by the language acquisition unit 105. Furthermore, the image selection unit 201 selects one or more images from the image group acquired by the image acquisition unit 101 that have similar characteristics to the reference image group, using the reference image group acquired by the reference image acquisition unit 104.

[0035] In addition, the image selection unit 201 can also select one or more images from the reference image group that are related to the language contained in the reference image or have characteristics similar to those of the reference image group, for example, using the method disclosed in JP 2023-027897 A.

[0036] Collation unit 202: The matching unit 202 matches one or more images among the group of images acquired by the image acquisition unit 101 with the above-mentioned geospatial information and the overhead image, and calculates a correspondence relationship between the spatial position in the geospatial information and this image. Then, the matching unit 202 calculates spatial coordinates of one or more images from the correspondence relationship, and outputs the calculated spatial coordinates as corresponding spatial coordinates.

[0037] Specifically, the matching unit 202 targets one or more images from among the images belonging to the selected image group selected by the image selection unit 201. The matching unit 202 matches the target image (hereinafter referred to as the "target image") with the geospatial information and the overhead image, calculates a correspondence relationship between the spatial position in the geospatial information and the target image, and further calculates the spatial coordinates of the target image using this correspondence relationship. The matching unit 202 can also calculate the correspondence relationship between the spatial position in the geospatial information and the target image using, for example, the method disclosed in JP 2002-032013 A.

[0038] The matching unit 202 can also perform a process of matching the target image with the reference image and specifying the position of the object in the target image using the matching result. For example, assume that the target image is an image including the state of the object after a disaster, and the reference image is an image of an area including the object before the disaster. In this case, the matching unit 202 matches a portion of the target image that satisfies a criterion for determining that the image is not damaged (or that satisfies a criterion for determining that the image is less damaged) with the reference image.

[0039] In addition, the criteria for determining that damage is minor include, for example, the existence of man-made structures such as buildings, bridges, and hospitals after the disaster, and the existence of natural objects such as mountains and rivers.

[0040] The matching unit 202 specifies the position of the object in the reference image based on the portion determined to match (or be similar) by the matching. When the target image and the reference image are both images taken before the disaster, or when the target image and the reference image are both images taken after the earthquake disaster, the matching unit 202 may specify the portion of the reference image that matches (or is similar) to the object as the position of the object.

[0041] Information superimposition unit 203: The information superimposing unit 203 performs superimposition display on the portions of each image in the selected image group selected by the image selecting unit 201 that correspond to the corresponding spatial coordinates output by the matching unit 202, using the geographical information acquired by the geographical information acquiring unit 102. Then, the information superimposing unit 203 outputs the superimposed image as superimposed geospatial information.

[0042] In addition to the geospatial information, the information overlay unit 203 can also use the overlay parameters corresponding to each image acquired by the overlay parameter acquisition unit 106 to overlay the overlay intensity, attributes, range, characters, etc. onto the image.

[0043] Specifically, as shown in Fig. 3 to Fig. 9, the information superimposing unit 203 generates a concentric pattern indicating the strength and range of superimposition using the superimposition parameters corresponding to each image acquired by the superimposition parameter acquiring unit 106. Then, the information superimposing unit 203 superimposes the generated pattern on the geospatial information. The information superimposing unit 203 can also generate a pattern of any shape, such as a triangle or a rectangle, instead of a concentric pattern. Furthermore, the information superimposing unit 203 can also generate a pattern that displays a probability distribution such as a Gaussian distribution with contour lines or pseudocolor, and superimpose this on the geospatial information.

[0044] Furthermore, the information superimposing unit 203 can also superimpose attributes and characters on the geospatial information by using the superimposing parameters corresponding to each image acquired by the superimposing parameter acquiring unit 106. More specifically, when "collapsed building" corresponding to the "attribute" and "urgent rescue required" corresponding to the "characters" are acquired as the superimposing parameters, the information superimposing unit 203 directly superimposes these attributes and characters on the geospatial information (see Figs. 6 to 9).

[0045] Selection update section 204: The selection update unit 204 reselects further images from the group of selected images selected by the image selection unit 201 based on at least one of the reference image and the reference language information in accordance with user instructions, and outputs the reselected group of images as a reselected group of images.

[0046] Specifically, the selection update unit 204 reselects, for example, one or more images from the selected image group selected by the image selection unit 201, which have image features similar to the image they represent, using language information. The selection update unit 204 can also reselect, for example, one or more images from the selected image group selected by the image selection unit 201, which have image features similar to the reference image group, using a reference image group. More specifically, the selection update unit 204 can also reselect an image from the selected image group, using, for example, the method disclosed in the above-mentioned JP 2023-027897 A.

[0047] Overlay update unit 205: The overlay update unit 205 uses the overlay geospatial information output by the information overlay unit 203 to re-overlay display the images in the reselected image group output by the selection update unit 204. Furthermore, the overlay update unit 205 also performs different re-overlay display for the images that were not selected as part of the reselected image group, and outputs these re-overlay results as re-overlay geospatial information.

[0048] Furthermore, the overlay update unit 205 uses the overlay parameters corresponding to each image acquired by the overlay parameter acquisition unit 106 in addition to the overlay geospatial information to overlay the overlay intensity, attributes, range, text, etc. onto each image in the reselected image group, thereby updating each image.

[0049] Specifically, as shown in Fig. 3 to Fig. 9, the overlay update unit 205 generates a concentric pattern indicating the strength and range of overlay using the overlay parameters corresponding to each image acquired by the overlay parameter acquisition unit 106. Then, the overlay update unit 205 overlays the generated pattern on the geospatial information to update each image. In addition to the concentric pattern, the overlay update unit 205 can also generate a pattern of any shape, such as a triangle or a rectangle. Furthermore, the overlay update unit 205 can generate a pattern that displays a probability distribution such as a Gaussian distribution as contour lines or pseudo-colors, and overlays this on the geospatial information to update it.

[0050] Furthermore, the superimposition update unit 205 can also superimpose attributes and characters on the geospatial information by using the superimposition parameters corresponding to each image acquired by the superimposition parameter acquisition unit 106. Specifically, when the "attribute" "collapsed building" and the "character" "urgent rescue required" are acquired as the superimposition parameters, the superimposition update unit 205 directly superimposes these attributes or characters on the geospatial information to update the image.

[0051] Display section 301: 3, the display unit 301 displays on a screen the superimposed geospatial information output by the information superimposition unit 203 or the re-superimposed geospatial information output by the superimposition update unit 205. Furthermore, as shown in FIGS. 4 to 9, the display unit 301 can also display information acquired by the image acquisition unit 101, the geographic information acquisition unit 102, the overhead image acquisition unit 103, the reference image acquisition unit 104, and the language acquisition unit 105 in addition to the superimposed geospatial information or the re-superimposed geospatial information in response to an instruction from a user.

[0052] Specifically, the display unit 301 displays the reference language information entered by the user, such as "Please find an image of XX city", "Please find an area where a building has collapsed", "Numbers 1 and 4 are target images", and "Numbers 3, 6, and 7 are non-target images", acquired by the language acquisition unit 105, as shown in Figures 6 to 9.

[0053] 7 to 9, for example, assume that a collapsed building is searched for by the selection update unit 204 from among images selected by the image selection unit 201 from among an image group previously acquired by the image acquisition unit 101. In this case, the display unit 301 displays, from among the searched images, an image group whose position has been specified by the matching unit 202. Furthermore, the display unit 301 can also display a map on which the image number is superimposed around the specified position on the map and on which an intensity map is superimposed.

[0054] The display unit 301 can also display an image including the status of the object and information capable of identifying the position of the object in the area. The display unit 301 can also create information that associates the position of the object in the reference image with the status of the object in the target image, and display the created information.

[0055] Examples of the above-mentioned "information that enables the position of an object to be identified" include information that indicates how the situation of an object and the position of the object are connected by a line, and information that indicates how the same code is assigned to the situation of an object and the position of the object.

[0056] The display unit 301 can display a map of the area instead of a reference image of the area. The display unit 301 can also display the reference image and an image showing the map. Furthermore, the display unit 301 can create information including the situation of the object in the target image and information that can identify the position of the object in the reference image for each of a plurality of target images, and display this on one screen.

[0057] In addition, when the target image shows the state of objects after a disaster, the display unit 301 can calculate the ratio between the number of objects in the reference image and the number of objects in the target image, and display the calculated ratio. Furthermore, the display unit 301 can display the number of objects in the target image. Such a display process can also be called a process of displaying the state of damage in the area captured in the reference image.

[0058] In the embodiment, the information displayed on the display unit 301 is not limited to the above-mentioned information. In the embodiment, the display unit 301 can also display, for example, the user's history in more detail.

[0059] [Device operation] Next, the operation of information display device 100 will be described with reference to FIG. 10 and other figures. FIG. 10 is a flow diagram showing an example of the operation of the information display device. In the following description, reference will be made to FIGS. 1 to 9 as appropriate. In addition, in the first embodiment, an information display method is implemented by operating information display device 100. Therefore, in the embodiments, the description of the information display method will be replaced with the description of the operation of the information display device below.

[0060] 10, first, the image acquisition unit 101 acquires a group of images (S201). Next, the geographic information acquisition unit 102 acquires geospatial information (S202). Next, the overhead image acquisition unit 103 acquires overhead images such as aerial photographs or satellite images (S203). Note that sources from which these images and the like are acquired include a server device, a terminal device, and the like connected to the information display device 100 via a network or the like.

[0061] Next, the image selection unit 201 selects an image group from the image group acquired in step S201 in response to a user's instruction (S204). Next, the reference image acquisition unit 104 acquires images related to the selected image group (S205). Next, the language acquisition unit 105 acquires reference language information related to the selected image group (S206). Next, the superimposition parameter acquisition unit 106 acquires superimposition parameters necessary for superimposition (S207).

[0062] Next, the collation unit 202 links spatial coordinates to the images selected in step S204 that have not been linked to spatial coordinates (S208). Next, the information superimposition unit 203 superimposes superimposed geospatial information, such as highlighting, on the area of ​​the corresponding spatial coordinates (S209). Next, the display unit 301 displays the geographic information on which the superimposed geospatial information is superimposed (S210).

[0063] Next, the selection update unit 204 selects an image again from the image group selected in step S204 (S211). Next, the overlay update unit 205 overlays the overlay geospatial information again on the corresponding spatial coordinates in the image group reselected in step S211, thereby performing an update (S212).

[0064] Next, the display unit 301 displays the geographic information on which the finally updated superimposed geospatial information is superimposed, that is, the re-superimposed geospatial information, on the screen of the display device 200 (S213).

[0065] Thereafter, the selection update unit 204 judges whether or not the user has instructed to end the process (S214). If the result of the judgment in step S214 is that the user has not instructed to end the process, the selection update unit 204 executes step S211 again. On the other hand, if the result of the judgment in step S214 is that the user has instructed to end the process, the process in the information display device 100 ends.

[0066] [Specific example] A specific example of the operation of the information display device will be described in detail below with reference to FIGS.

[0067] For example, as shown in FIG. 3, when a user first inputs "Please search for images of XX city" to the information display device 100 via his / her terminal device, the language acquisition unit 105 acquires the input content as reference language information.

[0068] As a result, the image selection unit 201 selects an image associated with XX city from the image group previously acquired by the image acquisition unit 101. Furthermore, the collation unit 202 specifies the position of the selected image.

[0069] Then, the display unit 301 superimposes the language acquired by the language acquisition unit 105 and the response thereto on the selected image, and further superimposes the image number around the specified position on the map. The display unit 301 also superimposes an intensity map on the selected image.

[0070] Next, as shown in FIG. 4, it is assumed that the user further inputs "Please find areas where buildings have collapsed" to the information display device 100 via his / her terminal device or the like.

[0071] In this case, the selection update unit 204 searches for an image of a collapsed building from the image group previously acquired by the image acquisition unit 101 and selected by the image selection unit 201, and newly selects the searched image. Furthermore, the collation unit 202 specifies the position of the newly selected image.

[0072] Then, the display unit 301 superimposes the language and the response acquired by the language acquisition unit 105 on the newly selected image, and further superimposes the image number and intensity around the position specified on the map. At this time, some of the images may be different from the image intended by the user. For images not selected by the selection update unit 204, the image number may not be displayed.

[0073] Furthermore, as shown in Fig. 6, it is assumed that the user inputs "images 1 and 4 are target images" and "images 3, 6 and 7 are not target images" via a terminal device or the like. In this case, as shown in Figs. 7 to 9, the selection update unit 204 searches for similar or dissimilar images to these specified images from among the image group previously acquired by the image acquisition unit 101 and selected by the image selection unit 201. Furthermore, the matching unit 202 specifies the position of the searched image.

[0074] The display unit 301 then superimposes the language and the response acquired by the language acquisition unit 105 on the searched image, and further superimposes the image number and intensity around the specified position on the map. At this time, some of the images may be different from the image intended by the user.

[0075] Furthermore, when the user performs a designation operation (for example, by placing the mouse cursor over the image), the display unit 301 can redisplay a specific image on the map. Note that the image numbers of images that were not selected by the selection update unit 204 do not need to be displayed.

[0076] [Effects of the embodiment] As described above, in the first embodiment, the information display device 100 can display information that allows identification of the status and position of an object in a disaster-affected area on an image. Therefore, the information display device 100 enables rapid understanding and identification of the disaster situation.

[0077] [program] The program may be, for example, a program that causes a computer to execute steps S201 to S214 shown in Fig. 10. By installing and executing this program in a computer, in the embodiment, information display device 100 and information display method can be realized. In this case, the processor of the computer functions as image acquisition unit 101, geographic information acquisition unit 102, overhead image acquisition unit 103, reference image acquisition unit 104, language acquisition unit 105, superimposition parameter acquisition unit 106, image selection unit 201, matching unit 202, information superimposition unit 203, selection update unit 204, and superimposition update unit 205 to perform processing. Examples of the computer include a general-purpose PC, a smartphone, and a tablet terminal device.

[0078] The program may be executed by a computer system constructed by a plurality of computers. In this case, for example, one or more computers may function as any one of the image acquisition unit 101, the geographic information acquisition unit 102, the overhead image acquisition unit 103, the reference image acquisition unit 104, the language acquisition unit 105, the superimposition parameter acquisition unit 106, the image selection unit 201, the matching unit 202, the information superimposition unit 203, the selection update unit 204, and the superimposition update unit 205.

[0079] [Physical configuration] Here, a computer that realizes the information display device 100 by executing a program in the embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram showing an example of a computer that realizes the information display device.

[0080] 11, a computer 410 includes a CPU (Central Processing Unit) 411, a main memory 412, a storage device 413, an input interface 414, a display controller 415, a data reader / writer 416, and a communication interface 417. These components are connected to each other via a bus 421 so as to be able to communicate data with each other.

[0081] Furthermore, the computer 410 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 411. In this embodiment, the GPU or FPGA can execute the programs in the embodiments.

[0082] The CPU 411 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 413, into the main memory 412, and executes each code in a predetermined order to perform various calculations. The main memory 412 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0083] Moreover, the program in the embodiment is provided in a state stored in a computer-readable recording medium 420. The program in the embodiment may be distributed on the Internet connected via the communication interface 417.

[0084] Specific examples of the storage device 413 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 414 mediates data transmission between the CPU 411 and an input device 418 such as a keyboard and a mouse. The display controller 415 is connected to a display device 419 and controls the display on the display device 419.

[0085] The data reader / writer 416 mediates data transmission between the CPU 411 and the recording medium 420, reads out a program from the recording medium 420, and writes the processing results of the computer 410 to the recording medium 420. The communication interface 417 mediates data transmission between the CPU 411 and other computers.

[0086] Specific examples of recording medium 420 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as a flexible disk, or optical recording media such as a CD-ROM (Compact Disk Read Only Memory).

[0087] The information display device 100 can be realized not by a computer with a program installed, but by hardware corresponding to each part, for example, an electronic circuit. Furthermore, the information display device 100 may be partially realized by a program and the remaining part by hardware. In the embodiment, the computer is not limited to the computer shown in FIG.

[0088] The specific configuration of the present disclosure is not limited to the above-described embodiment, and any modifications that do not depart from the gist of the present invention are included in the present invention.

[0089] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Industrial Applicability]

[0090] The present disclosure is useful for a system that displays information showing a disaster situation or the like. [Explanation of symbols]

[0091] 100 Information display device 101 Image acquisition unit 102 Geographic Information Acquisition Department 103 Bird's-eye view image acquisition unit 104 Reference image acquisition unit 105 Language Acquisition Department 106 Superposition parameter acquisition unit 200 Display device 201 Image Selection Section 202 Collation Unit 203 Information Overlay Unit 204 Selection Update Section 205 Overlay Update Unit 301 Display section 410 Computer 411 CPU 412 Main memory 413 Storage device 414 Input Interface 415 Display Controller 416 Data Reader / Writer 417 Communication Interface 418 Input Device 419 Display device 420 Recording media 421 Bus

Claims

1. A matching unit that identifies a position of the object in the overhead image based on a result of matching between an overhead image of an area including the object before the disaster and a portion of a target image including the state of the object after the disaster that satisfies a criterion for determining that the object has been less damaged; a display unit that displays an image including a state of the object and information that allows a position of the object in the overhead image to be identified; An information display device comprising:

2. an image acquisition unit that acquires one or more images as an image group; a geographic information acquisition unit that acquires geographic information to be superimposed on a two-dimensional plane or a three-dimensional space as geospatial information; an overhead image acquisition unit that acquires one or more images obtained by photographing a target area from above as an overhead image; an image selection unit that selects one or more images as a selected image group from the acquired image group based on at least one of a reference image and reference language information; Further comprising: The collation unit is one or more of the target images among the acquired image group are collated with the geospatial information and the bird's-eye image, and a correspondence relationship between a spatial position in the geospatial information and the image group is calculated from a collation result, and the corresponding spatial coordinates are output as corresponding spatial coordinates. The information display device according to claim 1 .

3. an information superimposition unit that uses the geospatial information to perform a superimposition display on a portion of each image of the selected image group that corresponds to each of the coordinates of the corresponding spatial coordinates, and outputs the superimposed image as superimposed geospatial information; a selection update unit that reselects images from the selected image group based on at least one of a reference image and reference language information and outputs the reselected image group; a superimposition update unit that uses the superimposed geospatial information to perform re-superimposition display on the images of the reselected image group that have been output, and performs different re-superimposition display on the images that have not been selected as part of the reselected image group, and outputs the re-superimposition results as re-superimposed geospatial information; Further comprising: The display unit displays the superimposed geospatial information or the re-superimposed geospatial information. The information display device according to claim 2 .

4. a reference image acquisition unit that acquires one or more images to be used as the reference image as a reference image group; a language acquisition unit that acquires a language used as the reference language information; a superimposition parameter acquisition unit that acquires at least one of intensity, attribute, range, and character for superimposition as a superimposition parameter, as a parameter necessary for performing the superimposition display on each image of the acquired image group; Further comprising: the image selection unit selects one or more images from the acquired image group based on at least one of the reference language information and the reference image group, and outputs the selected image group as a selected image group; the matching unit matches one or more images among the images belonging to the selected image group with the geospatial information and the overhead image, calculates a correspondence relationship between a spatial position in the geospatial information and one or more images among the images belonging to the selected image group from a matching result, and outputs the corresponding spatial coordinates as corresponding spatial coordinates; the information superimposition unit performs a superimposed display using the superimposition parameters in addition to the geospatial information; the overlay update unit performs re-overlay display using the overlay parameters in addition to the overlay geospatial information. The information display device according to claim 3 .

5. the display unit further displays at least one of the images of the image group, the geospatial information, the overhead image, the images of the reference image group, and the reference language information. The information display device according to claim 4.

6. the information superimposing unit generates a pattern indicating a superimposition strength and a range based on the superimposition parameters, superimposes the generated pattern on the geospatial information, and further superimposes attributes and characters on the geospatial information based on the superimposition parameters; the superimposition update unit generates a pattern indicating a superimposition strength and a range based on the superimposition parameters, superimposes the generated pattern on the geospatial information, and further superimposes attributes and characters on the geospatial information based on the superimposition parameters; The information display device according to claim 4.

7. A matching step of identifying a position of the object in the overhead image based on a result of matching between an overhead image of an area including the object before the disaster and a portion of a target image including the state of the object after the disaster that satisfies a criterion for determining that the object has been less damaged; a display step of displaying an image including a situation of the object and information that allows a position of the object in the overhead image to be specified; 13. An information display method comprising:

8. On the computer, A matching step of identifying a position of the object in the overhead image based on a result of matching between an overhead image of an area including the object before the disaster and a portion of a target image including the state of the object after the disaster that satisfies a criterion for determining that the object has been less damaged; a display step of displaying an image including a situation of the object and information that allows a position of the object in the overhead image to be specified; A program characterized by causing a computer to execute the above steps.

Images