Disaster position specification system, disaster position specification method and disaster position specification program
The disaster location identification system uses image-based feature point matching and geographic data to accurately determine disaster locations, overcoming the limitations of cameras without measurement units and measurement errors, ensuring precise disaster location identification.
Patent Information
- Application Number
- JP2023083347
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-29
AI Technical Summary
Existing disaster location identification systems struggle to accurately pinpoint the location of a disaster when cameras without measurement units are used, and measurement errors lead to reduced accuracy, especially for distant disasters.
The system utilizes first and second feature points from images taken from different angles, matching them with geographic information to calculate focal length, azimuth, and depression angles, enabling precise disaster location determination without relying on camera measurement units.
This approach allows for high-accuracy disaster location identification, even with cameras lacking measurement units, by using image-based feature point matching and geographic data to calculate precise disaster locations.
Smart Images

Figure 2024166915000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a disaster location identification system, a disaster location identification method, and a disaster location identification program that identify a location where a disaster is occurring. [Background technology]
[0002] In recent years, a system has been known that captures an area using a camera installed at a high altitude and grasps the situation of a disaster that occurs in the area based on the captured images. In such a system, it is desirable to be able to identify the location where the disaster is occurring with high accuracy in order to accurately grasp the situation of the disaster.
[0003] Patent Document 1 discloses a system that displays on a screen an image captured by a camera installed in a remote location and a map corresponding to the image, and enables the captured image to be identified on the map. The system disclosed in Patent Document 1 acquires image capture information, such as the camera's image capture direction, the field of view of the image capture, and the image capture magnification, and reflects this information in map data acquired corresponding to the image. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2005-123750 A Summary of the Invention [Problem to be solved by the invention]
[0005] In the system disclosed in the above Patent Document 1, the measurement unit provided in the camera acquires each measurement value of the focal length of the camera, the direction of the camera, and the angle of the direction of the camera relative to the horizontal plane, and obtains the shooting information based on each acquired measurement value. When the technology of the above Patent Document 1 is applied to the identification of the disaster occurrence location, if a camera without a measurement unit is used, the shooting information cannot be acquired, and therefore the disaster occurrence location cannot be identified. For this reason, in the technology of the above Patent Document 1, when a camera without a measurement unit, which is a component for acquiring the shooting information, is used, the disaster occurrence location cannot be identified on the map, and it becomes difficult to identify the disaster occurrence location with high accuracy. Even when a camera with a measurement unit is used, it may be difficult to identify the disaster occurrence location with high accuracy due to measurement errors by the measurement unit. In addition, the further the disaster occurrence location is from the camera, the more significant the decrease in the accuracy of the location identification due to measurement errors. In other words, when identifying the disaster occurrence location far from the camera, even if there is a slight error in the measurement, it becomes difficult to identify the disaster occurrence location with high accuracy. As described above, the conventional technology disclosed in Patent Document 1 has a problem in that it is difficult to specify the location where a disaster has occurred with high accuracy.
[0006] The present disclosure has been made in consideration of the above, and aims to provide a disaster location identification system that can identify the location where a disaster has occurred with high accuracy. [Means for solving the problem]
[0007] In order to solve the above-mentioned problems and achieve the object, the disaster location identification system of the present disclosure includes a feature point registration unit that obtains feature point location information indicating the geographical location of the first feature point by matching first feature points, which are feature points of a first image captured of a region from a position overlooking the region, with geographic information of the region and registers the first feature point by saving the feature point location information; a feature point matching unit that matches second feature points, which are feature points of a second image captured of a state in which a disaster has occurred in the region, with each of the registered first feature points and identifies the first feature point corresponding to the second feature point; a shooting information calculation unit that calculates shooting information indicating the conditions at the time of shooting of the second image based on the feature point location information of each of the two or more first feature points identified by the feature point matching unit and shooting location information indicating the location at which the second image was captured; and a disaster location information calculation unit that calculates disaster location information indicating the geographical location of the point at which the disaster has occurred based on the shooting information and detection location information indicating the location at which the disaster was detected in the second image. Effect of the Invention
[0008] The disaster location identification system according to the present disclosure has an effect of being able to identify the location where a disaster has occurred with high accuracy. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a diagram showing a configuration example of a disaster location identification system according to a first embodiment. [Diagram 2] FIG. 1 is a diagram for explaining acquisition of a still image by the disaster location identification system according to the first embodiment. [Diagram 3] FIG. 11 is a diagram showing an example of a first image displayed by a feature point registration unit of the disaster location identification system according to the first embodiment; [Figure 4] FIG. 1 is a diagram showing an example of a two-dimensional map displayed by a feature point registration unit of the disaster location identification system according to the first embodiment; [Diagram 5] FIG. 1 is a diagram showing an example of feature point information stored in a feature point database of the disaster location identification system according to the first embodiment; [Figure 6]FIG. 1 is a diagram showing a configuration example of a disaster image analysis unit included in a disaster location identification system according to a first embodiment. [Figure 7] FIG. 1 is a diagram for explaining detection of a disaster by a disaster detection unit included in the disaster location specification system according to the first embodiment. [Figure 8] FIG. 1 is a diagram showing an example of feature point information generated by a disaster image analysis unit in the disaster location identification system according to the first embodiment; [Figure 9] FIG. 1 is a diagram showing an example of disaster information generated by a disaster image analysis unit in the disaster location identification system according to the first embodiment. [Figure 10] FIG. 1 is a diagram showing a configuration example of a disaster location identification unit included in a disaster location identification system according to a first embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a result of identifying a first feature point by a feature point matching unit included in the disaster location identification system according to the first embodiment; [Figure 12] FIG. 1 is a diagram showing an example of camera information stored in an image capturing information calculation unit of the disaster location specification system according to the first embodiment; [Figure 13] FIG. 1 is a diagram showing a configuration example of an imaging information calculation unit included in the disaster location specification system according to the first embodiment. [Figure 14] FIG. 1 is a first diagram for explaining an example of a method for calculating a focal length by an imaging information calculation unit included in the disaster location specification system according to the first embodiment; [Figure 15] FIG. 2 is a second diagram for explaining an example of a method for calculating a focal length by the photographing information calculation unit included in the disaster location specification system according to the first embodiment; [Figure 16] FIG. 3 is a third diagram for explaining an example of a method for calculating a focal length by the photographing information calculation unit included in the disaster location specification system according to the first embodiment; [Figure 17] FIG. 4 is a fourth diagram for explaining an example of a method for calculating a focal length by the photographing information calculation unit included in the disaster location specification system according to the first embodiment; [Figure 18] FIG. 5 is a fifth diagram for explaining an example of a method for calculating a focal length by the photographing information calculation unit of the disaster location specification system according to the first embodiment; [Figure 19]FIG. 1 is a diagram for explaining an example of a method for calculating an azimuth angle by an imaging information calculation unit included in the disaster location specification system according to the first embodiment. [Figure 20] FIG. 1 is a diagram for explaining an example of a method for calculating disaster location information by a disaster location information calculation unit included in the disaster location identification system according to the first embodiment. [Figure 21] 1 is a flowchart showing an example of an operation procedure of a pre-analysis device included in the disaster location identification system according to the first embodiment. [Figure 22] 1 is a flowchart showing an example of an operation procedure of a disaster analysis device included in a disaster location identification system according to a first embodiment. [Figure 23] FIG. 1 is a first diagram for explaining a still image when the shooting direction is nearly horizontal in the first embodiment; [Figure 24] FIG. 2 is a second diagram for explaining a still image when the shooting direction is nearly horizontal in the first embodiment; [Diagram 25] FIG. 1 is a diagram showing a configuration example of a processing circuit according to a first embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] A disaster location identification system, a disaster location identification method, and a disaster location identification program according to embodiments will be described in detail below with reference to the drawings.
[0011] Embodiment 1 FIG. 1 is a diagram showing a configuration example of a disaster location identification system 10 according to a first embodiment. The disaster location identification system 10 is a system for identifying a location where a disaster has occurred in a region. Disasters targeted for location identification by the disaster location identification system 10 include, for example, fire, building collapse, or flooding, and include natural disasters and man-made disasters. The disaster location identification system 10 includes a camera 11, an operation device 12, an encoding device 13, a pre-analysis device 14, a disaster analysis device 15, and a display device 16.
[0012] The camera 11 is connected to each of the operation device 12 and the encoding device 13 so as to be able to communicate with each other. The camera 11 is installed in a position overlooking an area to be monitored for occurrence of a disaster. For example, the camera 11 is installed at a high place such as the top of a tall building. The camera 11 captures an image of the area from a position overlooking the area. The camera 11 has a swivel function for performing a swivel operation such as a panning operation and a tilting operation. The swivel function allows the camera 11 to capture an image of the surroundings of the camera 11 over a wide range. The camera 11 transmits the image data generated by the image capture to the encoding device 13. The disaster location identification system 10 may output the image data to the outside of the disaster location identification system 10 so that the image captured by the camera 11 can be displayed on a device outside the disaster location identification system 10.
[0013] The operation device 12 is a device that remotely controls the camera 11. The operation device 12 is an input device that allows manual information input, such as a joystick, an operation panel, a mouse, or a keyboard. In this case, the operation device 12 remotely controls the camera 11 according to an operation of the operation device 12 by an operator or the like. The operation device 12 may also be a device that automatically remotely controls the camera 11. The operation device 12 that automatically remotely controls the camera 11 remotely controls the camera 11 according to a preset procedure, for example.
[0014] The encoding device 13 executes encoding processing of video data. The encoding device 13 periodically generates still images that are snapshots of the video data after encoding processing. The still images generated by the encoding device 13 can be said to be images of an area photographed from a position overlooking the area. The encoding device 13, the pre-analysis device 14, and the disaster analysis device 15 are connected to each other so that they can communicate with each other. The encoding device 13 outputs the generated still images to the pre-analysis device 14 and the disaster analysis device 15.
[0015] The pre-analysis device 14 executes a process in a preparation stage before the occurrence of a disaster. The pre-analysis device 14 acquires a still image captured during normal times when no disaster has occurred from the encoding device 13, and executes an analysis process for the still image. Hereinafter, the still image captured during normal times when no disaster has occurred is also referred to as a first image.
[0016] The pre-analysis device 14 includes a feature point extraction unit 21, a feature point registration unit 22, a geographic information storage unit 23, and a feature point information storage unit 24. The feature point extraction unit 21 extracts first feature points, which are feature points of the first image, from the first image. Details of the feature points will be described later.
[0017] The geographic information storage unit 23 stores geographic information of an area to be monitored for occurrence of a disaster. The geographic information is two-dimensional map information or three-dimensional topographical information. The geographic information includes information indicating features that exist on the earth's surface and location information indicating the geographical location of a location shown on the two-dimensional map or three-dimensional topographical model. The disaster location identification system 10 receives, for example, geographic information acquired from a provider of geographic information. The geographic information input to the disaster location identification system 10 is stored in the geographic information storage unit 23.
[0018] The feature point information storage unit 24 stores a feature point database in which feature point information is stored. The feature point information stored in the feature point information storage unit 24 includes feature point position information, which is information indicating the geographical position of the first feature point. One example of the feature point position information is coordinates indicating the geographical position of the first feature point, which is a combination of information on latitude, longitude, and altitude. The altitude information included in the feature point position information may be information on altitude. The feature point database accumulates feature point information on the first feature point extracted by the feature point extraction unit 21.
[0019] The feature point registration unit 22 obtains feature point position information by matching the first feature points extracted by the feature point extraction unit 21 with the geographic information read out from the geographic information storage unit 23. The feature point registration unit 22 stores the obtained feature point position information in the feature point information storage unit 24. The feature point registration unit 22 registers the first feature points by storing the feature point position information in the feature point information storage unit 24. The feature point registration unit 22 registers the first feature points extracted from each of a plurality of first images captured while changing at least one of the azimuth angle and depression angle of the shooting direction.
[0020] The disaster analysis device 15 detects the occurrence of a disaster by analyzing still images. When the occurrence of a disaster is detected, the disaster analysis device 15 executes a process for identifying the location of the disaster. Hereinafter, the still image in which the occurrence of a disaster is detected is also referred to as the second image. The second image can be said to be an image taken from a position overlooking the area, showing the state in which the disaster occurs in the area.
[0021] The disaster analysis device 15 includes a disaster image analysis unit 31, a disaster location identification unit 32, and a disaster information storage unit 33. The disaster image analysis unit 31 acquires still images from the encoding device 13 and executes analysis processing of the still images. The disaster image analysis unit 31 detects the occurrence of a disaster by analyzing the acquired still images. In addition, the disaster image analysis unit 31 extracts feature points by analyzing the acquired still images.
[0022] When the occurrence of a disaster is detected from the acquired still image, the disaster image analysis unit 31 identifies the position where the occurrence of the disaster was detected in the still image where the occurrence of the disaster was detected. The disaster image analysis unit 31 generates detection position information indicating the position where the occurrence of the disaster was detected in the second image. In addition, when the occurrence of a disaster is detected from the acquired still image, the disaster image analysis unit 31 analyzes the second image, which is the still image where the occurrence of the disaster was detected, to extract second feature points, which are feature points of the second image, from the second image.
[0023] The disaster location identification unit 32 executes a process for identifying the location where a disaster has occurred. The disaster location identification unit 32 includes a feature point matching unit 34, a photographing information calculation unit 35, and a disaster location information calculation unit 36.
[0024] The feature point matching unit 34 matches the second feature point extracted by the disaster image analysis unit 31 with each of the multiple registered first feature points, and identifies the first feature point corresponding to the second feature point. The first feature point corresponding to the second feature point is a first feature point having the same geographical position as the second feature point. Note that the first feature point and the second feature point having the same geographical position are not limited to the case where the geographical position of the first feature point and the geographical position of the second feature point completely match, but also include the case where the first feature point and the second feature point can be considered to exist in the same geographical position. The feature point matching unit 34 identifies the first feature point for each of two or more second feature points extracted from the second image. In this way, the feature point matching unit 34 identifies two or more first feature points for one second image.
[0025] The shooting information calculation unit 35 reads out feature point position information of each of the two or more first feature points identified by the feature point matching unit 34 from the feature point information storage unit 24. The shooting information calculation unit 35 calculates shooting information indicating the conditions at the time of shooting the second image based on the feature point position information of each of the two or more first feature points identified by the feature point matching unit 34 and shooting position information indicating the position at which the second image was shot. The shooting information calculated by the shooting information calculation unit 35 includes information on the focal length at the time of shooting and information on the azimuth angle and depression angle of the shooting direction.
[0026] The disaster information storage unit 33 stores a disaster database in which disaster information is stored. The disaster information stored in the disaster information storage unit 33 includes disaster location information calculated by the disaster location information calculation unit 36. One example of disaster location information is the coordinates of the location where a disaster occurs, which is a combination of latitude, longitude, and altitude information. The altitude information included in the disaster location information may be altitude information. The disaster database accumulates disaster information including disaster location information. Note that the disaster location information is not limited to the coordinates of the location where a disaster occurs, and may be the address of the location where a disaster occurs.
[0027] The disaster location information calculation unit 36 calculates disaster location information indicating the geographical location of a point where a disaster has occurred, based on the detection location information generated by the disaster image analysis unit 31 and the photography information calculated by the photography information calculation unit 35. The disaster location information calculation unit 36 stores disaster information including the calculated disaster location information in the disaster information storage unit 33. The disaster location information calculation unit 36 registers the disaster location information by storing the disaster information in the disaster information storage unit 33.
[0028] The display device 16 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display. The display device 16 is communicably connected to the disaster analysis device 15. The display device 16 reads out disaster information from the disaster information storage unit 33. The display device 16 displays the read out disaster information. For example, the display device 16 displays a two-dimensional map or a three-dimensional terrain model based on the geographic information, and displays the disaster position indicated in the read out disaster information on the two-dimensional map or the three-dimensional terrain model.
[0029] Next, a description will be given of details of operations and processes executed by the disaster location identification system 10 before the occurrence of a disaster. Fig. 2 is a diagram for explaining acquisition of a still image by the disaster location identification system 10 according to the first embodiment.
[0030] Camera 11 performs a panning operation, which is a horizontal swinging operation, and a tilting operation, which is a vertical swinging operation, in accordance with remote control by operating device 12. The two double-headed arrows in FIG. 2 indicate that camera 11 performs a panning operation and a tilting operation. Camera 11 changes the shooting direction by at least one of the panning operation and the tilting operation. The shooting direction is the direction in which camera 11 is pointed, and is the direction of the optical axis of camera 11. Camera 11 also changes the focal length in accordance with remote control by operating device 12. Camera 11 changes the viewing angle and magnification by changing the focal length. Camera 11 shoots an area while appropriately changing the shooting direction, viewing angle, and magnification.
[0031] The disaster location identification system 10 may constantly monitor occurrence of disasters in an area by automatically remotely operating the camera 11 via the operation device 12 to capture images of the area. Alternatively, the disaster location identification system 10 may capture images of an area by an operator or the like remotely operating the camera 11 via the operation device 12. The disaster location identification system 10 may be capable of switching the remote operation of the camera 11 between automatic and manual.
[0032] Encoding device 13 executes encoding processing of video data input from camera 11, and generates still images from the encoded video data. Rectangle 41 indicated by a dashed line in Fig. 2 represents the range captured by camera 11. Fig. 2 shows a schematic diagram of how a plurality of still images are captured with different azimuth angles of the shooting direction.
[0033] The still image output from the encoding device 13 is input to a feature point extraction unit 21 of the pre-analysis device 14. The feature point extraction unit 21 extracts a first feature point from a first image, which is an input still image.
[0034] A feature point is a point detected as a local area that is a feature of an image from which the feature points are extracted. For example, a corner of a subject or a part with a different brightness from the surroundings can be detected as a feature point. For example, a general-purpose method for extracting feature points can be used as a method for extracting the first feature points by the feature point extraction unit 21. The method for extracting the first feature points by the feature point extraction unit 21 is arbitrary. Feature points are robust against image scale, image rotation, and the like. Therefore, the disaster location identification system 10 can match the feature points of the first image with the feature points of the second image when feature points at the same geographical positions exist in the first image and the second image.
[0035] The feature point extraction unit 21 executes a process of extracting a first feature point for each of a plurality of first images captured by changing at least one of the azimuth angle and depression angle of the shooting direction. The feature point extraction unit 21 outputs feature point information including information indicating the first image from which the first feature point was extracted and in-image feature point position information indicating the position of the first feature point in the first image to the feature point registration unit 22. The in-image feature point position information indicating the position of the first feature point in the first image is, for example, the coordinates of the first feature point in the coordinate system of the first image.
[0036] The feature point registration unit 22 obtains feature point position information of the first feature point extracted by the feature point extraction unit 21. As an example, the feature point registration unit 22 displays the first image showing the first feature point and a two-dimensional map side by side, and accepts an operation to specify a point on the two-dimensional map that corresponds to the first feature point on the first image. At that time, the feature point registration unit 22 reads out two-dimensional map information, which is geographic information, from the geographic information storage unit 23, and displays the two-dimensional map based on the two-dimensional map information.
[0037] For example, the pre-analysis device 14 includes a display device that displays the first image and a two-dimensional map, and an input device that accepts an operation to specify a point corresponding to the first feature point. The display device and input device included in the pre-analysis device 14 are not illustrated. The feature point registration unit 22 may cause a display device 16 provided outside the pre-analysis device 14 in the disaster position identification system 10 to display the first image and the two-dimensional map.
[0038] FIG. 3 is a diagram showing an example of a first image displayed by the feature point registration unit 22 of the disaster location identification system 10 according to the first embodiment. FIG. 3 shows an example of the first image and an example of an object 42 representing the first feature point. The object 42 shown in FIG. 3 is a circle surrounding the first feature point. The feature point registration unit 22 draws the object 42 on the first image based on the in-image feature point position information input from the feature point extraction unit 21. FIG. 3 shows a plurality of objects 42. In the example shown in FIG. 3, a plurality of first feature points are extracted from one first image. Note that the display mode of the object 42 is not limited to that shown in FIG. 3 and may be any mode.
[0039] Fig. 4 is a diagram showing an example of a two-dimensional map displayed by the feature point registration unit 22 of the disaster location identification system 10 according to the first embodiment. Fig. 4 shows an example of a map including the range shown in the first image shown in Fig. 3 among the areas to be monitored for occurrence of a disaster.
[0040] An operator or the like operates an input device to specify a point on the two-dimensional map that corresponds to the first feature point. For example, the operator or the like clicks the point on the two-dimensional map that corresponds to the first feature point with the input device to specify the point that corresponds to the first feature point. The feature point registration unit 22 acquires information indicating the geographical position of the specified point from the geographical information. As a result, the feature point registration unit 22 obtains feature point position information that indicates the geographical position of the first feature point. The feature point registration unit 22 adds the obtained feature point position information to the feature point information acquired from the feature point extraction unit 21, and stores the feature point information including the feature point position information in the feature point information storage unit 24.
[0041] In this way, the feature point registration unit 22 obtains feature point position information by manually matching the first feature points with the geographic information, and registers the first feature points. The feature point registration unit 22 registers each of the multiple first feature points by specifying a location on the two-dimensional map that corresponds to the first feature point for each of the multiple first feature points shown in the first image. Note that the method of obtaining feature point position information by the feature point registration unit 22 is not limited to the above method, and may be any method.
[0042] In the above description, the geographical information displayed by the feature point registration unit 22 is a two-dimensional map, but the geographical information displayed by the feature point registration unit 22 may be a three-dimensional topographical model. In this case, an operator or the like operates an input device to specify a point on the three-dimensional topographical model that corresponds to the first feature point.
[0043] The feature point registration unit 22 registers the first feature points extracted from each of the multiple first images captured by changing at least one of the azimuth angle and depression angle of the shooting direction. This allows the disaster location identification system 10 to register first feature points for a wider range around the camera 11 compared to when the azimuth angle and depression angle are both fixed.
[0044] Fig. 5 is a diagram showing an example of feature point information stored in the feature point database of the disaster location identification system 10 according to the first embodiment. Fig. 5 shows an example of a table in which feature point information is stored in the feature point database. In this table, a record for each extracted first feature point is accumulated.
[0045] In the example shown in Fig. 5, the feature point information includes information on each of the following items: feature point ID (IDentifier), camera ID, still image file, feature point matching information, in-image feature point position information, feature point position information, and plane rectangular coordinates. The in-image feature point position information in the feature point information shown in Fig. 5 is information that indicates the position of the first feature point in the first image.
[0046] In FIG. 5, the item of feature point ID indicates a feature point ID that is an identifier of the extracted first feature point. The item of camera ID indicates a camera ID that is an identifier of the camera 11 that captured the first image. The camera ID "Camera01" indicates the camera 11 shown in FIG. 1. The item of still image file indicates information indicating the first image from which the first feature point is extracted. Here, the information indicating the first image from which the first feature point is extracted is the file name of the first image from which the first feature point is extracted. The item of feature point matching information indicates feature point matching information that is information used in matching the first feature point. The item of in-image feature point position information indicates the x coordinate and y coordinate that are in-image feature point position information of the first feature point. The item of feature point position information indicates the longitude, latitude, and altitude of the geographical position of the first feature point. The item of plane rectangular coordinates indicates the X coordinate and Y coordinate of the plane rectangular coordinate system that is information indicating the geographical position of the first feature point.
[0047] The plane rectangular coordinate system is a coordinate system with a north-south axis as the X axis and an east-west axis as the Y axis, centered on a certain fixed position on the earth. The feature point registration unit 22 converts the longitude and latitude information into the X coordinate and the Y coordinate of the plane rectangular coordinate system, thereby registering the plane rectangular coordinate. The disaster location identification system 10 may use the plane rectangular coordinate instead of the longitude and latitude information of the feature point position information in the process after the process of registering the first feature point. The disaster location identification system 10 may calculate the plane rectangular coordinate in the process after the process of registering the first feature point. The feature point database may not include the plane rectangular coordinate. In the example shown in FIG. 5, a record with "-" written in the column of feature point position information and the column of plane rectangular coordinate indicates that the geographical position of the first feature point was not identified.
[0048] Next, details of the operation and processing executed by the disaster location identification system 10 when a disaster occurs will be described. Fig. 6 is a diagram showing an example of the configuration of the disaster image analysis unit 31 included in the disaster location identification system 10 according to the first embodiment. The disaster image analysis unit 31 includes a disaster detection unit 37, a feature point extraction unit 38, and a disaster occurrence determination unit 39. The still image output from the encoding device 13 is input to each of the disaster detection unit 37 and the feature point extraction unit 38.
[0049] The disaster detection unit 37 detects the occurrence of a disaster for each type of disaster by analyzing the still images. The disaster detection unit 37 has, for example, a function for detecting a fire and a function for detecting the collapse of a building. Taking the function for detecting a fire as an example, the disaster detection unit 37 detects the occurrence of a fire by detecting flames or smoke from a still image. A detailed description of the method by which the disaster detection unit 37 detects the occurrence of a disaster will be omitted.
[0050] The disaster detection unit 37 outputs information indicating whether or not a fire has been detected by the function for detecting fire, and information indicating whether or not a building collapse has been detected by the function for detecting building collapse, to the disaster occurrence determination unit 39. In this way, the disaster detection unit 37 outputs the result of detecting the occurrence of a disaster for each type of disaster to the disaster occurrence determination unit 39. Note that disasters detected by the disaster detection unit 37 are not limited to fires and building collapses. It is sufficient for the disaster detection unit 37 to detect the occurrence of at least one phenomenon known as a disaster.
[0051] When the disaster detection unit 37 detects the occurrence of a disaster, it generates disaster information, which is information related to the detected disaster. The disaster information includes information indicating the second image in which the disaster was detected, information indicating the type of the detected disaster, and detection position information. The detection position information is information indicating the position at which the occurrence of the disaster was detected in the second image, which is a still image in which the occurrence of the disaster was detected. The detection position information is, for example, coordinates indicating the position at which the occurrence of the disaster was detected in the coordinate system of the second image. The disaster detection unit 37 outputs the generated disaster information to the disaster occurrence determination unit 39.
[0052] Fig. 7 is a diagram for explaining the detection of a disaster by the disaster detection unit 37 of the disaster location identification system 10 according to the first embodiment. Here, a case where a flame is detected from a still image by a fire detection function is taken as an example. Fig. 7 shows an example of a still image in which a flame is detected. The disaster detection unit 37 generates detection location information regarding the location of the fire by determining the coordinates of the position of the flame in the still image.
[0053] The disaster location identification system 10 may display an object 43 indicating the location of the occurrence of a disaster when, for example, a still image in which a disaster is detected is displayed by the display device 16. In the example shown in Fig. 7, an object 43 indicating the location of an occurrence of a fire is shown. The location of the object 43 is determined based on the detected location information. Note that the display mode of the object 43 is not limited to that shown in Fig. 7 and may be any mode.
[0054] The feature point extraction unit 38 extracts feature points from the input still image. When the disaster detection unit 37 detects the occurrence of a disaster, the feature point extraction unit 38 analyzes the second image, which is a still image in which the occurrence of the disaster is detected, to extract second feature points, which are feature points of the second image, from the second image. The feature point extraction unit 38 outputs information indicating the second image from which the second feature points have been extracted, and feature point information including information indicating the positions of the second feature points in the second image, to the disaster occurrence determination unit 39. The information indicating the positions of the second feature points in the second image is, for example, the coordinates of the second feature points in the coordinate system of the second image.
[0055] The disaster occurrence determination unit 39 obtains the result of detecting the occurrence of a disaster from the disaster detection unit 37 and determines whether or not a disaster has occurred. When it is determined that a disaster has occurred, the disaster occurrence determination unit 39 outputs the still image which is the second image, the feature point information, and the disaster information to the disaster position identification unit 32.
[0056] Fig. 8 is a diagram showing an example of feature point information generated by the disaster image analysis unit 31 in the disaster location identification system 10 according to the first embodiment. Fig. 8 shows a table showing an example of feature point information generated by the disaster image analysis unit 31.
[0057] In the example shown in Fig. 8, the feature point information includes information on each of the items of feature point ID, camera ID, still image file, feature point matching information, and in-image feature point position information. The in-image feature point position information in the feature point information shown in Fig. 8 is information that indicates the position of the second feature point in the second image.
[0058] In FIG. 8, the feature point ID item indicates a feature point ID that is an identifier of the extracted second feature point. The camera ID item indicates a camera ID that is an identifier of the camera 11 that captured the second image. The still image file item indicates information indicating the second image from which the second feature point was extracted. Here, the information indicating the second image from which the second feature point was extracted is the file name of the second image from which the second feature point was extracted. The feature point matching information item indicates feature point matching information that is information used in matching the second feature point. The in-image feature point position information item indicates the x and y coordinates that are in-image feature point position information of the second feature point.
[0059] 9 is a diagram showing an example of disaster information generated by the disaster image analysis unit 31 in the disaster location identification system 10 according to the first embodiment. In FIG. 9, a table showing an example of disaster information generated by the disaster image analysis unit 31 is shown.
[0060] In the example shown in FIG. 9, the disaster information includes information on each of the items of disaster ID, camera ID, still image file, disaster type, detection position information, and disaster position information. The disaster ID item indicates a disaster ID that is an identifier of a disaster detected by the disaster detection unit 37. The camera ID item indicates a camera ID that is an identifier of the camera 11 that captured the second image in which the disaster was detected. The still image file item indicates information indicating the second image in which the disaster was detected. The information indicating the second image is the file name of the second image, as in FIG. 8. The disaster type item indicates the type of disaster detected by the disaster detection unit 37. The detection position information item indicates x and y coordinates that are detection position information indicating the position where the occurrence of the disaster was detected. The disaster position information item is disaster position information indicating the geographical position of the point where the disaster is occurring, and indicates longitude, latitude, and altitude.
[0061] The disaster location information is calculated by the disaster location identification unit 32. When the disaster information is generated by the disaster image analysis unit 31, the disaster information does not include the disaster location information. In Fig. 9, the disaster location information column is left blank to indicate that the disaster location information has not been calculated. When the disaster location information is calculated by the disaster location identification unit 32, the disaster location information is added to the disaster information.
[0062] 10 is a diagram showing a configuration example of the disaster location identification unit 32 included in the disaster location identification system 10 according to the first embodiment. The feature point information on the second feature point, which is the feature point information output from the disaster image analysis unit 31, is input to the feature point matching unit 34. The feature point matching unit 34 reads out the feature point information on the registered first feature point from the feature point information storage unit 24.
[0063] The feature point matching unit 34 matches the feature point information on the second feature point with the feature point information on each of the multiple first feature points. The feature point matching unit 34 identifies one first feature point that is determined to be optimal as a first feature point whose geographic position matches that of the second feature point from among the registered first feature points. In this way, the feature point matching unit 34 matches the second feature point with each of the multiple registered first feature points to identify the first feature point whose geographic position matches that of the second feature point.
[0064] The feature point matching unit 34 reads out feature point position information indicating the geographical position of the identified first feature point from the record for the identified first feature point in the feature point database. In this way, the feature point matching unit 34 acquires the feature point position information for the identified first feature point. The feature point matching unit 34 outputs the acquired feature point position information to the shooting information calculation unit 35.
[0065] FIG. 11 is a diagram showing an example of a result of first feature points being identified by the feature point matching unit 34 of the disaster location identification system 10 according to the first embodiment. FIG. 11 shows an example of a table summarizing information on the first feature points identified by the feature point matching unit 34 from among the first feature points registered in the feature point database. The information shown in FIG. 11 can be said to be information showing a result of extracting the first feature points from the feature point database through matching by the feature point matching unit 34. Hereinafter, the information shown in FIG. 11 will be referred to as extraction result information. The feature point matching unit 34 outputs the extraction result information to the shooting information calculation unit 35.
[0066] In the example shown in FIG. 11, the extraction result information includes information on each of the items of extraction result ID, camera ID, feature point ID, registered feature point ID, in-image feature point position information, and feature point position information. In FIG. 11, the extraction result ID item represents the extraction result ID, which is an identifier of a record of the extraction result information. The camera ID represents the camera ID, which is an identifier of the camera 11 that captured the second image. The feature point ID item represents the feature point ID, which is an identifier of the second feature point. The registered feature point ID item represents the feature point ID, which is an identifier of the extracted first feature point. The in-image feature point position information item represents the x and y coordinates, which are in-image feature point position information of the second feature point. The feature point position information item represents information indicating the geographical position of the extracted first feature point, and represents longitude, latitude, and altitude.
[0067] FIG. 11 shows an example of extraction result information obtained by matching feature point information on the first feature point shown in FIG. 5 with feature point information on the second feature point shown in FIG. 8. For example, assume that the first feature point with feature point ID "101" in FIG. 5 is identified as the first feature point whose geographical position coincides with the second feature point with feature point ID "1" in FIG. 8. The record with extraction result ID "A" in FIG. 11 is a record of extraction result information on the first feature point. The camera ID, feature point ID, and in-image feature point position information in the record with extraction result ID "A" are obtained from the feature point information on the second feature point shown in FIG. 8. The registered ID and feature point position information in the record with extraction result ID "A" are obtained from the feature point information on the first feature point shown in FIG. 5.
[0068] 11 is a record of extraction result information when the first feature point with feature point ID "103" in Fig. 5 is identified for the second feature point with feature point ID "2" in Fig. 8. Also, the record of extraction result ID "C" in Fig. 11 is a record of extraction result information when the first feature point with feature point ID "104" in Fig. 5 is identified for the second feature point with feature point ID "4" in Fig. 8.
[0069] The photographing information calculation unit 35 acquires feature point position information for the first feature point identified by the feature point matching unit 34 by acquiring the extraction result information output from the feature point matching unit 34. The feature point matching unit 34 acquires feature point position information for each of the two or more first feature points identified by the feature point matching unit 34. The photographing information calculation unit 35 also stores camera information that is information about the camera 11 that captured the second image. The camera information includes shooting position information that indicates the position of the camera 11. The photographing information calculation unit 35 calculates the photographing information based on the feature point position information for each of the two or more first feature points and the shooting position information.
[0070] Fig. 12 is a diagram showing an example of camera information stored in the photography information calculation unit 35 of the disaster location identification system 10 according to the first embodiment. Fig. 12 shows a table showing an example of camera information. In the example shown in Fig. 12, the camera information includes information on each of the items of camera ID, photography position information, and plane rectangular coordinates. The photography position information item is information indicating the geographical position of the camera 11, and indicates longitude, latitude, and altitude. The plane rectangular coordinate item is information indicating the geographical position of the camera 11, and indicates the X coordinate and Y coordinate of the plane rectangular coordinate system.
[0071] 12 shows the shooting position information and plane rectangular coordinates for the camera ID "Camera01." When multiple cameras 11 are used to shoot a still image, the camera information for each camera 11 is stored in the shooting information calculation unit 35.
[0072] 13 is a diagram showing a configuration example of the imaging information calculation unit 35 included in the disaster location identification system 10 according to the first embodiment. The imaging information calculation unit 35 has a focal length calculation function, an azimuth angle calculation function, and a depression angle calculation function. The imaging information calculation unit 35 calculates the focal length of the camera 11 at the time of imaging by the focal length calculation function. The imaging information calculation unit 35 calculates the azimuth angle of the imaging direction of the camera 11 by the azimuth angle calculation function. The imaging information calculation unit 35 calculates the depression angle of the imaging direction of the camera 11 by the depression angle calculation function. Specific examples of processing by each of the focal length calculation function, the azimuth angle calculation function, and the depression angle calculation function will be described below.
[0073] Fig. 14 is a first diagram for explaining an example of a method for calculating a focal length by the imaging information calculation unit 35 of the disaster location identification system 10 according to the first embodiment. Fig. 14 shows a plane rectangular coordinate system with a point O as the origin. The point O is the geographical position of the camera 11, and represents the position indicated by the imaging position information. In Fig. 14, the direction of the arrow representing the X-axis of the plane rectangular coordinate system represents the north direction, and the direction of the arrow representing the Y-axis of the plane rectangular coordinate system represents the east direction.
[0074] Each of points A, B, and C is a geographical position of a first feature point registered in the feature point database, and represents a position indicated by the feature point position information. Point o represents the center position of image plane 44 of the still image. Each of points a, b, and c represents a position of a first feature point on image plane 44, and represents a position indicated by the in-image feature point position information. LOS is a linear direction from point O to point o, and represents the shooting direction of camera 11. Distance l is the distance between point O and image plane 44, and is the focal length of camera 11.
[0075] The photographing information calculation unit 35 calculates the angles ∠XOA between the X-axis and the straight line OA, ∠XOB between the X-axis and the straight line OB, and ∠XOC between the X-axis and the straight line OC. ∠XOA, ∠XOB, and ∠XOC are the azimuth angles of points A, B, and C, respectively. The angles of ∠XOA, ∠XOB, and ∠XOC are angles measured clockwise with north being 0 degrees. Based on these angles, the photographing information calculation unit 35 calculates θ, which is the angle of ∠AOB. AB and θ, the angle of ∠BOC BC and θ, the angle of ∠COA CA We ask for:
[0076] Fig. 15 is a second diagram for explaining an example of a method for calculating a focal length by the imaging information calculation unit 35 included in the disaster location identification system 10 according to the first embodiment. Fig. 16 is a third diagram for explaining an example of a method for calculating a focal length by the imaging information calculation unit 35 included in the disaster location identification system 10 according to the first embodiment.
[0077] FIG. 15 shows the points shown in FIG. 14 and the image plane 44 shown in FIG. 14 viewed from oblique directions relative to the x-axis and y-axis. In FIG. 15, the x-axis and y-axis are two axes perpendicular to each other on the image plane 44. The direction indicated by the arrow in the x-axis direction is the positive direction. The direction indicated by the arrow in the y-axis direction is the positive direction. The coordinates of points o, a, b, and c on the image plane 44 are defined as (0,0), (a x ,a y ), (b x ,b y ), and (c x ,c y ) The axis perpendicular to the x-axis and y-axis is the z-axis. The positive direction of the z-axis, as indicated by the arrow, is the opposite direction to the LOS.
[0078] FIG. 16 shows the points shown in FIG. 15 and image plane 44 shown in FIG. 15 viewed from the negative side of the y axis. The left-right direction in FIG. 16 is the direction of the x axis, and the up-down direction in FIG. 16 is the direction of the z axis. The direction perpendicular to the paper surface in FIG. 16 is the direction of the y axis. Here, as shown in FIG. 16, if the position of camera 11 in the coordinate system of image plane 44 is point L, the coordinates of point L are (0,0,l). The shooting information calculation unit 35 specifies the position of point L to calculate distance l, which is the focal length.
[0079] In FIG. 16, each of points a, b, and c shown in FIG. 15 is replaced with a line of y=0 and z=0 in the image plane 44, that is, a point on the x-axis. That is, in FIG. 16, the coordinates of each of points a, b, and c are expressed as follows: x ,0,0), (b x ,0,0), and (c x ,0,0). Note that c x x <0 x holds true.
[0080] FIG. 17 is a fourth diagram for explaining an example of a method for calculating a focal length by the photographing information calculation unit 35 of the disaster location identification system 10 according to the first embodiment. FIG. 17 shows three circles, abL passing through the three points L, a, and b, bcL passing through the three points L, b, and c, and caL passing through the three points L, c, and a, which are assumed based on the points L, a, b, and c shown in FIG. 16. Point L is the intersection of the three circles. The position of point L can be identified by finding a solution from the equations that represent each of these three circles.
[0081] Fig. 18 is a fifth diagram for explaining an example of a method for calculating a focal length by the imaging information calculation unit 35 included in the disaster location identification system 10 according to the first embodiment. Fig. 18 shows a circle abL, which is one of the three circles shown in Fig. 17. Here, an example of finding the equation of the circle abL will be explained by taking the circle abL as an example.
[0082] Here, the vertex of an isosceles triangle with the line segment ab as its base, and its position on the circle abL, is called point L'. By obtaining the coordinates of the three points on the circle abL, the equation of the circle abL can be obtained. The coordinates of points a and b are (a x ,0,0) and (b x Since the coordinates of point L' are obtained as abL, the equation of the circle abL can be obtained by obtaining the coordinates of point L'.
[0083] The angle of ∠aL'b is the angle of ∠AOB, θ AB If the inscribed angle is the same, then point L' can be said to be on the circumference of circle abL. If the intersection point of the perpendicular line of the line segment ab that passes through point L' and the line segment ab is point o', then the coordinates of point o' are ((a x +b x ) / 2,0,0). From this, the coordinates of point L' are ((a x +b x ) / 2,0,l'). Furthermore, since △ao'L' is a right-angled triangle, ∠aL'o'=θ AB / 2 holds. From the above, the z coordinate l' of point L' is expressed by the following formula (1). In formula (1), the symbol with a bar above "ao'" represents the length of the line segment ao'.
[0084]
number
[0085] The coordinates of points a, b, and L' are expressed by the following equation (2).
[0086]
number
[0087] Based on the coordinates of points a, b, and L', the three equations shown in the following equation (3) can be created.
[0088]
number
[0089] By solving the three equations shown in (3) for α, β, and γ, we can determine the constant α AB , β AB , and γ AB This gives us the equation of the circle abL.
[0090] For the circle bcL, as in the case of the circle abL, the constant α BC , β BC , and γ BC By obtaining the equation of the circle bcL, we can obtain the equation of the circle caL. As with the case of the circle abL, the equation of the circle caL can be obtained by obtaining the constant α CA , β CA , and γ CA By obtaining the above equation, we can obtain the equation of the circle caL.
[0091] From the above, the equations of the circles abL, bcL, and caL can be obtained as shown in the following equation (4).
[0092]
number
[0093] By solving these equations shown in formula (4), two intersections of the circle abL and the circle bcL, two intersections of the circle bcL and the circle caL, and two intersections of the circle caL and the circle abL are obtained. The position where the distance between the obtained intersections is smaller than, for example, a preset ε is set as point L. By specifying the position of point L in the above manner, the distance l, which is the focal length, is obtained. Note that the method of calculating the focal length by the photographing information calculation unit 35 is not limited to the above method. The photographing information calculation unit 35 can calculate the focal length by any method.
[0094] 19 is a diagram for explaining an example of a method for calculating an azimuth angle by the photographing information calculation unit 35 of the disaster location identification system 10 according to the first embodiment. In FIG. 19, points A, B, C, O, a, b, and c and an angle θ AB , angle θ BC , and angle θ CA The angles and LOS are the same as those shown in FIG. 14. As an example, the photographing information calculation unit 35 calculates the azimuth angle θ A The angle θ between LOS and the line segment OA LA By adding these, the azimuth angle of the shooting direction θ L Calculate.
[0095] The coordinates of point a are (a x ,a y ) and distance l to find the angle θ LA is expressed by the following equation (5).
[0096]
number
[0097] Azimuth angle θ L θ L =θ A -θ LA and equation (5). The azimuth angle θ L By calculating the average of the results, the azimuth angle θ L It should be noted that the method of calculating the azimuth angle of the shooting direction by the shooting information calculation unit 35 is not limited to the above method. The shooting information calculation unit 35 can calculate the azimuth angle of the shooting direction by any method.
[0098] Angle of depression φ A is the angle between the horizontal plane and the line segment OA. The coordinates of point a are (a x ,a y ) and distance l, the angle φ LA is expressed by the following equation (6).
[0099]
number
[0100] Elevation angle φ of the shooting direction L is φ L =φ A -φ LA The elevation angle φ can be calculated based on points A, B, and C. L By calculating the average of the results, the elevation angle φ L It should be noted that the method for calculating the elevation angle of the shooting direction by the shooting information calculation unit 35 is not limited to the above method. The shooting information calculation unit 35 can calculate the elevation angle of the shooting direction by any method.
[0101] 20 is a diagram for explaining an example of a method for calculating disaster location information by the disaster location information calculation unit 36 included in the disaster location identification system 10 according to the first embodiment. The disaster information output from the disaster image analysis unit 31 is input to the disaster location information calculation unit 36. The imaging information calculated by the imaging information calculation unit 35 is input to the disaster location information calculation unit 36.
[0102] The photographing information calculation unit 35 calculates the direction vector of the LOS based on the photographing information, such as the focal length, azimuth angle, and depression angle. The LOS direction vector passes through point o, which is the center of the image plane 44. A direction vector 45 from the position of the camera 11 toward the disaster position passes through a position 46 on the image plane 44 where the disaster position is captured. The position 46 is a position indicated by the detected position information generated by the disaster image analysis unit 31. The photographing information calculation unit 35 calculates the direction vector 45 toward the disaster position by performing a conversion that rotates the LOS direction vector so that it passes through the position 46.
[0103] The disaster location information calculation unit 36 reads out the geographic information from the geographic information storage unit 23. For example, the disaster location information calculation unit 36 calculates the latitude, longitude, and altitude of the disaster location by determining the intersection 48 between the three-dimensional topographical model 47, which is the geographic information, and the direction vector 45 toward the disaster location. As described above, the disaster location information calculation unit 36 calculates disaster location information indicating the geographic location of the point where the disaster occurs, based on the detection location information generated by the disaster image analysis unit 31 and the photography information calculated by the photography information calculation unit 35. Note that the method of calculating the disaster location information by the disaster location information calculation unit 36 is not limited to the above method. The disaster location information calculation unit 36 can calculate the disaster location information by any method.
[0104] The disaster location information calculation unit 36 adds the determined disaster location information to the disaster information input to the disaster location information calculation unit 36 , and stores the disaster information including the disaster location information in the disaster information storage unit 33 .
[0105] In the above description, the feature point matching unit 34 identifies two or more first feature points for one second image. From the viewpoint of making it easier to identify the disaster occurrence location, the feature point matching unit 34 may identify three or more first feature points for one second image. In the disaster location identification system 10, the feature point registration unit 22 of the pre-analysis device 14 registers first feature points extracted at various magnifications for one shooting direction. This makes it possible for the disaster location identification system 10 to identify a sufficient number of first feature points from the second image to identify the disaster occurrence location, regardless of the magnification of the second image when a disaster occurs.
[0106] If two or more feature points or three or more feature points are not extracted from one second image, the disaster location identification system 10 may urge the operator, etc., by an announcement or the like, to change at least one of the focal length and the shooting direction and capture the second image again. The operator, etc., changes at least one of the focal length and the shooting direction by operating the operation device 12. This allows the disaster location identification system 10 to extract two or more feature points or three or more feature points from the second image acquired after at least one of the focal length and the shooting direction is changed.
[0107] Alternatively, when two or more feature points or three or more feature points are not identified for one second image, the disaster location identification system 10 may automatically adjust at least one of the focal length and the shooting direction via the operation device 12. In this case as well, the disaster location identification system 10 can extract two or more feature points or three or more feature points from the second image acquired after at least one of the focal length and the shooting direction is changed.
[0108] When the image captured by the camera 11 is displayed on a device external to the disaster location identification system 10, the disaster location identification system 10 may add an object representing a feature point to the image output to the external device. This allows the operator or the like to operate the operation device 12 while checking from the image whether or not the second image contains a number of feature points sufficient to identify the disaster occurrence location. The disaster location identification system 10 may also display the number of feature points contained in the image together with the image. This allows the operator or the like to operate the operation device 12 while checking the number of feature points contained in the image. The disaster location identification system 10 may also display the image captured by the camera 11 on the display device 16.
[0109] The disaster location identification system 10 may select one of the multiple cameras 11 to acquire a still image. In this case, when the second image does not contain a number of feature points that can identify the disaster occurrence location, the disaster location identification system 10 may urge the operator, etc., by an announcement, etc., to switch the camera 11 that acquires the still image. When the camera 11 can be selected by operating the operation device 12, the operator, etc., switches the camera 11 that acquires the still image by operating the operation device 12. Alternatively, the disaster location identification system 10 may automatically switch the camera 11 that acquires the still image by the operation device 12 when the second image does not contain a number of feature points that can identify the disaster occurrence location. In this way, the disaster location identification system 10 can obtain a second image that includes a number of feature points that can identify the disaster occurrence location.
[0110] When two or more feature points or three or more feature points are extracted from one second image, the disaster location identification system 10 may zoom the camera 11 to such an extent that all of the extracted feature points are within the shooting range. By zooming the camera 11, the disaster location identification system 10 can further improve the accuracy of identifying the disaster location.
[0111] Next, a description will be given of the operation of the pre-analysis device 14. Fig. 21 is a flowchart showing an example of an operation procedure of the pre-analysis device 14 included in the disaster location identification system 10 according to the first embodiment.
[0112] In step S1, the feature point extraction unit 21 of the pre-analysis device 14 extracts a first feature point from a first image. The feature point extraction unit 21 outputs feature point information including feature point position information in the image of the first feature point to the feature point registration unit 22.
[0113] In step S2, the feature point registration unit 22 of the pre-analysis device 14 obtains feature point position information by matching the first feature point with the geographic information. In step S3, the feature point registration unit 22 registers the first feature point by storing feature point information including the feature point position information in the feature point information storage unit 24.
[0114] In step S4, the feature point registration unit 22 judges whether or not the registration of all the first feature points capable of identifying geographical positions among the first feature points extracted from the first image in step S1 has been completed. If there are first feature points capable of identifying geographical positions and for which registration has not been completed (step S4, No), the pre-analysis device 14 repeats the procedure from step S2 for the first feature points for which registration has not been completed. On the other hand, if the registration of all the first feature points capable of identifying geographical positions has been completed (step S4, Yes), the pre-analysis device 14 ends the operation according to the procedure shown in Fig. 21. The pre-analysis device 14 repeats the operation according to the procedure shown in Fig. 21 for each of the multiple first images input to the pre-analysis device 14.
[0115] Next, a description will be given of the operation of the disaster analysis device 15. Fig. 22 is a flowchart showing an example of an operation procedure of the disaster analysis device 15 included in the disaster location specification system 10 according to the first embodiment.
[0116] In step S11, the disaster detection unit 37 of the disaster analysis device 15 detects the occurrence of a disaster by analyzing the still image. When the disaster detection unit 37 detects the occurrence of a disaster, it generates disaster information, which is information related to the detected disaster. The disaster detection unit 37 outputs the disaster information including detection position information indicating the position where the occurrence of the disaster was detected to the disaster occurrence determination unit 39.
[0117] In step S12, the feature point extraction unit 38 of the disaster analysis device 15 extracts a second feature point from the second image, which is a still image in which the occurrence of a disaster is detected. The feature point extraction unit 38 outputs feature point information of the second feature point to the disaster occurrence determination unit 39. When it is determined that a disaster has occurred, the disaster occurrence determination unit 39 outputs the still image, which is the second image, the feature point information, and the disaster information to the disaster position identification unit 32 of the disaster analysis device 15.
[0118] In step S13, the feature point matching unit 34 of the disaster location identification unit 32 matches the second feature point extracted in step S12 with each of the multiple first feature points registered in step S3, and identifies the first feature point corresponding to the second feature point. The feature point matching unit 34 outputs feature point position information of the identified first feature point to the photography information calculation unit 35 of the disaster location identification unit 32.
[0119] In step S14, the photographing information calculation unit 35 calculates photographing information based on the feature point position information of each of the two or more first feature points and the photographing position information indicating the position where the second image was photographed. The photographing information calculation unit 35 outputs the calculated photographing information to the disaster position information calculation unit 36 of the disaster position identification unit 32.
[0120] In step S15, the disaster position information calculation unit 36 calculates disaster position information based on the detection position information indicating the position where the occurrence of the disaster was detected in the second image and the shooting information. In step S16, the disaster position information calculation unit 36 stores the disaster information including the disaster position information in the disaster information storage unit 33. With the above, the disaster analysis device 15 ends the operation according to the procedure shown in FIG.
[0121] Next, the advantages of the disaster location identification system 10 in detecting a disaster will be described. Here, a comparative example will be taken of a case where shooting information is obtained based on measurement values and a disaster location is identified. In the comparative example, it is assumed that a measuring unit is provided in the camera that captures the image. The measurement value by the measuring unit provided in the camera contains errors, large and small. For example, an error in the azimuth angle and an error in the depression angle each cause a deviation between the vector of the shooting direction obtained by calculation and the line of sight of the actual image. If a deviation occurs between the vector of the shooting direction and the line of sight, a deviation will occur between the disaster occurrence position on the image and the calculated disaster occurrence position. Also, if there is an error in the focal length, a deviation will occur between the shooting range calculated based on the focal length and the actual shooting range, so a deviation will occur between the disaster occurrence position on the image and the calculated disaster occurrence position. The effect of such errors becomes more noticeable as the distance from the center of the image increases.
[0122] In addition, when monitoring the occurrence of a disaster in a region, the shooting direction is likely to be nearly horizontal in order to include distant locations from the camera in the shooting range. When the shooting direction is nearly horizontal, the more distant an object is, the smaller it will appear in the image, and the further away it is, the more difficult it is to distinguish the difference in position in the depth direction of the ground surface.
[0123] Fig. 23 is a first diagram for explaining a still image when the shooting direction is nearly horizontal in embodiment 1. Fig. 24 is a second diagram for explaining a still image when the shooting direction is nearly horizontal in embodiment 1. Fig. 23 shows a schematic example of a still image when the shooting direction is nearly horizontal.
[0124] It is assumed that a plurality of districts are captured in the still image shown in Fig. 23. In Fig. 23, lines representing the boundaries of five districts that are aligned in the shooting direction among the plurality of districts are shown. As shown in Fig. 24, it is assumed that the five districts are District A, District B, District C, District D, and District E, in that order from shooting position 52. Fig. 24 shows the boundaries of each of Districts A, B, C, D, and E when viewed from above.
[0125] In Fig. 23, a plurality of pins 51 arranged in the depth direction are drawn in a portion of the still image showing the ground surface. In Fig. 23, the plurality of pins 51 are arranged at equal intervals on the still image. Fig. 24 shows a top view of the plurality of pins 51 arranged on the ground surface in the manner shown in Fig. 23. As shown in Fig. 24, the intervals between the pins 51 become larger as they move away from the shooting position 52. This means that the depth direction length of the ground surface per pixel of the still image becomes longer as they move away from the shooting position 52. It becomes more difficult to identify the position in the depth direction as they move away from the shooting position 52.
[0126] In the comparative example, the deterioration of the accuracy of identifying the location due to the error of the measurement value becomes more significant as the disaster occurrence location is farther away from the photographing location 52. In other words, even if there is a slight error in the measurement value, the accuracy of identifying the location when identifying a distant disaster occurrence location becomes low.
[0127] In contrast, the disaster location identification system 10 according to the first embodiment calculates the photographing information based on the feature point location information of the first feature point and the photographing location information. That is, the disaster location identification system 10 does not obtain the photographing information from the measurement value by the measurement unit, but calculates the photographing information based on the information obtained from the first image and the second image, which are still images. The disaster location identification system 10 can identify the disaster occurrence location with high accuracy by excluding the measurement value, which is a parameter that may include an error, from the process of identifying the disaster occurrence location. The disaster location identification system 10 can prevent a situation in which the location identification accuracy drops significantly when identifying a disaster occurrence location in a distant place.
[0128] Furthermore, because disaster location identification system 10 can acquire shooting information based on information obtained from the first and second images, which are still images, camera 11 does not need to be equipped with a measuring unit for measuring focal length, azimuth angle, and depression angle. Even when using camera 11 that is not equipped with a measuring unit for acquiring shooting information, disaster location identification system 10 can identify the disaster occurrence location with high accuracy.
[0129] In the above description, the pre-analysis device 14 and the disaster analysis device 15 are input with still images captured by the same camera 11, but this is not limited to the above. The camera 11 capturing the still images input to the pre-analysis device 14 and the camera 11 capturing the still images input to the disaster analysis device 15 may be different from each other. The camera 11 capturing the still images input to the pre-analysis device 14 and the camera 11 capturing the still images input to the disaster analysis device 15 may be installed at the same position or at different positions.
[0130] Next, the hardware configuration of each of the pre-analysis device 14 and the disaster analysis device 15 will be described. Each of the pre-analysis device 14 and the disaster analysis device 15 is realized by a processing circuit. The processing circuit is a circuit in which a processor executes software. Each of the pre-analysis device 14 and the disaster analysis device 15 is realized by a computer system equipped with a processing circuit.
[0131] 25 is a diagram showing a configuration example of the processing circuit 60 according to the first embodiment. The processing circuit 60 includes an input unit 61, a processor 62, a memory 63, and an output unit 64. The input unit 61 is an interface circuit that receives data input from outside the processing circuit 60 and provides the data to the processor 62. The output unit 64 is an interface circuit that transmits data from the processor 62 or the memory 63 to outside the processing circuit 60.
[0132] The feature point extraction unit 21 and the feature point registration unit 22, which are processing units of the pre-analysis device 14, are realized by software, firmware, or a combination of software and firmware. The disaster image analysis unit 31 and the disaster position identification unit 32, which are processing units of the disaster analysis device 15, are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 63.
[0133] In the processing circuit 60 for realizing the pre-analysis device 14, the processor 62 reads and executes a program stored in the memory 63, thereby realizing the function of the pre-analysis device 14. That is, the processing circuit 60 for realizing the pre-analysis device 14 includes a memory 63 for storing a program that will result in the processing of the pre-analysis device 14 being executed. In the processing circuit 60 for realizing the disaster analysis device 15, the processor 62 reads and executes a program stored in the memory 63, thereby realizing the function of the disaster analysis device 15. That is, the processing circuit 60 for realizing the disaster analysis device 15 includes a memory 63 for storing a program that will result in the processing of the disaster analysis device 15 being executed. It can be said that the program stored in the memory 63 of the processing circuit 60 for realizing the pre-analysis device 14 and the program stored in the memory 63 of the processing circuit 60 for realizing the disaster analysis device 15 are what cause a computer to execute the procedure and method of the disaster location identification system 10.
[0134] The processor 62 is a central processing unit (CPU), a processing device, an arithmetic device, a microprocessor, a microcomputer, a processor, or a digital signal processor (DSP). The memory 63 is, for example, a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a digital versatile disk (DVD), such as a non-volatile or volatile semiconductor memory. The geographic information storage unit 23 and the feature point information storage unit 24 of the advance analysis device 14 and the disaster information storage unit 33 of the disaster analysis device 15 are realized by the memory 63.
[0135] At least one of the pre-analysis device 14 and the disaster analysis device 15 may include an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The program may be stored in a recording medium such as a CD (Compact Disc)-ROM or a DVD-ROM, and the recording medium may be provided to realize the procedures and methods of the disaster location identification system 10.
[0136] Each component of the disaster location identification system 10 does not have to be physically configured as shown in the figure. The specific form of distribution and integration of each component is not limited to that shown in the figure. Each component may be functionally or physically distributed in any unit, or may be integrated.
[0137] In the above description, the disaster location identification system 10 includes the pre-analysis device 14 and the disaster analysis device 15, which are separate devices, but this is not limited to the above. The disaster location identification system 10 may implement the functions of the pre-analysis device 14 and the disaster analysis device 15 in a single device. The display device 16 may be included in the disaster analysis device 15. The encoding device 13 may be included in each of the pre-analysis device 14 and the disaster analysis device 15.
[0138] According to the first embodiment, the feature point registration unit 22 collates the first feature point with the geographic information and registers the first feature point by storing the feature point position information of the first feature point. The feature point collation unit 34 collates the second feature point with each of the registered first feature points and identifies the first feature point corresponding to the second feature point. The shooting information calculation unit 35 calculates shooting information based on the feature point position information and shooting position information of each of the two or more first feature points. The disaster position information calculation unit 36 calculates disaster position information based on the detection position information indicating the position where the occurrence of the disaster was detected and the shooting information. The disaster position identification system 10 can identify the position where the disaster occurs with high accuracy by calculating the shooting information based on information obtained from the first image and the second image, which are still images.
[0139] The photographing information calculated by the photographing information calculation unit 35 includes information on the focal length at the time of photographing, and information on the azimuth angle and depression angle of the photographing direction. This enables the disaster location identification system 10 to calculate disaster location information based on the detected location information and the photographing information.
[0140] The feature point registration unit 22 registers the first feature points extracted from each of the first images captured by changing at least one of the azimuth angle and depression angle of the shooting direction. This enables the disaster location identification system 10 to register the first feature points for a wide range around the camera 11.
[0141] The disaster image analysis unit 31 detects the occurrence of a disaster and extracts a second feature point from a second image in which the occurrence of the disaster is detected. The feature point matching unit 34 of the disaster location identification unit 32 matches the second feature point extracted by the disaster image analysis unit 31 with each of the multiple registered first feature points to identify the first feature point corresponding to the second feature point. This enables the disaster location identification system 10 to calculate shooting information based on the feature point position information of the first feature point identified by the feature point matching unit 34. In addition, the disaster location identification system 10 can obtain detection position information for calculating disaster location information by detecting the occurrence of a disaster in the disaster image analysis unit 31.
[0142] As described above, the disaster location identification system 10 according to the first embodiment has an effect of being able to identify the location where a disaster has occurred with high accuracy.
[0143] The configurations shown in the above embodiments are examples of the contents of the present disclosure. The configurations of the embodiments can be combined with other known technologies. Part of the configurations of the embodiments can be omitted or modified without departing from the gist of the present disclosure.
[0144] Various aspects of the present disclosure are summarized below as appendices.
[0145] (Appendix 1) a feature point registration unit that obtains feature point position information indicating a geographical position of a first feature point by comparing a first image obtained by photographing an area from a position overlooking the area with geographical information of the area, and registers the first feature point by storing the feature point position information; a feature point matching unit that matches second feature points that are feature points of a second image capturing a state in which a disaster has occurred in the area with each of the registered multiple first feature points, and identifies the first feature points that correspond to the second feature points; a photographing information calculation unit that calculates photographing information indicating a condition at the time of photographing the second image based on the feature point position information of each of the two or more first feature points identified by the feature point matching unit and photographing position information indicating a position at which the second image was photographed; and a disaster position information calculation unit that calculates disaster position information indicating a geographical position of a point where the disaster is occurring based on detection position information indicating a position where the occurrence of the disaster is detected in the second image and the shooting information. A disaster location identification system. (Appendix 2) The second image is an image taken from a position overlooking the area, The shooting information includes information on the focal length at the time of shooting, and information on the azimuth angle and depression angle of the shooting direction. 2. A disaster location identification system according to claim 1. (Appendix 3) The feature point registration unit registers the first feature points extracted from each of the first images captured while changing at least one of an azimuth angle and a depression angle of a shooting direction. 3. A disaster location identification system according to claim 1 or 2. (Appendix 4) a disaster image analysis unit that detects the occurrence of the disaster by analyzing an image of the area captured from a position overlooking the area, and extracts the second feature point from the second image, which is an image in which the occurrence of the disaster is detected; The feature point matching unit matches the second feature points extracted by the disaster image analysis unit with each of the plurality of registered first feature points. 4. A disaster location identification system according to claim 1, (Appendix 5) A disaster location identification method for identifying a geographical location of a point where a disaster is occurring in a region by a computer, comprising: a step of obtaining feature point position information indicating a geographical position of a first feature point by matching a first image obtained by photographing the area from a position overlooking the area with geographical information of the area, and registering the first feature point by storing the feature point position information; A step of comparing second feature points, which are feature points of a second image capturing a state in which a disaster has occurred in the region, with each of the registered multiple first feature points, and identifying the first feature points corresponding to the second feature points; a step of estimating shooting information indicating a condition at the time of shooting the second image based on the feature point position information of each of the two or more first feature points identified in the step of identifying the first feature points and shooting position information indicating a position at which the second image was shot; and calculating disaster location information indicating a geographical location of a point where the disaster is occurring based on detection location information indicating a location where the occurrence of the disaster is detected in the second image and the shooting information. A disaster location identification method comprising: (Appendix 6) The second image is an image taken from a position overlooking the area, The shooting information includes information on the focal length at the time of shooting, and information on the azimuth angle and depression angle of the shooting direction. 6. A disaster location identification method according to claim 5, (Appendix 7) In the step of registering the first feature points, the first feature points extracted from each of the plurality of first images captured while changing at least one of an azimuth angle and a depression angle of a shooting direction are registered. 7. The disaster location identification method according to claim 5 or 6, (Appendix 8) The method further includes a step of detecting the occurrence of the disaster by analyzing an image of the area captured from a position overlooking the area, and extracting the second feature point from the second image, which is an image in which the occurrence of the disaster is detected; In the step of identifying the first feature points corresponding to the second feature points, the second feature points extracted from the second image are matched with each of a plurality of registered first feature points. 8. The disaster location identification method according to claim 5, further comprising: (Appendix 9) a step of registering the first feature point by comparing a first image obtained by photographing the area from a position overlooking the area with geographic information of the area to obtain feature point position information indicating the geographic position of the first feature point, and storing the feature point position information; A step of comparing second feature points, which are feature points of a second image capturing a state in which a disaster has occurred in the region, with each of the plurality of registered first feature points, and identifying the first feature points whose geographical positions coincide with those of the second feature points; a step of estimating shooting information indicating a condition at the time of shooting the second image based on the feature point position information of each of the two or more first feature points identified in the step of identifying the first feature points and shooting position information indicating a position at which the second image was shot; and calculating disaster location information indicating a geographical location of a point where the disaster is occurring based on detection location information indicating a location where the occurrence of the disaster is detected in the second image and the shooting information. A disaster location identification program comprising: [Explanation of symbols]
[0146] 10 disaster location identification system, 11 camera, 12 operation device, 13 encoding device, 14 pre-analysis device, 15 disaster analysis device, 16 display device, 21, 38 feature point extraction unit, 22 feature point registration unit, 23 geographic information storage unit, 24 feature point information storage unit, 31 disaster image analysis unit, 32 disaster location identification unit, 33 disaster information storage unit, 34 feature point matching unit, 35 shooting information calculation unit, 36 disaster location information calculation unit, 37 disaster detection unit, 39 disaster occurrence determination unit, 41 rectangle, 42, 43 object, 44 image plane, 45 directional vector, 46 position, 47 terrain model, 48 intersection, 51 pin, 52 shooting position, 60 processing circuit, 61 input unit, 62 processor, 63 memory, 64 output unit.
Claims
1. a feature point registration unit that obtains feature point position information indicating a geographical position of a first feature point by comparing a first image obtained by photographing an area from a position overlooking the area with geographical information of the area, and registers the first feature point by storing the feature point position information; a feature point matching unit that matches second feature points that are feature points of a second image capturing a state in which a disaster has occurred in the region with each of the plurality of registered first feature points, and identifies the first feature points that correspond to the second feature points; a photographing information calculation unit that calculates photographing information indicating a condition at the time of photographing the second image, based on the feature point position information of each of the two or more first feature points specified by the feature point matching unit and photographing position information indicating a position at which the second image was photographed; and a disaster position information calculation unit that calculates disaster position information indicating a geographical position of a point where the disaster is occurring based on detection position information indicating a position where the occurrence of the disaster is detected in the second image and the shooting information. A disaster location identification system.
2. The second image is an image taken from a position overlooking the area, The shooting information includes information on the focal length at the time of shooting, and information on the azimuth angle and depression angle of the shooting direction.
2. The disaster location identification system according to claim 1 .
3. The feature point registration unit registers the first feature points extracted from each of the first images captured while changing at least one of an azimuth angle and a depression angle of a shooting direction.
2. The disaster location identification system according to claim 1 .
4. a disaster image analysis unit that detects the occurrence of the disaster by analyzing an image of the area captured from a position overlooking the area, and extracts the second feature point from the second image, which is an image in which the occurrence of the disaster is detected; The feature point matching unit matches the second feature points extracted by the disaster image analysis unit with each of the plurality of registered first feature points.
4. The disaster location identification system according to claim 1, wherein the disaster location identification system is a disaster location identification system.
5. A disaster location identification method for identifying a geographical location of a point where a disaster is occurring in a region by a computer, comprising: a step of obtaining feature point position information indicating a geographical position of a first feature point by matching a first image obtained by photographing the area from a position overlooking the area with geographical information of the area, and registering the first feature point by storing the feature point position information; A step of comparing second feature points, which are feature points of a second image capturing a state in which a disaster has occurred in the area, with each of the registered multiple first feature points, and identifying the first feature points corresponding to the second feature points; a step of estimating shooting information indicating a condition at the time of shooting the second image based on the feature point position information of each of the two or more first feature points identified in the step of identifying the first feature points and shooting position information indicating a position at which the second image was shot; and calculating disaster position information indicating a geographical position of a point where the disaster is occurring based on detection position information indicating a position where the occurrence of the disaster is detected in the second image and the shooting information. A disaster location identification method comprising:
6. The second image is an image taken from a position overlooking the area, The shooting information includes information on the focal length at the time of shooting, and information on the azimuth angle and depression angle of the shooting direction.
6. The disaster location specifying method according to claim 5.
7. In the step of registering the first feature points, the first feature points extracted from each of the plurality of first images captured while changing at least one of an azimuth angle and a depression angle of a shooting direction are registered.
6. The disaster location specifying method according to claim 5.
8. The method further includes a step of detecting the occurrence of the disaster by analyzing an image of the area captured from a position overlooking the area, and extracting the second feature point from the second image, which is an image in which the occurrence of the disaster is detected; In the step of identifying the first feature points corresponding to the second feature points, the second feature points extracted from the second image are compared with each of a plurality of registered first feature points.
8. The disaster location specifying method according to claim 5, further comprising the step of:
9. a step of obtaining feature point position information indicating a geographical position of a first feature point by matching a first image obtained by photographing an area from a position overlooking the area with geographical information of the area, and registering the first feature point by storing the feature point position information; a step of comparing second feature points, which are feature points of a second image capturing a state in which a disaster has occurred in the area, with each of the plurality of registered first feature points, and identifying the first feature points whose geographical positions coincide with those of the second feature points; a step of estimating shooting information indicating a condition at the time of shooting the second image based on the feature point position information of each of the two or more first feature points identified in the step of identifying the first feature points and shooting position information indicating a position at which the second image was shot; and calculating disaster position information indicating a geographical position of a point where the disaster is occurring based on detection position information indicating a position where the occurrence of the disaster is detected in the second image and the shooting information. A disaster location identification program comprising:
Citation Information
Patent Citations
Video-map interlocking system and interlocking method thereof
JP2005123750A
