Earth surface situation assessment method, earth surface situation assessment device, and earth surface situation assessment program

Abstract

To improve the frequency of grasping a condition of an earth surface.SOLUTION: In an earth surface condition grasping method, a plurality of past images obtained by photographing an earth surface for each observation point with photographing means mounted on a plurality of flying bodies flying on different orbits are stored in a storage device while kept linked with data specifying the observation point and a photographing time; a new image is obtained by photographing a predetermined observation point on the earth surface with photographing means mounted on a first flying body flying on a first orbit; one immediate past image obtained by photographing the same point as the predetermined observation point with photographing means mounted on a second flying body flying on a second orbit different from the first orbit is selected from among the plurality of past images, and the selected past image is output; based on point information, the new image and the selected past image are aligned; and taking an influence of foreshortening into account, the new image and the selected past image after the alignment are compared to detect a changed region.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a method for assessing the Earth's surface situation, an apparatus for assessing the Earth's surface situation, and a program for assessing the Earth's surface situation, and in particular to a method for assessing the Earth's surface situation using SAR images acquired by a synthetic aperture radar (hereinafter also abbreviated as "SAR") mounted on a flying object (platform) such as an artificial satellite or an aircraft. [Background technology]

[0002] Artificial satellites are capable of observing the Earth's surface regardless of day or night or weather conditions, and the satellite images obtained from these observations are used to regularly monitor the conditions of the Earth's surface. Research and utilization of remote sensing has become increasingly active, and various methods have been proposed. In particular, radar interferometry technology utilizes the phase information contained in radio waves, and radar mounted on artificial satellites can be used to efficiently measure wide areas. Against this background, many orbiting satellites equipped with radar have been launched, improving the frequency and availability of radar images.

[0003] For example, various information processing systems have been developed that perform image analysis of the Earth's surface using radar images. Among these, a system that analyzes the Earth's surface using radar images acquired by a synthetic aperture radar (SAR) mounted on a flying object (platform) such as an artificial satellite or an aircraft (hereinafter abbreviated as "SAR image analysis") is known. Note that a synthetic aperture radar (SAR) is a radar that can acquire radar images by mounting a radar device on a flying object (platform) such as an aircraft or an artificial satellite and irradiating the Earth's surface with microwaves while the flying object (platform) is moving. Radar images (satellite images) acquired by SAR are also called the above-mentioned SAR images. Furthermore, an artificial satellite equipped with a SAR is also called a SAR satellite.

[0004] The main feature of SAR is that it can observe the Earth's surface regardless of weather or day or night by using microwaves that can penetrate clouds and other objects, and by observing from the air, it can observe a wide area of the Earth's surface at once. For this reason, SAR images are used to grasp the damage situation in disasters and to grasp the detailed behavior of ships at sea.

[0005] To understand the condition of the Earth's surface, changes (changed areas) on the Earth's surface (observation point) are detected by comparing two SAR images taken at a specific observation point under the same conditions (i.e., by a flying object flying on the same orbit) but at different times. Note that in this specification, "observation point" should be understood to mean not only a single point on the Earth's surface but also a specified area encompassing that point. An observation point is also referred to as an "imaging point."

[0006] For example, Patent Document 1 analyzes SAR images taken of the same area at different times. The "image analysis device" disclosed in Patent Document 1 includes a stable reflection point identification unit and a clustering unit. The stable reflection point identification unit identifies stable reflection points. The clustering unit performs clustering using the identified stable reflection points. By comparing examples of roads and buildings with the clustering results, users of the image analysis device can easily associate stable reflection points with roads or buildings. The relationship between stable reflection points and objects such as roads or buildings is visualized. Patent Document 1 also states that "one of the causes of difficulty in understanding the correspondence between stable reflection points and objects is a phenomenon called layover or collapse. Layover is a phenomenon in which multiple stable reflection points located in geographically distant locations appear mixed together in the same or nearby locations in an SAR image."

[0007] Patent Document 2 discloses a "ground change investigation method" that can obtain more accurate investigation results. In the interferometry process, multiple radar images of the ground to be investigated are used. These radar images are acquired at different times by at least one satellite. Each radar wave reflecting element is positioned to coincide with the line of sight of the satellite. The radar wave reflecting element has the characteristic of being rotatable so that it can coincide with the line of sight of satellites passing through different orbits. A least-squares error distribution algorithm including the true coordinates of the reference point is executed for the radar images acquired by the satellite. The algorithm is applied to the change between two radar images. A process for obtaining the corrected coordinate change at the current time is applied between the time when the second radar image was acquired and the current time. The position of the investigation point and the corrected change in its altitude are displayed on a map of a geographic information system.

[0008] Patent Document 3 discloses a "disaster occurrence prediction method and disaster occurrence prediction device" that predicts the occurrence of disasters and enables rapid response to disasters. The disaster occurrence prediction technology disclosed in Patent Document 3 automatically orders satellite image data based on periodically observed or predicted natural phenomena, and automatically analyzes the obtained satellite image data to predict the occurrence of disasters. More specifically, Patent Document 3 creates differential image data by taking the difference between satellite image data during abnormal times and satellite image data during normal times, and extracts anomalous areas where disasters are predicted to occur based on the differential image data. Patent Document 3 also states, "Note that the satellite image data during abnormal times and normal times are subjected to predetermined position correction, brightness correction, etc. so that they can be compared under the same conditions."

[0009] Patent Document 4 discloses an "interferometric synthetic aperture radar system" that matches observed targets between multiple synthetic aperture radars at different altitudes and maintains an appropriate correlation between the observation data of the two satellites. The "interferometric synthetic aperture radar system" disclosed in Patent Document 4 includes a first synthetic aperture radar satellite (first flying vehicle), a second synthetic aperture radar satellite (second flying vehicle), and an analyzer (flying vehicle control device). The first synthetic aperture radar satellite and the second synthetic aperture radar satellite are on different orbits. Based on the orbit of the second synthetic aperture radar satellite and the position of the observed target observed by the first synthetic aperture radar satellite, the system calculates the off-nadir angle of the second synthetic aperture radar satellite and generates a control signal indicating that the directivity of the directional antenna of the second synthetic aperture radar satellite be set to the calculated off-nadir angle. This allows the observed targets of the first synthetic aperture radar satellite and the second synthetic aperture radar satellite to be matched. Patent Document 4 states that "data obtained by two observations at different times can also be used to detect crustal movements." [Prior art documents] [Patent documents]

[0010] [Patent Document 1] International Publication No. 2018 / 123748 [Patent Document 2] Special Publication No. 2012-533744 [Patent Document 3] Japanese Patent Application Laid-Open No. 2003-281664 [Patent Document 4] Japanese Patent Application Laid-Open No. 2010-078396 [Non-patent literature]

[0011] [Non-Patent Document 1] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros: Image-to-Image Translation with Conditional Adversarial Networks. CVPR 2017: 5967-5976 Summary of the Invention [Problem to be solved by the invention]

[0012] However, the above-mentioned Patent Documents 1 to 4 have the following problems.

[0013] In other words, Patent Document 1 simply analyzes SAR images taken at different times of the same area. Patent Document 1 simply clusters each stable reflection point and uses the clusters of the clustered stable reflection points to enable association between stable reflection points and objects on a cluster-by-cluster basis. Furthermore, Patent Document 1 describes a phenomenon called "collapse," but merely states that this phenomenon is one of the causes that make it difficult to understand the association between stable reflection points and objects.

[0014] Patent Document 2 also merely discloses acquiring radar images at different times by at least one satellite. Patent Document 2 also discloses using a radar wave reflecting member as a reference point and making the radar wave reflecting member rotatable so that it can align with the line of sight of satellites passing through different orbits. Patent Document 2 also merely calculates corrected coordinates of the survey point and displays the amount of change on a map by executing an error distribution algorithm using the least squares method based on the raw coordinates of the survey point, the raw coordinates of the reference point, and the true coordinates of the reference point.

[0015] Patent Document 3 also merely discloses the technical idea of extracting anomalous areas where disasters are predicted to occur based on differential image data between satellite image data in an abnormal state and satellite image data in normal states. Although Patent Document 3 states that "image data that has undergone predetermined position correction and brightness correction, etc., is used," it does not state or teach any specific position correction or brightness correction.

[0016] As described above, all of Patent Documents 1 to 3 obtain SAR images (radar images: satellite image data) observed under the same conditions but at different times to grasp the condition of the Earth's surface. This limits the frequency with which the condition of the Earth's surface can be grasped. This is because the frequency with which an artificial satellite can photograph a specific observation point (photography point) from the same orbital position depends on the number of recurrence periods and the number of satellites. On the other hand, changing the recurrence period of an artificial satellite requires changing the orbit itself, which places significant constraints on the operation of the artificial satellite. Furthermore, increasing the number of artificial satellites requires enormous costs.

[0017] On the other hand, Patent Document 4 merely discloses the technical idea of aligning the observation targets of a first synthetic aperture radar satellite and a second synthetic aperture radar satellite, which are on different orbits. Therefore, in Patent Document 4, the first synthetic aperture radar satellite and the second synthetic aperture radar satellite emit radio waves toward the same observation target at approximately the same observation time and receive the reflected radio waves. Therefore, in Patent Document 4 as well, observation data obtained two times with a time difference (at different times) is required to detect crustal deformation. Furthermore, Patent Document 4 does not mention or even recognize the influence of collapse.

[0018] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide a method for assessing the condition of the Earth's surface, an apparatus for assessing the condition of the Earth's surface, and a program for assessing the condition of the Earth's surface, which can improve the frequency with which the condition of the Earth's surface can be assessed. [Means for solving the problem]

[0019] In one aspect of the present invention, a method for assessing the Earth's surface condition is provided, in which photographing means mounted on a plurality of flying objects flying on different orbits are used. Oblique irradiation of microwaves onto the Earth's surface a storage step of storing a plurality of past images obtained by photographing the Earth's surface at each observation point in a storage unit while linking the images to data specifying the observation point and the time of photographing; and By the oblique irradiationan acquisition step of acquiring new images obtained by photographing a predetermined observation point on the surface of the Earth; and selecting a new image from the plurality of past images stored in the storage unit by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit. By the oblique irradiation a selection step of selecting one most recent past image obtained by photographing the same location as the predetermined observation location and outputting the selected past image; and a registration step of aligning the new image with the selected past image based on location information; resulting from imaging the Earth's surface with oblique illumination and a comparison step of comparing the new image after alignment with the selected previous image, taking into account the effect of collapse, to detect a changed area.

[0020] In another aspect of the present invention, the Earth surface situation assessment device measures the Earth surface situation by using image capturing means mounted on a plurality of flying objects flying on different orbits. Oblique irradiation of microwaves onto the Earth's surface a storage unit that stores a plurality of past images obtained by photographing the Earth's surface at each observation point in a storage unit in a state where the images are linked to data that identifies the observation point and the time of photographing; and a storage unit that stores the images by a photographing means mounted on a first flying object that flies on a first orbit. By the oblique irradiation an acquisition means for acquiring new images obtained by photographing a predetermined observation point on the surface of the Earth; and an acquisition means for acquiring new images obtained by photographing a predetermined observation point on the surface of the Earth by photographing a second flying object flying on a second orbit different from the first orbit from the plurality of past images stored in the storage unit. By the oblique irradiation a selection means for selecting a most recent past image obtained by photographing the same location as the predetermined observation location and outputting the selected past image; and a positioning means for positioning the new image and the selected past image based on location information; resulting from imaging the Earth's surface with oblique illumination and a comparison means for detecting a changed area by comparing the new image after alignment with the selected previous image, taking into account the effect of collapse.

[0021] In another aspect of the present invention, the Earth surface situation assessment program is a program for causing a computer to acquire the Earth surface situation by using image capturing means mounted on a plurality of flying objects flying on different orbits. Oblique irradiation of microwaves onto the Earth's surfacea storage means for storing a plurality of past images obtained by photographing the Earth's surface at each observation point in a storage unit, the images being linked to data specifying at least the observation point and the time of photographing; and a photographing means mounted on a first flying object flying on a first orbit. By the oblique irradiation an acquisition means for acquiring new images obtained by photographing a predetermined observation point on the surface of the Earth; and an acquisition means for acquiring new images obtained by photographing a predetermined observation point on the surface of the Earth by photographing a second flying object flying on a second orbit different from the first orbit from the plurality of past images stored in the storage unit. By the oblique irradiation a selection means for selecting a most recent past image obtained by photographing the same location as the predetermined observation location and outputting the selected past image; and a positioning means for positioning the new image and the selected past image based on location information; resulting from imaging the Earth's surface with oblique illumination Taking into consideration the influence of the collapse, the new image after alignment is compared with the selected previous image, and functions as a comparison means for detecting a changed area. [Effects of the Invention]

[0022] According to the present invention, it is possible to increase the frequency with which the conditions of the Earth's surface can be grasped. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 10 is a diagram for explaining the principle of collapse. [Figure 2] This is a diagram for explaining that two SAR images with different tilts are captured when the satellite's orbital position is different at the time of capturing the images. [Figure 3] FIG. 10 is a diagram for explaining the effect of change detection using SAR images taken from different orbital positions. [Figure 4] 1 is a block diagram showing the configuration of an earth surface situation assessment device according to a first embodiment of the present invention. [Figure 5] 5 is a flowchart for explaining a change detection method implemented by a change detection device used in the earth's surface condition assessment device shown in FIG. 4. [Figure 6]FIG. 4 is a block diagram showing the configuration of an apparatus for assessing the state of the earth's surface according to a second embodiment of the present invention. [Figure 7] 7 is a flowchart for explaining a change detection method implemented by a change detection device used in the earth's surface condition assessment device shown in FIG. 6. [Figure 8] 1 is a block diagram showing the configuration of an earth surface situation assessment device according to a first embodiment of the present invention. [Figure 9] 9 is a flowchart for explaining the operation of the earth's surface condition assessment device shown in FIG. 8. [Figure 10] FIG. 10 is a diagram illustrating the configuration and operation of a change detection device according to a second embodiment of the present invention. [Figure 11] FIG. 10 is a diagram illustrating a method for learning an image generation model as another method for generating images with different viewpoints. [Figure 12] FIG. 10 is a diagram illustrating the configuration and operation of a change detection device according to a third exemplary embodiment of the present invention. [Figure 13] FIG. 10 is a diagram for explaining a method for training a change detection model to which trajectory information is input. [Figure 14] 1 is a block diagram showing an example of a hardware configuration of an earth surface situation assessment device according to each embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0024] First, an outline of the present invention will be described to facilitate understanding of the present invention.

[0025] Satellites on different orbits and sub-recurrent orbits have the opportunity to photograph the same observation point from different orbital positions. By using multiple SAR images taken by these satellites, we can improve the frequency with which we can understand the conditions of the Earth's surface.

[0026] However, due to the characteristics of SAR, the way buildings and terrain are inclined in SAR images varies depending on the orbital position of the satellite taking the image. Therefore, change extraction methods that simply compare the pixel values between two SAR images (i.e., change detection methods that do not consider the effects of inclination) are strongly affected by inclination. Therefore, change detection between two SAR images taken from different orbital positions carries the risk of many false positives, making it difficult to put into practical use.

[0027] Below, we will explain in more detail why it becomes difficult to detect changes with high accuracy due to the influence of tilt when using multiple SAR images taken from different orbital positions.

[0028] SAR images are subject to distortions caused by the characteristics of the sensor, one of which is foreshortening.

[0029] Figure 1 shows the principle behind this phenomenon. SAR uses oblique illumination to measure the distance between the sensor and a target on the ground and create an image. Therefore, when photographing tall objects such as buildings or mountains, the slope facing the radar will bend toward the radar and appear shorter. On the other hand, the slope facing away from the radar will appear longer.

[0030] Therefore, if the satellite's orbital position is different when the images are taken, two SAR images will be taken with different inclinations, as shown in Figure 2. Therefore, when detecting changes in the Earth's surface between two SAR images taken from different orbital positions, the simple comparison of pixel values as described above will detect the difference in inclination as a change.

[0031] Therefore, this invention utilizes a method for detecting changes between two SAR images captured from different orbital positions with fewer false positives. Such a change detection method can be realized, for example, by using a technology for generating SAR images from different viewpoints, such as Pix2Pix, as described in Non-Patent Document 1, or by using a change detection model created for each combination of orbits at the time of capture. This reduces the effect of inconsistency between two SAR images captured from different orbital positions, making it possible to perform change detection with fewer false positives.

[0032] In other words, in this invention, by using a system that utilizes a change detection model that inputs orbital information, it is possible to perform robust change detection even between two SAR images taken from different orbital positions, and to grasp the condition of the Earth's surface with high frequency.

[0033] The change detection method described above makes it possible to select a comparison SAR image for a newly acquired SAR image (new SAR image) regardless of the satellite's orbital position. Furthermore, it makes it possible to check for changes between the new SAR image and the most recent SAR image taken of the same observation point, thereby improving the frequency with which the state of the Earth's surface can be grasped.

[0034] By utilizing the present invention, as shown in FIG. 3, it is possible to confirm changes in the Earth's surface in a shorter time, and to correctly grasp the transition of the Earth's surface conditions.

[0035] Furthermore, as mentioned above, if changes in the Earth's surface can only be detected between two SAR images taken by a satellite from the same orbital position, it is necessary to increase the number of satellites passing in the same orbit in order to grasp the condition of the Earth's surface more frequently.

[0036] In contrast, this invention makes it possible to detect changes in the Earth's surface between two SAR images taken by satellites in different orbits. This allows existing SAR satellites that travel in different orbits to be utilized, thereby reducing the cost of developing new satellites.

[0037] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0038] [First embodiment] 4 is a block diagram showing the configuration of an earth surface condition assessment device 100 according to a first embodiment of the present invention. The illustrated earth surface condition assessment device 100 comprises a storage device 110 and a change detection device 120. The change detection device 120 comprises an accumulation unit 121, an acquisition unit 122, a selection unit 123, an alignment unit 124, and a comparison unit 125.

[0039] The accumulation unit 121 accumulates, in the storage device 110, a plurality of past images obtained by photographing the Earth's surface at each observation point using imaging means mounted on a plurality of flying objects flying on different orbits, with the images linked to data specifying the observation point and the time of photographing. Here, the "flying object" may be, for example, one selected from the group consisting of an artificial satellite, an aircraft, a drone, an airship, and a helicopter. More specifically, the flying object may be a SAR satellite, which is an artificial satellite equipped with a synthetic aperture radar (SAR) as the imaging means.

[0040] The acquisition unit 122 acquires new images obtained by photographing a predetermined observation point on the Earth's surface using an imaging means mounted on a first flying object flying on a first orbit. Here, if the flying object is the SAR satellite, each of the plurality of past images and the new image is an SAR image.

[0041] Next, an overview of a SAR satellite will be described. Although not shown, a SAR satellite comprises a satellite body, a SAR antenna, and a communication antenna. While orbiting in a predetermined orbital direction, the SAR satellite transmits (radiates) multiple transmission waves from the SAR antenna obliquely toward an observation point on the Earth's surface under the same conditions at multiple different times (periods). Here, "same conditions" refers to the same latitude, longitude, altitude, and radiation angle of the transmission waves. The "observation point" may be, for example, a rectangular area of 10 km x 10 km. In this example, the transmission waves consist of microwaves.

[0042] The SAR antenna receives backscattered waves, which are generated when each transmitted wave is backscattered by an observation point, as backscattered received wave data. Each backscattered received wave data represents a SAR image of the observation point at that observation time. The SAR satellite transmits this SAR image, linked to data specifying the observation point and the time of image capture, from its communication antenna to a terrestrial receiving station (not shown) installed on the ground. The terrestrial receiving station is connected to an earth surface situation assessment device 100 shown in FIG. 4.

[0043] Therefore, each of the storage unit 121 and the acquisition unit 122 receives SAR images linked to data specifying the observation point and the time of image capture from the terrestrial receiving station. Note that there is a one-to-one correspondence between the observation point and the position information of the SAR satellite, because the "diagonal direction" is a fixed direction for each SAR satellite.

[0044] The selection unit 123 selects, from among the multiple past images stored in the storage device 110, one most recent past image obtained by photographing the same location as the specified observation location by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit, and outputs the selected past image.

[0045] The alignment unit 124 aligns the new image with the selected previous image based on the location information. The comparison unit 125 compares the aligned new image with the selected previous image, taking into account the influence of collapse, to detect changed areas.

[0046] For example, the comparison unit 125 uses a method for generating images from different viewpoints (image generation model) to compare the new image after alignment with the selected previous image to detect the changed area. Alternatively, the comparison unit 125 uses a change detection model created for each combination of trajectories during shooting to compare the new image after alignment with the selected previous image to detect the changed area. The "image generation model" and "change detection model" will be described in detail later with reference to the drawings.

[0047] Next, the change detection method implemented in the change detection device 120 will be described with reference to FIG.

[0048] First, the accumulation unit 121 accumulates the above-mentioned multiple past images in the storage device 110 (step S101).

[0049] Subsequently, the acquisition unit 122 acquires the new image (step S102). The new image is also called the "latest image."

[0050] Then, the selection unit 123 selects the most recent past image from the storage device 110 and outputs the selected past image (step S103). The selected past image is also called a "comparison image."

[0051] Next, the positioning unit 124 aligns the new image (latest image) with the selected previous image (comparison image) based on the location information (step S104).

[0052] Finally, the comparison unit 125 compares the new image after alignment with the selected previous image, taking into account the influence of collapse as described above, to detect a changed area (step S105).

[0053] Next, the effects of the first embodiment will be described.

[0054] The first embodiment has the advantage of being able to grasp the condition of the Earth's surface more frequently, because changes between two images taken from different orbital positions are confirmed.

[0055] [Second embodiment] 6 is a block diagram showing the configuration of an earth surface situation assessment device 100A according to a second embodiment of the present invention. The illustrated earth surface situation assessment device 100A has the same configuration and operates similarly to the earth surface situation assessment device 100 shown in FIG. 4, except that the configuration and operation of the change detection device differ as described below and that the device further includes a display device 130. Therefore, the change detection device is given the reference numeral 120A. Components similar to those shown in FIG. 4 are given the same reference numerals, and for simplicity of explanation, only the differences will be described below.

[0056] The display device 130 is a display device such as an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), etc. The display device 130 has a function of displaying various information in response to instructions from the change detection device 120A.

[0057] Change detection device 120A has the same configuration and operates in the same manner as change detection device 120 shown in FIG. 4, except that change detection device 120A further includes identification unit 126 and display unit 127.

[0058] The specifying unit 126 specifies the coordinates of the changed area. The display unit 127 superimposes the changed area on the new image based on the specified coordinates, and displays the image on the display screen of the display device 130.

[0059] Next, a change detection method performed by change detection device 120A will be described with reference to Fig. 7. Steps S101 to S105 in Fig. 7 are the same as steps S101 to S105 in Fig. 5, so their description will be omitted and the following description will focus on the subsequent operations.

[0060] The identification unit 126 identifies the coordinates of the changed area (step S106).

[0061] Subsequently, the display unit 127 superimposes the changed area on the new image based on the identified coordinates, and displays the image on the display screen of the display device 130 (step S107).

[0062] Next, the effects of the second embodiment will be described.

[0063] In addition to the effects of the first embodiment, the second embodiment also has the effect of enabling accurate understanding of the transition of the Earth's surface conditions, because the changed area is displayed superimposed on the new image. [Example]

[0064] FIG. 8 is a block diagram showing the configuration of an earth surface situation assessment device 200 according to a first embodiment of the present invention. This first embodiment is an example in which the flying object is an artificial satellite (SAR satellite) 201 equipped with a SAR. Note that while FIG. 8 shows only one SAR satellite 201, there may be multiple SAR satellites flying on different orbits. Each SAR satellite 201 transmits SAR images and their metadata to a terrestrial receiving station 202. The metadata includes an image identifier (image ID) for identifying the SAR image, as well as the positional information of the SAR satellite when the SAR image was captured and the time of capture. Instead of the SAR satellite 201 and terrestrial receiving station 202, an external satellite image distribution organization 203 may be provided.

[0065] The illustrated earth surface condition assessment device 200 comprises a data server 210, a change detection device 220, and a change confirmation terminal 230.

[0066] The data server 210 includes a satellite image storage device 212 , a metadata storage device 214 , and a change detection result storage device 216 .

[0067] As will be described later, the satellite image storage device 212 stores new SAR images and multiple past SAR images corresponding to each image ID. The metadata storage device 214 stores metadata, as will be described later. Therefore, each SAR image stored in the satellite image storage device 212 is linked by the image ID to the corresponding metadata stored in the metadata storage device 214. The change detection result storage device 216 stores change detection results, as will be described later.

[0068] The change detection device 220 includes a comparison image selection unit 222, an image registration unit 224, and a change detection unit 226. The illustrated change detection device 220 does not include the accumulation unit 121 and acquisition unit 122 shown in FIGS.

[0069] As will be described later, the comparison image selection unit 222 selects one SAR image for comparison from among multiple past SAR images. As will be described later, the image alignment unit 224 aligns the new SAR image with the comparison SAR image. As will be described later, the change detection unit 226 performs change detection on the aligned new SAR image and comparison SAR image, taking into account the effects of collapsing.

[0070] The change checking terminal 230 displays the new SAR image and the changed area on the display screen, as will be described later.

[0071] The earth surface condition assessment device 200 operates generally as follows.

[0072] The Earth's surface condition assessment device 200 acquires new SAR images of an observation point and registers them in the data server 210. The registration of the new SAR image triggers the change detection device 220 to detect changed areas in the new SAR image, as described below. First, the change detection device 220 automatically selects a comparison SAR image to be used for change detection from multiple past SAR images that have been accumulated during normal times. The comparison image selection unit 222 of the change detection device 220 selects one past SAR image that captures the same observation point, regardless of the orbital position of the SAR satellite that captured it. The change detection unit 226 of the change detection device 220 performs change detection on the new SAR image and the selected comparison SAR image, taking into account the effects of collapsing, and identifies the coordinates of the changed area. Based on the change detection results, the operator's change confirmation terminal 230 displays the new SAR image and the changed area superimposed on each other.

[0073] Next, the operation of the earth's surface condition assessment device 200 will be further described with reference to the flowchart of FIG.

[0074] (S201: Registering a newly captured SAR image) The earth surface condition assessment device 200 registers new SAR images captured by the SAR satellite 201 and their metadata in the satellite image storage device 212 and metadata storage device 214 of the data server 210, respectively. Alternatively, the earth surface condition assessment device 200 registers new SAR images received from the external satellite image distribution organization 203 and their metadata in the satellite image storage device 212 and metadata storage device 214 of the data server 210, respectively (S301 in FIG. 8). As described above, the metadata includes an image ID that identifies the SAR image, and the positional information of the satellite when the SAR image was captured and the time of capture.

[0075] (S202: Select a past image for comparison) The registration of a new SAR image in the data server 210 triggers the comparison image selection unit 222 to select a SAR image for comparison. When selecting a SAR image for comparison, the comparison image selection unit 222 references the metadata of the SAR image registered in the metadata storage device 214 (S302 in FIG. 8) and selects a past SAR image that captures the same observation point. At this time, the position in the orbit of the SAR satellite when the SAR image was captured may be different. After selecting the SAR image for comparison, the comparison image selection unit 222 transmits the image ID of the new SAR image and the image ID of the SAR image for comparison to the image registration unit 224 (S303 in FIG. 8).

[0076] (S203: Alignment between SAR images) As a pre-processing step for using the change detection technology, the image registration unit 224 aligns the positional relationship between the new SAR image and the comparison SAR image on a pixel-by-pixel basis. More specifically, based on the image ID received from the comparison image selection unit 222, the image registration unit 224 references the new SAR image and the comparison SAR image (S304 in FIG. 8) and displays them on the change confirmation terminal 230 (S305 in FIG. 8). The operator inputs information about the same location for the displayed SAR image on the change confirmation terminal 230 (S306 in FIG. 8). Based on the input location information, the image registration unit 224 aligns the positions of both SAR images.

[0077] (S204: Automatic detection of change areas) After alignment, the change detection unit 226 performs change detection on the new SAR image and the comparison SAR image (S307 in FIG. 8) taking into account the effects of tilt. This makes it possible to detect changes with fewer false positives even between two SAR images taken from different orbital positions. The change detection unit 226 associates the coordinates of the changed area detected using this technique with the new SAR image and registers them in the change detection result storage device 216 of the data server 210 (S308 in FIG. 8).

[0078] (S205: Visualization of change areas) Based on the new SAR image and change detection result registered in the data server 210, the change checking terminal 230 of the worker displays the new SAR image and the changed area (S309 in FIG. 8).

[0079] According to the first embodiment, changes can be detected in a shorter time, and the transition of the situation on the Earth's surface can be correctly understood (see FIG. 3).

[0080] As mentioned above, in order to take into account the influence of tilting, the change detection unit 226 can use a method for generating a viewpoint-converted image (image generation model) or a change detection model created for each combination of trajectories during shooting. Below, the change detection device (change detection unit) shown in Figure 8 will be described, with an example using an image generation model as a "second embodiment" and an example using a change detection model as a "third embodiment." [Example]

[0081] 10 is a diagram showing the configuration and operation of a change detection device 220A according to a second embodiment of the present invention. The change detection device 220A shown in the figure includes a comparison image selection unit 222, an image registration unit 224, and a change detection unit 226A. In the following explanation, SAR images will also be referred to simply as images, and SAR satellites will also be referred to simply as satellites.

[0082] In this second embodiment, to reduce the impact of differences in viewpoints when capturing images from a satellite when comparing two SAR images, the comparison image is converted to the same viewpoint as the latest image, and the comparison image after viewpoint conversion is compared with the latest image. To achieve this, this second embodiment uses a simple method, such as orthorectification, which uses geometric transformations that use information about the satellite's position and orientation when the two images were captured. In this case, the SAR image can also be considered as distance information from the camera, and the two viewpoint conversions can be calculated if the camera's position and orientation at the time of capture are known.

[0083] To prepare an image for comparison, the comparison image selection unit 222 selects an image taken at the same photographing point in the past from the satellite image storage device 212 based on the coordinates of the photographing point where the latest image was taken and the photographing time.

[0084] Subsequently, the image positioning unit 224 trims the same photographic area from the latest image and the comparative image, thereby generating images in which the same points overlap when the two images are superimposed.

[0085] The change detection unit 226A first uses the viewpoint information (satellite position, satellite attitude, and shooting time) at which the latest image and the comparison image were captured to perform geometric transformations such as orthorectification to generate an image in which the comparison image has been converted to the viewpoint of the latest image (step S401).

[0086] Next, the change detection unit 226A detects changes based on the difference in pixel values at the same points between the trimmed comparison image converted to the same viewpoint as the latest image and the latest trimmed image (step S402).The change detection unit 226A creates a heat map or the like with the same resolution as the latest trimmed image as the change detection result (S308).

[0087] Another method for generating images with different viewpoints may be to use a method using AI (artificial intelligence) such as Pix2pix, as described in Non-Patent Document 1. This method is shown in FIG.

[0088] 11, in this method, first, SAR images of the same area on the Earth's surface taken from two orbits (hereinafter referred to as "orbit A" and "orbit B") are prepared. Then, in this method, "the image taken from orbit A, the orbit information (satellite position, attitude, and imaging time) of the image taken from orbit A, and the orbit information of the image taken from orbit B" are used as training data (input data) 510, and "the image taken from orbit B" is used as ground truth data 520, and an image generation model 530 is created by training the model.

[0089] Using this image generation model 530, the change detection unit 226A generates an image captured on trajectory A as an image from the viewpoint of trajectory B (S401). When detecting changes, the change detection unit 226A compares the latest image with an image generated from the comparison image using the viewpoint of the trajectory from which the latest image was captured, thereby detecting changes while suppressing the influence of differences in viewpoint (S402). [Example]

[0090] FIG. 12 is a diagram showing the configuration and operation of a change detection device 220B according to a third embodiment of the present invention. The illustrated change detection device 220B has a configuration and operates similarly to the change detection device 220A shown in FIG. 10, except for differences in the configuration and operation of the change detection unit, as described below. Therefore, the change detection unit is given the reference numeral 226B. Components similar to those shown in FIG. 10 are given the same reference numerals, and for simplicity, only the differences will be described below.

[0091] Therefore, change detection device 220B includes a comparison image selection unit 222, an image registration unit 224, and a change detection unit 226B. In the following explanation, an SAR image will also be referred to simply as an image, and an SAR satellite will also be referred to simply as a satellite.

[0092] In this third embodiment, changed areas are detected by learning images of the same area on the ground taken from two orbits and their orbit information as learning data, and manually annotated changed areas as ground truth data.

[0093] The operations of the comparison image selection section 222 and the image position adjustment section 224 are the same as those explained with reference to FIG. 10, so their explanation will be omitted and only the operation of the change detection section 226B will be explained below.

[0094] The change detection unit 226B receives the latest image, orbital information of the latest image (satellite position, satellite attitude, and shooting time), the comparison image, and orbital information of the comparison image as inputs and detects changes using the change detection model 630 (step S403).The change detection unit 226B then creates a heat map or the like with the same resolution as the latest cropped image as the change detection result (S308).

[0095] Next, a method for generating (learning) the change detection model 630 will be described with reference to Fig. 13. In this generation (learning) method, images captured on trajectory A and trajectory information, and images captured on trajectory B and trajectory information are used as learning data (input data) 610, and a change area map in which areas with changes have been manually annotated is used as ground truth data 620 for learning, to create the change detection model 630. At this time, the map in which areas with changes have been annotated has the same resolution as the images of trajectories A and B to be learned. When detecting changes, the latest image and its trajectory information, and a comparison image and its trajectory information are input to this change detection model 630, and a map of changed areas is output.

[0096] [Hardware configuration] The earth surface condition assessment device according to the above-described embodiment of the present invention may be realized by hardware, software, or a combination of hardware and software.

[0097] FIG. 14 is a block diagram showing an example of an information processing device (computer) that constitutes the earth's surface situation assessment device.

[0098] As shown in FIG. 14, the information processing device 700 includes a control unit (CPU: Central Processing Unit) 710, a memory unit 720, a ROM (Read Only Memory) 730, a RAM (Random Access Memory) 740, a communication interface 750, and a user interface 760.

[0099] The control unit (CPU) 710 can realize various functions of the Earth's surface situation assessment device by loading and executing programs stored in the storage unit 720 or ROM 730 into the RAM 740. The control unit (CPU) 710 may also have an internal buffer that can temporarily store data, etc.

[0100] The storage unit 720 is a large-capacity storage medium capable of storing various types of data, and can be realized by a storage medium such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Furthermore, when the information processing device 700 is connected to a communication network via the communication interface 750, the storage unit 720 may be cloud storage present on the communication network. Furthermore, the storage unit 720 may store a program that can be read by the control unit (CPU) 710.

[0101] The ROM 730 is a non-volatile storage device that can be configured with a flash memory or the like having a smaller capacity than the storage unit 720. The ROM 730 may also store a program that can be read by the control unit (CPU) 710. Note that it is sufficient that the program that can be read by the control unit (CPU) 710 is stored in at least one of the storage unit 720 and the ROM 730.

[0102] The program readable by the control unit (CPU) 710 may be non-temporarily stored in various computer-readable storage media and supplied to the information processing device 700. Examples of such storage media include magnetic tape, magnetic disks, magneto-optical disks, CD-ROMs (Compact Disc-Read Only Memory), CD-Rs (Compact Disc-Recordable), CD-R / Ws (Compact Disc-Rewritable), and semiconductor memories.

[0103] The RAM 740 is a semiconductor memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM), and can be used as an internal buffer for temporarily storing data and the like.

[0104] The communication interface 750 is an interface that connects the information processing device 700 to a communication network (not shown) via a wired or wireless connection. The communication interface 750 has a function of sending data received via the communication network or wirelessly to the control unit (CPU) 710. In the illustrated example, the communication interface 750 is connected to the terrestrial receiving station 202 or the external satellite image distribution organization 203 (see FIG. 8) via a wired or wireless connection.

[0105] The user interface 760 includes an input device and an output device. The input device includes a keyboard, a touch panel, a mouse, etc. The input device has the function of detecting an operator's operation and sending the operation information to the control unit (CPU) 710. The output device includes a display device such as an LCD (Liquid Crystal Display) or a PDP (Plasma Display Panel), and a printer. The output device has the function of displaying various information and printing out final results in response to instructions from the control unit (CPU) 710. Note that a display device with a touch panel may be used as a combination of an input device and an output device.

[0106] In other words, each unit of the above-described earth surface situation assessment devices 100, 100A, and 200 can be realized using a combination of hardware and software in a computer system. This computer system may include one or more processors and memories according to a desired configuration. In addition, in this computer system configuration, each unit can be realized by loading an earth surface situation assessment program into the memory and operating hardware such as one or more processors with a set of executable instructions or a set of codes based on the earth surface situation assessment program. In this case, if necessary, the earth surface situation assessment program may implement each unit in cooperation with functions provided by software such as an operating system, microprograms, and drivers.

[0107] The program data expanded in the memory may include, as appropriate, a set of execution instructions, a set of codes, table files, content data, etc. that cause the processor to operate as one or more of the above-mentioned units.

[0108] Furthermore, this computer system does not necessarily have to be constructed as a single device, but may be constructed by combining multiple servers / computers / virtual machines, etc., using so-called thin clients, distributed computing, or cloud computing. Furthermore, some or all of the components of the computer system may be replaced with hardware or firmware (for example, one or more LSIs (Large-Scale Integration), FPGAs (Field Programmable Gate Arrays), or a combination of electronic elements). Similarly, only some of the components may be replaced with hardware or firmware.

[0109] The earth surface condition assessment program may be non-temporarily recorded on a recording medium and distributed. The earth surface condition assessment program recorded on the recording medium is read into a memory via a wired or wireless connection or via the recording medium itself, and causes a processor or the like to operate.

[0110] In this specification, the term "recording medium" includes similar terms such as storage media, memory devices, and storage devices. Examples of such recording media include optical disks, magnetic disks, semiconductor memory devices, hard disk drives, and tape media. It is desirable that the recording medium be non-volatile. The recording medium may also be a combination of a volatile module (e.g., RAM: Random Access Memory) and a non-volatile module (e.g., ROM: Read Only Memory).

[0111] The present invention has been described using exemplary embodiments and examples. However, the specific configuration of the present invention is not limited to the above-described embodiments and examples, and modifications that do not deviate from the gist of the present invention are also included in the present invention. For example, modifications such as separating and merging the block configurations of the above-described embodiments and examples, or replacing procedures, are free as long as they satisfy the spirit of the present invention and the functions described, and the above description does not limit the present invention. Furthermore, in the above examples, only methods that utilize an "image generation model" or a "change detection model" have been described as methods that take into account the effects of collapse, but it goes without saying that other methods may also be adopted.

[0112] Furthermore, part or all of the above-described embodiments can also be described as follows: Note that the following supplementary notes do not limit the present invention in any way.

[0113] [Appendix 1] a storage step of storing, in a storage device, a plurality of past images obtained by photographing the Earth's surface at each observation point using photographing means mounted on a plurality of flying objects flying on mutually different orbits, while linking the images to data identifying the observation points and the times when the images were photographed; an acquisition step of acquiring new images obtained by photographing a predetermined observation point on the Earth's surface using an imaging means mounted on a first flying object flying on a first orbit; a selection step of selecting, from the plurality of past images stored in the storage device, one most recent past image obtained by photographing the same point as the predetermined observation point by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit, and outputting the selected past image; a registration step of registering the new image and the selected past image based on location information; a comparison step of detecting a changed area by comparing the new image after alignment with the selected previous image, taking into account the influence of the collapse; Earth surface situational awareness methods, including:

[0114] [Appendix 2] an identifying step of identifying coordinates of the changed region; a display step of superimposing and displaying the changed area on the new image based on the specified coordinates; 2. The method of claim 1, further comprising:

[0115] [Appendix 3] 3. The method for assessing the situation on the Earth's surface according to claim 1 or 2, wherein the aerial vehicle is one selected from the group consisting of an artificial satellite, an aircraft, a drone, an airship, and a helicopter.

[0116] [Appendix 4] The flying object is a synthetic aperture radar (SAR) satellite equipped with a SAR as the imaging means, each of the plurality of past images and the new image comprises a SAR image; 1. A method for assessing the Earth's surface condition as described in Appendix 3.

[0117] [Appendix 5] the comparing step detects the changed region by comparing the new image after alignment with the selected previous image using a technique for generating images from different viewpoints; 5. A method for assessing the state of the Earth's surface according to any one of appendices 1 to 4.

[0118] [Appendix 6] the comparing step uses a change detection model created for each combination of trajectories during imaging to compare the new image after alignment with the selected previous image to detect the changed area; 5. A method for assessing the state of the Earth's surface according to any one of appendices 1 to 4.

[0119] [Appendix 7] a storage means for storing, in a storage device, a plurality of past images obtained by photographing the Earth's surface at each observation point using photographing means mounted on a plurality of flying objects flying on mutually different orbits, in a state where the images are linked to data identifying the observation point and the time of photographing; an acquisition means for acquiring a new image obtained by photographing a predetermined observation point on the surface of the Earth using an imaging means mounted on a first flying object flying on a first orbit; a selection means for selecting, from the plurality of past images stored in the storage device, one most recent past image obtained by photographing the same point as the predetermined observation point by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit, and outputting the selected past image; a positioning means for positioning the new image and the selected past image based on location information; a comparison means for comparing the new image after alignment with the selected previous image, taking into account the influence of the collapse, to detect a changed area; Earth surface situational awareness equipment, including:

[0120] [Appendix 8] A specifying means for specifying the coordinates of the changed region; a display means for displaying the changed area superimposed on the new image based on the specified coordinates; 8. The Earth surface situation assessment device of claim 7, further comprising:

[0121] [Appendix 9] 9. The Earth surface situation assessment device according to claim 7 or 8, wherein the aerial vehicle is one selected from the group consisting of an artificial satellite, an aircraft, a drone, an airship, and a helicopter.

[0122] [Appendix 10] The flying object is a synthetic aperture radar (SAR) satellite equipped with a SAR as the imaging means, each of the plurality of past images and the new image comprises a SAR image; 10. The Earth surface situation assessment device according to claim 9.

[0123] [Appendix 11] the comparing means detects the changed area by comparing the new image after alignment with the selected previous image using a technique for generating images from different viewpoints; 11. The Earth surface situation assessment device according to any one of appendixes 7 to 10.

[0124] [Appendix 12] the comparison means uses a change detection model created for each combination of trajectories during photography to compare the new image after alignment with the selected previous image, and detects the changed area. 11. The Earth surface situation assessment device according to any one of appendixes 7 to 10.

[0125] [Appendix 13] Computer, a storage means for storing, in a storage device, a plurality of past images obtained by photographing the Earth's surface at each observation point using photographing means mounted on a plurality of flying objects flying on mutually different orbits, in a state where the images are linked to data identifying at least the observation point and the time of photographing; an acquisition means for acquiring a new image obtained by photographing a predetermined observation point on the surface of the Earth using an imaging means mounted on a first flying object flying on a first orbit; a selection means for selecting, from the plurality of past images stored in the storage device, one most recent past image obtained by photographing the same point as the predetermined observation point by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit, and outputting the selected past image; a positioning means for positioning the new image and the selected past image based on location information; a comparison means for comparing the new image after alignment with the selected previous image, taking into account the influence of the collapse, to detect a changed area; Earth surface situation awareness program to function as a

[0126] [Appendix 14] The computer A specifying means for specifying the coordinates of the changed region; a display means for displaying the changed area superimposed on the new image based on the specified coordinates; 14. The Earth surface situation awareness program of claim 13, further functioning as

[0127] [Appendix 15] The Earth surface situation assessment program according to claim 13 or 14, wherein the aerial vehicle is one selected from the group consisting of an artificial satellite, an aircraft, a drone, an airship, and a helicopter.

[0128] [Appendix 16] The flying object is a synthetic aperture radar (SAR) satellite equipped with a SAR as the imaging means, each of the plurality of past images and the new image comprises a SAR image; 14. The Earth surface situation awareness program described in Appendix 15.

[0129] [Appendix 17] the comparing means detects the changed area by comparing the new image after alignment with the selected previous image using a technique for generating images from different viewpoints; 17. A program for assessing the state of the Earth's surface according to any one of appendices 13 to 16.

[0130] [Appendix 18] the comparison means uses a change detection model created for each combination of trajectories during photography to compare the new image after alignment with the selected previous image, and detects the changed area. 17. A program for assessing the state of the Earth's surface according to any one of appendices 13 to 16. [Industrial Applicability]

[0131] The present invention can be widely used in the field of earth observation, such as disaster monitoring, ocean monitoring, oil field monitoring, agriculture, fisheries, and monitoring of activities in urban areas, where synthetic aperture radar is generally widely used. [Explanation of symbols]

[0132] 100, 100A Earth surface situation assessment device 110 Storage device 120, 120A Change detection device 121 Storage Unit 122 Acquisition Department 123 Selection Department 124 Alignment section 125 Comparison Section 126 Specific part 127 Display section 130 Display device 200 Earth surface situation assessment device 201 SAR satellite 202 Ground Receiving Station 203 External satellite image distribution agency 210 Data Server 212 Satellite Image Storage Device 214 Metadata Storage Device 216 Change detection result storage device 220, 220A, 220B Change detection device 222 Comparison Image Selection Section 224 Image Alignment Unit 226, 226A, 226B Change detection section 230 Change confirmation terminal 510 training data (input data) 520 correct data 530 Image Generation Model 610 Learning data (input data) 620 Correct data 630 Change Detection Model 700 Information Processing Devices 710 Control Unit (CPU) 720 Storage section 730 ROM 740 RAM 750 Communication Interface 760 User Interface

Claims

1. a storage step of storing, in a storage device, a plurality of past images obtained by photographing the Earth's surface at each observation point by obliquely irradiating the Earth's surface with microwaves using photographing means mounted on a plurality of flying objects flying on different orbits, in a state where the past images are linked to data identifying the observation point and the time of photographing; an acquisition step of acquiring a new image obtained by photographing a predetermined observation point on the surface of the Earth using the oblique illumination by an imaging means mounted on a first flying object flying on a first orbit; a selection step of selecting, from the plurality of past images stored in the storage device, one most recent past image obtained by photographing the same point as the predetermined observation point with the oblique illumination by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit, and outputting the selected past image; a registration step of registering the new image and the selected past image based on location information; a comparison step of detecting a change area by comparing the new image after alignment with the selected previous image, taking into account the influence of tilt caused by imaging the Earth's surface with the oblique illumination; Earth surface situational awareness methods, including:

2. an identifying step of identifying coordinates of the changed region; a display step of superimposing and displaying the changed area on the new image based on the specified coordinates; The method of claim 1 further comprising:

3. 3. The method for understanding the Earth's surface situation according to claim 1, wherein the air vehicle is one selected from the group consisting of an artificial satellite, an aircraft, a drone, an airship, and a helicopter.

4. the flying object is a synthetic aperture radar (SAR) satellite equipped with a SAR as the imaging means, each of the plurality of past images and the new image comprises a SAR image; 4. The method for assessing the Earth's surface condition according to claim 3.

5. the comparing step uses a method of generating an image obtained by converting the selected previous image to the same viewpoint as the new image, and compares the new image after alignment with the previous image after the viewpoint conversion to detect the changed area; 5. The method for assessing the state of the Earth's surface according to claim 1.

6. The comparison step uses images of the same area on the Earth's surface taken from the different orbits at the time of photography and their orbit information as learning data, and uses changed areas annotated with changed areas as correct answer data to create a change detection model, and compares the new image after alignment with the selected past image to detect the changed area.

5. The method for assessing the state of the Earth's surface according to claim 1.

7. a storage means for storing, in a storage device, a plurality of past images obtained by photographing the Earth's surface at each observation point by obliquely irradiating the Earth's surface with microwaves using photographing means mounted on a plurality of flying objects flying on different orbits, in a state where the images are linked to data specifying the observation point and the time of photographing; an acquisition means for acquiring a new image obtained by photographing a predetermined observation point on the surface of the Earth with the oblique illumination by an imaging means mounted on a first flying object flying on a first orbit; a selection means for selecting, from the plurality of past images stored in the storage device, one most recent past image obtained by photographing the same point as the predetermined observation point with the oblique illumination by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit, and outputting the selected past image; a positioning means for positioning the new image and the selected past image based on location information; a comparison means for detecting a change area by comparing the new image after alignment with the selected previous image, taking into consideration the influence of tilt caused by imaging the Earth's surface with the oblique illumination; Earth surface situational awareness equipment, including:

8. A specifying means for specifying the coordinates of the changed region; a display means for displaying the changed area superimposed on the new image based on the specified coordinates; 8. The Earth's surface situation awareness device of claim 7, further comprising:

9. 9. The Earth surface situation awareness system according to claim 7 or 8, wherein the air vehicle is one selected from the group consisting of an artificial satellite, an airplane, a drone, an airship, and a helicopter.

10. Computer, a storage means for storing, in a storage device, a plurality of past images obtained by photographing the Earth's surface at each observation point by obliquely irradiating the Earth's surface with microwaves using photographing means mounted on a plurality of flying objects flying on different orbits, in a state where the images are linked to data identifying at least the observation point and the time of photographing; an acquisition means for acquiring a new image obtained by photographing a predetermined observation point on the surface of the Earth with the oblique illumination by an imaging means mounted on a first flying object flying on a first orbit; a selection means for selecting, from the plurality of past images stored in the storage device, one most recent past image obtained by photographing the same point as the predetermined observation point with the oblique illumination by an imaging means mounted on a second flying object flying on a second orbit different from the first orbit, and outputting the selected past image; a positioning means for positioning the new image and the selected past image based on location information; a comparison means for detecting a change area by comparing the new image after alignment with the selected previous image, taking into consideration the influence of tilt caused by imaging the Earth's surface with the oblique illumination; Earth surface situation awareness program to function as a

Images