Change detection device and change detection method
The change detection device uses SAR tomography to reconstruct and remove layover-induced phase signals in SAR images, allowing accurate change detection without requiring post-change three-dimensional reconstruction, thus enhancing efficiency and reducing false positives.
Patent Information
- Application Number
- JP2023504909
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-09
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2041-03-09
AI Technical Summary
SAR images are affected by layover, which can cause false changes to be detected due to the mixing of signals from tall and low buildings, reducing the sensitivity of change detection using coherence, and existing methods to reduce layover require multiple image acquisitions to reconstruct three-dimensional structures, increasing effort and time.
A change detection device and method that reconstructs the three-dimensional structure of an observation area using SAR tomography, removes phase signals caused by layover, and performs coherent change detection based on coherence values without the need to reproduce the three-dimensional structure after changes occur.
Enables accurate change detection in areas with layover without the need for extensive image acquisition, reducing processing time and effort by eliminating the influence of layover through phase signal removal, thus maintaining sensitivity and accuracy.
Smart Images

Figure 0007582444000001 
Figure 0007582444000002 
Figure 0007582444000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a motion detection device and a motion detection method for performing motion detection based on SAR images. [Background technology]
[0002] There is a change detection technology that detects areas where the state of the earth's surface has changed based on images taken from a high altitude, such as images taken by a satellite.
[0003] Synthetic Aperture Radar (SAR) technology is a technology that allows a radar mounted on a flying object such as an artificial satellite or an aircraft to transmit and receive electromagnetic waves while the flying object is moving, and obtain images (hereinafter referred to as SAR images) equivalent to images taken by an antenna with a large aperture. Synthetic aperture radar is used, for example, to process the signals of waves reflected from the earth's surface and analyze surface displacement.
[0004] An image taken by a flying object such as an artificial satellite is called an observation image. In this specification, an observation image corresponds to an SAR image. Hereinafter, the flying object that transmits and receives electromagnetic waves is assumed to be an artificial satellite, but the flying object is not limited to an artificial satellite.
[0005] For example, Patent Document 1 describes a change detection technique that utilizes coherence (Coherent Change Detection). Coherence is calculated by the complex correlation of pixels at the same position in K (K≧2) SAR images. Let a pair of SAR images be (p, q), and let the components of the coherence matrix be c p,q p and q are each a value less than or equal to K, and each represent one of the K SAR images. For a pair of SAR images, the phase θ p,q (Specifically, the phase difference) is calculated. Then, exp(-jθ p,q ) is the absolute value of the averaged value of the coherence matrix element c p,qIn addition, the intensity in SAR image p is A p , the intensity in SAR image q is A q As, A p· A q· exp(-jθ p,q ) may be averaged.
[0006] The values represented by the components of the coherence matrix are taken as coherence values. Also, a coherence image is assumed in which the coherence values are pixel values.
[0007] A change detection device that uses a change detection technique that utilizes coherence performs change detection based on a coherence image. When the change detection device detects an area with a low coherence value in the coherence image, it determines that a change has occurred. For example, the change detection device generates a coherence image from an already acquired SAR image and a new SAR image that covers the same observation area as when the SAR image was acquired. As an example, a change in an area may occur due to the appearance of a new structure or the disappearance of an existing structure.
[0008] Hereinafter, already acquired SAR images may be referred to as stored SAR images. Newly acquired SAR images may be referred to as comparison SAR images. Multiple SAR images may be referred to as a SAR image group. For example, a group of already acquired SAR images may consist of multiple SAR images that have already been acquired. [Prior art documents] [Patent documents]
[0009] [Patent Document 1] European Patent Application Publication No. 3540462 [Patent Document 2] JP 2008-185375 A Summary of the Invention [Problem to be solved by the invention]
[0010] SAR images can be affected by layover. For example, in urban areas where there are buildings in the observation area, reflected waves from tall buildings, short buildings, and the ground are mixed in the radar mounted on the artificial satellite. As a result, the waveform of the received wave of the radar mounted on the artificial satellite becomes unstable. Specifically, a layover can occur in SAR images, where information on tall buildings overlaps with information on short buildings. In a situation where a layover occurs, there is a possibility that changes will occur between multiple SAR images even if there is no change in the observation area.
[0011] When a change occurs between multiple SAR images, the similarity between the SAR images decreases, and the coherence value in the coherence image decreases. In such a case, the sensitivity of change detection using coherence decreases. For example, even if no change actually occurs in the observation area, it may be mistakenly determined that a change has occurred.
[0012] There is a technology that utilizes the three-dimensional structure of the observation area estimated from the SAR images in order to reduce the effect of layover on the SAR images (see, for example, Patent Document 2). The device described in Patent Document 2 inputs multiple SAR images. The device converts the difference in distortion between the multiple SAR images into height. Furthermore, the device generates three-dimensional shape data (three-dimensional information) of the observation area using the height information.
[0013] We assume that a technology that utilizes 3D structures estimated from multiple SAR images is applied to a change detection device that uses change detection technology. A change detection device to which this technology is applied is referred to as an application device.
[0014] The application device generates 3D information for reducing the effects of layover from multiple SAR images before the change (before the change occurs in the observation area).The application device also generates 3D information for reducing the effects of layover from multiple newly acquired SAR images.The application device then performs change detection by referring to both sets of 3D information.
[0015] In order to perform the above-mentioned change detection, multiple SAR images before the change are necessary, and multiple SAR images newly acquired after the change are also necessary. That is, in order to reduce the effect of layover, a large number of SAR images are necessary. In addition, the application device performs processing for generating three-dimensional information using multiple SAR images before the change, and also performs processing for generating three-dimensional information using multiple SAR images newly acquired after the change. Therefore, many images need to be acquired in order to reconstruct the three-dimensional structure before and after the change. Therefore, image acquisition takes a lot of time and effort.
[0016] The present invention aims to provide a change detection device and a change detection method that can detect changes in a state where the effects of layover are reduced, without reproducing the three-dimensional structure after the change. [Means for solving the problem]
[0017] The change detection device according to the present invention includes a three-dimensional structure reconstruction means for reconstructing a three-dimensional structure of a predetermined area in an observation area, and a phase detector for detecting the phase of the predetermined area in a plurality of SAR images of the observation area by using the three-dimensional structure. information phase removing means for removing of a given area phase information and a change detection means for generating a coherence image from the SAR image pair from which the coherence has been removed, and detecting a change in the observation region based on the coherence values of pixels constituting the coherence image.
[0018] The change detection method according to the present invention reconstructs a three-dimensional structure of a specific area in an observation area, and uses the three-dimensional structure to estimate the phase of the specific area in a plurality of SAR images in which the observation area is captured. affection Remove the information, of a given area phase information A coherence image is generated from the SAR image pair from which the coherence noise has been removed, and changes in the observation area are detected based on the coherence values of the pixels that make up the coherence image.
[0019] The change detection program according to the present invention includes a process for causing a computer to reconstruct a three-dimensional structure of a specific area in an observation area, and a process for calculating the phase of the specific area in a plurality of SAR images in which the observation area is captured, using the three-dimensional structure. information and removing the of a given area phase information A coherence image is generated from the SAR image pair from which the coherence has been removed, and a process of detecting a change in the observation region based on the coherence values of the pixels constituting the coherence image is executed. Effect of the Invention
[0020] According to the present invention, it is possible to detect changes in a state where the influence of layover is reduced, without reproducing the three-dimensional structure after the change. [Brief description of the drawings]
[0021] [Figure 1] 1 is a block diagram showing an example of the configuration of a change detection device according to a first embodiment; [Diagram 2] 4 is a flowchart showing the operation of the change detection device of the first embodiment. [Diagram 3] FIG. 1 is an explanatory diagram for explaining SAR tomography. [Figure 4] 4 is a flowchart showing a three-dimensional structure reconstruction process in the first embodiment. [Diagram 5] 13 is a flowchart showing a phase signal removal process. [Figure 6] 13 is a flowchart illustrating a coherent change detection process. [Figure 7] FIG. 11 is a block diagram showing an example of the configuration of a change detection device according to a second embodiment. [Figure 8] 10 is a flowchart showing the operation of the change detection device according to the second embodiment. [Figure 9] 13 is a flowchart showing a three-dimensional structure reconstruction process in the second embodiment. [Figure 10] FIG. 13 is a block diagram showing an example of the configuration of a change detection device according to a third embodiment. [Figure 11]13 is a flowchart showing the operation of the change detection device of the third embodiment. [Figure 12] FIG. 13 is a block diagram showing a modification of the change detection device according to the third embodiment. [Figure 13] 13 is a flowchart showing the operation of a modification of the change detection device of the third embodiment. [Figure 14] FIG. 13 is a block diagram showing an example of the configuration of a change detection device according to a fourth embodiment. [Figure 15] FIG. 1 is a block diagram showing an example of a computer having a CPU. [Figure 16] FIG. 2 is a block diagram showing a main part of the change detection device. [Figure 17] FIG. 13 is a block diagram showing a main part of a change detection device according to another embodiment. [Figure 18] FIG. 13 is a block diagram showing a main part of a change detection device according to still another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0022] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0023] Embodiment 1. Fig. 1 is a block diagram showing an example of the configuration of a change detection device according to the first embodiment. The change detection device 100 shown in Fig. 1 includes a three-dimensional structure reconstruction unit 110, a phase removal unit 120, and a coherent change detection unit .
[0024] The three-dimensional structure reconstruction unit 110 reconstructs the three-dimensional structure of the observation area from a plurality of SAR images stored in the SAR image storage unit 700.
[0025] The phase removal unit 120 includes a phase signal estimation unit 121 and a phase signal removal unit 122. The phase signal estimation unit 121 estimates a phase signal of a predetermined region in the observation region from the SAR image pair. The predetermined region is a region where a layover may occur (layover region). The phase removal unit 120 removes the phase signal of the layover region.
[0026] As described above, for example, a layover may occur when a tall building is present in the observation area. The received signal from the layover area is a mixture of signals originating from various reflectors. Therefore, even if no change occurs on the ground, the similarity of the received signals between SAR images decreases, and the coherence decreases. In this embodiment, the phase removal unit 120 removes phase signals caused by height (height of the building) that causes a layover, by referring to the three-dimensional structure of the observation area. For example, the phase removal unit 120 uses three-dimensional point cloud data having information on reflection intensity and phase reproduced by SAR tomography to simulate the mixing of signals in the layover area. That is, the phase removal unit 120 reproduces a phase that is not related to changes on the ground (i.e., a phase caused by the height of the building). Then, the phase removal unit 120 removes signals of such phases from the observation signal.
[0027] Additionally, the phase signal due to height is expected to have a large variation between SAR images, so removing the phase signal due to height reduces the variation between SAR images when no actual change occurs, resulting in more accurate coherent change detection even in layover regions.
[0028] The coherent change detection unit 130 generates a coherence image from one SAR image among the SAR images stored in the SAR image storage unit 700 and a comparison target SAR image. The coherent change detection unit 130 uses the coherence image to determine whether or not a change has occurred in the observation area. The comparison target SAR image may be a newly acquired SAR image, or may be a SAR image selected from the multiple SAR images stored in the SAR image storage unit 700.
[0029] Next, the overall operation of the change detection device 100 of the first embodiment will be described with reference to the flowchart of FIG.
[0030] The three-dimensional structure reconstruction unit 110 reads out a group of existing SAR images from the SAR image storage unit 700. Then, the three-dimensional structure reconstruction unit 110 reconstructs the three-dimensional structure of a predetermined area in the observation area by using a SAR tomography technique (step S101).
[0031] In the phase removal unit 120, the phase signal estimation unit 121 reads out one SAR image from the SAR image storage unit 700. The phase signal estimation unit 121 also receives an SAR image to be compared. The phase signal estimation unit 121 estimates a phase signal in a layover region of a SAR image pair between the SAR image read out from the SAR image storage unit 700 and the SAR image to be compared, using a three-dimensional structure (step S102).
[0032] The phase signal removing unit 122 removes the phase signal estimated by the phase signal estimating unit 121 of the SAR image pair (step S103).
[0033] The coherent change detection unit 130 calculates a coherence value between the SAR images of the SAR image pair. The coherent change detection unit 130 generates a coherence image in which the pixel values are the coherence values. The coherent change detection unit 130 executes a coherent change detection process (step S104). That is, the coherent change detection unit 130 determines whether or not there is a part in which a change has occurred in the observation region.
[0034] As described above, the three-dimensional structure reconstruction unit 110 reconstructs the three-dimensional structure of a predetermined area by a three-dimensional structure reconstruction process using a SAR tomography technique. FIG. 3 is an explanatory diagram for explaining SAR tomography. In the example shown in FIG. 3, the artificial satellite advances in a direction perpendicular to the paper surface. That is, the azimuth direction is a direction perpendicular to the paper surface. Then, in an orbit of n (n=1 to N), the artificial satellite acquires SAR images of a predetermined observation area. In FIG. 3, r indicates the range direction, and s indicates the elevation direction. Also, N corresponds to the number of SAR images stored in the SAR image storage unit 700.
[0035] The observation (received signal) at a certain azimuth-range position is expressed as g=(g1,…,g N ) T Let T denote transposition. The complex reflectance distribution (three-dimensional distribution of complex reflectance) is defined as γ = [γ(s1), …, γ(s L )] T Let s l (l=1 to L) indicates a position (point) in the elevation direction. The elements of the steering matrix are R nl =exp(-4jπk n s l ) k n is the phase-to-altitude conversion coefficient.
[0036] The observed value g at the azimuth-range position, the complex reflection distribution γ, and the steering matrix R are related by the following equation (1).
[0037] g = Rγ (1)
[0038] The three-dimensional structure reconstruction unit 110 can estimate (calculate) the complex reflection distribution γ at each position in the elevation direction from the observation value g and the steering matrix R. The complex reflection distribution γ includes intensity information and phase information. Therefore, the three-dimensional structure reconstruction unit 110 can obtain three-dimensional point cloud data representing the three-dimensional structure by estimating the complex reflection distribution γ for each pixel. The three-dimensional point cloud data is data for each point in three-dimensional coordinates.
[0039] An example of the reflection intensity distribution is shown in the lower part of Fig. 3. In the example shown in Fig. 3, the intensity of the reflected wave from a low-rise house is higher than the intensity of the reflected wave from the earth's surface (ground). The intensity of the reflected wave from a high-rise building is higher than the intensity of the reflected wave from the ground.
[0040] FIG. 4 is a flowchart showing the three-dimensional structure reconstruction process executed by the three-dimensional structure reconstruction unit 110. As shown in FIG.
[0041] The three-dimensional structure reconstruction unit 110 reads out an existing SAR image group (multiple SAR images: in this case, about 10 or more for SAR tomography) from the SAR image storage unit 700. Then, the three-dimensional structure reconstruction unit 110 estimates (calculates) a steering matrix R1 from the observation value g1 of the SAR image (step S111). The three-dimensional structure reconstruction unit 110 also calculates a complex reflection distribution γ using equation (1) (in this case, g1=R1γ) (step S112).
[0042] FIG. 5 is a flowchart showing the phase signal removal process executed by the phase removal unit 120. As shown in FIG.
[0043] The phase signal estimation unit 121 in the phase removal unit 120 reads out an existing SAR image from the SAR image storage unit 700. The phase signal estimation unit 121 also inputs a SAR image to be compared. The phase signal estimation unit 121 causes interference between the existing SAR image and the SAR image to be compared for each pixel of the SAR image, and estimates (calculates) a phase-altitude conversion coefficient for converting the phase included in each pixel after interference into height (step S121).
[0044] The phase signal estimation unit 121 estimates the steering matrix R all is estimated (calculated) (step S122). all is a steering matrix for each point based on all existing SAR images and SAR image data to be compared. Furthermore, the phase signal estimator 121 estimates the complex reflection distribution γ and the steering matrix R all From this, for each pixel of the SAR image, the received signal model g model The received signal model g is estimated (calculated) (step S123). model is the model inferred from the three-dimensional structure, i.e., the predicted observations of the layover region.
[0045] The phase signal removal unit 122 in the phase removal unit 120 removes the observed signal (observed value, i.e., pixel value of the SAR image) g obs (Step S124). obsare the observation signals of the existing SAR image and the comparison target SAR image, respectively.
[0046] The phase signal removal unit 122 calculates a received signal model g for each pixel of the SAR image as expressed by equation (2). model The phase signal predicted from the observed value g obs (step S125).
[0047] g obs exp(-j∠g model ) ···(2)
[0048] The phase remover 120 passes the pair of SAR images from which the phase of the predetermined region has been removed to the coherent change detector 130 .
[0049] FIG. 6 is a flowchart showing the coherent change detection process executed by the coherent change detector 130. As shown in FIG.
[0050] The coherent change detection unit 130 calculates a coherence value between the SAR images of the SAR image pair from the phase removal unit 120 (step S131). In other words, the coherent change detection unit 130 generates a coherence image in which the pixel value is the coherence value. The coherent change detection unit 130 determines whether or not there is a portion in the observation region where the coherence value is smaller than, for example, a predetermined threshold value (step S132). If such a portion exists, the coherent change detection unit 130 determines that a change has occurred in the observation region (step S133). If such a portion does not exist, the coherent change detection unit 130 determines that no change has occurred in the observation region (step S134).
[0051] The coherent change detection unit 130 determines that a change has occurred in the observation region when, for example, the values (coherence values) of all pixels in a region (portion) of a predetermined size in the coherence image are smaller than a predetermined threshold value. The coherent change detection unit 130 may determine that a change has occurred in the observation region when a predetermined ratio or more of pixels in the region of the predetermined size have coherence values smaller than the predetermined threshold value.
[0052] As described above, in this embodiment, the three-dimensional structure reconstruction unit 110 reconstructs the three-dimensional structure of the layover region using the SAR tomography technique. The phase removal unit 120 reconstructs the received signal model g from the three-dimensional structure. model Then, the phase removal unit 120 estimates the received signal model g model The phase that is removed is, for example, the phase due to building height. Since building height causes layover, the phase-removed observed signal is a signal that has had the effects of layover removed.
[0053] Therefore, the coherent change detector 130 can perform coherent change detection processing based on the observation signal from which the phase caused by the height of the building has been removed. In other words, the coherent change detector 130 can perform coherent change detection processing from which the effects of layover have been removed. Furthermore, the coherent change detector 130 can perform accurate coherent change detection processing even in layover areas.
[0054] Furthermore, the motion detection device 100 eliminates the need to acquire a sufficient number of SAR images to reconstruct the three-dimensional structure after the change, compared to when the motion detection device 100 performs a process of generating three-dimensional information using multiple SAR images before the change and also performs a process of generating three-dimensional information using multiple SAR images newly acquired after the change. This is because the motion detection device 100 of this embodiment does not require a process of generating three-dimensional information using multiple SAR images to be compared.
[0055] It is also possible to use a DEM (Digital Elevation Model) to reconstruct a three-dimensional structure and remove the phase signal in the layover area by referring to the three-dimensional structure. However, when SAR tomography is used, more precise phase signal removal can be performed than when DEM is used. This is because when a three-dimensional structure is reconstructed using DEM, only phase information related to large structures is reflected in the three-dimensional structure, whereas when a three-dimensional structure is reconstructed using SAR tomography, phase signals can be estimated for each structure, such as buildings. Also, when a three-dimensional structure is reconstructed using SAR tomography, accurate phase signal estimation can be performed even if structures are distributed in a complex manner in the observation area.
[0056] In addition, when a three-dimensional structure is reconstructed using SAR tomography, a received signal model closer to the actual three-dimensional structure is created compared to when the three-dimensional structure is reconstructed using DEM. This is because when SAR tomography is used, three-dimensional point cloud data having information on reflection intensity and phase is generated. In addition, when a three-dimensional structure is reconstructed using SAR tomography, there is an advantage that a received signal model closer to the actual three-dimensional structure is created compared to when the three-dimensional structure is reconstructed using data acquired by LiDAR (Light Detection and Ranging) as described later.
[0057] Embodiment 2. Fig. 7 is a block diagram showing an example of the configuration of a change detection device according to the second embodiment. The change detection device 200 shown in Fig. 7 includes a three-dimensional structure reconstruction unit 110, a phase removal unit 220, and a coherent change detection unit 230. The configuration and operation of the three-dimensional structure reconstruction unit 110 are the same as those in the first embodiment.
[0058] In this embodiment, for example, SAR images of one observation region obtained from each orbit (path) of a plurality of mutually offset orbits (multipaths) of an artificial satellite are stored in the SAR image storage unit 700. The phase removal unit 220 inputs the plurality of SAR images obtained from the plurality of orbits.
[0059] When an aircraft is used as the flying object, the degree of freedom in selecting a path is greater than when an artificial satellite is used.
[0060] The phase removal unit 220 includes a phase signal estimation unit 221 and a phase signal removal unit 222. In this embodiment, the phase signal estimation unit 221 inputs a SAR image group including a plurality of SAR images stored in the SAR image storage unit 700 and one or more SAR images to be compared. The one or more SAR images to be compared may all be newly acquired SAR images, or may include both a SAR image selected from the plurality of SAR images stored in the SAR image storage unit 700 and a newly acquired SAR image. The phase signal estimation unit 221 estimates phase signals of a predetermined region (layover region) of all SAR images constituting the SAR image group. The phase removal unit 220 removes phase signals of the layover regions of all SAR images.
[0061] The coherent change detection unit 230 generates a coherence image for all SAR image pairs. The coherent change detection unit 230 uses the generated multiple coherence images to determine whether or not a change has occurred in the observation area. All SAR image pairs are combinations of all SAR images that make up the SAR image group. As an example, if the SAR image group includes four SAR images, there are six (=4C2) SAR image pairs.
[0062] Next, the overall operation of the motion detection device of the second embodiment will be described with reference to the flowchart of FIG.
[0063] The three-dimensional structure reconstruction unit 110 operates in the same manner as in the first embodiment to reconstruct the three-dimensional structure of a predetermined area in the observation area (step S101).
[0064] In the phase removing unit 220, the phase signal estimating unit 221 estimates phase signals in a predetermined region (layover region) of all SAR images by referring to the three-dimensional structure (step S202).
[0065] The phase signal removing unit 222 removes the phase signals estimated by the phase signal estimating unit 221 for all SAR images (step S203). Note that the specific method of estimating and removing the phase signals is the same as the method in the first embodiment (see FIG. 5, etc.).
[0066] The phase removal unit 220 passes all SAR images from which the phases of the predetermined region have been removed to the coherent change detection unit 230. The coherent change detection unit 230 executes a coherent change detection process (step S204). That is, the coherent change detection unit 130 determines whether or not a change has occurred in the observation region.
[0067] FIG. 9 is a flowchart showing the coherent change detection process executed by the coherent change detector 230. As shown in FIG.
[0068] The coherent change detection unit 230 calculates the coherence values of all combinations (pairs) of SAR images in the SAR image group (including multiple SAR images stored in the SAR image storage unit 700 and one or more SAR images to be compared) (step S230).
[0069] In addition, the coherent change detection unit 230 calculates a typical coherence value for determining whether or not a change has occurred in the observation region (step S231). The typical coherence value corresponds to a coherence value between SAR images of a SAR image pair in which there is no change or the change is smaller than a predetermined tolerance.
[0070] In order to obtain a typical coherence value, the coherent change detection unit 230, for example, arranges the coherence values calculated for all SAR image pairs two-dimensionally. That is, the coherent change detection unit 230 generates a coherence matrix having the calculated coherence values as elements. When image numbers are assigned to the SAR images, the horizontal axis of the two-dimensional arrangement corresponds to the image number of one SAR image in all SAR image pairs. The vertical axis of the two-dimensional arrangement corresponds to the image number of the other SAR image in all SAR image pairs.
[0071] The coherent change detector 230 determines, for example, the largest coherence value in the two-dimensional array as the typical coherence value.
[0072] The coherent change detection unit 230 compares the coherence values between the SAR images of all SAR image pairs with a typical coherence value. If there are multiple SAR image pairs that exhibit coherence values that are significantly different from the typical coherence value, the coherent change detection unit 230 determines that a change has occurred in the observation region (step S233). If there are no such SAR image pairs, the coherent change detection unit 230 determines that no change has occurred in the observation region (step S234).
[0073] In the case of a coherence value that is significantly different from a typical coherence value, the "difference" is a preset value. In the case of a plurality of SAR images that exhibit significantly different coherence values, the "multiple" are preset values.
[0074] In this embodiment, a plurality of SAR images obtained through multiple paths are used in coherent change detection, thereby obtaining the effect of improving the robustness of coherent change detection in addition to the effect of the first embodiment.
[0075] Generally, the acquisition time of each SAR image acquired by a multipath is different, so if a SAR image group contains multiple SAR images to be compared, it is possible to recognize the time when a change occurred in the observation area from the acquisition time of the SAR image to be compared in a SAR image pair that exhibits a coherence value that is significantly different from the typical coherence value.
[0076] Embodiment 3. Fig. 10 is a block diagram showing an example of the configuration of a change detection device according to the third embodiment. The change detection device 300 shown in Fig. 10 includes a three-dimensional structure reconstruction unit 110, a phase removal unit 320, and a coherent change detection unit 230. The configuration and operation of the three-dimensional structure reconstruction unit 110 are the same as those in the first and second embodiments. The configuration and operation of the coherent change detection unit 230 are the same as those in the second embodiment.
[0077] The phase removal unit 320 includes a phase signal estimation unit 221, a pixel designation unit 321, and a phase signal removal unit 223. The configuration and operation of the phase signal estimation unit 221 are the same as those in the second embodiment. That is, the phase signal estimation unit 221 inputs a SAR image group including a plurality of SAR images stored in the SAR image storage unit 700 and one or more SAR images to be compared. The phase signal estimation unit 221 estimates phase signals of a predetermined region (layover region) for all SAR images constituting the SAR image group.
[0078] The phase signal remover 223 removes the phase signals of the pixels specified by the pixel designator 321 from all the SAR images.
[0079] The pixel designation unit 321 specifies, from the pixels of each SAR image, pixels from which the phase signal is to be removed.
[0080] Next, the operation of the motion detection device of the third embodiment will be described with reference to the flowchart of FIG.
[0081] The three-dimensional structure reconstruction unit 110 operates in the same manner as in the first and second embodiments to reconstruct a three-dimensional structure of a predetermined region in an observation region (step S101). In the phase removal unit 220, the phase signal estimation unit 221 operates in the same manner as in the first and second embodiments to estimate phase signals of the predetermined region for all SAR images (step S202).
[0082] The pixel designation unit 321 calculates the average intensity of each pixel of the SAR image across all the SAR images (each of the average intensities of pixels at the same position in a plurality of SAR images) (step S301).
[0083] If the average intensity of a pixel is greater than a predetermined value (a threshold value for filtering pixel selection), the pixel designation unit 321 identifies the pixel as a pixel whose phase signal should be removed (step S302). In that case, the process proceeds to step S203A. If the average intensity of the pixel is equal to or less than the predetermined value, the process proceeds to step S204. In step S302, the pixel designation unit 321 targets all SAR images and compares the intensity of each pixel in each SAR image with the average intensity of the pixel. Then, if the intensity of a pixel is greater than the average intensity of the pixel, the pixel designation unit 321 identifies the pixel in the SAR image (the SAR image in which the pixel is the target of comparison processing with the average intensity among all SAR images) as a pixel whose phase signal should be removed.
[0084] In step S203A, the phase signal remover 222 removes the phase signals of the pixels specified by the pixel designator 321 for all SAR images. The specific method of estimating and removing the phase signals is the same as that in the first embodiment (see FIG. 5, etc.). However, in this embodiment, the phase signal remover 222 estimates the phase signals of the received signal model g from the pixels specified by the pixel designator 321 in the observation signal. model Then, the process proceeds to step S204.
[0085] In step S204, the coherent change detection unit 230 executes the coherent change detection process in the same manner as in the second embodiment.
[0086] In this embodiment, when the average intensity of a pixel is greater than a predetermined value, the phase signal of the pixel in the SAR image is removed. That is, a pixel with low intensity is not subject to phase signal removal. The reliability of phase information of a pixel with low intensity may be low. If a coherent change detection process is performed after a phase signal removal process is performed on such a pixel, there is a risk that a coherent change will be erroneously detected. In the case where a phase signal removal process is not performed on a pixel with low intensity as in this embodiment, it is expected that the possibility of erroneous detection of a coherent change will be reduced.
[0087] [Variations] Fig. 12 is a block diagram showing a modification of the change detection device of the third embodiment. The change detection device 310 shown in Fig. 12 includes a three-dimensional structure reconstruction unit 110, a phase removal unit 330, and a coherent change detection unit 230. The phase removal unit 330 includes a pixel designation unit 321, a phase signal estimation unit 221, and a phase signal removal unit 222. Unlike the third embodiment, in this modification, the pixel designation unit 321 operates before the phase signal is estimated.
[0088] The configuration and operation of the three-dimensional structure reconstruction section 110 are the same as those in the first to third embodiments. The configuration and operation of the coherent change detection section 230 are the same as those in the second and third embodiments.
[0089] Next, the operation of the change detection device 310 according to the modified example of the third embodiment will be described with reference to the flowchart of FIG.
[0090] The pixel designation unit 321 calculates the average intensity of each pixel of the SAR image across all the SAR images (each of the average intensities of pixels at the same position in a plurality of SAR images) (step S301).
[0091] If the average intensity of the pixel is greater than a predetermined value (a threshold value for filtering pixel selection), the process proceeds to step S101 (step S302). If the average intensity of the pixel is equal to or less than the predetermined value, the process proceeds to step S204. In step S302, the pixel designation unit 321 compares the intensity of each pixel in each SAR image with the average intensity of the pixel for all SAR images. If the pixel designation unit 321 determines that the intensity of the pixel is greater than the average intensity of the pixel, the process proceeds to step S101. If not, the processes of steps S101, S202, and S203 are skipped.
[0092] The processes of steps S101, S202, and S203 are the same as those in the second embodiment. That is, the three-dimensional structure reconstruction unit 110 operates in the same manner as in the second embodiment to reconstruct the three-dimensional structure of a predetermined region in the observation region (step S101). In the phase removal unit 220, the phase signal estimation unit 221 estimates phase signals of the predetermined region of all SAR images by referring to the three-dimensional structure, as in the second embodiment (step S202). In the phase removal unit 330, the phase signal removal unit 222 removes the phase signals estimated by the phase signal estimation unit 221 for all SAR images, as in the second embodiment (step S203).
[0093] Then, the coherent change detection unit 230 executes the coherent change detection process in the same manner as in the second embodiment (step S204).
[0094] In this embodiment, when the average intensity of a pixel is greater than a predetermined value, a process of estimating a phase signal using a three-dimensional structure (step S202) and a process of removing a phase signal (step S203) are performed. That is, for pixels with low intensity, the process of estimating a phase signal and the process of removing a phase signal are not performed. As described above, the reliability of phase information of a pixel with low intensity may be low. If a coherent change detection process is performed after the phase signal removal process is performed on such a pixel, there is a risk that a false detection of a coherent change may occur. In the case where the process of estimating a phase signal and the process of removing a phase signal are not performed on pixels with low intensity as in this embodiment, it is expected that the possibility of false detection of a coherent change will be reduced.
[0095] The motion detection devices 300, 310 of the above-described third embodiment and its modified example are configured by applying a pixel designation unit 321 to the motion detection device 200 of the second embodiment. However, the pixel designation unit 321 may also be applied to the motion detection device 100 of the first embodiment.
[0096] Embodiment 4. In the first to third embodiments, in order to reconstruct the three-dimensional structure of a predetermined region in the observation region, the three-dimensional structure reconstruction unit 110 acquires three-dimensional point cloud data representing the three-dimensional structure by using a SAR tomography technique that uses a plurality of SAR images stored in the SAR image storage unit 700. However, the three-dimensional point cloud data may be acquired without using the SAR tomography technique.
[0097] In the fourth embodiment, the change detection device uses other types of three-dimensional data, rather than three-dimensional point cloud data from SAR images, to reconstruct the three-dimensional structure of a predetermined area. Examples of such three-dimensional data include three-dimensional point cloud data acquired by LiDAR, digital surface model (DSM: Digital Surface Mode), and data containing three-dimensional information generated from images acquired by optical satellites, etc. (polygon data of buildings, etc.). In the following, an example is given in which the change detection device uses data acquired by LiDAR, but 3D imaging devices other than LiDAR can also be used.
[0098] 14 is a block diagram showing an example of the configuration of a change detection device according to the fourth embodiment. A change detection device 400 shown in FIG.
[0099] Three-dimensional point cloud information (three-dimensional point cloud data) including the reflection intensity of each point in the observation area is stored in advance in the LiDAR data storage unit 800. The information (data) including the reflection intensity is three-dimensional point cloud data based on the analysis result of reflected light from the observation area received by a general 3D (dimension) LiDAR.
[0100] The three-dimensional structure reconstruction unit 111 estimates three-dimensional point cloud data in the SAR image from the three-dimensional point cloud data stored in the LiDAR data storage unit 800. The three-dimensional structure reconstruction unit 111 estimates three-dimensional point cloud data in the SAR image based on, for example, a simulation of a satellite that acquires the SAR image. As a procedure for simulating the satellite that acquires the SAR image, for example, a method is given in which a DSM created from the three-dimensional point cloud data stored in the LiDAR data storage unit 800 is input, coordinate conversion is performed to a SAR coordinate system (azimuth-range coordinates), and positions of one-time scattering, two-time scattering, three-time scattering, and radar shadow are estimated from the shape of the DSM and the position information of the SAR satellite, and reflection intensity and phase are estimated taking into account the reflection coefficient and roughness of the reflection surface. The three-dimensional structure reconstruction unit 111 essentially converts the three-dimensional point cloud data stored in the LiDAR data storage unit 800 into three-dimensional point cloud data in the SAR image.
[0101] The phase remover 120 and the coherent change detector 130 operate in the same manner as those in the first embodiment.
[0102] In this embodiment, the change detection device 400 does not execute a process of reconstructing a three-dimensional structure by SAR tomography. Therefore, the change detection device 400 can reduce the processing time compared to the change detection device 100 of the first embodiment.
[0103] The change detection device 400 of this embodiment is configured by applying a three-dimensional structure reconstruction unit 111 that uses three-dimensional point cloud data obtained by LiDAR to the change detection device 100 of the first embodiment. However, the three-dimensional structure reconstruction unit 111 may also be applied to the change detection device 200 of the second embodiment, the change detection device 300 of the third embodiment, and the change detection device 310 of the modified example.
[0104] As described above, the change detection device of each of the above embodiments generates 3D point cloud data by SAR tomography or the like, and uses the 3D point cloud data to remove phase components of a predetermined region (layover region) from the observation signal. The sensitivity of change detection by such a change detection device does not decrease even when a layover occurs in the observation region. Furthermore, the change detection device can perform coherent change detection with high accuracy even in the layover region.
[0105] In addition, the motion detection device does not need to execute a process to generate 3D point cloud data for the SAR image to be compared. In other words, the motion detection device can execute sensitive motion detection even if there is only one SAR image to be compared. Therefore, there is no need to collect the SAR image to be compared, and the processing load on the motion detection device does not increase significantly. As a result, the coherent change detection process can be executed in a short time.
[0106] Furthermore, since the change detection device can perform coherent change detection in a short time, it can quickly detect changes when used for purposes such as periodically monitoring an observation area.
[0107] Each of the above embodiments can be realized by hardware or by a computer program. Also, some of the components in each of the above embodiments can be realized by hardware and the other parts can be realized by software.
[0108] FIG. 15 is a block diagram showing an example of a computer having a CPU. The computer is implemented in the change detection device of each of the above-mentioned embodiments. The CPU 1000 executes processing according to a change detection program (software element: code) stored in the storage device 1001 to realize each function in the above-mentioned embodiments. That is, it realizes the functions of the three-dimensional structure reconstruction units 110 and 111, the phase removal units 120, 220, 320, and 330, and the coherent change detection units 130 and 230 in the change detection devices 100, 200, 300, 310, and 400 shown in FIG. 1, FIG. 7, FIG. 10, FIG. 12, and FIG. 14.
[0109] The storage device 1001 is, for example, a non-transitory computer readable medium. The non-transitory computer readable medium includes various types of tangible storage media. Specific examples of the non-transitory computer readable medium include a magnetic recording medium (for example, a hard disk), a magneto-optical recording medium (for example, a magneto-optical disk), a CD-ROM (Compact Disc-Read Only Memory), a CD-R (Compact Disc-Recordable), a CD-R / W (Compact Disc-ReWritable), and a semiconductor memory (for example, a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), and a flash ROM).
[0110] The program may also be stored in various types of transitory computer readable media, to which the program is supplied, for example, via a wired or wireless communication path, i.e., via an electrical signal, an optical signal, or an electromagnetic wave.
[0111] The memory 1002 is realized by, for example, a random access memory (RAM), and is a storage means for temporarily storing data when the CPU 1000 executes processing. A configuration is also conceivable in which a program held in the storage device 1001 or a temporary computer-readable medium is transferred to the memory 1002, and the CPU 1000 executes processing based on the program in the memory 1002.
[0112] Fig. 16 is a block diagram showing the main parts of the change detection device. The change detection device 10 shown in Fig. 16 includes a three-dimensional structure reconstruction unit (three-dimensional structure reconstruction means) 11 (realized by the three-dimensional structure reconstruction units 110 and 111 in the embodiment) that reconstructs the three-dimensional structure of a predetermined area (for example, a layover area) in the observation area, a phase removal unit (phase removal means) 12 (realized by the phase removal units 120, 220, 320, and 330 in the embodiment) that removes phase signals of a predetermined area in a plurality of SAR images in which the observation area is photographed using the three-dimensional structure, and a change detection unit (change detection means) 13 (realized by the coherent change detection units 130 and 230 in the embodiment) that generates a coherence image from the phase-removed SAR image pair and detects changes in the observation area based on the coherence values of the pixels that constitute the coherence image.
[0113] Fig. 17 is a block diagram showing the main parts of a change detection device according to another embodiment. In the change detection device 10 shown in Fig. 17, the three-dimensional structure reconstruction unit 11 includes a three-dimensional point cloud data estimation unit 11A (implemented by a three-dimensional structure reconstruction unit 110 in the embodiment) that estimates three-dimensional point cloud data having information on reflection intensity and phase as a three-dimensional structure using SAR tomography.
[0114] Fig. 18 is a block diagram showing main parts of a change detection device according to still another embodiment. In the change detection device 10 shown in Fig. 18, the three-dimensional structure reconstruction unit 11 includes a three-dimensional point cloud data conversion unit 11B (which is realized by a three-dimensional structure reconstruction unit 111 that performs a simulation related to SAR in the embodiment) that uses data including three-dimensional information (for example, three-dimensional point cloud data obtained by LiDAR) to estimate three-dimensional point cloud data having information on reflection intensity and phase in a SAR image.
[0115] Some or all of the above embodiments may be described as follows, but are not limited to the following:
[0116] (Appendix 1) A three-dimensional structure reconstruction means for reconstructing a three-dimensional structure of a predetermined area in an observation area; a phase removing means for removing a phase signal of the predetermined region in a plurality of SAR images of the observation region by utilizing the three-dimensional structure; a change detection means for generating a coherence image from the phase-removed SAR image pair and detecting a change in the observation area based on the coherence values of pixels constituting the coherence image; A change detection device comprising:
[0117] (Appendix 2) The three-dimensional structure reconstruction means includes a three-dimensional point cloud data estimation means for estimating three-dimensional point cloud data having information on reflection intensity and phase as a three-dimensional structure using SAR tomography. The change detection device of appendix 1.
[0118] (Appendix 3) The phase removing means removes a phase signal of the predetermined region in each SAR image in a group of SAR images acquired by a radar mounted on an aircraft, The change detection means generates coherence images from a plurality of SAR image pairs selected from the group of SAR images, and detects a change in the observation area based on whether or not there is a coherence image among the coherence images that includes a coherence value whose difference from a typical coherence value is greater than a predetermined value. 2. A change detection device according to claim 1 or claim 2.
[0119] (Note 4) The phase removing means removes phase signals of pixels in the SAR image whose intensities are greater than a predetermined intensity value. 4. The change detection device of any of claims 1 to 3.
[0120] (Appendix 5) The three-dimensional structure reconstruction means includes a three-dimensional point cloud data conversion means for estimating three-dimensional point cloud data having information on reflection intensity and phase in the SAR image using data including three-dimensional information. The change detection device of appendix 1.
[0121] (Appendix 6) Reconstruct the three-dimensional structure of a given area in the observation field, removing phase signals of the predetermined region in a plurality of SAR images in which the observation region is photographed, using the three-dimensional structure; A coherence image is generated from the phase-removed SAR image pair, and changes in the observation area are detected based on the coherence values of pixels that constitute the coherence image. Change detection methods.
[0122] (Appendix 7) As a 3D structure, 3D point cloud data containing reflection intensity and phase information is estimated using SAR tomography. Change detection method of Appendix 6.
[0123] (Appendix 8) A phase signal of the predetermined region in each SAR image in a group of SAR images acquired by a radar mounted on an aircraft is removed, A coherence image is generated from a plurality of SAR image pairs selected from the group of SAR images, and a change in the observation area is detected based on whether or not there is a coherence image among the coherence images that includes a coherence value whose difference from a typical coherence value is greater than a predetermined value. 8. The change detection method of claim 6 or 7.
[0124] (Appendix 9) Remove the phase signal of pixels in the SAR image whose intensity is greater than a predetermined intensity value. 9. The change detection method of any of claims 6 to 8.
[0125] (Appendix 10) Using data containing 3D information, estimate 3D point cloud data containing information on reflection intensity and phase in SAR images. Change detection method of Appendix 6.
[0126] (Appendix 11) To the computer, A process of reconstructing a three-dimensional structure of a predetermined area in an observation area; a process of removing phase signals of the predetermined region in a plurality of SAR images in which the observation region is captured, using the three-dimensional structure; generating a coherence image from the phase-removed SAR image pair and detecting a change in the observation area based on the coherence values of pixels constituting the coherence image; A computer-readable recording medium storing a change detection program for executing the above.
[0127] (Appendix 12) A process for estimating 3D point cloud data having information on reflection intensity and phase as a 3D structure using SAR tomography in a computer 12. The computer-readable recording medium of claim 11, storing a change detection program for executing the above.
[0128] (Appendix 13) To the computer, A process of reconstructing a three-dimensional structure of a predetermined area in an observation area; a process of removing phase signals of the predetermined region in a plurality of SAR images in which the observation region is captured, using the three-dimensional structure; generating a coherence image from the phase-removed SAR image pair and detecting a change in the observation area based on the coherence values of pixels constituting the coherence image; A change detection program to execute the following.
[0129] (Appendix 14) A process for estimating 3D point cloud data having information on reflection intensity and phase as a 3D structure using SAR tomography in a computer The change detection program of Appendix 13 that executes the above.
[0130] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]
[0131] 10. Change detection device 11 3D structure reconstruction unit 11A 3D point cloud data estimation unit 11B 3D point cloud data conversion section 12 Phase Removal Section 13 Change detection section 100,200,300,310,400 Change detection device 110,111 3D structure reconstruction unit 120,220,320,330 Phase removal section 121,221 Phase signal estimation section 122,222,223 Phase signal removal section 130,230 Coherent Change Detection Unit 321 Pixel specification section 700 SAR image storage unit 800 LiDAR data storage unit 1000 CPU 1001 Storage device 1002 Memory
Claims
1. a three-dimensional structure reconstruction means for reconstructing a three-dimensional structure of a predetermined area in an observation area; a phase removing means for removing phase information of the predetermined region in a plurality of SAR images of the observation region by utilizing the three-dimensional structure; a change detection means for generating a coherence image from the SAR image pair from which the phase information of the predetermined region has been removed, and detecting a change in the observation region based on the coherence values of pixels constituting the coherence image; A change detection device comprising:
2. The three-dimensional structure reconstruction means includes a three-dimensional point cloud data estimation means for estimating, as the three-dimensional structure, three-dimensional point cloud data having reflection intensity and phase information using SAR tomography. The change detection device of claim 1 .
3. the phase removing means removes phase information of the predetermined region in each SAR image in a group of SAR images acquired by a radar mounted on the flying object; The change detection means generates coherence images from a plurality of SAR image pairs selected from the group of SAR images, and detects a change in the observation area based on whether or not there is a coherence image among the coherence images that includes a coherence value whose difference from a typical coherence value is greater than a predetermined value. The change detection device according to claim 1 or 2.
4. The phase removing means removes phase information of pixels in the SAR image whose pixel intensity is greater than a predetermined intensity value. A change detection device according to any one of claims 1 to 3.
5. The three-dimensional structure reconstruction means includes a three-dimensional point cloud data conversion means for estimating three-dimensional point cloud data having reflection intensity and phase information in the SAR image using data including three-dimensional information. The change detection device of claim 1 .
6. Reconstructing a three-dimensional structure of a predetermined area in the observation area; removing phase information of the predetermined region in a plurality of SAR images of the observation region by utilizing the three-dimensional structure; A coherence image is generated from the SAR image pair from which the phase information of the predetermined region has been removed, and a change in the observation region is detected based on the coherence values of pixels constituting the coherence image. Change detection methods.
7. As a three-dimensional structure, three-dimensional point cloud data having reflection intensity and phase information is estimated using SAR tomography. The method of claim 6.
8. removing phase information of the predetermined region in each SAR image in a group of SAR images acquired by a radar mounted on an air vehicle; A coherence image is generated from a plurality of SAR image pairs selected from the group of SAR images, and a change in the observation region is detected based on whether or not there is a coherence image among the coherence images that includes a coherence value whose difference from a typical coherence value is greater than a predetermined value. A method for detecting change according to claim 6 or 7.
9. On the computer, A process of reconstructing a three-dimensional structure of a predetermined area in an observation area; A process of removing phase information of the predetermined region in a plurality of SAR images of the observation region by using the three-dimensional structure; generating a coherence image from the SAR image pair from which the phase information of the predetermined region has been removed, and detecting a change in the observation region based on the coherence values of pixels constituting the coherence image; A change detection program to execute the following.
10. A process of estimating, in a computer, three-dimensional point cloud data having reflection intensity and phase information using SAR tomography as a three-dimensional structure.
10. The change detection program according to claim 9, which executes the following:
Citation Information
Patent Citations
Coherence change detection techniques
EP3540462A1
3D shape calculation device of SAR image, and distortion correction device of SAR image
JP2008185375A
Displacement correction processing device and displacement correction processing program
JP2021021703A
Coherence change detection techniques
US20190285741A1
Method for processing multi-pass radar data for sensing and analysing multiple components of non-stationary scatterers
WO2008125929A2
