Building displacement determination device, method for determining building displacement, and program

Abstract

To provide a building displacement determination device, a method for determining a building displacement, and a program which can determine a displacement of a building more efficiently.SOLUTION: The building displacement determination device includes: a first image generation unit for generating a first image showing a distribution of a height index according to the altitude of a ground surface including a feature ground in a first period of time and in a second period of time; a second image generation unit for generating a second image showing a distribution of a vegetation index indicating the amount of vegetation in the feature surface in the first period of time and in the second period of time; a synthesis image generation unit for generating a synthesis image formed by overlapping the positions of the first image and the second image with respect to the ground surface in the first period of time and in the second period of time; and a detection unit for detecting a displacement of a building on the basis of the synthesis image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This invention relates to a building movement interpretation device, a building movement interpretation method, and a program.

Background Art

[0002] Conventionally, there is a technique for detecting changes (movements) in the range of a building based on differences in images taken from above at different times. Visible light images from above include image portions viewed obliquely according to the viewing angle of the imaging device. These are processed into ortho-images viewed from directly above, and the planar view position is specified.

[0003] In Patent Document 1, image data is generated by synthesizing a DSM (Digital Surface Model) image representing the height distribution of the ground surface with a precise ortho-image, and height changes are detected. Thereby, it is possible to detect with higher accuracy than before the movements in portions where it is difficult to make a determination with visible light images that apparently change due to differences in seasons, sunlight, etc.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, if irregularities on the ground surface and the inclination of the building surface according to shooting from an oblique direction remain, it becomes difficult to accurately align the position with the DSM image, and there arises a problem that it is not easily possible to accurately obtain the movement of the building.

[0006] An object of this invention is to provide a building movement interpretation device, a building movement interpretation method, and a program that can more efficiently determine the movement of a building.

Means for Solving the Problems

[0007] To achieve the above object, the present disclosure provides a first image generation unit that generates first images respectively representing the distribution of height indicators corresponding to the elevations of the ground surface including the ground object surfaces in a first period and a second period different from the first period; a second image generation unit that generates second images respectively representing the distribution of vegetation indicators indicating the amount of vegetation on the ground surface in the first period and the second period; a composite image generation unit that generates a composite image obtained by overlapping and aligning the positions of the first images and the second images related to the ground surface in the first period and the second period; a detection unit that detects the movement of a building based on the composite image; and a building movement interpretation device comprising the above.

Advantages of the Invention

[0008] According to the present disclosure, there is an effect that the movement of a building can be determined more efficiently.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. [First Embodiment] FIG. 1 is a block diagram showing the functional configuration of an information processing apparatus 1 which is a building movement interpretation apparatus according to the first embodiment.

[0011] The information processing apparatus 1 includes a control unit 11, a storage unit 12, an input / output interface 13 (I / F), a display unit 14, an operation reception unit 15, and the like.

[0012] The control unit 11 comprehensively controls the operation of the information processing apparatus 1. The control unit 11 has a processor that performs arithmetic processing. The processor may be a single general-purpose CPU (Central Processing Unit), or may have a plurality of CPUs, and these may perform arithmetic processing in parallel or independently according to the application or the like. The processor may include those specialized for specific arithmetic processing, image processing, or the like. The control unit 11 performs various control processes by reading and executing a program 120 and the like from the storage unit 12.

[0013] The storage unit 12 has a RAM (Random Access Memory) and a non-volatile memory, and stores various data. The RAM provides a working memory space for the control unit 11 and stores temporary data. The non-volatile memory stores and holds a program 120, setting data, and the like. The non-volatile memory is, for example, a flash memory, an HDD (Hard Disk Drive), or the like, but is not limited thereto. The storage unit 12 may have a ROM (Read Only Memory). An initial control program and the like may be stored in the ROM. The program 120 includes control programs related to the analysis image generation process, the learning control process, and the movement area extraction process described later. The program 120 includes a building movement detection model 121.

[0014] The input / output interface 13 performs input / output of data between the outside of the information processing apparatus 1 (including peripheral devices). The input / output interface 13 has a connection terminal 131 and a communication unit 132. The connection terminal 131 includes, for example, a USB (Universal Serial Bus) terminal, a LAN (Local Area Network) connector, and the like. The communication unit 132 controls communication according to a communication protocol (protocol) related to a LAN such as TCP / IP, for example.

[0015] As peripheral devices, there may be included a database device 21 which is an auxiliary storage device, and an optical reading device 22 which reads a portable storage medium (optical disk) such as a CDROM, a DVD, and a Blu-ray (registered trademark). Further, a magnetic tape may be included in the portable storage medium, and a reading device for reading this magnetic tape may be included in the peripheral devices.

[0016] Data that can be acquired by the information processing apparatus 1 from the outside via the input / output interface 13 includes captured image data 201. The captured image data 201 includes captured data of visible light and near-infrared light on the ground surface captured from above. The captured image data is an image with a resolution that can discriminate the movement of the building to be detected, particularly partial demolition or extension. If such a resolution is obtained, the imaging method is not particularly limited, for example, a UAV (unmanned aerial vehicle, including drones), an aircraft, a satellite, or the like. Further, synthetic image data 202 and movement interpretation data 203 may be stored in the database device 21 or the like. Although the synthetic image data 202 and the movement interpretation data 203 will be described later, these may be stored in the storage unit 12.

[0017] The display unit 14 performs display on the display screen based on the control of the control unit 11. The display screen is, for example, a liquid crystal display or an organic EL (Electro-Luminescent) display, but is not limited thereto.

[0018] The operation reception unit 15 receives an input operation from the outside and outputs an operation signal corresponding to the input operation to the control unit 11. The operation reception unit 15 includes, for example, a keyboard and a pointing device. The pointing device may be a mouse. The display unit 14 and / or the operation reception unit 15 may be peripheral devices of the information processing apparatus 1. That is, these may be attached to the main body (computer) of the information processing apparatus 1 including the control unit 11, the storage unit 12, and the input / output interface 13.

[0019] Next, generation of data used to determine a change in a building will be described. A change in a building means that the range of the building as seen from above changes when the building is newly constructed, demolished, or renovated. Changes in a building include new construction, renovation (including extension or partial demolition), and disappearance. If a building part that is higher than the ground surface appears in new data compared to old data, it is new construction or extension. If a building part that is higher than the ground surface in old data disappears in new data, it is partial demolition or disappearance (total demolition).

[0020] In this embodiment, DSM (Digital Surface Model) data and NDVI (Normalized Difference Vegetation Index) data are used when discriminating the range in which building movement has occurred. DSM data indicates the distribution of height indices that identify the height (elevation) of each point on the ground surface, including the surface of features such as buildings, by combining multiple visible light imaging pictures acquired by stereoscopic photography from above. NDVI data indicates the distribution of index values normalized by dividing the difference value obtained by subtracting the reflectance of red light from the reflectance of near-infrared light by the sum of the reflectance of near-infrared light and the reflectance of red light, based on visible light imaging pictures and near-infrared light pictures taken from above. The NDVI index is known as a vegetation index indicating the amount of vegetation, that is, an index in which the value becomes larger at locations with more vegetation. Based on the differences in height related to DSM data and vegetation conditions related to NDVI data at each location in a certain first period and a second period (different from the first period) for detecting building movement relative to the first period, the change in height and whether the change in height is due to building movement are discriminated. The DSM data and NDVI data for each period may be generated by combining visible light imaging pictures and near-infrared light pictures taken at the same timing and position. In this case, both data can be easily generated with their positions aligned.

[0021] Among these, only DSM data alone cannot accurately discriminate whether the change in height between old and new data is due to building movement. Conventionally, there is a technique for discriminating whether a building is a roof from a visible light imaging picture. In this technique, it is known that misjudgments often occur regarding whether it is a building due to differences in the way sunlight hits, such as differences in seasons, time zones, weather, etc.

[0022] Therefore, in the present embodiment, the visible light image is used for the generation of DSM data, that is, the acquisition of height information, and NDVI data is used for the discrimination of building areas. As described above, NDVI data is an index representing vegetation. Since there are usually no plants on the rooftops of buildings, the NDVI index value shows a small value. That is, the movement of the building is discriminated based on the part where the height has changed and is not vegetation. However, even in such cases, misjudgment may still occur. In the present disclosure, the elevation distribution, the distribution of NDVI index values, and the relationship between the changing part and the non-changing part can be further considered by the building movement detection model 121 (trained model), which is a machine learning model.

[0023] As described above, DSM data and NDVI data may be separately input to the building movement detection model 121. However, in the present embodiment, in order to easily and clearly define the movement area in pixel units, the composite image data 202 obtained by combining the DSM data and the NDVI data obtained from the captured image data 201 is used as the input data. For this input data, the building movement detection model 121 outputs data on the movement area of the building.

[0024] The composite image data 202 is data in which the positions of a total of 4 images of DSM data and NDVI data at two times to be compared are aligned and overlaid in 4 layers (layers). The alignment may be performed for the 4 images at once. Alternatively, the alignment of the 2 images at the same time may be performed respectively, and after the first composite image in the first period and the second composite image in the second period are generated, the 2-layer data of these two periods are further aligned to obtain a 4-layer composite image. In the latter case, if the data at the same position can be obtained by previously determining each pixel position according to absolute geographical coordinates at specific positions and intervals, the alignment of the 2-layer data can be easily performed. That is, the composite image data 202 is data in which the coordinates of each image data in the 4 layers are aligned. By using such composite image data 202 for the training of the building movement detection model 121, the machine learning of each feature point can be performed accurately at once.

[0025] FIG. 2 is a flowchart showing the control procedure of the analysis image generation process executed by the control unit 11 of the information processing apparatus 1. This analysis image generation process is started based on, for example, a predetermined input operation performed by the user designating the original image data to be analyzed.

[0026] The control unit 11 acquires the photographed image data 201 (S1). The control unit 11 sets a target area for determining the movement of the building with respect to the photographed image data 201, and generates DSM data for a range including at least the target area (S2). As described above, based on the image data photographed from a plurality of (at least three) positions and orientations at the same time, the height index at each horizontal position is specified.

[0027] Here, abnormal values may be mixed in the DSM data. For abnormalities, there are various factors, such as wavy shapes such as water surfaces, corrugated sheets, and corrugated roofs, which appear dispersed at each position. Therefore, the control unit 11 may mechanically adjust a value that is significantly deviated from the surroundings as an adjustment unit (S31). For example, the adjustment unit obtains the average value μ of the DSM values, that is, the height, and the standard deviation σ of the variation within a predetermined pixel range including the target pixel. The adjustment unit associates these values with each pixel. The adjustment unit determines whether the pixel value is within the reference variation range from the average value for each pixel, thereby determining whether it is an abnormal value. The reference variation range may be represented by, for example, μ±3σ. The abnormal value may be adjusted and changed to, for example, the boundary value of the reference variation range. For example, when the abnormal value is larger than μ+3σ, the abnormal value may be changed to the value of μ+3σ. Here, ±3σ is used as the reference variation range according to the determination of outliers in the normal distribution, but the reference variation range may be other values, such as ±2σ or ±4σ.

[0028] The control unit 11 generates NDVI data for a range including the target area from the captured image data 201 (S3). As described above, the control unit 11 obtains the NDVI index for each coordinate position using the red light image and the near-infrared light image in the captured image data 201, and obtains data related to the distribution of the NDVI index. The DSM data and the NDVI data may be defined in advance with geographical coordinates, that is, pixel positions such as latitude and longitude, as described above. In this case, in the captured image data, the visible light image capturing device and the near-infrared light image capturing device may be the same or different, and the images captured by different capturing devices may be obtained at different timings. The amount of deviation in the capturing positions by different capturing devices may be held in advance.

[0029] The control unit 11 performs a synthesis process by aligning the pixel positions of the DSM image and the NDVI image as the first image generation unit or the second image generation unit (S4; first image generation step, first image generation means), second image generation step, second image generation means). As described above, when the pixel positions of each image are defined by geographical coordinates or the like, it is only necessary to specify the same pixel position of the DSM image and the NDVI image based on the geographical coordinates.

[0030] The control unit 11 determines whether composite images for two time periods of the same target area have been obtained respectively (S5). When it is determined that the composite images for two time periods have not been obtained (S5; N), the process of the control unit 11 returns to process S1. In process S1, processes S1 to S4 based on the captured image data 201 of another time period (second time period) different from the already generated composite image are executed.

[0031] When it is determined that synthetic images of two periods have been obtained (S5; Y), the control unit 11, as a synthetic image generation unit, generates four-layer image data obtained by further synthesizing synthetic images of two periods of the same target area as image data to be analyzed (synthetic image data) (S6; synthetic image generation step, synthetic image generation means). The control unit 11, as a normalization processing unit, normalizes the distribution of the index values (height index and vegetation index) of each layer (S7). After the synthetic image is generated, the original DSM image and NDVI image may be stored as they are and reused later, or may be deleted each time.

[0032] Normalization is a process of adjusting the numerical ranges of DSM data and NDVI. The DSM data has a difference of several meters to about several hundred meters according to the height of the building in meters. On the other hand, the NDVI index value is obtained within the range of ±1. Therefore, these numerical ranges can differ by about two to four digits. By normalizing so that these numerical ranges become the same level, it is possible to reduce the occurrence of unnecessary weighting in the processing in the machine learning model described later. Normalization may be performed so that the average value of each index value is 0 and the variance is 1, but it may be different from these.

[0033] The synthetic image data in which the data of each layer is normalized may be output to the database device 21 again and stored in the synthetic image data 202. Alternatively, the synthetic image data may be stored in the storage unit 12. Then, the control unit 11 ends the analysis image generation process.

[0034] A part of the synthetic image data 202 to be analyzed obtained in this way is attached with correct answer data, and is used as teacher data, which is a set of the correct answer data and the synthetic image data associated with the correct answer data, for the learning and evaluation of the building change detection model 121. By inputting the synthetic image data 202 other than the above part into the machine-learned building change detection model 121, the building change detection model 121 extracts and outputs the difference between the building areas of two periods in the range of the ground surface included in the synthetic image as data of the building change part (change data).

[0035] The building change detection model 121 has an algorithm related to image recognition. Examples of the algorithm related to image recognition include those using a convolutional neural network. Here, a U-NET using a fully convolutional network may be used. Ground truth data is attached to a part of the synthesized image data 202 obtained as described above, and a further part of it is used for the training of the building change detection model 121, and the rest is used for the evaluation of the building change detection model 121. The training data and the evaluation data each contain data including the changed parts of each change type at a predetermined ratio. The synthesized image data used as the other training data and evaluation data may be determined randomly, or data not including the changed parts may be selected. At this time, each layer image data of the four-layer image for machine learning can be handled three-dimensionally. The position in the layer direction may be such that the DSM and NDVI of the first period are arranged side by side, and then the DSM data and NDVI of the second period are arranged side by side, as described above. Machine learning can be performed more effectively when the DSM data and NDVI data of the same period are arranged adjacent to each other and the set of these two data is arranged in the order of the shooting times than when arranged in other orders. However, the arrangement order of the four-layer image is not limited to this. Also, when a comparison algorithm between two-layer image data is used, it may be possible to process in an order different from the actual layer order in the synthesized image data 202 by separately specifying which layer the two-layer data (DSM and NDVI) of the same period belongs to.

[0036] The correct data can be obtained by the user or the person in charge visually inspecting the synthetic image data to identify the parts where changes have occurred and determining them together with the identification information of the change type. Note that the generation of the correct data does not necessarily have to be based only on DSM data and NDVI data. The correct data may be generated with reference to the entire visible light image, separately provided construction information data, etc. The synthetic image data associated with this correct data is input into the building change detection model 121. Then, the parameters are adjusted by the error backpropagation method or the like based on the difference between the output result of the building change detection model 121 and the correct data, so that the building change detection model 121 is learned. The learned building change detection model 121 is evaluated based on the synthetic image data for evaluation. The synthetic image data associated with the correct data is input into the building change detection model 121, and the degree of agreement and the tendency of the difference between the output result and the correct data are evaluated. The building change detection model 121 that has obtained a degree of agreement and a tendency of difference at a level that can withstand practical use is acquired as a learned model.

[0037] Figure 3 is a diagram showing examples of each layer data of the synthetic image and the correct data. Figures 3(a) and 3(b) are diagrams respectively representing the distributions (normalized) of the DSM data and the NDVI data at the first time period by luminance values. Figures 3(c) and 3(d) are diagrams respectively representing the distributions (normalized) of the DSM data and the NDVI data at the second time period by luminance values.

[0038] In the DSM data of Figures 3(a) and 3(c), the roofs of the buildings that are higher than the surroundings are arranged in a substantially rectangular shape and are white. Among the roofs, especially in the case of gabled roofs, the highest (brightest) area in the center extends linearly. Also, within the site around the building, there may be some areas that are slightly white and one level higher, such as in front of the entrance, storage areas, and roofed parking spaces. In addition to this, the one-level higher land in the upper right and the vicinity of the tops of the trees are shown in white. These have many parts where the contours show a curved shape and are not in a nice arc shape.

[0039] In the NDVI data of FIGS. 3(b) and 3(d), fields, plantings or weeds along roads, and tree parts are shown relatively white. That is, these are the parts where vegetation exists. Among the parts shown with higher height and brightness than the surroundings in FIGS. 3(a) and 3(c), the tops of trees, etc., are shown in FIGS. 3(b) and 3(d).

[0040] The size of the target range may be determined according to the number of pixels, etc., such that the time required for image processing falls within an appropriate range, and is not particularly limited. For the four-layer composite image determined in this way, in FIG. 3(e), the pixels in the range where changes have occurred are set as white data. That is, new buildings are located near the upper right and the center to the left of the image respectively. This change area may be further provided with information on whether it is an increased part (new construction, extension) or a decreased part (demolition, loss) of the building.

[0041] FIG. 4 is a diagram showing the control procedure of the learning control process by the control unit 11 and an example of the learning result. The learning control process shown in FIG. 4(a) may be started, for example, based on a predetermined input operation by the user.

[0042] The control unit 11 sets a specified number of target areas including the locations where there are building changes in the composite image data 202 (S11). The control unit 11 randomly sets a specified number of target areas in a range that does not overlap with the target areas already set in the composite image data 202 (S12).

[0043] The control unit 11 acquires and sets the correct data for the set target areas (S13). The correct data may be, for example, data generated by the user or the like as described above and for which a registration operation has been performed. The control unit 11 inputs a part of the composite image data of the target area for which the correct data has been set to the building change detection model 121, and learns the building change detection model 121 based on the output result and the correct data (S14).

[0044] The control unit 11 inputs the remaining composite image data of the target area where the correct data is set into the building change detection model 121, and evaluates the performance of the building change detection model 121 based on the output result and the correct data (S15). The control unit 11 determines whether the performance is at a level that can be used in practice (OK) (S16). If it is determined that the performance is at a level that can be used in practice (S16; Y), the control unit 11 acquires the learned model and designates it as an available machine learning model (S17). Then, the control unit 11 ends the learning control process. If it is determined that the performance is not at a level that can be used in practice (S16; N), the control unit 11 ends the learning control process without acquiring the learned model.

[0045] In this case, the control unit 11 may perform the learning control process again based on additional correct data while leaving the current settings. Alternatively, the control unit 11 may initialize the learned model once and perform learning on the initial building change detection model 121 based on the newly set target area and correct data.

[0046] FIG. 4(b) is a diagram showing an example of the result output by the learned building change detection model 121 after interpreting the area where the building has changed in the composite images of FIGS. 3(a) to (d). Compared with the correct data shown in FIG. 3(e), the area where the building has changed is generally correctly detected.

[0047] By using the building change detection model 121 learned as described above to extract the area where the building has changed, the interpretation of the building change is performed. FIG. 5 is a flowchart showing the control procedure of the change area extraction process by the control unit 11. This change area extraction process including the building change interpretation method of the present embodiment is executed by reading the program 120 from the storage unit 12 in response to a predetermined input operation by the user or the like.

[0048] The composite image generation unit acquires (S21) composite image data (normalized) including the target range for extracting the area where building movement has occurred. That is, the composite image data 202 is obtained by the above-described analysis image generation process. The composite image data may be generated together with the learning data and the evaluation data, or may be generated separately. In particular, after the learning of the building movement detection model 121, the composite image generation unit may generate the composite image data 202 by the analysis image generation process based on the captured image data 201 at any two separate times. As described above, since the pixel positions of each of the four-layer images of the composite image data 202 are aligned, the accuracy related to feature discrimination is likely to be improved.

[0049] As a detection unit, the control unit 11 inputs the composite image data 202 to the learned building movement detection model 121 (S22). The detection unit acquires the output result from the building movement detection model 121 and extracts the movement area of the building (S23). The processes of S22 and S23 correspond to the detection step of the building movement interpretation method of the present embodiment and correspond to the detection means in the program 120. The result of the extracted movement area is stored in the database device 21 or the storage unit 12 as the movement interpretation data 203. Then, the control unit 11 ends the movement area extraction process.

[0050] [Second Embodiment] As described above, the discrimination of building movement using NDVI and DSM has few misclassifications, but there is still a possibility of misidentifying objects other than buildings, especially objects that are in contact with the building or partially protrude from within the building, as the building or part of the building. Such objects to be misidentified include, for example, automobiles. Therefore, in one embodiment, a machine learning model for detecting automobiles is prepared, and by excluding the detected area of the automobiles from the building movement area, the possibility of misjudging the presence or absence of automobiles as building movement is reduced. Also, as described above, water surfaces and buildings with corrugated roofs, which are assumed to cause abnormal DSM values, can also be learned as misidentification targets and excluded.

[0051] FIG. 6 is a block diagram showing the functional configuration of the information processing apparatus 1a according to the second embodiment. The program 120a stored in the storage unit 12 is different from the program 120. Other configurations are the same, and the same reference numerals are assigned to the same configurations and detailed descriptions thereof are omitted.

[0052] In addition to the building movement detection model 121, the program 120a stores a misrecognition target detection model 122. The misrecognition target detection model 122 is a machine learning model for detecting the state of an object or the ground surface that is the misrecognition target.

[0053] The misrecognition target detection model 122 for detecting a misrecognition target may be learned for each misrecognition target based on a visible light image. That is, a misrecognition target detection model or the like is generated in advance. The image used for learning may include a band other than visible light. For example, the image used for learning may include a near-infrared light image used for generating NDVI data. In this case, an improvement in the detection accuracy of an automobile having a metal part or a building having a wavy roof is expected. Further, the DSM data may be included in the image used for learning the misrecognition target. For example, a two-layer composite image in which the DSM data and the NDVI data are aligned and overlapped may be used for learning as described above. Also, a two-layer composite image in which the DSM data and a visible light image or a near-infrared image are aligned and overlapped may be used for learning. In this case, an improvement in the detection accuracy is expected based on the height of an automobile or a roof or the occurrence of an abnormal value of the DSM. The type of the image used for learning may be different for each misrecognition target. The object of the misrecognition target may not be entirely visible. For example, an automobile partially visible from a parking space of a building may be learned to be detectable in a partially visible state.

[0054] FIG. 7 is a flowchart showing the control procedure of the movement area extraction process according to the second embodiment. In this movement area extraction process, the processes of S41 to S45 are added to the movement area extraction process of the first embodiment. Other processes are the same, and the same reference numerals are assigned to the same process contents and detailed descriptions thereof are omitted.

[0055] The detection unit acquires the captured image data 201 including the target area (S41). The detection unit inputs the acquired captured image into the learned misrecognition target detection model 122 (S42). The detection unit inputs the captured image of the first period to obtain an output, and further inputs the captured image of the second period to obtain an output. The captured image is the same type of image as the image used for learning, for example, a visible light image. Alternatively, the captured image may be a near-infrared light image, a two-layer composite image obtained by aligning and overlapping DSM data and NDVI data, a two-layer composite image obtained by aligning and overlapping DSM data and a visible light image, or a two-layer composite image obtained by aligning and overlapping DSM data and a near-infrared light image. The DSM data and the NDVI data may be the data generated by the composite image generation unit.

[0056] The detection unit acquires the output result of the misrecognition target detection model 122 and identifies the pixels within a predetermined misrecognition target area (misrecognition target area) (S43). The misrecognition target area may be determined, for example, by an OR process in which the positions detected in either of the two-period images are added together. Also, for example, the position detected in both of the two-period images may be determined by an AND process. The AND process is useful when identifying the pixels of a building having a corrugated roof and no movement. The detection unit sets a polygon shape that collectively represents the pixels of the misrecognition target area (S44). By polygonizing the misrecognition target area, it becomes easier to make a judgment when there is an overlap with the movement area. During polygonization, minute noise may be removed by a morphological transformation or the like. Then, the process proceeds to the process of process S21 by the composite image generation unit.

[0057] After process S23, the detection unit polygons the extracted movement area (S45). That is, the detection unit sets a polygon shape representing a pixel group in the result obtained in process S23. Each polygon represents an individual abnormal area. The detection unit excludes the portion of the polygonized abnormal area that overlaps with the misrecognition target area (S46). The detection unit calculates, for example, the degree of overlap between the polygon of the abnormal area and the polygon of the misrecognition target area. The degree of overlap may be obtained, for example, by Intersection over Union (IoU). The detection unit compares the obtained degree of overlap with a predetermined reference value. The reference value may be determined in advance based on examples and the like. The detection unit excludes the pixels within the polygon of the abnormal area whose degree of overlap is equal to or greater than the reference value from the result obtained in process S23. Then, the control unit 11 ends the movement area extraction process.

[0058] [Modification Example] Note that the present invention is not limited to the above-described embodiment, and various modifications are possible. For example, in the above, the machine learning model was used to interpret the movement of the building based on the synthetic image, but it is not necessary to use the machine learning model. The moving part of the building may be extracted by taking a logical product under appropriate conditions from the differences between various types of images and performing a process of removing the noise part.

[0059] Also, the DSM data is not limited to being generated only from visible light images. It may be generated using laser measurement or the like. Alternatively, if the height distribution is obtained as the first image, this data does not have to be DSM data. It may be height distribution data generated according to other formats. Alternatively, the height data may be a height distribution map obtained by polygonizing based on a house plan or the like.

[0060] In addition, in the above description, an example using NDVI as a vegetation index was given, but the vegetation index is not limited to this. Other indices using the reflectance of near-infrared light and red light may also be used. Alternatively, other indices such as an index related to the difference in reflectance between red light and green light (GRVI) or an index using the difference in reflectance between blue light and green light may be used as the vegetation index.

[0061] In addition, in the above, the change of the building was detected by the composite image data 202 combining DSM data and NDVI data, but it can also be used for other purposes. For example, the composite image data 202 may be used for detecting vehicles, detecting movement, detecting changes in land use such as the change from farmland to a parking lot, land reclamation, or detecting sediment deposition or outflow.

[0062] In addition, in the above embodiment, the analysis image generation process, the learning control process, and the change area extraction process are executed by a single information processing device 1, but it is not limited to this. Some of these processes may be executed by an information processing device different from the information processing device 1. The composite image data generated by the information processing device 1 may be read by another information processing device and the machine learning model may be learned. The information processing device 1 may acquire the learned machine learning model learned by another information processing device and use it as the building change detection model 121. Alternatively, a program that uses the learned machine learning model of another information processing device may extract the area where the building has changed. In addition, the DSM data and the NDVI data may be generated externally, and the process related to the synthesis thereof may be performed by the information processing device 1. In these cases, the program related to each process may be divided according to the operation range of each information processing device.

[0063] Also, in the above description, as an example of a computer-readable medium for storing the program 120 related to the control such as the analysis image generation process of the present invention, the storage unit 12 composed of a non-volatile memory such as an HDD or a flash memory is described, but the present invention is not limited thereto. As other computer-readable media, other non-volatile memories such as MRAM, and portable recording media such as CD-ROMs and DVD discs can be applied. Further, a carrier wave is also applied to the present invention as a medium for providing the data of the program according to the present invention via a communication line. In addition, the specific configurations, the contents and procedures of the processing operations shown in the above embodiments can be appropriately changed without departing from the gist of the present invention. The scope of the present invention includes the scope of the invention described in the claims and the equivalent scope thereof.

[0064] As described above, the information processing apparatus 1 of the present embodiment includes a control unit 11. The control unit 11, as a first image generation unit, generates DSM data, which is a first image representing the distribution of height indexes corresponding to the elevations of the ground surface including the ground surface at the first time and the second time different from the first time, respectively. The control unit 11, as a second image generation unit, generates NDVI data, which is a second image representing the distribution of vegetation indexes indicating the amount of vegetation on the ground surface at the first time and the second time, respectively. The control unit 11, as a composite image generation unit, generates a composite image in which the positions related to the ground surface of the DSM data and the NDVI data at the first time and the second time are aligned and superimposed. The control unit 11, as a detection unit, detects the movement of a building based on the composite image. In this way, by using the DSM data related to the height distribution and the NDVI data which is a vegetation index in combination to detect the movement of buildings, it is possible to reduce misjudgments mainly based on the influence of solar radiation etc. in visible light images and accurately detect the movement of buildings. That is, both the DSM data and the NDVI data have less influence from differences such as season, shooting time zone, weather, and shooting methods compared to visible light images. Therefore, in detecting the movement of buildings, these influences can be reduced and more stable results can be obtained. Also, at this time, by generating the above-mentioned four-layer composite image data, the information processing device 1 can avoid the labor and uncertainty such as alignment for each comparison process, and can easily and efficiently detect the movement of buildings.

[0065] Further, the composite image generation unit synthesizes the DSM data of the first period and the NDVI data of the first period to generate a first composite image, and synthesizes the DSM data of the second period and the NDVI data of the second period to generate a second composite image. Then, the composite image generation unit may generate a four-layer composite image by aligning and overlapping the positions of the first composite image and the second composite image. In this way, by generating the two-layer composite image data for each period and then generating the composite image data by overlapping the composite images of both periods, it becomes easy to combine the composite image data of any two periods.

[0066] Also, the control unit 11 may standardize the height index of the DSM data and the vegetation index of the NDVI data as a standardization processing unit. Thereby, unnecessary weighting caused apparently by the two parameters can be eliminated, and both parameters can be used with substantially the same weight for determining the movement of buildings.

[0067] Alternatively, the vegetation index may be a normalized difference vegetation index (NDVI data) based on the difference in reflectance of red light and near-infrared light. Since it is widely used as a vegetation index and is an index obtained from a combination of a captured image using near-infrared light, for which images are acquired in many measurements, and a captured image using visible light, it can be widely used for detecting building movements in many areas. Also, NDVI data has been widely used conventionally, and stable quantitative accuracy of the index can be expected.

[0068] Alternatively, as an adjustment unit, when the height index in each of the DSM data exceeds the reference variation range, for example, ±3σ from the average value, the control unit may adjust the height index outside the reference variation range to a value within the reference variation range. Abnormal height data may be regarded as a characteristic structure and may have an adverse effect on the determination of the building and its movement. By mechanically adjusting the height at such points within the reference variation range, false determination regarding building movement can be reduced, and the extraction accuracy of the movement location can be improved.

[0069] Alternatively, the detection unit may detect the false recognition target, such as a vehicle, which is different from the building on the ground surface, from the input of the ground surface image, by using a false recognition target detection model 122 that is machine-learned to output the false recognition target. The detection unit may exclude the abnormal region where the false recognition target is located from the building movement region. In the above building movement detection, there may be a false recognition target that is difficult to distinguish from the building. By detecting such a false recognition target separately in advance using a dedicated false recognition target detection model 122 and excluding it from the building movement region, the possibility that the presence or absence of this false recognition target is misjudged as a building movement can be reduced, and the extraction accuracy of the movement location can be improved.

[0070] In addition, the detection unit may input the composite image to the building movement detection model 121 that has been machine-learned to output building movement data in the range of the ground surface included in the composite image with respect to the input of the composite image, and acquire the building movement data. That is, instead of simply extracting the difference between images with two parameters, by using a machine learning model to extract the building movement area, it is possible to more accurately exclude noise and differences other than confusing buildings according to the shape and size of the difference part, etc., and extract the building movement area.

[0071] In addition, the building movement interpretation method of this embodiment includes the following steps. (1) A first image generation step of generating DSM data representing the distribution of height indexes corresponding to the elevations of the ground surface including the ground surface vegetation in the first period and the second period different from the first period, respectively. (2) A second image generation step of generating NDVI data representing the distribution of vegetation indexes indicating the amount of vegetation on the ground surface in the first period and the second period, respectively. (3) A composite image generation step of generating a composite image in which the positions related to the ground surface of the first image and the second image in the first period and the second period are aligned and overlaid. (4) A detection step of detecting building movement based on the composite image. Such a building movement interpretation method can reduce false judgments and detect building movement more accurately by using DSM data related to height distribution and NDVI data, which is a vegetation index, in combination to detect building movement. In addition, since it does not use special measurement data, it can be easily and accurately used for building movement in a wide area. And by generating a composite image in advance, the alignment at the time of detection becomes easy, and it becomes easy to consider not only the comparison of image layers at the same position but also the relationship with surrounding pixels. Therefore, this building movement interpretation method can extract building movement more efficiently.

[0072] In addition, by installing and executing the program 120 related to the above building movement interpretation method on a computer, it is possible to easily interpret the building movement by an ordinary computer without using special processing or hardware.

Explanation of symbols

[0073] 1 Information processing apparatus 11 Control unit 12 Memory unit 120 Program 121 Building movement detection model 122 False recognition target detection model 13 Input / output interface 131 Connection terminal 132 Communication unit 14 Display unit 15 Operation reception unit 21 Database apparatus 22 Optical reading apparatus 201 Photographed image data 202 Composite image data 203 Movement judgment data

Claims

1. a first image generating unit that generates first images representing a distribution of height indices according to an elevation of a ground surface including a feature surface at a first time point and at a second time point different from the first time point; a second image generating unit configured to generate second images each representing a distribution of a vegetation index indicating an amount of vegetation on the ground surface at the first time point and the second time point; a composite image generating unit that generates a composite image by aligning positions of the first image and the second image at the first time point and the second time point with respect to the ground surface and superimposing the first image and the second image at the first time point and the second time point; A detection unit that detects changes in a building based on the composite image; A building change interpretation device comprising:

2. The composite image generating unit includes: synthesizing the first image at the first time point and the second image at the first time point to generate a first synthetic image; synthesizing the first image at the second time and the second image at the second time to generate a second synthetic image; generating a four-layer composite image by aligning and superimposing the first composite image and the second composite image; 2. The building change interpretation device according to claim 1.

3. The building change interpretation device according to claim 1 , further comprising a standardization processing unit that standardizes the height index of the first image and the vegetation index of the second image.

4. 2. The building change interpretation device according to claim 1, wherein the vegetation index is a normalized vegetation index based on a difference between red light and near infrared light reflectance.

5. The building change interpretation device according to claim 1 , further comprising an adjustment unit that adjusts the height index outside the standard variation range to a value within the standard variation range when the height index in each of the first images exceeds a standard variation range.

6. The building change interpretation device according to claim 1, wherein the detection unit detects the mistaken object using a mistaken object detection model that has been machine-learned to output a predetermined mistaken object that is different from the building on the ground surface in response to an input image of the ground surface, and excludes an abnormal area in which the mistaken object is located from the change area of ​​the building.

7. The detection unit inputs the synthetic image into a building change detection model that has been machine-trained to output building change data in the range of the ground surface included in the synthetic image in response to the input of the synthetic image, and obtains the building change data. The building change interpretation device according to any one of claims 1 to 6.

8. a first image generating step of generating first images each representing a distribution of height indices according to an elevation of the ground surface including the surface of a feature at a first time point and at a second time point different from the first time point; a second image generating step of generating second images each representing a distribution of a vegetation index indicating the amount of vegetation on the ground surface at the first time point and the second time point; a composite image generating step of generating a composite image by aligning positions of the first image and the second image at the first time point and the second time point with respect to the earth's surface and superimposing the first image and the second image; A detection step of detecting a change in a building based on the composite image; A method for interpreting building changes, including:

9. Computer, a first image generating means for generating first images each representing a distribution of height indices according to an elevation of the ground surface including a feature surface at a first time point and at a second time point different from the first time point; a second image generating means for generating second images representing a distribution of a vegetation index indicating the amount of vegetation on the ground surface at the first time point and the second time point, a composite image generating means for generating a composite image by aligning positions of the first image and the second image at the first time point and the second time point with respect to the ground surface and superimposing the first image and the second image; A detection means for detecting a change in a building based on the composite image; A program that functions as a

Images