3D roof model generation device and computer program

Abstract

This invention provides a 3D roof model generation device and computer program capable of generating 3D roof models with higher accuracy. [Solution] The 3D roof model generation device includes: an acquisition unit that acquires a 2D image showing a roof and a 3D point cloud obtained by measuring the region containing the roof; a 2D roof surface model generation unit that generates a plurality of 2D roof surface models, each having a planar shape, from the 2D image; a 3D roof surface model generation unit that generates a plurality of 3D roof surface models by aligning each of the plurality of 2D roof surface models with respect to the 3D point cloud; a correction unit that corrects the positions of at least two 3D roof surface models based on the positional relationship of at least two of the plurality of 3D roof surface models; and a 3D roof model generation unit that generates a 3D roof model based on the corrected 3D roof surface models.

Description

[Technical Field]

[0001] This invention relates to a 3D roof model generation device and a computer program. [Background technology]

[0002] The PLATEAU Project, a nationwide urban digital twin project promoted by the Ministry of Land, Infrastructure, Transport and Tourism in collaboration with various players, is developing 3D urban models as platform data for urban activities. The level of detail of buildings included in the 3D urban models is defined from LOD (Level of Detail) 0 to LOD4. LOD0 is a model that does not have height information, and LOD1 is the simplest model that has height information. In contrast, LOD2 is a model that expresses the unevenness of the exterior and has information such as roof shape. Generally, in the creation of LOD2 buildings, roof drawing (creation of 3D roof models) accounts for 85% of the total workload. Furthermore, such roof drawing relies on manual drawing work by workers, resulting in low work efficiency. Therefore, there is a need to improve productivity by automating roof drawing.

[0003] Patent Document 1 discloses a 3D model generation device that generates a 3D model based on 3D point cloud data and 2D building outline data. This 3D model generation device overlays the 3D point cloud data and 2D building outline data based on positional information and divides the area within the building outline indicated by the 2D building outline data into a grid mesh of a predetermined size. The 3D model generation device estimates the roof surface by grouping the divided grid meshes based on the distribution trend of the 3D point cloud data. The 3D model generation device corrects the roof surface by correcting the grid mesh based on information from other grid meshes adjacent to the grid mesh, and generates a 3D model based on the corrected roof surface. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2024-5750 [Overview of the project] [Problems that the invention aims to solve]

[0005] There is a need to generate 3D roof models with higher accuracy. In particular, there is a need to generate the positional relationships of roof surfaces with high accuracy, taking into account the continuity of multiple roof surfaces.

[0006] The present invention aims to provide a 3D roof model generation device and computer program capable of generating 3D roof models with higher accuracy. [Means for solving the problem]

[0007] The 3D roof model generation apparatus according to the present invention includes: an acquisition unit that acquires a 2D image showing a roof and a 3D point cloud obtained by measuring the region including the roof; a 2D roof surface model generation unit that generates a plurality of 2D roof surface models, each having a planar shape, from the 2D image; a 3D roof surface model generation unit that generates a plurality of 3D roof surface models by aligning each of the plurality of 2D roof surface models with respect to the 3D point cloud; a correction unit that corrects the positions of at least two 3D roof surface models based on the positional relationship of at least two of the plurality of 3D roof surface models; and a 3D roof model generation unit that generates a 3D roof model based on the corrected 3D roof surface models.

[0008] Furthermore, in the 3D roof model generation apparatus according to the present invention, it is preferable that the correction unit corrects the position of the 3D roof surface models by aligning the 3D shared vertices of pairs of 3D roof surface models that correspond to pairs of 2D roof surface models that share a common vertex, and whose 3D shared vertices corresponding to the said common vertex are close to each other in 3D space.

[0009] Furthermore, in the 3D roof model generation apparatus according to the present invention, it is preferable that the correction unit groups together 3D roof surface models that correspond to 2D roof surface models that share a common vertex and whose 3D common vertices corresponding to the said common vertex are in close proximity to each other in 3D space, and corrects the position of each 3D roof surface model belonging to each group based on the positional relationship of the 3D roof surface models belonging to each group.

[0010] Furthermore, in the 3D roof model generation apparatus according to the present invention, it is preferable that the correction unit corrects the position of the 3D roof surface models belonging to each group by performing regression analysis for each group based on an error item relating to the distance from the point cloud corresponding to the 3D roof surface model belonging to each group from the 3D point cloud, and an error item relating to the distance between the mutually corresponding 3D shared vertices of the 3D roof surface models belonging to each group.

[0011] Furthermore, in the 3D roof model generation apparatus according to the present invention, it is preferable that the correction unit corrects the position of the 3D roof surface model by performing regression analysis on a point cloud obtained by projecting a 3D point cloud corresponding to a 3D roof surface model whose inclination angle approximates a predetermined fixed angle onto a plane whose normal is a 3D direction vector corresponding to the fixed angle.

[0012] Furthermore, in the 3D roof model generation apparatus according to the present invention, it is preferable that the correction unit uses a point cloud from the 3D point cloud that is within a predetermined distance from the 3D roof surface model as the 3D point cloud corresponding to the 3D roof surface model.

[0013] Furthermore, in the 3D roof model generation apparatus according to the present invention, it is preferable that the 3D roof surface model generation unit generates a 3D roof surface model by aligning a specific 2D roof surface model with a point among the points included in the 3D point cloud that is located in the region corresponding to the specific 2D roof surface model in the horizontal direction and is located at a predetermined distance or more from a position corresponding to the boundary line of the specific 2D roof surface model.

[0014] In addition, in the three-dimensional roof model generation device according to the present invention, the three-dimensional roof surface model generation unit estimates a three-dimensional roof surface from a two-dimensional roof surface model and a three-dimensional point cloud, and sets a weight for each point in the three-dimensional point cloud according to the distance between each point in the three-dimensional point cloud and the estimated three-dimensional roof surface, and generates a three-dimensional roof surface model from the three-dimensional point cloud based on the set weight.

[0015] In addition, the computer program according to the present invention acquires a two-dimensional image showing a roof and a three-dimensional point cloud obtained by measuring an area including the roof, generates a plurality of two-dimensional roof surface models each having a planar shape from the two-dimensional image, generates a plurality of three-dimensional roof surface models by aligning each of the plurality of two-dimensional roof surface models with the three-dimensional point cloud, corrects the positions of at least two of the plurality of three-dimensional roof surface models based on the positional relationship between at least two of the plurality of three-dimensional roof surface models, and causes a computer to generate a three-dimensional roof model based on the corrected three-dimensional roof surface models.

Effect of the Invention

[0016] The three-dimensional roof model generation device and the computer program according to the present invention can generate a three-dimensional roof model with higher accuracy.

Brief Description of the Drawings

[0017] [Figure 1] It is a block diagram showing a schematic configuration of a three-dimensional roof model generation device 100. [Figure 2] It is a flowchart showing an example of the flow of generation processing. [Figure 3] (A) shows an example of an orthoimage, (B) shows an example of a DSM image, (C) shows an example of a roof line, (D) shows a schematic diagram for explaining the main direction of the roof line, (E) shows an example of a roof line, and (F) shows an example of a two-dimensional roof surface model. [Figure 4](A) shows an example of a DSM point cloud, (B) shows a schematic diagram to explain the target point cloud, (C) shows a schematic diagram to explain a 3D roof surface model, and (D) shows a schematic diagram to explain a 3D roof surface model. [Figure 5] (A) shows a schematic diagram to explain the angle of inclination, and (B) and (C) show schematic diagrams to explain the correction of the angle of inclination. [Figure 6] (A) and (B) show schematic diagrams to explain proximity, and (C) shows a schematic diagram to explain correction. [Figure 7] (A) and (B) show examples of three-dimensional roof models. [Modes for carrying out the invention]

[0018] Various embodiments of the present invention will be described below with reference to the drawings. Please note that the technical scope of the present invention is not limited to these embodiments, but extends to the invention described in the claims and its equivalents.

[0019] Figure 1 is a block diagram showing the schematic configuration of the 3D roof model generation device 100.

[0020] The 3D roof model generation device 100 is a personal computer, a notebook personal computer, a server, etc. The 3D roof model generation device 100 includes an operating device 101, a display device 102, a communication device 103, a storage device 110, and a processing circuit 120, etc.

[0021] The operating device 101 has input devices such as a keyboard and a mouse, and an interface circuit that acquires signals from the input devices. It accepts operations from the user and outputs a signal corresponding to the user's input to the processing circuit 120.

[0022] The display device 102 is an example of an output unit. The display device 102 has a display made of liquid crystal, organic EL, etc., and an interface circuit that outputs image data to the display, and displays the image data on the display according to instructions from the processing circuit 120.

[0023] The communication device 103 is an example of an output unit. The communication device 103 is equipped with a wired or wireless communication interface circuit and connects the 3D roof model generation device 100 to a communication network. The communication device 103 performs wired communication according to a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol). Alternatively, the communication device 103 may perform wireless communication according to the IEEE (Institute of Electrical and Electronics Engineers) 802.11 standard. The communication device 103 transmits information supplied from the processing circuit 120 to an external device. The communication device 103 also supplies information received from the external device to the processing circuit 120.

[0024] The storage device 110 includes, for example, semiconductor memory such as RAM (Random Access Memory) or ROM (Read Only Memory), a fixed disk device such as a hard disk, or a portable storage device such as an optical disc. The storage device 110 stores computer programs, data, etc., used for processing by the processing circuit 120. The computer program is installed in the storage device 110 from a server (not shown) via a communication device 103. Alternatively, the computer program may be installed in the storage device 110 from a computer-readable portable recording medium using a known setup program, etc. The portable recording medium is, for example, a CD-ROM or DVD-ROM. The computer program may also be distributed from a server or the like and installed in the storage device 110.

[0025] The processing circuit 120 is, for example, a CPU (Central Processing Unit). The processing circuit 120 may also be an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), etc. The processing circuit 120 is connected to the operating device 101, display device 102, communication device 103, and storage device 110, etc., and controls each of these parts. The processing circuit 120 reads the program stored in the storage device 110 and operates according to the read program, thereby functioning as an acquisition unit 121, a roof line extraction unit 122, a 2D roof surface model generation unit 123, a 3D roof surface model generation unit 124, a correction unit 125, a 3D roof model generation unit 126, and an output control unit 127. The processing circuit 120 generates a 3D roof model representing the roof of a building.

[0026] Figure 2 is a flowchart showing an example of the generation process performed by the 3D roof model generation device 100.

[0027] The following describes an example of the operation of the 3D roof model generation device 100, referring to the flowchart shown in Figure 2. The operation flow described below is mainly executed by the processing circuit 120 in cooperation with each element of the 3D roof model generation device 100, based on a program stored in the memory device 110 beforehand.

[0028] First, the acquisition unit 121 acquires a two-dimensional image showing the roof of the building (step S101). The acquisition unit 121 acquires an orthophoto and / or a DSM (Digital Surface Model) image as a two-dimensional image. When the acquisition unit 121 acquires both an orthophoto and a DSM image, it acquires orthophotos and DSM images showing the same roof so that the same position of the same roof is displayed at corresponding positions in the images.

[0029] Figure 3(A) shows an example of an orthomosaic image. As shown in Figure 3(A), an orthomosaic image is an image that has been transformed from an aerial or satellite photograph taken from above, in which positional displacement of the included image has been removed, so that the image is displayed at the correct size and position without tilt, as if viewed from directly above.

[0030] Figure 3(B) shows an example of a DSM image. DSM is an elevation model that digitally models the surface, including features such as buildings and trees, and further represents it in a tiled manner. As shown in Figure 3(B), the grayscale value of each pixel in the DSM image is set to a value corresponding to the height of the corresponding position on the surface. For example, the grayscale value of each pixel in the DSM image is set to a value normalized within the grayscale range of the height of the corresponding position on the surface. Alternatively, an nDSM (Normalized Digital Surface Model) image may be used instead of a DSM image. nDSM is a model that shows the height of features such as buildings and trees from the ground surface. For example, the grayscale value of each pixel in the nDSM image is set to a value normalized within the grayscale range of the height of the corresponding position.

[0031] Next, the roof line extraction unit 122 extracts roof lines from the 2D image acquired by the acquisition unit 121 (step S102). Roof lines are straight lines (edges) included in the outer frame (boundary line) of one or more planes included in the roof. For example, the roof line extraction unit 122 extracts roof lines from a 2D image using a learning model. The learning model is pre-trained using multiple training data, each including an image and the position of the roof lines included in that image, to output roof line information about the roof lines included in the input image when an image is input. As roof line information, the learning model outputs the position (coordinates) of both ends (vertices) of each roof line, and the confidence level of each roof line for each roof line in the image. The confidence level of a roof line indicates the likelihood of it being a roof line; for example, the higher the likelihood that it is a roof line, the higher the confidence level, and the lower the likelihood that it is not a roof line, the lower the confidence level. HEAT (Holistic Edge Attention Transformer for Structured Reconstruction), neural networks, etc., are used as learning models. HEAT is a deep learning model that extracts wireframes (edges). HEAT integrates and utilizes both the coordinate features and image features of edges. In addition, HEAT has a branch that determines edges using only coordinate information. Furthermore, when determining the truth value of an edge, HEAT considers its relationship with other edge candidates using a Self Attention mechanism. The 3D roof model generation device 100 can extract roof lines from a 2D image with high accuracy by using HEAT. The learning model has multiple input channels. The roof line extraction unit 122 inputs, for example, the red, green, and blue components of the orthomosaic image and the DSM image to each input channel of the learning model and obtains roof line information output from the learning model. The roof line extraction unit 122 extracts straight lines as roof lines, which are lines that connect the coordinates of the vertices of each roof line in the image, as shown in the acquired roof line information.

[0032] Figure 3(C) shows an example of roof lines extracted from a 2D image using HEAT. As shown in Figure 3(C), the roof line extraction unit 122 extracts each edge E connecting the pairs of vertices T of each roof line included in the roof line information as a roof line.

[0033] The roof line extraction unit 122 may extract edge pixels from a 2D image whose difference in grayscale value between adjacent pixels is greater than or equal to a threshold, detect straight lines from the extracted edge pixels using the least squares method or Hough transform, and extract each detected straight line as a roof line. In this case, the roof line extraction unit 122 may calculate the reliability such that the reliability increases as the number of edge pixels located within a predetermined range from the extracted roof line increases, and decreases as the number of edge pixels located within a predetermined range from the extracted roof line decreases. Furthermore, the roof line extraction unit 122 may extract roof lines from either an orthophoto or a DSM image, or both.

[0034] Next, the roof line extraction unit 122 deletes roof lines with low reliability from among the extracted roof lines (step S103). The roof line extraction unit 122 deletes roof lines from all extracted roof lines whose reliability is below a predetermined reliability threshold. By deleting roof lines with low reliability, the 3D roof model generation device 100 can reduce the influence of roof patterns or dirt, etc., and generate a 2D roof surface model with high accuracy.

[0035] Next, the roof line extraction unit 122 merges vertices that are close to each other into a single vertex from among the vertices of each extracted roof line (step S104). The roof line extraction unit 122 identifies multiple vertices whose distance from each other is less than or equal to a predetermined first distance threshold as vertices that are close to each other and merges them into a single vertex. By merging vertices that are close to each other, the roof line extraction unit 122 connects roof lines that have vertices that are close to each other. In the example shown in Figure 3(C), vertices T1 and T3 that are close to each other are merged into one, and vertices T2 and T4 that are close to each other are merged into one. As a result, roof line (edge) E1 having vertices T1 and T2 and roof line E2 having vertices T3 and T4 are merged into a single roof line. Note that if merging multiple roof lines results in roof lines that are completely duplicated, the roof line extraction unit 122 keeps only one of the roof lines and deletes the others. The 3D roof model generation device 100 can generate a 2D roof surface model with high accuracy by integrating vertices that are close to each other, thereby reducing the influence of parts that are not clearly captured in the 2D image.

[0036] Next, the roof line extraction unit 122 simplifies the polyline formed by two roof lines that share a vertex (step S105). First, the roof line extraction unit 122 extracts pairs of roof lines that share a vertex from among the currently remaining roof lines. Next, the roof line extraction unit 122 extracts pairs of roof lines as polylines where the distance between the line connecting the vertices that the extracted roof line pairs do not share and the vertices that the pairs share is less than or equal to a predetermined second distance threshold. Next, the roof line extraction unit 122 deletes the vertices that the extracted roof line pairs share, deletes the roof line pairs, and extracts the line connecting the vertices that the pairs do not share as a new roof line. In the example shown in Figure 3(C), roof line E3 having vertices T1 and T5, and roof line E4 having vertices T5 and T6 are extracted as a pair of roof lines that share vertex T5. Since the distance D between the line L connecting vertices T1 and T6, which are not shared by roof lines E3 and E4, and vertex T5, ​​which is shared by roof lines E3 and E4, is less than or equal to the second distance threshold, the pair of roof lines E3 and E4 is extracted as a polyline. Then, vertex T5 is deleted, roof lines E3 and E4 are deleted, and instead, the line L connecting vertices T1 and T6 is extracted as the roof line. The 3D roof model generation device 100 can generate a 2D roof surface model with high accuracy by simplifying the polyline formed by the two roof lines, thereby reducing the effects of distortion and other factors in the 2D image.

[0037] Next, the roof line extraction unit 122 snaps (moves to adjacent roof lines) vertices that are not connected to any other roof lines and are close to other roof lines to the adjacent roof line (hereinafter sometimes referred to as adjacent roof lines) (step S106). The roof line extraction unit 122 extracts roof lines from the currently remaining roof lines that have vertices that are not connected to any other roof lines (hereinafter sometimes referred to as unconnected vertices). The roof line extraction unit 122 determines whether there are any roof lines whose distance from the extracted roof line to the unconnected vertices is less than or equal to the third distance threshold, and if so, extracts that roof line as an adjacent roof line. When an adjacent roof line is extracted, the roof line extraction unit 122 snaps the unconnected vertices to the adjacent roof line. The roof line extraction unit 122 extends the roof line with the unconnected vertices to the snapped adjacent roof line (i.e., connects it to the adjacent roof line), and also divides the adjacent roof line into two roof lines at the snapped unconnected vertices. In the example shown in Figure 3(C), vertex T7 of roof line E6 is extracted as a disconnected vertex, and roof line E5, whose distance from vertex T7 is less than or equal to the third distance threshold, is extracted as a nearby roof line. Vertex T7 is snapped to roof line E5, roof line E6, which has vertex T7, is extended to roof line E5, and roof line E5 is divided into a roof line connecting vertex T8 and the snapped vertex T7, and a roof line connecting vertex T9 and the snapped vertex T7. The 3D roof model generation device 100 can generate a 2D roof surface model with high accuracy by snapping disconnected vertices adjacent to nearby roof lines to nearby roof lines, thereby reducing the influence of parts that are not clearly captured in the 2D image.

[0038] Next, the roof line extraction unit 122 deletes intersecting roof lines (step S107). The roof line extraction unit 122 determines whether there are any pairs of intersecting roof lines among the currently remaining roof lines, and if such pairs exist, it deletes the roof line with the lower reliability from that pair. By deleting intersecting roof lines, the 3D roof model generation device 100 can reduce the influence of roof patterns or dirt, etc., and generate a 2D roof surface model with high accuracy.

[0039] Next, the roof line extraction unit 122 deletes roof lines with endpoints (step S108). The roof line extraction unit 122 extracts and deletes roof lines with unconnected vertices from the remaining roof lines as roof lines with endpoints. In the example shown in Figure 3(C), vertex T10 of roof line E7 is extracted as an unconnected vertex, and roof line E7 is deleted. By deleting roof lines with endpoints, the 3D roof model generation device 100 can reduce the influence of roof patterns or dirt, etc., and generate a 2D roof surface model with high accuracy.

[0040] Next, the roof line extraction unit 122 calculates the main direction of the roof lines (step S109). First, the roof line extraction unit 122 identifies the extension direction of each remaining roof line and groups the roof lines whose respective extension directions form an angle less than or equal to a predetermined first angle threshold (e.g., 5°). Next, for each group, the roof line extraction unit 122 uses the length of the roof lines belonging to each group as a weight (the longer the roof line, the greater the weight) and calculates a weighted average of the extension directions of the roof lines belonging to each group as the extension direction of that group. Next, the roof line extraction unit 122 calculates the total length of the roof lines belonging to each group and identifies the first group with the largest calculated total and the second group with the second largest calculated total. Next, the roof line extraction unit 122 determines whether the angle formed by the extension direction of the first group and the direction perpendicular to the extension direction of the second group is less than or equal to the second angle threshold (e.g., 5°). If the angle is less than or equal to the second angle threshold, the roof line extraction unit 122 uses the sum of the lengths of the roof lines belonging to the first group and the sum of the lengths of the roof lines belonging to the second group as weights (the larger the sum of the lengths of the roof lines, the greater the weight) to calculate a weighted average of the extension direction of the first group and the direction perpendicular to the extension direction of the second group as the main direction of the roof lines. On the other hand, if the angle is greater than the second angle threshold, the roof line extraction unit 122 identifies the extension direction of the first group as the main direction of the roof lines.

[0041] Figure 3(D) is a schematic diagram illustrating the main directions of roof lines. Figure 3(D) shows five roof lines E11 to E15. In this example, roof lines E11 and E12, whose extension directions are similar to each other, are classified as Group A; roof lines E13 and E14, whose extension directions are similar to each other, are classified as Group B; and roof line E15, for which there are no roof lines with similar extension directions, is classified as Group C. In Figure 3(D), the direction from left to right is defined as the left-right direction, the direction sloping downwards to the right relative to the horizontal direction is defined as the - direction, and the direction sloping upwards to the right relative to the left-right direction is defined as the + direction. The extension direction A11 of Group A is sloping by -5.0 with respect to the left-right direction, and the total length of the roof lines belonging to Group A is 20m. The extension direction A12 of Group B is sloping by +91.0 with respect to the left-right direction, and the total length of the roof lines belonging to Group B is 15m. The extension direction A13 of group C is tilted by +40.0 degrees relative to the left-right direction, and the total length of the roof lines belonging to group C is 10m. In this case, group A, which has the largest total length of roof lines, is identified as the first group, and group B, which has the second largest total length of roof lines, is identified as the second group. The main direction of the roof lines is set to {20[m]×(-5.0°)+15[m]×(91.0°-90.0°)} / (20[m]+15[m])=-2.4°.

[0042] Next, the roof line extraction unit 122 corrects each roof line and its vertices based on the main direction of the roof lines (step S110). The roof line extraction unit 122 groups together one or more roof lines from among the currently remaining roof lines in which the angle between the extension direction of the roof line and the main direction is less than or equal to a predetermined third angle threshold (e.g., 5°) and which are continuous with each other. For each group, the roof line extraction unit 122 estimates (calculates) a straight line for the vertices of the roof lines belonging to each group using a conditional least squares method, which has the condition that the direction of the estimated straight line coincides with the main direction. The roof line extraction unit 122 also groups together one or more roof lines from among the currently remaining roof lines in which the angle between the extension direction of the roof line and the direction perpendicular to the main direction is less than or equal to the third angle threshold and which are continuous with each other. The roof line extraction unit 122 estimates (calculates) a straight line for each group of roof lines, using a conditional least squares method that has the condition that the direction of the estimated straight line coincides with the direction perpendicular to the main direction. Next, the roof line extraction unit 122 calculates the perpendicular points (points where a perpendicular line drawn from point P to line L intersects line L) from the vertices of the roof lines belonging to each group to the estimated straight line, and moves each vertex to the calculated perpendicular point position to place each roof line belonging to the group on the estimated straight line. Next, for each roof line that was connected to the moved roof line, the roof line extraction unit 122 calculates the intersection point between the moved roof line and the roof line that was connected to it, and corrects the position of each roof line's vertex to the calculated intersection point position. Specifically, the roof line extraction unit 122 aligns the extension direction of a roof line among a plurality of roof lines, where the angle between the extension direction and the main direction or a direction perpendicular to the main direction is less than or equal to the third angle threshold, with the main direction or a direction perpendicular to the main direction. The 3D roof model generation device 100 can generate a 2D roof surface model with high accuracy by correcting each roof line and its vertices based on the main direction of the roof lines, thereby reducing the effects of distortion and other factors in the 2D image.

[0043] Next, the roof line extraction unit 122 deletes unnecessary vertices (step S111). The roof line extraction unit 122 extracts pairs of roof lines from among the remaining roof lines that share vertices and have the same extension direction. Next, the roof line extraction unit 122 deletes the vertices shared by the extracted pairs of roof lines, deletes those pairs, and extracts a straight line connecting the vertices that the extracted pairs of roof lines do not share as a new roof line. By deleting unnecessary vertices and reducing the number of roof lines, the 3D roof model generation device 100 can reduce the processing time and processing load required for the processing described later.

[0044] Figure 3(E) shows an example of roof lines adjusted by the roof line extraction unit 122. The roof lines shown in Figure 3(E) are those adjusted by the processing in steps S103 to S111 compared to the roof lines shown in Figure 3(C). As shown in Figure 3(E), overlapping or adjacent roof lines in Figure 3(C) are merged, and most roof lines are corrected to extend in the main direction A1 or in the direction A2 perpendicular to the main direction.

[0045] Next, the 2D roof surface model generation unit 123 generates multiple 2D roof surface models, each having a planar shape, from each of the currently remaining roof lines (step S112). A 2D roof surface model is a two-dimensional (planar) polygon. The 2D roof surface model generation unit 123 detects the smallest unit area enclosed by the currently remaining roof lines and not containing any other roof lines as a single plane, and generates a 2D roof surface model representing each detected plane. In this way, the 2D roof surface model generation unit 123 generates multiple 2D roof surface models from a 2D image.

[0046] Figure 3(F) shows an example of a two-dimensional roof surface model. The roof lines shown in Figure 3(F) represent the two-dimensional roof surface model U generated from the roof lines shown in Figure 3(E). As shown in Figure 3(F), a two-dimensional roof surface model U is generated for each region enclosed by roof lines that does not contain any other roof lines.

[0047] Next, the acquisition unit 121 acquires a 3D point cloud obtained by measuring the area including the building roof shown in the 2D image acquired in step S101 (step S113). The acquisition unit 121 acquires a 3D point cloud, such as a DSM point cloud. The 3D point cloud is created from a point cloud created by automatic processing of aerial photographs (SfM (Structure from Motion) or MVS (Multi-View Stereos)). The 3D point cloud may also be calculated by calculating the position of the reflection point when a laser is shone from the aircraft toward the ground surface of the area including the building roof.

[0048] Figure 4(A) shows an example of a DSM point cloud. As shown in Figure 4(A), each point P included in the DSM point cloud indicates the location of the reflection point of a laser beam that was irradiated from above and reflected off a building roof or the ground surface, etc.

[0049] Next, the 3D roof surface model generation unit 124 extracts target point clouds corresponding to each 2D roof surface model from the points included in the 3D point cloud acquired by the acquisition unit 121, for each 2D roof surface model generated by the 2D roof surface model generation unit 123 (step S114). The 3D roof surface model generation unit 124 identifies the region corresponding to each 2D image in the horizontal direction within the 3D space containing the 3D point cloud, and identifies the region corresponding to each 2D roof surface model and the position corresponding to the boundary line of each 2D roof surface model. For each 2D roof surface model, the 3D roof surface model generation unit 124 extracts as target point clouds the points included in the 3D point cloud that are located within the region corresponding to each 2D roof surface model in the horizontal direction and are at least a predetermined distance from the position corresponding to the boundary line of each 2D roof surface model. The predetermined distance is generally set in advance based on the size of the chamfer (C chamfer, R chamfer, or thread chamfer, etc.) formed at the edge of the roof, and is set to a distance equivalent to, for example, 10 cm.

[0050] Figure 4(B) shows a schematic diagram to explain the target point cloud. As shown in Figure 4(B), first, point cloud P1 (white circles and black circles) located within the region corresponding to the 2D roof surface model U in the horizontal direction is extracted from the points included in the 3D point cloud. Next, from the extracted point cloud P1, point cloud P2 (black circles) located within a predetermined distance D1 from the position corresponding to the boundary line B of the 2D roof surface model U is excluded from the target point cloud, and only point cloud P3 (white circles) located at a predetermined distance D1 or more from the position corresponding to the boundary line B of the 2D roof surface model U is extracted as the target point cloud.

[0051] The 3D roof surface model generation unit 124 may also extract all points located within the region corresponding to each 2D roof surface model in the horizontal direction from among the points included in the 3D point cloud as the target point cloud.

[0052] Next, the 3D roof surface model generation unit 124 estimates a 3D roof surface from the extracted target point cloud for each 2D roof surface model generated by the 2D roof surface model generation unit 123 (step S115). The 3D roof surface model generation unit 124 estimates the 3D roof surface using, for example, a RANSAC (Random Sample Consensus) based least squares method. First, the 3D roof surface model generation unit 124 extracts a predetermined number (e.g., 1000) of combinations of three points from the extracted target point cloud, and calculates a plane that passes through the three points included in each combination for each extracted combination. For each calculated plane, the 3D roof surface model generation unit 124 calculates the number of points in the target point cloud whose distance from each plane is less than or equal to a predetermined fourth distance threshold (e.g., 10 cm). On the plane with the largest number of calculated points, the 3D roof surface model generation unit 124 estimates the region corresponding to the 2D roof surface model in the horizontal direction as the 3D roof surface.

[0053] The 3D roof surface model generation unit 124 may estimate the 3D roof surface using a conventional least squares method or the like, rather than a RANSAC-based method.

[0054] Next, the 3D roof surface model generation unit 124 generates a 3D roof surface model for each 2D roof surface model generated by the 2D roof surface model generation unit 123, based on the extracted target point cloud and the estimated 3D roof surface (step S116). The 3D roof surface model generation unit 124 excludes points from the extracted target point cloud whose distance from the estimated 3D roof surface is greater than the fourth distance threshold as outliers. Next, the 3D roof surface model generation unit 124 sets a weight for each point in the target point cloud according to the distance between the remaining target point cloud and the estimated 3D roof surface (the shorter the distance, the greater the weight), and uses a weighted least squares method or the like to calculate the plane that minimizes the sum of the squares of the weighted distances to each point in the target point cloud. On the calculated plane, the 3D roof surface model generation unit 124 generates a polygon that represents the region corresponding to the 2D roof surface model in the horizontal direction as a 3D roof surface model.

[0055] Figure 4(C) shows a schematic diagram illustrating the 3D roof surface model. As shown in Figure 4(C), first, the 3D roof surface F1 is estimated from the target point group P3 (white and black circles) extracted for each 2D roof surface model. Next, points P4 (black circles) whose distance from the 3D roof surface F1 is greater than the fourth distance threshold D2 are excluded from the target point group P3 as outliers. Then, for each point P5 (white circle) included in the remaining target point group, a weight is set for each point P5 according to the distance D5 between it and the 3D roof surface F1, and the 3D roof surface model is generated on the plane F2 where the sum of the squares of the weighted distances to each point P5 is minimized.

[0056] Figure 4(D) shows a schematic diagram illustrating the three-dimensional roof surface model. As shown in Figure 4(D), each three-dimensional roof surface model V is generated by aligning each two-dimensional roof surface model U shown in Figure 3(F) with the three-dimensional point cloud P shown in Figure 4(A).

[0057] In this way, the 3D roof surface model generation unit 124 generates multiple 3D roof surface models by aligning each of the multiple 2D roof surface models with respect to a 3D point cloud.

[0058] In particular, the 3D roof surface model generation unit 124 estimates the 3D roof surface from the 2D roof surface model and the 3D point cloud. The 3D roof surface model generation unit 124 then sets a weight for each point in the 3D point cloud according to the distance between each point in the 3D point cloud and the estimated 3D roof surface, and generates a 3D roof surface model from the 3D point cloud based on the set weights. The 3D roof model generation device 100 can estimate the 3D roof surface model with higher accuracy by estimating the 3D roof surface model in two stages from each point in the 3D point cloud. The 3D roof surface model generation unit 124 may also generate the 3D roof surface model using a normal least squares method or the like without setting a weight for each point included in the target point cloud. Furthermore, the 3D roof surface model generation unit 124 may generate the 3D roof surface model without excluding points from the target point cloud that are greater than the fourth distance threshold as outliers.

[0059] Furthermore, the 3D roof surface model generation unit 124 generates a 3D roof surface model by aligning each 2D roof surface model with a target point cloud set for each 2D roof surface model, that is, points located within the region corresponding to each 2D roof surface model in the horizontal direction and at a predetermined distance or more from the position corresponding to the boundary line of each 2D roof surface model. Chamfers may be formed at the edges of the roof. The 3D roof model generation device 100 can calculate the inclination of the 3D roof surface model with greater accuracy in the horizontal direction by not using points near the position corresponding to the boundary line of the 2D roof surface model in the calculation of the 3D roof surface model.

[0060] Next, the 3D roof surface model generation unit 124 corrects the inclination angle of the generated 3D roof surface model in a specific direction (step S117).

[0061] Figure 5(A) shows a schematic diagram illustrating the inclination angle. As shown in Figure 5(A), the specific direction is set to the direction in the two-dimensional space where the two-dimensional roof surface model is generated, that is, the direction parallel to the horizontal plane in the three-dimensional space where the three-dimensional roof surface model is generated. The specific direction is set to, for example, the main direction of the roof lines or a direction perpendicular to the main direction. The specific direction may be set to any direction parallel to the horizontal plane. The inclination angle of the three-dimensional roof surface model in the specific direction is set to the angle between a first straight line extending in the specific direction and a second straight line that passes through the first straight line and extends vertically, intersecting the vertical plane and the three-dimensional roof surface model.

[0062] The 3D roof surface model generation unit 124 calculates the inclination angle for each 3D roof surface model. The 3D roof surface model generation unit 124 extracts 3D roof surface models in which the difference between the inclination angle and a predetermined fixed angle is less than or equal to a fourth angle threshold (e.g., 5°) as 3D roof surface models in which the inclination angle approximates the fixed angle, and corrects them so that the inclination angle becomes the fixed angle. The fixed angle is set to, for example, 0°. The fixed angle may be set to any angle between 0° and less than 90°. When the fixed angle is 0°, if a 3D roof surface model in which the difference between the inclination angle and the fixed angle is less than or equal to the fourth angle threshold has a line corresponding to a roof line extending in a specific direction, that line is corrected so that it becomes parallel to the horizontal plane. The 3D roof surface model generation unit 124 sets a 3D direction vector that extends along a vertical plane and whose angle with a specific direction is a fixed angle, and sets an orthogonal plane with the set 3D direction vector as the normal. The 3D roof surface model generation unit 124 projects the 3D point cloud corresponding to each 3D roof surface model onto an orthogonal plane for each 3D roof surface model whose inclination angle is to be corrected, and calculates a straight line on the orthogonal plane that minimizes the sum of the squares of the distances from the 2D point cloud projected onto the orthogonal plane. The correction unit 125 corrects the inclination angle of the 3D roof surface model in a specific direction by placing the 3D roof surface model on a plane that passes through the calculated straight line and is parallel to the set 3D direction vector. As a result, the 3D roof model generation device 100 can reduce the error (fluctuation) in the inclination angle for each 3D roof surface model.

[0063] Figures 5(B) and (C) show schematic diagrams to explain the correction of the inclination angle. Figures 5(B) and (C) show an example where the specific direction is the main direction, the fixed angle is 0°, and the 3D roof surface model is slightly inclined with respect to the horizontal direction. As shown in Figure 5(B), when the fixed angle is 0°, the 3D direction vector is set to the same direction as the specific direction, i.e., parallel to the horizontal plane, and the orthogonal plane normalized to the 3D direction vector is set to be parallel to the vertical plane. As shown in Figure 5(C), a straight line is calculated using the least squares method or the like for the 2D point cloud projected onto the orthogonal plane from the 3D point cloud corresponding to the 3D roof surface model, and the corrected 3D roof surface model is placed on a plane that passes through the calculated straight line and is parallel to the 3D direction vector (specific direction). As a result, if the 3D roof surface model that was slightly inclined with respect to the horizontal direction has a line corresponding to a roof line extending in the specific direction, that line will be placed parallel to the horizontal direction.

[0064] Next, the correction unit 125 determines whether the 3D roof models in each pair are in close proximity to each other in 3D space, for each pair of 3D roof models generated from a pair of 2D roof models that share one or more shared vertices (step S118). A pair of 3D roof models generated from a pair of 2D roof models that share shared vertices is an example of a pair of 3D roof models corresponding to a pair of 2D roof models that share shared vertices. For each pair of 3D roof models, the correction unit 125 calculates the vertical distance between the points corresponding to each shared vertex in the 2D roof model within each 3D roof model in 3D space, i.e., the pair of 3D shared vertices shared by each 3D roof model. If the vertical distance between at least one pair of 3D shared vertices shared by the 3D roof models in each pair is less than or equal to a predetermined fifth distance threshold (for example, a value equivalent to 10 cm), the correction unit 125 determines that the 3D shared vertices corresponding to the shared vertices of that pair of 3D roof models are in close proximity to each other in 3D space. In that case, the correction unit 125 determines that the 3D roof surface models related to that pair are in close proximity, and that the roof surfaces in real space corresponding to the 3D roof surface models related to that pair are connected to each other. On the other hand, if the vertical distance between pairs of all 3D shared vertices shared by the 3D roof surface models related to each pair is greater than the fifth distance threshold, the correction unit 125 determines that the 3D shared vertices corresponding to the shared vertices of the pair of 3D roof surface models are not in close proximity to each other in 3D space. In that case, the correction unit 125 determines that the 3D roof surface models related to each pair are not in close proximity, and that the roof surfaces in real space corresponding to the 3D roof surface models related to that pair are not connected to each other.

[0065] Figures 6(A) and 6(B) show schematic diagrams to illustrate whether three-dimensional roof surface models are close to each other in three-dimensional space. In the example shown in Figure 6(A), two-dimensional roof surface models U1 and U2 share common vertices T11 and T12, and two-dimensional roof surface models U2 and U3 share common vertices T13 and T14. As shown in Figure 6(B), three-dimensional roof surface models V1 and V2 generated from two-dimensional roof surface models U1 and U2 have three-dimensional shared vertices S11 and S21 corresponding to the shared vertex T11, and three-dimensional shared vertices S12 and S22 corresponding to the shared vertex T12, respectively. Furthermore, three-dimensional roof surface models V2 and V3 generated from two-dimensional roof surface models U2 and U3 have three-dimensional shared vertices S23 and S33 corresponding to the shared vertex T13, and three-dimensional shared vertices S24 and S34 corresponding to the shared vertex T14, respectively. The distance D4 between 3D shared vertices S11 and S21, and the distance D5 between 3D shared vertices S12 and S22 are sufficiently small, and it is determined that 3D roof surface models V1 and V2 are in close proximity to each other. On the other hand, the distance D6 between 3D shared vertices S23 and S33, and the distance D7 between 3D shared vertices S24 and S34 are sufficiently large, and it is determined that 3D roof surface models V2 and V3 are not in close proximity to each other.

[0066] As shown in Figure 6(A), even if two shared vertices are located at the same position in two-dimensional space, as shown in Figure 6(B), the corresponding three-dimensional shared vertices may be located at different vertical positions in three-dimensional space. Three-dimensional shared vertices corresponding to two roof surfaces that are actually connected will be close to each other, while three-dimensional shared vertices corresponding to two roof surfaces that overlap when viewed from the vertical but are not actually connected will be far apart. Therefore, the three-dimensional roof model generation device 100 can determine the positional relationship between two roof surfaces with high accuracy based on the distance between the three-dimensional shared vertices.

[0067] Next, the correction unit 125 classifies each 3D roof surface model into one or more groups (step S119). The correction unit 125 groups each 3D roof surface model so that 3D roof surface models that have been determined to be close to each other are included in the same group. That is, the correction unit 125 groups 3D roof surface models that correspond to 2D roof surface models that share a common vertex and whose 3D shared vertices corresponding to that common vertex are close to each other in 3D space. On the other hand, the correction unit 125 groups each 3D roof surface model so that 3D roof surface models that do not share a common 3D vertex and 3D roof surface models that have been determined not to be close to each other are not included in the same group. In the example shown in Figure 6(B), 3D roof surface models V1 and V2, which have been determined to be close to each other, are classified into the same group. On the other hand, 3D roof surface model V3, which does not share a common 3D vertex with 3D roof surface model V1 and has been determined not to be close to 3D roof surface model V2, is classified into a different group from 3D roof surface models V1 and V2.

[0068] Next, the correction unit 125 corrects the position of the 3D roof surface model belonging to each of the classified 3D roof surface models (step S120).

[0069] The correction unit 125 corrects the position of the 3D roof surface models so that the sum of the squares of the distances from points in the 3D point cloud corresponding to each 3D roof surface model belonging to a group to each 3D roof surface model, and the sum of the squares of the vertical distances between the 3D shared vertices shared by each pair of 3D roof surface models belonging to a group that share a 3D shared vertex, is minimized. The correction unit 125 uses the point cloud within a predetermined distance from each 3D roof surface model generated (or corrected) by the 3D roof surface model generation unit 124 as the 3D point cloud corresponding to each 3D roof surface model. This allows the 3D roof surface model generation device 100 to appropriately correct the position of the 3D roof surface models. Alternatively, the correction unit 125 may use the point cloud within a predetermined distance from each 3D roof surface model as the 3D point cloud corresponding to each 3D roof surface model. Furthermore, the correction unit 125 may use points from the target point group corresponding to each 3D roof surface model, excluding those excluded as outliers in step S116, as the 3D point group corresponding to each 3D roof surface model. Alternatively, the correction unit 125 may use all of the target point groups corresponding to each 3D roof surface model as the 3D point group corresponding to each 3D roof surface model. Alternatively, the correction unit 125 may use all points located in the 3D space containing the 3D point group, in the horizontal direction, within the region corresponding to the 2D roof surface model corresponding to the 3D roof surface model, as the 3D point group corresponding to each 3D roof surface model. In a 3D space having x and y axes extending horizontally and a z axis extending vertically, the plane k on which each 3D roof surface model is arranged is defined by the following equation (1).

number

[0070] At this time, the correction unit 125 calculates the parameters (a k , b k , d k ) of each plane so that the objective function defined by the following equation (2) is minimized.

Equation

[0071] Figure 6(C) shows a schematic diagram illustrating the correction of the position of the 3D roof surface model. The 3D roof surface models V1 and V2 are fitted to the corresponding point clouds P11 and P21, respectively, and are positioned so that the 3D shared vertices S11 and S12 are located at the same position as the 3D shared vertices S21 and S22, respectively.

[0072] In this way, the correction unit 125 corrects the position in the height direction of at least two three-dimensional roof surface models based on the positional relationship of the two three-dimensional roof surface models. In particular, the correction unit 125 corrects the position of the three-dimensional roof surface models by aligning the three-dimensional shared vertices of a pair of three-dimensional roof surface models that correspond to a pair of two-dimensional roof surface models that share a common vertex and whose three-dimensional shared vertices corresponding to that common vertex are close to each other in three-dimensional space. As a result, the three-dimensional roof model generation device 100 can arrange two three-dimensional roof surface models, each corresponding to two roof surfaces that are connected to each other in real space, so as to connect them appropriately in three-dimensional space.

[0073] Furthermore, the correction unit 125 corrects the position of the 3D roof surface models belonging to each group based on the positional relationship of the 3D roof surface models belonging to each group, for each group into which the 3D roof surface models have been classified based on proximity. As a result, the 3D roof model generation device 100 can appropriately extract two 3D roof surface models corresponding to two roof surfaces that are mutually connected in real space, and appropriately connect these two 3D roof surface models in 3D space. In particular, the correction unit 125 corrects the position of the 3D roof surface models belonging to each group by performing regression analysis for each classified group based on error items related to the distance from the point cloud corresponding to the 3D roof surface models belonging to each group in the 3D point cloud, and error items related to the distance between the mutually corresponding 3D shared vertices of the 3D roof surface models belonging to each group. Regression analysis includes methods such as the nonlinear least squares method. As a result, the 3D roof model generation device 100 can arrange the two 3D roof surface models corresponding to two roof surfaces that are mutually connected in real space so as to be appropriately connected in 3D space, while also arranging each 3D roof surface model in an appropriate position according to the 3D point cloud.

[0074] The correction unit 125 may also correct the position of each 3D roof surface model, taking into account the constraints, for 3D roof surface models that have constraints. A constraint is, for example, fixing the inclination angle of a 3D roof surface model to a fixed angle when the inclination angle in a specific direction approximates a fixed angle. In this case, the correction unit 125 extracts the 3D roof surface model whose inclination angle in a specific direction has been corrected in step S117, i.e., the 3D roof surface model whose inclination angle approximates a fixed angle, as a 3D roof surface model with constraints. The correction unit 125 corrects the position of each 3D roof surface model, i.e., the 3D roof surface model whose inclination angle approximates a fixed angle, so that the inclination angle of each 3D roof surface model with constraints becomes a fixed angle.

[0075] The orthogonal plane described in step S117, that is, the straight line on the orthogonal plane whose normal is a three-dimensional direction vector corresponding to a fixed angle, is defined by the following equation (3).

number

[0076] If all three-dimensional roof surface models belonging to the group have constraints, the correction unit 125 adjusts the parameters (a) of each plane so that the objective function defined by equation (4) below is minimized, instead of the objective function defined by equation (2) above. k ,b k d k ) and the parameter (α) of each line k ,β k Calculate ).

number

[0077] If a group includes both a 3D roof surface model with constraints and a 3D roof surface model without constraints, the correction unit 125 adjusts the parameters (a) of each plane so that the objective function defined by equation (5) below is minimized, instead of the objective function defined by equation (2) above. k ,b k d k ) and the parameter (α) of each line k ,β k Calculate ).

number

[0078] The correction unit 125 corrects the positions of the respective three-dimensional roof surface models so as to be arranged in a region corresponding to the corresponding two-dimensional roof surface model in the horizontal direction on a plane defined by the calculated respective parameters (a k , b k , d k ), (α k , β k ). Thereby, the three-dimensional roof surface model generation device 100 can correct the position while maintaining the inclination angle at a fixed angle for a three-dimensional roof surface model whose inclination angle approximates the fixed angle. For example, when the fixed angle is 0°, the three-dimensional roof surface model generation device 100 can arrange a line corresponding to a roof line extending in a specific direction substantially parallel to the horizontal plane at an appropriate position while arranging the line parallel to the horizontal plane for a three-dimensional roof surface model having the line.

[0079] In this way, the correction unit 125 corrects the position of the 3D roof surface model by performing regression analysis on a 2D point cloud obtained by projecting a 3D point cloud corresponding to a 3D roof surface model whose inclination angle approximates a predetermined fixed angle onto an orthogonal plane whose normal is a 3D direction vector corresponding to the fixed angle. Regression analysis includes methods such as the nonlinear least squares method. As a result, the 3D roof model generation device 100 can position a 3D roof surface model with a constrained inclination angle so that it has that constrained inclination angle, and can position the 3D roof surface model with a more appropriate inclination. Furthermore, the 3D roof model generation device 100 can reduce the order of the objective function, making it easier to solve the minimization problem and reducing the processing time and processing load required for the generation process.

[0080] For example, the correction unit 125 corrects the position of each 3D roof surface model so that the objective function defined by equation (2) above is minimized. In this case, the correction unit 125 can correct the position of each 3D roof surface model so that the 3D shared vertices shared by a pair of 3D roof surface models are located at the same position (same height). Alternatively, the correction unit 125 may correct the position of each 3D roof surface model so that the objective function defined by equation (4) above is minimized. In this case, the correction unit 125 can correct the position of each 3D roof surface model so that the 3D shared vertices shared by a pair of 3D roof surface models are located at the same position (same height), and the inclination angle of a 3D roof surface model whose inclination angle approximates a fixed angle becomes a fixed angle. Alternatively, the correction unit 125 may correct the position of each 3D roof surface model so that the objective function obtained by removing the third term from the objective function defined by equation (4) above is minimized. In that case, the correction unit 125 can correct the position of each 3D roof surface model so that the inclination angle of the 3D roof surface model whose inclination angle approximates a fixed angle becomes a fixed angle. Alternatively, the correction unit 125 may correct the position of each 3D roof surface model so that the objective function obtained by removing the second term from the objective function defined in equation (2) above is minimized.

[0081] Next, the 3D roof model generation unit 126 generates a 3D roof model based on the 3D roof surface model corrected by the correction unit 125 (step S121). The 3D roof model generation unit 126 generates the 3D roof model by combining each 3D roof surface model in 3D space.

[0082] Figure 7(A) shows an example of a 3D roof model created by combining 3D roof surface models before correction, and Figure 7(B) shows an example of a 3D roof model created by combining 3D roof surface models after correction. As shown in Figure 7(A), in the 3D roof model created by combining 3D roof surface models before correction, a gap Q occurs between 3D roof surface model V4 and 3D roof surface model V5. On the other hand, as shown in Figure 7(B), in the 3D roof model created by combining 3D roof surface models after correction, there is no gap between 3D roof surface model V4 and 3D roof surface model V5, and 3D roof surface model V4 and 3D roof surface model V5 are properly connected.

[0083] Next, the output control unit 127 outputs the 3D roof model generated by the 3D roof model generation unit 126 by displaying it on the display device 102 or by transmitting it to an external device via the communication device 103 (step S122), thus ending the series of steps.

[0084] Note that any of the processes in steps S103-S111 and S114-S119 may be omitted.

[0085] As explained above, the 3D roof model generation device 100 corrects the positions of at least two of the multiple 3D roof surface models based on the positional relationship of those two 3D roof surface models. This enables the 3D roof model generation device 100 to generate 3D roof models with higher accuracy.

[0086] Generally, laser point clouds offer high measurement accuracy but high acquisition costs. On the other hand, DSM point clouds offer low acquisition costs but low measurement accuracy. Therefore, when a 3D roof model is generated directly from a DSM point cloud, the accuracy (reproducibility of the roof) of the generated 3D roof model will be low. Furthermore, 2D images such as orthomosaic images offer high measurement accuracy, and the accuracy (reproducibility of the roof surface) of the 2D roof surface model generated from 2D images is high. However, information in the vertical direction cannot be obtained from 2D images. The 3D roof model generation device 100 generates a 2D roof surface model from a 2D image, and while maintaining its shape, aligns the generated 2D roof surface model with the 3D point cloud. In addition, the 3D roof model generation device 100 corrects the position of each 3D roof surface model using 3D shared vertices shared by 3D roof surface models corresponding to interconnected roof surfaces. As a result, the 3D roof model generation device 100 can generate a 3D roof model with high accuracy and low cost, taking into account the continuity of multiple roof surfaces.

[0087] Furthermore, the 3D roof model generation device 100 can easily generate a 3D roof model using only general-purpose data such as a 2D image showing the roof and a 3D point cloud obtained by measuring the area containing the roof, without using building contour data. In addition, since the 3D roof model generation device 100 can generate 3D roof models for all types of roofs using the same method, it can generate 3D roof models for various types of roofs while suppressing complexity in the generation process. Moreover, the 3D roof model generation device 100 can generate a 3D roof model with higher accuracy by aligning (fitting) the 2D roof surface model to the 3D point cloud.

[0088] Those skilled in the art will understand that various changes, substitutions, and modifications can be made without departing from the spirit and scope of the present invention. For example, the embodiments and modifications described above may be combined as appropriate within the scope of the invention. [Explanation of symbols]

[0089] 100 3D roof model generation device, 121 acquisition unit, 122 roof line extraction unit, 123 2D roof surface model generation unit, 124 3D roof surface model generation unit, 125 correction unit, 126 3D roof model generation unit, 127 output control unit

Claims

1. An acquisition unit that acquires a two-dimensional image showing the roof and a three-dimensional point cloud obtained by measuring the region containing the roof, A two-dimensional roof surface model generation unit generates a plurality of two-dimensional roof surface models, each having a planar shape, from the aforementioned two-dimensional image. A three-dimensional roof surface model generation unit generates multiple three-dimensional roof surface models by aligning each of the multiple two-dimensional roof surface models with respect to the three-dimensional point cloud, A correction unit that corrects the positions of at least two three-dimensional roof surface models based on the positional relationship of at least two of the plurality of three-dimensional roof surface models, The system includes a three-dimensional roof model generation unit that generates a three-dimensional roof model based on the corrected three-dimensional roof surface model, The correction unit corrects the position of the three-dimensional roof surface models by aligning the three-dimensional shared vertices of pairs of three-dimensional roof surface models that correspond to pairs of two-dimensional roof surface models that share a common vertex, and whose three-dimensional shared vertices corresponding to that common vertex are close to each other in three-dimensional space. A three-dimensional roof model generation device characterized by the following features.

2. An acquisition unit that acquires a two-dimensional image showing the roof and a three-dimensional point cloud obtained by measuring the region containing the roof, A two-dimensional roof surface model generation unit generates a plurality of two-dimensional roof surface models, each having a planar shape, from the aforementioned two-dimensional image. A three-dimensional roof surface model generation unit generates multiple three-dimensional roof surface models by aligning each of the multiple two-dimensional roof surface models with respect to the three-dimensional point cloud, A correction unit that corrects the positions of at least two three-dimensional roof surface models based on the positional relationship of at least two of the plurality of three-dimensional roof surface models, The system includes a three-dimensional roof model generation unit that generates a three-dimensional roof model based on the corrected three-dimensional roof surface model, The correction unit groups together three-dimensional roof surface models from among the plurality of two-dimensional roof surface models that share a common vertex and whose three-dimensional common vertex corresponding to that common vertex is close to each other in three-dimensional space, and corrects the position of each three-dimensional roof surface model belonging to each group based on the positional relationship of the three-dimensional roof surface models belonging to each group. A three-dimensional roof model generation device characterized by the following features.

3. The correction unit corrects the position of the three-dimensional roof surface models belonging to each group by performing regression analysis for each group based on an error item relating to the distance from the point cloud corresponding to the three-dimensional roof surface model belonging to each group from the three-dimensional point cloud, and an error item relating to the distance between the mutually corresponding three-dimensional shared vertices of the three-dimensional roof surface models belonging to each group, as described in claim 2.

4. The correction unit corrects the position of the three-dimensional roof surface model by performing regression analysis on a point cloud obtained by projecting a three-dimensional point cloud corresponding to a three-dimensional roof surface model whose inclination angle approximates a predetermined fixed angle onto a plane whose normal is a three-dimensional direction vector corresponding to the fixed angle, the three-dimensional roof surface model generation apparatus according to claim 2.

5. The three-dimensional roof model generation apparatus according to claim 3 or 4, wherein the correction unit uses a point cloud from the three-dimensional point cloud that is within a predetermined distance from the three-dimensional roof surface model as a three-dimensional point cloud corresponding to the three-dimensional roof surface model.

6. The three-dimensional roof surface model generation device according to claim 1, wherein the three-dimensional roof surface model generation unit generates the three-dimensional roof surface model by aligning a specific two-dimensional roof surface model with a point among the points included in the three-dimensional point cloud that is located in the horizontal direction within the region corresponding to the specific two-dimensional roof surface model and is located at a predetermined distance or more from a position corresponding to the boundary line of the specific two-dimensional roof surface model.

7. The three-dimensional roof surface model generation unit is, The three-dimensional roof surface is estimated from the two-dimensional roof surface model and the three-dimensional point cloud. A three-dimensional roof model generation apparatus according to claim 1, comprising setting a weight for each point in the three-dimensional point cloud according to the distance between each point in the three-dimensional point cloud and the estimated three-dimensional roof surface, and generating the three-dimensional roof surface model from the three-dimensional point cloud based on the set weights.

8. We obtain a 2D image showing the roof and a 3D point cloud obtained by measuring the region containing the roof. From the aforementioned two-dimensional image, multiple two-dimensional roof surface models, each having a planar shape, are generated. By aligning each of the aforementioned two-dimensional roof surface models with respect to the three-dimensional point cloud, multiple three-dimensional roof surface models are generated. Based on the positional relationship of at least two of the aforementioned three-dimensional roof surface models, the positions of the at least two three-dimensional roof surface models are corrected. The computer is instructed to generate a three-dimensional roof model based on the corrected three-dimensional roof surface model. In the correction described above, the position of the three-dimensional roof surface models is corrected by aligning the three-dimensional shared vertices of a pair of three-dimensional roof surface models that correspond to a pair of two-dimensional roof surface models that share a common vertex, and whose three-dimensional shared vertices corresponding to that common vertex are in close proximity to each other in three-dimensional space. A computer program characterized by the following features.

9. We obtain a 2D image showing the roof and a 3D point cloud obtained by measuring the region containing the roof. From the aforementioned two-dimensional image, multiple two-dimensional roof surface models, each having a planar shape, are generated. By aligning each of the aforementioned two-dimensional roof surface models with respect to the three-dimensional point cloud, multiple three-dimensional roof surface models are generated. Based on the positional relationship of at least two of the aforementioned three-dimensional roof surface models, the positions of the at least two three-dimensional roof surface models are corrected. The computer is instructed to generate a three-dimensional roof model based on the corrected three-dimensional roof surface model. In the correction described above, among the plurality of two-dimensional roof surface models, three-dimensional roof surface models that correspond to two-dimensional roof surface models that share a common vertex and whose three-dimensional common vertex corresponding to that common vertex is in close proximity to each other in three-dimensional space are grouped together, and for each group, the position of the three-dimensional roof surface model belonging to each group is corrected based on the positional relationship of the three-dimensional roof surface models belonging to each group. A computer program characterized by the following features.

Images