Three-dimensional model generation apparatus, three-dimensional model generation method, and three-dimensional model generation program

Abstract

To easily generate a three-dimensional model in which a position of a structure provided on a wall surface is reproduced.SOLUTION: A three-dimensional model generation apparatus includes: frontalization means 102 which generates a frontal image by performing projective transformation so as to orthographically project a texture image on a predetermined wall surface of a building model on the basis of a relationship between the texture image included in the building model and coordinates of geometry; learning model generation means 112 which generates a learning model by learning combinations of the frontal images and frame body information, as training data; extraction means 115 which extracts a frame body from a newly generated frontal image using the learning model generated by the learning model generation means 112; and geometrization means 117 which calculates coordinates in the geometry of the frame body extracted by the extraction means 115 and synthesizes the frame body with the geometry on the basis of the calculated coordinates.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a three-dimensional model generating device, a three-dimensional model generating method, and a three-dimensional model generating program that generate a three-dimensional model to which information about wall structures such as windows and doors is added based on a texture image attached to a building model. [Background technology]

[0002] There are methods for generating 3D models that use 3D point cloud data generated using a laser scanner or aerial photographs.

[0003] When generating a 3D model of a building, images such as aerial photographs may be pasted onto the 3D model in order to reproduce the appearance of the actual building. The images pasted onto the 3D model are called texture images.

[0004] Patent Document 1 describes technology related to a wall information collection system in which, for each wall of a building, information regarding a wall image or a memory location on a server where the wall image is stored is associated and stored in order to attach images to the walls of a 3D digital model of the building.

[0005] Non-Patent Document 1 also discloses a technique relating to a method for automatically generating a 3D building model of LOD (Level of Detail) 2. Non-Patent Document 1 discloses a technique for cutting out an image to be pasted onto the 3D model of the corresponding building from an aerial photograph, and pasting it onto the generated 3D model as a texture image. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2008-198086 A [Non-patent literature]

[0007] [Non-Patent Document 1] Development of LOD2 automatic generation tool using AI etc. and OSS technology verification report, [online], [Retrieved October 27, 2023], PLATEAU Technical Report, Internet<URL:https: / / www.mlit.go.jp / plateau / file / libraries / doc / plateau_tech_doc_0056_ver01.pdf> Summary of the Invention [Problem to be solved by the invention]

[0008] However, in the techniques described in Patent Document 1 and Non-Patent Document 1, a texture image is pasted onto a 3D model of a building, but the wall structures such as windows and doors contained in this texture image do not have 3D coordinates as geometry.

[0009] If the 3D coordinates of wall structures such as windows and doors on the walls of the generated 3D model can be expressed, it could be used to consider the installation positions of solar panels on walls to achieve high power generation efficiency, and to simulate the spread of fire in buildings. For such purposes, there was a need to generate 3D models that included the position coordinates of wall structures such as windows and doors on the walls of buildings.

[0010] In addition, LOD (Level of Detail) 3 represents the three-dimensional shapes of wall structures such as windows and doors on the walls of buildings, but there is no established technology for automatic creation, making mass production difficult.

[0011] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a 3D model generation device, a 3D model generation method, and a 3D model generation program that easily generate a 3D model that represents the position of a wall structure installed on the wall of a building. [Means for solving the problem]

[0012] In order to achieve the above object, a first feature of the 3D model generating device according to the present invention is to A three-dimensional model generating device that generates a three-dimensional model to which wall structure information is added based on a texture image pasted on a building model, comprising: a projection transformation unit that performs orthogonal projection of the texture image onto a predetermined wall surface of the architectural model based on a relationship between the texture image and a geometry coordinate included in the acquired architectural three-dimensional model, thereby generating an orthogonal image; a learning model generating means for generating a learning model by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; an extraction means for extracting a frame from the orthodontic image newly generated by the orthodontic means, using the learning model generated by the learning model generation means; a geometry generating means for calculating coordinates of the frame body extracted by the extraction means in the geometry, and synthesizing the frame body with the geometry based on the calculated coordinates; The reason is that it is equipped with the following features.

[0013] A second feature of the 3D model generating device according to the present invention is The orthogonalizing means is The wall normal direction of the geometry expressed in the XYZ coordinate system is rotated so as to overlap with the Y axis, and the XZ direction of the geometry is aligned with the UV direction of the texture image expressed in the UV coordinate system. Then, the normalized image is generated by projecting the UV coordinates of the image onto the XZ coordinates of the geometry. It is characterized by:

[0014] A third feature of the 3D model generating device according to the present invention is The learning model generation means generating a plurality of learning models for object detection by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; The extraction means includes: Using a plurality of object detection methods, each using a learning model generated by the learning model generation means, a bounding box enclosing a frame body is extracted from the orientation image newly generated by the orientation means, and a frame body is extracted from the orientation image based on a reliability score indicating a reliability that the extracted bounding box corresponds to the frame body. The point is...

[0015] A fourth feature of the 3D model generating device according to the present invention is The learning model generation means generating a learning model for object detection by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; The extraction means includes: Using the learning model generated by the learning model generation means, a bounding box enclosing a frame body is extracted from the newly generated orthodontic image by the orthodontic means, and when the extracted bounding box overlaps with another bounding box, the overlapping bounding boxes are integrated based on a reliability score indicating the reliability that the bounding box is the frame body, thereby extracting the frame body from the orthodontic image.

[0016] A fifth feature of the 3D model generating device according to the present invention is The orthogonalizing means is Based on the relationship with the coordinates of the geometry included in the three-dimensional model of the building, if the walls of the three-dimensional model share sides with the same coordinates and the normals to each wall face in the same direction, the walls are integrated, and based on the relationship between the texture image and the coordinates of the geometry included in the three-dimensional model of the building, a normalized image is generated by performing a projection transformation so that the texture image is orthogonally projected onto a predetermined wall surface of the three-dimensional model into which the walls are integrated. The point is...

[0017] A sixth feature of the 3D model generating device according to the present invention is The geometry generation means includes: The region of the frame extracted by the extraction means is hollowed out from the wall surface of the geometry, and the remaining wall surface is formed by combining triangles. The point is...

[0018] A first feature of the 3D model generating method according to the present invention is to A three-dimensional model generating device that generates a three-dimensional model to which information about a wall structure is added based on a texture image pasted on a building model, a projection transformation step of orthogonally projecting the texture image onto a predetermined wall surface of the architectural model based on a relationship between the texture image and a geometry coordinate included in the acquired architectural three-dimensional model to generate an orthogonal image; a learning model generating step of generating a learning model by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; an extraction step of extracting a frame from the orientation image newly generated by the orientation step, using the learning model generated by the learning model generation step; a geometry generation step of calculating coordinates of the frame body in the geometry extracted by the extraction step, and synthesizing the frame body with the geometry based on the calculated coordinates; The purpose of the present invention is for a computer to execute the above.

[0019] A first feature of the three-dimensional model generation program according to the present invention is to A 3D model generation program executed by a computer that generates a 3D model to which information on wall structures such as windows and doors is added based on a texture image attached to an architectural model, a projection transformation step of orthogonally projecting the texture image onto a predetermined wall surface of the architectural model based on a relationship between the texture image and a geometry coordinate included in the acquired architectural three-dimensional model to generate an orthogonal image; a learning model generating step of generating a learning model by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; an extraction step of extracting a frame from the orientation image newly generated by the orientation step, using the learning model generated by the learning model generation step; a geometry generation step of calculating coordinates of the frame body in the geometry extracted by the extraction step, and synthesizing the frame body with the geometry based on the calculated coordinates; The objective of the present invention is to have a computer execute the above. Effect of the Invention

[0020] According to the three-dimensional model generating device, the three-dimensional model generating method, and the three-dimensional model generating program of the present invention, it is possible to easily generate a three-dimensional model that accurately reproduces the positions of wall structures provided on a wall surface. [Brief description of the drawings]

[0021] [Figure 1] 1 is a schematic diagram showing the schematic configuration of a three-dimensional model generating device according to an embodiment of the present invention; [Diagram 2] 1A is a diagram showing an example of a three-dimensional model of a building, FIG. 1B is a diagram showing an example of a texture image, and FIG. 1C is a diagram showing an example of a normalized image. [Diagram 3] FIG. 13 is a schematic diagram showing an example in which the same wall surface of a building model is composed of a plurality of surfaces. [Figure 4] FIG. 13 is a schematic diagram showing a normalized image and frame information. [Diagram 5]1 is an explanatory diagram illustrating ensemble processing by an extraction means included in a 3D model generating device according to an embodiment of the present invention. (a) is a diagram showing an example of a newly oriented image generated by the orientation means, (b) is a diagram showing an example of frame extraction when a learning model generated by Mask R-CNN is used, (c) is a diagram showing an example of frame extraction when a learning model generated by YOLOX-s is used, and (d) is a diagram showing an example of integrating frames by ensemble processing. [Figure 6] FIG. 6 is an enlarged view of a portion of FIG. 5(d), and is an explanatory diagram for explaining a model of overlap processing in the three-dimensional model generating device according to one embodiment of the present invention. [Figure 7] FIG. 2 is an explanatory diagram for explaining a model of geometry processing by a geometry generating means of the three-dimensional model generating device according to one embodiment of the present invention; [Figure 8] FIG. 2 is an explanatory diagram illustrating the contents of boundary processing in an extraction means of the three-dimensional model generating device according to one embodiment of the present invention. [Figure 9] FIG. 2 is an explanatory diagram illustrating the contents of boundary processing in a geometry generating means of the three-dimensional model generating device according to one embodiment of the present invention. [Figure 10] 4 is a flowchart showing the processing contents in a three-dimensional model generating device according to an embodiment of the present invention. [Figure 11] 4 is a flowchart showing the processing contents in a three-dimensional model generating device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The same or equivalent parts and components are designated by the same or equivalent reference numerals throughout the drawings. However, it should be noted that the drawings are schematic and may differ from the actual product. In addition, the drawings may include parts with different dimensional relationships and ratios.

[0023] The embodiments described below are merely examples of devices for embodying the technical idea of ​​the present invention, and the technical idea of ​​the present invention does not limit the arrangement of each component to that described below. Various modifications can be made to the technical idea of ​​the present invention within the scope of the claims.

[0024] A three-dimensional model generating device according to one embodiment of the present invention will be described below.

[0025] FIG. 1 is a schematic diagram showing the schematic configuration of a three-dimensional model generating device according to an embodiment of the present invention.

[0026] As shown in FIG. 1, the 3D model generating device 1 includes a building 3D model acquisition means 101, an orientation means 102, a log storage means 103, an orientation image storage means 104, a frame information acquisition means 111, a learning model generation means 112, a learning model storage means 113, an extraction means 115, a geometry means 117, and a 3D model data storage means 118.

[0027] The building 3D model acquisition means 101 acquires a building 3D model. Here, the building 3D model is targeted for LOD2 to which a texture image is attached.

[0028] Three-dimensional point cloud data is, for example, a collection of points with coordinate data calculated based on the reflected light when a laser scanner or the like is mounted on an aircraft such as an airplane or drone and laser light is emitted to the ground. Note that, here, the three-dimensional point cloud data is measurement data obtained by an airplane or drone as described above, but is not limited to measurement data obtained by irradiation from an airplane or drone, and may be data that includes at least three-dimensional coordinate data.

[0029] In recent years, a method for automatically generating a 3D building model of LOD (Level of Detail) 2 has been disclosed. The 3D building model acquired by the building 3D model acquisition means 101 is a building model of LOD 2. LOD 2 has information on the faces, lines, and vertices that constitute the building's external shape.

[0030] Similarly, the texture images attached to the building models are also taken from airplanes, drones, car cameras, handheld cameras, etc.

[0031] The orthogonalization means 102 generates an orthogonalized image by performing projective transformation so as to orthogonally project the texture image onto a specified wall surface of the architectural model, based on the relationship between the texture image and the coordinates of the geometry contained in the architectural 3D model.

[0032] FIG. 2(a) is a diagram showing an example of a 3D model of a building, FIG. 3(b) is a diagram showing an example of a texture image, and FIG. 2(c) is a diagram showing an example of a normalized image.

[0033] As shown in Fig. 2(a), the 3D building model G101 is an LOD2 level building model G101a generated from 3D point cloud data, and texture images G101b, G101c, etc., which are wall images, are attached to the walls of the building. Here, two surfaces are shown, but texture images are attached to the roof surface and four wall surfaces on the front, back, left and right.

[0034] The texture image itself may not match the shape of the surface to which it is pasted, due to the influence of the surface to which it is pasted and the shooting angle when the photograph is taken. As an example, as shown in Figure 2(b) G102, it has a trapezoid shape with the top side longer than the bottom side, resulting in a deformed image. In addition, the texture image does not have position information for frames installed inside walls such as windows, so the position of the window surface is not accurate.

[0035] Therefore, the orthogonalization means 102 generates an orthogonalized image by performing projective transformation so as to orthogonally project the texture image onto a specified wall surface of the architectural model, based on the relationship between the texture image and the coordinates of the geometry.

[0036] Specifically, the orientation correction means 102 orients the object so that the normal to the geometry in the XYZ coordinate system is on the YZ plane. Then, the orientation correction means 102 rotates the object so that the normal to the geometry in the XYZ coordinate system overlaps with the Y axis.

[0037] Then, the orientation correction means 102 aligns the XZ direction of the geometry with the UV direction of the texture image in the UV coordinate system, and then generates an oriented image by performing projective transformation from the UV plane of the texture image to the XZ plane of the geometry.

[0038] As shown in Figure 2(c), the orthogonal image G103 obtained by the orthogonalization means 102 represents one side of the building when the building model is viewed from the front, so it can be said that the position and size of the frame body provided in the wall surface such as a window surface included in the orthogonal image G103 are correctly positioned relative to the geometric shape of the building.

[0039] In this case, when the same wall surface of the building model is composed of multiple surfaces, the orthogonal image may be divided.

[0040] FIG. 3 is a schematic diagram showing an example in which the same wall surface of a building model is made up of a plurality of surfaces.

[0041] As shown in Fig. 3, the wall surface G201 and the wall surface G202 have a triangular shape, and the faces share a side L101 with the same coordinates. That is, the wall surface G201 and the wall surface G202 both share a side L101 with vertices P101 and P102 as vertices. The normals of the wall surface G201 and the wall surface G202 are oriented in the same direction.

[0042] In this case, the orientation means 102 integrates the wall surface G201 and the wall surface G202 into a single wall surface.

[0043] Similarly, wall surface G203 and wall surface G204 have a triangular shape, the faces share a side with the same coordinates, and their respective normals point in the same direction, so the orientation correction means 102 integrates wall surface G203 and wall surface G204 as the same wall surface.

[0044] However, since the wall surface G201 and the wall surface G203 do not share a side that has the same coordinates, the orientation correction means 102 does not integrate the wall surface G201 and the wall surface G203 as the same wall surface.

[0045] Incidentally, a threshold value may be set so as to allow a predetermined width to determine whether the normal directions are the same or not.

[0046] In this way, the orthogonalization means 102 integrates wall surfaces when the surfaces share a side with the same coordinates and their normals point in the same direction. The orthogonalization means 102 generates an orthogonalized image by performing projective transformation so as to orthogonally project the texture image onto a predetermined wall surface of the integrated building model. In addition, the orthogonalization means 102 generates an orthogonalization log when generating the orthogonalized image.

[0047] The log storage means 103 is configured with a storage medium such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores the orientation log generated by the orientation means 102. The orientation log includes the file name of the 3D building model which is the target data, the wall name indicating which wall of the building model it is, a coordinate string indicating the correspondence between the UV coordinates of the texture image and the XYZ coordinates of the geometry, the normal vector of the wall of the geometry, the input and output arrangement paths, and the like.

[0048] The orthodontic image storage means 104 stores the orthodontic image obtained by the orthodontic means 102 performing projective transformation.

[0049] The frame information acquiring means 111 acquires frame information indicating a frame included in the normal image from an input device (not shown) based on a user input.

[0050] FIG. 4 is a schematic diagram showing a normalized image and frame information.

[0051] As shown in FIG. 4, the orthogonal image G301 includes a plurality of window surfaces, doors, and the like, which are frames.

[0052] While viewing the orthogonal image G301 displayed on a display device (not shown), the user specifies the position and size of the window surface using an input device such as a mouse so as to surround the window surface with a rectangle.

[0053] For example, when a user uses an input device to draw a frame G311 in a rectangular shape surrounding a window surface, the frame information acquisition means 111 may specify the position and size of the frame G311 using the XYZ coordinates of vertices P301 to P304, which are the vertices of the frame G311, or may specify the position and size of the frame G311 using the XYZ coordinates of vertex P301, which is the vertex of the frame G311, and the direction in the XYZ coordinate system of the frame G311 and the lengths L301 and L302 of the vertical and horizontal sides.

[0054] The learning model generating means 112 generates a learning model by learning using AI (Artificial Intelligence) with a combination of the orthogonal image and the frame information acquired by the frame information acquiring means 111 as teacher data.

[0055] Furthermore, the learning model generating means 112 may generate multiple learning models using multiple object detection methods, for example, to execute ensemble processing. Specifically, the learning model generating means 112 generates multiple learning models by learning, as teacher data, a normalized image and frame information indicating a frame in the normalized image, using multiple object detection methods, such as Mask R-CNN and YOLOX-s.

[0056] The learning model storage means 113 stores the learning model generated by the learning model generation means 112. As described above, when a plurality of learning models are generated by the learning model generation means 112, each of the generated learning models is stored.

[0057] The extraction means 115 uses the learning model generated by the learning model generation means 112 and stored in the learning model storage means 113 to extract a frame corresponding to the wall structure from the orientation image newly generated by the orientation means 102. Here, the wall structure refers to a structure provided on the wall surface of a building, such as a door or window surface.

[0058] Furthermore, when a plurality of learning models are generated by the learning model generating means 112, the extracting means 115 may be configured to execute an ensemble process.

[0059] Fig. 5 is an explanatory diagram explaining the ensemble processing by the extraction means 115. Fig. 5(a) is a diagram showing an example of a newly oriented image generated by the orientation means 102, Fig. 5(b) is an example of a frame body extraction when a learning model generated by Mask R-CNN is used, Fig. 5(c) is an example of a frame body extraction when a learning model generated by YOLOX-s is used, and Fig. 5(d) is an example of a frame body integration by ensemble processing.

[0060] The extraction means 115 uses, for example, the oriented image shown in FIG. 5(a) as an input image, and extracts a bounding box surrounding the frame from the oriented image using a learning model generated by Mask R-CNN.

[0061] In the example shown in FIG. 5(b), the extraction means 115 extracts bounding boxes G401 to G404 that enclose the frame body, etc., using a learning model generated by Mask R-CNN.

[0062] In this case, when the extraction means 115 extracts the bounding boxes G401 to G404 using the learning model generated by Mask R-CNN, it outputs the reliability score corresponding to each of the bounding boxes G401 to G404. The reliability score indicates the reliability of the frame.

[0063] Here, the confidence score corresponding to the bounding box G401 is output as "0.80", the confidence score corresponding to the bounding box G402 is output as "0.60", the confidence score corresponding to the bounding box G403 is output as "0.85", and the confidence score corresponding to the bounding box G404 is output as "0.90". Note that the confidence scores shown in the figure are displayed as values ​​multiplied by 100.

[0064] In addition, the extraction means 115 extracts a bounding box surrounding the frame from the orthodontic image, for example, using the orthodontic image shown in FIG. 5(a) as an input image and using a learning model generated by YOLOX-s.

[0065] In the example shown in FIG. 5(c), the extraction means 115 uses the learning model generated by YOLOX-s to extract a bounding box G411 that surrounds the frame body, and outputs a reliability score corresponding to the bounding box G411, etc.

[0066] Here, the confidence score corresponding to the bounding box G411 is output as "0.95".

[0067] Therefore, the extraction means 115 extracts a frame from the normalized image based on the reliability score corresponding to the extracted bounding box.

[0068] The bounding box G411 having the higher reliability score among the reliability scores corresponding to the bounding boxes G401, G402 extracted using the learning model generated by Mask R-CNN, and the reliability score corresponding to the bounding box G411 extracted using the learning model generated by YOLOX-s, is extracted as the frame.

[0069] As a result, a bounding box G411 is extracted as a frame, as shown in FIG.

[0070] Similarly, the confidence score corresponding to the bounding box G403 extracted using the learning model generated by Mask R-CNN is output as "0.85", and the confidence score corresponding to the bounding box G404 is output as "0.90". For the corresponding frame, no bounding box is extracted in the learning model generated by YOLOX-s.

[0071] Therefore, as shown in FIG. 5(d), bounding boxes G403 and G404 extracted using a learning model generated by Mask R-CNN are extracted as frames.

[0072] In addition, when window surfaces that are frame bodies detected by object detection positionally overlap each other, such as bounding boxes G403 and G404, overlap processing may be performed by keeping those with high reliability and eliminating those with low reliability.

[0073] FIG. 6 is an enlarged view of a part of FIG. 5(d), and is an explanatory diagram for explaining a model of the overlap processing in the three-dimensional model generating device 1 according to one embodiment of the present invention.

[0074] As shown in Fig. 6(a), the extraction means 115 extracts bounding boxes G403 and G404 for one frame using a learning model generated by Mask R-CNN. At this time, it is assumed that the bounding boxes G403 and G404 overlap with each other with a certain IoU (Intersection over Union) or more.

[0075] Therefore, the extraction means 115 executes overlap processing to keep those with high reliability scores and eliminate those with low reliability scores. Specifically, the reliability score corresponding to the bounding box G403 extracted using the learning model generated by Mask R-CNN is output as "0.85", and the reliability score corresponding to the bounding box G404 is output as "0.90", so the extraction means 115 extracts the bounding box G404 with the highest reliability score from the bounding box G403 and the bounding box G404 as the frame.

[0076] That is, when the extracted bounding box overlaps with other bounding boxes with a certain IoU (Intersection over Union) or more, the extraction means 115 extracts the bounding box with the highest confidence score as the frame. Note that although Non-Maximum Suppression (NMS) is given as an example of the ensemble method here, Non-Maximum Weighted (NMW) or Weighted Boxes Fusion (WBF) may also be used.

[0077] The geometry generating means 117 calculates the coordinates in the geometry of the frame body extracted by the extraction means 115, and synthesizes the frame body with the geometry based on the calculated coordinates, thereby generating a three-dimensional model.

[0078] FIG. 7 is an explanatory diagram for explaining a model of geometry processing by the geometry generating means 117 of the three-dimensional model generating device 1 according to one embodiment of the present invention.

[0079] The geometry creation means 117 reads the orientation log stored in the log storage means 103, and outputs a frame body as shown in FIG. 7(a) as a polygon based on the coordinates of the vertices in the XYZ coordinate system of the frame body extracted by the extraction means 115 and included in the orientation log.

[0080] In the example shown in Fig. 7(a), frame bodies G601, G602, etc. are output as polygons. Frame body G601 is located on the front wall surface of the geometry, and frame body G601 is located on the side wall surface of the geometry.

[0081] The geometry generation means 117 generates a 3D model in which positional information is added to the frame bodies by combining the frame bodies G601, G602 output as polygons with geometry G603, which is a building model, as shown in Figure 7(b).

[0082] The 3D model data storage means 118 is configured with a storage medium such as a hard disk drive (HDD) or a solid state drive (SSD), and stores the 3D model generated by the geometry generation means 117.

[0083] In this way, the 3D model generating device 1 according to one embodiment of the present invention can generate a 3D model with added position information of windows, doors, etc., making it possible to grasp the position coordinates and size information of wall structures such as windows and doors installed on a wall. This is thought to be useful, for example, in considering the installation position of solar panels on a wall to achieve high power generation efficiency, and in simulating the spread of fire in a building.

[0084] Here, although there are cases where the frame extracted by the extraction means 115 approaches the boundary of the wall surface, in an actual building, it is rare for frames such as window frames and doors to be provided near the boundary of the wall surface.

[0085] Therefore, in the 3D model generating device 1, the geometry generating means 117 may execute boundary processing. In the boundary processing, when the distance between the frame extracted by the extraction means 115 and the boundary of the wall surface on which the frame exists in the geometry is less than a predetermined threshold value LTh, the geometry generating means 117 moves the frame so that the distance becomes the threshold value LTh.

[0086] FIG. 8 is an explanatory diagram illustrating the contents of boundary processing in the extraction means 115 of the three-dimensional model generating device 1 according to one embodiment of the present invention.

[0087] As shown in FIG. 8(a), a frame G711 extracted by the extraction means 115 may be close to a boundary G701 of a wall surface.

[0088] Therefore, when the distance between the frame body G711 extracted by the extraction means 115 and the boundary of the wall surface G701 in which the frame body G711 exists in the geometry is less than a predetermined threshold value LTh, the geometry generation means 117 generates the wall surface G702 by moving the frame body G711 so that the distance becomes the threshold value LTh, as shown in Figure 8 (b).

[0089] This makes it possible to generate a natural 3D model in which window frames, doors, and other frames are not provided near the boundaries of the wall surface.

[0090] In addition, some software may not be able to correctly read concave polygons, which are figures in which one or more of the interior angles is greater than 180 degrees.

[0091] Therefore, the geometry generating means 117 of the 3D model generating device 1 according to one embodiment of the present invention may form the wall surfaces by combining triangles in order to eliminate concave polygons. Furthermore, the region of the frame extracted by the extraction means 115 may be hollowed out from the wall surfaces in the geometry, and the remaining wall surfaces may be formed by combining triangles.

[0092] FIG. 9 is an explanatory diagram illustrating the contents of boundary processing in the geometry generation means 117 of the three-dimensional model generating device 1 according to one embodiment of the present invention.

[0093] Fig. 9(a) shows an example of a concave polygonal wall surface that does not have a frame such as a window surface, a door, etc. In this case, for example, as shown in Fig. 9(b), by dividing the concave polygonal wall surface shown in Fig. 9(a) by internal boundary lines L801 to L803 and configuring the wall surface as a combination of triangles, it is possible to generate a 3D model having a wall surface that excludes concave polygons.

[0094] 9(c), when a frame body G801 extracted by the extraction means 115 is provided on a wall surface, the geometry generating means 117 cuts out the area of ​​the frame body G801 extracted by the extraction means 115 from the wall surface. Then, the geometry generating means 117 divides the wall surface remaining after the area of ​​the frame body G801 is cut out along the internal boundary lines L811-L819 to configure the wall surface as a triangular combination, thereby making it possible to generate a 3D model having a wall surface excluding concave polygons even when the frame body G801 is provided on the wall surface.

[0095] 10 and 11 are flow charts showing the process contents in the three-dimensional model generating device 1 according to one embodiment of the present invention.

[0096] As shown in FIG. 10, in step S101, the building three-dimensional model acquisition means 101 acquires a building three-dimensional model.

[0097] In step S103, the orientation correction means 102 detects walls from the acquired three-dimensional building model.

[0098] In step S105, the orientation correction means 102 integrates wall surfaces that share a side with the same coordinates and whose normals face in the same direction.

[0099] In step S107, the orthogonalization means 102 generates an orthogonalized image by performing projective transformation so as to orthogonally project the texture image onto a predetermined wall surface of the architectural model, based on the relationship between the texture image and the coordinates of the geometry.

[0100] In step S109, the orientation means 102 generates an orientation log when generating the orientation image.

[0101] In step S121, the extraction means 115 extracts a frame from the orientation-corrected image newly generated by the orientation correction means 102, using the learning model generated by the learning model generation means 112 and stored in the learning model storage means 113.

[0102] In step S123, when a plurality of learning models have been generated by the learning model generating means 112, the extracting means 115 executes an ensemble process.

[0103] In step S125, the extraction means 115 performs overlap processing, leaving those with high reliability and eliminating those with low reliability.

[0104] In step S127, the extraction means 115 outputs a log file including a log relating to the extracted frame body.

[0105] In step S141, the geometry generation means 117 executes boundary processing.

[0106] In step S143, the geometry generating means 117 calculates the coordinates in the geometry of the frame body extracted by the extraction means 115, and synthesizes the frame body with the geometry based on the calculated coordinates.

[0107] In step S145, the geometry generating means 117 cuts out the area of ​​the frame G801 extracted by the extraction means 115 from the wall surface.

[0108] In step S147, the geometry generating means 117 generates a three-dimensional model.

[0109] In step S149, the geometry generating means 117 outputs a log file including a log relating to the three generated dimensional models.

[0110] As described above, the 3D model generating device 1 according to one embodiment of the present invention can generate a 3D model in which position information is added to the frame, so that the position information and size information of the frame of a window, door, etc. provided on the wall of the generated 3D model can be accurately grasped. This is considered to be useful, for example, in examining the installation position of a solar panel on a wall to achieve high power generation efficiency, or in simulating the spread of a fire in a building.

[0111] The above-described embodiment can also be realized by executing a program installed on a computer. [Explanation of symbols]

[0112] 1. 3D model generation device 101 Means of acquiring 3D building models 102 Means of Orthogonalization 103 Log storage means 104 Oriented image storage means 111 Frame information acquisition means 112 Learning model generation means 113 Learning model storage means 115 Extraction means 117 Geometry Method 118 3D model data storage means

Claims

1. A three-dimensional model generating device that generates a three-dimensional model to which wall structure information is added based on a texture image pasted on a building model, comprising: a projection transformation unit that performs orthogonal projection of the texture image onto a predetermined wall surface of the architectural model based on a relationship between the texture image and a geometry coordinate included in the acquired architectural three-dimensional model, thereby generating an orthogonal image; a learning model generating means for generating a learning model by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; an extraction means for extracting a frame from the orthodontic image newly generated by the orthodontic means, using the learning model generated by the learning model generation means; a geometry generating means for calculating coordinates of the frame body extracted by the extraction means in the geometry, and synthesizing the frame body with the geometry based on the calculated coordinates; A three-dimensional model generating device comprising:

2. The orthogonalizing means is The wall normal direction of the geometry expressed in the XYZ coordinate system is rotated so as to overlap with the Y axis, and the XZ direction of the geometry is aligned with the UV direction of the texture image expressed in the UV coordinate system, and then the normalized image is generated by projecting the image from the UV coordinates to the XZ coordinates of the geometry.

2. The three-dimensional model generating device according to claim 1.

3. The learning model generation means generating a learning model for detecting a plurality of objects by using a plurality of object detection methods and learning a combination of the normalized image and frame information indicating a frame included in the normalized image as training data; The extraction means includes: Using the multiple learning models generated by the learning model generation means, a bounding box surrounding a frame body is extracted from the orientation image newly generated by the orientation means, and a frame body is extracted from the orientation image based on a reliability score indicating a reliability that the extracted bounding box corresponds to the frame body.

2. The three-dimensional model generating device according to claim 1.

4. The learning model generation means generating a learning model for object detection by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; The extraction means includes: Using the learning model generated by the learning model generation means, a bounding box enclosing a frame body is extracted from the newly generated oriented image by the oriented image generation means, and when the extracted bounding box overlaps with another bounding box, the overlapping bounding boxes are integrated based on a reliability score indicating a reliability that the bounding box is the frame body, thereby extracting the frame body from the oriented image.

2. The three-dimensional model generating device according to claim 1.

5. The orthogonalizing means is Based on the relationship with the coordinates of the geometry included in the three-dimensional model of the building, if the wall surfaces of the three-dimensional model share a side with the same coordinates and the normals to each wall surface are oriented in the same direction, the wall surfaces are integrated, and based on the relationship between the texture image and the coordinates of the geometry included in the three-dimensional model of the building, the texture image is orthogonally projected onto a predetermined wall surface of the three-dimensional model into which the wall surfaces are integrated, thereby generating an orthogonalized image.

2. The three-dimensional model generating device according to claim 1.

6. The geometry generation means includes: The region of the frame extracted by the extraction means is hollowed out from the wall surface of the geometry, and the remaining wall surface is formed by combining triangles.

2. The three-dimensional model generating device according to claim 1.

7. A three-dimensional model generating device that generates a three-dimensional model to which information about a wall structure is added based on a texture image pasted on a building model, a projection transformation step of orthogonally projecting the texture image onto a predetermined wall surface of the architectural model based on a relationship between the texture image and a geometry coordinate included in the acquired architectural three-dimensional model to generate an orthogonal image; a learning model generating step of generating a learning model by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; an extraction step of extracting a frame from the orientation image newly generated by the orientation step, using the learning model generated by the learning model generation step; a geometry generation step of calculating coordinates of the frame body in the geometry extracted by the extraction step, and synthesizing the frame body with the geometry based on the calculated coordinates; A three-dimensional model generating method characterized in that the above steps are executed by a computer.

8. A three-dimensional model generation program executed by a computer that generates a three-dimensional model to which information on wall structures such as windows and doors is added based on a texture image attached to an architectural model, a projection transformation step of orthogonally projecting the texture image onto a predetermined wall surface of the architectural model based on a relationship between the texture image and a geometry coordinate included in the acquired architectural three-dimensional model to generate an orthogonal image; a learning model generating step of generating a learning model by learning a combination of the orthodontic image and frame information indicating a frame included in the orthodontic image as training data; an extraction step of extracting a frame from the orientation image newly generated by the orientation step, using the learning model generated by the learning model generation step; a geometry generation step of calculating coordinates of the frame body in the geometry extracted by the extraction step, and synthesizing the frame body with the geometry based on the calculated coordinates; A three-dimensional model generating program that causes a computer to execute the above steps.

Images