Existing Building Information Model Reconstruction Methods Based on Image Target Detection and UAVs

Abstract

This invention relates to a method for reconstructing existing building information models based on image target detection and unmanned aerial vehicles (UAVs), comprising the following steps: acquiring images of the four facades of a target building taken by a UAV; employing a deep learning object detection model (YOLO V9) to obtain a detection map containing window detection boxes and center point markers, window position information, and window size information; determining the correspondence between the pixel length captured in the image and the actual length; performing edge detection processing on the detection map, and combining the correspondence to obtain the boundary position information of the walls and the actual coordinates of the windows based on the walls; performing parametric modeling of the building envelope based on the boundary position information of the walls, while simultaneously acquiring the building's floor information and the floor plan layout of each floor, thus completing the reconstruction of the entire building information model. Compared with existing technologies, this invention can quickly and easily identify the position and size of the building envelope, thereby facilitating modeling, and has a certain degree of automation and low cost.

Description

Technical Field

[0001] This invention relates to the fields of civil engineering and computer vision interaction technology, and in particular to a method for reconstructing existing building information models based on image target detection and unmanned aerial vehicles (UAVs). Background Technology

[0002] With the widespread application of information technology in the civil engineering field, the entire lifecycle of a building, from design and construction to later operation and maintenance, relies on BIM models. Through BIM models, designers can gain a more intuitive understanding of the building's structure and function, enabling more precise design. Simultaneously, BIM models can provide construction workers with detailed construction guidance, helping them better understand the design intent and reducing misunderstandings and errors during construction.

[0003] However, some existing buildings constructed earlier lack design and construction drawings, causing many inconveniences for subsequent operation, maintenance, and renovation, especially in old residential area renovation projects. Current methods for reconstructing existing building information models (BIMs) heavily rely on architectural CAD drawings. For buildings lacking construction drawings, it is extremely difficult to quickly and easily model them; the dimensions and locations of many components need to be measured, which is time-consuming and labor-intensive. Therefore, a simple, efficient, and rapid automated modeling method is urgently needed for existing building BIM models. Summary of the Invention

[0004] The purpose of this invention is to overcome the problem of difficulty in reconstructing existing building information models without construction drawings in the prior art, and to provide a method for reconstructing existing building information models based on image target detection and UAVs. This method can quickly and easily identify the location and size of the building envelope, thereby facilitating modeling. It has a certain degree of automation and low cost.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for reconstructing existing building information models based on image target detection and unmanned aerial vehicles (UAVs) includes the following steps:

[0007] Images of the four facades of the target building captured by a drone are obtained. A trained deep learning object detection YOLO V9 model is used to perform object detection on the acquired images, resulting in a detection map containing window detection boxes and center point markers, window position information, and window size information.

[0008] Based on the window size information and the determined window type size, the correspondence between the pixel length captured in this shot and the actual length is determined;

[0009] Edge detection processing is performed on the detection image to obtain the boundary line of the wall, and the boundary coordinate points on the boundary line are picked up. Combined with the correspondence, the boundary position information of the wall and the actual coordinates of the window based on the wall surface are obtained.

[0010] Parametric modeling of the building envelope is performed based on the boundary location information of the wall. At the same time, the floor information and floor plan of each floor are obtained based on the actual coordinates of the windows on the wall, thus completing the reconstruction of the entire building information model.

[0011] Furthermore, multiple images were taken of the same facade.

[0012] Furthermore, when multiple images are taken of the same facade, target detection is performed on each image, and the detection images corresponding to each image are stitched together to obtain an overall facade detection image, and then the edge detection processing is performed.

[0013] Furthermore, the SIFT feature point detection algorithm is used to perform the image stitching.

[0014] Furthermore, the drone shots in all of the images were taken from the same angle.

[0015] Furthermore, during the training of the deep learning object detection YOLO V9 model, the building facade images collected by the UAV are combined with images from other publicly available building window object detection datasets to form a training dataset, and each image in the dataset is labeled and processed into YOLO annotation format.

[0016] Furthermore, the other publicly available building window target detection datasets are filtered and then combined with the building facade images collected by the UAV.

[0017] Furthermore, the correspondence is represented by a scaling factor under the current shooting state, which is expressed as follows:

[0018]

[0019] In the formula, s represents the scaling factor under the current shooting condition; D window This indicates the physical length or width of a window selected at a specific proportion; d window This represents the average pixel length of the detection box length or width that has been determined to belong to a window of this size on the image.

[0020] Furthermore, the process of obtaining the boundary position information of the wall and the actual coordinates of the window based on the wall surface includes:

[0021] The detection image is converted into a black and white binary image, making the boundaries of the wall and the boundaries of the nearby windows visible;

[0022] By querying, the coordinates of the four corner points of the wall can be obtained, and then the length and height of the wall can be obtained;

[0023] The coordinates of the upper left corner of the wall are set as the origin of the wall coordinates. Based on the correspondence, the actual position coordinates and dimensions of the window are calculated.

[0024] Furthermore, Dynamo is used to reconstruct the entire building information model.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. The present invention provides a method for reconstructing existing building information models. By collecting a few images, the positions of windows and walls can be quickly identified and recorded within minutes using a computer or other equipment. Then, Revit modeling can be performed within minutes using a pre-edited program. The method is highly automated and does not require manual step-by-step processing based on architectural drawings. The entire process is short and can easily achieve modeling, facilitating the maintenance and renovation of existing buildings.

[0027] 2. This invention can identify and record the position of each window on a wall using a single image, filling a gap in the field of architectural modeling for automatic identification of building envelope positions, and achieving a high degree of automation. By identifying the size and position of building windows, this invention can also, to some extent, detect quality issues such as window size deformation and changes in window position due to settlement.

[0028] 3. This invention does not have high requirements for equipment. In terms of data acquisition, it only requires the ability to take orthographic projections of building facades. Drones can be used, and in some scenarios, mobile phones can also be used for shooting. In terms of data processing and modeling, it does not have high requirements for computer equipment such as computers. Laptops can meet the requirements.

[0029] 4. The building information model reconstructed by this invention can carry the results of quality inspection and evaluation of existing building envelopes, providing a carrier for the information management of existing buildings. Attached Figure Description

[0030] Figure 1 This is a flowchart of the present invention;

[0031] Figure 2 This is a diagram showing the detection results of YOLO V9 in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of image stitching in an embodiment of the present invention;

[0033] Figure 4This is a schematic diagram of the boundary positioning of the wall in an embodiment of the present invention;

[0034] Figure 5 This is a flowchart of the Dynamo modeling process in an embodiment of the present invention. Detailed Implementation

[0035] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0036] like Figure 1 As shown, this embodiment provides a method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicles (UAVs), including the following steps:

[0037] S1: Images of the building's exterior facade were captured using a drone camera, with each image containing several windows. Images from other publicly available building window object detection datasets were added to the collected images to form a window object detection dataset. Each image in the dataset was labeled and processed into YOLO annotation format.

[0038] In this step, when creating the window object detection dataset, the images in the public dataset are filtered and re-labeled. The images are taken as unobstructed as possible, and the shooting angle is directly facing the building facade. Finally, the dataset needs to be processed into the YOLO annotation format.

[0039] S2: Build a deep learning object detection YOLO V9 model, train and tune the window object detection dataset in step S1, and save the trained YOLO V9 model and training weights.

[0040] S3: Use a drone to capture images of the four facades of the target building. To ensure the accuracy of the model's object detection, multiple images can be taken from the same facade for object detection, and then stitched together to ensure as few occlusions as possible. Based on the weights obtained during training, use the YOLO V9 model to perform object detection on the captured images, obtaining detection maps containing window bounding boxes and center point markers, as well as the position and size information of the windows, and storing this information in a text file.

[0041] refer to Figure 2 In this embodiment, step S3 yields an image containing the detection box and the center point of the window with specific pixel values, as well as an Excel spreadsheet containing the window size and coordinates.

[0042] S4: In the same image, since the shooting distance is usually tens or even hundreds of meters, the difference in actual size of window components in the central and surrounding areas of the image caused by perspective effects and lens distortion is not considered. The correspondence between pixel length and actual length can be obtained based on the dimensions of a specific window type.

[0043] Specifically, in step S4, the length and position of the pixels containing the window detection box will not be changed in subsequent image processing of the detection results. Therefore, it is necessary to determine the correspondence between the pixel length and the actual length under this shooting condition in this step. The ratio between the pixel length and the actual size can be determined by the average pixel length of a certain predetermined window height or width. The formula for calculating the scaling factor for each image is as follows:

[0044]

[0045] In the formula, s represents the scaling factor under the current shooting conditions; D window This indicates the physical length or width of a window selected at a specific proportion; d window This represents the average pixel length of the detection box length or width that has been determined to belong to a window of this size on the image.

[0046] S5: If multiple images are used for window detection on the same facade, these images need to be stitched together based on feature points to obtain the detection results for the building facade. If a single image covers the entire envelope of a facade, skip this step and proceed to the next step.

[0047] refer to Figure 3 In step S5, the image stitching uses the SIFT feature point detection algorithm. First, feature points are extracted from the reference image and the image to be stitched using the SIFT algorithm. Next, feature points in the two images are matched using Euclidean distance or other similarity metrics. Based on the matched feature point pairs, the optimal viewpoint transformation matrix between the images is estimated using algorithms such as Random Sample Consensus (RANSAC). Finally, the images are aligned and fused according to the transformation matrix to achieve seamless stitching.

[0048] It should be noted that the images to be stitched will inevitably be scaled during the image stitching process. However, this scaling will not affect the window size determined in step S4, but will only change the position of the stitched window. The position of the window will need to be re-determined in subsequent steps.

[0049] S6: Perform edge detection processing on the detected image to obtain the wall boundary line. Then, pick the pixels on the wall boundary to obtain the boundary coordinate points of the wall, and thus obtain the boundary position information of the wall. The position information of the window can be obtained by marking the center point of the window in the image.

[0050] refer to Figure 4 As shown, in step S6, after stitching together the detection images of the same facade, the coordinates of the window can be easily obtained and recorded based on the marking of specific pixel values ​​at the center point of the window. For the wall boundary, the position of the wall boundary needs to be determined by picking up pixels on the wall boundary. Finally, the actual coordinates of the window based on the wall surface need to be calculated, as follows:

[0051] Step S6.1: First, perform an edge contour detection process on the image. This will convert the image into a black and white binary image, which will clearly show the boundaries of the wall and the boundaries of the nearby windows.

[0052] Step S6.2: In order to accurately pick up the boundary outline of the wall, it is necessary to manually click the image containing the boundary of the wall to query the coordinates. By querying the coordinates of any point on each boundary, the position information of the boundary line of the wall can be obtained, as well as the coordinates of the four corner points, and then the length and height of the wall can be obtained.

[0053] Step S6.3: Based on the identification of the special pixel values ​​of the window center point on the stitched facade image, the window's coordinates based on the image can be obtained. The coordinates of the upper left corner of the wall are set as the origin of the wall's coordinates. The processed coordinates obtained by subtracting the upper left corner coordinates of the wall from the window's coordinates are the window's coordinates based on the wall. The obtained coordinates and the window dimensions obtained in Step S3 are recorded in the same Excel spreadsheet. Based on the scaling factor obtained in Step S4, i.e., the correspondence, the actual position coordinates and dimensions of the window can be calculated. The calculation formula is as follows:

[0054] W i =w i ×s

[0055] H i =h i ×s

[0056] X i =x i ×s+X wall

[0057] Y i =Y wall -y i ×s

[0058] In the formula: W i H i X i Y i Let represent the width, height, X coordinate, and Y coordinate of the i-th detected window in the image in planar coordinates of its actual size, respectively; s represents the scaling factor; and w represents the width, height, X coordinate, and Y coordinate of the i-th detected window in the image. i hi x i y i Let X represent the width (in pixels), height (in pixels), and X and Y coordinates (based on the wall) of the i-th detected window in the image. wall Y wall These represent the coordinates of the wall's origin in actual space, which need to be determined in conjunction with the actual spatial location of the building.

[0059] Considering that the window coordinates are in a three-dimensional coordinate system, the above calculation formula still applies. We only need to change the Y coordinate in the two-dimensional plane to the Z coordinate in the three-dimensional plane. Due to the different orientation of the facade, when the building facade is parallel to the XOZ plane, the X coordinate of the plane is the X coordinate in the three-dimensional plane; when the building facade is parallel to the YOZ plane, the X coordinate of the plane is the Y coordinate in the three-dimensional plane.

[0060] S7: Once you have the dimensions and location information of the building envelope, you can use Dynamo to perform parametric modeling of the building envelope.

[0061] refer to Figure 5 As shown, this demonstrates how to generate windows of different sizes and types in Revit based on Excel information at different locations using Dynamo. Taking a window on a single wall as an example, the specific method is as follows:

[0062] S7.1: First, import the family file of the window into Revit.

[0063] S7.2: Create a wall in Dynamo according to its actual location.

[0064] S7.3: Import the window coordinate column values ​​from the Excel spreadsheet to generate a two-dimensional list. Based on the list, create windows of the same family type at specific locations on the wall, and adjust these windows to different family types according to the window names in the Excel spreadsheet.

[0065] S7.4: Read the window size information from Excel and assign it to the family type of different window names so that each window with a different name has different size information.

[0066] By executing steps S7.3 and S7.4 in sequence, all windows on the wall can be generated.

[0067] S8: By using the location of windows and other semantic information about the building, we can roughly obtain the building's floor information and the floor plan of each floor, and then establish the building information model of the entire building.

[0068] Specifically, in step S8, walls and windows of other facades are generated, and the building model is improved based on other semantic information about the building, thereby realizing the reconstruction of the existing building information model.

[0069] In this embodiment, semantic information includes material information of the building's exterior walls, material type of windows, etc.

[0070] In a preferred embodiment, after obtaining the floor plan of each floor based on the location of the windows and the actual apartment layout, further refined modeling can be performed. Some building walls are not perfectly rectangular; some areas have depth, which can also be manually adjusted.

[0071] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] In another embodiment, an electronic device is also provided, including one or more processors, a memory, and one or more programs stored in the memory, the one or more programs including instructions for performing the methods described above.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for reconstructing existing building information models based on image target detection and unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Images of the four facades of the target building captured by a drone are obtained. A trained deep learning object detection YOLO V9 model is used to perform object detection on the acquired images, resulting in a detection map containing window detection boxes and center point markers, window position information, and window size information. Based on the window size information and the determined window type size, the correspondence between the pixel length captured in this shot and the actual length is determined; Edge detection processing is performed on the detection image to obtain the boundary line of the wall, and the boundary coordinate points on the boundary line are picked up. Combined with the correspondence, the boundary position information of the wall and the actual coordinates of the window based on the wall surface are obtained. Parametric modeling of the building envelope is performed based on the boundary position information of the wall. At the same time, the floor information and floor plan of each floor are obtained based on the actual coordinates of the windows on the wall, thus completing the reconstruction of the entire building information model. Multiple images were taken of the same exterior facade; When multiple images are taken of the same facade, target detection is performed on each image, and the detection images corresponding to each image are stitched together to obtain an overall facade detection image, and then the edge detection processing is performed. During the training of the deep learning object detection YOLO V9 model, images of building facades collected by drones are combined with images from other publicly available building window object detection datasets to form a training dataset, and each image in the dataset is labeled and processed into YOLO annotation format. The correspondence is represented by a scaling factor under the current shooting state, which is expressed as follows: In the formula, s represents the scaling factor under the current shooting conditions; This indicates the physical length or width of a window selected at a specific proportion; This represents the average pixel length of the detection box length or width that has been identified as belonging to a window of this size in the image. The process of obtaining the boundary position information of the wall and the actual coordinates of the window based on the wall surface includes: The detection image is converted into a black and white binary image, making the boundaries of the wall and the boundaries of the nearby windows visible; By querying, the coordinates of the four corner points of the wall can be obtained, and then the length and height of the wall can be obtained; Set the coordinate point of the upper left corner of the wall as the origin of the wall coordinate system, and calculate the actual position coordinates and dimensions of the window based on the aforementioned correspondence. The entire building information model was reconstructed using Dynamo.

2. The method for reconstructing existing building information models based on image target detection and unmanned aerial vehicles according to claim 1, characterized in that, The image stitching is achieved using the SIFT feature point detection algorithm.

3. The method for reconstructing existing building information models based on image target detection and UAVs according to claim 1, characterized in that, The drone shots in all of the images were taken from the same angle.

4. The method for reconstructing existing building information models based on image target detection and UAVs according to claim 1, characterized in that, The other publicly available building window target detection datasets were filtered and then combined with the building facade images collected by the UAV.

Images