Existing building information model reconstruction method based on image target detection and unmanned aerial vehicle

Abstract

The invention relates to an existing building information model reconstruction method based on image target detection and an unmanned aerial vehicle, and the method comprises the following steps: obtaining pictures, shot by the unmanned aerial vehicle, of four external vertical faces of a target building, and employing a deep learning target detection YOLO V9 model, obtaining a detection graph containing a window detection frame and a central point mark, window position information and window size information; determining the corresponding relation between the pixel length of the shooting and the actual length; performing edge detection processing on the detection image, and combining the corresponding relation to obtain boundary position information of the wall and actual coordinates of the window based on the wall surface; and performing parametric modeling of the building envelope based on the boundary position information of the wall, and simultaneously obtaining floor information of the building and the plane layout of each floor to complete reconstruction of the whole building information model. Compared with the prior art, the position and the size of the building envelope can be quickly, simply and conveniently recognized, modeling is further facilitated, a certain automation degree is achieved, and cost is low.

Description

Technical Field

[0001] The present invention relates to the technical field of civil engineering and computer vision interaction, and in particular to a method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle. Background Art

[0002] With the extensive application of information technology in the civil engineering field, the entire life cycle of buildings, from design and construction to later operation and maintenance, relies on the BIM model of the building. Through the BIM model, designers can understand the structure and function of the building more intuitively and make more accurate designs. At the same time, the BIM model can also provide detailed construction guidance for construction personnel, helping them better understand the design intent and reduce misunderstandings and errors during the construction process.

[0003] However, some existing buildings built earlier lack design and construction drawings, which brings many inconveniences to the subsequent operation and maintenance management and renovation, especially in the renovation projects of old communities. The current reconstruction method of existing building information models relies heavily on the design CAD drawings of the building. It is almost difficult to quickly and easily model buildings that lack construction drawings. The size and position of many components need to be measured, which is time-consuming and labor-intensive. Therefore, a simple, efficient and fast automated modeling method is urgently needed for the BIM model of existing buildings. Summary of the invention

[0004] The purpose of the present invention is to overcome the problem of difficulty in reconstructing existing building information models in the absence of construction drawings in the above-mentioned prior art, and to provide a method for reconstructing existing building information models based on image target detection and drones, which can quickly and easily identify the position and size of the building envelope, thereby facilitating modeling, has a certain degree of automation, and is low in cost.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle comprises the following steps:

[0007] Obtain images of the four facades of the target building taken by a drone, use the trained deep learning target detection YOLO V9 model to perform target detection on the acquired images, and obtain a detection map containing window detection frames and center point marks, window position information, and window size information;

[0008] Based on the window size information and the determined window type size, determining a correspondence between the pixel length of this shooting and the actual length;

[0009] Performing edge detection processing on the detection image to obtain the boundary line of the wall, and picking up boundary coordinate points on the boundary line, and combining the corresponding relationship to obtain boundary position information of the wall and actual coordinates of the window based on the wall surface;

[0010] Based on the boundary position information of the wall, parametric modeling of the building envelope structure is performed, and at the same time, based on the actual coordinates of the window based on the wall, the floor information of the building and the plane layout of each floor are obtained to complete the reconstruction of the entire building information model.

[0011] Furthermore, a plurality of pictures are taken corresponding to the same facade.

[0012] Furthermore, when multiple pictures are taken corresponding to the same facade, after target detection is performed on each picture, the detection images corresponding to each picture are stitched to obtain an overall detection image of the facade, and then the edge detection process is performed.

[0013] Furthermore, the image stitching is achieved by using a SIFT feature point detection algorithm.

[0014] Furthermore, the drone shooting angles of the multiple pictures are completely consistent.

[0015] Furthermore, when training the deep learning target detection YOLO V9 model, the building facade images collected by the drone are combined with images in other public building window target detection data sets to form a training data set, and each image in the data set is annotated and processed into the YOLO annotation format.

[0016] Furthermore, the other public building window target detection datasets are screened and combined with the building facade images collected by the drone.

[0017] Furthermore, the corresponding relationship is represented by a proportional conversion factor under the current shooting state, and the proportional conversion factor under the current shooting state is expressed as:

[0018]

[0019] Where, s represents the proportional conversion factor under the current shooting state; D window Indicates the physical length or width of the window with a certain ratio; d window It indicates the average pixel length of the length or width of the detection box that belongs to the window of this size determined on the image.

[0020] Furthermore, the process of obtaining the boundary position information of the wall and the actual coordinates of the window based on the wall surface includes:

[0021] Convert the detection image into a black and white binary image to make the boundaries of the wall and nearby windows visible;

[0022] By querying the coordinates of the four corner points of the wall, we can get the length and height of the wall.

[0023] The coordinate point at the upper left corner of the wall is set as the coordinate origin of the wall, and the actual position coordinates and size of the window are calculated based on the corresponding relationship.

[0024] Furthermore, Dynamo is used to achieve reconstruction of the entire building information model.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. The method of the present invention provides a method for reconstructing an existing building information model. By collecting several pictures, the positions of windows and walls can be quickly identified and recorded through computers and other equipment within a few minutes. Revit modeling can be performed within a few minutes using an edited program. The method has a high degree of automation and does not require manual refinement step by step through architectural drawings. The whole process takes a short time and can easily realize modeling, which facilitates the maintenance and renovation of existing buildings.

[0027] 2. The present invention can identify and record the position of each window on the wall through a picture, which fills the gap in the automatic identification of the position of the enclosure structure in the field of building modeling to a certain extent, and has a high degree of automation. By identifying the size and position of the building windows, the present invention can also check the quality problems such as the size deformation and the change of the window position caused by the settlement of the window to a certain extent.

[0028] 3. The present invention has low requirements on equipment. In terms of data collection, it only needs to be able to take orthographic projection images of building facades. UAVs can be used, and mobile phones can also be used for shooting in some scenarios. In terms of data processing and modeling, it does not have high requirements on computer equipment such as computers, and laptops can meet the requirements.

[0029] 4. The building information model reconstructed by the present invention can carry the results of quality inspection and evaluation of existing building envelope structures, providing a carrier for information management of existing buildings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the present invention;

[0031] Figure 2 This is a YOLO V9 detection result diagram in an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of image stitching in an embodiment of the present invention;

[0033] Figure 4This is a schematic diagram of the boundary positioning of a wall in an embodiment of the present invention;

[0034] Figure 5 This is a Dynamo modeling flow chart in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0036] like Figure 1 As shown, this embodiment provides a method for reconstructing an existing building information model based on image target detection and drone, comprising the following steps:

[0037] S1: Use drone cameras to take pictures of the four sides of the building facade, each picture contains several windows. Add pictures from other public building window target detection datasets to the collected pictures to form a window target detection dataset. Annotate each picture in the dataset and process it into the YOLO annotation format.

[0038] In this step, when creating a window target detection dataset, the images in the public dataset are screened and re-labeled. The pictures taken are as unobstructed as possible and the shooting angle is facing the building facade. Finally, the dataset needs to be processed into the YOLO labeling format.

[0039] S2: Establish a deep learning target detection YOLO V9 model, train and adjust the parameters of the window target detection dataset in step S1, and save the trained YOLO V9 model and training weights.

[0040] S3: Use a drone to take pictures of the four facades of the target building. In order to ensure the accuracy of the model's target detection, multiple pictures can be taken on the same facade for target detection, and then stitched together to ensure that the pictures are as unobstructed as possible. According to the weights obtained through training, the YOLO V9 model is used to perform target detection on the captured pictures, and a detection map containing the window detection frame and center point mark as well as the window position and size information is obtained, and this information is stored in a text file.

[0041] refer to Figure 2 As described above, in this embodiment, an image of the window center point including the detection frame and the specific pixel value and an Excel table including the window size and coordinates are obtained through step S3.

[0042] S4: In the same image, since the shooting distance is usually tens of meters or even hundreds of meters, the actual size of the window components in the center area and the surrounding area of ​​the image caused by perspective effect and lens distortion is different. According to the size of a certain window type, the corresponding relationship between pixel length and actual length can be obtained.

[0043] Specifically, in step S4, the length and position of the pixel where the window detection frame is located will not be changed in the subsequent image processing of the detection result, so it is necessary to determine the corresponding relationship between the pixel length and the actual length under this shooting situation in this step. The proportional relationship with the actual size can be determined by the average pixel length of a certain determined window height or width. The calculation formula of the proportional conversion factor of each image is as follows:

[0044]

[0045] Where s represents the ratio conversion factor under the current shooting state; D window Indicates the physical length or width of the window with a certain ratio; d window It indicates the average pixel length of the length or width of the detection box that belongs to the window of this size determined on the image.

[0046] S5: If multiple images are used for window target detection on the same facade, it is necessary to perform image stitching based on feature points on these images to obtain the detection results of the building facade. If one image covers the entire enclosure structure of a facade, skip this step and proceed to the next step.

[0047] refer to Figure 3 The image stitching in step S5 uses the SIFT feature point detection algorithm. First, the SIFT algorithm is used to extract feature points from the reference image and the image to be stitched. Then, the feature points of the two images are matched by Euclidean distance or other similarity measurement methods. Based on the matched feature point pairs, the random sampling consensus (RANSAC) and other algorithms are used to estimate the optimal perspective transformation matrix between the images. Finally, the images are aligned and fused according to the transformation matrix to achieve seamless stitching.

[0048] It should be noted that, since the image stitching process inevitably causes the image to be stitched to be scaled, this processing will not affect the window size determined in step S4, but will only change the position of the stitched window, and the window position needs to be re-determined in subsequent steps.

[0049] S6: Perform edge detection on the detected image to obtain the boundary line of the wall. Then pick up the pixels on the boundary of the wall to obtain the boundary coordinate points of the wall, and then obtain the boundary position information of the wall. The position information of the window can be obtained by marking the center point of the window in the picture.

[0050] refer to Figure 4 As shown, in step S6, after the detection images of the same facade are spliced, the coordinates of the window can be easily obtained and recorded according to the marking of the specific pixel value of the center point of the window. For the boundary of the wall, it is necessary to determine the position of the boundary of the wall by picking up the pixels on the boundary of the wall, and finally calculate the actual coordinates of the window based on the wall, as follows:

[0051] Step S6.1: First, perform an edge contour detection process on the image to convert the image into a black and white binary image, so that the boundaries of the wall and the boundaries of the nearby windows can be clearly seen.

[0052] Step S6.2: In order to accurately pick up the boundary contour line of the wall, you need to manually click the mouse to query the coordinates of the image containing the boundary of the wall. By querying the coordinates of any point on each boundary, you can get the position information of the boundary line of the wall, and you can also get the coordinates of the four corner points, and then get the length and height of the wall.

[0053] Step S6.3: The image-based coordinates of the window can be obtained by identifying the special pixel value of the center point of the window on the spliced ​​facade image. The coordinate point of the upper left corner of the wall is set as the coordinate origin of the wall. The processed coordinates obtained by subtracting the coordinates of the upper left corner of the wall from the coordinates of the window are the coordinates of the window based on the wall. The obtained coordinates and the window size obtained in step S3 are recorded in the same Excel table. According to the proportional conversion factor obtained in step S4, that is, the corresponding relationship, the actual position coordinates and size of the window can be calculated. The calculation formula is as follows:

[0054] W i =w i ×s

[0055] H i =h i ×s

[0056] X i =x i ×s+X wall

[0057] Y i =Y wall -y i ×s

[0058] Where: W i , H i , X i , Y i They represent the width, height, X coordinate and Y coordinate of the ith detected window in the image in the plane coordinates of the actual size, s represents the scale conversion factor, and w i 、hi 、x i ,y i Respectively represent the width pixel length, height pixel length and X coordinate and Y coordinate based on the wall of the i-th detected window on the image. wall , Y wall They respectively represent the coordinate origin of the wall in the actual space coordinates, which need to be determined in combination with the actual space position of the building.

[0059] Considering that the window coordinates are in a three-dimensional coordinate system, the above calculation formula is still applicable. You only need to change the Y coordinate in the two-dimensional plane to the Z coordinate in three dimensions. Due to the different facade orientations, when the building facade is parallel to the XOZ plane, the X coordinate of the plane is the X coordinate in three dimensions; when the building facade is parallel to the YOZ plane, the X coordinate of the plane is the Y coordinate in three dimensions.

[0060] S7: Once we have the size and location information of the building envelope, we can use Dynamo to perform parametric modeling of the building envelope.

[0061] refer to Figure 5 As shown in the figure, based on Dynamo, windows of different sizes and types are generated in different locations in Revit according to Excel information. Taking the windows of a wall as an example, the specific method is as follows:

[0062] S7.1: First import the window family file into Revit.

[0063] S7.2: Create a wall in Dynamo according to the actual location.

[0064] S7.3: Import the values ​​of the window coordinate column in the Excel table to generate a two-dimensional list, create windows of the same family type at specific locations on the wall based on the list, and adjust these windows to different family types according to the window names in Excel.

[0065] S7.4: Read the size information of the window in Excel and assign it to the family types of different window names, so that each window with a different name has different size information.

[0066] By executing step S7.3 and step S7.4 in sequence, all windows on the wall can be generated.

[0067] S8: Through the location of windows and other semantic information of the building, we can roughly obtain the floor information of the building and the floor plan of each floor, and then establish the entire building information model of the building.

[0068] Specifically, in step S8, walls and windows of other facades are generated, and the building model is improved according to other semantic information related to the building, so as to realize the reconstruction of the existing building information model.

[0069] In this embodiment, the semantic information includes material information of the building's exterior walls, material types of windows, and the like.

[0070] In a preferred embodiment, after obtaining the floor plan of each floor according to the position of each window and the actual apartment type on site, further refined modeling can be performed. Some building walls are not completely rectangular in shape, and some places have some depth, which can also be adjusted manually.

[0071] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0072] In another embodiment, an electronic device is also provided, including one or more processors, a memory, and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the method described above.

[0073] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicles, characterized in that: The following steps are involved: Obtain images of the four facades of the target building taken by a drone, use the trained deep learning target detection YOLO V9 model to perform target detection on the acquired images, and obtain a detection map containing window detection frames and center point marks, window position information, and window size information; Based on the window size information and the determined window type size, determining a correspondence between the pixel length of this shooting and the actual length; Performing edge detection processing on the detection image to obtain the boundary line of the wall, and picking up boundary coordinate points on the boundary line, and combining the corresponding relationship to obtain boundary position information of the wall and actual coordinates of the window based on the wall surface; Based on the boundary position information of the wall, parametric modeling of the building envelope structure is performed, and at the same time, based on the actual coordinates of the window based on the wall, the floor information of the building and the plane layout of each floor are obtained to complete the reconstruction of the entire building information model.

2. The method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle according to claim 1, characterized in that: Multiple pictures are taken corresponding to the same facade.

3. The method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle according to claim 2 is characterized in that: When multiple pictures are taken corresponding to the same facade, after target detection is performed on each picture, the detection images corresponding to each picture are stitched to obtain an overall detection image of the facade, and then the edge detection process is performed.

4. The method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle according to claim 3 is characterized in that: The image stitching is achieved by using SIFT feature point detection algorithm.

5. The method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle according to claim 2, characterized in that: The drone shooting angles of the multiple pictures are exactly the same.

6. The method for reconstructing an existing building information model based on image target detection and drone according to claim 1, characterized in that: When training the deep learning target detection YOLO V9 model, the building facade images collected by the drone are combined with images in other public building window target detection datasets to form a training dataset, and each image in the dataset is annotated and processed into the YOLO annotation format.

7. The method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle according to claim 6, characterized in that: The other public building window target detection datasets are screened and combined with the building facade images collected by the drone.

8. The method for reconstructing an existing building information model based on image target detection and drone according to claim 1, characterized in that: The corresponding relationship is represented by a proportional conversion factor under the current shooting state, and the proportional conversion factor under the current shooting state is expressed as: Where, s represents the proportional conversion factor under the current shooting state; D window Indicates the physical length or width of the window with a certain ratio; d window It indicates the average pixel length of the length or width of the detection box that belongs to the window of this size determined on the image.

9. The method for reconstructing an existing building information model based on image target detection and unmanned aerial vehicle according to claim 1, characterized in that: The process of obtaining the boundary position information of the wall and the actual coordinates of the window based on the wall surface includes: Convert the detection image into a black and white binary image to make the boundaries of the wall and nearby windows visible; By querying the coordinates of the four corner points of the wall, we can get the length and height of the wall. The coordinate point at the upper left corner of the wall is set as the coordinate origin of the wall, and the actual position coordinates and size of the window are calculated based on the corresponding relationship.

10. The method for reconstructing an existing building information model based on image target detection and drone according to claim 1, characterized in that: The reconstruction of the entire building information model is achieved using Dynamo.

Images