Method and device for identifying appearance damage of building facade

Abstract

The application relates to the technical field of building identification, and discloses a building facade appearance damage identification method and device. According to a target long-range image of a target building, a target long-range building model is constructed, which is closely combined with a three-dimensional building space in the identification process of the appearance damage area of the target building facade. Further, based on the polygon contour of the target building and the elevation information of the target building, which are extracted in combination with the target long-range building model, a target close-range image of the target building is collected by generating a close-range aerial photography planning path of the target building according to the coordinate information of the polygon contour of the target building and the elevation information of the target building, and finally the appearance damage area of the target building facade is identified. Therefore, the appearance damage identification method of the building facade is favorable for enhancing the accuracy of the identification of the appearance damage area of the target building facade, and is further favorable for the accurate maintenance of the appearance damage area.

Description

Technical Field

[0001] This invention relates to the field of building identification technology, specifically to a method and apparatus for identifying appearance damage to building facades. Background Technology

[0002] Building facades refer to the roof and all external envelope components. Identifying building facades enhances both the aesthetics and structural integrity of a building.

[0003] In related technologies, building facade identification mainly relies on manual visual inspection or simple tools, which suffers from low efficiency, high subjectivity, dangers of working at heights, and difficulty in accurately identifying the location of damage to the building facade. However, with the popularization of drone photography technology, building facade damage areas are generally identified using two-dimensional image processing data collected by drones. Because two-dimensional image processing cannot be effectively integrated with three-dimensional building spatial information, the identification results are not accurate enough, thus limiting the precise repair of building facade damage. Summary of the Invention

[0004] This invention provides a method and apparatus for identifying exterior damage to building facades, in order to solve the problem that the identification of damaged areas on building facades using two-dimensional image processing is not accurate enough because two-dimensional image processing cannot be effectively integrated with three-dimensional building spatial information.

[0005] According to a first aspect, the present invention provides a method for identifying appearance damage to a building facade, the method comprising: Obtain a distant view image of the target building; Construct a model of the target building from a distant view image; Based on the target building model, extract the polygonal outline and elevation information of the target building; Based on the polygonal outline of the target building, determine the coordinate information of the polygonal outline of the target building; Based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building, a close-up aerial photography planning path for the target building is generated. Collect close-up images of the target building according to the planned close-up aerial photography path; Based on the close-up image of the target building, identify the areas of exterior damage to the target building's facade. Construct a close-up model of the target building based on the close-up image of the target building; Based on the close-up building model of the target building and the area of ​​appearance damage on the exterior facade of the target building, locate the location of the appearance damage on the exterior facade of the target building. Based on the close-up architectural model of the target building and the area of ​​exterior damage to the target building facade, locate the location of the exterior damage to the target building facade, including: Select target 2D pixels from the area of ​​external damage on the exterior facade of the target building; Obtain the two-dimensional coordinates of the target two-dimensional pixel; By utilizing the intrinsic parameters of the aerial camera, the two-dimensional coordinates of the target two-dimensional pixels are transformed into distortion-free coordinates in the camera coordinate system; Using the extrinsic parameters of the aerial camera, the direction vector of the target ray in the camera coordinate system is transformed to the world coordinate system, and the starting position of the target ray is obtained; In the world coordinate system, obtain all rays that originate from the starting position of the target ray and pass through the target pixel; Determine the target's three-dimensional pixels from the close-up architectural model; Determine the optimal intersection point between the target's 3D pixels and all rays; The optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

[0006] In this embodiment, a target distant building model is constructed based on the target building's distant view image. This model is closely integrated with the 3D architectural space during the identification of the exterior damage areas of the target building's facade. Furthermore, based on the target distant building model, the polygonal outline and elevation information of the target building are extracted. Then, according to the coordinate information of the target building's polygonal outline and elevation information, a close-up aerial photography planning path is generated to collect close-up images of the target building, ultimately identifying the exterior damage areas of the target building's facade. Therefore, the exterior damage identification method for building facades in this embodiment enhances the accuracy of identifying exterior damage areas of the target building's facade, thereby facilitating precise repair of the damaged areas in the future.

[0007] In some optional implementations, the location of the surface damage to the target building facade is determined based on the target close-up building model and the area of ​​surface damage to the target building facade, including: Select target 2D pixels from the area of ​​external damage on the exterior facade of the target building; Obtain the two-dimensional coordinates of the target two-dimensional pixel; By utilizing the intrinsic parameters of the aerial camera, the two-dimensional coordinates of the target two-dimensional pixels are transformed into distortion-free coordinates in the camera coordinate system; Using the extrinsic parameters of the aerial camera, the direction vector of the target ray in the camera coordinate system is transformed to the world coordinate system, and the starting position of the target ray is obtained; In the world coordinate system, obtain all rays that originate from the starting position of the target ray and pass through the target pixel; Determine the target's three-dimensional pixels from the close-up architectural model; Determine the optimal intersection point between the target's 3D pixels and all rays; The optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

[0008] This embodiment, through the above-described implementation method, aims to accurately determine the location of external damage to the exterior facade of a target building.

[0009] In some alternative implementations, a target distant view building model is constructed based on the target distant view image of the target building, including: Analyze the feature points of the target building's distant view image using 3D architectural tools; Based on the feature points of the target building's distant view image, the position and orientation of the target building in the three-dimensional building space are calculated using 3D building tools; Based on the position and orientation of the target building in three-dimensional architectural space, a dense point cloud is generated using three-dimensional architectural tools; Use 3D building tools to convert dense point clouds into target distant building models.

[0010] This embodiment constructs a target distant building model based on the target building's distant view image through the above implementation method, and closely integrates it with the three-dimensional building space during the process of identifying the appearance damage area of ​​the target building's facade.

[0011] In some optional implementations, the polygonal outline of the target building is extracted based on the target distant building model, including: Based on the target building model, generate an orthophoto of the target building using 3D building tools; The first neural network model is used to identify whether each target pixel in the target orthophoto is a building pixel. The building pixels in the target orthophoto are sequentially binarized and morphologically processed. Extract the outline pixel boundary point sequence of the target building from the building pixels in the processed target orthophoto; Convert the sequence of outline pixels of the target building into a polygon outline of the target building.

[0012] This embodiment aims to accurately extract the polygonal outline of the target building through the above-described implementation method.

[0013] In some optional implementations, a close-up aerial photography planning path for the target building is generated based on the coordinate and elevation information of the target building's polygonal outline; including: Based on the polygonal outline of the target building, determine the orientation angle of the aerial camera to ensure that the optical axis of the aerial camera is perpendicular to the exterior of the target building; Based on the lateral coverage width and side overlap requirements of the aerial camera relative to the exterior of the target building, calculate the flight path spacing of the close-up aerial photography planning path for the target building. Calculate the waypoint spacing of the target building based on the directional coverage length and directional overlap requirements of the aerial camera relative to the exterior of the target building; Based on the coordinate information of the polygonal outline of the target building, the elevation information of the target building, the flight path spacing and waypoint spacing, a close-up aerial photography planning path for the target building is generated.

[0014] This embodiment, through the above implementation method, aims to generate a more suitable close-up aerial photography planning path for the target building, so as to facilitate the accurate acquisition of close-up images of the target building.

[0015] According to a second aspect, the present invention provides a device for identifying appearance damage to a building facade, the device comprising: The image acquisition module is used to acquire distant images of the target building; The first construction module is used to construct a model of the target building from the target distant view image. The contour extraction module is used to extract the polygonal contour and elevation information of the target building based on the target distant building model. The information determination module is used to determine the coordinate information of the polygonal outline of the target building based on the polygonal outline of the target building; The path generation module is used to generate a close-up aerial photography planning path for the target building based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building. The image acquisition module is used to acquire close-up images of the target building according to the planned close-up aerial photography path. The damage identification module is used to identify the areas of external damage to the facade of the target building based on a close-up image of the target building. The second construction module is used to construct a close-up model of the target building based on the close-up image of the target building. The location module is used to locate the position of the surface damage on the exterior of the target building based on the target close-up building model and the area of ​​surface damage on the target building facade. The location positioning module is specifically used for: Select target 2D pixels from the area of ​​external damage on the exterior facade of the target building; Obtain the two-dimensional coordinates of the target two-dimensional pixel; By utilizing the intrinsic parameters of the aerial camera, the two-dimensional coordinates of the target two-dimensional pixels are transformed into distortion-free coordinates in the camera coordinate system; Using the extrinsic parameters of the aerial camera, the direction vector of the target ray in the camera coordinate system is transformed to the world coordinate system, and the starting position of the target ray is obtained; In the world coordinate system, obtain all rays that originate from the starting position of the target ray and pass through the target pixel; Determine the target's three-dimensional pixels from the close-up architectural model; Determine the optimal intersection point between the target's 3D pixels and all rays; The optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

[0016] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the building facade appearance damage identification method described in the first aspect or any corresponding embodiment thereof.

[0017] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the building facade appearance damage identification method described in the first aspect or any corresponding embodiment thereof.

[0018] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the method for identifying appearance damage to building facades described in the first aspect or any corresponding embodiment. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of a first method for identifying exterior damage to building facades according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the heading overlap according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the lateral overlap according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a second process for identifying appearance damage to building facades according to an embodiment of the present invention. Figure 5 This is a schematic diagram of the projection ray method according to an embodiment of the present invention; Figure 6 This is a structural block diagram of a building facade appearance damage identification device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0023] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0024] According to an embodiment of the present invention, an embodiment of a method for identifying appearance damage to building facades is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] This embodiment provides a method for identifying exterior damage to building facades, which can be used with aerial cameras, mobile terminals such as mobile phones and tablets. Figure 1 This is a flowchart of a method for identifying exterior damage to building facades according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain a distant view image of the target building.

[0026] Specifically, the aerial camera, equipped with a high-definition visible light drone, captures distant images of the target building. The drone flies along a preset route and at a preset altitude.

[0027] Step S102: Construct a target distant view building model based on the target distant view image of the target building.

[0028] In some optional implementations, step S102 above, which involves constructing a target distant building model based on the target distant view image of the target building, includes: Step a1: Analyze the feature points of the target building's distant view image using 3D architectural tools.

[0029] Specifically, the 3D building tool can be the Metashape software. Feature points from the target distant view image are input into the 3D building tool, which then analyzes these feature points using feature detection algorithms.

[0030] Step a2: Based on the feature points of the target building's distant view image, calculate the position and orientation of the target building in the three-dimensional building space using a 3D building tool.

[0031] Step a3: Based on the position and orientation of the target building in the three-dimensional architectural space, generate a dense point cloud using three-dimensional architectural tools.

[0032] The Metashape software can use matching algorithms to perform denser matching on each pixel of a target building's distant image, generating a dense point cloud containing millions or even billions of points. This dense point cloud accurately describes the surface geometry of the target building.

[0033] Step a4: Use 3D building tools to convert the dense point cloud into a target distant building model.

[0034] Specifically, the dense point cloud is converted into a polygonal mesh surface using 3D building tools, which is the target distant building model mentioned above.

[0035] This embodiment constructs a model of the target building in the distance, which facilitates the accurate extraction of the polygonal outline of the target building.

[0036] Step S103: Extract the polygonal outline and elevation information of the target building based on the target distant building model.

[0037] In some specific implementations, the polygonal outline of the target building is extracted based on the target distant building model, including: Step b1: Based on the target distant building model, generate a target orthophoto of the target building using 3D building tools.

[0038] Specifically, based on the target distant building model described above, the 3D building tool uses orthophoto projection technology to project the target distant image of the target building onto a horizontal plane, generating a target orthophoto of the target building. This target orthophoto has a uniform scale and orientation, no perspective distortion, and can be used for outline extraction of the target building and subsequent flight path planning.

[0039] Step b2: Use the first neural network model to identify whether each target pixel in the target orthophoto is a building pixel.

[0040] Specifically, the first neural network model employs a deep learning semantic segmentation model based on architectures such as U-Net. This model is trained on a large dataset of orthophotos labeled with building outlines, with the input data being target orthophotos. After training, the first neural network model can perform pixel-by-pixel classification of the input orthophotos and output recognition results. These results include whether each target pixel is a building pixel or a non-building pixel.

[0041] Step b3 involves performing binarization and morphological processing on the building pixels in the target orthophoto image.

[0042] Specifically, the building pixels output from the target distant building model are converted into black-and-white binary images using a preset threshold. Morphological processing includes opening and closing operations. Opening operations can eliminate small noise points and separate adhered buildings. Closing operations can fill small holes and broken gaps inside the building outline.

[0043] Step b4: Extract the outline pixel boundary point sequence of the target building from the building pixels in the processed target orthophoto.

[0044] Step b5: Convert the sequence of outline pixel boundary points of the target building into the polygon outline of the target building.

[0045] Several feature extraction algorithms, including but not limited to Marching Squares and OpenCV's findContours algorithm, are used to extract the contour pixel boundary point sequence of the target orthophoto from the binarized image. This sequence is then further converted into a vector polygon, representing the polygonal contour of the target building.

[0046] Step S104: Determine the coordinate information of the polygonal outline of the target building based on the polygonal outline of the target building.

[0047] In some specific implementations, step S104 above, determining the coordinate information of the polygonal outline of the target building based on the polygonal outline of the target building, includes: The coordinate information of the target building's polygonal outline includes its longitude and latitude. Based on the target building's polygonal outline, geographic information system (GIS) tools are used to determine the latitude and longitude coordinates of the target building's polygonal outline.

[0048] Geographic Information System (GIS) tools are GIS software. In GIS software, the vertex coordinates of each polygon outline are the latitude and longitude coordinates of the polygon outline.

[0049] Step S105: Generate a close-up aerial photography planning path for the target building based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building.

[0050] In some specific implementations, step S105 above, which generates a close-up aerial photography planning path for the target building based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building, includes: Step c1: Determine the orientation angle of the aerial camera based on the polygonal outline of the target building to ensure that the optical axis of the aerial camera is perpendicular to the exterior of the target building. For example, if the polygonal outline of the target building is a rectangle, calculate the normal direction of each side of the rectangle. This normal direction is the camera angle at which the aerial camera is shooting the target building.

[0051] Step c2: Calculate the flight path spacing of the close-up aerial photography planning path for the target building based on the lateral coverage width and side overlap requirements of the aerial camera relative to the target building facade.

[0052] Specifically, aerial overlap requirements include: lateral overlap requirements and side overlap requirements. Lateral overlap refers to the percentage overlap between two adjacent photos taken along the same flight path, typically 70%–85% in engineering practice, with 80% being preferred. Side overlap refers to the percentage lateral overlap between adjacent flight paths, typically 60%–80% in engineering practice, with 70% being preferred. Overlap = Overlap length / Photo coverage length × 100%.

[0053] Specifically, such as Figure 2 The diagram shown illustrates the heading overlap. Figure 3 As shown, this represents the lateral overlap requirement.

[0054] In a specific example, the lateral coverage width of the aerial camera relative to the ground can be calculated using the following formula:

[0055] In another specific example, the lateral coverage width of the aerial camera relative to the ground can also be calculated using the following formula:

[0056] in, The horizontal coverage width of the aerial camera relative to the ground. This is the preset flight distance of the aerial camera from the exterior of the target building. The first size parameter of the aerial camera. The focal length of the aerial camera. This is the first field of view angle of the aerial camera.

[0057] Furthermore, based on the lateral coverage width of the aerial camera relative to the ground, the flight path spacing for the close-up aerial photography of the target building is calculated using the following formula.

[0058]

[0059] in, For the distance between routes, For lateral overlap requirements.

[0060] Step c3: Calculate the waypoint spacing of the target building based on the directional coverage length and directional overlap requirements of the aerial camera relative to the exterior of the target building.

[0061] In a specific example, the directional coverage length of the aerial camera relative to the ground can be calculated using the following formula:

[0062] In another specific example, the directional coverage width of the aerial camera relative to the ground can also be calculated using the following formula:

[0063] in, This refers to the directional coverage length of the aerial camera relative to the ground. This is the preset flight distance of the aerial camera from the exterior of the target building. This is the second size parameter for the aerial camera. The focal length of the aerial camera. This is the second field of view angle for the aerial camera.

[0064] Furthermore, based on the directional coverage width of the aerial camera relative to the ground, the flight path spacing of the close-up aerial photography planning path for the target building is calculated using the following formula.

[0065]

[0066] in, The distance between waypoints, For heading overlap requirements.

[0067] The order of steps c1 and c2 above is not important.

[0068] Step c4: Based on the coordinate information of the polygonal outline of the target building, the elevation information of the target building, the flight path spacing and waypoint spacing, generate a close-up aerial photography planning path for the target building.

[0069] Specifically, based on the obtained flight path spacing and waypoint spacing, the coordinate information (longitude and latitude coordinates) of the polygonal outline of the target building is further combined to generate a close-up aerial photography planning path for the target building.

[0070] Step S106: Collect close-up images of the target building according to the planned close-up aerial photography path.

[0071] Specifically, using a high-resolution aerial camera to fly along a planned close-up aerial photography path of the target building and acquire close-up images of the building's facade helps ensure the accuracy of the close-up image acquisition. This close-up imagery covers multiple perspectives of the building's facade.

[0072] Step S107: Identify the areas of exterior damage on the facade of the target building based on the close-up image of the target building.

[0073] In some specific implementations, based on a close-up image of the target building, the area of ​​external damage to the target building is identified, including: The second neural network model is used to identify the appearance damage areas in the close-up image of the target.

[0074] Specifically, the second neural network model employs a data-driven deep learning semantic segmentation model, including but not limited to DeepLabv3+, Mask R-CNN, and U-Net YOLO networks. This second neural network model was pre-trained using a large amount of close-up image data of building facades.

[0075] In some alternative implementations, the second neural network model is trained through the following steps: Step d1: Obtain historical close-up images of the target building.

[0076] Step d2 involves preprocessing the historical close-up image of the target building.

[0077] Preprocessing methods for historical close-up images include, but are not limited to, grayscale transformation enhancement, edge detection, linear transformation processing, threshold segmentation, morphological processing, texture analysis, color segmentation, region segmentation, and shape analysis processing.

[0078] Step d2: Extract historical crack features and historical peeling features corresponding to historical close-up images.

[0079] Step d3: Input the historical crack features and historical peeling features corresponding to the historical close-up images into the second neural network model for training, to obtain the recognition results of the appearance damage areas corresponding to the historical close-up images, and the loss value between the recognition results of the appearance damage areas corresponding to the historical close-up images and the true labels of the historical close-up images. The loss value is used to update the parameter values ​​of the second neural network model.

[0080] Updating the parameter values ​​of the second neural network model is equivalent to performing transfer learning and fine-tuning on it. For example, based on the ground truth labels of historical close-up images, the second neural network model is fine-tuned to identify the appearance damage regions corresponding to those historical close-up images. During training, the cross-entropy loss function is used, and the dataset of historical close-up images is expanded through data augmentation techniques such as rotation, cropping, and brightness adjustment to improve the robustness of the second neural network model.

[0081] Building upon this, this embodiment can also utilize a validation set to evaluate the performance of the second neural network model. For example, accuracy can be measured using metrics such as intersection-over-union ratio, precision, and recall, and the learning rate, network structure, or loss function can be adjusted as necessary to improve the performance of the second neural network model. After achieving the desired effect through training, the performance of the second neural network model is tested and verified using test set data of historical close-up images.

[0082] The method for identifying exterior damage to building facades in this embodiment constructs a target distant building model based on a distant view image of the target building. This model is then closely integrated with the 3D architectural space during the identification of damage areas on the building facade. Furthermore, by combining the distant view model with the extracted polygonal outline of the target building, and based on the position coordinates of this polygonal outline, a close-up aerial photography path is generated to acquire close-up images of the target building, ultimately identifying the damage areas on the building facade. Therefore, this method enhances the accuracy of identifying damage areas on the building facade, thus facilitating precise repairs of the damaged areas.

[0083] This embodiment provides a method for identifying exterior damage to building facades, which can be used with aerial cameras, mobile terminals such as mobile phones and tablets. Figure 4 This is a flowchart of a method for identifying exterior damage to building facades according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps: Step S101: Obtain a distant view image of the target building.

[0084] Step S102: Construct a target distant view building model based on the target distant view image of the target building.

[0085] Step S103: Extract the polygonal outline of the target building based on the target distant building model.

[0086] Step S104: Determine the coordinate information of the polygonal outline of the target building based on the polygonal outline of the target building.

[0087] Step S105: Generate a close-up aerial photography planning path for the target building based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building.

[0088] Step S106: Collect close-up images of the target building according to the planned close-up aerial photography path.

[0089] Step S107: Identify the areas of exterior damage on the facade of the target building based on the close-up image of the target building.

[0090] For specific details of steps S101-S107 above, please refer to the above embodiment, and they will not be repeated here.

[0091] Step S108: Construct a close-up model of the target building based on the close-up image of the target building.

[0092] The process of constructing the near-view building model of the target is the same as the process of constructing the far-view building model of the target. Please refer to steps a1-a3 above for details, which will not be repeated here.

[0093] Step S109: Based on the target close-up building model and the appearance damage area of ​​the target building facade, locate the appearance damage location of the target building facade.

[0094] In some optional implementations, step S109 above, locating the location of the appearance damage on the target building facade based on the target close-up building model and the appearance damage area of ​​the target building facade, includes: Step e1: Select target two-dimensional pixels from the appearance damage area of ​​the target building facade.

[0095] Step e2: Obtain the two-dimensional coordinates of the target two-dimensional pixel.

[0096] For example, such as Figure 5 The diagram shown is a schematic of the projection ray method. Figure 5 In the process, a target two-dimensional pixel P is selected in the area of ​​appearance damage on the exterior of the target building, and the two-dimensional coordinates of the target two-dimensional pixel are obtained. .

[0097] Step e3: Using the intrinsic parameters of the aerial camera, the two-dimensional coordinates of the target two-dimensional pixels are transformed into distortion-free coordinates in the camera coordinate system.

[0098] Using camera intrinsic parameters, the two-dimensional coordinates of the target two-dimensional pixels are obtained. Transformed to distortion-free coordinates in camera coordinate system .

[0099] Step e4: Using the extrinsic parameters of the aerial camera, transform the direction vector of the target ray in the camera coordinate system to the world coordinate system, and obtain the starting position of the target ray.

[0100] The direction vector of the target ray in the camera coordinate system Transform to the world coordinate system. The starting point of this target ray is the starting position of the camera center in the world coordinate system. .

[0101] Step e5: In the world coordinate system, obtain all rays that start from the starting position of the target ray and pass through the target pixel.

[0102] For example, in the world coordinate system, the equations of all rays originating from the camera center Ci and passing through the target 2D pixel P can be expressed as: .

[0103] in, Represents the ray direction vector. This represents the Euclidean distance from the center Ci of the aerial camera to point P along the ray direction.

[0104] Step e6: Determine the target's three-dimensional pixels from the target's close-up architectural model.

[0105] Step e7: Determine the optimal intersection point between the target 3D pixel and all rays.

[0106] Step e8: The optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

[0107] In theory, all rays corresponding to point P should intersect at the actual location of point P. However, due to noise, calibration errors, matching errors, etc., rays usually do not intersect precisely at a single point.

[0108] Finding the optimal intersection point involves identifying a 3D pixel X within the near-field architectural model of the target that minimizes the sum of its distances to all rays. This is a least-squares optimization problem. A commonly used solution is: The problem can be formulated as minimizing the sum of the squares of the sines of the angles between all ray direction vectors di and vector (X - Ci), which can be solved by minimizing the sum of the squares of the angles between them.

[0109]

[0110] This problem can be transformed into solving a system of linear equations AX = B, which can be solved using singular value decomposition (SVD) or normal equations.

[0111] Furthermore, the calculated three-dimensional spatial point X is the precise three-dimensional coordinate (X, Y, Z) of the target two-dimensional pixel point P in the world coordinate system on the target near-view building model, that is, the optimal intersection point between the target three-dimensional pixel point and all rays, and this optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

[0112] This embodiment, based on identifying the areas of external damage to the target building's facade, further combines a close-up model of the target building to locate the location of the external damage. This facilitates precise location of the target building, which in turn enables targeted repairs of the damaged areas.

[0113] This embodiment also provides a building facade appearance damage identification device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated for details already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0114] This embodiment provides a device for identifying appearance damage to building facades, such as... Figure 6 As shown, it includes: Image acquisition module 601 is used to acquire a distant view image of the target building; The first construction module 602 is used to construct a target distant building model based on the target distant image of the target building; The contour extraction module 603 is used to extract the polygonal contour and elevation information of the target building based on the target distant building model. The coordinate determination module 604 is used to determine the coordinate information of the polygonal outline of the target building based on the polygonal outline of the target building. The path generation module 605 is used to generate a close-up aerial photography planning path for the target building based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building. Image acquisition module 606 is used to acquire close-up images of the target building according to the close-up aerial photography planning path of the target building; The damage identification module 607 is used to identify the appearance damage areas of the exterior facade of the target building based on the close-up image of the target building. The second construction module is used to construct a close-up model of the target building based on the close-up image of the target building. The location module is used to locate the position of the surface damage on the exterior of the target building based on the target close-up building model and the area of ​​surface damage on the target building facade. The location module is further used for: Select target 2D pixels from the area of ​​external damage on the exterior facade of the target building; Obtain the two-dimensional coordinates of the target two-dimensional pixel; By utilizing the intrinsic parameters of the aerial camera, the two-dimensional coordinates of the target two-dimensional pixels are transformed into distortion-free coordinates in the camera coordinate system; Using the extrinsic parameters of the aerial camera, the direction vector of the target ray in the camera coordinate system is transformed to the world coordinate system, and the starting position of the target ray is obtained; In the world coordinate system, obtain all rays that originate from the starting position of the target ray and pass through the target pixel; Determine the target's three-dimensional pixels from the close-up architectural model; Determine the optimal intersection point between the target's 3D pixels and all rays; The optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

[0115] In some alternative implementations, the first building module 602 is further configured to: Analyze the feature points of the target building's distant view image using 3D architectural tools; Based on the feature points of the target building's distant view image, the position and orientation of the target building in the three-dimensional building space are calculated using 3D building tools; Based on the position and orientation of the target building in three-dimensional architectural space, a dense point cloud is generated using three-dimensional architectural tools; Use 3D building tools to convert dense point clouds into target distant building models.

[0116] In some alternative implementations, the contour extraction module 603 is further configured to: Based on the target building model, generate an orthophoto of the target building using 3D building tools; The first neural network model is used to identify whether each target pixel in the target orthophoto is a building pixel. The building pixels in the target orthophoto are sequentially binarized and morphologically processed. Extract the outline pixel boundary point sequence of the target building from the building pixels in the processed target orthophoto; Convert the sequence of outline pixels of the target building into a polygon outline of the target building.

[0117] In some alternative implementations, the path generation module 605 is further configured to: Based on the polygonal outline of the target building, determine the orientation angle of the aerial camera to ensure that the optical axis of the aerial camera is perpendicular to the exterior of the target building; Based on the lateral coverage width and side overlap requirements of the aerial camera relative to the exterior of the target building, calculate the flight path spacing of the close-up aerial photography planning path for the target building. Calculate the waypoint spacing of the target building based on the directional coverage length and directional overlap requirements of the aerial camera relative to the exterior of the target building; Based on the coordinate information of the polygonal outline of the target building, the elevation information of the target building, the flight path spacing and waypoint spacing, a close-up aerial photography planning path for the target building is generated.

[0118] The building facade appearance damage identification device provided in this embodiment of the invention can execute the building facade appearance damage identification method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0119] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0120] The following is a detailed reference. Figure 7 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 702 or a program loaded from memory 708 into random access memory (RAM) 703. In RAM... 703 also stores various programs and data required for the operation of the electronic device. The processor 701, ROM 702, and RAM 703 are interconnected via bus 704. The input / output (I / O) interface 705 is also connected to bus 704.

[0121] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0122] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a memory 708, or installed from a ROM 702. When the computer program is executed by the processor 701, it performs the functions defined in the building facade appearance damage identification method of the embodiments of the present invention.

[0123] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0124] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the method for identifying exterior damage to building facades shown in the above embodiments is implemented.

[0125] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0126] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for identifying appearance damage to building facades, characterized in that, The method includes: Obtain a distant view image of the target building; Based on the target distant view image of the target building, construct a target distant view building model; Based on the target building model, extract the polygonal outline and elevation information of the target building; Based on the polygonal outline of the target building, determine the coordinate information of the polygonal outline of the target building; Based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building, a close-up aerial photography planning path for the target building is generated. Acquire close-up images of the target building according to the planned close-up aerial photography path; Based on the close-up image of the target building, identify the areas of exterior damage to the target building's facade. Based on the close-up image of the target building, construct a close-up model of the target building; Based on the target close-up building model and the area of ​​appearance damage to the target building facade, locate the position of the appearance damage to the target building facade. Based on the close-up architectural model of the target building and the area of ​​exterior damage to the target building facade, locate the location of the exterior damage to the target building facade, including: Select target 2D pixels from the area of ​​external damage on the exterior facade of the target building; Obtain the two-dimensional coordinates of the target two-dimensional pixel; By utilizing the intrinsic parameters of the aerial camera, the two-dimensional coordinates of the target two-dimensional pixels are transformed into distortion-free coordinates in the camera coordinate system; Using the extrinsic parameters of the aerial camera, the direction vector of the target ray in the camera coordinate system is transformed to the world coordinate system, and the starting position of the target ray is obtained; In the world coordinate system, obtain all rays that originate from the starting position of the target ray and pass through the target pixel; Determine the target's three-dimensional pixels from the close-up architectural model; Determine the optimal intersection point between the target's 3D pixels and all rays; The optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

2. The method according to claim 1, characterized in that, Based on the target distant view image of the target building, a target distant view building model is constructed, including: The feature points of the target building's distant view image are analyzed using 3D architectural tools; Based on the feature points of the target building's distant view image, the position and orientation of the target building in three-dimensional architectural space are calculated using 3D architectural tools; Based on the position and orientation of the target building in three-dimensional architectural space, a dense point cloud is generated using three-dimensional architectural tools; The dense point cloud is converted into the target distant building model using 3D building tools.

3. The method according to claim 1, characterized in that, Based on the target building model, extract the polygonal outline of the target building, including: Based on the target building model, generate a target orthophoto of the target building using 3D building tools; The first neural network model is used to identify whether each target pixel in the target orthophoto is a building pixel; The building pixels in the target orthophoto are sequentially binarized and morphologically processed. Extract the outline pixel boundary point sequence of the target building from the building pixels in the processed target orthophoto; The sequence of outline pixel boundary points of the target building is converted into the polygon outline of the target building.

4. The method according to claim 1, characterized in that, Based on the coordinate and elevation information of the polygonal outline of the target building, a close-up aerial photography planning path for the target building is generated; including: Based on the polygonal outline of the target building, determine the orientation angle of the aerial camera to ensure that the optical axis of the aerial camera is perpendicular to the exterior of the target building; Based on the lateral coverage width and side overlap requirements of the aerial camera relative to the exterior of the target building, calculate the flight path spacing of the close-up aerial photography planning path for the target building. Calculate the waypoint spacing of the target building based on the directional coverage length and directional overlap requirements of the aerial camera relative to the exterior of the target building; Based on the coordinate information of the polygonal outline of the target building, the elevation information of the target building, the flight path spacing, and the waypoint spacing, a close-up aerial photography planning path for the target building is generated.

5. A device for identifying surface damage to building facades, characterized in that, The device includes: The image acquisition module is used to acquire distant images of the target building; The first construction module is used to construct a target distant view building model based on the target distant view image of the target building; The contour extraction module is used to extract the polygonal contour and elevation information of the target building based on the target distant building model. The information determination module is used to determine the coordinate information of the polygonal outline of the target building based on the polygonal outline of the target building; The path generation module is used to generate a close-up aerial photography planning path for the target building based on the coordinate information of the polygonal outline of the target building and the elevation information of the target building. The image acquisition module is used to acquire close-up images of the target building according to the planned close-up aerial photography path of the target building; The damage identification module is used to identify the appearance damage areas of the exterior facade of the target building based on the close-up image of the target building. The second construction module is used to construct a close-up model of the target building based on the close-up image of the target building. The location module is used to locate the position of the surface damage on the exterior of the target building based on the target close-up building model and the area of ​​surface damage on the target building facade. The location positioning module is specifically used for: Select target 2D pixels from the area of ​​external damage on the exterior facade of the target building; Obtain the two-dimensional coordinates of the target two-dimensional pixel; By utilizing the intrinsic parameters of the aerial camera, the two-dimensional coordinates of the target two-dimensional pixels are transformed into distortion-free coordinates in the camera coordinate system; Using the extrinsic parameters of the aerial camera, the direction vector of the target ray in the camera coordinate system is transformed to the world coordinate system, and the starting position of the target ray is obtained; In the world coordinate system, obtain all rays that originate from the starting position of the target ray and pass through the target pixel; Determine the target's three-dimensional pixels from the close-up architectural model; Determine the optimal intersection point between the target's 3D pixels and all rays; The optimal intersection point is taken as the location of the appearance damage on the exterior of the target building.

6. An electronic device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the building facade appearance damage identification method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the building facade appearance damage identification method according to any one of claims 1 to 4.

8. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the building facade appearance damage identification method according to any one of claims 1 to 4.

Images