Building facade door and window automatic completion method based on occlusion evidence inference

Abstract

This invention discloses an automatic door and window completion method for building facades based on occlusion evidence, belonging to the field of 3D reconstruction technology. The method includes: detecting occlusion areas and extracting boundary evidence and locally visible door and window fragments; generating candidate door and window predictions using a multi-evidence source supplementation strategy, sequentially supplementing cross-facade evidence and statistical priors of similar buildings when the regularity of visible doors and windows on the same facade is insufficient; correcting and weighting the candidate predictions using locally visible door and window fragments as geometric constraints to obtain door and window completion parameters and a comprehensive confidence level; back-mapping the completion parameters to point cloud data for consistency verification, adjusting the weights and re-fusing if the verification fails, and outputting the results according to the confidence level after passing the verification. This method improves the sufficiency of the basis for door and window inference under occlusion conditions and the verifiability of the completion results, making the reliability of the completion results quantifiable and assessable, and can be more effectively used for urban real-scene 3D reconstruction or refined modeling of existing buildings.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, specifically to a method for automatically completing building facade doors and windows based on occlusion evidence. Background Technology

[0002] In 3D reconstruction of building facades, conventional techniques address the occlusion of doors and windows caused by trees, temporary structures, or blind spots in scanning. These techniques utilize the visible portions of the facade for door and window detection and geometric fitting, then infer or fill in the occluded areas. This approach yields relatively complete reconstruction results when the occlusion area is small and the facade is highly regular. However, when the occlusion area is large or the facade bays are irregularly arranged, the observation constraints provided by the visible portions are severely insufficient. This makes it difficult to reliably infer the position, size, and style of doors and windows within the occluded area, leading to missing components or geometric errors in the reconstructed model.

[0003] To address the aforementioned issues, several approaches have been developed in related technologies. For example, facade grid analysis methods based on translational symmetry or shape priors establish regular grids on facade images or point clouds and infer occluded openings using the positional patterns of repeating units; deep learning-based facade occlusion analysis and image inpainting methods use convolutional networks or generative adversarial networks to perform pixel-level inpainting of occluded areas in the image or depth domain; and methods based on a 3D semantic reconstruction pipeline utilize point cloud or image evidence from a single facade to detect and fit door and window components.

[0004] However, the inventors discovered that the above methods still have the following shortcomings in practical applications: Raster-based methods, when the occlusion area is large or the facade has irregular openings, cannot summarize a stable repetition period from the visible part, leading to a direct failure to determine reconstruction failure after rule summarization fails, lacking a mechanism to degrade to using other evidence sources when the raster is unavailable; Depth-generative methods perform pixel-level completion in the image or depth map domain, neither explicitly using the geometric contour of the occlusion boundary to constrain the spatial range of the completed area, nor verifying the geometric consistency between the generated result and the visible fragment, resulting in structural distortion in the completion result and failing to provide confidence levels to prompt manual review; Single-facade semantic reconstruction methods lack sufficient observation constraints when the visible proportion of the same facade is too low, only able to mark the occluded area as unknown or degrade it to wall filling, failing to utilize statistical priors from other facades of the same building or even similar buildings for evidence transfer. Furthermore, the above methods typically output all completion results indiscriminately, without labeling the sufficiency of evidence and the credibility of results for different areas, making it difficult for downstream quality control to allocate review resources differentiatedly.

[0005] The inventors further analyzed that at least part of the root cause of the above-mentioned shortcomings is that the completion process of the existing method relies on a single type of evidence source. When the evidence source is unavailable under occlusion conditions, there is a lack of alternative inference basis. Furthermore, there is a lack of effective consistency verification between the completion result and the actual observation data, making it difficult to make a quantitative judgment on the reliability of the completion result.

[0006] Based on this, the present invention proposes an automatic door and window completion method for building facades based on occlusion evidence, which can improve the sufficiency of the basis for door and window inference under occlusion conditions and the verifiability of the completion results, so as to realize evidence-based inference and graded verifiable output in the door and window completion process in occluded areas. Summary of the Invention

[0007] To at least partially overcome the problems existing in related technologies, this application proposes an automatic door and window completion method for building facades based on occlusion evidence. The method provides verifiable spatial constraints for inference based on the boundary geometric features of the occluded area and locally visible door and window fragments, which helps to improve the sufficiency and verifiability of the geometric basis for the door and window completion results in the occluded area.

[0008] This application provides a method for automatically completing building facade doors and windows based on occlusion evidence inference. The method includes the following steps: Acquire multi-source facade data, perform main plane fitting and orthographic projection on each facade, and generate frontal projection map, point cloud density map and bidirectional index table between the frontal projection map and the point cloud data for each facade. Based on the orthographic projection map and the point cloud density map, the occlusion area is detected, the boundary contour of the occlusion area is extracted as boundary evidence, and locally visible door and window fragments are extracted in the expansion neighborhood of the occlusion area. Using the visible doors and windows in the frontal projection as the first source of evidence, when the grid regularity score of the visible doors and windows is not lower than a preset threshold, the window opening position in the occluded area is extrapolated according to the arrangement pattern of the visible doors and windows, a first candidate prediction is generated and associated with the first confidence level; otherwise, the corresponding occluded area is marked as the second source of evidence. Using other facades of the same building as the second source of evidence, in response to the mark of transferring to the second source of evidence, the window parameters of other facades that are isomorphic to the current facade are retrieved, and after geometric transformation, they are migrated to the current occlusion area to generate a second candidate prediction and associate it with a second confidence level. When no migrateable reference window is found, the corresponding occlusion area is marked as transferring to the third source of evidence. Using statistical priors of similar buildings as the third source of evidence, in response to the marker of transferring to the third source of evidence, matching statistical prior entries are retrieved, and after consistency testing, a third candidate prediction is generated and associated with a third confidence level. Using the locally visible door and window segments as geometric constraints, the predicted values ​​and confidence levels of each candidate prediction for the same occluded area are corrected, and then weighted and fused according to the corrected confidence levels to obtain the door and window completion parameters and comprehensive confidence level of the occluded area. Based on the bidirectional index table, the door and window completion parameters are back-mapped to the point cloud data, and consistency verification is performed with the boundary evidence. If the verification fails, the confidence weight of each candidate prediction is adjusted, and the weighted fusion is re-executed until the verification is passed or the preset iteration limit is reached. The completion results that pass the verification are divided into confidence levels according to the comprehensive confidence level and then output.

[0009] The technical solution provided in this application extracts boundary contours and locally visible door and window fragments during the occlusion detection stage, providing verifiable spatial constraints for inference. This ensures that the completion process is conducted within defined geometric boundaries, thus maintaining consistency between the geometric range of the completion result and the true boundary of the occluded area. A multi-evidence source supplementation strategy is employed, sequentially supplementing cross-facade evidence and statistical priors of similar buildings when the regularity of visible doors and windows on the same facade is insufficient. This allows for the generation of source-labeled candidate predictions even when a single evidence source fails, thereby producing well-founded inference results even in scenarios with low facade visibility. Finally, the predicted values ​​and confidence levels of each candidate prediction are evaluated using locally visible door and window fragments as geometric constraints. After correction, weighted fusion is performed, ensuring that the predicted values ​​and weights involved in the fusion are verified and adjusted based on the visible segments. This improves the geometric consistency between the completed parameters and the actual visible parts. By mapping the completed parameters back to the point cloud data via a bidirectional index table and verifying their consistency with boundary evidence, weight adjustments and re-fusion are triggered when the verification fails, forming a quantifiable closed-loop verification mechanism. This ensures that the output completed results are effectively verified against the actual observation data. By classifying the completed results that pass the verification into levels based on their comprehensive confidence, the completed results for different occluded areas are explicitly distinguished according to the sufficiency of the inferred evidence, thus providing a quantifiable reference for downstream differentiated management. Attached Figure Description

[0010] Figure 1 A flowchart illustrating an embodiment of the automatic completion method for building facade doors and windows based on occlusion evidence provided in this application; Figure 2 This is a schematic diagram illustrating the priority order of multi-source evidence in an obscured area in one embodiment of this application. Detailed Implementation

[0011] To make the purpose, technical solution and advantages of this application clearer, the technical solution of this application will be described in detail below.

[0012] Based on the background technology, existing symmetry / repetition pattern perception facade analysis methods, such as facade raster analysis based on translational symmetry and occluded facade analysis based on MRF (Markov Random Field) shape prior, achieve good results when the occluded area is small and the facade itself has strict translational symmetry. However, when the occlusion area is large (e.g., the canopy of an entire tree blocks more than two spans) or the facade itself has irregular spans (corner units, large windows of ground-floor shops, added components), the regular grid cannot be used to summarize a stable repetition period from the visible part. Existing literature clearly points out that this kind of grammar / grid-based method "can repair the occluded window openings, but the calculation time is large, and when the occlusion is severe, the grammar rules themselves cannot be summarized, thus leading to the failure of facade reconstruction." That is, this type of method has a brittle failure mode from "no grid to summarize" to "direct failure judgment" and lacks a mechanism to degenerate to use other sources of evidence when the grid is unavailable.

[0013] Deep learning-based methods for facade occlusion resolution and image inpainting (such as multi-scale row-column convolutional networks for occluded facade resolution, and methods that use Pix2Pix generative adversarial networks to orthophoto-project point clouds into depth / color images, then inpaint and map them back to 3D) learn the statistical relationship between occluded and visible areas in a data-driven manner. Within the architectural style range covered by the training data, they can generate visually coherent inpainting results. However, these methods are essentially pixel-level generation in the image / depth domain. The positions and sizes of doors and windows are implicitly determined by the network. They neither explicitly use the geometric contours of the occluded object boundaries to constrain the spatial range of the inpainted area, nor do they verify the geometric consistency between the generated results and the actual visible segments. This results in the boundaries of the inpainted door and window openings not matching the actual occlusion boundaries. Furthermore, when the facade style of the test building exceeds the distribution of the training set, the generated results exhibit structural distortion, and the system itself cannot provide a confidence level to indicate the need for manual review.

[0014] Existing 3D semantic facade reconstruction pipelines (such as LoD3 semantic reconstruction methods based on the physical characteristics of laser ranging and architectural priors, and facade structure extraction methods combining semantic information) mainly utilize point cloud / image evidence of a single facade to complete the detection and geometric fitting of door and window components. When the visible proportion of openings is high, they can obtain relatively complete geometric and semantic results. However, when the visible observation proportion of the same facade is too low (e.g., below 30%), methods that rely solely on evidence within that facade lack sufficient observation constraints. They can usually only mark the obscured areas as "unknown" or degenerate into wall filling. They fail to establish an evidence transfer and fusion mechanism between other facades of the same building, or even statistical priors of similar buildings. This results in the direct abandonment of door and window information that could have been constrained and inferred through cross-facade and cross-building evidence, reducing the completeness of LoD3-level fine modeling.

[0015] To address the shortcomings of existing technologies in three aspects—raster grammar failure under severe irregular occlusion, lack of geometric evidence constraints and confidence in image generative completion, and direct abandonment of inference when single-facade evidence is insufficient—this invention provides an automatic door and window completion method for building facades based on occlusion evidence inference. This method systematically extracts boundary geometric evidence of the occluded area and performs weighted inference by fusing multi-source evidence according to the priority order of "visible fragments of this facade - other facades of the same building - statistical priors of similar buildings." Simultaneously, it verifies the geometric consistency between the inference results and the boundary evidence and outputs confidence labels, ensuring that the door and window completion results are verifiable and graded.

[0016] In one embodiment, such as Figure 1 As shown, the automatic completion method for building facade doors and windows based on occlusion evidence proposed in this application includes the following steps: Step S110: Obtain multi-source facade data, perform main plane fitting and orthographic projection on each facade, and generate frontal projection map, point cloud density map and bidirectional index table between frontal projection map and point cloud data for each facade.

[0017] Specifically, in this step, the multi-source facade data includes at least point clouds and images. The main plane fitting can use the RANSAC algorithm to provide an accurate facade reference plane for subsequent orthographic projection. The resolution of the orthographic projection is 2 cm / pixel, and the geometric registration error between the orthographic projection and the point cloud data does not exceed ±0.05m. The above resolution and registration accuracy together provide the necessary accuracy basis for subsequent occlusion detection, geometric correction, and reverse mapping verification.

[0018] For example, in one specific implementation, the multi-source facade data includes oblique photogrammetric dense point clouds, ground laser scan point clouds, and close-up images of the target building. The point density of the oblique photogrammetric dense point cloud is no less than 300 points / m². 2 The point density of the ground laser scanning point cloud is no less than 800 points / m².2 The ranging accuracy is ±3mm@50m, and the resolution of the close-range image is no less than 4000×3000 pixels. The above point density and resolution requirements are determined based on the typical output capabilities of oblique photogrammetry and ground scanning equipment under normal operating parameters, respectively, which can provide a sufficient number of sampling points for an orthophoto grid of 2cm / pixel. Taking the main facade normal vector of the building's outer contour polygon (coordinate system WGS-84 or CGCS2000) as the reference, the RANSAC algorithm is used to perform principal plane fitting for each facade. The interior point distance threshold is set to 0.03m, and the iteration limit is 2000 times to extract the point cloud subset belonging to that facade. The interior point distance threshold is set according to the empirical tolerance of facade flatness, and the iteration limit ensures that the algorithm can converge when the number of facade point clouds is large. The facade point cloud is orthographically projected along its local coordinate system, with the horizontal direction as the u-axis, the vertical direction as the v-axis, and the normal as the w-axis. This generates a depth map, color map, and normal residual map with a resolution of 2 cm / pixel. This resolution matches the typical dimensions of building door and window components (e.g., window frame width is usually no less than 5 cm), effectively distinguishing door and window boundaries. The point cloud density map records the number of points within each 2 cm × 2 cm grid, used to determine the observation sufficiency of each grid in subsequent occlusion detection. Pose registration is performed on the near-view image and the facade reconstruction results. The registration algorithm uses PnP combined with RANSAC, with a reprojection error threshold of 1.5 pixels. The image is projected onto the same uv plane and aligned pixel-level with the orthographic point cloud map. A bidirectional index table is established between each pixel of the orthographic projection map and the point cloud data. The geometric registration error does not exceed ±0.05 m. This error threshold is on the same order of magnitude as the orthographic projection resolution, ensuring that when the 2D inference results are mapped back to the 3D point cloud via the bidirectional index table, the verification window accurately covers the corresponding point cloud area. This bidirectional index table supports querying the corresponding 3D point cloud from 2D pixel coordinates, and retrieving the pixel coordinates of the 3D point cloud in its orthographic projection.

[0019] In this way, step S110 provides a unified spatial reference for subsequent occlusion detection and establishes a mapping channel between two-dimensional and three-dimensional for the three-dimensional verification of the completed parameters.

[0020] Then, in step S120, the occlusion area is detected based on the orthographic projection map (including depth map, color map and normal residual map) and point cloud density map. The boundary contour of the occlusion area is extracted as boundary evidence, and locally visible door and window fragments are extracted in the dilated neighborhood of the occlusion area.

[0021] In some embodiments, the detection of occlusion regions in this step is achieved by traversing the frontal projection map and the point cloud density map grid by grid and performing occlusion determination. Grids that meet the conditions of point cloud density being lower than a density threshold, depth value variance being greater than a variance threshold, or missing depth values ​​are marked as occlusion candidates. Connectivity component marking is performed on the occlusion candidates and isolated candidates with an area smaller than an area threshold are removed to obtain the occlusion region. The density threshold, variance threshold, and area threshold are all preset values.

[0022] Specifically, for example, for each 2cm×2cm grid, if the point cloud density is below a density threshold, the depth value variance is above a variance threshold, or the depth value is missing, the grid is marked as an occlusion candidate. Here, the density threshold is set to 2 points / grid, corresponding to 50 points / m. 2 A density lower than this indicates that the point cloud sampling within the grid is insufficient to support reliable geometric inference; the depth variance is calculated within a 5×5 neighborhood, with a variance threshold of 0.15m. 2 A variance exceeding this threshold indicates drastic depth variations in the area, exhibiting typical characteristics of non-planar shading objects such as vegetation. After labeling, 8-neighborhood connected component labeling is performed on the shading candidate meshes, aggregating spatially continuous shading candidates into connected components, and removing those with areas smaller than the area threshold of 0.04m. 2 Isolated candidates are selected to exclude local holes caused by point cloud noise and retain occluded areas with actual geometric significance. In this process, the occlusion type can be determined based on the depth characteristics and point cloud distribution characteristics of each occluded area. The occlusion types can be divided into four categories: vegetation occlusion, viewing blind spots, scanning dead angles, and temporary structures. Different processing strategies can be adopted for different types of occluded areas in subsequent boundary extraction and evidence inference.

[0023] In some embodiments, the extraction of boundary evidence employs different methods depending on the presence or absence of a depth value for the occluded region in the orthographic projection. Specifically, extracting the boundary contour of the occluded region includes: When the depth value of the shading area exists and the difference between the depth of the shading area and the depth of the main plane of the facade is greater than the difference threshold of 0.2m, it indicates that the area is the shading object itself located in front of the facade, such as a tree canopy or scaffolding. In this case, the Douglas-Peucker algorithm is used to extract the two-dimensional contour polygon of the shading object on the frontal projection with a tolerance of 0.03m. The number of contour vertices is simplified to a level that can express the overall shape of the shading object, which serves as boundary evidence of the shading area.

[0024] Regarding the selection of the aforementioned difference threshold of 0.2m, this threshold is used to distinguish between independent obstructions located in front of the facade and the local concave and convex structures of the facade itself. The distance between the window sills, decorative moldings, and other concave and convex components of the building facade and the main plane is usually no more than 0.2m, while the clearance between independent obstructions such as tree canopies and scaffolding and the facade is generally greater than this value. Therefore, using 0.2m as the boundary can effectively filter out the interference of the geometric undulations of the facade itself on the determination of obstructions. Regarding the selection of the aforementioned preset tolerance of 0.03m for the Douglas-Peucker algorithm, this value is on the same order of magnitude as the resolution of orthographic projection, which can retain the main shape features of the obstruction outline while filtering out the contour jaggedness caused by local undulations of the point cloud.

[0025] When the depth value of an obstructed area is missing, it indicates that the area is a scanning blind spot or a viewing blind zone. In this case, the outer boundary of the connected domain is directly used as boundary evidence. Furthermore, when the depth value of an obstructed area exists but the depth difference with the corresponding main plane of the facade is no greater than the difference threshold of 0.2m, it indicates that although the area has depth data, its distance from the main plane of the facade is too close to be considered an independent obstruction. It may be a local uneven structure of the facade itself or an attachment closely to the facade. In this case, the same treatment method as for the missing depth value is used, directly using the outer boundary of the connected domain as boundary evidence.

[0026] Based on the extracted boundary evidence, locally visible door and window fragments are extracted from the expanded neighborhood of the occluded area. For example, for each occluded area, its expanded neighborhood is taken with an expansion radius of 0.3m. Within this neighborhood, a search is conducted to determine if locally visible geometric fragments belonging to the door and window category exist. These fragments include remnants of door and window components with clear geometric features, such as the right-angled edges of window frames, window sills, and vertical edges of door openings. If such fragments are found, their endpoint coordinates, edge direction angles, and lengths are recorded as locally visible door and window fragments of that occluded area. It should be noted that the aforementioned expansion radius of 0.3m is based on the typical dimensions of door and window components on building facades, which can effectively capture residual visible door and window components adjacent to the occlusion boundary without excessively expanding the search range. The detection results for each occluded area are organized in the form of a structured record table, including the area number, area area, occlusion type, a list of boundary polygon vertex coordinates, and a list of locally visible door and window fragments, for use in subsequent steps.

[0027] In this way, the occluded area is transformed from a simple point cloud void into a structured object with boundary geometric constraints and local fragment evidence, providing verifiable spatial constraints for subsequent inferences.

[0028] Based on step S120, subsequent steps S130, S140, and S150 are performed, combined with... Figure 2As shown in the technical solution of this application, the generation of candidate predictions for doors and windows within the obstructed area adopts a strategy of sequentially supplementing three levels of evidence sources according to priority. The three levels of evidence sources are the arrangement pattern of visible doors and windows on the same facade (pattern candidates), window opening parameters on other facades of the same building (migration candidates), and statistical priors of similar buildings (priority candidates), with their priorities decreasing in that order. When a high-priority evidence source cannot produce a valid candidate prediction due to insufficient available information under obstruction conditions, it is automatically supplemented to the next lower priority evidence source, rather than directly abandoning the inference when a single evidence source fails. The processing steps of the three levels of evidence sources are explained below.

[0029] Step S130: Using the visible doors and windows in the frontal projection map as the first source of evidence, when the grid regularity score of the visible doors and windows is not lower than the preset threshold, the window positions in the occluded area are extrapolated according to the arrangement rules of the visible doors and windows, a first candidate prediction is generated and associated with the first confidence level; otherwise, the corresponding occluded area is marked as the second source of evidence.

[0030] The inputs to this step include the facade color map in the orthographic projection generated in step S110, the occlusion record table generated in step S120, and the set of visible door and window rectangles extracted from the orthographic projection by the door and window detector. The set of visible door and window rectangles can be output from the upstream facade component detection sub-process. In practice, this sub-process can employ well-known opening detection methods, such as a window opening detector based on Faster R-CNN or HOG+SVM. Each rectangle contains center coordinates, width, and height parameters. The occlusion record table provides the location and extent information of each occluded area, used to determine which occluded areas require grid extrapolation.

[0031] In some embodiments, the preset threshold is 0.6, and the calculation of the grid regularity score includes: performing row and column clustering on visible doors and windows to obtain the actual number of detected window openings in each floor row, the expected number of window openings estimated by the facade width, and the standard deviation and mean of the distance between adjacent windows. The grid regularity score is equal to the ratio of the actual number of detected window openings to the expected number of window openings multiplied by the distribution attenuation factor calculated based on the standard deviation and mean.

[0032] Specifically, in the row-column clustering process, one-dimensional kernel density clustering is performed on the vertical coordinates of the visible door and window centers. For example, a bandwidth of 0.4m is used, which is consistent with the typical floor height of conventional residential buildings and can effectively distinguish doors and windows in different floor rows. The clustering yields a set of candidate floor rows. Then, within each row of the set, an equal-interval test is performed on the horizontal coordinates of the visible doors and windows, and the sample standard deviation and mean of the distance between adjacent doors and windows are calculated.

[0033] As mentioned above, raster regularity score This is equal to the ratio of the actual number of window openings detected in that row to the expected number of window openings estimated based on the facade width and average spacing, multiplied by the distribution attenuation factor. The distribution attenuation factor can be an exponential function, as shown in the following expression: (1) In expression (1), Standard deviation, The mean, This represents the actual number of window openings detected by the bank. To be based on facade width and The estimated number of windows. When the standard deviation is small relative to the mean, the decay factor approaches 1; when the standard deviation is large relative to the mean, the decay factor approaches 0, and the grid regularity score decreases accordingly.

[0034] The aforementioned preset threshold of 0.6 is based on statistical analysis of the regularity of facades for different building types. This value can distinguish between regular bays with stable repetition cycles and irregular bays caused by corner units, large windows in ground-floor shops, or added components. When the grid regularity score is not lower than 0.6, the grid assumption for that row is deemed usable. The periodic parameters are recorded, including the average spacing between adjacent doors and windows, the average row height, and the average width and height of visible doors and windows. The window sill positions that should exist in the obstructed area are extrapolated according to these periodic parameters to generate a first candidate prediction and associate it with a first confidence level. The source of this first candidate prediction is the extrapolation of the same facade grid, which is used to trace the source of evidence during subsequent weighted fusion.

[0035] In practice, the first confidence level is taken as the raster regularity score of that row. When the raster regularity score is below 0.6, the raster hypothesis for that row is deemed unusable, and the corresponding occluded area is marked as a second source of evidence.

[0036] The evaluation results of the grid hypothesis are output in the form of a parameter table, which includes the availability flag and periodic parameters for each row. For rows that are determined to be available, a mapping table between the occlusion area and the threshold prediction is further output for use in subsequent fusion steps.

[0037] The quantitative evaluation of the grid regularity score can be used to explicitly identify unreliable grid assumptions when severe occlusion leads to an insufficient number of visible doors and windows. This avoids forced extrapolation when the arrangement pattern itself cannot be reliably generalized, and the occluded area is then transferred to the next source of evidence for processing.

[0038] Step S140: Using other facades of the same building as the second source of evidence, in response to the marker of transferring to the second source of evidence, retrieve the window parameters of other facades that are isomorphic to the current facade, migrate them to the current occlusion area after geometric transformation, generate a second candidate prediction and associate it with the second confidence score (Score_transfer). When no transferable reference window is found, mark the corresponding occlusion area as the third source of evidence.

[0039] The inputs for this step include the list of occluded areas marked as transitioning to the second evidence source output in step S130, and the building facade identifiers and their orientation information determined after master plane fitting in step S110. The set of visible doors and windows detected on each facade is provided by the corresponding door and window detection results for each facade. The building facade identifiers include the normal and orientation information of each facade, used to determine the spatial relationship between facades. The set of visible doors and windows consists of the window opening parameters extracted and confirmed by the door and window detectors on each facade.

[0040] In some embodiments, the criteria for determining isomorphism with the current facade are that the facade orientation difference is 90° or 270° and the floor height difference between the facades of the target building is less than the floor height tolerance, or the facades have the same orientation but different floors. The determination of facade isomorphism can also be combined with the facade material classification results. Material classification can employ texture or spectral classification methods known in the art to distinguish differences in door and window configurations on facades of different materials within the same building. The floor height tolerance is set to 0.3m, which matches the bandwidth of the row and column clustering in step S130, ensuring that doors and windows on different facades on the same floor have consistency in the height direction.

[0041] Geometric transformation employs different methods depending on the spatial relationship between the isomorphic facade and the current facade. When the isomorphic facades are at an angle (i.e., the orientation difference is 90° or 270°), the windowsill height and door / window height of the reference doors and windows remain unchanged, while the door / window width and spacing are scaled according to the bay width ratio between the facade to be completed and the reference facade. In practice, the bay width can be measured from the facade outline generated in step S110, and the scaling ratio reflects the difference in bay dimensions between facades with different orientations. When the isomorphic facades are opposite facades (i.e., facades with the same orientation but opposite each other), the door / window width and height of the opposite facade on the same floor are directly used as a reference without geometric scaling, because opposite facades usually have the same bay division method in terms of building function.

[0042] The second confidence level is determined based on the number of independent facades providing references and the consistency of width and height between the references. For example, when searching for transferable reference window openings, the existing visible door and window geometry parameters of the corresponding floor are retrieved within the isomorphic facade class. When there are two or more independent facades in the isomorphic facade class providing consistent references and the difference in width and height between the references is less than 10%, the second confidence level is set to a higher value of 0.8. When there is only one reference facade, the second confidence level is set to a medium value of 0.5. The difference in the second confidence level reflects the improved reliability of multiple facades providing consistent references compared to a single reference. When the isomorphic facade of the corresponding floor itself is also missing window opening parameters due to obstruction, i.e., no transferable reference window openings are found, the corresponding obstructed area is marked as a third source of evidence.

[0043] In this step, the results of cross-facade migration are output in the form of an evidence table. Each record includes the source facade identifier, the geometrically transformed width and height of doors and windows, the predicted location range, and a second confidence level, for use in subsequent steps. For obstructed areas where no migration reference can be obtained, a marker is output indicating that the data has been transferred to a third source of evidence.

[0044] Through the above-mentioned cross-facade semantic evidence transfer, the basis for inferring the obscured area is expanded from a single facade to other facades of the same building. When there are insufficient visible doors and windows on the same facade but there is isomorphic visible evidence on other facades of the building, it is still possible to obtain candidate predictions with geometric constraints, rather than directly determining that it cannot be completed.

[0045] Step S150: Using the statistical prior of similar buildings as the third source of evidence, in response to the label of transferring to the third source of evidence, retrieve the matching statistical prior entries, generate the third candidate prediction after consistency test and associate it with the third confidence level (Score_prior).

[0046] The inputs to this step include the list of obstructed areas marked as being transferred to the third evidence source, output from step S140, and the pre-acquired metadata of the target building. The target building's metadata includes its functional type, construction year range, and number of floors above ground, which can be derived from urban component surveys or planning application data and provided as external input information before the execution of this method. Furthermore, this step relies on a pre-constructed statistical prior library of similar buildings, organized by building functional type, construction year range, and number of floors range. Each entry includes the mean and standard deviation of standard window width, the mean and standard deviation of standard window height, the mean and standard deviation of window sill height, the mean and standard deviation of window spacing, and the sample size.

[0047] In some embodiments, retrieving matching statistical prior entries specifically involves: performing exact matching based on building function type, construction year range, and number of floors; and performing degenerate matching based on function type when no exact match is found. The consistency test is a statistical hypothesis test. The prior is deemed applicable when the measured statistical value of the width of visible doors and windows does not differ significantly from the prior distribution. The third confidence level is determined based on the matching accuracy and the test results. When the consistency test fails, the statistical value of the width sample of visible doors and windows is used to replace the prior parameters to generate a third candidate prediction.

[0048] For example, during prior retrieval, exact matching is prioritized using triples of building function type, construction year range, and number of floors range. If a corresponding entry for this triple exists in the prior database, an exact match is found. If no exact match is found, the search degenerates to matching only by function type. For instance, if an exact match for "residential" is missing, the search degenerates to matches for "residential" but without restrictions on construction year or number of floors. In this case, the matching level is lowered, and the third confidence level is correspondingly reduced.

[0049] After matching a priori entry, the mean and standard deviation of the window width, mean and standard deviation of the window height, and mean of the windowsill height are read. A consistency test is performed between the actual door and window width samples determined in step S130 or S140 within the same building and the prior distribution. When there is no significant difference between the measured samples and the prior distribution, the prior is deemed applicable, and the third confidence level is determined based on the matching accuracy, with the third confidence level for exact matching being higher than that for degenerate matching. When the consistency test fails, it indicates a systematic difference between the door and window component dimensions of the target building and the statistical distribution in the prior library. In this case, the mean and standard deviation of the measured door and window width samples already determined within the building are used to replace the prior parameters, and a new third candidate prediction is generated, with the third confidence level lower than the corresponding value when the prior is applicable.

[0050] In practice, for example, a one-sample t-test can be used to test consistency, with a significance level of 0.05. The prior is considered valid when the measured mean falls within the range of the prior mean plus or minus 1.96 standard deviations. In this case, the third confidence level is 0.4 for exact matches and 0.24 for degenerate matches. When the test fails, the third confidence level is 0.3 for exact matches and 0.18 for degenerate matches. For example, when an exact entry for the function type "residential" is missing, degenerate matching can be performed to entries for residential types, regardless of year or number of floors.

[0051] After completing the verification and parameter determination, a third candidate prediction is generated for the occluded area based on the parameters that have passed the verification or correction. The predicted door and window widths and heights are taken as the corresponding average values, the windowsill height is taken as the a priori or measured average value, and the position is determined according to the row and column positions extrapolated by the grid in step S130. When there is no grid position reference, it is generated according to the uniform bay assumption.

[0052] In this step, the results of the prior matching are output in the form of an evidence table. Each record contains the matching level, third confidence level, predicted door and window width, height and location, which are used for fusion in subsequent steps.

[0053] Through the matching and consistency test of the statistical priors of similar buildings, even when there is insufficient visible evidence on each facade of the building itself, it can still provide a basis for predicting the door and window parameters of the obscured area based on statistical regularities, serving as a fallback basis for inference and avoiding abandoning the inference directly at the point where the chain of evidence breaks.

[0054] Step S160: Using locally visible door and window segments as geometric constraints, the predicted values ​​and confidence levels of each candidate prediction for the same occluded area are corrected, and then weighted and fused according to the corrected confidence levels to obtain the door and window completion parameters and comprehensive confidence level of the occluded area.

[0055] The input for this step includes the first candidate prediction, the second candidate prediction, and the third candidate prediction generated in steps S130, S140, and S150, respectively, as well as the locally visible door and window fragments of each occluded area extracted in step S120. Each candidate prediction includes the prediction center coordinates, prediction width, prediction height, windowsill height, confidence score, and source marker. The source marker is used to trace the type of evidence source on which the candidate prediction is based.

[0056] In some embodiments, using partially visible door and window segments as geometric constraints, the predicted values ​​and confidence levels of each candidate prediction for the same occlusion area are corrected, and then weighted and fused according to the corrected confidence levels. Specifically, the following processing is included: First, alignment and pairing are performed on candidate predictions that overlap in location intervals within the same occlusion area (e.g., overlap rate greater than 50%). For example, by judging the center coordinates and width and height range of each candidate prediction, multiple candidates pointing to the same window location in space are grouped together for unified processing in subsequent correction and fusion.

[0057] Secondly, when a locally visible door / window segment overlaps with a candidate prediction in location, the endpoint coordinates or edge orientation angle of the locally visible door / window segment are used as hard constraints to fine-tune the prediction center coordinates, prediction width, or prediction height of the candidate prediction. This ensures that the distance from the endpoint of the locally visible door / window segment to the boundary of the candidate prediction window opening is less than a preset tolerance, and the confidence level of the candidate prediction is multiplied by a preset boosting factor. The preset tolerance can be set according to the orthophoto resolution, for example, 0.05m, to ensure that the corrected prediction matches the visible segment geometrically. The boosting factor can be a value greater than 1, for example, 1.2, so that the candidate prediction verified by the locally visible segment receives higher weight in subsequent fusion, reflecting the corrective effect of actual observation evidence on the inference results.

[0058] In each candidate prediction, the confidence score reflects the reliability of the evidence upon which the candidate prediction is based, and is calculated from the evidence source step that generates the candidate. As mentioned earlier, the first confidence score is the grid regularity score in step S130, the second confidence score is determined by the number of independent facades provided for reference and the consistency between references in step S140, and the third confidence score is determined by the prior matching accuracy and consistency test results in step S150. In the weighted fusion below, the weight of each candidate prediction is taken as its corrected final confidence score.

[0059] Therefore, each candidate prediction is weighted by its corrected final confidence level. A weighted average formula is used to weight and fuse the prediction center coordinates, prediction width, and prediction height of each candidate prediction, where the weight of each candidate prediction is its corrected final confidence level. The weighted average formula can be expressed as: (2) In expression (2), Indicates the fusion result. Let i be the final confidence level of candidate i after correction. This is the predicted value corresponding to candidate i. Using this method, the door and window completion parameters for the obstructed area can be obtained, including center coordinates, width, height, and windowsill height.

[0060] When the sum of the confidence scores of all candidate predictions within the same occluded area is lower than a preset lower threshold, no deterministic completion result is generated; only an insufficient evidence flag is output. The preset lower threshold can be set to 0.3.

[0061] The overall confidence level output in this step It can be calculated as follows: (3) Where m is the number of candidate predictions participating in the fusion. The final confidence level of the i-th candidate prediction after correction (0≤ ≤1). This calculation method averages the corrected confidence scores of each candidate, ensuring the overall confidence score naturally falls within the range of 0 to 1. The overall confidence score reflects the average confidence level of the predictions from all participating candidate predictions; the higher the confidence score of each candidate, the higher the overall confidence score; conversely, the lower the confidence score of each candidate, the lower the overall confidence score. When only one candidate prediction is available, the overall confidence score equals the confidence score of that candidate, reflecting the reality that the completion result relies on only a single source of evidence and has limited sufficiency of evidence.

[0062] After the weighted fusion is completed, the style parameters of the doors and windows can be further determined. The style parameters include semantic attributes such as window frame style, window sill structure, and door leaf material. These parameters do not participate in the weighted fusion of the above geometric parameters, but are determined by a voting system: the style tag with the highest confidence among all candidates is taken as the final style.

[0063] If the candidate with the highest confidence level comes from a third source of evidence, namely statistical priors of similar buildings, then add a source label to the style tag to indicate that the style parameter is based on statistical priors rather than direct observations, and it is recommended to pay attention to this in subsequent applications.

[0064] The completion results for each occluded area are output in a structured format, including door and window completion parameters, style labels, overall confidence level, and the proportion of evidence sources. The proportion of evidence sources is the percentage of the final confidence level of each candidate prediction in the total confidence level.

[0065] Through the above correction and fusion processing, multiple candidate predictions for the same occluded area are integrated into unified door and window completion parameters. Locally visible fragments provide geometric anchors for fusion, so that the final result takes into account both the statistical advantages of multi-source evidence and the geometric constraints of actual observation.

[0066] Based on step S160, step S170 is performed, in which the door and window completion parameters are back-mapped to the point cloud data based on the bidirectional index table, and consistency verification is performed with the boundary evidence. If the verification fails, the confidence weight of each candidate prediction is adjusted, and weighted fusion is re-executed until the verification is passed or the preset iteration limit is reached. The completion results that pass the verification are divided into confidence levels according to the comprehensive confidence level and then output.

[0067] The inputs for this step include the door and window completion parameters and overall confidence scores for each occluded area output in step S160, the boundary evidence extracted in step S120, and the bidirectional index table and original point cloud data generated in step S110.

[0068] In some embodiments, consistency verification includes the following processing: First, the boundary evidence is expanded by a preset distance to obtain the expanded boundary. The window opening rectangles corresponding to the door / window completion parameters are then compared with the expanded boundary for inclusion. If the area of ​​the window opening rectangle exceeding the expanded boundary exceeds a preset threshold, it is considered a boundary inconsistency. The expansion distance can be set according to the orthographic projection resolution, for example, 0.1m, and the threshold can be set to 15%. Boundary inclusion verification is used to verify whether the geometric range of the completed window opening falls within the acceptable boundary of the occlusion area. Completion results with excessively large excess proportions may exceed the actual occlusion range, resulting in low geometric reliability.

[0069] Secondly, based on a bidirectional index table, the door and window completion parameters are mapped back to the point cloud data, and the percentage of residual point cloud points within the tolerance range before and after the window opening is statistically analyzed. When the percentage of points exceeds a preset residual threshold, it is considered a point cloud inconsistency. The tolerance range before and after is, for example, ±0.15m, and the residual threshold is, for example, 8%. Point cloud inconsistencies indicate that there are still many measured point clouds in the three-dimensional space corresponding to the completed window opening, meaning that the actual location may be a wall surface rather than a window opening, and the completion result conflicts with the original observation data.

[0070] If any condition of boundary inconsistency or point cloud contradiction is triggered, the verification will be deemed unsuccessful.

[0071] When a verification fails, the confidence weights of each candidate prediction for the corresponding occluded area are adjusted. Specifically, the confidence weights of candidate predictions involving locally visible door and window segments are multiplied by a first adjustment coefficient, and the confidence weights of third candidate predictions are multiplied by a second adjustment coefficient. Then, the weighted fusion in step S160 is re-executed. The first adjustment coefficient is greater than 1, for example, 1.5, and the second adjustment coefficient is less than 1, for example, 0.5. This allows locally verified candidate segments to receive higher weights in the re-fusion, while the weights of candidates based on statistical priors are correspondingly reduced. If the completed parameters after re-fusion still fail verification, the weights can be adjusted and fused again until verification is passed or a preset iteration limit is reached. The preset iteration limit is, for example, 2 iterations. Completed results that fail verification after exceeding the iteration limit are downgraded to manual review marks and no longer automatically output deterministic geometric parameters.

[0072] The completed results that pass the verification are then output after being categorized into confidence levels based on their overall confidence. In some embodiments, the confidence levels include three levels: high confidence auto-completion, medium confidence (suggesting sampling), and low confidence (suggesting manual verification), each corresponding to different preset threshold ranges for the overall confidence. For example, an overall confidence level of not less than 0.75 is labeled as high confidence auto-completion, an overall confidence level between 0.45 and 0.75 is labeled as medium confidence (suggesting sampling), and an overall confidence level below 0.45 is labeled as low confidence (suggesting manual verification).

[0073] The completion results include the door and window completion parameters and overall confidence levels of each obscured area. After being combined with the door and window detection results of the visible area, they form a complete building facade door and window model for downstream applications.

[0074] To facilitate understanding of the technical solution of this application, the following explanation will be provided in conjunction with an engineering case study in a specific scenario.

[0075] This embodiment involves a detailed 3D modeling project of a six-story brick-concrete residential building in a city's residential community. The building, constructed in 1998, has an approximately rectangular floor plan, measuring 42 meters east-west and 16 meters north-south, with a floor height of 2.9 meters. The south facade is obstructed by rows of camphor trees within the community's greenbelt, and the north facade suffers from blind spots in the oblique photography view due to a distance of less than 6 meters from adjacent buildings. Both facades exhibit significant geometrical deficiencies in doors and windows. The complete implementation process of the method of this invention is described below: Data Acquisition and Hardware Environment: The project used a Dapeng UAV equipped with a five-lens oblique camera (equivalent focal length 24mm, single-lens resolution 5472×3648 pixels) to perform oblique photogrammetry flight over the building and the surrounding approximately 0.3 square kilometers of the block. The forward overlap was 82%, the lateral overlap was 65%, and the ground resolution (GSD) was 2.8cm. Simultaneously, a Leica RTC360 ground laser scanner was used to set up 9 scanning stations around the building, with a single-station ranging accuracy of ±1mm@10m and a nominal point density of approximately 6000 points / m at a 10-meter ranging distance. 2 The measured point cloud density of the facade after registration and fusion is approximately 850 points / m². 2 A total of 312 close-up images were collected along the walkways surrounding the buildings using a handheld street view acquisition device (5760×3840 pixels resolution). The oblique photogrammetry data underwent aerial triangulation and dense reconstruction using ContextCapture, outputting a dense point cloud of approximately 460 million points and a triangular mesh model (OBJ format, approximately 28 million faces). The TLS point cloud and the oblique photogrammetry point cloud were fused into a unified point cloud through ICP fine registration (initial pose provided by common control points, with a root mean square error of 0.018m after registration). The software environment was the Feidu Technology CIM+ modeling processing platform (Windows Server 2022, dual Intel Xeon Gold 6338 processors, 256GB RAM, NVIDIA RTX A6000×2). Point cloud processing was based on PCL 1.13 and a self-developed C++ module, while the raster detection and fusion inference modules were implemented using Python 3.10.

[0076] The implementation steps include: S201: Using the building's outer contour polygon (output from the previous block reconstruction process, CGCS2000 coordinates, with the four main facade normals being due south, due north, due east, and due west) as the baseline, RANSAC principal plane fitting was performed on the south, north, east, and west facades respectively, with an interior point distance threshold of 0.03m. The fitted interior point percentages for the four facades were 91.2% (south), 88.7% (north), 95.4% (east), and 94.8% (west), respectively. The point cloud of each facade was orthophoto-projected into a depth map and color map with a resolution of 2cm / pixel. The projection size of the south facade was 2100×145 pixels (corresponding to an actual area of ​​42m×2.9m×6 floors), and the north facade had the same size. After PnP+RANSAC pose registration of 312 close-up street scene images, 287 images were successfully registered (success rate 92.0%) with an average reprojection error of 1.1 pixels, achieving pixel-level alignment with the point cloud orthophoto. The remaining 25 images failed to register due to excessive occlusion and fewer than 10 feature points. They were recorded but not included in subsequent fusion.

[0077] S202: Perform occlusion detection on the south facade frontal projection. Calculate the point cloud density using a 2cm×2cm grid, with ρ_min set to 2 points / grid (corresponding to 50 points / m). 2 The data was analyzed, and the depth variance of the 5×5 neighborhood was calculated. The results showed that there were four connected obstruction areas on the south facade with an area exceeding 0.04m². 2 After removing the threshold values, the largest single area is located between the third and fourth floors, with a horizontal range of approximately 8.2 meters (corresponding to two bays), an area of ​​approximately 23.8 square meters, and a depth value 0.6 to 2.1 meters shallower than the main facade plan with a variance of 0.31 meters. 2 The area was identified as being obstructed by vegetation (camphor tree canopy). The Douglas-Peucker algorithm (tolerance 0.03m) was used to extract the canopy outline polygon, reducing the number of vertices to 34. Within a 0.3m expanded neighborhood at the edge of this obstructed area, a residual visible fragment (a 0.42m long right-angled side of the window frame, with an orientation angle perpendicular to the facade normal, indicating high confidence) was detected at the lower left corner of a window opening on the third floor and recorded as local evidence F_1. The north facade suffered from a blind spot due to oblique photography, resulting in a complete lack of depth in the middle section from the third to the fifth floors (approximately 31.5 square meters), with no residual visible fragments; the obstruction type was labeled as "blind spot."

[0078] S203: Perform window row and column clustering on each floor of the south facade. The first floor is a commercial facade with irregular window order, so grid modeling is skipped; among the five floors from the second to the sixth floor, the number of visible windows on the first, second, fifth, and sixth floors are 9, 8, 7, and 8 respectively (the expected number of windows per floor is 10, corresponding to 5 bays × 2 spans), and the standard deviation of the spacing is... The values ​​are 0.04m, 0.05m, 0.09m, and 0.04m respectively, with an average value of All are around 3.6m, calculated according to Formula 1. are 0.87, 0.79, 0.61 and 0.83 respectively, all ≥0.6, so the grids are determined to be available; only 3 window openings are visible on the third floor (7 are missing due to occlusion), the expected number of window openings is 10, =0.24, it is determined that the grid hypothesis of this row is unavailable, and the marking proceeds to S204. 6 window openings are visible on the fourth floor, =0.58, which is close to the critical value, so it is determined to be unavailable, and the process also proceeds to S204. Check whether there is occlusion in the available areas of the second, fifth and sixth floors. After verification, there is no occlusion and missing in the second, fifth and sixth floors, so the grid extrapolation in this case only verifies the effectiveness of the grid model on the non-occluded floors, and no sill position prediction that needs to be output is generated.

[0079] S204: The grids of the third and fourth floors on the south facade are unavailable, so search for isomorphic facades. Since the east and west facades of the building are gable walls (without window sequence), only the south and north facades are isomorphic (with opposite orientations, they are opposite facades rather than corner-related facades; in this case, the proportional transformation of corner bays is not applicable, and according to the empirical rule that "the types of window components on different facades of the same floor are usually consistent", the window width and height of the opposite facade on the same floor are directly used as references without geometric scaling). The corresponding floors of the third and fourth floors on the north facade are also missing due to blind view areas, so the mutual inspection of the north and south facades is both unavailable, Score_transfer=0, and the process proceeds to S205.

[0080] S205: Search the statistical prior database of similar buildings, perform exact matching according to the triplet (building function type = residential building, construction year range = 1990-1999, number of above-ground floors = 6 floors), and hit the entry in the prior database (measured statistics of 214 similar residential buildings in the sample size), the average window width w̄_prior=1.5m, standard deviation s_w=0.12m, the average window height h̄_prior=1.6m, standard deviation s_h=0.10m, the average sill height sill̄_prior=0.9m, κ=1.0 (exact matching). The one-sample t-test is performed on the measured window width samples (24 samples in total, average 1.48m, standard deviation 0.09m) confirmed by S203 on the second, fifth and sixth floors of the building and the prior distribution, t=0.82<t_critical(0.05,23)=2.069, the test passes (the measured average falls within the range of prior ±1.96×s_w=[1.265m,1.735m]), it is determined that the prior is applicable, Score_prior=0.4×1.0=0.4. Based on this, sill position prediction is generated for the missing window openings on the third and fourth floors according to the positions extrapolated by the grid (reusing μ_gap=3.6m horizontal cycle verified on the second floor, combined with the actual starting boundaries of the third and fourth floors respectively), and the predicted width and height adopt w̄_prior=1.5m and h̄_prior=1.6m.

[0081] S206: Fuse the sill position candidates in the occluded area of the third floor. There are three-way candidates on the third floor: S203 grid extrapolation (this row =0.24, deemed unusable and no candidate generated, therefore this path is missing), S204 cross-facade migration (Score_transfer=0, no candidate generated), S205 prior candidate (Score_prior=0.4). Since the occluded area also contains the locally visible fragment F_1 extracted by S202 (the 0.42m window frame edge at the lower left corner of a window on the third floor), fragment correction is performed on the prior candidate S205: the coordinates of the predicted lower left corner of the window opening are aligned with the endpoint of F_1, and after fine-tuning, the center coordinates are offset by 0.06m, and the width is fine-tuned from 1.5m to 1.46m to match the F_1 orientation angle constraint. After correction, the confidence enhancement coefficient λ=1.2, so the final confidence of this candidate S_205'=0.4×1.2 = 0.48. For the remaining six obstructed window sill locations on the third floor (excluding the F_1 covered location), only the S205 prior candidate (Score_prior=0.4, no fragment correction) has a confidence level of 0.4, which is greater than τ_low=0.3, so it is adopted, and the output predicted width and height are the prior values ​​of 1.5m×1.6m. The situation for the seven sill locations in the obstructed area on the fourth floor is similar to that on the third floor. One of the sill locations has a locally visible fragment (a straight line on the windowsill, 0.35m long), with a corrected confidence level of 0.48, while the other six have a confidence level of 0.4.

[0082] S207: Perform a geometric consistency check on all 13 completed windows output by S206 (7 for three layers, 7 for four layers, and 13 in total after removing duplicate counts corresponding to F_1). Boundary inclusion verification: All 13 completed window rectangles fall within the corresponding occlusion area expansion boundary (expansion 0.1m), and the proportion of the area exceeding the boundary is less than 15% of the threshold, so all passed; Point cloud residual verification: The point cloud is back-mapped to the original point cloud to count the proportion of points within ±0.15m. Among them, the residual point proportion of one threshold (the second one on the fourth floor) reaches 11.3%, which exceeds the threshold of ξ=8%, and is judged as "point cloud contradiction" (this point was verified to be an air conditioner outdoor unit installation hole rather than a normal window hole), triggering feedback re-inference - increasing the weight of the locally visible fragment S202 in this area to λ=1.5, and reducing the prior weight of S205 to 0.5×0.4=0.2 for re-fusion. Since there is no locally visible fragment in this area that can be used for re-anchoring, the overall confidence level drops to 0.2 after the first iteration, which is lower than τ_low=0.3. The second iteration still cannot improve it. Finally, this threshold is downgraded and marked as "manual verification". The geometry is not automatically output for the time being, and only the occlusion area mark is retained. The remaining 12 completed window openings passed the verification: 2 of them (corresponding to F_1 and the local fragment positions on the fourth floor) had a comprehensive confidence score of 0.48, marked as "medium confidence, random sampling recommended"; the remaining 10 had a comprehensive confidence score of 0.4, also marked as "medium confidence, random sampling recommended" (not reaching the high confidence threshold of 0.75, because the evidence sources were all prior fallbacks rather than direct observations). Due to the lack of any locally visible fragments on the north facade and the similar lack of evidence on the same floor of the south facade (mutually empty evidence), all 10 threshold positions in the middle section of the third to fifth floors could only obtain prior candidates with a score_prior=0.4, with a comprehensive confidence score of 0.4, also marked as "medium confidence, random sampling recommended". No candidates were judged as "low confidence" or triggered manual review, because the prior matching itself was valid and there were no contradictory point cloud residuals.

[0083] Processing time and results statistics: In this embodiment, the south and north facade occlusion detection (S202) takes about 18 seconds per facade, grid modeling and scoring (S203) takes about 6 seconds per facade, cross-facade migration retrieval (S204) takes about 3 seconds, prior library matching and consistency check (S205) takes about 2 seconds, multi-source weighted fusion (S206) with 13 thresholds takes less than 1 second in total, geometric consistency check and one feedback iteration (S207) takes about 9 seconds, and the total time for the entire process (excluding the initial reconstruction in S101) for a single building is about 47 seconds. Ultimately, a total of 23 window openings were completed on the south and north facades of the residential building. Among them, 22 were output with deterministic geometric parameters and labeled with confidence levels (2 medium-confidence segment correction results and 20 medium-confidence pure prior results), and 1 was downgraded to a manual verification mark. Compared with the previous method of manually checking each building of obstructed facade window openings used in this project (the original manual verification of a single six-story residential building of this type took an average of about 25 minutes), the method of this invention can reduce the time for automated processing of the deterministic output part to less than 1 minute, and accurately narrow the scope of manual verification to a single suspected component with geometric inconsistencies, rather than manually confirming all 23 obstructed window openings one by one. Theoretical analysis expects that in the scenario of batch modeling of similar multi-story residential communities, it can significantly reduce the proportion of manual verification time per unit building.

[0084] Based on the above embodiments, compared with the prior art, the present invention has the following beneficial effects: (a) This invention refines the binary judgment of whether a whole facade grid is valid into a row-by-row judgment through a grid-based scoring mechanism. For usable grid assumptions, window opening positions are extrapolated periodically. For unusable parts, cross-facade semantic evidence transfer is explicitly performed. When evidence within the same building is insufficient, it is further transferred to a statistical prior library of similar buildings for matching, forming a three-level evidence supplementation mechanism of "same facade grid → cross-facade transfer → statistical prior." This ensures that even when the same facade grid is unavailable, obstructed areas can still obtain evidence-based completion results, preventing the overall process from being interrupted due to the failure of a single evidence source. In severely obstructed scenarios, the completion rate of door and window areas in this invention is significantly improved compared to methods relying solely on single facade grid extrapolation, avoiding the brittle failure problem of "directly judging failure when there is no grid to summarize."

[0085] (b) This invention explicitly extracts the polygonal outline of the occluded object as boundary evidence and extracts locally visible door and window fragments within the expanded neighborhood of the occluded region, upgrading the "region to be completed" from pixel holes to a structured object with geometric boundary constraints. Using the endpoint coordinates and edge direction angles of the locally visible door and window fragments as hard constraints, the prediction center coordinates, prediction width, or prediction height of the candidate predictions are fine-tuned, ensuring that the distance from the endpoints of the local fragments to the candidate prediction boundary is less than a preset tolerance, thus subjecting the completion result to the constraints of actual visible geometry. Based on this, the completion result is further compared with the expanded boundary of the boundary evidence for inclusion, and a residual point cloud conformity check is performed based on a bidirectional index table and reverse-mapped to the original point cloud. If both checks fail, feedback re-inference is triggered; if inconsistent, the weights are automatically adjusted and re-fused. By extracting boundary evidence and locally visible fragments as verifiable evidence, the completion result is traceable, and the boundary position error can be controlled within a preset tolerance range. Compared to generative methods without geometric constraints, this approach can detect and correct completion results that contradict actual observations earlier.

[0086] (c) This invention takes the average of the final confidence scores of all candidate predictions after correction as the comprehensive confidence score. The comprehensive confidence score naturally falls in the range of 0 to 1, reflecting the improvement in the reliability of the completion result due to mutual corroboration of multiple sources of evidence. The completion results are divided into three levels according to the comprehensive confidence score: high confidence score automatic completion, medium confidence score suggested sampling inspection, and low confidence score suggested manual verification. The source of evidence is recorded in the style label (e.g., "Source: Statistical Prior"). Downstream modelers can allocate quality inspection resources differently according to the confidence score level, concentrating the workload of manual review in the low confidence score area. Compared with the traditional operation method of sampling all completion results indiscriminately, this can significantly reduce the proportion of unnecessary manual verification.

[0087] (d) This invention establishes a cross-facade semantic evidence transfer mechanism, which transfers isomorphic door and window parameters from other facades of the same building to the current area to be completed through geometric transformation; it also establishes a statistical prior library matching and consistency verification mechanism for similar buildings, using statistical priors as a fallback when internal building evidence is insufficient, and avoiding systematic bias caused by mechanically applying priors through consistency verification. This expands the scope of evidence utilization from a single facade to the entire building and even the statistical patterns of similar buildings. Even in scenarios with poor long-term observation conditions, such as the rear facade, it can still provide evidence-based geometric results superior to those marked as "unknown," significantly improving the component completeness rate of the building's LoD3 level refined model.

[0088] In summary, this invention achieves a transformation from pixel-level generative completion to verifiable geometric inference through a complete closed loop of "boundary evidence extraction → multi-level evidence supplementation → weighted fusion → reverse verification → confidence leveling". This makes the completion results verifiable, gradable and traceable, providing reliable technical support for city-level real-scene 3D modeling and digital twin base production.

[0089] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0090] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0091] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0092] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for automatically completing building facade doors and windows based on occlusion evidence, characterized in that, Includes the following steps: Acquire multi-source facade data, perform main plane fitting and orthographic projection on each facade, and generate frontal projection map, point cloud density map and bidirectional index table between the frontal projection map and the point cloud data for each facade. Based on the orthographic projection map and the point cloud density map, the occlusion area is detected, the boundary contour of the occlusion area is extracted as boundary evidence, and locally visible door and window fragments are extracted in the expansion neighborhood of the occlusion area. Using the visible doors and windows in the frontal projection as the first source of evidence, when the grid regularity score of the visible doors and windows is not lower than a preset threshold, the window opening position in the occluded area is extrapolated according to the arrangement pattern of the visible doors and windows, a first candidate prediction is generated and associated with the first confidence level; otherwise, the corresponding occluded area is marked as the second source of evidence. Using other facades of the same building as the second source of evidence, in response to the mark of transferring to the second source of evidence, the window parameters of other facades that are isomorphic to the current facade are retrieved, and after geometric transformation, they are migrated to the current occlusion area to generate a second candidate prediction and associate it with a second confidence level. When no migrateable reference window is found, the corresponding occlusion area is marked as transferring to the third source of evidence. Using statistical priors of similar buildings as the third source of evidence, in response to the marker of transferring to the third source of evidence, matching statistical prior entries are retrieved, and after consistency testing, a third candidate prediction is generated and associated with a third confidence level. Using the locally visible door and window segments as geometric constraints, the predicted values ​​and confidence levels of each candidate prediction for the same occluded area are corrected, and then weighted and fused according to the corrected confidence levels to obtain the door and window completion parameters and comprehensive confidence level of the occluded area. Based on the bidirectional index table, the door and window completion parameters are back-mapped to the point cloud data, and consistency verification is performed with the boundary evidence. If the verification fails, the confidence weight of each candidate prediction is adjusted, and the weighted fusion is re-executed until the verification is passed or the preset iteration limit is reached. The completion results that pass the verification are divided into confidence levels according to the comprehensive confidence level and then output.

2. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The multi-source facade data includes at least point clouds and images. The main plane fitting adopts the RANSAC algorithm. The resolution of the orthographic projection is 2cm / pixel. The geometric registration error between the orthographic projection and the point cloud data does not exceed ±0.05m.

3. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The detection obstruction area includes: The frontal projection map and the point cloud density map are traversed grid by grid. Occlusion determination is performed. Grids that meet the conditions of point cloud density being lower than the density threshold, depth value variance being greater than the variance threshold, or missing depth values ​​are marked as occlusion candidates. Connectivity component marking is performed on the occlusion candidates and isolated candidates with an area smaller than the area threshold are removed to obtain the occlusion region. The density threshold, variance threshold, and area threshold are all preset values.

4. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The step of extracting the boundary contour of the occluded region includes: When the depth value of the occluded area exists in the frontal projection and the difference between the depth of the occluded area and the depth of the main plane of the facade where the occluded area is located is greater than the difference threshold, the occluded area is determined to be the occluded object itself. The Douglas-Peucker algorithm is used to extract the two-dimensional contour polygon of the occluded area in the frontal projection with a preset tolerance as the boundary evidence. When the depth value is missing, or when the depth value exists and the depth difference with the corresponding facade main plane is not greater than the difference threshold, the outer boundary of the occluded area is used as the boundary evidence.

5. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The preset threshold is 0.6; The calculation of the grid regularity score includes: performing row and column clustering on the visible doors and windows to obtain the actual number of detected window openings in each floor row, the expected number of window openings estimated based on the facade width, and the standard deviation and mean of the distance between adjacent windows. The grid regularity score is equal to the ratio of the actual number of detected window openings to the expected number of window openings multiplied by the distribution attenuation factor calculated based on the standard deviation and mean.

6. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The criteria for determining whether a building is isomorphic to the current facade are: the facade orientation difference is 90° or 270° and the floor height difference between each facade of the target building is less than the floor height tolerance, or the facades have the same orientation but different floors. The geometric transformation includes: when the isomorphic facades are in a corner relationship, the windowsill height and door / window height of the reference doors and windows remain unchanged, and the door / window width and door / window spacing are scaled according to the ratio of the width of the facade to be completed to the width of the reference facade; when the isomorphic facades are in a face-to-face relationship, the door / window width and height of the opposite facade on the same floor are directly used as a reference. The second confidence level is determined based on the number of independent facades that provide references and the consistency of width and height among the references.

7. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The statistical prior entries for the retrieval matching are specifically as follows: exact matching is performed according to the functional type, construction year range, and number of floors of the building; when there is no exact match, degenerate matching is performed according to the functional type. The consistency test is a statistical hypothesis test. When the measured statistical value of the width of the visible doors and windows is not significantly different from the prior distribution, the prior is deemed applicable. The third confidence level is determined based on the matching accuracy and the test result. When the consistency test fails, the statistical value of the width sample of the visible doors and windows is used to replace the prior parameter to generate the third candidate prediction.

8. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The step of using the locally visible door and window segments as geometric constraints to correct the predicted values ​​and confidence levels of each candidate prediction for the same occlusion area, and then performing weighted fusion based on the corrected confidence levels, includes: Alignment pairing is performed on candidate predictions that overlap in location intervals within the same occlusion area; When the locally visible door and window segment overlaps with the candidate prediction in position, the endpoint coordinates or edge direction angle of the locally visible door and window segment are used as hard constraints to fine-tune the prediction center coordinates, prediction width, or prediction height of the candidate prediction, so that the distance from the endpoint of the locally visible door and window segment to the boundary of the candidate prediction window opening is less than a preset tolerance, and the confidence of the candidate prediction is multiplied by a preset boosting factor. The prediction center coordinates, prediction width and prediction height of each candidate prediction after correction are weighted and fused using a weighted average formula, where the weight of each candidate prediction is its final confidence level after correction.

9. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 1, characterized in that, The consistency check includes: The window opening rectangle corresponding to the door and window completion parameters is compared with the expansion boundary of the boundary evidence. When the area of ​​the window opening rectangle exceeding the expansion boundary exceeds a threshold, it is determined to be a boundary inconsistency. Based on the bidirectional index table, the door and window completion parameters are mapped in reverse to the point cloud data. The percentage of residual point cloud points within the tolerance range before and after the window opening plane is counted. When the percentage of points exceeds the residual threshold, it is determined to be a point cloud contradiction. The inconsistency of the boundaries or the contradiction of the point cloud triggered a failure to pass the verification.

10. The method for automatically completing building facade doors and windows based on occlusion evidence inference according to claim 9, characterized in that, The adjustment of the confidence weights of each candidate prediction includes: The confidence weights of the candidate predictions for the locally visible door and window segments in the corresponding occluded area are multiplied by a first adjustment coefficient, and the confidence weights of the third candidate predictions are multiplied by a second adjustment coefficient. Then, the weighted fusion is performed again.

Images