Window detection and completion method and system based on spatial distribution guidance

Abstract

The application belongs to the technical field of computer vision, and discloses a window detection and completion method and system based on spatial distribution guidance, which comprises the following steps: acquiring a first image of a target building, detecting the first image based on a target detection algorithm, and outputting a second image containing the boundary boxes of all windows; clustering the boundary boxes, generating a grid structure in the second image based on the clustering result, and obtaining a third image; traversing the grid structure, obtaining the blank part of the grid structure, generating a new boundary box to complete the blank part, and obtaining a fourth image; optimizing and verifying the boundary boxes in the fourth image, merging the verified boundary boxes with the boundary boxes in the second image, and outputting a fifth image containing the window detection and completion result of the target building; and the application can obtain a high-precision and complete window detection result.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for window detection and completion based on spatial distribution guidance. Background Technology

[0002] Window detection is a key step in building facade construction analysis and semantic segmentation, and it is of great significance for subsequent steps such as building modeling, energy consumption assessment and style reconstruction.

[0003] Currently, the mainstream window detection methods are based on deep learning-based object detection algorithms, such as the YOLO series, Faster R-CNN, and RetinaNet. These methods achieve automatic detection of window objects by training convolutional neural networks on large-scale architectural image datasets.

[0004] However, traditional object detection methods rely solely on local feature recognition, making them prone to missing detections in real architectural images. They ignore the global structural information of windows, which typically exhibit a regular grid pattern in building facades. Structural priors are crucial for completing missed windows, and conventional detection models cannot automatically infer this pattern. While existing research attempts to complete windows through image segmentation or pattern matching, these methods largely rely on manual thresholding or template matching, making them difficult to adapt to the diverse distribution patterns of different buildings. Furthermore, due to the lack of a joint verification mechanism with the detection model, the completed results are prone to introducing erroneous windows.

[0005] Existing technologies still have insufficient capabilities in detecting and supplementing building facade windows; therefore, existing technologies need further improvement. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a window detection and completion method and system based on spatial distribution guidance, in order to solve the problem of low fusion accuracy in existing window detection and completion methods.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows:

[0008] In a first aspect, the present invention provides a window detection and completion method based on spatial distribution guidance, comprising:

[0009] A first image of the target building is acquired, and the first image is detected based on the target detection algorithm to output a second image containing the bounding boxes of all windows.

[0010] Cluster the bounding boxes, and generate a mesh structure in the second image based on the clustering results to obtain the third image;

[0011] Traverse the mesh structure to obtain the empty parts of the mesh structure, generate new bounding boxes to fill the empty parts, and obtain the fourth image;

[0012] The bounding boxes in the fourth image are optimized and verified. The verified bounding boxes are then merged with the bounding boxes in the second image to output a fifth image containing the window detection and completion results of the target building.

[0013] In one implementation, the step of detecting the first image based on an object detection algorithm and outputting the bounding boxes of all windows in the first image includes:

[0014] The first image is normalized and subjected to elevation geometry correction to obtain a corrected image;

[0015] The corrected image is input into a pre-trained target detection model, which outputs the detection boxes of all windows in the first image and the confidence scores of the detection boxes.

[0016] Overlapping detection boxes are removed based on the nonmaximum suppression method, and detection boxes with confidence scores lower than a preset confidence threshold are discarded.

[0017] The remaining detection boxes are used as bounding boxes, and a second image containing the bounding boxes of all windows is output.

[0018] In one implementation, clustering the bounding boxes and generating a mesh structure in the second image based on the clustering results to obtain a third image includes:

[0019] Calculate the center point coordinates, width, and height of the bounding box;

[0020] Based on the center point coordinates, width, and height of the bounding box, one-dimensional clustering is performed on the bounding box to obtain row clusters and column clusters;

[0021] Based on the row centers of the row clusters and the column centers of the column clusters, calculate the row spacing and column spacing of the grid;

[0022] Based on the row spacing and column spacing, a grid structure is generated in the second image to obtain the third image.

[0023] In one implementation, the step of performing one-dimensional clustering on the bounding box based on the center point coordinates, width, and height to obtain row clusters and column clusters includes:

[0024] Calculate the interquartile range of the bounding box based on its width and height;

[0025] Calculate the size range based on the interquartile range, and remove bounding boxes whose width and height do not meet the size range.

[0026] Calculate the average width and average height of the remaining bounding boxes;

[0027] Based on the center point coordinates of the bounding box, the average width, and the average height, one-dimensional row clustering and one-dimensional column clustering are performed on the remaining bounding boxes in sequence to obtain row clusters and column clusters.

[0028] In one implementation, traversing the mesh structure to obtain the missing parts of the mesh structure, generating new bounding boxes to fill the missing parts, and obtaining a fourth image includes:

[0029] Map the bounding box to the mesh structure;

[0030] Traverse the mesh structure to obtain the empty parts of the missing bounding boxes in the mesh structure;

[0031] Based on the neighboring bounding boxes of the missing part, a new bounding box is generated to fill the missing part, resulting in the fourth image.

[0032] In one implementation, generating a new bounding box to fill in the missing portion based on the neighboring bounding boxes of the missing portion to obtain a fourth image includes:

[0033] Based on the traversal results of the mesh structure, calculate the adjacent bounding boxes of the missing parts;

[0034] Based on the neighboring bounding boxes, new candidate boxes are generated, and the size and position information of the new candidate boxes are calculated.

[0035] Perform geometric consistency filtering on new candidate boxes, and retain candidate boxes whose intersection-union ratio with neighboring bounding boxes is within a preset intersection-union ratio threshold;

[0036] The retained candidate boxes are used as new bounding boxes, and the missing parts in the third image are filled in with the new bounding boxes to obtain the fourth image.

[0037] In one implementation, optimizing and verifying the bounding boxes in the fourth image, merging the verified bounding boxes with the bounding boxes in the second image, and outputting a fifth image containing the window detection and completion results of the target building includes:

[0038] The bounding boxes in the fourth image are optimized and verified using a pre-configured refined verification model.

[0039] The verified bounding box is merged with the bounding box in the second image to obtain a fifth image containing the window detection and completion results of the target building;

[0040] Output the fifth image.

[0041] Secondly, the present invention provides a window detection and completion system based on spatial distribution guidance, comprising:

[0042] The image detection module is used to acquire a first image of the target building, detect the first image based on the target detection algorithm, and output a second image containing the bounding boxes of all windows.

[0043] A mesh generation module is used to cluster the bounding boxes and generate a mesh structure in the second image based on the clustering results to obtain a third image;

[0044] The window completion module is used to traverse the grid structure, obtain the missing parts of the grid structure, generate new bounding boxes to complete the missing parts, and obtain the fourth image.

[0045] The verification output module is used to optimize and verify the bounding boxes in the fourth image, merge the verified bounding boxes with the bounding boxes in the second image, and output a fifth image containing the window detection and completion results of the target building.

[0046] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, the memory storing a window detection and completion program based on spatial distribution guidance, wherein the window detection and completion program based on spatial distribution guidance, when executed by the processor, is used to implement the operation of the window detection and completion method based on spatial distribution guidance as described in the first aspect.

[0047] Fourthly, the present invention also provides a computer-readable storage medium storing a window detection and completion program based on spatial distribution guidance, wherein the window detection and completion program based on spatial distribution guidance, when executed by a processor, is used to implement the operation of the window detection and completion method based on spatial distribution guidance as described in the first aspect.

[0048] The present invention, by employing the above technical solution, has the following effects:

[0049] This invention uses an object detection algorithm to detect the first image, obtaining bounding boxes containing all windows. The bounding boxes of the fourth image are then optimized and validated. A multi-stage coupled detection framework is employed. The initial detection stage uses object detection models with different architectures, while the validation stage uses an independent detection model different from the initial model. This allows for cross-model verification, effectively reducing the false detection rate during the completion process and enhancing the system's robustness and generalization ability in complex scenarios. By generating a mesh structure and utilizing the grid-like distribution of windows on building facades, the window distribution structure of the building is automatically inferred, effectively restoring the spatial relationships of missing windows and significantly improving the structural integrity and geometric consistency of the detection results. For missing parts, an adaptive size calculation and candidate box filtering mechanism is applied. Based on the statistical characteristics of local neighboring windows, the width and height of candidate boxes are adaptively calculated, and candidate boxes with large positional deviations or abnormal sizes are eliminated, thereby improving the accuracy and stability of the completion results. In summary, this invention can obtain high-precision and complete window detection results for building facades. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0051] Figure 1 This is a flowchart of the window detection and completion method based on spatial distribution guidance in this invention.

[0052] Figure 2 This is an exterior view of the target building in one implementation of the present invention.

[0053] Figure 3 This is a schematic diagram of the facade identification of a target building that includes the bounding box of all windows in one implementation of the present invention.

[0054] Figure 4 This is a schematic diagram of the mesh structure generated in one implementation of the present invention.

[0055] Figure 5 This is a schematic diagram of target building facade recognition that includes a grid structure in one implementation of the present invention.

[0056] Figure 6 This is a schematic diagram of building facade recognition that includes detecting bounding boxes, mesh structures, and completing bounding boxes in one implementation of the present invention.

[0057] Figure 7This is a refined schematic diagram of building facade recognition, including detection bounding boxes, mesh structures, and completed bounding boxes, in one implementation of the present invention.

[0058] Figure 8 This is a building facade recognition image output in one implementation of the present invention, which includes the final window detection and completion results.

[0059] Figure 9 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0060] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0062] Exemplary methods

[0063] Traditional object detection methods rely solely on local feature recognition, making them prone to missing detections in real-world architectural images. They ignore the global structural information of windows, which typically exhibit a regular grid pattern on building facades. Structural priors are crucial for completing missed windows, a pattern that conventional detection models cannot automatically infer. While existing research attempts to complete windows through image segmentation or pattern matching, these methods largely rely on manual thresholding or template matching, making them ill-suited for the diverse distribution patterns of different buildings. Furthermore, the lack of a joint verification mechanism with the detection model easily introduces erroneous windows into the completed results.

[0064] To address the above technical problems, this invention provides a window detection and completion method based on spatial distribution guidance, comprising: acquiring a first image of the target building; detecting the first image based on a target detection algorithm to output a second image containing bounding boxes of all windows; clustering the bounding boxes; generating a grid structure in the second image based on the clustering results to obtain a third image; traversing the grid structure to obtain the missing parts of the grid structure; generating new bounding boxes to complete the missing parts to obtain a fourth image; optimizing and verifying the bounding boxes in the fourth image; merging the verified bounding boxes with the bounding boxes in the second image to output a fifth image containing the window detection and completion results of the target building; this invention can obtain high-precision and complete window detection results.

[0065] like Figure 1 As shown, this embodiment of the invention provides a window detection and completion method based on spatial distribution guidance, including the following steps:

[0066] Step S100: Obtain a first image of the target building, perform target detection on the first image based on the target detection algorithm, and output a second image containing the bounding boxes of all windows.

[0067] Specifically, in one implementation of this embodiment, step S100 includes the following steps:

[0068] Step S101: Obtain the first image of the target building.

[0069] It should be noted that window detection is a key step in building facade structure analysis and semantic segmentation. Therefore, the first image obtained in this embodiment is the exterior facade image of the building, and the obtained exterior facade image of the building must contain windows. Window detection and completion results cannot be obtained from exterior facade images of buildings without windows.

[0070] like Figure 2 The image shown is the exterior view of the target building obtained in this embodiment, i.e., the first image of the target building.

[0071] Step S102: Normalize and perform facade geometry correction on the first image to obtain a corrected image.

[0072] In this embodiment, the first image is normalized in size and corrected using perspective transformation or elevation correction algorithms. In elevation correction, the corner coordinates (x, y) of the first image belong to the image pixel coordinate system, which is as follows: origin: top left corner of the image; x: increasing to the right; y: increasing downwards.

[0073] Specifically, using manual or automatic extraction methods, the four corner points of the building's facade are extracted from the first image and represented as follows: Each corner point In homogeneous coordinates, it is represented as After homography matrix After transformation ( (representing the real number field), to obtain the corresponding point on the target plane (orthogonal plane). ,Right now:

[0074] ;

[0075] Then, it is restored to non-homogeneous coordinates, that is, to actual pixel coordinates, in order to achieve facade geometry correction. The corresponding formula is as follows:

[0076] ;

[0077] ;

[0078] in, . , These are the first two components of the homogeneous coordinates of the point after perspective transformation, which have not yet been normalized. It is the third component of the homogeneous coordinate vector after perspective transformation, and is used as a scale factor. and It is the two-dimensional non-homogeneous coordinates (actual pixel coordinates) of the point in the target orthophoto plane obtained by normalizing the homogeneous coordinates.

[0079] It should be noted that the formulas contain subscripts. This is the standard notation for describing homography matrices in the fields of computer vision and photogrammetry. It has no independent physical meaning; its significance lies solely in its role as a whole in forming a 3×3 homography matrix. This matrix describes the projective mapping relationship from one plane to another.

[0080] Step S103: Input the corrected image into the pre-trained target detection model and output the detection boxes of all windows in the first image and the confidence scores corresponding to the detection boxes.

[0081] In this embodiment, the pre-trained object detection model is a convolutional neural network-based model, such as the YOLO series. It performs preliminary window detection on the input first image and outputs the bounding boxes of all detected windows in the first image, along with their corresponding confidence scores. The model uses a deep convolutional neural network (CNN) to perform multi-scale convolutions on the input image, extracting significant features such as window edges, textures, and linear structures from the building facade. The YOLO backbone outputs multi-level feature maps, enabling the model to simultaneously focus on large-scale and small-scale windows. On the feature maps, the model predicts a set of candidate bounding boxes and their parameters (center point, width, and height) for each location. By learning from a large amount of labeled window data, the model can regress the bounding boxes most likely corresponding to the window locations from texture features and geometric structures.

[0082] It should be noted that the object detection model can also be RetinaNet, Faster R-CNN, DETR or other object detection models based on the Transformer architecture, in order to adapt to different hardware platforms and accuracy and performance requirements.

[0083] Step S104 involves removing overlapping detection boxes using a non-maximum suppression method and discarding detection boxes with confidence scores below a preset confidence threshold. Specifically, first, detection boxes are sorted from highest to lowest confidence score, and the highest-confidence detection box is selected as the retained box. Then, the Intersection over Union (IoU) between the selected box and the remaining candidate boxes is calculated. If the IoU between a candidate box and the retained box exceeds the IoU threshold, it is considered a duplicate detection and is suppressed. The formula for calculating the IoU between detection boxes A and B is:

[0084] ;

[0085] Represented as a detection box The area.

[0086] It should be noted that this embodiment removes duplicate detection boxes using a non-maximum suppression method and discards detection boxes with a confidence level lower than a preset confidence threshold.

[0087] In this embodiment, the confidence threshold is set to 0.2 and the IoU threshold is set to 0.25.

[0088] Step S105: Use the remaining detection boxes as bounding boxes and output a second image containing the bounding boxes of all windows.

[0089] like Figure 3 The image shown is a schematic diagram of the facade recognition of a target building containing bounding boxes of all windows, i.e., the second image. By using the remaining detection boxes as window bounding boxes, a second image containing bounding boxes of all windows is output. The coordinate information, confidence score, and corresponding window category of all bounding boxes in the second image are also output for spatial distribution analysis. For example, Figure 3 The data only includes the window category. The corresponding category for the window bounding box is the green window detection box, and the green number part is the corresponding confidence level. The position of the bounding box can be determined based on the position of the green window detection box.

[0090] like Figure 1 As shown, this embodiment of the invention provides a window detection and completion method based on spatial distribution guidance, including the following steps:

[0091] Step S200: Cluster the bounding boxes, generate a grid structure in the second image based on the clustering results, and obtain the third image.

[0092] It should be noted that clustering the bounding boxes, that is, using the window bounding box detection results in step S100 to infer the window spatial distribution pattern of the building facade, can automatically restore the grid arrangement pattern of the building facade and provide structural priors for subsequent missing window detection boxes.

[0093] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0094] Step S201: Calculate the center point coordinates, width, and height of the bounding box.

[0095] In this embodiment, the coordinate information of all bounding boxes output in step S100 is obtained, and the center point coordinates of all bounding boxes are calculated based on the obtained coordinate information. ,width and height .

[0096] Specifically, the coordinates of the top-left corner of the current bounding box are defined as follows: The coordinates of the lower right corner are The formula for calculating the coordinates of the corresponding center point is as follows:

[0097] ;

[0098] ;

[0099] The formula for calculating the corresponding width is as follows:

[0100] ;

[0101] The formula for calculating the corresponding height is as follows:

[0102] ;

[0103] It should be noted that, in addition to using the coordinates of the top left and bottom right corners to calculate the center point coordinates, width, and height of the bounding box, another set of opposite corners, such as the bottom left and top right corners, can also be used. The calculation formula needs to be adjusted according to the selected opposite corners.

[0104] Step S202: Based on the center point coordinates, width, and height of the bounding box, perform one-dimensional clustering on the bounding box to obtain row clusters and column clusters.

[0105] In one implementation of this embodiment, step S202 further includes the following steps:

[0106] Step S202a: Calculate the interquartile range of the bounding box based on the width and height of the bounding box.

[0107] Specifically, for all the window bounding boxes obtained in the second image, there is a set of widths of the bounding boxes. and high set Calculate the 25th percentile of the width and height sets respectively. This indicates that 25% of the data in the width or height set is less than or equal to this value, reflecting the typical range of "smaller size" window frames; 75th percentile This indicates that 75% of the data in the width or height set is less than or equal to this value, reflecting the typical range of "larger size" window frames. (Based on the 25th percentile) and the 75th percentile Calculate the interquartile range of the bounding box The formula is as follows:

[0108] .

[0109] Step S202b: Calculate the size range based on the interquartile range, and remove bounding boxes whose width and height do not meet the size range.

[0110] Specifically, based on box plot rules, the acceptable size range for the bounding box is defined as follows:

[0111] ;

[0112] Based on the defined size range, the bounding box of a width and height that do not meet the size range is considered an outlier and is removed.

[0113] Step S202c: Calculate the average width and average height of the remaining bounding boxes.

[0114] It should be noted that the average width of the remaining bounding box is calculated. and average height The remaining wide set after removing outlier bounding boxes can be used. and high set Direct calculation is possible; however, the overall difference in window size and type on the same building facade will not be too large, so it can also be calculated using a wide set. and high set The median is used to obtain the reliable average width of the remaining bounding boxes. and average height .

[0115] In step S202d, based on the center point coordinates of the bounding box, the average width, and the average height, one-dimensional row clustering and one-dimensional column clustering are performed on the remaining bounding boxes in sequence to obtain row clusters and column clusters.

[0116] In this embodiment, the set of center points of the bounding box is obtained based on the coordinates of the center points of the bounding box. for:

[0117] ;

[0118] in and These are the horizontal and vertical coordinates of the center of the detection box, respectively.

[0119] Move all center points in the set of center points of the bounding box along... and The direction is sorted and one-dimensional row clustering is performed. In this embodiment, one-dimensional clustering includes one-dimensional row clustering and one-dimensional column clustering.

[0120] In this embodiment, one-dimensional column clustering is performed. First, the horizontal distance threshold is defined as:

[0121] ;

[0122] Set of center points Points in Sort the sequences in ascending order to obtain the following sequence:

[0123] ;

[0124] The horizontal clusters (column clusters) are defined as follows:

[0125] ;

[0126] When the distance between two adjacent points exceeds a threshold, a new clustering process begins, i.e.:

[0127] At that time, new column clusters are created. ;

[0128] The mean of each column's clusters is the center position of that column:

[0129] .

[0130] To perform one-dimensional row clustering, the vertical distance threshold is first defined as:

[0131] ;

[0132] Set of center points Points in Sort the sequences in ascending order to obtain the following sequence:

[0133] ;

[0134] Vertical clusters (row clusters) are defined as follows:

[0135] ;

[0136] When the distance between two adjacent points exceeds a threshold, a new clustering process begins, i.e.:

[0137] At that time, new column clusters are created. ;

[0138] The mean of each column's clusters is the center position of that column:

[0139] .

[0140] Step S203: Calculate the row spacing and column spacing of the grid based on the row center of the row cluster and the column center of the column cluster.

[0141] In this embodiment, row clusters are obtained. The center of line Clustering column center Sort them according to coordinates and calculate the row spacing of the grid. and column spacing .

[0142] The obtained row center set was identified as The column center set is .

[0143] Sort all row centers in the row center set in ascending order of their coordinates, and we have:

[0144] ;

[0145] Sort all column centers in the column center set in ascending order of their coordinates, and we have:

[0146] ;

[0147] Calculate the sequence of adjacent center distances:

[0148] ;

[0149] ;

[0150] The median of the adjacent center spacing sequence is used as an estimate to obtain the row spacing and column spacing, as shown in the following formula:

[0151] ;

[0152] ;

[0153] like or Since the median cannot be calculated from the difference between adjacent values, a backtracking estimation based on the average window width / height is used to obtain the row spacing and column spacing. The formulas for the row spacing and column spacing are modified as follows:

[0154] ;

[0155] ;

[0156] in, The empirical coefficient is 1.2, which is selected in this embodiment.

[0157] Step S204: Generate a grid structure in the second image based on the row spacing and the column spacing to obtain the third image.

[0158] In this embodiment, a structured result dictionary is output, which contains the inferred center coordinates of each row and column, as well as typical spacing between columns and rows, typical window size, and other data.

[0159] like Figure 4 As shown, this is a schematic diagram of the generated mesh structure. In this embodiment, the generated mesh structure is obtained based on the output structured result dictionary. By directly generating the corresponding mesh structure in the second image, it is possible to obtain, as shown... Figure 5 The third image shown is a schematic diagram of the identification of the target building facade containing a grid structure.

[0160] like Figure 1 As shown, this embodiment of the invention provides a window detection and completion method based on spatial distribution guidance, including the following steps:

[0161] Step S300: Traverse the mesh structure to obtain the empty parts of the mesh structure, generate new bounding boxes to fill the empty parts, and obtain the fourth image.

[0162] It should be noted that there may be missing parts in the identified mesh structure. It is necessary to intelligently fill in the missing parts that have not been identified by the detection model based on the mesh structure. By intelligently inferring and filling in the undetected window areas based on the spatial distribution pattern of the building facade, the overall completeness and recall rate of the detection can be improved.

[0163] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0164] Step S301: Map the bounding box onto the mesh structure.

[0165] In this embodiment, the mesh structure obtained in step S200 is acquired, and the bounding box of the window detected in step S100 is mapped to the mesh cell of the mesh structure to form a detection mesh mapping table.

[0166] Step S302: Traverse the mesh structure to obtain the empty parts of the missing bounding boxes in the mesh structure.

[0167] In this embodiment, the method for traversing the grid structure is to traverse the grid index matrix and find the grid cells that are not occupied by the detection box, that is, the empty parts of the grid structure that are missing bounding boxes.

[0168] Step S303: Based on the neighboring bounding boxes of the missing part, generate a new bounding box to fill the missing part, and obtain the fourth image.

[0169] In this embodiment, intelligent completion is performed on each missing part in the grid structure. The intelligent completion is based on the neighboring bounding boxes of the missing part, that is, the bounding boxes of the windows detected in step S100. By calculating the median width and median height of the local bounding boxes of the windows, an adaptive window estimate is obtained. Then, a new bounding box is generated based on the adaptive window estimate to complete the missing part in the third image, and finally a fourth image with the missing part completed is obtained.

[0170] In one implementation of this embodiment, step S303 further includes the following steps:

[0171] Step S303a: Calculate the neighboring bounding boxes of the missing parts based on the traversal results of the mesh structure.

[0172] In this embodiment, by traversing the grid index matrix, the grid cells not occupied by the detection box, i.e. the empty parts, are found. For each empty grid cell, the bounding boxes of the detected windows in the neighboring grid are found, and the median width and median height of the local bounding box of the window are calculated.

[0173] In this embodiment, the adjacent grid of the missing part can be limited to a 3×3 area in the grid structure.

[0174] Step S303b: Generate new candidate boxes based on the neighboring bounding boxes, and calculate the size and position information of the new candidate boxes.

[0175] Specifically, based on the predicted center point coordinates and estimated dimensions, the coordinates of the top-left corner of the newly generated candidate window frame are calculated. and the coordinates of the bottom right corner And set the coordinates according to the image width and height. To truncate, we have:

[0176] ;

[0177] ;

[0178] ;

[0179] ;

[0180] Therefore, according to the above formula, the newly generated candidate boxes are limited to the range of the third image.

[0181] Step S303c: Perform geometric consistency screening on the new candidate boxes and retain candidate boxes whose intersection-union ratio with neighboring bounding boxes is within the preset intersection-union ratio threshold.

[0182] In this embodiment, geometric consistency screening is performed on all new candidate boxes: if the overlap (IoU) between a candidate box and the bounding box of a detected window exceeds a set threshold, it is considered redundant and discarded; non-overlapping candidate boxes are retained, and their grid distance to the nearest detected window is calculated. The system sorts them in ascending order based on distance, assuming that candidate boxes that are closer are more likely to be actual missed windows, and high-confidence candidate boxes are retained first.

[0183] In step S303d, the retained candidate boxes are used as new bounding boxes, and the missing parts in the third image are filled in using the new bounding boxes to obtain the fourth image.

[0184] In this embodiment, the retained candidate boxes are used as new bounding boxes, and the new bounding boxes are used to fill in the missing parts in the third image, resulting in the following: Figure 6 The fourth image shown is a schematic diagram of building facade recognition, which includes the detection of bounding boxes, mesh structure, and completion of bounding boxes. Figure 6 The detection bounding box in the middle is still represented by the green detection box, while the completed bounding box is represented by the inferred position box with a blue dashed line.

[0185] like Figure 1 As shown, this embodiment of the invention provides a window detection and completion method based on spatial distribution guidance, including the following steps:

[0186] Step S400: Optimize and verify the bounding boxes in the fourth image, merge the verified bounding boxes with the bounding boxes in the second image, and output a fifth image containing the window detection and completion results of the target building.

[0187] It should be noted that traditional bounding box refinement and verification methods usually rely on a single detection model or a fixed threshold strategy, which leads to false detections and redundant completions in complex scenarios and lacks constraints on structural consistency. In this embodiment, the fourth image containing the completed bounding box obtained in step S300 is further refined and verified to eliminate erroneous speculations and improve the accuracy of the detection results.

[0188] In this embodiment, a coupled refinement mechanism is adopted, which allows the selection of a verification model different from the initial detection model for secondary detection and confirmation, thereby achieving flexible multi-model collaboration.

[0189] Specifically, in one implementation of this embodiment, step S400 includes the following steps:

[0190] Step S401: Use a pre-configured refinement verification model to optimize and verify the bounding boxes in the fourth image.

[0191] In this embodiment, a pre-configured refined verification model is loaded. The refined verification model can be flexibly configured according to different hardware platforms and detection requirements. The model is not limited to the YOLO series and can also adopt other mainstream target detection frameworks. The refined model and the initial detection model can be different versions of the same architecture or completely different network structures to achieve complementarity of detection strategies.

[0192] For each bounding box in the fourth image, the corresponding candidate region is cropped from the original building facade image, i.e., the first image. To enhance the local context information, the candidate region is expanded by a proportional boundary extension area around it during cropping.

[0193] In this embodiment, the proportion of the boundary extension area corresponding to the candidate region is 20% to 50%.

[0194] Each cropped candidate region is input into a pre-configured refined validation model for object detection, and the output includes the detection box coordinates, confidence score, and window category label.

[0195] For the obtained detection results, three constraints are applied sequentially to obtain the refined and verified bounding boxes. If multiple detection boxes satisfy the three constraints, the detection box with the highest confidence is selected as the final refined result for that region. The refined result is as follows: Figure 7 As shown, Figure 7 The fourth image is the refined version, which is a schematic diagram of the building facade recognition including the detected bounding boxes, mesh structure, and completed bounding boxes. The green box is the window corresponding to the bounding box obtained in the initial detection, that is, the window corresponding to the detected box. The blue dashed box is the window corresponding to the candidate box inferred in step S300, that is, the inferred position box. The yellow box is the refined detection box after the final refined model verification. The yellow box in the figure is the completed box, which is the refined detection box. The corresponding confidence score is also given.

[0196] Step S402: Merge the verified bounding box with the bounding box in the second image to obtain a fifth image containing the window detection and completion results of the target building.

[0197] In this embodiment, the verified bounding boxes are mapped to the second image by mapping the candidate boxes to the cropping positions of the original image. That is, the verified bounding boxes are restored from the local area to the global image coordinate system of the building facade based on the bounding box coordinates, and the corresponding confidence information and window category are recorded.

[0198] Finally, the verified bounding boxes are merged with the bounding boxes of all windows detected in the second image to obtain the fifth image containing the window detection and completion results of the target building.

[0199] Step S403: Output the fifth image.

[0200] like Figure 8 The image shown is the building facade recognition image output in this embodiment, which includes the final window detection and completion results. Figure 8 Only the bounding boxes that have been verified and the bounding boxes of all windows detected in the second image are retained. That is, only the green detection boxes and the yellow completion boxes that have been verified are retained in the second image. By removing the used grid structure, the output results are more significant and the image is cleaner. The retained confidence information is also given.

[0201] This embodiment achieves the following technical effects through the above technical solution:

[0202] (1) The overall method of distribution-guided optimization based on grid prior is used. This embodiment utilizes the characteristic that the windows on the building facade are distributed in a grid pattern. Through spatial clustering and row and column spacing analysis of the detection results, the window distribution structure of the building can be automatically inferred, which can effectively restore the spatial relationship of missing windows and significantly improve the structural integrity and geometric consistency of the detection results.

[0203] (2) The missing part completion uses an adaptive size calculation and candidate box screening mechanism. Based on the statistical characteristics of local neighborhood windows, the width and height of the candidate box are adaptively calculated. In the screening stage, a geometric screening algorithm based on intersection-union ratio (IoU) and area ratio constraints is introduced to automatically remove candidate boxes with large position deviations or abnormal sizes, thereby improving the accuracy and stability of the completion results.

[0204] (3) The target detection part uses a multi-stage coupled detection framework, which has good model compatibility. In the initial detection stage, target detection models with different architectures can be used, while in the refinement stage, independent detection models different from the initial model can be used for verification, which can realize cross-model mutual verification. This design effectively reduces the false detection rate in the completion process and enhances the robustness and generalization ability of the system in complex scenarios.

[0205] Exemplary device

[0206] Based on the above embodiments, the present invention also provides a window detection and completion system based on spatial distribution guidance, comprising:

[0207] The image detection module is used to acquire a first image of the target building, detect the first image based on the target detection algorithm, and output a second image containing the bounding boxes of all windows.

[0208] A mesh generation module is used to cluster the bounding boxes and generate a mesh structure in the second image based on the clustering results to obtain a third image;

[0209] The window completion module is used to traverse the grid structure, obtain the missing parts of the grid structure, generate new bounding boxes to complete the missing parts, and obtain the fourth image.

[0210] The verification output module is used to optimize and verify the bounding boxes in the fourth image, merge the verified bounding boxes with the bounding boxes in the second image, and output a fifth image containing the window detection and completion results of the target building.

[0211] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 9 As shown.

[0212] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a computer-readable storage medium and internal memory; the computer-readable storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the computer-readable storage medium; the interface is used to connect to external devices; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or other devices.

[0213] When executed by the processor, this computer program is used to implement a window detection and completion method based on spatial distribution guidance.

[0214] It will be understood by those skilled in the art that Figure 9 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0215] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a spatially distributed guided window detection and completion program, which, when executed by the processor, is used to implement the operations of the spatially distributed guided window detection and completion method described above.

[0216] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a spatially distributed guided window detection and completion program, which, when executed by a processor, is used to implement the operations of the spatially distributed guided window detection and completion method described above.

[0217] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.

[0218] In summary, this invention provides a window detection and completion method based on spatial distribution guidance, comprising: acquiring a first image of the target building; detecting the first image based on a target detection algorithm; outputting a second image containing bounding boxes of all windows; clustering the bounding boxes; generating a grid structure in the second image based on the clustering results to obtain a third image; traversing the grid structure to obtain the missing parts of the grid structure; generating new bounding boxes to complete the missing parts to obtain a fourth image; optimizing and verifying the bounding boxes in the fourth image; merging the verified bounding boxes with the bounding boxes in the second image; and outputting a fifth image containing the window detection and completion results of the target building. This invention can obtain high-precision and complete window detection results.

[0219] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A window detection and completion method based on spatial distribution guidance, characterized in that, include: A first image of the target building is acquired, and the first image is detected based on the target detection algorithm to output a second image containing the bounding boxes of all windows. Cluster the bounding boxes, and generate a mesh structure in the second image based on the clustering results to obtain the third image; Traverse the mesh structure to obtain the empty parts of the mesh structure, generate new bounding boxes to fill the empty parts, and obtain the fourth image; The bounding boxes in the fourth image are optimized and verified. The verified bounding boxes are merged with the bounding boxes in the second image, and a fifth image containing the window detection and completion results of the target building is output. The step of clustering the bounding boxes and generating a mesh structure in the second image based on the clustering results to obtain the third image includes: Calculate the center point coordinates, width, and height of the bounding box; Based on the center point coordinates, width, and height of the bounding box, one-dimensional clustering is performed on the bounding box to obtain row clusters and column clusters; Based on the row centers of the row clusters and the column centers of the column clusters, calculate the row spacing and column spacing of the grid; Based on the row spacing and column spacing, a grid structure is generated in the second image to obtain the third image; The process of traversing the mesh structure to obtain the missing parts of the mesh structure, generating new bounding boxes to fill the missing parts, and obtaining the fourth image includes: Map the bounding box to the mesh structure; Traverse the mesh structure to obtain the empty parts of the missing bounding boxes in the mesh structure; Based on the neighboring bounding boxes of the missing part, a new bounding box is generated to fill the missing part, resulting in a fourth image; The process of generating new bounding boxes to fill in the missing parts based on the neighboring bounding boxes of the missing parts, resulting in a fourth image, includes: Calculate the median width and median height of the local bounding box to obtain an adaptive window estimate. Generate a new bounding box based on the adaptive window estimate to fill in the missing part and obtain the fourth image.

2. The window detection and completion method based on spatial distribution guidance according to claim 1, characterized in that, The step of detecting the first image based on the object detection algorithm and outputting the bounding boxes of all windows in the first image includes: The first image is normalized and subjected to elevation geometry correction to obtain a corrected image; The corrected image is input into a pre-trained target detection model, which outputs the detection boxes of all windows in the first image and the confidence scores of the detection boxes. Overlapping detection boxes are removed based on the nonmaximum suppression method, and detection boxes with confidence scores lower than a preset confidence threshold are discarded. The remaining detection boxes are used as bounding boxes, and a second image containing the bounding boxes of all windows is output.

3. The window detection and completion method based on spatial distribution guidance according to claim 1, characterized in that, The method involves performing one-dimensional clustering on the bounding box based on its center point coordinates, width, and height to obtain row clusters and column clusters, including: Calculate the interquartile range of the bounding box based on its width and height; Calculate the size range based on the interquartile range, and remove bounding boxes whose width and height do not meet the size range. Calculate the average width and average height of the remaining bounding boxes; Based on the center point coordinates of the bounding box, the average width, and the average height, one-dimensional row clustering and one-dimensional column clustering are performed on the remaining bounding boxes in sequence to obtain row clusters and column clusters.

4. The window detection and completion method based on spatial distribution guidance according to claim 1, characterized in that, The process of generating new bounding boxes to fill in the missing parts based on the neighboring bounding boxes of the missing parts, resulting in a fourth image, includes: Based on the traversal results of the mesh structure, calculate the adjacent bounding boxes of the missing parts; Based on the neighboring bounding boxes, new candidate boxes are generated, and the size and position information of the new candidate boxes are calculated. Perform geometric consistency filtering on new candidate boxes, and retain candidate boxes whose intersection-union ratio with neighboring bounding boxes is within a preset intersection-union ratio threshold; The retained candidate boxes are used as new bounding boxes, and the missing parts in the third image are filled in with the new bounding boxes to obtain the fourth image.

5. The window detection and completion method based on spatial distribution guidance according to claim 1, characterized in that, The process of optimizing and validating the bounding boxes in the fourth image, merging the validated bounding boxes with the bounding boxes in the second image, and outputting a fifth image containing the window detection and completion results of the target building includes: The bounding boxes in the fourth image are optimized and verified using a pre-configured refined verification model. The verified bounding box is merged with the bounding box in the second image to obtain a fifth image containing the window detection and completion results of the target building; Output the fifth image.

6. A window detection and completion system based on spatial distribution guidance, used to implement the window detection and completion method based on spatial distribution guidance as described in any one of claims 1-5, characterized in that, include: The image detection module is used to acquire a first image of the target building, detect the first image based on the target detection algorithm, and output a second image containing the bounding boxes of all windows. A mesh generation module is used to cluster the bounding boxes and generate a mesh structure in the second image based on the clustering results to obtain a third image; The window completion module is used to traverse the grid structure, obtain the missing parts of the grid structure, generate new bounding boxes to complete the missing parts, and obtain the fourth image. The verification output module is used to optimize and verify the bounding boxes in the fourth image, merge the verified bounding boxes with the bounding boxes in the second image, and output a fifth image containing the window detection and completion results of the target building.

7. A terminal, characterized in that, include: The processor and memory, the memory storing a spatially distributed guided window detection and completion program, which, when executed by the processor, is used to implement the operation of the spatially distributed guided window detection and completion method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a window detection and completion program based on spatial distribution guidance, which, when executed by a processor, is used to implement the operation of the window detection and completion method based on spatial distribution guidance as described in any one of claims 1-5.

Images