Building facade layout completion methods, devices, terminals and storage media

Abstract

The building facade layout completion method, apparatus, terminal, and storage medium provided by this invention belong to the field of target detection technology. The method includes: performing target detection on an image of a building facade to obtain a set of rectangular objects; performing component extraction and component clustering on the set of rectangular objects to obtain several vertical groups and several horizontal groups; determining the set of neighboring rectangular objects for each rectangular object within all vertical and horizontal groups; generating a set of neighbor vectors for each neighboring rectangular object within the set of neighboring rectangular objects; generating candidate rectangular objects; verifying the candidate rectangular objects; updating the set of rectangular objects if the verification is successful; obtaining the final set of rectangular objects and outputting the corresponding rectangular layout. This invention directly processes the set of rectangular objects to generate candidate rectangular objects and updates the set of rectangular objects when a candidate rectangular object passes verification, thus promptly meeting the user's completion needs.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method, device, terminal and storage medium for completing the layout of building facades. Background Technology

[0002] In instance detection tasks for building facade windows or balconies, due to occlusion, changes in lighting, and inherent limitations of detection algorithms, targets are often missed, resulting in compromised structural integrity of the detection results. Therefore, post-processing is necessary to complete the missing detection frames and restore the spatial continuity of the detection results.

[0003] In existing technologies, graph-based processing methods can be used to complete missing bounding boxes. This method models the bounding boxes as graph nodes, uses edges to represent spatial relationships, and completes missing nodes through graph reasoning. However, this method is time-consuming in the graph construction and reasoning stages, and cannot meet users' completion needs in a timely manner.

[0004] Therefore, existing technologies have shortcomings and need to be improved and developed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method, device, terminal and storage medium for completing the building facade layout, in order to address the above-mentioned deficiencies of the prior art, and to solve the problem that the prior art is unable to meet the user's completion needs in a timely manner.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] In a first aspect, embodiments of the present invention provide a method for completing the facade layout of a building, the method comprising:

[0008] Target detection is performed on the image of the building facade to obtain a set of rectangular objects. The set of rectangular objects contains several rectangular objects, and each rectangular object corresponds to a detected target component, which is a window or balcony.

[0009] Component extraction and component clustering are performed on the set of rectangular objects to obtain several vertical groups and several horizontal groups. The set of neighboring rectangular objects of each rectangular object in all the vertical groups and all the horizontal groups is determined, and a set of neighboring vectors of each neighboring rectangular object in the set of neighboring rectangular objects is generated.

[0010] Traverse all the vertical groups and all the horizontal groups, and generate candidate rectangle objects based on each rectangle object in the current group and its corresponding neighbor vector group;

[0011] The candidate rectangle objects are verified, and if the verification passes, the set of rectangle objects is updated.

[0012] The process iteratively performs clustering and grouping on the set of rectangular objects, traverses all vertical groups and all horizontal groups, and verifies the candidate rectangular objects until the termination condition is met, thereby obtaining the final set of rectangular objects and outputting the corresponding rectangular layout.

[0013] In one implementation, the rectangular object is defined by the x-coordinate of its top-left corner, y-coordinate of its top-left corner, x-coordinate of its bottom-right corner, and y-coordinate of its bottom-right corner; component extraction and component clustering are performed on the set of rectangular objects to obtain several vertical groups and several horizontal groups, including:

[0014] The coordinate components of the rectangular object set are extracted to obtain the top left x-coordinate component array, the top left y-coordinate component array, the bottom right x-coordinate component array, and the bottom right y-coordinate component array of all rectangular objects.

[0015] Mean-shift clustering is performed on all the top-left horizontal coordinate component arrays, the top-left vertical coordinate component arrays, the bottom-right horizontal coordinate component arrays, and the bottom-right vertical coordinate component arrays respectively to generate corresponding cluster label sets;

[0016] Based on the entire set of cluster labels, the set of rectangular objects is divided into several vertical groups and several horizontal groups.

[0017] In one implementation, determining the set of neighboring rectangle objects for each rectangle object within all vertical groups and all horizontal groups includes:

[0018] Each rectangle object within all the vertical groups and all the horizontal groups is sequentially designated as the current rectangle object, and the following operations are performed:

[0019] Determine the group to which the current rectangular object belongs, and calculate the centroid coordinates of the current rectangular object;

[0020] Take the other rectangle objects within the same group as the target rectangle objects in sequence, and perform the following operations on each target rectangle object:

[0021] Calculate the centroid coordinates of the target rectangular object;

[0022] Construct a line segment from the centroid coordinates of the current rectangle object to the centroid coordinates of the target rectangle object;

[0023] Detect whether the line segment intersects with a third-party rectangle object, wherein the third-party rectangle object is a rectangle object within the same group other than the current rectangle object and the target rectangle object;

[0024] If the line segment has no intersection with the third-party rectangle object, then the target rectangle object is determined to belong to the set of neighboring rectangle objects;

[0025] If the line segment intersects with the third-party rectangle object, then the target rectangle object is determined not to belong to the set of neighboring rectangle objects.

[0026] In one implementation, the centroid coordinates include a centroid x-coordinate and a centroid y-coordinate; generating a neighbor vector group for each neighbor rectangle object within the neighbor rectangle object set includes:

[0027] Subtracting the centroid x-coordinate of the corresponding rectangle from the centroid x-coordinate of each neighbor rectangle in the set of neighbor rectangles yields the horizontal centroid offset.

[0028] Subtract the centroid ordinate of the corresponding rectangle from the centroid ordinate of each neighbor rectangle in the set of neighbor rectangle objects to obtain the vertical centroid offset.

[0029] The width of a neighboring rectangle is obtained by subtracting the horizontal coordinate of its top-left corner from the horizontal coordinate of its bottom-right corner in the set of neighboring rectangle objects.

[0030] The height of a neighboring rectangle is obtained by subtracting the ordinate of its top-left corner from the ordinate of its bottom-right corner in the set of neighboring rectangle objects.

[0031] The neighbor vector group of each neighbor rectangle object is composed of the horizontal centroid offset, the vertical centroid offset, the width of the neighbor rectangle object, and the height of the neighbor rectangle object.

[0032] In one implementation, candidate rectangle objects are generated based on each rectangle object within the current group and its corresponding neighbor vector group, including:

[0033] Generate candidate rectangle objects by sequentially combining each rectangle object in the current group with the neighbor vector group of each neighbor rectangle object. The generation steps include:

[0034] The centroid abscissa of the rectangular object is added to the horizontal centroid offset in the neighbor vector group to obtain the centroid abscissa of the candidate rectangular object.

[0035] The centroid ordinate of the rectangular object is added to the vertical centroid offset in the neighbor vector group to obtain the centroid ordinate of the candidate rectangular object.

[0036] Use the width of the neighboring rectangle object as the width of the candidate rectangle object;

[0037] Use the height of the neighboring rectangle object as the height of the candidate rectangle object;

[0038] The x-coordinates of the top-left and bottom-right corners of the candidate rectangle are obtained from the centroid x-coordinate and the width of the candidate rectangle.

[0039] The ordinates of the top left and bottom right corners of the candidate rectangle are obtained from the centroid ordinate and height of the candidate rectangle.

[0040] In one implementation, the candidate rectangle objects are validated, and if the validation passes, the set of rectangle objects is updated, including:

[0041] Perform physical constraint verification on the candidate rectangular objects;

[0042] If the physical constraint verification result is passed, then structural constraint verification is performed.

[0043] In structural constraint verification, a copy of the set of rectangle objects is created as a verification set, and the candidate rectangle objects are added to the verification set.

[0044] Component extraction and component clustering are performed on the verification set to determine the number of cluster centers after component clustering, and to determine whether the number of cluster centers after component clustering meets the preset constraints.

[0045] If the number of cluster centers after component clustering meets the preset constraints, then the candidate rectangle object is added to the rectangle object set.

[0046] In one implementation, physical constraint verification of the candidate rectangle object includes:

[0047] Detect whether the candidate rectangle object overlaps with the rectangle objects in the rectangle object set;

[0048] Detect whether the candidate rectangular object exceeds the building facade boundary.

[0049] Secondly, embodiments of the present invention also provide a building facade layout completion device, the device comprising:

[0050] The target detection module is used to perform target detection on the image of the building facade to obtain a set of rectangular objects. The set of rectangular objects contains several rectangular objects, and each rectangular object corresponds to a detected target component, which is a window or balcony.

[0051] The neighbor vector generation module is used to perform component extraction and component clustering on the set of rectangular objects to obtain several vertical groups and several horizontal groups, determine the set of neighbor rectangular objects of each rectangular object in all the vertical groups and all the horizontal groups, and generate a set of neighbor vectors for each neighbor rectangular object in the set of neighbor rectangular objects.

[0052] The candidate rectangle object generation module is used to traverse all the vertical groups and all the horizontal groups, and generate candidate rectangle objects based on each rectangle object in the current group and its corresponding neighbor vector group.

[0053] The verification module is used to verify the candidate rectangle objects, and if the verification passes, the rectangle object set is updated.

[0054] The iteration module is used to iteratively perform clustering and grouping on the set of rectangular objects, traverse all the vertical groups and all the horizontal groups, and verify the candidate rectangular objects until the termination condition is met, so as to obtain the final set of rectangular objects and output the corresponding rectangular layout.

[0055] Thirdly, embodiments of the present invention also provide a terminal, the terminal comprising: a memory, a processor, and a building facade layout completion program stored in the memory and executable on the processor, wherein when the building facade layout completion program is executed by the processor, it implements the steps of a building facade layout completion method as described above.

[0056] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a building facade layout completion program, which can be executed to implement the steps of a building facade layout completion method as described above.

[0057] The beneficial effects of this invention are as follows: This invention performs target detection on images of building facades to obtain a set of rectangular objects; it then performs component extraction and component clustering on the set of rectangular objects to obtain several vertical groups and several horizontal groups; it determines the set of neighboring rectangular objects for each rectangular object within all vertical and horizontal groups, and generates a set of neighbor vectors for each neighboring rectangular object within the set of neighboring rectangular objects; it generates candidate rectangular objects; it verifies the candidate rectangular objects, and updates the set of rectangular objects if the verification is successful; finally, it obtains the final set of rectangular objects and outputs the corresponding rectangular layout. This invention directly processes the set of rectangular objects to generate candidate rectangular objects, and updates the set of rectangular objects when a candidate rectangular object passes verification, thus promptly meeting the user's completion needs. Attached Figure Description

[0058] Figure 1 This is a flowchart of a preferred embodiment of the building facade layout completion method in this invention.

[0059] Figure 2 This is a flowchart of component extraction and component clustering for a set of rectangular objects in this invention.

[0060] Figure 3This is a schematic diagram of a scenario in this invention where the line segment does not intersect with the target rectangular object.

[0061] Figure 4 This is a schematic diagram of a scenario in this invention where a line segment intersects with a target rectangular object.

[0062] Figure 5 This is a flowchart of the completion process of the present invention.

[0063] Figure 6 This is a schematic diagram of a preferred embodiment of the building facade layout completion device of the present invention.

[0064] Figure 7 This is a schematic diagram of the terminal structure of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0066] In instance detection tasks for building facade windows and balconies, due to occlusion, changes in lighting, and inherent limitations of detection algorithms, target omissions often occur, resulting in compromised structural integrity of the detection results. Therefore, post-processing is necessary to complete the missing detection frames and restore the spatial continuity of the detection results.

[0067] In existing technologies, graph-based processing methods can be used to complete missing bounding boxes. This method models the bounding boxes as graph nodes, uses edges to represent spatial relationships, and completes missing nodes through graph reasoning. However, this method is time-consuming in the graph construction and reasoning stages, and cannot meet users' completion needs in a timely manner.

[0068] To address the aforementioned deficiencies in existing technologies, this invention provides a method, apparatus, terminal, and storage medium for completing building facade layouts. The method includes: performing target detection on an image of the building facade to obtain a set of rectangular objects; performing component extraction and component clustering on the set of rectangular objects to obtain several vertical groups and several horizontal groups; determining the set of neighboring rectangular objects for each rectangular object within all vertical and horizontal groups; generating a set of neighbor vectors for each neighboring rectangular object within the set of neighboring rectangular objects; generating candidate rectangular objects; verifying the candidate rectangular objects; updating the set of rectangular objects if the verification is successful; obtaining the final set of rectangular objects and outputting the corresponding rectangular layout. This invention directly processes the set of rectangular objects to generate candidate rectangular objects and updates the set of rectangular objects when a candidate rectangular object passes verification, thus promptly meeting the user's completion needs.

[0069] Please see Figure 1The building facade layout completion method described in this embodiment of the invention includes the following steps:

[0070] Step S100: Perform target detection on the image of the building facade to obtain a set of rectangular objects. The set of rectangular objects contains several rectangular objects, and each rectangular object corresponds to a detected target component, which is a window or balcony.

[0071] Specifically, this invention utilizes a trained object detection model to perform object detection on images of building facades, obtaining a set of rectangular objects. This set of rectangular objects can be represented as... ,in, Indicates a rectangular index subscript. This indicates the total number of rectangular objects. Indicates the first A rectangular object.

[0072] Please see Figure 1 The building facade layout completion method described in this embodiment of the invention further includes the following steps:

[0073] Step S200: Perform component extraction and component clustering on the set of rectangular objects to obtain several vertical groups and several horizontal groups. Determine the set of neighboring rectangular objects for each rectangular object in all the vertical groups and all the horizontal groups, and generate a set of neighboring vectors for each neighboring rectangular object in the set of neighboring rectangular objects.

[0074] Specifically, a rectangle object is defined by the x-coordinate of its top-left corner, the y-coordinate of its top-left corner, the x-coordinate of its bottom-right corner, and the y-coordinate of its bottom-right corner. It can be represented as ,in, The x-coordinate of the top left corner. The top-left ordinate is the vertical coordinate. The x-coordinate of the bottom right corner. The vertical coordinate is the bottom right corner.

[0075] In one implementation, component extraction and component clustering are performed on the set of rectangular objects to obtain several vertical groups and several horizontal groups, including:

[0076] The coordinate components of the rectangular object set are extracted to obtain the top left x-coordinate component array, the top left y-coordinate component array, the bottom right x-coordinate component array, and the bottom right y-coordinate component array of all rectangular objects.

[0077] Mean-shift clustering is performed on all the top-left horizontal coordinate component arrays, the top-left vertical coordinate component arrays, the bottom-right horizontal coordinate component arrays, and the bottom-right vertical coordinate component arrays respectively to generate corresponding cluster label sets;

[0078] Based on the entire set of cluster labels, the set of rectangular objects is divided into several vertical groups and several horizontal groups.

[0079] Specifically, the flowchart for component extraction and component clustering of a rectangular object set is as follows: Figure 2 As shown, the coordinate components of each rectangle in the collection of rectangle objects are extracted, specifically the top-left x-coordinate, top-left y-coordinate, bottom-right x-coordinate, and bottom-right y-coordinate of each rectangle, forming an array of the top-left x-coordinate components for all rectangle objects. Top-left y-coordinate component array The lower right corner x-coordinate component array and the bottom right y-coordinate component array Then, mean-shift clustering is performed on each component data to generate a corresponding cluster label set. After performing mean-shift clustering on the top-left x-coordinate component array, the top-left y-coordinate component array, the bottom-right x-coordinate component array, and the bottom-right y-coordinate component array to generate corresponding cluster label sets, the process further includes: sequentially determining the number of cluster centers in the cluster label set corresponding to the top-left x-coordinate component array, the top-left y-coordinate component array, the bottom-right x-coordinate component array, and the bottom-right y-coordinate component array. Determining the number of cluster centers for each component array is used for subsequent structural constraint verification.

[0080] Mean shift clustering is a nonparametric clustering algorithm based on kernel density estimation (KDE). Its core idea is to iteratively calculate the density gradient of data points, move the points to regions with higher density, and eventually converge to a local density maximum (i.e., the cluster center).

[0081] For a given data point At point Kernel density estimation at [location] It can be represented as:

[0082] ;

[0083] In the formula, It's a kernel function, using a Gaussian kernel. Perform kernel density estimation, It is the spatial location of the density to be evaluated. n The number of samples in the dataset is denoted by , e is a natural constant, z is the variable for the Gaussian kernel, and h is the bandwidth, which controls the smoothness of the Gaussian kernel.

[0084] To find the direction of maximum density, the gradient is calculated. And on The formula is derived to obtain the mean shift vector. The calculation formula is as follows:

[0085] ;

[0086] The mean shift vector indicates the distance from the current position. The direction in which it moves to the density center of its neighborhood.

[0087] Through iteration along The cluster centers can be obtained by moving the direction until convergence.

[0088] The iterative update formula is as follows:

[0089] ;

[0090] In the formula, For point In the The position of the step, For point In the t The position of the step, For in position The mean shift vector calculated at [location]. After convergence. Become a cluster center. The original data points are assigned cluster labels based on their final convergence location. .

[0091] For the top left x-coordinate component array Top-left y-coordinate component array The lower right corner x-coordinate component array and the bottom right y-coordinate component array The rectangular objects in the dataset are clustered using the mean-shift clustering method, which yields separate label sets. , , and .in, The set of cluster labels corresponding to the top-left x-coordinate component array can be represented as: . Composed of multiple cluster labels Composition, clustering tags Indicates the first The cluster label of the top-left x-coordinate of a rectangular object. It has alignment properties. Indicates the total number of clusters; The set of cluster labels corresponding to the top-left y-coordinate component array can be represented as: . Composed of multiple cluster labels Composition, clustering tags Indicates the first Cluster label of the top-left y-coordinate of a rectangular object; The set of cluster labels corresponding to the lower right x-coordinate component array can be represented as: . Composed of multiple cluster labels Composition, clustering tags Indicates the first Cluster label of the bottom right x-coordinate of a rectangular object. The set of cluster labels corresponding to the bottom right y-coordinate component array can be represented as: . Composed of multiple cluster labels Composition. Clustering tags Indicates the first Cluster label of the bottom right ordinate of a rectangular object.

[0092] After obtaining the clustering label set, the rectangular object set is divided into several vertical groups and several horizontal groups based on the entire clustering label set. This includes: grouping rectangular objects with the same clustering label at both the top-left and bottom-right horizontal coordinates into the same vertical group; and grouping rectangular objects with the same clustering label at both the top-left and bottom-right vertical coordinates into the same horizontal group. All vertical groups can be represented as follows: ,in, Indicates the number of vertical groups. For the first k There are vertical groups. All horizontal groups can be represented as follows: ,in, Indicates the number of horizontal groups. For the first k Horizontal grouping. Rectangles belonging to the same group are those with similar width or height attributes and are horizontally or vertically aligned. Rectangular objects that do not share the same cluster label at both their top-left and bottom-right horizontal coordinates are not added to the same vertical group. Similarly, rectangular objects that do not share the same cluster label at both their top-left and bottom-right vertical coordinates are not added to the same horizontal group.

[0093] This invention employs mean-shift clustering to independently group rectangular coordinate components, automatically extracting horizontally / vertically aligned clusters to form a structured label set. This method eliminates the need for manually setting template parameters or labeling data, adaptively discovering layout patterns and solving the problem of traditional clustering methods ignoring size and orientation.

[0094] In addition to using mean-shift clustering, the DBSCAN clustering algorithm or the K-means clustering algorithm can also be used to cluster the top-left horizontal coordinate component array, the top-left vertical coordinate component array, the bottom-right horizontal coordinate component array, and the bottom-right vertical coordinate component array respectively, generating corresponding cluster label sets.

[0095] In one implementation, determining the set of neighboring rectangle objects for each rectangle object within all vertical groups and all horizontal groups includes:

[0096] Each rectangle object within all the vertical groups and all the horizontal groups is sequentially designated as the current rectangle object, and the following operations are performed:

[0097] Determine the group to which the current rectangular object belongs, and calculate the centroid coordinates of the current rectangular object;

[0098] Take the other rectangle objects within the same group as the target rectangle objects in sequence, and perform the following operations on each target rectangle object:

[0099] Calculate the centroid coordinates of the target rectangular object;

[0100] Construct a line segment from the centroid coordinates of the current rectangle object to the centroid coordinates of the target rectangle object;

[0101] Detect whether the line segment intersects with a third-party rectangle object, wherein the third-party rectangle object is a rectangle object within the same group other than the current rectangle object and the target rectangle object;

[0102] If the line segment has no intersection with the third-party rectangle object, then the target rectangle object is determined to belong to the set of neighboring rectangle objects;

[0103] If the line segment intersects with the third-party rectangle object, then the target rectangle object is determined not to belong to the set of neighboring rectangle objects.

[0104] Specifically, scenarios where the line segment does not intersect the target rectangle object include... Figure 3 As shown, in this case, the target rectangle object is always a neighboring rectangle object of the current rectangle object. The scenario where a line segment intersects with the target rectangle object is as follows: Figure 4 As shown, in this case, the target rectangle object is not a neighboring rectangle object of the current rectangle object. The mathematical form of this process can be expressed as:

[0105] ;

[0106] in, For third-party rectangle objects, For the current rectangle object, For the target rectangle object, Let the centroid of the current rectangle be... Let the centroid of the target rectangle be... This represents the line segment between the two centroids.

[0107] In addition to using the above method to determine neighboring rectangle objects, the Euclidean distance between each current rectangle object and the target rectangle object in its group can also be calculated. When the Euclidean distance is less than or equal to a preset threshold, the target rectangle object is identified as a neighboring rectangle object; when the Euclidean distance is greater than the preset threshold, the corresponding target rectangle object is identified as a third-party rectangle object.

[0108] In one implementation, the centroid coordinates include a centroid x-coordinate and a centroid y-coordinate; generating a neighbor vector group for each neighbor rectangle object within the neighbor rectangle object set includes:

[0109] Subtracting the centroid x-coordinate of the corresponding rectangle from the centroid x-coordinate of each neighbor rectangle in the set of neighbor rectangles yields the horizontal centroid offset.

[0110] Subtract the centroid ordinate of the corresponding rectangle from the centroid ordinate of each neighbor rectangle in the set of neighbor rectangle objects to obtain the vertical centroid offset.

[0111] The width of a neighboring rectangle is obtained by subtracting the horizontal coordinate of its top-left corner from the horizontal coordinate of its bottom-right corner in the set of neighboring rectangle objects.

[0112] The height of a neighboring rectangle is obtained by subtracting the ordinate of its top-left corner from the ordinate of its bottom-right corner in the set of neighboring rectangle objects.

[0113] The neighbor vector group of each neighbor rectangle object is composed of the horizontal centroid offset, the vertical centroid offset, the width of the neighbor rectangle object, and the height of the neighbor rectangle object.

[0114] Specifically, the neighbor vector set can be represented as The calculation formula is as follows:

[0115] ;

[0116] In the formula, This is the horizontal centroid offset. The x-coordinate of the centroid of the neighboring rectangle object. The x-coordinate of the centroid of the corresponding rectangular object. This is the vertical centroid offset. The ordinate of the centroid of the neighboring rectangle object. The x-coordinate of the centroid of the corresponding rectangular object. The width of the neighboring rectangle object. The x-coordinate of the bottom right corner of the neighboring rectangle object. The x-coordinate of the top-left corner of the neighboring rectangle object. The height of the neighboring rectangle object. The y-coordinate of the bottom right corner of the neighboring rectangle object. The ordinate is the top-left corner coordinate of the neighboring rectangular object. This invention transforms the adjacency relationships of rectangular objects into a vector group, represented by the centroid connection vector and size parameters. Neighbor relationships are dynamically defined through the no-occlusion principle, replacing traditional fixed templates or global graph models. This approach supports flexible modeling of locally non-uniform arrangements, avoids dependence on global mesh rules, and improves robustness to occlusion and deformation.

[0117] Please see Figure 1 The method for completing the building facade layout according to an embodiment of the present invention further includes the following steps:

[0118] Step S300: Traverse all the vertical groups and all the horizontal groups, and generate candidate rectangle objects based on each rectangle object in the current group and its corresponding neighbor vector group.

[0119] Specifically, generating candidate rectangle objects based on each rectangle object within the current group and its corresponding neighbor vector group includes: sequentially generating candidate rectangle objects by combining each rectangle object within the current group with the neighbor vector group of each neighbor rectangle object. The generation steps include: adding the centroid x-coordinate of the rectangle object to the horizontal centroid offset in the neighbor vector group to obtain the centroid x-coordinate of the candidate rectangle object; adding the centroid y-coordinate of the rectangle object to the vertical centroid offset in the neighbor vector group to obtain the centroid y-coordinate of the candidate rectangle object; using the width of the neighbor rectangle object as the width of the candidate rectangle object; using the height of the neighbor rectangle object as the height of the candidate rectangle object; obtaining the top-left and bottom-right x-coordinates of the candidate rectangle object from its centroid x-coordinate and width; and obtaining the top-left and bottom-right y-coordinates of the candidate rectangle object from its centroid y-coordinate and height. Multiple candidate rectangle objects can be generated in this way.

[0120] Please see Figure 1 The method for completing the building facade layout according to an embodiment of the present invention further includes the following steps:

[0121] Step S400: Verify the candidate rectangle objects. If the verification is successful, update the rectangle object set.

[0122] Specifically, physical constraint verification and structural constraint verification are performed on candidate rectangle objects in sequence, and the rectangle object set is updated only if both verifications are passed.

[0123] In one implementation, the candidate rectangle objects are validated, and if the validation passes, the set of rectangle objects is updated, including:

[0124] Perform physical constraint verification on the candidate rectangular objects;

[0125] If the physical constraint verification result is passed, then structural constraint verification is performed.

[0126] In structural constraint verification, a copy of the set of rectangle objects is created as a verification set, and the candidate rectangle objects are added to the verification set.

[0127] Component extraction and component clustering are performed on the verification set to determine the number of cluster centers after component clustering, and to determine whether the number of cluster centers after component clustering meets the preset constraints.

[0128] If the number of cluster centers after component clustering meets the preset constraints, then the candidate rectangle object is added to the rectangle object set.

[0129] Specifically, physical constraint verification is performed on the candidate rectangular objects, including: detecting whether the candidate rectangular object overlaps with rectangular objects in the set of rectangular objects; and detecting whether the candidate rectangular object exceeds the building facade boundary. If the physical constraint verification fails, the candidate rectangular object is discarded directly. After passing the physical constraint verification, the candidate rectangular object is added to the verification set, and component extraction and component clustering are performed again. Component extraction and component clustering are performed on the verification set to determine the number of cluster centers after component clustering, and to determine whether the number of cluster centers after component clustering meets the preset constraint conditions, including: obtaining the number of cluster centers generated by the previous component clustering; and determining whether the preset constraint conditions are met based on the relationship between the number of cluster centers generated by the previous component clustering and the number of cluster centers generated by the current component clustering. The number of cluster centers generated by the previous component clustering includes: the number of cluster centers in the cluster label set corresponding to the top-left x-coordinate component array of the previous component clustering, the number of cluster centers in the cluster label set corresponding to the top-left y-coordinate component array of the previous component clustering, the number of cluster centers in the cluster label set corresponding to the bottom-right x-coordinate component array of the previous component clustering, and the number of cluster centers in the cluster label set corresponding to the bottom-right y-coordinate component array of the previous component clustering; the number of cluster centers generated by the current component clustering includes: the number of cluster centers in the cluster label set corresponding to the top-left x-coordinate component array of the current component clustering, the number of cluster centers in the cluster label set corresponding to the top-left y-coordinate component array of the current component clustering, the number of cluster centers in the cluster label set corresponding to the bottom-right x-coordinate component array of the current component clustering, and the number of cluster centers in the cluster label set corresponding to the bottom-right y-coordinate component array of the current component clustering.

[0130] The preset constraint condition is that at least one of the horizontal or vertical constraints must be satisfied, which can be expressed as: In the formula, To constrain the strictness parameter, , To satisfy the number of horizontal constraints, To satisfy the number of vertical constraints, This represents the number of constraints that are actually satisfied.

[0131] The horizontal constraint is that the number of cluster centers in the label set corresponding to the top-left y-coordinate component array of the previous component cluster is equal to the number of cluster centers in the label set corresponding to the top-left x-coordinate component array of the current component cluster, and the number of cluster centers in the label set corresponding to the bottom-right x-coordinate component array of the previous component cluster is equal to the number of cluster centers in the label set corresponding to the bottom-right x-coordinate component array of the current component cluster.

[0132] The mathematical expression for the horizontal constraint is:

[0133] ;

[0134] In the formula, This represents the number of cluster centers in the set of cluster labels corresponding to the top-left y-coordinate component array of the current component cluster. The number of cluster centers in the set of cluster labels corresponding to the top-left y-coordinate component array of the previous component clustering. This represents the number of cluster centers in the set of cluster labels corresponding to the bottom right x-coordinate component array of the current component cluster. This represents the number of cluster centers in the cluster label set corresponding to the lower right x-coordinate component array of the previous component clustering.

[0135] The vertical constraint is defined as follows: the number of cluster centers in the label set corresponding to the top-left y-coordinate component array of the previous component cluster is equal to the number of cluster centers in the label set corresponding to the top-left y-coordinate component array of the current component cluster, and the number of cluster centers in the label set corresponding to the bottom-right y-coordinate component array of the previous component cluster is equal to the number of cluster centers in the label set corresponding to the bottom-right y-coordinate component array of the current component cluster. The mathematical expression for the vertical constraint is:

[0136] ;

[0137] In the formula, This represents the number of cluster centers in the set of cluster labels corresponding to the top-left y-coordinate component array of the current component cluster. This represents the number of cluster centers in the set of cluster labels corresponding to the top-left y-coordinate component array of the previous component clustering. This represents the number of cluster centers in the set of cluster labels corresponding to the bottom right y-coordinate component array of the current component cluster. This represents the number of cluster centers in the cluster label set corresponding to the lower right y-coordinate component array of the previous component clustering.

[0138] when When 1, it means that the candidate rectangle object only satisfies the vertical or horizontal constraint. This causes the candidate rectangle object to only satisfy the condition outside the current rectangle layout's bounding box, and not inside the bounding box. It can only expand outwards, not be completed internally. When When =2, it means that the candidate rectangle object satisfies both the horizontal and vertical constraints, and can only be filled in from the inside.

[0139] Understandably, when the structure validation fails, the candidate rectangle object is discarded.

[0140] This invention generates candidate boxes based on neighbor vector groups, verifies compatibility in real time through physical and structural constraints, and iteratively optimizes until convergence. This approach ensures the structural consistency of the completed box with the existing layout, avoids disrupting the original alignment relationship, and supports strict or lenient completion strategies.

[0141] Please see Figure 1 The method for completing the building facade layout according to an embodiment of the present invention further includes the following steps:

[0142] Step S500: Iteratively perform clustering and grouping on the set of rectangular objects, traverse all the vertical and horizontal groups, and verify the candidate rectangular objects until the termination condition is met, to obtain the final set of rectangular objects and output the corresponding rectangular layout.

[0143] Specifically, the termination condition is the length of the set of rectangular objects in this iteration. Equal to the length of the collection of rectangular objects in the last iteration The length of the rectangular object set is the number of cluster centers obtained after component clustering.

[0144] The flowchart of the completion process of this invention is as follows: Figure 5 As shown, the process comprises four stages: initialization, candidate rectangle object processing, constraint verification, and iteration control. In the initialization stage, component extraction and clustering are performed on the rectangle object set. In the candidate rectangle object processing stage, the grouping type (vertical or horizontal) is determined, and neighboring rectangle objects are identified within the corresponding group, generating candidate rectangle objects. In the constraint verification stage, physical constraint verification is performed first, followed by structural constraint verification. During structural constraint verification, candidate rectangle objects are combined with a copy of the rectangle object set, and component extraction and clustering are performed again. It is then determined whether the preset constraints are met. If they are met, the candidate rectangle object is added to the original rectangle object set to update the set; otherwise, it is discarded. In the iteration control stage, it is determined whether the iteration is complete. If it is complete, the final rectangle layout corresponding to the rectangle object set is output.

[0145] In one embodiment, such as Figure 6 As shown, based on the above-mentioned method for completing building facade layout, the present invention also provides a device for completing building facade layout, the device comprising:

[0146] The target detection module 100 is used to perform target detection on the image of the building facade to obtain a set of rectangular objects. The set of rectangular objects contains a number of rectangular objects, and each rectangular object corresponds to a detected target component, which is a window or a balcony.

[0147] The neighbor vector generation module 200 is used to perform component extraction and component clustering on the set of rectangular objects to obtain several vertical groups and several horizontal groups, determine the set of neighbor rectangular objects of each rectangular object in all the vertical groups and all the horizontal groups, and generate a set of neighbor vectors of each neighbor rectangular object in the set of neighbor rectangular objects.

[0148] The candidate rectangle object generation module 300 is used to traverse all the vertical groups and the horizontal groups, and generate candidate rectangle objects based on each rectangle object in the current group and its corresponding neighbor vector group.

[0149] The verification module 400 is used to verify the candidate rectangle objects, and if the verification passes, the rectangle object set is updated.

[0150] The iteration module 500 is used to iteratively perform clustering and grouping on the set of rectangular objects, traverse all the vertical groups and all the horizontal groups, and verify the candidate rectangular objects until the termination condition is met, so as to obtain the final set of rectangular objects and output the corresponding rectangular layout.

[0151] It should be noted that the foregoing explanation of the method embodiment for completing the building facade layout also applies to the building facade layout completion device of this embodiment, and will not be repeated here.

[0152] Based on the above embodiments, the present invention also provides a terminal, the structural schematic diagram of which is as follows: Figure 7 As shown. The terminal includes a processor, memory, network interface, and display screen connected via a device bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operating devices and a building facade layout completion program. The internal memory provides an environment for the operation of the operating devices and the building facade layout completion program stored in the non-volatile storage medium. The network interface is used for communication with external terminals via a network connection. When the building facade layout completion program is executed by the processor, it implements the steps of any of the above-described building facade layout completion methods. The display screen can be a liquid crystal display (LCD) or an e-ink display.

[0153] Those skilled in the art will understand that Figure 7 The structural schematic diagram shown is only a schematic diagram of a part of the structure related to the present invention solution, and does not constitute a limitation on the terminal on which the present invention solution is applied. The specific terminal may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0154] In one embodiment, a terminal is provided, the terminal including a memory, a processor, and a building facade layout completion program stored in the memory and executable on the processor. When the building facade layout completion program is executed by the processor, it implements the steps of any of the building facade layout completion methods provided in the embodiments of the present invention.

[0155] This invention also provides a computer-readable storage medium storing a building facade layout completion program. When the building facade layout completion program is executed by a processor, it implements the steps of any of the building facade layout completion methods provided in this invention.

[0156] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0158] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0159] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0160] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0161] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not mean that the essence of the corresponding technical solutions deviates from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for completing the facade layout of a building, characterized in that, The method includes: Target detection is performed on the image of the building facade to obtain a set of rectangular objects. The set of rectangular objects contains several rectangular objects, and each rectangular object corresponds to a detected target component, which is a window or balcony. Component extraction and component clustering are performed on the set of rectangular objects to obtain several vertical groups and several horizontal groups. The set of neighboring rectangular objects of each rectangular object in all the vertical groups and all the horizontal groups is determined, and a set of neighboring vectors of each neighboring rectangular object in the set of neighboring rectangular objects is generated. Traverse all the vertical groups and all the horizontal groups, and generate candidate rectangle objects based on each rectangle object in the current group and its corresponding neighbor vector group; The candidate rectangle objects are verified, and if the verification passes, the set of rectangle objects is updated. Iteratively perform clustering and grouping on the set of rectangular objects, traverse all the vertical groups and all the horizontal groups, and verify the candidate rectangular objects until the termination condition is met, to obtain the final set of rectangular objects and output the corresponding rectangular layout. Determine the set of neighboring rectangle objects for each rectangle object within all the vertical groups and all the horizontal groups, including: Each rectangle object within all the vertical groups and all the horizontal groups is sequentially designated as the current rectangle object, and the following operations are performed: Determine the group to which the current rectangular object belongs, and calculate the centroid coordinates of the current rectangular object; Take the other rectangle objects within the same group as the target rectangle objects in sequence, and perform the following operations on each target rectangle object: Calculate the centroid coordinates of the target rectangular object; Construct a line segment from the centroid coordinates of the current rectangle object to the centroid coordinates of the target rectangle object; Detect whether the line segment intersects with a third-party rectangle object, wherein the third-party rectangle object is a rectangle object within the same group other than the current rectangle object and the target rectangle object; If the line segment has no intersection with the third-party rectangle object, then the target rectangle object is determined to belong to the set of neighboring rectangle objects; If the line segment intersects with the third-party rectangle object, then the target rectangle object is determined not to belong to the set of neighboring rectangle objects.

2. The method for completing the building facade layout according to claim 1, characterized in that, The rectangular object is defined by the x-coordinate of the top left corner, the y-coordinate of the top left corner, the x-coordinate of the bottom right corner, and the y-coordinate of the bottom right corner; Component extraction and component clustering are performed on the set of rectangular objects to obtain several vertical groups and several horizontal groups, including: The coordinate components of the rectangular object set are extracted to obtain the top left x-coordinate component array, the top left y-coordinate component array, the bottom right x-coordinate component array, and the bottom right y-coordinate component array of all rectangular objects. Mean-shift clustering is performed on all the top-left horizontal coordinate component arrays, the top-left vertical coordinate component arrays, the bottom-right horizontal coordinate component arrays, and the bottom-right vertical coordinate component arrays respectively to generate corresponding cluster label sets; Based on the entire set of cluster labels, the set of rectangular objects is divided into several vertical groups and several horizontal groups.

3. The method for completing the building facade layout according to claim 1, characterized in that, The centroid coordinates include the centroid x-coordinate and the centroid y-coordinate; Generate a set of neighbor vectors for each neighbor rectangle object within the set of neighbor rectangle objects, including: Subtracting the centroid x-coordinate of the corresponding rectangle from the centroid x-coordinate of each neighbor rectangle in the set of neighbor rectangles yields the horizontal centroid offset. Subtract the centroid ordinate of the corresponding rectangle from the centroid ordinate of each neighbor rectangle in the set of neighbor rectangle objects to obtain the vertical centroid offset. The width of a neighboring rectangle is obtained by subtracting the horizontal coordinate of its top-left corner from the horizontal coordinate of its bottom-right corner in the set of neighboring rectangle objects. The height of a neighboring rectangle is obtained by subtracting the ordinate of its top-left corner from the ordinate of its bottom-right corner in the set of neighboring rectangle objects. The neighbor vector group of each neighbor rectangle object is composed of the horizontal centroid offset, the vertical centroid offset, the width of the neighbor rectangle object, and the height of the neighbor rectangle object.

4. The method for completing the building facade layout according to claim 3, characterized in that, Candidate rectangle objects are generated based on each rectangle object in the current group and its corresponding neighbor vector group, including: Generate candidate rectangle objects by sequentially combining each rectangle object in the current group with the neighbor vector group of each neighbor rectangle object. The generation steps include: The centroid abscissa of the rectangular object is added to the horizontal centroid offset in the neighbor vector group to obtain the centroid abscissa of the candidate rectangular object. The centroid ordinate of the rectangular object is added to the vertical centroid offset in the neighbor vector group to obtain the centroid ordinate of the candidate rectangular object. Use the width of the neighboring rectangle object as the width of the candidate rectangle object; Use the height of the neighboring rectangle object as the height of the candidate rectangle object; The x-coordinates of the top-left and bottom-right corners of the candidate rectangle are obtained from the centroid x-coordinate and the width of the candidate rectangle. The ordinates of the top left and bottom right corners of the candidate rectangle are obtained from the centroid ordinate and height of the candidate rectangle.

5. The method for completing the building facade layout according to claim 2, characterized in that, The candidate rectangle objects are validated, and if the validation passes, the set of rectangle objects is updated, including: Perform physical constraint verification on the candidate rectangular objects; If the physical constraint verification result is passed, then structural constraint verification is performed. In structural constraint verification, a copy of the set of rectangle objects is created as a verification set, and the candidate rectangle objects are added to the verification set. Component extraction and component clustering are performed on the verification set to determine the number of cluster centers after component clustering, and to determine whether the number of cluster centers after component clustering meets the preset constraints. If the number of cluster centers after component clustering meets the preset constraints, then the candidate rectangle object is added to the rectangle object set.

6. The method for completing the building facade layout according to claim 5, characterized in that, Physical constraint verification of the candidate rectangular object includes: Detect whether the candidate rectangle object overlaps with the rectangle objects in the rectangle object set; Detect whether the candidate rectangular object exceeds the building facade boundary.

7. A building facade layout completion device, characterized in that, include: The target detection module is used to perform target detection on the image of the building facade to obtain a set of rectangular objects. The set of rectangular objects contains several rectangular objects, and each rectangular object corresponds to a detected target component, which is a window or balcony. The neighbor vector generation module is used to perform component extraction and component clustering on the set of rectangular objects to obtain several vertical groups and several horizontal groups, determine the set of neighbor rectangular objects of each rectangular object in all the vertical groups and all the horizontal groups, and generate a set of neighbor vectors for each neighbor rectangular object in the set of neighbor rectangular objects. Determine the set of neighboring rectangle objects for each rectangle object within all the vertical groups and all the horizontal groups, including: Each rectangle object within all the vertical groups and all the horizontal groups is sequentially designated as the current rectangle object, and the following operations are performed: Determine the group to which the current rectangular object belongs, and calculate the centroid coordinates of the current rectangular object; Take the other rectangle objects within the same group as the target rectangle objects in sequence, and perform the following operations on each target rectangle object: Calculate the centroid coordinates of the target rectangular object; Construct a line segment from the centroid coordinates of the current rectangle object to the centroid coordinates of the target rectangle object; Detect whether the line segment intersects with a third-party rectangle object, wherein the third-party rectangle object is a rectangle object within the same group other than the current rectangle object and the target rectangle object; If the line segment has no intersection with the third-party rectangle object, then the target rectangle object is determined to belong to the set of neighboring rectangle objects; If the line segment intersects with the third-party rectangle object, then the target rectangle object is determined not to belong to the set of neighboring rectangle objects; The candidate rectangle object generation module is used to traverse all the vertical groups and all the horizontal groups, and generate candidate rectangle objects based on each rectangle object in the current group and its corresponding neighbor vector group. The verification module is used to verify the candidate rectangle objects, and if the verification passes, the rectangle object set is updated. The iteration module is used to iteratively perform clustering and grouping on the set of rectangular objects, traverse all the vertical groups and all the horizontal groups, and verify the candidate rectangular objects until the termination condition is met, so as to obtain the final set of rectangular objects and output the corresponding rectangular layout.

8. A terminal, characterized in that, The terminal includes: a memory, a processor, and a building facade layout completion program stored in the memory and executable on the processor. When the building facade layout completion program is executed by the processor, it implements the steps of the building facade layout completion method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a building facade layout completion program, which, when executed by a processor, implements the steps of the building facade layout completion method as described in any one of claims 1-6.

Images