Building window extraction method and system based on object-level thermal infrared attribute features

Abstract

The present invention relates to a method and system for extracting building windows based on object-level thermal infrared attribute features, the method comprising segmenting point clouds based on spatial position features, visible light attribute features and thermal infrared attribute features of point clouds, extracting point cloud objects; achieving rough extraction of building facade window areas by statistically analyzing point cloud object features; and achieving precise optimization of building windows based on the rough extraction results of the window areas by using a grammar model of building window distribution and energy function minimization optimization. The present invention firstly uses the difference between the building facade window temperature and the wall temperature to perform rough extraction of window targets, and then uses the symmetry grammar model of the facade repetitive structure to achieve precise optimization of window extraction, which can significantly improve the extraction accuracy of building windows and can be widely used in building window extraction.

Description

Technical Field

[0001] The invention relates to a method and system for extracting building windows based on object-level thermal infrared attribute features, and relates to the technical field of three-dimensional photogrammetry. Background Art

[0002] Three-dimensional building models play an important role in visualization analysis, urban planning, indoor and outdoor pedestrian navigation, performance evaluation, virtual reality and other fields. Doors and windows are basic elements of building facades. Accurately identifying the locations of doors and windows on building facades and the topological relationships between them is crucial for building modeling. CityGML is the main international standard for urban building applications. According to the level of detail (LoD) of building depiction, the CityGML standard divides three-dimensional building models into five different levels (LoD0-LoD4). The larger the value, the more detailed the building model. Specifically, doors and windows are defined as the opening areas of the building facade. Starting from the LoD3 level, the building model requires detailed modeling of doors and windows. Therefore, the detailed extraction of doors and windows is crucial for constructing the building information model (BIM) under the geographic information system. Furthermore, the arrangement and combination of doors and windows on the facade can also be used to assist in determining the architectural style and style. In addition, in building energy consumption detection, doors and windows are generally considered to be the main source of heat leakage, and the proportion of door and window area to the total building area is an important indicator for evaluating building energy consumption. Therefore, accurately identifying doors and windows of buildings from point cloud data has become a research hotspot in the field of photogrammetry.

[0003] There are two main methods for identifying doors and windows from building facade point clouds: point feature-based methods and object-oriented methods. Point feature-based methods only use local features of point clouds and lack spatial context and global information, so the classification accuracy of weak local features, weak texture areas and large-scale targets is poor. Object-oriented methods generally include model fitting methods, region growing methods, feature clustering methods and energy optimization methods. Among them, the model fitting method is difficult to handle protrusions on the building facade (such as balconies, chimneys) and complex detailed structural information; the main advantage of the region growing method is high computational efficiency, and the disadvantage is that it is sensitive to noise and density changes in point clouds; feature clustering is also not robust to density changes, noise and outliers in point clouds because the neighborhood size and growth criteria need to be determined in advance; the advantage of the energy optimization method is that it is robust to noise, but the method has a large amount of calculation and low efficiency in point cloud segmentation.

[0004] In order to accurately extract doors and windows on the facade of a building, the prior art also discloses pattern recognition methods and hole-based methods. The advantage of the pattern recognition method is that it can process low-density point cloud data and has strong robustness, but it requires the pre-definition of basic features with repetitive and symmetrical structures. Since windows appear as texture-free, transparent, highly reflective and highly transmissive planes, they generally appear as holes in laser point cloud data. The hole-based method is to identify holes in three-dimensional point cloud data, but this type of method can only extract rectangular window structures on the facade of a building, and the computational efficiency is low. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to provide a building window extraction method and system that can automatically extract building windows from three-dimensional point cloud data with visible light attribute information and thermal infrared attribute information.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a building window extraction method based on object-level thermal infrared attribute features, comprising:

[0008] Based on the spatial position features, visible light attribute features and thermal infrared attribute features of the point cloud, point cloud segmentation is performed to extract point cloud objects;

[0009] By counting the object-level thermal infrared attribute features of the point cloud, the rough extraction of the window area of ​​the building facade is achieved;

[0010] Based on the rough extraction results of the window area, the precise optimization of the building windows is achieved through the grammatical model of the building window distribution and the minimization optimization of the energy function.

[0011] Furthermore, point cloud segmentation is performed based on the spatial position features, visible light attribute features and thermal infrared attribute features of the point cloud to extract point cloud objects, including:

[0012] A multi-scale voxel segmentation method is used to segment the point cloud into a set of supervoxels;

[0013] Taking the supervoxel plane and non-planar discrete points obtained by multi-scale supervoxel segmentation as basic units, the initial plane segmentation of the building facade point cloud is achieved through the graph cut optimization method of the adjacency graph.

[0014] Furthermore, the supervoxel planes and non-planar discrete points obtained by multi-scale supervoxel segmentation are used as basic units, and the initial plane segmentation of the building facade point cloud is achieved through the graph cut optimization method of the adjacency graph, including the construction of the basic unit adjacency graph, including:

[0015] For the basic unit V iFor example, all basic units whose distance from the center is less than the circular search range r are included in the adjacency graph to construct the basic unit V i The adjacency graph G i = {V i ,E}, assuming that the basic unit V i With the basic unit V j Adjacent, connecting basic unit V i With V j The edge between them is w ij :

[0016]

[0017] Among them, ω n ,ω RGB ,ω T and ω f Represents the parameters of the Gaussian kernel function, which respectively control the weights of the basic unit normal vector, visible light attribute characteristics, thermal infrared attribute characteristics and curvature. and They represent the normal vector distance, visible light attribute feature distance, thermal infrared attribute feature distance and curvature distance between adjacent basic units respectively:

[0018]

[0019] Among them, n i and n j Represents the basic unit V i With V j The normal vector of Represents the normal vector n i With n j The cosine value of the angle between the two is used to measure the basic unit V i With V j The normal vector similarity of (R i ,G i ,B i ) and (R j ,G j ,B j ) represents the basic unit V i With V j Visible light properties characteristics; Used to measure the basic unit V i With V j Similarity of visible light properties; Thermal i Thermal j Represents the basic unit V i With V j Thermal infrared properties characteristics; Used to measure the basic unit V i With V jThe thermal infrared attribute similarity of i and f j Represents the basic unit V i With V j The curvature of Used to measure the basic unit V i With V j The curvature similarity.

[0020] Furthermore, the supervoxel planes and non-planar discrete points obtained by multi-scale supervoxel segmentation are used as basic units, and the initial plane segmentation of the building facade point cloud is achieved through the graph cut optimization method of the adjacency graph, which also includes graph cut optimization based on the adjacency graph. The specific process is as follows:

[0021] Each basic unit V i is considered as an initial segmentation set S i ;

[0022] Arrange the edge weights of all adjacency graphs from small to large;

[0023] By iteratively comparing two adjacent segmentation sets S i With S j The edge weight w between ij and their internal differences i with I j , gradually merge adjacent segmentation sets with smaller edge weights until the iteration is terminated, and the graph cut optimization of the entire point cloud is achieved.

[0024] Furthermore, by counting the features of the point cloud objects, the rough extraction of the window area of ​​the building facade is achieved, including:

[0025] Based on the designed building window feature WI, object-level window index features are extracted;

[0026] The window feature threshold method is used to extract windows from the building facade point cloud. That is, when the object-level window index feature is greater than the threshold, the object is considered to be a window. The window extraction optimization method is used to further improve the window extraction result to obtain the rough window detection result.

[0027] Furthermore, the expression of the building window feature WI is:

[0028] Among them, the thermal infrared attribute feature Tem of the segmented object is expressed by calculating the average thermal infrared attribute temperature of all points in the object, and the volume feature Volume of the segmented object = ΔX×ΔY×ΔZ, wherein ΔX, ΔY and ΔZ represent the distance range span of the segmented object on the X-axis, Y-axis and Z-axis respectively. The larger the window feature value WI of the segmented object, the more likely the object is a window.

[0029] Furthermore, based on the rough extraction results of the window area, the building window is finely optimized through the grammatical model of the building window distribution and the energy function minimization optimization, including:

[0030] Construct a grammatical model of building window distribution: Use the grid fitting method to construct a hypothetical window set, take the midline of the X direction as the symmetry axis, and use the rough detection results of the windows as sampling points. Reconstruct the missing windows through the symmetry principle in the Gestalt law to obtain the reconstructed virtual grid window set G;

[0031] Building facade window reasoning based on energy function minimization can achieve precise optimization of building windows.

[0032] Furthermore, based on the reasoning of building facade windows with energy function minimization, precise optimization of building windows is achieved, including:

[0033] Design energy function E: E = γ(E G→C +E C→G )+(1-γ)(E α +E β ), where γ is used to measure the weight parameter between the window inference result and the window rough extraction result, E G→C The grid points G(g ij ) and the window rough extraction result C(c ij ), E C→G The distance from the window coarse extraction center to its nearest grid: Among them, α ij It is used to measure the weight of projecting the grid point G to the window rough extraction result C, β k Used to measure the reliability of mapping the window rough extraction result C to the grid point G;

[0034] By minimizing the energy function E, we get the objective function: g 1 ,g 2 ,{α ij},{β k}=argmin(E), where g 1 and g 2 is the plane orthogonal vector g 1 With g 2 ;

[0035] The iterative Gauss-Newton method is used to minimize the objective function. After each iteration, the window optimization inference result with a better energy function value is stored. When the energy function no longer decreases, the iteration is terminated to obtain the precise optimization extraction result of the window.

[0036] In a second aspect, the present invention provides a building window extraction system based on object-level thermal infrared attribute features, the system comprising:

[0037] A point cloud object extraction unit is configured to perform point cloud segmentation based on spatial position features, visible light attribute features, and thermal infrared attribute features of the point cloud, and extract point cloud objects;

[0038] A window rough extraction unit is configured to achieve rough extraction of the window area of ​​the building facade by counting the thermal infrared attribute features of the object in the point cloud;

[0039] The window optimization unit is configured to achieve precise optimization of building windows based on the coarse extraction results of the window area through the grammatical model of the building window distribution and the energy function minimization optimization.

[0040] In a third aspect, the present invention provides an electronic device, which includes at least a processor and a memory, wherein a computer program is stored in the memory, and when the processor runs the computer program, the computer program is executed to implement the method described.

[0041] The present invention adopts the above technical solution, which has the following advantages:

[0042] 1. The present invention firstly utilizes the difference between the window temperature and the wall temperature of the building facade to perform rough extraction of window targets, and then utilizes the symmetry grammar model of the facade repetitive structure to realize precise optimization of window extraction, which can significantly improve the extraction accuracy of building windows.

[0043] 2. The present invention adopts the methods of semantic segmentation, object-level feature extraction and repeated structure grammar model optimization to significantly reduce the salt and pepper effect in the point cloud classification results and improve the integrity and shape regularity of the window extraction results.

[0044] 3. The present invention integrates the geometric features, visible light attribute features and thermal infrared attribute features of the point cloud to achieve semantic segmentation of multi-source feature fusion, and uses grammar model reasoning based on energy function optimization to significantly improve the extraction accuracy of windows with regular contour shapes.

[0045] In summary, the present invention can be widely used in building window extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Throughout the accompanying drawings, the same reference numerals are used to represent the same components. In the accompanying drawings:

[0047] Figure 1 It is a flow chart of a building window extraction method according to an embodiment of the present invention;

[0048] Figure 2The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0050] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0051] Although the terms first, second, third, etc. can be used in the text to describe multiple elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms can only be used to distinguish an element, component, region, layer or section from another region, layer or section. Unless the context clearly indicates, terms such as "first", "second" and other numerical terms do not imply order or sequence when used in the text. Therefore, the first element, component, region, layer or section discussed below can be referred to as the second element, component, region, layer or section without departing from the teaching of the example embodiments.

[0052] For ease of description, spatially relative terms may be used herein to describe the relationship of one element or feature relative to another element or feature as shown in the figures, such as "inside", "outside", "inner side", "outer side", "below", "above", etc. Such spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation depicted in the figures.

[0053] Embodiment 1: Figure 1 As shown, the building window extraction method based on object-level thermal infrared attribute features provided in this embodiment includes:

[0054] S1. Point cloud semantic segmentation: Point cloud segmentation is performed based on the spatial position features, visible light attribute features, and thermal infrared attribute features of the point cloud to extract point cloud objects, including:

[0055] S11. Generate multi-scale supervoxels

[0056] In order to realize automatic extraction of windows from building facade point clouds, the present invention first uses a multi-scale supervoxel segmentation method to realize point cloud downsampling and semantic information extraction. In which, a multi-scale supervoxel is generated:

[0057] First, the point cloud is divided into the maximum scale (r max ) is divided into a set of supervoxels;

[0058] Next, each supervoxel is classified as planar or non-planar according to its salient features;

[0059] Then, the non-planar supervoxel is further subdivided at a smaller scale (r = r-Δr);

[0060] Repeat the above steps until the scale parameter r is less than the preset minimum scale parameter (r min )until.

[0061] Specifically, the present invention uses a multi-scale voxel connectivity segmentation method (VCCS) to segment the point cloud into a set of supervoxels. In the process of supervoxel segmentation, the present invention considers the spatial distance features between points, visible light attribute features, and thermal infrared attribute features, and sets the weight values ​​of these three features to be equal. For each supervoxel, the following rules are used to determine whether the supervoxel belongs to a planar area or a non-planar area to complete the segmentation. The judgment method is as follows:

[0062] First, the covariance matrix is ​​calculated using all discrete points within the supervoxel s:

[0063]

[0064] Among them, p i represents a discrete point within a supervoxel s, represents the average position of all discrete points within the supervoxel s, M represents the covariance matrix of the supervoxel s, and k represents the number of all discrete points within the supervoxel s.

[0065] Next, principal component analysis is used to calculate the eigenvalues ​​(λ 1 >λ 2 >λ 3 ) and the eigenvector (e 1 ,e 2 ,e 3 ).

[0066] Then, the saliency features of each supervoxel are calculated.

[0067]

[0068] Among them, m 1 、m 2 and m 3 represents the salient features calculated using the eigenvalues ​​of supervoxels.

[0069] Finally, the supervoxels are classified as planar regions or non-planar regions according to the set criteria of formula (3).

[0070]

[0071] For a supervoxel s, only when its salient feature (m 1 ,m 2 ,m 3 ) satisfies the conditions of formula (3), the supervoxel s is considered to be a planar area; otherwise, the supervoxel s is considered to belong to a non-planar area and needs to be further divided. After the supervoxel segmentation and plane classification of the point cloud are realized, the basic processing unit in the point cloud is no longer a discrete point, but a planar supervoxel and a non-planar discrete point, which greatly reduces the number of processing units in the point cloud.

[0072] S12. Graph cut optimization based on supervoxel adjacency graph

[0073] The present invention uses the supervoxel planes and non-planar discrete points obtained by multi-scale supervoxel segmentation as basic units, and realizes the initial plane segmentation of the building facade point cloud through the graph cut optimization method of the adjacency graph as the point cloud segmentation result for subsequent window extraction. The algorithm generates multiple planes by iteratively fusing adjacent basic units with similar statistical characteristics. The calculation process includes: basic unit adjacency graph construction and region growing based on graph cut optimization.

[0074] S121. Construction of basic unit adjacency graph

[0075] For the basic unit V i For example, all basic units whose distance from the center is less than the circular search range r are included in the adjacency graph to construct the basic unit V i The adjacency graph G i = {V i ,E}. Assume that the basic unit V i With the basic unit V j adjacency, therefore, connects the basic unit V i With V j The edge between ij It is obtained by calculation using formula (4).

[0076]

[0077] Among them, ω n ,ω RGB ,ω T and ω f The parameters of the Gaussian kernel function respectively control the weights of the basic unit normal vector, visible light attribute features, thermal infrared attribute features and curvature. In this embodiment, these observation weights are set to equal weight observations. and They represent the normal vector distance, visible light attribute characteristic distance, thermal infrared attribute characteristic distance and curvature distance between adjacent basic units, respectively, as shown in formula (5):

[0078]

[0079] Among them, n i and n j Represents the basic unit V i With V j The normal vector of Represents the normal vector n i With n j The cosine value of the angle between the two is used to measure the basic unit V i With V j The normal vector similarity of (R i ,G i ,B i ) and (R j ,G j ,B j ) represents the basic unit V i With V j The visible light attribute characteristics are obtained by calculating the characteristic mean of all discrete points in the basic unit. Used to measure the basic unit V i With V j Similarity of visible light properties; Thermal i Thermal j Represents the basic unit V i With V j The thermal infrared property characteristics are also obtained by calculating the characteristic mean of all discrete points in the basic unit. Used to measure the basic unit V i With V j The thermal infrared attribute similarity of i and f j Represents the basic unit V i With V j The curvature of Used to measure the basic unit V i With Vj It should be noted that the basic unit normal vector n is the minimum eigenvalue λ in the covariance matrix M. 3 The corresponding eigenvector e 3 The curvature f is obtained through the four arithmetic operations of the eigenvalues, and the calculation method is shown in formula (6).

[0080]

[0081] S122. Graph cut optimization based on adjacency graph

[0082] After constructing the adjacency graph of the basic unit, the present invention realizes high-precision segmentation based on the local adjacency graph through the graph cut optimization method. The specific process is as follows:

[0083] First, each basic unit V i is considered as an initial segmentation set S i ;

[0084] Next, arrange the edge weights w of all adjacency graphs from small to large. The smaller the edge weights between adjacent supervoxels, the more similar the two supervoxels are, and the more they should be merged into the same segmentation.

[0085] Then, by iteratively comparing two adjacent segmentation sets S i With S j The edge weight w between ij and their internal differences i with I j , gradually merge the adjacent segmentation sets with smaller edge weights until the iteration ends, and the graph cut optimization of the entire point cloud can be achieved. For example: Assume that the basic unit V i Belongs to the partition set S a , basic unit V j Belongs to the partition set S b , there is an edge w between them ij connected, if w ij The value is less than the preset threshold ε ab , then we can split the set S a and S b Merge into a segmentation set, the specific threshold ε ab The setting method is shown in formula (7):

[0086]

[0087] Among them, |S a | and |S b | represents the partition set S a and S b The number of points, θ is a preset threshold, I a and Ib Represents the partition set S a and S b The internal difference of is represented by the maximum edge weight inside it. In the process of graph cut optimization of adjacency graph, the iterative method of depth-first search (DFS) can be used.

[0088] S2. Rough extraction of building facade windows

[0089] After point cloud segmentation, multi-scale point cloud segmentation objects are obtained. By statistically analyzing the features of the segmented objects, the automatic rough extraction of the window area of ​​the building facade can be achieved. Specifically, it can be divided into two steps: object-level window index feature extraction and rough window detection based on threshold and post-processing.

[0090] S21. Extracting object-level window index features

[0091] Due to the mirror reflection effect, windows always reflect the temperature of surrounding objects rather than their own temperature. When observing the building facade from the ground, most windows on the building facade, especially those located on higher floors, always reflect the cold sky, so the temperature of the windows is lower than that of the walls of the building facade. In addition, compared with wall objects, the volume of window objects is smaller. Therefore, among the objects on the building facade, the lower the temperature value of the object and the smaller the volume value, the greater the possibility that the object is a building window. Therefore, the expression of the building window feature (WI) designed by the present invention is shown in formula (8).

[0092]

[0093] The thermal infrared attribute feature Tem of the segmented object is expressed by calculating the average thermal infrared attribute temperature of all points in the object, and the volume feature Volume of the segmented object is expressed by formula (9).

[0094] Volume = ΔX × ΔY × ΔZ (9)

[0095] Among them, ΔX, ΔY and ΔZ represent the distance range span of the segmented object on the X axis, Y axis and Z axis respectively. The larger the window feature value (WI) of the segmented object, the more likely the object is a window.

[0096] S22, rough window detection based on threshold and post-processing, the specific process includes:

[0097] First, the windows are roughly extracted from the building facade point cloud using the window feature threshold method, that is, when the object-level window index feature is greater than a threshold (such as 15, which is taken as an example but not limited to this), the object is considered to be a window;

[0098] Then, the window extraction optimization method is used to further improve the window extraction results. For example, the normal vector of the building facade window should be parallel to the Z axis. Therefore, when the angle between the normal vector of the initial window object and the Z axis is less than a threshold (such as 20°, for example, not limited to this), the object is considered to be a window. In addition, when the area of ​​the initial window object is large (such as greater than 3 square meters, for example, not limited to this) and connected to the ground (such as less than 0.2m from the bottom of the building model, for example, not limited to this), the object is considered to be a door of the building rather than a window;

[0099] After the above operations, the rough detection result C of the window can be obtained.

[0100] S3. Optimization of building windows

[0101] There are still many missed detection and false detection problems in the rough extraction results of building windows. Therefore, the present invention first estimates the grammatical model of building window distribution, and then optimizes the window inference results by minimizing the energy function. The specific process is as follows:

[0102] S31. Constructing a grammar model of building window distribution

[0103] Project the point cloud to the XOY facade plane, calculate the center line in the X direction, and use the symmetry in the Gestalt law to complete the missing windows on the building facade. The goal of the model construction stage is to achieve parameter estimation of the building facade model with a regular structure. In order to reliably detect the repeated structural pattern of windows in the building facade, the present invention uses a grid fitting method to construct an assumed window set, using the center line in the X direction as the symmetry axis and the rough detection results of the windows as sampling points. The missing windows are reconstructed through the symmetry principle in the Gestalt law to obtain a reconstructed virtual grid window set G. The specific grid position G = g ij It can be obtained by two plane orthogonal generators g 1 With g 2 constitute, satisfy g ij =ig 1 +jg 2 , where g ij The value range of is determined by the point cloud range projected to the XOY plane, but the specific generator g 1 With g 2 The size of needs to be obtained by minimizing the energy function.

[0104] S32. Reasoning of building facade windows based on energy function minimization

[0105] In order to obtain the optimized building facade window reasoning results, the present invention designs the following energy function to determine the optimal window combination. The energy function includes: minimizing the grid point G(g ij) and the window rough extraction result C(c ij ), as shown in formula (10):

[0106]

[0107] Among them, c ij is the distance from the grid point position g ij The nearest window coarse extraction center, α ij It is used to measure the weight of projecting the grid point G to the window rough extraction result C.

[0108] Similarly, the present invention also considers the distance from the window coarse extraction center to its nearest grid, as shown in formula (11):

[0109]

[0110] Among them, g k Indicates the rough extraction result of the distance window c k The nearest grid point location, β k Used to measure the reliability of mapping the window rough extraction result C to the grid point G. α ij and β k It is used to detect possible loopholes and outliers in the window inference results. ij and β k When α is close to 0, it corresponds to holes and outliers in the inference results; when α ij and β k When it is close to 1, it means that there is a high-precision and reliable match between the inference result and the grid point location.

[0111] In addition, in order to maximize the number of effective point pairs of grid point inference positions and window rough extraction centers, the present invention further provides the following two energy functions:

[0112]

[0113]

[0114] Combining the above four different energy functions, we get the overall objective function, as shown in formula (14):

[0115] E=γ(E G→C +E C→G )+(1-γ)(E α +E β ) (14)

[0116] Among them, γ is used to measure the weight parameter between the window inference result and the window coarse extraction result.

[0117] By minimizing the energy function E, we can get the plane orthogonal vector g 1 With g 2 The value of and weight parameter α ij and β k The value of is shown in formula (15).

[0118] g 1 ,g 2 ,{α ij},{β k}=argmin(E) (15)

[0119] The present invention uses an iterative Gauss-Newton method to minimize the objective function shown in formula (15). Since the present invention does not assume any prior knowledge about point cloud holes and outliers, the weight parameter α ij and β k is initialized to 1. The grid generator g 1 and g 2 The initial value of is obtained by using the initial extraction result of the window with symmetry on both sides of the midline in the X direction. The minimization process of the energy function is implemented in an iterative manner. After each iteration, the window optimization inference result with a better energy function value is stored. The iteration is terminated when the energy function no longer decreases. After multiple iterations, the influence of point cloud holes and Liqun noise on the rough extraction results can be eliminated as much as possible, and the refined optimization extraction result of the window can be obtained.

[0120] Embodiment 2: The above-mentioned embodiment 1 provides a method for extracting building windows based on object-level thermal infrared attribute features. Correspondingly, this embodiment provides a system for extracting building windows based on object-level thermal infrared attribute features. The system provided in this embodiment can implement the method for extracting building windows based on object-level thermal infrared attribute features of embodiment 1, and the system can be implemented by software, hardware, or a combination of software and hardware. For the convenience of description, the description of this embodiment is divided into various units and described separately according to their functions. Of course, the functions of each unit can be implemented in the same or one or more software and / or hardware during implementation. For example, the system may include integrated or separate functional modules or functional units to perform the corresponding steps in each method of embodiment 1. Since the system of this embodiment is basically similar to the method embodiment, the description process of this embodiment is relatively simple. For relevant parts, please refer to the partial description of embodiment 1. The embodiment of the system for extracting building windows based on object-level thermal infrared attribute features provided by the present invention is only schematic.

[0121] Specifically, the building window extraction system based on object-level thermal infrared attribute features provided in this embodiment includes:

[0122] A point cloud object extraction unit is configured to perform point cloud segmentation based on spatial position features, visible light attribute features, and thermal infrared attribute features of the point cloud, and extract point cloud objects;

[0123] A window rough extraction unit is configured to achieve rough extraction of the window area of ​​the building facade by counting the point cloud object features;

[0124] The window optimization unit is configured to achieve precise optimization of building windows based on the coarse extraction results of the window area through the grammatical model of the building window distribution and the energy function minimization optimization.

[0125] Embodiment 3: This embodiment provides an electronic device corresponding to the method for extracting building windows based on object-level thermal infrared property features provided in Embodiment 1. The electronic device may be an electronic device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of Embodiment 1.

[0126] like Figure 2 As shown, the electronic device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Component (EISA) bus, etc. The memory stores a computer program that can be run on the processor, and when the processor runs the computer program, it executes the method for extracting building windows based on object-level thermal infrared attribute features provided in the first embodiment. Those skilled in the art can understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computing device to which the scheme of the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] In some implementations, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), optical disk and other media that can store program codes.

[0128] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0129] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some implementations", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0130] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A building window extraction method based on object-level thermal infrared attribute features, characterized in that include: Based on the spatial position features, visible light attribute features and thermal infrared attribute features of the point cloud, point cloud segmentation is performed to extract point cloud objects; By counting the features of point cloud objects, rough extraction of window areas on building facades is achieved, including: Based on the designed building window feature WI, object-level window index features are extracted; The window feature threshold method is used to extract windows from the building facade point cloud. That is, when the object-level window index feature is greater than the threshold, the object is considered to be a window. The window extraction optimization method is used to further improve the window extraction result to obtain the rough window detection result. The expression of building window feature WI is: Among them, the thermal infrared attribute characteristics of the segmented object The volume features of the segmented object are expressed by calculating the average temperature of the thermal infrared properties of all points in the object. ,in, , and Respectively represent the distance range span of the segmented object on the X-axis, Y-axis and Z-axis. The larger the window feature value WI of the segmented object is, the more likely the object is a window. Based on the rough extraction results of the window area, the building window is finely optimized through the grammatical model of the building window distribution and the energy function minimization optimization, including: Construct a grammatical model of building window distribution: Use the grid fitting method to construct a hypothetical window set, take the midline of the X direction as the symmetry axis, and use the rough detection results of the windows as sampling points. Reconstruct the missing windows through the symmetry principle in the Gestalt law to obtain a reconstructed virtual grid window set ; Building facade window reasoning based on energy function minimization can achieve precise optimization of building windows.

2. The building window extraction method based on object-level thermal infrared attribute features according to claim 1 is characterized in that: Point cloud segmentation is performed based on the spatial position features, visible light attribute features, and thermal infrared attribute features of the point cloud to extract point cloud objects, including: A multi-scale voxel connection segmentation method is used to segment the point cloud into a set of supervoxels; Taking the supervoxel plane and non-planar discrete points obtained by multi-scale supervoxel segmentation as basic units, the initial plane segmentation of the building facade point cloud is achieved through the graph cut optimization method of the adjacency graph.

3. The building window extraction method based on object-level thermal infrared attribute features according to claim 2 is characterized in that: The supervoxel plane and non-planar discrete points obtained by multi-scale supervoxel segmentation are used as basic units, and the initial plane segmentation of the building facade point cloud is achieved through the graph cut optimization method of the adjacency graph, including the construction of the basic unit adjacency graph, including: For the basic unit For example, all the objects whose distance from the center is less than the circular search range The basic unit is incorporated into the adjacency graph to construct the basic unit The adjacency graph of , assuming that the basic unit With the basic unit Adjacency, connecting basic units and The edge between : in, , , and Represents the parameters of the Gaussian kernel function, which respectively control the weights of the basic unit normal vector, visible light attribute characteristics, thermal infrared attribute characteristics and curvature. , , and They represent the normal vector distance, visible light attribute feature distance, thermal infrared attribute feature distance and curvature distance between adjacent basic units respectively: in, and Represents the basic unit and The normal vector of Representation vector and The cosine of the angle between the two is used to measure the basic unit and The normal vector similarity of ; and Represents the basic unit and Visible light properties characteristics; Used to measure the basic unit and Similarity of visible light properties; and Represents the basic unit and Thermal infrared properties characteristics; Used to measure the basic unit and Similarity of thermal infrared properties; and Respectively represent the basic unit and The curvature of Used to measure the basic unit and The curvature similarity.

4. The building window extraction method based on object-level thermal infrared attribute features according to claim 3 is characterized in that: The supervoxel plane and non-planar discrete points obtained by multi-scale supervoxel segmentation are used as basic units, and the initial plane segmentation of the building facade point cloud is achieved through the graph cut optimization method of the adjacency graph, which also includes graph cut optimization based on the adjacency graph. The specific process is as follows: Each basic unit is considered as an initial segmentation set ; Arrange the edge weights of all adjacency graphs from small to large; By iteratively comparing two adjacent segmentation sets and The edge weight between and their internal differences and , gradually merge adjacent segmentation sets with smaller edge weights until the iteration is terminated, and the graph cut optimization of the entire point cloud is achieved.

5. The building window extraction method based on object-level thermal infrared attribute features according to claim 1 is characterized in that: Building facade window reasoning based on energy function minimization can achieve precise optimization of building windows, including: Design energy function E: ,in, It is used to measure the weight parameter between the window inference result and the window rough extraction result. Grid points Rough extraction results with windows The distance between The distance from the window coarse extraction center to its nearest grid: ; ,in, Used to measure the grid points Projection to the window rough extraction result The weight of Used to measure the rough extraction results of windows To grid point Reliability; By minimizing the energy function E, we get the objective function: ,in, and is a plane orthogonal vector and ; The iterative Gauss-Newton method is used to minimize the objective function. After each iteration, the window optimization inference result with a better energy function value is stored. When the energy function no longer decreases, the iteration is terminated to obtain the precise optimization extraction result of the window.

6. A building window extraction system for implementing the building window extraction method based on object-level thermal infrared attribute features as described in any one of claims 1 to 5, characterized in that: The system includes: A point cloud object extraction unit is configured to perform point cloud segmentation based on spatial position features, visible light attribute features, and thermal infrared attribute features of the point cloud, and extract point cloud objects; A window rough extraction unit is configured to achieve rough extraction of the window area of ​​the building facade by counting the point cloud object features; The window optimization unit is configured to achieve precise optimization of building windows based on the coarse extraction results of the window area through the grammatical model of the building window distribution and the energy function minimization optimization.

7. An electronic device, comprising at least a processor and a memory, wherein a computer program is stored in the memory, wherein: When the processor runs the computer program, the method according to any one of claims 1 to 5 is implemented.

Images