A plane feature detection method, device and equipment for a real scene three-dimensional model

Abstract

The application discloses a plane feature detection method for a real scene three-dimensional model. The application first analyzes the real scene three-dimensional model into a primitive set, then comprehensively considers the spatial proximity feature, the surface feature and the color feature of the primitive to construct a primitive heterogeneity distance formula, and clusters the primitives into a series of clusters with uniform properties and regular boundaries, extracts the plane features based on the clusters by using the Euclidean space distance and the cosine measurement value, and uses a meta heuristic optimization algorithm to improve the automation degree of the method. The application can effectively detect large planes in the real scene three-dimensional model, has the advantages of high recognition rate, high efficiency and easy generation of regular boundary planes, and can provide support for real scene three-dimensional model building extraction, objectization, data compression, model simplification and roof extraction and the like.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of surveying data processing, and particularly relates to a planar feature detection method, device and equipment for a real scene three-dimensional model. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology and optical sensor technology, airborne LiDAR and oblique photogrammetry technology can obtain large-scale three-dimensional ground information at low cost, quickly and accurately. In particular, the unmanned aerial vehicle oblique photogrammetry technology can effectively collect the coordinates and texture information of the top and side of buildings, and is increasingly important in the construction of three-dimensional digital cities and smart cities. The collected three-dimensional ground information exists in the form of point clouds after processing, and these point clouds can be used to further generate a digital orthophoto model (DOM), a digital surface model (DSM) and a real scene three-dimensional model.

[0003] Compared with two-dimensional images (such as DSM and DOM), a real scene three-dimensional model contains detailed three-dimensional geometric and texture features. Compared with three-dimensional point clouds, a real scene three-dimensional model has the advantages of spatial continuity and explicit adjacency, and occupies less disk and memory, because some geometrically irrelevant points are filtered out in the process of reconstructing a real scene three-dimensional model from point clouds. Compared with three-dimensional models in the field of computer graphics, a real scene three-dimensional model represents large-scale fine three-dimensional ground information and has a larger number of primitives. Therefore, a real scene three-dimensional model is widely used in various types of 3D geographic applications and spatial analysis.

[0004] However, due to the fully automated mechanism of real scene modeling software such as Photomesh and ContextCapture, a continuous and overall mesh model is constructed, and therefore the real scene three-dimensional model generated has the phenomenon of "one skin", that is, all ground objects are represented by a three-dimensional mesh, which makes it difficult to query and analyze semantically and cannot meet the diversified application requirements. Therefore, objectification and entity of the real scene three-dimensional model is an urgent demand in the construction of three-dimensional digital cities. Although a large number of studies have emerged in the field of ground filtering, ground object extraction, scene segmentation and spatial clustering of DSM, DOM and point cloud data, research on the most important data segmentation and ground object extraction of real scene three-dimensional models for three-dimensional digital cities is rare.

[0005] Generally, the planar feature detection of the real scene three-dimensional model is a basic common technology, which can promote the subsequent research or work of the real scene three-dimensional model building extraction, objectification, data compression, model simplification and roof extraction. However, due to the complexity of the real scene three-dimensional model itself, there is no method or research on the planar feature detection of the real scene three-dimensional model at home and abroad at present. Therefore, the application provides a planar feature detection method, device and equipment for the real scene three-dimensional model, which can quickly detect the planar feature from the large-area real scene three-dimensional model. SUMMARY

[0006] In order to fill the technical blank, the application provides a planar feature detection method for the real scene three-dimensional model, which comprises the following steps:

[0007] Step S1, all tiles of the real scene three-dimensional model are spliced into a complete three-dimensional model, and the real scene three-dimensional model is parsed, so that the real scene three-dimensional model is parsed into a geometric primitive set;

[0008] Step S2, a primitive heterogeneity distance formula is constructed for evaluating the similarity between two primitives;

[0009] Step S3, based on the primitive heterogeneity distance formula, a heterogeneity cost function of the cluster is constructed and its constraint condition is determined, which is used for judging the sum of the heterogeneity cost of all clusters:

[0010] Step S4, based on the heterogeneity cost function and its constraint condition, an energy optimization function is constructed and solved, so that the real scene three-dimensional model is over-divided into a cluster set with uniform properties and regular boundaries;

[0011] Step S5, based on the cluster set with uniform properties and regular boundaries, the cluster set is efficiently planar detected in combination with the threshold value of the planar detection, so as to generate a set of boundary-regular planar sets;

[0012] Step S6, the planar feature detection is optimized based on the meta-heuristic algorithm, so as to automatically find the best threshold value in the planar detection process in step S5, and obtain a more ideal planar detection result.

[0013] Further, the construction of the primitive heterogeneity distance formula D(·) for evaluating the similarity between two primitives in step S2 can comprehensively consider the spatial proximity feature, surface feature and color feature of the primitive, and the heterogeneity distance formula between the primitives p i and p j is constructed as: D(p i ,p j )=μ1D s (p i ,p j )+μ2D e (pi ,p j )+μ3D c (p i ,p j ), where D s (p i ,p j D e (p i ,p j ) and D c (p i ,p j ) represent the surface feature difference distance, spatial proximity distance, and color difference distance between two normalized primitives, respectively, and μ1, μ2, and μ3 are the corresponding weighting factors for the three.

[0014] Furthermore, the spatial proximity distance D between the two primitives E (p i ,p j ) You can choose the European-style spatial distance: For p i The q-th vertex; p i and p j The difference in surface features between two primitives can be measured using a normalized cosine similarity formula. and p i and p j The normal vector, i.e. p i and p j Color difference distance D between two primitives c (p i ,p j The distance is calculated within the CIELab linear color space, using a distance formula that better aligns with the non-linear perception of human vision. in, For the basic element p i Average texture color value in CIE Lab space.

[0015] Furthermore, the normal vector of the triangular primitive is calculated using the spatial coordinates of its three vertices v1, v2, and v3:

[0016] Average color texture value The calculation method is as follows: calculate the primitive p in the Adobe RGB color space. iThe space range in y direction and the number of scanning lines, from top to bottom, for any scanning line, intersect all edges of the primitive with the scanning line, sort the horizontal coordinates obtained by intersection from left to right, the edge intersected by the scanning line odd times is an in-edge, and the edge intersected by the scanning line even times is an out-edge; then, the spatial coordinates of the pixel on the scanning line between the in-edge and the out-edge are calculated by interpolation, the U and V coordinates of all pixel points inside the primitive are calculated by using the barycentric coordinate method, and the U and V coordinates are wherein S a is the area of the triangle formed by the coordinates of the pixel point and the v1 and v2 vertices of the primitive, S b is the area of the triangle formed by the pixel point and the v1 and v3 vertices of the primitive, and S c is the area of the triangle formed by the pixel point and the v2 and v3 vertices of the primitive, U1, U2, U3, V1, V2 and V3 are the UV coordinates of v1, v2 and v3 respectively, S t =S a +S b +S c After all the UV coordinates are solved, the texture values are obtained from the texture image corresponding to the primitive by using the UV coordinates, the average value of all the texture values is taken as the texture value of the primitive, and then the Adobe RGB color space is converted into the CIE Lab color space.

[0017] Further, the heterogeneity cost function J(·) of the cluster is constructed and the constraint condition thereof is determined, and specifically, the heterogeneity cost function J(·) of the cluster is constructed and the constraint condition thereof is determined. wherein r ij =1 represents that the primitive p i may represent a central primitive of a cluster, and the cluster contains all non-central primitives satisfying r ij =0; the constraint condition of J(r ij ) is wherein I(·) is an exponential function, k represents the expected number of clusters; D(p i ,p j ) represents the heterogeneity distance formula between the i-th primitive p i and the j-th primitive p j , and n is the number of primitives.

[0018] Further, the energy function E(·) is constructed and solved, so that the real scene three-dimensional model is over-divided into a set of clusters ψ with uniform properties and regular boundaries, and specifically, the energy function is set as The bottom-up merging-based energy minimization method can be selected for solving the energy equation, so that each primitive in except the central primitive set is mapped according to the mapping function assigning them to individual clusters, wherein D(p, cp i ) is the inter-primitive heterogeneity distance formula, and each primitive is assigned to the cluster center primitive with which it has the smallest total heterogeneity distance;

[0019] Further, efficiently planar detection is performed on the cluster set ψ to generate a set of boundary-regularized candidate planes, which includes:

[0020] S501: For the cluster set ψ, select a cluster in the set ψ as a seed cluster of a plane S m , and remove the seed cluster from the cluster set ψ, wherein the plane S m is composed of a subset of the cluster set ψ;

[0021] S502: Calculate the set ψ' of k1 neighboring clusters around the seed cluster, and for each neighboring cluster in ψ', determine whether it has the same property as the seed cluster according to a similarity criterion, if the plane similarity criterion is satisfied, i.e. then incorporate the neighboring cluster into the plane S m containing the seed cluster, and remove the neighboring cluster from the set ψ, if the plane similarity criterion is not satisfied, then do not perform any operation on the cluster, where θ is an angle threshold, represents the cluster, and D s represents the cosine similarity measure;

[0022] S503: Take each cluster newly incorporated into the plane S m in the S502 process as a new seed cluster of the plane S m , and iteratively perform the step S502 until no cluster in ψ satisfies the plane similarity criterion;

[0023] S504: Iteratively perform the S501-S503 process until ψ is empty, and save each detected plane S m in each iteration to form a candidate plane feature set S;

[0024] S505: Post-process the candidate plane feature set S, remove candidate planes with less than k2 clusters in S, and remove candidate planes with an area less than k3, and take the remaining candidate planes as the final plane detection result.

[0025] Further, plane feature detection optimization based on meta-heuristic algorithms, which includes:

[0026] S601: Determine the value range of each threshold value of the plane feature detection process, i.e. the value range of θ and k3;

[0027] S602: determining an unsupervised planar feature evaluation criterion to evaluate the quality of the result of the planar feature detection process;

[0028] S603: using a meta-heuristic optimization algorithm to iteratively select a set of threshold values from the range of values, execute the step S5, and evaluate the quality of the detection using the unsupervised planar feature evaluation criterion of S602 to find the most suitable parameter values of the threshold values.

[0029] The application also provides a device for detecting planar features of a real-world three-dimensional model, comprising the following modules:

[0030] A real-world three-dimensional model parsing module for inputting a real-world three-dimensional model, stitching all the tiles of the acquired real-world three-dimensional model into a complete three-dimensional model, and parsing the real-world three-dimensional model into a set of geometric primitives;

[0031] A primitive heterogeneity distance formula module for inputting a primitive heterogeneity distance formula and calculating the similarity between two primitives according to the inputted two primitives;

[0032] A heterogeneity cost function module for inputting a heterogeneity cost function and its constraint conditions and calculating the sum of the heterogeneity costs of the clusters according to the inputted set of clusters;

[0033] An energy optimization function module for inputting an energy optimization function and over-segmenting the inputted set of geometric primitives of the real-world three-dimensional model into a set of clusters with uniform properties and regular boundaries;

[0034] A planar detection module for inputting the threshold values of planar detection and performing efficient planar detection based on the set of clusters with uniform properties and regular boundaries to generate a set of boundary-regular planes;

[0035] A meta-heuristic optimization module for automatically finding the most suitable values of the threshold values of the planar detection module without manual and empirical setting and obtaining more ideal planar detection results.

[0036] An electronic device comprising a distributed memory, a processor, and a computer program stored in the memory and executable in the processor, wherein the processor implements the steps of the planar feature detection method for a real-world three-dimensional model in the above-mentioned solutions when executing the computer program.

[0037] A computer-readable storage medium storing computer software instructions, wherein the computer program is executed by a processor to implement the steps of the planar feature detection method for a real-world three-dimensional model in the above-mentioned solutions.

[0038] Compared with the prior art, the application has the following advantages and beneficial effects:

[0039] (1) In view of the fact that no effective method has been proposed for detecting planar features in real scene 3D models, the application proposes a planar feature automatic detection method for real scene 3D models.

[0040] (2) The application first considers the spatial proximity feature, surface feature and color feature of the primitives to over-segment the primitives of the real scene 3D model, and then generates a group of clusters with uniform properties, and then performs planar feature detection based on the clusters, which has the advantages of high efficiency and easy generation of boundary rules.

[0041] (3) The method proposed in the application adopts a meta-heuristic algorithm optimization in the planar detection process, which can automatically find the threshold value in the planar detection process without manual experience setting, and achieves more ideal planar detection results. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a planar feature detection method flowchart for real scene 3D models in the embodiment of the application.

[0043] Figure 2 is a real scene 3D model used in the embodiment of the application.

[0044] Figure 3 is a local primitive detail view of a real scene 3D model used in the embodiment of the application.

[0045] Figure 4 is a group of clusters with uniform internal properties generated after over-segmentation of the real scene 3D model in the embodiment of the application.

[0046] Figure 5 is a planar feature detected from the real scene 3D model used in the embodiment of the application.

[0047] Figure 6 is a planar feature detected from the real scene 3D model used in the embodiment of the application.

[0048] Figure 7 is a module diagram of a planar feature detection device for real scene 3D models in another embodiment of the application.

[0049] Figure 8 is a structure block diagram of a planar feature detection device for real scene 3D models in another embodiment of the application. DETAILED DESCRIPTION

[0050] For the convenience of those skilled in the art to understand and implement the present application, the present application is further described in detail below in conjunction with the embodiments. It should be understood that the embodiments described herein are only used for illustration and explanation of the present application and should not be regarded as limiting the present application.

[0051] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered to be limiting, but merely an example embodiment. One skilled in the art will recognize other modifications that can be made to fall within the scope and spirit of the present disclosure.

[0052] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above, and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0053] These and other characteristics of the present application will become apparent from the following description of the preferred forms given, by way of non-limiting example, with reference to the attached drawings.

[0054] It should also be understood that, although the present application has been described herein with reference to particular embodiments, many alternatives, modifications, and variations will be apparent to those skilled in the art from the disclosure herein. It is therefore the intention that limitations intended to define the present application be interpreted in the broadest possible manner consistent with the scope of the appended claims.

[0055] It should be understood, however, that the disclosed embodiments are merely examples of the present disclosure, which can be practiced in various ways. Well-known and / or repetitive functionality and structure are not described in detail to avoid obscuring the present disclosure. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present disclosure in virtually any appropriately detailed structure. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0056] The specification can use phrases such as "in one embodiment", "in another embodiment", "in yet another embodiment", or "in other embodiments", which can refer to one or more embodiments under the same or different embodiments according to the present disclosure.

[0057] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0058] As Figure 1 shown, the embodiments of the present application provide a plane feature detection method for a real scene three-dimensional model, comprising:

[0059] S1, reading the osgb format real scene three-dimensional model as Figure 2 shown, splicing all the tiles of the real scene three-dimensional model into a complete three-dimensional model, and analyzing the real scene three-dimensional model, and analyzing the real scene three-dimensional model into a geometric primitive set Each primitive stores its index, vertex coordinates and texture mapping coordinates, etc. n represents the number of primitives read. As shown in the figure, the collection primitive of the real scene three-dimensional model is a triangular primitive. After the analysis is completed, the model is composed of 3077862 vertices and 5694838 triangular primitives, so n is equal to 5694838; Figure 3

[0060] S2, construct the primitive heterogeneity distance formula D(·) to evaluate the similarity between two primitives. Considering the spatial proximity feature, surface feature and color feature of the primitive, the heterogeneity distance formula between p i and p j is constructed as: D(p i , p j ) = μ1D s (p i , p j ) + μ2D e (p i , p j ) + μ3D c (p i , p j ), wherein D s (p i , p j ), D e (p i , p j ) and D c (p i , p j ) are respectively selected as the normalized surface feature difference distance, spatial proximity distance and color difference distance between two primitives, μ1, μ2 and μ3 are respectively the weight factors corresponding to the three, the value range of the three is [0, 1], the user can specify the three parameters according to his own needs, and in the embodiment, the values are respectively 0.5, 0.2 and 1;

[0061] Wherein, the spatial proximity distance D e (p i , p j ) between two primitives can be selected as the Euclidean space distance: is the qth vertex of p i , because the primitive type is a triangular primitive, so q is at most 3; the surface feature difference distance between p i and p j adopts the normalized cosine similarity measurement formula and are respectively p i ​and p j The normal vector, i.e. p i and p j Color difference distance D between two primitives c (p i ,p j The distance is calculated within the CIE Lab linear color space, using a distance formula that better aligns with the non-linear perception of human vision. In the formula, For the basic element p i Average texture color value in CIELab space;

[0062] in, The method for calculating the normal vector of each primitive element: Taking a triangle primitive element as an example, the normal vector of the primitive element is calculated using the spatial coordinates of its three vertices v1, v2, and v3.

[0063] Among them, the basic element p i Average color texture value The calculation can be based on the scanline algorithm, specifically: taking the calculation of color texture values ​​in the Adobe RGB color space as an example, the calculation of primitive p... i The spatial range and number of scan lines in the y-direction are determined. From top to bottom, for any scan line, all edges of the primitive are intersected with that scan line. From left to right, the x-coordinates of the intersections are sorted; edges intersected an odd number of times are considered inbound edges, and edges intersected an even number of times are considered outbound edges. Then, the spatial coordinates of the pixels on the scan line between the inbound and outbound edges are calculated using interpolation. The UV coordinates of all pixels within the primitive are calculated using the barycentric coordinate method, with their U and V coordinates being... Where S a Let S be the area of ​​the triangle formed by the coordinates of the pixel and the vertices v1 and v2 of the primitive. b S is the area of ​​the triangle formed by the pixel and the vertices v1 and v3 of the primitive. c Let S be the area of ​​the triangle formed by the pixel and the vertices v2 and v3 of the primitive, where U1, U2, U3, V1, V2, and V3 are the UV coordinates of v1, v2, and v3, respectively. t =S a +S b +S c After all UV coordinates are solved, the texture value is obtained from the texture image corresponding to the primitive using the UV coordinates, and the average of all texture values ​​is taken as the texture value of the primitive. Then, the Adobe RGB color space is converted to the CIE Lab color space.

[0064] S3, Based on the aforementioned primitive heterogeneity distance formula D(·), construct the heterogeneity cost function for the cluster. wherein if r ij = 1 represents the central element p i of a cluster, and the cluster contains all non-central elements satisfying r ij = 0; J(r ij ) is used to judge the sum of the heterogeneity cost of all clusters, and the constraint condition of J(r ij ) is set as wherein k represents the expected number of clusters, and I(·) is an index function, i.e., taking x as an example, if x = 1, then I(x) = 1, otherwise, I(x) = 0;

[0065] wherein the value of the expected number of clusters k is converted into a cluster resolution R which is convenient to set, i.e., wherein X max , X min , Y max and Y min are respectively the maximum and minimum values of the X coordinates of the element set , and the maximum and minimum values of the Y coordinates of the element set ;

[0066] S4, based on the heterogeneity cost function J(r ij ) and its constraint condition , an energy optimization function E(r is constructed For the solution of the energy equation, a bottom-up merging-based energy minimization method is selected to assign each element in except the central element set to a cluster according to a mapping function , wherein D(p, cp i ) is the element heterogeneity distance formula, and the sum of the heterogeneity distance of each element to the central element of the cluster to which it is classified is minimized, so that the element set of the input real three-dimensional model is over-divided into a cluster set ψ which is uniform in nature and regular in boundary;

[0067] wherein the bottom-up merging-based energy minimization method is specifically, firstly, a regularization term is added to the energy optimization function E(r ij ), i.e., where λ is a regularization parameter, a larger λ value will result in a smaller deviation of the final cluster number from k, but the weight of the heterogeneity distance metric will be reduced. The initial value of the regularization parameter λ is set to the median of the minimum heterogeneity distance between each element and its adjacent elements, and then it is doubled every iteration. At the beginning, all elements are set as the center elements of the clusters, and they are merged from bottom to top in pairs until the number of clusters drops to k;

[0068] The pseudo-color map of the set of clusters ψ generated by the process is shown in FIG. 2, where each color represents a cluster, and different colors represent different clusters. Figure 4

[0069] S5, based on the set of clusters ψ with uniform properties and regular boundaries, efficiently planar detection is performed on the cluster set ψ to generate a set of boundary-regular planes, and the process includes:

[0070] S501: for the cluster set ψ, a cluster in the set ψ is selected as a seed cluster of a plane S m , and the seed cluster is removed from the cluster set ψ, where the plane S m is composed of a subset of the cluster set ψ;

[0071] As a preferred embodiment, the centroid of each cluster in the set ψ is calculated, and then the curvature of all cluster centroids is calculated, and the set ψ is sorted in ascending order of centroid curvature. Then the seed cluster is selected from the cluster set ψ in order.

[0072] S502: calculate the set ψ' of 8 adjacent clusters around the seed cluster, and for each adjacent cluster in ψ', determine whether it has the same properties as the seed cluster according to the cosine similarity metric criterion, if the cosine similarity metric criterion is satisfied, i.e. , then the adjacent cluster is merged into the plane S m containing the seed cluster, and the adjacent cluster is removed from the set ψ, if the cosine similarity metric criterion is not satisfied, no operation is performed on the cluster, where θ is an angle threshold;

[0073] where the cosine similarity metric criterion is where and are the normal vectors of the cluster and the cluster , respectively, and the normal vector of the cluster is calculated as the average of the normal vectors of all elements in the cluster;

[0074] S503: take each cluster newly merged into the plane S m in the S502 process as a plane S m ​new seed cluster, the step S502 is iteratively executed until there is no cluster in ψ satisfying the cosine similarity measure criterion;

[0075] S504: iteratively execute S501-S503 until ψ is empty, save the planes detected in each iteration S m to form a candidate plane feature set S;

[0076] S505: post-process the candidate plane feature set S, remove the candidate planes in S whose number of clusters is less than 3, remove the candidate planes whose area is less than k3, and take the remaining candidate planes as the final plane detection result.

[0077] The pseudo-color map of the set of boundary regular planes generated by the process is shown in Figure 5 , where each color represents a plane, and different colors represent different planes;

[0078] S6, as shown in Figure 6 , the Jaya meta-heuristic algorithm is used to optimize the plane feature detection process to automatically find the values of the threshold θ and k3 in the S5 plane detection process without manual and empirical setting, and a more ideal plane detection result is obtained. The process includes:

[0079] S601: first, determine the value range of θ and k3 in the plane feature detection process, respectively θ∈[10°, 45°], k3∈[2, 20];

[0080] S602: determine the unsupervised plane feature evaluation criterion to evaluate the quality of the result of the plane feature detection process;

[0081] where the unsupervised plane feature evaluation criterion is In the formula, S i is a plane in the set S, ω i is the ratio of the area of the plane S i to the sum of all planes in the set S, C f (·) and C i (·) are used to measure the flatness coefficient and completeness coefficient of a plane, respectively;

[0082] Further, In the formula, p j is a primitive in the plane S i , A is the area of p j , l is the number of primitives constituting the plane S i , a i , b i and c i are the coefficients of the plane S ithe length, width and height of the bounding rectangle of, min{·} is used to find the minimum value of the set elements;

[0083] Further, wherein, and are the second and third eigenvalues after principal component analysis of the plane S i

[0084] S603: using Jaya meta-heuristic optimization algorithm, iteratively select a set of threshold values from the value range, execute S5, and use the unsupervised plane feature evaluation criterion of S602 to evaluate the quality of detection to find the most appropriate threshold parameter value, that is, wherein θ' and k3' are the most appropriate values of θ and k3 found.

[0085] As Figure 7 shown, another embodiment of the present application also provides a plane feature detection device for real three-dimensional models, comprising:

[0086] a real three-dimensional model analysis module for inputting a real three-dimensional model, stitching all the tiles of the acquired real three-dimensional model into a complete three-dimensional model, and analyzing the real three-dimensional model into a set of geometric primitives;

[0087] a primitive heterogeneity distance formula module for inputting a primitive heterogeneity distance formula D(·), and calculating the similarity between two primitives according to the inputted two primitives;

[0088] a heterogeneity cost function module for inputting a heterogeneity cost function and its constraint conditions, and calculating the sum of the heterogeneity costs of the clusters according to the inputted set of clusters:

[0089] an energy optimization function module for inputting an energy optimization function E(·), and over-segmenting the inputted set of geometric primitives of the real three-dimensional model into a set of clusters ψ with uniform properties and regular boundaries;

[0090] a plane detection module for user inputting plane detection threshold values, and performing efficient plane detection based on the set of clusters ψ with uniform properties and regular boundaries to generate a set of boundary-regular planes;

[0091] a meta-heuristic optimization module for automatically finding the most appropriate threshold values of the plane detection module without manual and empirical setting, and achieving more ideal plane detection results.

[0092] wherein the real three-dimensional model analysis module requests the user to input at least one real three-dimensional model and analyzes the real three-dimensional model, including:

[0093] ​Model input interface: according to the file path of the real scene three-dimensional model input by the user, if the model is stored in the form of page LOD (level of detail) tile, recursively traverse to obtain all leaf node tiles of the model, that is, the highest resolution three-dimensional model tiles, and splice these tiles into a complete real scene three-dimensional model;

[0094] Model analysis interface: analyze the spliced real scene three-dimensional model, and parse it into a geometric primitive set, each primitive saves its index, vertex coordinates and texture mapping coordinates and other information;

[0095] Output interface: output the parsed primitive set.

[0096] Primitive heterogeneity distance formula module, comprising:

[0097] Primitive heterogeneity distance formula D(·) input interface: input the primitive heterogeneity distance formula;

[0098] Heterogeneity distance calculation interface: calculate the similarity between two primitives according to the input of the two primitives;

[0099] Output interface: output the calculated heterogeneity distance.

[0100] Heterogeneity cost function module, comprising:

[0101] Heterogeneity cost function input interface: for inputting the heterogeneity cost function and its constraint conditions;

[0102] Heterogeneity cost calculation interface: calculate the sum of the heterogeneity costs of the clusters according to the input of the cluster set;

[0103] Output interface: output the calculated heterogeneity cost.

[0104] Energy optimization function module, comprising:

[0105] Energy optimization function input interface: for inputting the energy optimization function E(·);

[0106] Over-segmentation interface: over-segment the geometric primitive set of the real scene three-dimensional model into a cluster set ψ with uniform properties and regular boundaries;

[0107] Output interface: output the over-segmented cluster set.

[0108] Plane detection module, comprising:

[0109] Parameter input interface: for receiving the specified parameters of plane detection;

[0110] Plane detection calculation interface: perform efficient plane detection to generate a set of boundary-regular planes;

[0111] Output interface: output the detected planar features.

[0112] Meta-heuristic optimization module, comprising:

[0113] Meta-heuristic optimization calculation interface: execute the meta-heuristic optimization method to automatically find the most suitable value of the threshold value of the planar detection module;

[0114] Result output interface: output the detected planar features after meta-heuristic optimization.

[0115] As shown in Figure 8 Another embodiment of the present application also provides a device for detecting planar features of a real scene three-dimensional model, the device comprising a distributed memory, a processor and a computer program stored in the memory and executable in the processor, wherein the processor executes the computer program to implement the steps of the method for detecting planar features of a real scene three-dimensional model, comprising:

[0116] stitching all the tiles of the real scene three-dimensional model into a complete three-dimensional model and parsing the real scene three-dimensional model to obtain a set of geometric primitives

[0117] constructing a primitive heterogeneity distance formula D(·) for evaluating the similarity between two primitives;

[0118] constructing a cluster heterogeneity cost function J(·) based on the primitive heterogeneity distance formula D(·) and determining its constraint condition, for judging the sum of the heterogeneity costs of all clusters:

[0119] constructing and solving an energy optimization function E(·) based on the heterogeneity cost function J(·) and its constraint condition, thereby over-segmenting the real scene three-dimensional model into a set of clusters ψ with uniform properties and regular boundaries;

[0120] based on the set of clusters ψ with uniform properties and regular boundaries, efficiently detecting the clusters in the set ψ to generate a set of planar surfaces with regular boundaries;

[0121] planar feature detection optimization based on a meta-heuristic algorithm to automatically find the value of the threshold value in the planar detection process of step 5 without manual and empirical setting, and to obtain more ideal planar detection results.

[0122] It should be noted that the above-mentioned embodiment provides the device for detecting the planar feature of the real three-dimensional model, and only the above-mentioned division of the program module is used for example to illustrate the device for detecting the planar feature of the real three-dimensional model, and in actual application, the above-mentioned processing can be completed by different program modules according to the needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-mentioned processing. In addition, the device for detecting the planar feature of the real three-dimensional model and the method for detecting the planar feature of the real three-dimensional model provided in the above-mentioned embodiment belong to the same concept, and the specific implementation process is shown in the method embodiment, which will not be repeated here.

[0123] The memory in the embodiment of the application is used for storing various types of data to support the operation of the device for detecting the planar feature of the real three-dimensional model. Examples of the data include any computer program used for operating the electronic device for detecting the planar feature of the real three-dimensional model.

[0124] The method for detecting the planar feature of the real three-dimensional model disclosed in the embodiment of the application can be applied to a processor or implemented by the processor. The processor can be an integrated circuit chip with the processing capability of signals. In the implementation process, each step of the method for detecting the planar feature of the real three-dimensional model can be completed by the integrated logic circuit of hardware in the processor or the instruction in the form of software. The processor can be a general processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the application, the hardware decoding processor can be directly embodied to execute the steps, or the hardware and software modules in the decoding processor can be combined to execute the steps. The software module can be located in a storage medium, and the storage medium is located in the memory. The processor reads the information in the memory and combines the hardware to complete the steps of the method for detecting the planar feature of the real three-dimensional model provided in the embodiment of the application.

[0125] The planar feature detection apparatus for a live three-dimensional model in the exemplary embodiment can be implemented with one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), FPGAs, general purpose processors, controllers, microcontrollers (MCUs), microprocessors (Microprocessors), or other electronic elements for executing the aforementioned methods.

[0126] It is to be understood that the memory can be a volatile memory or a nonvolatile memory, and can also include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The memory described in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0127] In the example embodiment, the present embodiment also provides a storage medium, i.e. a computer storage medium, specifically a computer readable storage medium, such as a memory storing a computer program executable by a processor of a device for detecting planar features from a real-world three-dimensional model, the steps of the method for detecting planar features from a real-world three-dimensional model. The computer readable storage medium can be a ROM, a PROM, an EPROM, an EEPROM, a Flash Memory, a magnetic surface memory, an optical disc, or a CD-ROM, etc.

[0128] In summary, the present application detects planar features from a real-world three-dimensional model, and provides support for subsequent work such as building extraction, objectization, data compression, model simplification, and roof extraction, etc. from the real-world three-dimensional model, and has a high degree of automation.

[0129] It should be understood that parts not described in detail in the specification are all prior art.

[0130] It should be understood that the above description of the embodiments is detailed, and therefore should not be considered as limiting the scope of patent protection of the present application. Any modifications, equivalent replacements, improvements, etc. made without departing from the scope of protection of the claims of the present application shall fall within the scope of protection of the present application. The scope of protection of the present application shall be subject to the appended claims.

Claims

1. A planar feature detection method for a real scene three-dimensional model, characterized in that, The method comprises the following steps: Step S1, all tiles of a real scene three-dimensional model are spliced into a complete three-dimensional model, and the real scene three-dimensional model is parsed into a geometric primitive set; Step S2, a primitive heterogeneity distance formula is constructed to evaluate the similarity between two primitives; The primitive heterogeneity distance formula for evaluating the similarity between two primitives in step S2 is denoted as D(·), which is constructed by comprehensively considering the spatial proximity feature, surface feature and color feature of the primitives. The heterogeneity distance formula between the i-th primitive p i and the j-th primitive p j is constructed as: D(p i ,p j ) = μ1D s (p i ,p j ) + μ2D e (p i ,p j ) + μ3D c (p i ,p j ), wherein D s (p i ,p j ), D e (p i ,p j ) and D c (p i ,p j ) are respectively the normalized surface feature difference distance, spatial proximity distance and color difference distance between the two primitives, and μ1, μ2 and μ3 are respectively the weight factors corresponding to the three distances, and the value range of the three is [0, 1]. Step S3, based on the primitive heterogeneity distance formula, a cluster heterogeneity cost function is constructed and its constraint condition is determined to judge the sum of the heterogeneity costs of all clusters; In step S3, the heterogeneity cost function of the cluster is denoted as J(·) and its constraint is determined, specifically: wherein If r ij = 1 represents the basic element p i , and the cluster contains all non-central basic elements satisfying r ij = 0; the constraint of J(r ij ) is wherein I(·) is an exponential function, k represents the expected number of clusters; D(p i , p j ) represents the heterogeneity distance formula between the i-th basic element p i and the j-th basic element p j , and n is the number of basic elements; Step S4, based on the heterogeneity cost function and its constraint condition, an energy optimization function is constructed and solved to over-segment the real scene three-dimensional model into a set of clusters with uniform properties and regular boundaries; Step S5, based on the set of clusters with uniform properties and regular boundaries, efficient plane detection is performed on the cluster set in combination with a plane detection threshold to generate a set of boundary-regular planes; Step S6, plane feature detection optimization is performed based on a meta-heuristic algorithm to automatically find the optimal threshold in the plane detection process in step S5 to obtain more ideal plane detection results.

2. The method of claim 1, wherein: Spatial proximity distance D between two primitives e (p i ,p j ) The Euclidean distance is chosen: For the qth vertex of primitive p i , q is at most 3 if the primitive is a triangle; p i and p j The surface feature difference distance between two primitives can be measured by the normalized cosine similarity formula and are the normal vectors of p i and p j , i.e. p i and p j The color difference distance D c (p i ,p j ) is computed in the CIE Lab linear color space, using a distance formula that is more consistent with the human visual non-linear perception, i.e. where, is the average texture color value of primitive p i in the CIE Lab space.

3. The method of claim 2, wherein: The normal vector calculation method of the primitive is: using the space coordinates of the three vertices v1, v2 and v3 of the triangular primitive to calculate the normal vector of the primitive: Average color texture value The calculation method is as follows: calculating the primitive p in the Adobe RGB color space i In the spatial range in the y direction and the number of scan lines, from top to bottom, for any scan line, intersect all edges of the primitive with the scan line, and sort the horizontal coordinates obtained by intersection from left to right. The edge intersected by the odd scan line is the entry edge, and the edge intersected by the even scan line is the exit edge. Then, the spatial coordinates of the pixels on the scan line between the entry edge and the exit edge are calculated by interpolation. The barycentric coordinate method is used to calculate the UV coordinates of all pixel points in the primitive. The U and V coordinates are respectively Where S a is the area of the triangle formed by the coordinates of the pixel point and the v1 and v2 vertices of the primitive, S b is the area of the triangle formed by the pixel point and the v1 and v3 vertices of the primitive, and S c is the area of the triangle formed by the pixel point and the v2 and v3 vertices of the primitive. U1, U2, U3, V1, V2, and V3 are respectively the UV coordinates of v1, v2, and v3. S t =S a +S b +S c After all the UV coordinates are solved, the texture value is obtained from the texture image corresponding to the primitive using the UV coordinates, and the average value of all texture values is taken as the texture value of the primitive. Then, the Adobe RGB color space is converted into the CIE Lab color space.

4. The method of claim 1, wherein: In step S4, let the energy function be E(·), and let the real scene three-dimensional model be over-segmented into a set of clusters with uniform properties and regular boundaries, denoted as ψ, which is specifically: let the energy function be An up-down based merging energy minimization method is selected to solve the energy function, so as to segment the geometric primitive set of the input real scene three-dimensional model into the set of clusters ψ. Each primitive in the geometric primitive set except the central primitive set is assigned to a cluster according to a mapping function where D(·) is a primitive heterogeneity distance formula, and the total sum of the heterogeneity distances between each primitive and the central primitive of the cluster to which it is classified is minimized, so as to over-segment the geometric primitive set of the input real scene three-dimensional model into the set of clusters ψ with uniform properties and regular boundaries.

5. The method of claim 1, wherein: The specific implementation of step S5 is as follows: S501: For the cluster set ψ, select one cluster in the set ψ as a plane S m seed cluster, and remove the seed cluster from the cluster set ψ m is composed of a subset of the cluster set ψ; S502: Compute the set of k1 neighboring clusters around the seed cluster ψ ′ For each neighboring cluster in ψ ′ , determine whether it has the same properties as the seed cluster according to a similarity criterion, i.e. if it satisfies the planar similarity criterion, i.e. then incorporate it into the planar seed cluster S m and remove it from the set ψ, if it does not satisfy the planar similarity criterion, then do nothing, where θ is an angle threshold, denotes the cluster, D s denotes the cosine similarity measure; S503: incorporating the new cluster in the plane S m of S502 as a new seed cluster of the plane S m S502 is iteratively performed until no cluster in ψ satisfies the plane similarity criterion. S504: iteratively perform the process of S501-S503 until ψ is empty, save the detected planes S in each iteration m to constitute a candidate plane feature set S; S505: post-processing is performed on the candidate plane feature set S to remove candidate planes with a cluster number less than k2 and a candidate plane with an area less than k3, and the remaining candidate planes are taken as the final plane detection results.

6. The method of claim 5, wherein: The plane feature detection optimization based on the meta-heuristic algorithm in step S6 comprises the following steps: S601: the value range of each threshold in the plane feature detection process, i.e. the value range of θ and k3, is determined; S602: an unsupervised plane feature evaluation criterion is determined to evaluate the quality of the results of the plane feature detection process; S603: a meta-heuristic optimization algorithm is adopted to iteratively select a set of threshold values from the value range, execute step S5, and evaluate the quality of the detection using the unsupervised plane feature evaluation criterion in S602 to find the most suitable parameter value of the threshold.

7. An apparatus for planar feature detection of a real scene three-dimensional model, characterized by, The method comprises the following modules: A real scene three-dimensional model parsing module is configured to input a real scene three-dimensional model, splice all tiles of the acquired real scene three-dimensional model into a complete three-dimensional model, and parse the real scene three-dimensional model into a geometric primitive set; A primitive heterogeneity distance formula module is configured to input a primitive heterogeneity distance formula and calculate the similarity between two input primitives; A primitive heterogeneity distance formula for evaluating the similarity between two primitives is denoted as D(·), which is constructed by comprehensively considering the spatial proximity feature, surface feature and color feature of the primitives, and the heterogeneity distance formula between the i-th primitive p i and the j-th primitive p j is constructed as: D(p i ,p j ) = μ1D s (p i ,p j ) + μ2D R (p i ,p j ) + μ3D c (p i ,p j ), wherein D s (p i ,p j ), D e (p i ,p j ) and D c (p i ,p j ) are respectively the normalized surface feature difference distance, spatial proximity distance and color difference distance between the two primitives, μ1, μ2 and μ3 are respectively the weight factors corresponding to the three, and the value range of the three is [0, 1]; A heterogeneity cost function module is configured to input a heterogeneity cost function and its constraint condition and calculate the sum of the heterogeneity costs of a set of input clusters; Let the heterogeneity cost function of a cluster be denoted as J(·) and its constraints be determined as follows: where If r ij = 1 indicates that the primitive p i is the central primitive of a cluster, and this cluster contains all the non-central primitives that satisfy r ij = 0; the constraints of J(r ij ) are where I(·) is an exponential function, k represents the expected number of clusters; D(p i , p j ) represents the heterogeneity distance formula between the i-th primitive p i and the j-th primitive p j , and n is the number of primitives. An energy optimization function module is configured to input an energy optimization function and over-segment a geometric primitive set of an input real scene three-dimensional model into a set of clusters with uniform properties and regular boundaries; A plane detection module is configured to input a plane detection threshold and perform efficient plane detection based on a set of clusters with uniform properties and regular boundaries to generate a set of boundary-regular planes; A meta-heuristic optimization module is configured to perform plane feature detection optimization based on a meta-heuristic algorithm to automatically find the most suitable value of the threshold of the plane detection module to obtain more ideal plane detection results.

8. An electronic device comprising a distributed memory, a processor and a computer program in the memory and executable in the processor, characterized in that: The processor implements the steps of the method for detecting planar features of a real-world three-dimensional model according to any one of claims 1 to 6 when executing the computer program.

Images