A Geographic Entity-Oriented Classification Method and System for Real-Scene 3D Models

Abstract

This invention discloses a method and system for classifying real-world 3D models based on geographic entity guidance, relating to the field of 3D data processing technology. The method includes the following steps: Step S1, performing initial oversegmentation on the input 3D mesh model based on geometric features to generate a set of 3D regions; Step S2, determining the optimal projection coordinate system for each 3D region and generating fixed-resolution 2D image patches through adaptive orthogonal projection; Step S3, extracting the 3D ontology features of each 3D region and the 2D image features of the 2D image patches; Step S4, fusing the 3D ontology features and the 2D image features to obtain a global feature vector; Step S5, inputting the global feature vector into a pre-trained classification model and outputting the geographic entity category to which each 3D region belongs. This invention achieves geographic entity classification of real-world 3D models through adaptive projection, feature fusion, and classification.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional data processing technology, specifically to a method and system for classifying real-world three-dimensional models based on geographic entity guidance. Background Technology

[0002] With the rapid development of real-scene 3D modeling technology, its application in fields such as smart cities and digital twins is becoming increasingly widespread. Accurate geographic entity classification is the foundation for subsequent applications of real-scene 3D models, such as quality inspection and the creation of digital twin platforms.

[0003] However, most existing classification methods rely on features from a single perspective or a single modality, making it difficult to fully capture the complete features of geographic entities. Secondly, in terms of feature extraction, they often only focus on single factors in appearance texture or geometric shape, failing to effectively integrate multi-source features, resulting in limited classification accuracy for geographic entities with complex shapes.

[0004] For example, Chinese patent CN111460193A discloses a 3D model classification method based on multimodal information fusion. The method includes the following steps: extracting view information and panoramic image information of the 3D model using an open source; extracting point cloud information of the 3D model using a point cloud library; learning a multimodal network model by training a multimodal network independently using a single-modal network; obtaining prediction scores of the 3D model under different modalities using the multimodal network model; aggregating the prediction scores under different modalities by constructing an aggregation function to generate a final prediction score; and processing the final prediction score using the Softmax algorithm to generate class labels. This invention characterizes the 3D model by fusing information from three modalities, providing more comprehensive information than using only single-modal information, thus improving the accuracy of 3D model classification.

[0005] For example, Chinese patent CN106845543A discloses a 3D model classification method based on feature matching, including: extracting Gaussian kernels from the training vector set and defining them as multi-view training Gaussian kernels and single-view training Gaussian kernels respectively; defining the Gaussian kernels as multi-view training feature libraries and single-view training feature libraries respectively; constructing an objective function using multi-view training features and single-view training features and iteratively minimizing it to obtain a feature matching matrix; randomly selecting one view from the multi-view color view set of each object in the data to be classified to obtain the initial single-view view and category label of each object; extracting the convolutional neural network features of the single-view view and calculating the single-view Gaussian kernel of the features, defining it as a single-view feature library; multiplying the single-view feature library by its transformation function to obtain the mapped features; multiplying the view training features by another transformation function to obtain the mapped features; and calculating the cosine distance between features to obtain the similarity. This invention avoids the limitation that features must be in the same space. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a method and system for classifying real-world 3D models based on geographic entity guidance.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The geographic entity-oriented real-world 3D model classification method includes the following steps:

[0009] Step S1: Perform initial oversegmentation based on geometric features on the input 3D mesh model to generate a set of 3D regions;

[0010] Step S2: Determine the optimal projection coordinate system for each of the three-dimensional regions, and generate a two-dimensional image block with a fixed resolution through adaptive orthogonal projection;

[0011] Step S3: Extract the three-dimensional ontological features of each three-dimensional region and the two-dimensional image features of each two-dimensional image block;

[0012] Step S4: Fuse the three-dimensional ontology features and the two-dimensional image features to obtain a global feature vector;

[0013] Step S5: Input the global feature vector into the pre-trained classification model and output the geographic entity category to which each three-dimensional region belongs.

[0014] Furthermore, step S1 specifically includes the following steps:

[0015] Step S1.1: For each vertex in the 3D mesh model, calculate its features in the dimensions of color, normal, curvature, and spatial position, and construct the vertex feature vector;

[0016] Step S1.2: Based on the vertex feature vector, calculate the comprehensive feature difference value between any two adjacent vertices;

[0017] Step S1.3: Based on the comprehensive feature differences, the entire three-dimensional mesh is divided into a set of three-dimensional regions through an iterative region growing algorithm.

[0018] Further, in step S1.2, the specific formula for calculating the comprehensive feature difference value is as follows:

[0019]

[0020] in, Represents vertices and vertex The comprehensive characteristic difference value, This represents the Euclidean distance between the color vectors of two vertices in the Lab color space. This represents the Euclidean distance between the normal vectors of two vertices. This represents the Euclidean distance between the three-dimensional coordinates of two vertices. This represents the difference in the absolute values ​​of the curvature at two vertices. , , and These represent the corresponding weighting coefficients, where the sum of the weighting coefficients is 1.

[0021] Further, in step S1.3, the iterative region growth algorithm specifically includes: arranging all vertices in ascending order of curvature value, selecting vertices with curvature lower than a preset curvature threshold as initial seed points; initializing each initial seed point as an initial growth region; traversing the spatial neighborhood of each initial growth region boundary vertex, merging adjacent vertices with a comprehensive feature difference value less than a feature threshold into the current growth region; iteratively executing the above process until all vertices are assigned to the corresponding growth regions, and finally outputting a set of three-dimensional regions.

[0022] Furthermore, the specific method for determining the feature threshold includes: when the number of vertices contained in the region is less than the preset minimum number of vertices, a fixed basic threshold is used as the feature threshold; when the number of vertices contained in the region is greater than or equal to the preset minimum number of vertices, a dynamic threshold is calculated based on the internal feature variance of the region as the feature threshold.

[0023] The formula for calculating the dynamic threshold is as follows:

[0024]

[0025] in, Indicates the dynamic feature threshold. Indicates the basic threshold. This represents the variance of characteristics within a region. Represents the sensitivity coefficient. This represents the reference variance.

[0026] Furthermore, step S2 specifically includes the following steps:

[0027] Step S2.1: Collect the three-dimensional coordinates of all vertices within the three-dimensional region to form a vertex coordinate point set;

[0028] Step S2.2: Calculate the centroid and covariance matrix of the vertex coordinate point set;

[0029] Step S2.3: Perform eigenvalue decomposition on the covariance matrix to obtain three mutually orthogonal eigenvectors, thereby constructing the optimal projected coordinate system;

[0030] Step S2.4: Using the centroid as the projection target point, project along the six positive and negative axes of the optimal projection coordinate system;

[0031] Step S2.5: For each projection direction, generate a two-dimensional image block with a fixed resolution through orthogonal projection. Each two-dimensional image block is a data set containing RGB color, depth information and surface normal data.

[0032] Furthermore, in step S2.2, the specific formula for the covariance matrix is ​​as follows:

[0033]

[0034] in, Represents the covariance matrix. This represents the number of vertices in the three-dimensional region. Represents the vertex index. Represents the first point set Coordinates of a point, Represents the centroid coordinates of the point set.

[0035] Furthermore, in step S3, the three-dimensional ontological features include geometric features and spatial distribution features, wherein the geometric features include the region surface area, volume, compactness, and the length, width, and height of the three-dimensional bounding box, and the spatial distribution features include the region elevation and the height of the region centroid from the ground.

[0036] The two-dimensional image features are extracted from the RGB color, depth information and surface normal data of the two-dimensional image block using a convolutional neural network. Max pooling is performed on the same channel features under all viewpoints, and the pooled feature vectors are concatenated to obtain the two-dimensional image features.

[0037] Furthermore, step S4 specifically includes the following steps:

[0038] Based on the first projection matrix determined in advance through principal component analysis, the two-dimensional image feature vector is subjected to dimensionality reduction processing to obtain a dimensionality-reduced two-dimensional feature vector.

[0039] Based on the pre-calculated maximum and minimum values, the 3D ontology feature vectors are subjected to maximum-minimum normalization processing;

[0040] The reduced 2D feature vector is concatenated with the normalized 3D ontology feature vector to form the global feature vector.

[0041] Furthermore, step S5 specifically includes the following steps:

[0042] The global feature vector is input into a pre-trained random forest classifier;

[0043] The input features are voted on using decision trees in the random forest classifier;

[0044] Aggregate the voting results of all decision trees and output the confidence score of each geographic entity category in probabilistic form;

[0045] The category with the highest confidence level is selected as the geographic entity category to which the three-dimensional region belongs.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] 1. This invention, through multi-view adaptive projection and fusion of three-dimensional and two-dimensional features, can accurately identify different types of geographical entities such as buildings, roads, and water bodies, laying a solid foundation for subsequent quality inspection.

[0048] 2. This invention uses a method for determining the optimal projection coordinate system based on principal component analysis to ensure that geographic entity features are captured from the most representative perspective, thereby improving the relevance and effectiveness of feature extraction.

[0049] 3. This invention adopts an automatic classification method based on region growing and feature fusion, which reduces the reliance on professional quality inspectors, effectively reduces labor costs, and solves the problem of high costs in traditional methods. Attached Figure Description

[0050] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0051] Figure 1 This is a flowchart illustrating an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the region growing algorithm according to an embodiment of the present invention;

[0053] Figure 3 This is a flowchart of the random forest classification decision-making process according to an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] like Figure 1 As shown, the geographic entity-oriented real-world 3D model classification method includes the following steps:

[0056] Step S1: Perform initial oversegmentation based on geometric features on the input 3D mesh model to generate a set of 3D regions;

[0057] Step S2: Determine the optimal projection coordinate system for each of the three-dimensional regions, and generate a two-dimensional image block with a fixed resolution through adaptive orthogonal projection;

[0058] Step S3: Extract the three-dimensional ontological features of each three-dimensional region and the two-dimensional image features of each two-dimensional image block;

[0059] Step S4: Fuse the three-dimensional ontology features and the two-dimensional image features to obtain a global feature vector;

[0060] Step S5: Input the global feature vector into the pre-trained classification model and output the geographic entity category to which each three-dimensional region belongs.

[0061] Step S1 specifically includes the following steps:

[0062] Step S1.1: For each vertex in the 3D mesh model, calculate its features in the dimensions of color, normal, curvature, and spatial position, and construct the vertex feature vector;

[0063] The vertex feature vector specifically includes the following features:

[0064] Geometric features include vertex normals and curvature estimation;

[0065] Image features include: mapping model textures back to 3D vertices, calculating vertex color values ​​in the RGB color space, and converting to the CIELab space to obtain perceptually uniform color features.

[0066] Spatial features include vertex coordinates.

[0067] Step S1.2: Based on the vertex feature vector, calculate the comprehensive feature difference value between any two adjacent vertices;

[0068] Step S1.3: Based on the comprehensive feature differences, the entire three-dimensional mesh is divided into a set of three-dimensional regions through an iterative region growing algorithm.

[0069] In step S1.2, the specific formula for calculating the comprehensive feature difference value is as follows:

[0070]

[0071] in, Represents vertices and vertex The comprehensive characteristic difference value, This represents the Euclidean distance between the color vectors of two vertices in the Lab color space. This represents the Euclidean distance between the normal vectors of two vertices. This represents the Euclidean distance between the three-dimensional coordinates of two vertices. This represents the difference in the absolute values ​​of the curvature at two vertices. , , and These represent the corresponding weighting coefficients, where the sum of the weighting coefficients is 1.

[0072] The specific settings for the weighting coefficients are as follows:

[0073] Color vector weights The highest value is 0.4, because most geographic entities have obvious texture features;

[0074] Normal weight A value of 0.3 indicates sensitivity to geometric boundaries.

[0075] Curvature weight Setting it to 0.2 is specific to entities of particular shapes;

[0076] Spatial weight The minimum value is set to 0.1, which mainly serves as a smoothing constraint.

[0077] like Figure 2 As shown, in step S1.3, the iterative region growth algorithm includes: arranging all vertices in ascending order of curvature value, selecting vertices with curvature lower than a preset curvature threshold as initial seed points; initializing each initial seed point as an initial growth region; traversing the spatial neighborhood of each initial growth region boundary vertex, merging adjacent vertices with a comprehensive feature difference value less than a feature threshold into the current growth region; iteratively executing the above process until all vertices are assigned to the corresponding growth regions, and finally outputting a set of three-dimensional regions.

[0078] The curvature threshold is determined based on the average curvature of the entire 3D mesh model;

[0079] The specific method for determining the feature threshold includes: when the number of vertices contained in the region is less than the preset minimum number of vertices, a fixed basic threshold is used as the feature threshold; when the number of vertices contained in the region is greater than or equal to the preset minimum number of vertices, a dynamic threshold is calculated based on the internal feature variance of the region as the feature threshold.

[0080] The formula for calculating the dynamic threshold is as follows:

[0081]

[0082] in, Indicates the dynamic feature threshold. Indicates the basic threshold. This represents the variance of characteristics within a region. Represents the sensitivity coefficient. This represents the reference variance.

[0083] Step S2 specifically includes the following steps:

[0084] Step S2.1: Collect the three-dimensional coordinates of all vertices within the three-dimensional region to form a vertex coordinate point set;

[0085] Step S2.2: Calculate the centroid and covariance matrix of the vertex coordinate point set;

[0086] Step S2.3: Perform eigenvalue decomposition on the covariance matrix to obtain three mutually orthogonal eigenvectors, thereby constructing the optimal projected coordinate system;

[0087] The three eigenvalues ​​of the covariance matrix are arranged from largest to smallest, each corresponding to a eigenvector. The eigenvector corresponding to the largest eigenvalue represents the direction in which the region extends the longest in three-dimensional space. The eigenvector corresponding to the second largest eigenvalue represents the direction in which the region extends the second longest in a plane perpendicular to the eigenvector corresponding to the largest eigenvalue. The eigenvector corresponding to the smallest eigenvalue represents the direction in which the region is thinnest, which usually corresponds to the normal direction of a planar region.

[0088] Step S2.4: Using the centroid as the projection target point, project along the six positive and negative axes of the optimal projection coordinate system. These six perspectives ensure that the observation is performed from the most important structural direction of the region, avoiding the loss of information from a single perspective.

[0089] Step S2.5: For each projection direction, generate a two-dimensional image block with a fixed resolution through orthogonal projection. Each two-dimensional image block is a data set containing RGB color, depth information and surface normal data.

[0090] Among them, RGB colors retain color and appearance information by directly rendering the original texture map of the area;

[0091] Depth information records the vertical distance from the surface point of the 3D region corresponding to each pixel to the projection plane.

[0092] The surface normal records the unit normal vector of the surface point of the three-dimensional region corresponding to each pixel.

[0093] In step S2.2, the specific formula for the covariance matrix is ​​as follows:

[0094]

[0095] in, Represents the covariance matrix. This represents the number of vertices in the three-dimensional region. Represents the vertex index. Represents the first point set Coordinates of a point, Represents the centroid coordinates of the point set.

[0096] In step S3, the three-dimensional ontological features include geometric features and spatial distribution features. The geometric features include the region surface area, volume, compactness, and the length, width, and height of the three-dimensional bounding box. The spatial distribution features include the region elevation and the height of the region's centroid from the ground.

[0097] The two-dimensional image features are extracted from the RGB color, depth information and surface normal data of the two-dimensional image block using a convolutional neural network. Max pooling is performed on the same channel features under all viewpoints, and the pooled feature vectors are concatenated to obtain the two-dimensional image features.

[0098] Specifically, the convolutional neural network extracts features including three-channel feature extraction;

[0099] In the RGB color channel, the input data is an RGB color image patch with a size of 224 pixels × 224 pixels × 3 channels. The ResNet-34 network pre-trained on the ImageNet dataset is used as the basic feature extractor. The last fully connected classification layer of the original ResNet-34 is removed, and all convolutional and pooling layers are retained. Features are extracted from the fourth convolutional block of the network, and the output feature map has a dimension of 7×7×2048. After global average pooling, a 2048-dimensional feature vector is obtained.

[0100] In the depth information channel, the input data is a single-channel depth map, which is converted into a 224×224×3 pseudo-color image through channel copying. The ResNet-34 network with the same structure as the RGB channel is used. Features are extracted from layer 4 and obtained by global average pooling to obtain a 2048-dimensional depth feature vector.

[0101] In the surface normal channel, the input data is a three-channel normal map, with each channel corresponding to the X, Y, and Z components of the normal vector. A 2048-dimensional normal feature vector is obtained through the same feature extraction process.

[0102] Step S4 specifically includes the following steps:

[0103] Based on the first projection matrix determined in advance through principal component analysis, the two-dimensional image feature vector is subjected to dimensionality reduction processing to obtain a dimensionality-reduced two-dimensional feature vector.

[0104] Based on the pre-calculated maximum and minimum values, the 3D ontology feature vectors are subjected to maximum-minimum normalization processing;

[0105] The reduced 2D feature vector is concatenated with the normalized 3D ontology feature vector to form the global feature vector.

[0106] Among them, dimensionality reduction of two-dimensional image features uses principal component analysis for linear dimensionality reduction, specifically including:

[0107] Perform PCA on the two-dimensional feature set of all training samples, calculate the first d principal component directions to form the first projection matrix, and calculate the mean vector of the training set features.

[0108] The dimensionality-reduced two-dimensional feature vector is obtained by subtracting the mean vector from the two-dimensional image features and multiplying it by the first projection matrix.

[0109] Among them, dimension d is determined by setting the cumulative variance contribution rate threshold to 95%;

[0110] 3D ontology feature normalization processes the 3D ontology feature vectors using max-min normalization, specifically including:

[0111] Calculate the minimum and maximum values ​​for each three-dimensional feature dimension on the training set;

[0112] Max-min normalization maps all feature dimensions to the interval [0, 1].

[0113] The dimensionality-reduced two-dimensional features and the normalized three-dimensional features are directly concatenated into vectors to form the final global feature vector.

[0114] like Figure 3 As shown, step S5 specifically includes the following steps:

[0115] The global feature vector is input into a pre-trained random forest classifier;

[0116] The input features are voted on using decision trees in the random forest classifier;

[0117] Aggregate the voting results of all decision trees and output the confidence score of each geographic entity category in probabilistic form;

[0118] The category with the highest confidence level is selected as the geographic entity category to which the three-dimensional region belongs.

[0119] The number of decision trees in the random forest classifier is set to be between 100 and 500, the maximum depth of each decision tree is limited to 10 to 20 layers, the number of features considered when splitting a node is the square root of the total dimension of the global feature vector, the minimum number of samples for a leaf node is set to 1 to 5, and Gini impurity is used as the node splitting criterion.

[0120] The examples described herein are merely preferred embodiments of the invention and are not intended to limit the concept and scope of the invention. Any modifications and improvements made by those skilled in the art to the technical solutions of the invention without departing from the design concept of the invention should fall within the protection scope of the invention.

Claims

1. A geographic entity-oriented real-scene 3D model classification method, characterized in that, Includes the following steps: Step S1: Perform initial oversegmentation based on geometric features on the input 3D mesh model to generate a set of 3D regions; Step S2: Determine the optimal projection coordinate system for each of the three-dimensional regions, and generate a two-dimensional image block with a fixed resolution through adaptive orthogonal projection; Step S3: Extract the three-dimensional ontological features of each three-dimensional region and the two-dimensional image features of each two-dimensional image block; Step S4: Fuse the three-dimensional ontology features and the two-dimensional image features to obtain a global feature vector; Step S5: Input the global feature vector into the pre-trained classification model and output the geographic entity category to which each three-dimensional region belongs; Specifically, step S1 includes the following steps: Step S1.1: For each vertex in the 3D mesh model, calculate its features in the dimensions of color, normal, curvature, and spatial position, and construct the vertex feature vector; Step S1.2: Based on the vertex feature vector, calculate the comprehensive feature difference value between any two adjacent vertices; Step S1.3: Based on the comprehensive feature differences, the entire three-dimensional mesh is divided into a set of three-dimensional regions using an iterative region growing algorithm; In step S1.2, the specific formula for calculating the comprehensive feature difference value is as follows: in, Represents vertices and vertex The comprehensive characteristic difference value, This represents the Euclidean distance between the color vectors of two vertices in the Lab color space. This represents the Euclidean distance between the normal vectors of two vertices. This represents the Euclidean distance between the three-dimensional coordinates of two vertices. This represents the difference in the absolute values ​​of the curvature at two vertices. , , and These represent the corresponding weight coefficients, where the sum of the weight coefficients is 1; In step S1.3, the iterative region growth algorithm specifically includes: arranging all vertices in ascending order of curvature value, selecting vertices with curvature lower than a preset curvature threshold as initial seed points; initializing each initial seed point as an initial growth region; traversing the spatial neighborhood of each initial growth region boundary vertex, merging adjacent vertices with a comprehensive feature difference value less than a feature threshold into the current growth region; iteratively executing the above process until all vertices are assigned to the corresponding growth regions, and finally outputting a set of three-dimensional regions; Step S2 specifically includes the following steps: Step S2.1: Collect the three-dimensional coordinates of all vertices within the three-dimensional region to form a vertex coordinate point set; Step S2.2: Calculate the centroid and covariance matrix of the vertex coordinate point set; Step S2.3: Perform eigenvalue decomposition on the covariance matrix to obtain three mutually orthogonal eigenvectors, thereby constructing the optimal projected coordinate system; Step S2.4: Using the centroid as the projection target point, project along the six positive and negative axes of the optimal projection coordinate system; Step S2.5: For each projection direction, generate a two-dimensional image block with a fixed resolution through orthogonal projection. Each two-dimensional image block is a data set containing RGB color, depth information and surface normal data.

2. The method according to claim 1, characterized in that, The specific method for determining the feature threshold includes: when the number of vertices contained in the region is less than the preset minimum number of vertices, a fixed basic threshold is used as the feature threshold; when the number of vertices contained in the region is greater than or equal to the preset minimum number of vertices, a dynamic threshold is calculated based on the internal feature variance of the region as the feature threshold. The formula for calculating the dynamic threshold is as follows: in, Indicates the dynamic feature threshold. Indicates the basic threshold. This represents the variance of characteristics within a region. Represents the sensitivity coefficient. This represents the reference variance.

3. The method according to claim 2, characterized in that, In step S2.2, the specific formula for the covariance matrix is ​​as follows: in, Represents the covariance matrix. This represents the number of vertices in the three-dimensional region. Represents the vertex index. Represents the first point set Coordinates of a point, Represents the centroid coordinates of the point set.

4. The method according to claim 3, characterized in that, In step S3, the three-dimensional ontological features include geometric features and spatial distribution features. The geometric features include the region surface area, volume, compactness, and the length, width, and height of the three-dimensional bounding box. The spatial distribution features include the region elevation and the height of the region's centroid from the ground. The two-dimensional image features are extracted from the RGB color, depth information and surface normal data of the two-dimensional image block using a convolutional neural network. Max pooling is performed on the same channel features under all viewpoints, and the pooled feature vectors are concatenated to obtain the two-dimensional image features.

5. The method according to claim 4, characterized in that, Step S4 specifically includes the following steps: Based on the first projection matrix determined in advance through principal component analysis, the vector composed of the two-dimensional image features is subjected to dimensionality reduction processing to obtain a dimensionality-reduced two-dimensional feature vector. Based on the pre-calculated maximum and minimum values, the vector composed of the three-dimensional ontology features is subjected to maximum and minimum normalization processing. The reduced two-dimensional feature vector is concatenated with the normalized vector composed of the three-dimensional ontological features to form the global feature vector.

6. The method according to claim 5, characterized in that, Step S5 specifically includes the following steps: The global feature vector is input into a pre-trained random forest classifier; The input features are voted on using decision trees in the random forest classifier; Aggregate the voting results of all decision trees and output the confidence score of each geographic entity category in probabilistic form; The category with the highest confidence level is selected as the geographic entity category to which the three-dimensional region belongs.

Images