Lightweight three-dimensional model retrieval method based on point cloud improved VFH features

Abstract

The invention belongs to the field of automation and graphics of mechanical design and assembly, and provides a lightweight three-dimensional model retrieval method based on point cloud improved VF H features, comprising the following steps: step 1, processing and converting existing three-dimensional model data, and normalizing; 2, extracting the improved VFH features of the pcd file based on a PCL library, intercepting the first 135 dimensions of the improved VFH features as point cloud features, and storing the point cloud features; 3, calculating the feature center of the three-dimensional model point cloud according to the category, and taking the feature center as the feature of the category; 4, extracting improved VFH features of the three-dimensional model to be retrieved; 5, the Euclidean distance between the feature and the category center feature is calculated to classify the model; and step 6, calculating Euclidean distances between the features of all models in the class and sorting the Euclidean distances so as to complete a retrieval task. According to the method, the problem that the retrieval efficiency of a deep learning method is low is solved, and meanwhile, the retrieval effect on three-dimensional models with large category differences is good, so that the retrieval time consumption can be reduced under the condition that the retrieval effect is ensured.

Description

Technical Field

[0001] The invention belongs to the field of automation and graphics of mechanical design and assembly, and specifically relates to a lightweight three-dimensional model retrieval method based on three-dimensional point cloud improved VFH features. Background Art

[0002] Three-dimensional models are favored by the industrial design and manufacturing fields due to their intuitive and relatively realistic characteristics. At present, the vigorous development of CAD technology has made the acquisition and processing of three-dimensional models easier, which has also led to an explosive growth in the number of three-dimensional models.

[0003] Directly designing a 3D model is often complicated, but improving existing 3D models can greatly reduce the workload. However, there are two problems with text-based model retrieval methods: each model has more than one shape feature, so it is difficult to describe it with a short text; different naming habits may result in different features corresponding to the same text. Therefore, the need for algorithms to retrieve 3D models with shapes that meet the requirements from a huge 3D model database is becoming more and more urgent, and content-based 3D model retrieval methods are also booming.

[0004] Existing content-based 3D model retrieval algorithms are divided into learning-based methods and feature descriptor-based algorithms, among which:

[0005] 1) Learning-based methods can extract features that are not explainable to humans through deep learning, and the effect is generally better than that of manually extracted features. However, learning-based methods require a lot of time for training, and each time when the data is updated in large batches, additional training is required, so the time cost is very high. In addition, learning-based methods require a large amount of data, but enterprises may not have enough data for learning, so lighter algorithms are needed.

[0006] 2) Traditional feature descriptor-based methods design a feature descriptor that can describe the entire model and retrieve the model based on the feature descriptor. Although some methods consider extracting local features of the model and combining them into global features in a certain way, they do not consider that different local features have different representation effects in the entire model. Complex local features have a stronger representation effect on the entire model, but existing methods do not emphasize the weight of complex local features on the entire model.

[0007] Some explanations of terms involved in the present invention:

[0008] Point cloud: A form used to describe a three-dimensional model. The surface of the three-dimensional model is sampled into several points, and the three-dimensional coordinates of each point are recorded as the point cloud of the three-dimensional model.

[0009] pcd: point cloud data, a file format for storing three-dimensional point clouds.

[0010] Normalization: The coordinates of the point cloud are proportionally compressed between 0 and 1, thereby ignoring the effect of model size on shape retrieval.

[0011] PCL library: C++ library for processing point clouds, including a series of functions such as point cloud loading, preprocessing, feature extraction, saving and visualization.

[0012] KDTree: K-dimensional tree, a method for retrieving the K nearest points to a point in an unordered point cloud.

[0013] VFH feature: Viewpoint Feature Histogram, a global feature used to describe the overall characteristics of a three-dimensional point cloud. It first calculates the FPFH feature of each point for all global points as a component describing the shape of the point cloud; then uses the average coordinates of all points to fit a center point, and uses the direction of the line connecting the statistical point and the fitting center point and the angle information of the viewing angle as a model describing the model pose, and counts them in a histogram as a global feature of the entire point cloud. The VFH feature in the PCL library is described as a 308-dimensional vector, of which the first 135 (45×3) dimensions are features that describe the angle information of the point cloud, which characterizes the shape of the point cloud; the 136th to 180th dimensions are features that describe the distance difference of the point cloud, which characterizes the size of the point cloud and defaults to 0 in the normalized point cloud; the 181st to 308th dimensions are used to describe the six-degree-of-freedom pose of the point cloud, which is not used in this invention. Category label: used to indicate the category of the model, such as car, bed, etc. Each model must have one and only one label.

[0014] Euclidean distance: It is used to describe the distance between two vectors. Its value is equal to the square root of the sum of the squares of the differences in the corresponding dimensions of the two vectors. Summary of the invention

[0015] In view of the problems existing in the prior art, the present invention proposes a lightweight three-dimensional model retrieval method based on the global features of the three-dimensional model, which mainly includes the steps of preprocessing and normalization of model data, extraction of model features, extraction of features of the model to be retrieved and distance measurement based on Euclidean distance.

[0016] The main purpose of the present invention is to provide a lightweight three-dimensional model retrieval method for small data sets, which uses an improved VFH feature to describe the characteristics of the model. The improved feature takes into account the different degrees of influence of local complexity on global features, so that the weight of complex local features on overall features is higher, thereby making global features more deeply affected by complex local features.

[0017] Technical solution of the present invention:

[0018] A lightweight 3D model retrieval method based on point cloud improved VFH features includes the following steps:

[0019] Step 1: load the existing 3D model library, convert 3D model files of different formats into point cloud files, and normalize the point cloud files;

[0020] Step 2, extracting the improved VFH features of the point cloud file obtained in step 1, intercepting the first 135 dimensions as model features, and storing the obtained features in the file;

[0021] The improved VFH feature calculation process is as follows:

[0022] First, for any point p in the point cloud i , calculate its SPFH feature;

[0023] Then, the FPFH feature of each point is calculated based on the SPFH feature weight;

[0024] Then, the local area near each point in the point cloud (radius is the same as the search radius r n ) Point cloud fits plane and sphere, and counts the number of points N that belong to the fitting plane respectively p and the number of points N that belong to the fitting sphere s And the total number of local internal points N, and calculate the complexity weight of the point:

[0025] Finally, the FPFH features of each point are weighted and summed according to the complexity weight to obtain the improved VFH features:

[0026]

[0027] Where n is the number of points in the entire point cloud.

[0028] The improved VFH feature takes into account the different influences of local complexity on global features, making the weight of complex local features higher for overall features, so that global features are more deeply affected by complex local features.

[0029] Step 3, according to the labels of the 3D models in the existing 3D model library, calculate the class center features of each category and store the results in a file;

[0030] Step 4: The three-dimensional model to be retrieved is used to extract improved VFH features according to the methods described in steps 1 and 2, and the first 135 dimensions are intercepted as model features;

[0031] Step 5: Calculate the Euclidean distance between the model features of the 3D model to be retrieved and the class center features of each category in the 3D model library, and classify the 3D model to be retrieved into the category with the smallest Euclidean distance;

[0032] Step 6: Calculate the Euclidean distance between the features of the 3D model to be retrieved and the features of all models in the class classified in step 5, and select several models with the smallest Euclidean distance as the retrieval result output.

[0033] Furthermore, in step 2, the steps for obtaining model features are:

[0034] Step 2.1, load the preprocessed normalized pcd point cloud file;

[0035] Step 2.2, calculate the normal vector of the point based on the position information of other points within a certain radius from the point, and repeat this operation to obtain the normal vector information of each point;

[0036] Step 2.3, use the RANSAC algorithm to perform plane and spherical fitting on the local point cloud of each point, and record the plane fitting degree and spherical fitting degree of each point;

[0037] Step 2.4, based on the position information and normal vectors of other points within a certain radius from the point and the fitting degree in step 2.3, count the position relationship characteristics of the point and the surrounding points, repeat this operation, and accumulate the statistical results into a histogram, which is the feature of the model;

[0038] Step 2.5, discard the feature intervals representing the size and posture of the model, and retain the feature intervals representing the surface shape of the model, that is, retain the first 135-dimensional features and discard the remaining features to obtain the model features of the model;

[0039] Step 2.6, save the category label and model features of the model at the same time.

[0040] Furthermore, the specific steps of calculating the class center features of each category in step 3 include:

[0041] Step 3.1, create a 135-dimensional array for counting the sum of eigenvalues ​​of each dimension;

[0042] Step 3.2, load a model feature and accumulate its 135-dimensional features into the array described in step 3.1;

[0043] Step 3.3, for models belonging to the same category label, execute step 3.2 in sequence to obtain the feature sum of the category, and divide it by the number of models with the category label to obtain the class center feature of the model of this category and save it.

[0044] Step 3.4: For each different category label, execute steps 3.2 and 3.3 respectively to obtain the class center model of each category and save the corresponding label.

[0045] Furthermore, the specific steps of classifying according to the Euclidean distance between the three-dimensional model feature and the class center feature in step 5 include:

[0046] Step 5.1, load the model features and all class center features respectively;

[0047] Step 5.2: For each dimension, calculate the square of the difference between the model feature and the feature value of the class center dimension, and add them up for all dimensions. The square root of the total value is the Euclidean distance between the model feature and the class center feature.

[0048] Step 5.3, for each class center, calculate the Euclidean distance according to the method described in step 5.2, and take the class with the smallest Euclidean distance as the class of the model.

[0049] Furthermore, the specific steps of classifying according to the Euclidean distance between the three-dimensional model features and the intra-class model features in step 6 include:

[0050] Step 6.1, load the model features and the features of all models corresponding to their categories respectively;

[0051] Step 6.2: For each dimension, calculate the square of the difference between the model feature and the feature value of the corresponding dimension of the model within the class, and add them up for all dimensions. Finally, take the square root of the total value, which is the Euclidean distance between the model feature and the model feature within the class.

[0052] Step 6.3, for each model of this type, calculate the Euclidean distance according to the method described in step 5.2, and take several models with the smallest Euclidean distance as the final retrieval results.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention designs a lightweight algorithm for 3D model retrieval based on the global features of the 3D model. It uses the first 135 dimensions of the improved VFH feature to represent the model, ignores the size and posture information of the model, and makes the feature focus on the complex parts of the model, so that the complex local parts that can better represent the entire model have a greater weight in the entire feature, and can ensure the recognition accuracy of the model when the inter-class features are obvious. In addition, the feature-based 3D model retrieval method does not require training, and uses the method of classification before retrieval, thereby reducing the time overhead of the retrieval process and improving the retrieval efficiency, so that good retrieval speed and efficiency can be guaranteed even when there are many samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the overall search process of the present invention;

[0056] Figure 2 Schematic diagram of the difference in normal vector angles according to an embodiment of the present invention;

[0057] Figure 3 This is a flow chart of VFH feature acquisition according to an embodiment of the present invention;

[0058] Figure 4 This is a flow chart of a method for calculating a class center feature according to an embodiment of the present invention;

[0059] Figure 5 This is a flow chart of a method for classifying models to be detected according to an embodiment of the present invention;

[0060] Figure 6 This is a flowchart of the retrieval result sequence output according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The technical solution provided by the present invention will be further described below in conjunction with the accompanying drawings.

[0062] A lightweight 3D model retrieval method based on point cloud improved VFH features, such as Figure 1 As shown, the following steps are included:

[0063] Step 1, load the existing 3D model library, convert 3D model files of different formats, such as off or obj, into pcd files, and normalize the pcd point cloud files;

[0064] Among them, the normalization method for the pcd file is:

[0065] Determine the maximum and minimum values ​​of the three-dimensional coordinates (x, y, z) of all points p in the point cloud, set them as max and min respectively, and then for any point p i (x i ,y i ,z i ), its coordinates become

[0066]

[0067] Among them, (x i ,y i ,z i ) is a point p in the point cloud i The coordinate value of is in the range of (-∞, +∞);

[0068] (x′ i ,y′ i ,z′ i ) is the corresponding point p in the normalized point cloud iThe coordinate value ranges from [0, 1].

[0069] max and min are the maximum and minimum coordinate values in the point cloud respectively.

[0070] Step 2: Based on the PCL library, extract the improved VFH features of the pcd point cloud file obtained in Step 1, intercept the first 135 dimensions as the model features, and store the obtained features in a txt file.

[0071] The model feature is a 135-dimensional vector.

[0072] Among them, the specific calculation steps of the improved VFH feature are as follows:

[0073] Such as Figure 2 , first, for each point p i , estimate its normal vector and calculate the difference in the normal vector triple angle with all neighboring points p n within the nearby radius r j (0 < j ≤ k, and j ≠ i, where k is the number of neighboring points within the search radius r i near p n ) and count the k groups of angle differences into a feature histogram, which is called the SPFH feature of point p . Among them, the angle calculation formula is: i u = n

[0074] u = n i

[0075]

[0076] w = u × v

[0077] α = v · n j

[0078]

[0079] θ = arctan(w · n j , u · n j )

[0080] Among them, u, v, and w are the coordinate axis directions of the coordinate system determined by the normal vector n i of point p j and the connection line between p i and p j ; n i and n j are the normal vectors of p i and p j respectively; α, θ are the parameters to be calculated representing the difference in normal vector angles.

[0081] Then, the FPFH feature of each point in the point cloud is calculated by weighting the SPFH feature:

[0082]

[0083] Where i≠j, FPFH(p i ) is any point p i The feature histogram of k is the radius r around the point n The number of inner neighbors, ω j For point p j The weight is generally taken as p i distance.

[0084] Then, the local area near each point in the point cloud (radius is the same as the search radius r n ) Point cloud fits plane and sphere, and counts the number of points N that belong to the fitting plane respectively p and the number of points N that belong to the fitting sphere s And the total number of local internal points N, and calculate the complexity weight of the point:

[0085]

[0086] Finally, the FPFH features of each point are weighted and summed according to the complexity weight to obtain the improved VFH features:

[0087]

[0088] Where n is the number of points in the entire point cloud.

[0089] Specifically, the process of obtaining the improved VFH feature is as follows: Figure 3 As shown, the steps to obtain model features are:

[0090] Step 2.1, load the point cloud into cloud (pcl::PointCloud <pcl::pointxyz>::Ptr);

[0091] Step 2.2, calculate the normal vector of the point based on the position information of other points within a certain radius from the point, and repeat this operation to obtain the normal vector information of each point;

[0092] Specifically, use the normal vector estimator in the PCL library (pcl::NormalEstimation <pcl::

[0093] PointXYZ,pcl::Normal>), and set its estimation object to cloud, and the nearby point cloud retrieval method to KDTree (pcl::search::KdTree <pcl::pointxyz>::Ptr), set the search radius r n (r n >0, for normalized point clouds, generally 0 <r n <0.05, estimate the normal vector, and store the extracted normal vector in cl oudnormals(pcl::PointCloud <pcl::normal>::Ptr);

[0094] Step 2.3, use the RANSAC algorithm to perform plane and spherical fitting on the local point cloud of each point, and record the plane fitting degree and spherical fitting degree of each point;

[0095] Specifically, the RANSAC algorithm is used to search for local regions near each point in the point cloud (with the same radius as the search radius r n ) Point cloud fits plane and sphere, and counts the number of points N that belong to the fitting plane respectively p and the number of points N that belong to the fitting sphere s And the total number of local internal points N, and calculate the complexity weight w of the point c :

[0096] Step 2.4, based on the position information and normal vectors of other points within a certain radius from the point and the fitting degree in step 2.3, count the position relationship characteristics of the point and the surrounding points, repeat this operation, and accumulate the statistical results into a histogram, which is the feature of the model;

[0097] Specifically, the VFH feature extractor (VFHEstimation<pcl::PointXYZ,pcl::Normal,pcl::VFHSignature308> ), set the extraction object to cloud, the corresponding normal vector to cloudnormals, and set the search radius r vfh (r vfh >r n , usually r vfh =2r n ), the nearby point cloud retrieval method is KDTree, and when extracting the local features of each point, the result is multiplied by w c Finally, the result is stored in vfhs(pcl::PointCloud <pcl::vfhsignature308>::Ptr);

[0098] Step 2.5, discard the feature intervals representing the size and posture of the model, and retain the feature intervals representing the surface shape of the model, that is, retain the first 135-dimensional features and discard the remaining features to obtain the model features of the model;

[0099] Specifically, the first 135-dimensional data of vfhs are stored in txt files row by row, and the category labels of the model are also stored.

[0100] Step 2.6, save the category label and model features of the model at the same time.

[0101] Step 3: Calculate the class center features of each category according to the labels of the 3D models in the existing 3D model library, and store the results in a file. Specifically, Figure 4 As shown, the method for calculating the class center feature of the category is:

[0102] Step 3.1, create a 135-dimensional array SumFeature

[135] (int) for counting the sum of feature values ​​of each dimension, and initialize it to 0; create a variable Num(int) for counting the number of models in this category, and initialize it to 0;

[0103] Step 3.2, read the class labels (a total of classNum) of the models obtained in step 2 and the feature vectors of the models within the class by class, and execute the subsequent steps for each class and model within the class respectively;

[0104] Step 3.3, for each in-class model Object of a single class, execute in sequence: load a feature vector Object of the in-class model, store it in the 135-dimensional array Feature

[135] (int), for each array subscript i (0<=i<=134), add the array elements in Feature to the array elements of the corresponding subscript in the SumFeature array, and then increment Num;

[0105] Step 3.4, for the features and array SumFeature of a single class and the number of models Num, for each array element, divide it by the number of models Num that have the label of this category, and get the class center feature MeanFeature of this class model, and finally save the class center feature and the corresponding label to a txt file;

[0106] Step 4: Extract the improved VFH features from the 3D model to be retrieved according to the methods described in steps 1 and 2, and extract the first 135 dimensions as the model features.

[0107] Step 5: Calculate the Euclidean distance between the features of the 3D model to be retrieved and the class center features of each category, and classify the 3D model to be retrieved into the category with the smallest Euclidean distance. Specifically, Figure 5 As shown, the method for calculating the Euclidean distance and completing the classification of the model to be detected is:

[0108] Step 5.1, load the model features and all class center features into the feature array InputVFHS

[135] (int) and MeanFeature

[135] (int), and create the Euclidean distance array dis

[134] (int) and the Euclidean distance variable Dis(int)

[0109] Step 5.2, for each array index i of the class center feature, calculate dis[i] = (InputVFHS[i] - MeanFeature[i]) 2 And add them to Dis, and finally square Dis to get the Euclidean distance between the model feature to be retrieved and the class center feature;

[0110] Step 5.3, for each class center, calculate the Euclidean distance according to the method described in step 5.2, and take the class with the smallest Euclidean distance as the class of the model.

[0111] Step 6: Calculate the Euclidean distance between the features of the 3D model to be retrieved and the features of all models in the class classified in step 5, and select several models with the smallest Euclidean distance as the retrieval result output.

[0112] Specifically, Figure 6 As shown in the figure, the method for calculating the Euclidean distance between the model feature and the intra-class model feature and outputting the retrieval result sequence is:

[0113] Step 6.1, load the model features and the features of an intra-class model corresponding to its category into InputVFHS

[135] (int) and Feature

[135] (int), respectively, and establish a mapping D{num,dis}(map) between the order of InputVFHS and the Euclidean distance between the intra-class model Feature;

[0114] Step 6.2, calculate the Euclidean distance between the model feature and each intra-class model according to the method described in step 5, and add the mapping between the Euclidean distance and the model order into D;

[0115] Step 6.3, after completing the Euclidean distance calculation, sort the mapping pairs in D from small to large according to Dis;

[0116] Step 6.4, output several Num with the smallest Dis in turn, and the in-class model model at the corresponding position is the retrieval result, and output each retrieval result.

[0117] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application. < / pcl::normal> < / pcl::pointxyz> < / pcl::pointxyz>

Claims

1. A lightweight 3D model retrieval method based on point cloud improved VFH features, characterized in that: The following steps are involved: Step 1: load the existing 3D model library, convert 3D model files of different formats into point cloud files, and normalize the point cloud files; Step 2, extracting the improved VFH features of the point cloud file obtained in step 1, intercepting the first 135 dimensions as model features, and storing the obtained features in the file; The improved VFH feature calculation process is as follows: First, for any point p in the point cloud i , calculate its SPFH feature; Then, the FPFH feature of each point is calculated based on the SPFH feature weight; Then, for each point in the point cloud, the radius near the search radius r n The local point cloud is fitted to the plane and sphere, and the number of points N in the fitting plane is counted respectively p and the number of points N that belong to the fitting sphere s And the total number of local internal points N, and calculate the complexity weight of the point: Finally, the FPFH features of each point are weighted and summed according to the complexity weight to obtain the improved VFH features: Where n is the number of points in the entire point cloud; Step 3, according to the labels of the 3D models in the existing 3D model library, calculate the class center features of each category and store the results in a file; Step 4: The three-dimensional model to be retrieved is used to extract improved VFH features according to the methods described in steps 1 and 2, and the first 135 dimensions are intercepted as model features; Step 5: Calculate the Euclidean distance between the model features of the 3D model to be retrieved and the class center features of each category in the 3D model library, and classify the 3D model to be retrieved into the category with the smallest Euclidean distance; Step 6: Calculate the Euclidean distance between the features of the 3D model to be retrieved and the features of all models in the class classified in step 5, and select several models with the smallest Euclidean distance as the retrieval result output.

2. The lightweight 3D model retrieval method based on point cloud improved VFH feature according to claim 1, characterized in that: In step 2, the steps to obtain model features are: Step 2.1, load the preprocessed normalized point cloud file; Step 2.2, calculate the normal vector of the point based on the position information of other points within a certain radius from the point, and repeat this operation to obtain the normal vector information of each point; Step 2.3, use the RANSAC algorithm to perform plane and spherical fitting on the local point cloud of each point, and record the plane fitting degree and spherical fitting degree of each point; Step 2.4, based on the position information and normal vectors of other points within a certain radius from the point and the fitting degree in step 2.3, count the position relationship characteristics of the point and the surrounding points, repeat this operation, and accumulate the statistical results into a histogram, which is the feature of the model; Step 2.5, discard the feature intervals representing the size and posture of the model, and retain the feature intervals representing the surface shape of the model, that is, retain the first 135-dimensional features and discard the remaining features to obtain the model features of the model; Step 2.6, save the category label and model features of the model at the same time.

3. A lightweight 3D model retrieval method based on global features of 3D point clouds as claimed in claim 1, characterized in that: The specific steps for calculating the class center features of each category in step 3 include: Step 3.1, create a 135-dimensional array for counting the sum of eigenvalues ​​of each dimension; Step 3.2, load a model feature and accumulate its 135-dimensional features into the array described in step 3.1; Step 3.3: For models with the same category label, execute step 3.2 in sequence to obtain the feature sum of the category, and divide it by the number of models with the category label to obtain the class center feature of the model and save it; Step 3.4: For each different category label, execute steps 3.2 and 3.3 respectively to obtain the class center model of each category and save the corresponding label.

4. A lightweight 3D model retrieval method based on global features of 3D point clouds as claimed in claim 1, characterized in that: The specific steps of classifying according to the Euclidean distance between the three-dimensional model feature and the class center feature in step 5 include: Step 5.1, load the model features and all class center features respectively; Step 5.2: For each dimension, calculate the square of the difference between the model feature and the feature value of the class center dimension, and add them up for all dimensions. The square root of the total value is the Euclidean distance between the model feature and the class center feature. Step 5.3, for each class center, calculate the Euclidean distance according to the method described in step 5.2, and take the class with the smallest Euclidean distance as the class of the model.

5. A lightweight 3D model retrieval method based on global features of 3D point clouds as claimed in claim 1, characterized in that: The specific steps of classifying according to the Euclidean distance between the three-dimensional model features and the intra-class model features in step 6 include: Step 6.1, load the model features and the features of all models corresponding to their categories respectively; Step 6.2: For each dimension, calculate the square of the difference between the model feature and the feature value of the corresponding dimension of the model within the class, and add them up for all dimensions. Finally, take the square root of the total value to get the Euclidean distance between the model feature and the model feature within the class. Step 6.3, for each model of this type, calculate the Euclidean distance according to the method described in step 5.2, and take several models with the smallest Euclidean distance as the final retrieval results.

Images