Lightweight 3D model retrieval method based on point cloud improved VFH feature

Abstract

The application belongs to the field of mechanical design assembly automation and graphics, and proposes a light three-dimensional model retrieval method based on improved VFH features of point cloud, comprising the following steps: step 1, processing and converting the existing three-dimensional model data, and normalizing; step 2, extracting the improved VFH features of the pcd file based on the PCL library, and storing the first 135 dimensions as the point cloud features; step 3, calculating the feature center of the three-dimensional model point cloud as the feature of the category; step 4, extracting the improved VFH features of the three-dimensional model to be retrieved; step 5, calculating the Euclidean distance between the category center features to classify the model; step 6, calculating the Euclidean distance between the features of all models in the category and sorting to complete the retrieval task. The application solves the problem of low retrieval efficiency of the deep learning method, and has good retrieval effect for three-dimensional models with large category differences, so that the retrieval time can be reduced while ensuring the retrieval effect.

Description

Technical Field

[0001] The present invention belongs to the field of automation and graphics of mechanical design and assembly, and specifically relates to a lightweight three-dimensional model retrieval method based on improved VFH features of three-dimensional point clouds. Background Art

[0002] Three-dimensional models are favored by the industrial design and manufacturing fields due to their intuitive and relatively realistic characteristics. At present, the vigorous development of CAD technology has made the acquisition and processing of three-dimensional models easier, which has also led to an explosive growth in the number of three-dimensional models.

[0003] Directly designing a 3D model is often complex, but improving existing 3D models can significantly reduce the workload. However, text-based model retrieval methods present two challenges: each model has more than one shape feature, making it difficult to describe it with a single text; and different naming conventions can lead to different features corresponding to the same text. Consequently, the need for algorithms to retrieve 3D models with the required shape from vast 3D model databases is becoming increasingly urgent, leading to the rapid development of content-based 3D model retrieval methods.

[0004] Existing content-based 3D model retrieval algorithms are divided into learning-based methods and feature descriptor-based algorithms, among which:

[0005] 1) Learning-based methods can extract features that are difficult for humans to interpret through deep learning, and their effectiveness is generally better than manually extracted features. However, learning-based methods require a significant amount of time for training, and each large-scale data update requires additional training, resulting in a significant time overhead. Learning-based methods also require large amounts of data, but enterprises may not have sufficient data for learning, so lighter algorithms are needed.

[0006] 2) Traditional feature descriptor-based methods design a feature descriptor that describes the entire model and use it to retrieve the model. Although some methods consider extracting local features of the model and combining them into global features in a certain way, they do not consider that different local features have different representational effects on the entire model. Complex local features have a stronger representation effect on the entire model. However, existing methods do not emphasize the weight of complex local features on the overall model.

[0007] Some explanations of terms involved in the present invention:

[0008] Point cloud: A form used to describe a three-dimensional model. The surface of the three-dimensional model is sampled into several points, and the three-dimensional coordinates of each point are recorded as the point cloud of the three-dimensional model.

[0009] pcd: point cloud data, a file format to store 3D point cloud.

[0010] normalization: scale the coordinates of point cloud between 0 and 1, so that the model size has no effect on shape retrieval.

[0011] PCL library: a C++ library to handle point cloud, including a series of functions such as loading, preprocessing, extracting features, saving and visualizing point cloud.

[0012] KDTree: K-dimensional tree, a way to retrieve the nearest K points to a certain point in an unordered point cloud.

[0013] VFH feature: Viewpoint Feature Histogram, a global feature to describe the overall characteristics of a 3D point cloud. It first calculates the FPFH feature of each point with respect to all points in the global as a component to describe the shape of the point cloud; then a center point is fitted using the average coordinates of all points, and the angle information between the direction of the connecting line of the point and the viewing angle is counted as a model to describe the pose of the model, and they are counted into a histogram as the global feature of the entire point cloud. The VFH feature in the PCL library is described as a 308-dimensional vector, of which the first 135 (45x3) dimensions are features describing the angle information of the point cloud, which represents the shape of the point cloud; the 136th to 180th dimensions are features describing the distance difference of the point cloud, which represents the size of the point cloud, which is 0 by default in the normalized point cloud; the 181st to 308th dimensions are used to describe the six degrees of freedom pose of the point cloud, which is not used in the present invention. Class label: used to represent the category of the model, such as car, bed, etc. Each model must have a label and can only have one label.

[0014] Euclidean distance: a criterion for describing the distance between two vectors, the value is equal to the square root of the sum of the square of the difference between the corresponding dimensions of the two vectors. SUMMARY

[0015] In view of the problems existing in the prior art, the present application provides a lightweight three-dimensional model retrieval method based on global features of three-dimensional models, mainly including the steps of preprocessing and normalization of model data, extraction of model features, extraction of features of the model to be retrieved, and distance measurement based on Euclidean distance.

[0016] The main purpose of the present application is to provide a lightweight three-dimensional model retrieval method for small data sets, which uses improved VFH features to describe the features of the model. The improved features take into account the different effects of local complexity on global features, so that the weight of complex local features on the overall features is higher, so that the global features are more deeply affected by complex local features.

[0017] The technical scheme of the present application:

[0018] A light three-dimensional model retrieval method based on improved VFH features of point cloud, comprising the following steps:

[0019] Step 1: Load the existing three-dimensional model library, convert three-dimensional model files of different formats into point cloud files, and normalize the point cloud files;

[0020] Step 2: Extract the improved VFH features of the point cloud files obtained in step 1, and take the first 135 dimensions as model features, and store the obtained features in a file;

[0021] The improved VFH feature calculation process is as follows:

[0022] First, for any point p in the point cloud i , calculate its SPFH feature;

[0023] Then, according to the SPFH feature, the FPFH feature of each point is calculated;

[0024] Then, for each point in the point cloud, the local (radius same as the retrieval radius r n ) point cloud around the point is fitted with a plane and a sphere, respectively, and the number of points belonging to the fitted plane N p and the number of points belonging to the fitted sphere N s and the total number of points in the local N are counted, and the complexity weight of the point is calculated:

[0025] Finally, the FPFH features of each point are weighted and summed according to the complexity weight to obtain the improved VFH feature:

[0026]

[0027] Where n is the number of points in the entire point cloud.

[0028] The improved VFH feature takes into account the different effects of local complexity on global features, so that complex localities have higher weights on overall features, so that global features are more deeply affected by complex localities.

[0029] Step 3: According to the labels of the three-dimensional models in the existing three-dimensional model library, calculate the class center features of each class, and store the results in a file;

[0030] Step 4: The three-dimensional model to be retrieved extracts the improved VFH feature according to the method described in steps 1 and 2, and takes the first 135 dimensions as the model feature;

[0031] Step 5: Calculate the Euclidean distance between the model features of the three-dimensional model to be searched and the class center features of each category in the three-dimensional model library, and classify the three-dimensional model to be searched into the category with the smallest Euclidean distance;

[0032] Step 6: Calculate the Euclidean distance between the features of the three-dimensional model to be searched and all model features in the category classified in step 5, and take several models with the smallest Euclidean distance as the search result output.

[0033] Further, in step 2, the steps of obtaining model features are:

[0034] Step 2.1, load the pre-processed normalized pcd point cloud file;

[0035] Step 2.2, calculate the normal vector of the point based on the position information of other points within a certain radius from the point, and repeat the operation to obtain the normal vector information of each point;

[0036] Step 2.3, use the ransac algorithm to respectively fit a plane and a sphere to the local point cloud of each point, and record the plane fitting degree and the sphere fitting degree of each point;

[0037] Step 2.4, based on the position information and normal vector of other points within a certain radius from the point and the fitting degree fitted in step 2.3, the position relationship features of the point and the surrounding points are counted, and the counting results are accumulated in a histogram, which is the feature of the model;

[0038] Step 2.5, discard the feature interval representing the size and pose of the model, and retain the feature interval representing the surface shape of the model, that is, retain the first 135 features and discard the remaining features, to obtain the model features of the model;

[0039] Step 2.6, save the category label and model features of the model at the same time.

[0040] Further, the specific steps of calculating the class center features of each category in step 3 include:

[0041] Step 3.1, establish a 135-dimensional array for counting the sum of feature values of each dimension;

[0042] Step 3.2, load a model feature, and accumulate the 135-dimensional features thereof in the array described in step 3.1;

[0043] Step 3.3, for models belonging to the same category label, sequentially execute step 3.2 to obtain the feature sum of the category, and divide the feature sum by the number of models having the category label to obtain the class center features of the category and save them.

[0044] Step 3.4, for each different category label, respectively execute step 3.2 and step 3.3 to obtain the class center model of each category and save the corresponding label.

[0045] Further, the specific steps of classification in step 5 according to the Euclidean distance between the three-dimensional model feature and the class center feature include:

[0046] Step 5.1, respectively load the model feature and all class center features;

[0047] Step 5.2, for each dimension, respectively calculate the square of the difference between the feature value of the model feature and the class center dimension, and accumulate all dimensions, and take the square root of the total value as the Euclidean distance between the model feature and the class center feature.

[0048] Step 5.3, for each class center, respectively calculate the Euclidean distance according to the method described in step 5.2, and take the class with the smallest Euclidean distance as the class of the model.

[0049] Further, the specific steps of classification in step 6 according to the Euclidean distance between the three-dimensional model feature and the intra-class model feature include:

[0050] Step 6.1, respectively load the model feature and the features of all models corresponding to its category;

[0051] Step 6.2, for each dimension, respectively calculate the square of the difference between the feature value of the model feature and the corresponding dimension of the intra-class model, and accumulate all dimensions, and finally take the square root of the total value as the Euclidean distance between the model feature and the intra-class model feature.

[0052] Step 6.3, for each model in the category, respectively calculate the Euclidean distance according to the method described in step 5.2, and take the several models with the smallest Euclidean distance as the final retrieval result.

[0053] Compared with the prior art, the present application has the following beneficial effects:

[0054] The present application designs a lightweight algorithm for three-dimensional model retrieval based on the global feature of the three-dimensional model. It uses the first 135 dimensions of the improved VFH feature to represent the model, ignores the size and pose information of the model, and makes the feature focus on the complex part of the model, so that the complex local part of the whole model can be better represented in the whole feature, and the recognition accuracy of the model can be guaranteed in the case that the feature between categories is obvious. Moreover, the three-dimensional model retrieval method based on features does not need to be trained, and uses the method of classification first and then retrieval, thereby reducing the time overhead of the retrieval process and improving the retrieval efficiency, so that good retrieval speed and efficiency can be guaranteed in the case of more samples. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The whole retrieval process flowchart of the present application;

[0056] Figure 2 The normal vector angle difference schematic diagram of the embodiment of the present application;

[0057] Figure 3 The VFH feature acquisition flowchart of the embodiment of the present application;

[0058] Figure 4 The class center feature calculation method flowchart of the embodiment of the present application;

[0059] Figure 5 The model classification method flowchart of the embodiment of the present application;

[0060] Figure 6 The retrieval result sequence output flowchart of the embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions provided by the present application will be further described below with reference to the accompanying drawings.

[0062] A light-weight three-dimensional model retrieval method based on improved VFH features of point cloud, as shown in the figure, comprising the following steps: Figure 1

[0063] Step 1, load the existing three-dimensional model library, convert different formats of three-dimensional model files such as off or obj into pcd files, and normalize the pcd point cloud file;

[0064] The normalization method for the pcd file is:

[0065] Determine the maximum and minimum values of the three-dimensional coordinates (x, y, z) of all points p in the point cloud, and set them as max and min respectively, and then for any point p i (x i ,y i ,z i ), its coordinates become

[0066]

[0067] Where (x i ,y i ,z i ) is the coordinate value of a point p i in the point cloud, and the value range is (-∞, +∞);

[0068] (x′ i ,y′ i ,z′ i ) is the corresponding point p i ​The coordinate value of is in the range of [0, 1];

[0069] max and min are the maximum and minimum coordinate values ​​in the point cloud respectively.

[0070] Step 2: Based on the PCL library, extract the improved VFH features of the pcd point cloud file obtained in step 1, intercept its first 135 dimensions as model features, and store the obtained features in a txt file.

[0071] The model feature is a 135-dimensional vector.

[0072] The specific calculation steps of the improved VFH feature are as follows:

[0073] like Figure 2 , first, for each point p i , estimate its normal vector and calculate its radius r n All adjacent points p in j (0 <j≤k,且j≠i,其中k为p i Nearby search radius r n The difference in the normal vector three-element angle between the adjacent points (number of adjacent points) The k groups of angle differences are counted into a feature histogram, which is called point p i The SPFH characteristics of , where the angle calculation formula is:

[0074] u=n i

[0075]

[0076] w=u×v

[0077] α=v·n j

[0078]

[0079] θ=arctan(w·n j , u·n j )

[0080] Among them, u, v, w are based on point p i Normal vector n j and p i With p j The direction of the coordinate axis of the coordinate system determined by the line between them; n i and n j They are p i and p j Normal vector of ; α, θ is the parameter to be determined that represents the difference in the angle between the normal vectors.

[0081] Then, the FPFH feature of each point in the point cloud is calculated by SPFH feature weighting:

[0082]

[0083] where i≠j, FPFH(p i ) is the feature histogram of any point p i , k is the number of neighboring points within the radius r n around the point, ω j is the weight of the point p j , and is generally taken as the distance of p i .

[0084] Further, a plane and a sphere are fitted to the local (radius same as the search radius r n ) point cloud around each point in the point cloud, and the number of points belonging to the fitted plane N p and the number of points belonging to the fitted sphere N s as well as the total number of points in the local N are counted, and the complexity weight of the point is calculated:

[0085]

[0086] Finally, the FPFH feature of each point is weighted and summed according to the complexity weight to obtain the improved VFH feature:

[0087]

[0088] where n is the number of points in the entire point cloud.

[0089] Specifically, the process of obtaining the improved VFH feature is as shown in Figure 3 , and the steps of obtaining the model feature are:

[0090] Step 2.1, load the point cloud to cloud (pcl::PointCloud <pcl::pointxyz>::Ptr);

[0091] Step 2.2: Calculate the normal vector of the point based on the position information of other points within a certain radius from the point, and repeat this operation to obtain the normal vector information of each point;

[0092] Specifically, use the normal vector estimator in the PCL library (pcl::NormalEstimation <pcl::

[0093] PointXYZ,pcl::Normal>), and set its estimation object to cloud, and the nearby point cloud retrieval method to KDTree (pcl::search::KdTree <pcl::pointxyz>::Ptr), set the search radius r n (r n > 0, for the point cloud that has been normalized, generally take 0 < r n <0.05), estimate the normal vector, store the extracted normal vector in cloudnormals (pcl::PointCloud <pcl::normal>::Ptr) in the point cloud;

[0094] Step 2.3, using ransac algorithm to fit plane and sphere to each point's local point cloud, record each point's plane fitting degree and sphere fitting degree;

[0095] Specifically, using RANSAC algorithm to fit plane and sphere to each point's local point cloud (radius same as search radius r n ) in the point cloud, respectively, and counting the number of points belonging to the fitted plane N p and the number of points belonging to the fitted sphere N s and the total number of points in the local N c , and calculating the complexity weight w vfh of the point:

[0096] Step 2.4, based on the position information and normal vector of other points within a certain radius of the point and the fitting degree in step 2.3, count the position relationship features of the point and the surrounding points, repeat this operation, and accumulate the statistical results into a histogram, which is the feature of the model;

[0097] Specifically, using the VFH feature extractor (VFHEstimation<pcl::PointXYZ,pcl::Normal,pcl::VFHSignature308>) provided by PCL library, setting the extraction object as cloud, the corresponding normal vector as cloudnormals, setting the search radius r vfh (r vfh >r n , generally r vfh =2r n ), the nearest point cloud search method is KDTree, and when extracting the local feature of each point, multiply the result by w c , and finally store the result in vfhs (pcl::PointCloud <pcl::vfhsignature308>::Ptr) in the file;

[0098] Step 2.5, discard the feature intervals representing the model size and pose, and keep the feature intervals representing the model surface shape, i.e. keep the first 135 features and discard the rest, to obtain the model feature of the model;

[0099] Specifically, the first 135 features of vfhs are stored in a txt file by row, and the class label of the model is also stored.

[0100] Step 2.6, save the class label and the model feature of the model at the same time.

[0101] Step 3, according to the labels of the three-dimensional models in the existing three-dimensional model library, calculate the class center feature of each class, and store the result in a file. Specifically, as shown in the following formula, the method for calculating the class center feature of a class is: Figure 4

[0102] Step 3.1, establish a 135-dimensional array SumFeature

[135] (int) for counting the sum of feature values of each dimension, and initialize it to 0, and establish a variable Num(int) for counting the number of models in the class, and initialize it to 0;

[0103] Step 3.2, read the class label (a total of classNum) and the feature vector of the model within the class obtained in step 2 according to the class, and execute the following steps for each class and model within the class;

[0104] Step 3.3, for each model within a class Object, execute the following steps in turn: load the feature vector of a model within the class Object, which is stored in the 135-dimensional array Feature

[135] (int), for each array index i (0<=i<=134), respectively, add the array element in Feature to the array element in the corresponding index of the SumFeature array, and then Num is incremented;

[0105] Step 3.4, for the feature of a single class, the array SumFeature and the model number Num, for each array element, respectively, divide by the number of models Num that have the class label to obtain the class center feature MeanFeature of the class model, and finally save the class center feature and the corresponding label to a txt file;

[0106] Step 4: extract the improved VFH feature of the three-dimensional model to be searched according to the method described in steps 1 and 2, and take the first 135 dimensions as the model feature.

[0107] ​Step 5: Calculate the Euclidean distance between the feature of the three-dimensional model to be searched and the class center feature of each category, and classify the three-dimensional model to be searched into the category with the smallest Euclidean distance. Specifically, as shown in Figure 5 the method for calculating the Euclidean distance and completing the classification of the model to be searched is as follows:

[0108] Step 5.1, load the model feature and all class center features into the feature arrays InputVFHS

[135] (int) and MeanFeature

[135] (int), and establish the Euclidean distance array dis

[134] (int) and the Euclidean distance variable Dis(int)

[0109] Step 5.2, for each array subscript i of the class center feature, calculate dis[i] = (InputVFHS[i]-MeanFeature[i]) respectively 2 and add to Dis, and finally take the square root of Dis to obtain the Euclidean distance between the model feature to be searched and the class center feature;

[0110] Step 5.3, for each class center, calculate the Euclidean distance according to the method described in step 5.2, and take the category with the smallest Euclidean distance as the category of the model.

[0111] Step 6: Calculate the Euclidean distance between the feature of the three-dimensional model to be searched and all model features in the category classified in step 5, and take the several models with the smallest Euclidean distance as the search result output.

[0112] Specifically, as shown in Figure 6 the method for calculating the Euclidean distance between the model feature and the model feature in the category and outputting the search result sequence is as follows:

[0113] Step 6.1, load the model feature and the feature of one model in the category corresponding to the model feature into InputVFHS

[135] (int) and Feature

[135] (int) respectively, and establish the mapping D{num,dis}(map) between the Euclidean distance between InputVFHS and the model feature in the category;

[0114] Step 6.2, calculate the Euclidean distance between the model feature and each model in the category according to the method described in step 5, and add the mapping between the Euclidean distance and the model sequence to D;

[0115] Step 6.3, after completing the Euclidean distance calculation, sort the mapping pairs in D according to Dis from small to large;

[0116] Step 6.4, output the several Num with the smallest Dis in turn, and the model in the category corresponding to the position of the Num is the search result, and each search result is output.

[0117] The above description is only a description of the preferred embodiments of the present application and does not limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical content should be regarded as equivalent valid embodiments and fall within the scope of protection of the technical solution of the present application. < / pcl::normal> < / pcl::pointxyz> < / pcl::pointxyz>

Claims

1. A lightweight 3D model retrieval method based on point cloud improved VFH features, characterized by: The following steps are involved: Step 1: Load the existing 3D model library, convert 3D model files of different formats into point cloud files, and normalize the point cloud files; Step 2: extract the improved VFH features of the point cloud file obtained in step 1, intercept the first 135 dimensions as the model features, and store the obtained features in the file; The improved VFH feature calculation process is as follows: First, for any point in the point cloud , calculate its SPFH feature; Then, the FPFH feature of each point is calculated based on the SPFH feature weight; Then, the radius around each point in the point cloud is the same as the search radius The local point cloud is fitted on the plane and sphere, and the number of points in the fitting plane is counted respectively. and the number of points within the fitted sphere And the total number of local interior points , and calculate the complexity weight of the point: ; Finally, the FPFH features of each point are weighted and summed according to the complexity weight to obtain the improved VFH features: in, is the number of points in the entire point cloud; Step 3: Calculate the class center feature of each category based on the labels of the 3D models in the existing 3D model library and store the results in a file; Step 4: Extract the improved VFH features from the 3D model to be retrieved according to the methods described in steps 1 and 2, and extract the first 135 dimensions as the model features; Step 5: Calculate the Euclidean distance between the model features of the 3D model to be retrieved and the class center features of each category in the 3D model library, and classify the 3D model to be retrieved into the category with the smallest Euclidean distance; Step 6: Calculate the Euclidean distance between the features of the 3D model to be retrieved and the features of all models in the class classified in step 5, and select the models with the smallest Euclidean distance as the retrieval results.

2. The lightweight 3D model retrieval method based on point cloud improved VFH features according to claim 1, characterized in that: In step 2, the steps to obtain model features are: Step 2.1, load the pre-processed normalized point cloud file; Step 2.2: Calculate the normal vector of the point based on the position information of other points within a certain radius from the point, and repeat this operation to obtain the normal vector information of each point; Step 2.3, use the Ransac algorithm to perform plane and spherical fitting on the local point cloud of each point, and record the plane fitting degree and spherical fitting degree of each point; Step 2.4: Based on the position information and normal vectors of other points within a certain radius from the point and the fitting degree in step 2.3, calculate the positional relationship characteristics of the point and its surrounding points. Repeat this operation and accumulate the statistical results into a histogram, which is the feature of the model. Step 2.5: discard the feature intervals representing the model size and posture, and retain the feature intervals representing the model surface shape, that is, retain the first 135-dimensional features and discard the remaining features to obtain the model features of the model; Step 2.6: Save the category label and model features of the model at the same time.

3. The lightweight 3D model retrieval method based on point cloud improved VFH features according to claim 1 is characterized in that: The specific steps for calculating the class center features of each category in step 3 include: Step 3.1: Create a 135-dimensional array to calculate the sum of the eigenvalues ​​of each dimension. Step 3.2: Load a model feature and accumulate its 135-dimensional features into the array described in step 3.

1. Step 3.3: For models with the same category label, execute step 3.2 in sequence to obtain the feature sum of the category, and divide it by the number of models with the category label to obtain the class center feature of the model and save it; Step 3.4: For each different category label, execute steps 3.2 and 3.3 respectively to obtain the class center model of each category and save the corresponding label.

4. The lightweight 3D model retrieval method based on point cloud improved VFH features according to claim 1, characterized in that: The specific steps of classifying according to the Euclidean distance between the 3D model features and the class center features in step 5 include: Step 5.1, load the model features and all class center features respectively; Step 5.2: For each dimension, calculate the square of the difference between the model feature and the feature value of the class center dimension, and add them up for all dimensions. The square root of the total value is the Euclidean distance between the model feature and the class center feature; Step 5.3: For each cluster center, calculate the Euclidean distance according to the method described in step 5.2, and take the category with the smallest Euclidean distance as the category of the model.

5. The lightweight 3D model retrieval method based on point cloud improved VFH features according to claim 1, characterized in that: The specific steps of classifying according to the Euclidean distance between the 3D model features and all model features within the class in step 6 include: Step 6.1, load the model features and the features of all models corresponding to their categories respectively; Step 6.2: For each dimension, calculate the square of the difference between the model feature and the feature value of the corresponding dimension of the model within the class, and add them up for all dimensions. Finally, take the square root of the total value to obtain the Euclidean distance between the model feature and the feature of the model within the class. Step 6.3: For each model of this type, calculate the Euclidean distance according to the method described in step 6.2, and take the models with the smallest Euclidean distance as the final retrieval results.

Images