Model retrieval method and device based on three-dimensional model features, equipment and medium

Abstract

The application relates to a model retrieval method and device based on three-dimensional model features, equipment and a medium. The method comprises the following steps: obtaining point cloud data of a three-dimensional model displayed by a client and preprocessing the point cloud data, respectively fitting the preprocessed point cloud data through a polynomial function and a twin neural network model to extract features of the three-dimensional model and obtain an original feature vector of the three-dimensional model; clustering and compressing all the original feature vectors, and storing compressed vectors of the three-dimensional model and corresponding model information in a model database; importing point cloud data of a three-dimensional model to be queried into the client and obtaining an original feature vector to be queried, comparing and calculating distances between the original feature vector to be queried and compressed vectors stored in the model database, retrieving similar models in the model database, and displaying and interacting with the similar models in the client. The method can quickly retrieve and efficiently manage three-dimensional models in large-scale three-dimensional scene roaming.

Description

Technical Field

[0001] This application relates to the field of 3D model retrieval technology, and in particular to a model retrieval method, apparatus, device and medium based on 3D model features. Background Technology

[0002] 3D scene roaming utilizes computer technology to generate realistic 3D scenes, providing users with an immersive experience by building models and simulating complex environments and situations. 3D scene roaming often requires the creation of multiple realistic 3D scenes, each requiring a large number of 3D models. These 3D models not only need high realism but also high interactivity and operability so that users can perform real-time operations during scene roaming. This poses a significant challenge to model storage and retrieval. Therefore, 3D model retrieval—that is, how to efficiently retrieve the required models from a 3D model library—has become one of the hot research topics in multimedia and other fields today. Summary of the Invention

[0003] Therefore, it is necessary to provide a model retrieval method, apparatus, device, and medium based on 3D model features that can efficiently and quickly retrieve similar models and centrally manage and maintain 3D models, in order to address the above-mentioned technical problems.

[0004] A model retrieval method based on 3D model features, the method comprising:

[0005] Obtain the point cloud data of the 3D model displayed on the client and preprocess it to obtain the preprocessed point cloud data;

[0006] Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model. The preprocessed point cloud data is then input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model. The distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model. The lengths of the distribution feature vector and the representation feature vector are the same.

[0007] Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering and compression. Obtain the compressed vector of each 3D model and store the compressed vector and the corresponding model information in the model database.

[0008] Import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. Calculate the distance between the original feature vector to be queried and the compressed vector stored in the model database. Search for similar models in the model database and display and interact with the similar models on the client.

[0009] In one embodiment, point cloud data of the 3D model displayed by the client is acquired and preprocessed to obtain preprocessed point cloud data, including:

[0010] Obtain the point cloud data of the 3D model displayed on the client. When the 3D model contains multiple sub-models, divide the 3D model into multiple sub-models and obtain the corresponding point cloud data.

[0011] The global coordinates of all points in the point cloud data are restored by matrix coordinate transformation, and the model is standardized by overall translation and scaling of all points, so that the model is scaled to a unit cube bounding box.

[0012] By changing the global coordinates of all points so that the center of the unit cube bounding box coincides with the origin of the coordinate system in the point cloud data, the preprocessing of the point cloud data is completed and the preprocessed point cloud data is obtained.

[0013] In one embodiment, feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model, including:

[0014] By randomly sampling any two points in the preprocessed point cloud data and calculating the distance between the two points, the probability distribution of point distance shape in the 3D model is obtained.

[0015] The probability distribution of point distance shape is divided into several segments. A polynomial function is used to fit each segment of data, and the parameters of the fitted polynomial function are used as the distribution feature vector of the 3D model. The length of the distribution feature vector is consistent with the number of parameters of the polynomial function.

[0016] In one embodiment, preprocessed point cloud data is input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representational feature vector of the 3D model, including:

[0017] The preprocessed point cloud data is input into a pre-constructed Siamese neural network model. The Siamese neural network model includes a feature extraction module and a similarity comparison module connected in sequence. The input of the feature extraction module is the preprocessed point cloud data, and the output is the feature vector of the 3D model. The similarity comparison module consists of a pair of networks with shared weights. The two networks receive the feature vector output by the feature extraction module as input and output the label vector of the 3D model and perform similarity comparison on the label vectors of any two 3D models.

[0018] The label vectors of any two 3D models are input into a pre-constructed loss function expression for calculation. The Siamese neural network model is then trained and optimized based on the calculated loss function until a well-trained Siamese neural network model that meets the fitting accuracy requirements is obtained.

[0019] The preprocessed point cloud data is input into the feature extraction module of the trained Siamese neural network model for feature extraction, and the extracted feature vectors are used as the representation feature vectors of the 3D model.

[0020] In one embodiment, the pre-constructed loss function expression is:

[0021]

[0022] Where W represents the parameters of the Siamese neural network model, X1 and X2 represent the preprocessed point cloud data corresponding to the two 3D models, and y is the label for whether the 3D models are similar, y=1 indicates that the 3D models are similar, and y=0 indicates that the 3D models are not similar. For the similarity comparison module, F W For the feature extraction module, F W (X1) and F W (X2) represents converting X1 and X2 into feature vectors, respectively. This indicates that F will be respectively W (X1) and F W (X2) is converted into the corresponding label vector and the distance between the two label vectors is calculated, where m is a pre-set boundary threshold.

[0023] In one embodiment, the original feature vectors corresponding to all 3D models displayed by the client are obtained and clustered and compressed to obtain the compressed vector of each 3D model. The compressed vector and the corresponding model information are then stored in the model database, including:

[0024] Obtain the original feature vectors corresponding to all 3D models displayed on the client, and divide all the original feature vectors into n sub-segments, each sub-segment being W / n-dimensional, and each sub-segment containing the same number of sub-segment vectors as the original feature vectors; where W is the parameter of the Siamese neural network model;

[0025] Cluster the sub-segment vectors within each sub-segment sequentially. Each sub-segment has k cluster centers, resulting in a total of n×k cluster centers. Divide the original feature vector corresponding to each 3D model that needs to be stored in the model database into n sub-segments. Calculate the nearest cluster center for each sub-segment and map the sub-segment to one of the k cluster centers to obtain the n-dimensional compressed vector for each 3D model. Then, store the compressed vector and the corresponding model information in the model database.

[0026] In one embodiment, the point cloud data of the 3D model to be queried is imported into the client and the corresponding original feature vector to be queried is obtained. The distance between the original feature vector to be queried and the compressed vector stored in the model database is calculated by comparison. Similar models are retrieved in the model database and displayed and interacted with on the client, including:

[0027] Import the point cloud data of the 3D model to be queried into the client, and perform preprocessing and feature extraction on the point cloud data of the 3D model to be queried in sequence to obtain the original feature vector to be queried.

[0028] The original feature vector to be queried is divided into n sub-segments, and the distance between each sub-segment and the cluster center corresponding to each sub-segment in the n-dimensional compressed vector of each 3D model in the model database is calculated to obtain a cluster center query distance table of length k. According to the cluster center query distance table, the cluster center query distances corresponding to the n sub-segments of the original feature vector to be queried are added together to obtain the distance between the 3D model to be queried and the 3D models stored in the model database. The M 3D models with the shortest distance are selected as similar models and output to the client for display and interaction.

[0029] A model retrieval device based on 3D model features, the device comprising:

[0030] The preprocessing module is used to acquire the point cloud data of the 3D model displayed by the client and perform preprocessing to obtain preprocessed point cloud data.

[0031] The feature extraction module is used to extract features from the preprocessed point cloud data through polynomial function fitting to obtain the distribution feature vector of the 3D model; and to train the preprocessed point cloud data into a pre-constructed Siamese neural network model, and to extract features from the trained Siamese neural network model to obtain the representation feature vector of the 3D model; the distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model; wherein the lengths of the distribution feature vector and the representation feature vector are the same.

[0032] The clustering and compression module is used to obtain the original feature vectors corresponding to all 3D models displayed by the client and perform clustering and compression, obtain the compressed vector of each 3D model, and store the compressed vector and the corresponding model information into the model database.

[0033] The similarity retrieval module is used to import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. By comparing and calculating the distance between the original feature vector to be queried and the compressed vector stored in the model database, similar models are retrieved from the model database and displayed and interacted with on the client.

[0034] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0035] Obtain the point cloud data of the 3D model displayed on the client and preprocess it to obtain the preprocessed point cloud data;

[0036] Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model. The preprocessed point cloud data is then input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model. The distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model. The lengths of the distribution feature vector and the representation feature vector are the same.

[0037] Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering and compression. Obtain the compressed vector of each 3D model and store the compressed vector and the corresponding model information in the model database.

[0038] Import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. Calculate the distance between the original feature vector to be queried and the compressed vector stored in the model database. Search for similar models in the model database and display and interact with the similar models on the client.

[0039] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0040] Obtain the point cloud data of the 3D model displayed on the client and preprocess it to obtain the preprocessed point cloud data;

[0041] Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model. The preprocessed point cloud data is then input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model. The distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model. The lengths of the distribution feature vector and the representation feature vector are the same.

[0042] Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering and compression. Obtain the compressed vector of each 3D model and store the compressed vector and the corresponding model information in the model database.

[0043] Import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. Calculate the distance between the original feature vector to be queried and the compressed vector stored in the model database. Search for similar models in the model database and display and interact with the similar models on the client.

[0044] The above-mentioned model retrieval method based on 3D model features has the following beneficial effects:

[0045] 1. This application uses a feature extraction method that combines polynomial function fitting and neural network model to extract multidimensional features of 3D models, thereby improving the accuracy of 3D model feature extraction.

[0046] 2. This application performs clustering and compression on the extracted 3D model features and stores the model compression vector and corresponding model information in the model database, which facilitates centralized management and maintenance of 3D models, as well as classification and organization of 3D models, and enables efficient management and rapid retrieval of 3D models in large 3D scenes.

[0047] 3. This application helps users quickly and accurately retrieve similar models from the model database by extracting the original feature vector of the 3D model to be queried and comparing its distance with the compressed vector stored in the model database, thus realizing the rapid positioning and retrieval of 3D models. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a model retrieval method based on 3D model features in one embodiment;

[0049] Figure 2 This is a structural block diagram of a model retrieval device based on 3D model features in one embodiment;

[0050] Figure 3 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] In one embodiment, such as Figure 1 As shown, a model retrieval method based on 3D model features is provided, including the following steps:

[0053] Step S1: Obtain the point cloud data of the 3D model displayed on the client and perform preprocessing to obtain preprocessed point cloud data. The 3D model displayed on the client is one or more 3D models pre-imported by the user. The client obtains the imported models and then displays and interacts with them.

[0054] Step S2: Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model; the preprocessed point cloud data is then input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model; the distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model; wherein the lengths of the distribution feature vector and the representation feature vector are the same.

[0055] It is understandable that by combining polynomial function fitting and neural network models, the feature extraction method can extract multi-dimensional features of the 3D model, thereby improving the accuracy of 3D model feature extraction.

[0056] Step S3: Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering compression to obtain the compressed vector of each 3D model, and store the compressed vector and the corresponding model information in the model database.

[0057] It is understandable that by clustering and compressing the extracted 3D model features and storing the model compression vector and corresponding model information in the model database, it is convenient to centrally manage and maintain 3D models, as well as to classify and organize 3D models, thus enabling efficient management and rapid retrieval of 3D models in large 3D scenes.

[0058] Step S4: Import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. Calculate the distance between the original feature vector to be queried and the compressed vector stored in the model database. Search for similar models in the model database and display and interact with the similar models on the client.

[0059] It is understandable that by extracting the original feature vector of the 3D model to be queried and comparing its distance with the compressed vector stored in the model database, users can quickly and accurately retrieve the similar models they need from the model database, thus achieving rapid positioning and retrieval of 3D models.

[0060] In one embodiment, point cloud data of the 3D model displayed by the client is acquired and preprocessed to obtain preprocessed point cloud data, including:

[0061] Obtain the point cloud data of the 3D model displayed on the client. When the 3D model contains multiple sub-models, divide the 3D model into multiple sub-models and obtain the corresponding point cloud data.

[0062] The global coordinates of all points in the point cloud data are restored by matrix coordinate transformation, and the model is standardized by overall translation and scaling of all points, so that the model is scaled to a unit cube bounding box.

[0063] By changing the global coordinates of all points so that the center of the unit cube bounding box coincides with the origin of the coordinate system in the point cloud data, the preprocessing of the point cloud data is completed and the preprocessed point cloud data is obtained.

[0064] Specifically, after completing the preprocessing of the point cloud data, the process also includes: adjusting the camera position and orientation, and saving and outputting the preprocessing results of the 3D model.

[0065] It is understandable that by preprocessing the point cloud data of the 3D model, the 3D model can be scaled down to a unit cube bounding box, which effectively reduces the size and complexity of the point cloud data, reduces the amount of data occupied, and improves the processing speed of subsequent steps.

[0066] In one embodiment, feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model, including:

[0067] By randomly sampling any two points in the preprocessed point cloud data and calculating the distance between the two points, the point distance (D2) shape probability distribution of the 3D model is obtained.

[0068] The point distance shape probability distribution is divided into several segments. A polynomial function is used to fit each segment, and the parameters of the fitted polynomial function are used as the distribution feature vector of the 3D model. The length of the distribution feature vector is the same as the number of parameters of the polynomial function. Specifically, the D2 shape probability distribution of the 3D model can be divided into 16 segments. A 7th-order polynomial function is used to fit each segment, resulting in 128 parameters of the polynomial function. These 128 parameters are used as the distribution feature vector of the 3D model.

[0069] It is understandable that in order to incorporate the density of point cloud data into the distribution feature vector of the 3D model, the density is represented by the probability distribution of point distance. However, the probability distribution is difficult to directly convert into a computable vector. Therefore, this application obtains a vector representing this probability distribution through multinomial fitting.

[0070] In one embodiment, preprocessed point cloud data is input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representational feature vector of the 3D model, including:

[0071] The preprocessed point cloud data is input into a pre-constructed Siamese neural network model. The Siamese neural network model includes a feature extraction module and a similarity comparison module connected in sequence. The input of the feature extraction module is the preprocessed point cloud data, and the output is the feature vector of the 3D model. The similarity comparison module consists of a pair of networks with shared weights. The two networks receive the feature vector output by the feature extraction module as input and output the label vector of the 3D model and perform similarity comparison on the label vectors of any two 3D models.

[0072] The label vectors of any two 3D models are input into a pre-constructed loss function expression for calculation. The Siamese neural network model is then trained and optimized based on the calculated loss function until a well-trained Siamese neural network model that meets the fitting accuracy requirements is obtained.

[0073] The preprocessed point cloud data is input into the feature extraction module of the trained Siamese neural network model for feature extraction, and the extracted feature vectors are used as the representation feature vectors of the 3D model.

[0074] Specifically, the feature extraction module can be a multilayer perceptron model. The input of this module is N×3 point cloud data, which is converted into a feature matrix of size N×128 after passing through the multilayer perceptron. Then, the feature matrix is ​​converted into a 1×128 feature vector through max pooling, where N is the number of points in the point cloud data. The length of the feature vector is consistent with the length of the distributed feature vector and the number of parameters of the polynomial function, which is 128.

[0075] In one embodiment, the pre-constructed loss function expression is:

[0076]

[0077] Where W represents the parameters of the Siamese neural network model, X1 and X2 represent the preprocessed point cloud data corresponding to the two 3D models, and y is the label for whether the 3D models are similar, y=1 indicates that the 3D models are similar, and y=0 indicates that the 3D models are not similar. For the similarity comparison module, F W For the feature extraction module, F W (X1) and F W (X2) represents converting X1 and X2 into feature vectors, respectively. This indicates that F will be respectively W (X1) and F W(X2) is converted into the corresponding label vector, and the distance between the two label vectors is calculated, where m is a pre-set boundary threshold. It can be understood that training the Siamese neural network model involves backpropagating the error between the feature extraction module and the similarity comparison module, ensuring that similar input and output values ​​are as close to 1 as possible, and 0 otherwise.

[0078] In one embodiment, the original feature vectors corresponding to all 3D models displayed by the client are obtained and clustered and compressed to obtain the compressed vector of each 3D model. The compressed vector and the corresponding model information are then stored in the model database, including:

[0079] Obtain the original feature vectors corresponding to all 3D models displayed on the client, and divide all the original feature vectors into n sub-segments, each sub-segment being W / n-dimensional, and each sub-segment containing the same number of sub-segment vectors as the original feature vectors; where W is the parameter of the Siamese neural network model;

[0080] Cluster the sub-segment vectors within each sub-segment sequentially. Each sub-segment has k cluster centers, resulting in a total of n×k cluster centers. Divide the original feature vector corresponding to each 3D model that needs to be stored in the model database into n sub-segments. Calculate the nearest cluster center for each sub-segment and map the sub-segment to one of the k cluster centers to obtain the n-dimensional compressed vector for each 3D model. Then, store the compressed vector and the corresponding model information in the model database.

[0081] Specifically, all original feature vectors can be divided into four sub-segments, each with 128 / 4 = 32 dimensions. Then, the sub-segment vectors within each sub-segment are clustered sequentially, with each sub-segment having 256 cluster centers, resulting in a total of 4 × 256 cluster centers. The original feature vector corresponding to each 3D model to be stored in the model database is then divided into four sub-segments. The nearest cluster center for each sub-segment is calculated, and the sub-segment is mapped to one of the 256 cluster centers, resulting in a 4D compressed vector for each 3D model. The compressed vector and the corresponding model information are then stored in the model database.

[0082] In one embodiment, the point cloud data of the 3D model to be queried is imported into the client and the corresponding original feature vector to be queried is obtained. The distance between the original feature vector to be queried and the compressed vector stored in the model database is calculated by comparison. Similar models are retrieved in the model database and displayed and interacted with on the client, including:

[0083] Import the point cloud data of the 3D model to be queried into the client, and perform preprocessing and feature extraction on the point cloud data of the 3D model to be queried in sequence to obtain the original feature vector to be queried.

[0084] The original feature vector to be queried is divided into n sub-segments, and the distance between each sub-segment and the cluster center corresponding to each sub-segment in the n-dimensional compressed vector of each 3D model in the model database is calculated to obtain a cluster center query distance table of length k. According to the cluster center query distance table, the cluster center query distances corresponding to the n sub-segments of the original feature vector to be queried are added together to obtain the distance between the 3D model to be queried and the 3D models stored in the model database. The M 3D models with the shortest distance are selected as similar models and output to the client for display and interaction.

[0085] Specifically, the original feature vector to be queried can be divided into four sub-segments, and the distance between each sub-segment and the cluster center corresponding to each sub-segment in the 4D compressed vector of each 3D model in the model database can be calculated, resulting in a cluster center query distance table of length 256. According to the cluster center query distance table, the cluster center query distances corresponding to the four sub-segments of the original feature vector to be queried are added together to obtain the distance between the 3D model to be queried and the 3D models stored in the model database. The five 3D models with the shortest distances are selected as similar models and output to the client for display and interaction.

[0086] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0087] In one embodiment, such as Figure 2 As shown, a model retrieval device based on 3D model features is provided, comprising:

[0088] The preprocessing module 201 is used to acquire the point cloud data of the 3D model displayed by the client and perform preprocessing to obtain preprocessed point cloud data.

[0089] The feature extraction module 202 is used to extract features from the preprocessed point cloud data through polynomial function fitting to obtain the distribution feature vector of the 3D model; and to train the preprocessed point cloud data by inputting it into a pre-constructed Siamese neural network model, and to extract features based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model; the distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model; wherein the lengths of the distribution feature vector and the representation feature vector are the same;

[0090] The clustering compression module 203 is used to obtain the original feature vectors corresponding to all the 3D models displayed by the client and perform clustering compression, obtain the compressed vector of each 3D model, and store the compressed vector and the corresponding model information into the model database.

[0091] The similarity retrieval module 204 is used to import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. By comparing and calculating the distance between the original feature vector to be queried and the compressed vector stored in the model database, similar models are retrieved in the model database and displayed and interacted with on the client.

[0092] Specific limitations regarding the model retrieval device based on 3D model features can be found in the limitations of the model retrieval method based on 3D model features mentioned above, and will not be repeated here. Each module in the aforementioned model retrieval device based on 3D model features can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0093] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a model retrieval method based on 3D model features. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0094] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0095] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps:

[0096] Obtain the point cloud data of the 3D model displayed on the client and preprocess it to obtain the preprocessed point cloud data;

[0097] Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model. The preprocessed point cloud data is then input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model. The distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model. The lengths of the distribution feature vector and the representation feature vector are the same.

[0098] Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering and compression. Obtain the compressed vector of each 3D model and store the compressed vector and the corresponding model information in the model database.

[0099] Import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. Calculate the distance between the original feature vector to be queried and the compressed vector stored in the model database. Search for similar models in the model database and display and interact with the similar models on the client.

[0100] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0101] Obtain the point cloud data of the 3D model displayed on the client and preprocess it to obtain the preprocessed point cloud data;

[0102] Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model. The preprocessed point cloud data is then input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model. The distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model. The lengths of the distribution feature vector and the representation feature vector are the same.

[0103] Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering and compression. Obtain the compressed vector of each 3D model and store the compressed vector and the corresponding model information in the model database.

[0104] Import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. Calculate the distance between the original feature vector to be queried and the compressed vector stored in the model database. Search for similar models in the model database and display and interact with the similar models on the client.

[0105] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0106] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model retrieval method based on 3D model features, characterized in that, The method includes: Obtain the point cloud data of the 3D model displayed on the client and preprocess it to obtain the preprocessed point cloud data; Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model; the preprocessed point cloud data is then input into a pre-constructed Siamese neural network model for training, and feature extraction is performed based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model; the distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model; wherein the lengths of the distribution feature vector and the representation feature vector are the same. Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering and compression. Obtain the compressed vector of each 3D model and store the compressed vector and the corresponding model information in the model database. Import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. Calculate the distance between the original feature vector to be queried and the compressed vector stored in the model database. Retrieve similar models in the model database and display and interact with the similar models on the client.

2. The method according to claim 1, characterized in that, Obtain the point cloud data of the 3D model displayed on the client and preprocess it to obtain the preprocessed point cloud data, including: Obtain the point cloud data of the 3D model displayed on the client. When the 3D model contains multiple sub-models, divide the 3D model into multiple sub-models and obtain the corresponding point cloud data. The global coordinates of all points in the point cloud data are restored by matrix coordinate transformation, and the model is standardized by overall translation and scaling of all points, so that the model is scaled to a unit cube bounding box. By changing the global coordinates of all points so that the center of the unit cube bounding box coincides with the origin of the coordinate system in the point cloud data, the preprocessing of the point cloud data is completed and the preprocessed point cloud data is obtained.

3. The method according to claim 2, characterized in that, Feature extraction is performed on the preprocessed point cloud data using polynomial function fitting to obtain the distribution feature vector of the 3D model, including: By randomly sampling any two points in the preprocessed point cloud data and calculating the distance between the two points, the probability distribution of point distance shape in the three-dimensional model is obtained. The probability distribution of the point distance shape is divided into several segments. A polynomial function is used to fit each segment of data, and the parameters of the fitted polynomial function are used as the distribution feature vector of the three-dimensional model. The length of the distribution feature vector is consistent with the number of parameters of the polynomial function.

4. The method according to claim 2, characterized in that, The preprocessed point cloud data is input into a pre-constructed Siamese neural network model for training, and features are extracted based on the trained Siamese neural network model to obtain the representational feature vector of the 3D model, including: The preprocessed point cloud data is input into a pre-constructed Siamese neural network model. The Siamese neural network model includes a feature extraction module and a similarity comparison module connected in sequence. The input of the feature extraction module is the preprocessed point cloud data, and the output is a feature vector of the 3D model. The similarity comparison module consists of a pair of networks with shared weights. The two networks respectively receive the feature vector output by the feature extraction module as input, and output the label vector of the 3D model and perform a similarity comparison on the label vectors of any two 3D models. The label vectors of any two 3D models are input into a pre-constructed loss function expression for calculation, and the Siamese neural network model is trained and optimized based on the calculated loss function until a well-trained Siamese neural network model that meets the fitting accuracy requirements is obtained. The preprocessed point cloud data is input into the feature extraction module of the trained Siamese neural network model for feature extraction, and the extracted feature vector is used as the representation feature vector of the three-dimensional model.

5. The method according to claim 4, characterized in that, The pre-constructed loss function expression is as follows: Where W represents the parameters of the Siamese neural network model, X1 and X2 represent the preprocessed point cloud data corresponding to the two 3D models, and y is the label for whether the 3D models are similar, y=1 indicates that the 3D models are similar, and y=0 indicates that the 3D models are not similar. For the similarity comparison module, F W For the feature extraction module, F W (X1) and F W (X2) represents converting X1 and X2 into feature vectors, respectively. This indicates that F will be respectively W (X1) and F W (X2) is converted into the corresponding label vector and the distance between the two label vectors is calculated, where m is a pre-set boundary threshold.

6. The method according to claim 1, characterized in that, Obtain the original feature vectors corresponding to all 3D models displayed on the client and perform clustering and compression. Obtain the compressed vector for each 3D model and store the compressed vector and corresponding model information in the model database, including: Obtain the original feature vectors corresponding to all 3D models displayed on the client, and divide all the original feature vectors into n sub-segments, each sub-segment being W / n-dimensional, and each sub-segment containing the same number of sub-segment vectors as the original feature vectors; where W is the parameter of the Siamese neural network model; Cluster the sub-segment vectors within each sub-segment sequentially. Each sub-segment has k cluster centers, resulting in a total of n×k cluster centers. Divide the original feature vector corresponding to each 3D model to be stored in the model database into n sub-segments. Calculate the nearest cluster center for each sub-segment and map the sub-segment to one of the k cluster centers to obtain the n-dimensional compressed vector of each 3D model. Store the compressed vector and the corresponding model information in the model database.

7. The method according to claim 6, characterized in that, The point cloud data of the 3D model to be queried is imported into the client, and the corresponding original feature vector to be queried is obtained. The distance between the original feature vector to be queried and the compressed vector stored in the model database is calculated by comparison. Similar models are retrieved from the model database, and the similar models are displayed and interacted with on the client, including: Import the point cloud data of the 3D model to be queried into the client, and perform preprocessing and feature extraction on the point cloud data of the 3D model to be queried in sequence to obtain the original feature vector to be queried. The original feature vector to be queried is divided into n sub-segments, and the distance between each sub-segment and the cluster center corresponding to each sub-segment in the n-dimensional compressed vector of each 3D model in the model database is calculated to obtain a cluster center query distance table of length k. According to the cluster center query distance table, the cluster center query distances corresponding to the n sub-segments of the original feature vector to be queried are added together to obtain the distance between the 3D model to be queried and the 3D models stored in the model database. The M 3D models with the shortest distance are selected as similar models and output to the client for display and interaction.

8. A model retrieval device based on three-dimensional model features, characterized in that, The device includes: The preprocessing module is used to acquire the point cloud data of the 3D model displayed by the client and perform preprocessing to obtain preprocessed point cloud data. The feature extraction module is used to extract features from the preprocessed point cloud data through polynomial function fitting to obtain the distribution feature vector of the 3D model; and to train the preprocessed point cloud data by inputting it into a pre-constructed Siamese neural network model, and to extract features based on the trained Siamese neural network model to obtain the representation feature vector of the 3D model; the distribution feature vector and the representation feature vector are added together to obtain the original feature vector of the 3D model; wherein the lengths of the distribution feature vector and the representation feature vector are the same; The clustering and compression module is used to obtain the original feature vectors corresponding to all 3D models displayed by the client and perform clustering and compression, obtain the compressed vector of each 3D model, and store the compressed vector and the corresponding model information into the model database. The similarity retrieval module is used to import the point cloud data of the 3D model to be queried into the client and obtain the corresponding original feature vector to be queried. By comparing and calculating the distance between the original feature vector to be queried and the compressed vector stored in the model database, similar models are retrieved from the model database and displayed and interacted with on the client.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images