Identity recognition method and system based on 3D tooth point cloud multi-feature fusion

Abstract

The invention discloses an identity recognition method and system based on 3D tooth point cloud multi-feature fusion. The method comprises the following steps: collecting a 3D tooth point cloud of a to-be-recognized identity; inputting the 3D tooth point cloud of which the identity is to be recognized and each 3D tooth point cloud of which the identity is registered in the identity database into the trained tooth multi-feature fusion identity recognition backbone network, and outputting a registration result between the 3D tooth point cloud of which the identity is to be recognized and each 3D tooth point cloud of which the identity is registered; the backbone network comprises a color feature extraction module, a structural feature extraction module, a feature fusion module, a local sampling feature extraction module, a feature aggregation module, a coarse correspondence prediction module, a point cloud decoding module, a fine correspondence prediction module and a prediction registration module. And taking the identity information of the 3D tooth point cloud with the registered identity, which is optimally registered with the 3D tooth point cloud with the identity to be identified, as the identity label of the 3D tooth point cloud with the identity to be identified. According to the invention, identity recognition can be carried out based on the 3D tooth point cloud.

Description

Technical Field

[0001] The present invention relates to the field of identity recognition technology, and in particular to an identity recognition method and system based on 3D tooth point cloud multi-feature fusion. Background Art

[0002] The 3D tooth point cloud data collected by intraoral scanning instruments can accurately reflect the subtle differences and uniqueness of teeth, including geometric and structural features such as dental arch shape, interdental arrangement, tooth unevenness, and tooth-gingival boundary. Compared with recognition using 2D images, 3D point cloud recognition can obtain richer tooth anatomical structure features, is less susceptible to factors such as lighting, angle, and occlusion, and has stronger anti-interference ability and higher accuracy. In addition, with the development of digital technology, the acquisition of 3D oral data is becoming increasingly convenient and popular, laying a solid foundation for the construction of a large-scale 3D tooth point cloud identity database, which in turn can promote the continuous improvement of identity recognition technology based on 3D tooth point clouds.

[0003] The identity recognition technology based on 3D tooth point cloud needs to process the collected 3D tooth point cloud and extract representative tooth features. These features should be able to reflect the uniqueness of the teeth and are the basis of identity recognition. At present, the mainstream identity recognition method based on 3D tooth point cloud often uses an iterative algorithm to directly perform 3D-3D registration on the tooth point cloud and perform identity recognition based on the registration results. However, the registration process included in this method requires multiple rounds of iterations to find the optimal matching relationship between the two point clouds. This iterative process usually has a high computational complexity because it is necessary to continuously establish matching point pairs and adjust the posture in three-dimensional space to minimize the alignment error of the two point clouds. This process involves a large number of matrix operations and optimization iterations, which makes the operation efficiency low, especially when the point cloud data volume is large, the recognition efficiency is low. In addition, these mainstream methods usually directly use the spatial distance difference information between the points in the point cloud for registration. Once the tooth point cloud is collected from different devices, the coordinate system between the point clouds will often have a large deviation due to the difference in equipment, which increases the difficulty of registration and affects the accuracy of the recognition result. In addition, the mainstream method only relies on the spatial coordinate information of the tooth point cloud. The existing methods have obvious deficiencies in the robustness and accuracy of registration in actual identity recognition tasks. Summary of the invention

[0004] The present invention provides an identity recognition method and system based on 3D tooth point cloud multi-feature fusion to solve the problems existing in the above-mentioned prior art. The technical solution is as follows:

[0005] On the one hand, an identity recognition method based on 3D tooth point cloud multi-feature fusion is provided, comprising:

[0006] S1, collect 3D tooth point cloud of the identity to be identified;

[0007] S2, inputting the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity in the identity database into the trained tooth multi-feature fusion identity recognition backbone network, and outputting the registration result between the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity;

[0008] The tooth multi-feature fusion identity recognition backbone network includes: a color feature extraction module, a structural feature extraction module, a feature fusion module, a local sampling feature extraction module, a feature aggregation module, a coarse correspondence prediction module, a point cloud decoding module, a fine correspondence prediction module, and a prediction registration module;

[0009] S3. Using the identity information of the 3D tooth point cloud of the registered identity that is best aligned with the 3D tooth point cloud of the identity to be identified as the identity identifier of the 3D tooth point cloud of the identity to be identified.

[0010] On the other hand, an identity recognition system based on 3D tooth point cloud multi-feature fusion is provided, the system comprising:

[0011] The acquisition module is used to acquire the 3D tooth point cloud of the identity to be identified;

[0012] A registration module is used to input the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity in the identity database into the trained tooth multi-feature fusion identity recognition backbone network, and output the registration result between the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity;

[0013] The tooth multi-feature fusion identity recognition backbone network includes: a color feature extraction module, a structural feature extraction module, a feature fusion module, a local sampling feature extraction module, a feature aggregation module, a coarse correspondence prediction module, a point cloud decoding module, a fine correspondence prediction module, and a prediction registration module;

[0014] The identification module is used to use the identity information of the 3D tooth point cloud of the registered identity that is best aligned with the 3D tooth point cloud of the identity to be identified as the identity identifier of the 3D tooth point cloud of the identity to be identified.

[0015] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned identity recognition method based on 3D tooth point cloud multi-feature fusion.

[0016] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned identity recognition method based on multi-feature fusion of 3D tooth point cloud.

[0017] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0018] 1) A color feature extraction module and a structural feature extraction module of the tooth point cloud were designed, which can extract the effective color feature representation and structural feature representation of the point according to the color information and spatial coordinate information of the known point and its neighborhood points, and then fuse the extracted color features and structural features to enrich the feature information that can be used in the link of establishing the corresponding point relationship of the tooth point cloud before the identity recognition stage from multiple dimensions, thereby improving the accuracy of identity recognition.

[0019] 2) A fast end-to-end identity recognition network based on tooth point cloud was constructed. The point correspondence was established by using the key points at the neighborhood level and the points at the single point level of the tooth point cloud in a coarse-to-fine manner. This enabled the network to find matching point pairs more accurately, thereby improving the prediction efficiency of matching point pairs while ensuring the prediction accuracy of matching point pairs, thereby improving the efficiency of the identity recognition task.

[0020] 3) Before the corresponding point relationship in the network is established, the tooth point cloud is subjected to feature extraction by the designed local sampling feature extraction module, and the extracted features are used as the basis for establishing matching point pairs. This extracted feature will not change due to large changes in the three-dimensional coordinate system of the tooth point cloud caused by changes in the acquisition equipment, and is stable, thus improving the stability of identity recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 This is a flow chart of an identity recognition method based on 3D tooth point cloud multi-feature fusion provided by an embodiment of the present invention;

[0023] Figure 2 Schematic diagram of the backbone network structure of tooth multi-feature fusion identity recognition provided by an embodiment of the present invention;

[0024] Figure 3 It is an overall block diagram of the identity recognition method based on tooth multi-feature fusion provided by an embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of the structure of a color feature extraction module provided by an embodiment of the present invention;

[0026] Figure 5 is a color frequency histogram provided by an embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of the structure of a structural feature extraction module provided in an embodiment of the present invention;

[0028] Figure 7 is a structural frequency histogram provided by an embodiment of the present invention;

[0029] Figure 8 It is a schematic diagram of the structure of a local sampling feature extraction module provided by an embodiment of the present invention;

[0030] Fig. 9 is a schematic diagram of the structure of a feature aggregation module provided in an embodiment of the present invention;

[0031] Fig.10 is a schematic diagram of the structure of a coarse correspondence prediction module provided by an embodiment of the present invention;

[0032] Fig.11 is a schematic diagram of the structure of a point cloud decoding module provided by an embodiment of the present invention;

[0033] Fig.12 is a schematic diagram of the structure of a precise correspondence prediction module provided by an embodiment of the present invention;

[0034] Fig.13 is a schematic diagram of the structure of a prediction and registration module provided by an embodiment of the present invention;

[0035] Fig.14 It is a block diagram of an identity recognition system based on 3D tooth point cloud multi-feature fusion provided by an embodiment of the present invention;

[0036] Fig.15 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0038] The embodiment of the present invention provides an identity recognition method based on 3D tooth point cloud multi-feature fusion, which can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The flowchart of an identity recognition method based on 3D tooth point cloud multi-feature fusion is shown, and the processing flow of the method may include the following steps:

[0039] S1, collect 3D tooth point cloud of the identity to be identified;

[0040] The embodiment of the present invention collects the 3D tooth point cloud of the identity to be identified through an intraoral scanner.

[0041] S2, inputting the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity in the identity database into the trained tooth multi-feature fusion identity recognition backbone network, and outputting the registration result between the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity;

[0042] like Figure 2 As shown, the tooth multi-feature fusion identity recognition backbone network includes: a color feature extraction module, a structural feature extraction module, a feature fusion module, a local sampling feature extraction module, a feature aggregation module, a coarse correspondence prediction module, a point cloud decoding module, a fine correspondence prediction module, and a prediction registration module;

[0043] like Figure 3 As shown, the training process of the tooth multi-feature fusion identity recognition backbone network includes:

[0044] 1) Tooth point cloud data collection.

[0045] The embodiment of the present invention collects 3D tooth point cloud data of multiple individuals through an intraoral scanner, and records the identity information of the collected individuals in text form.

[0046] 2) Combining manual adjustment with the optimized registration algorithm ICP, a transformation matrix RT is obtained that can register two tooth point clouds scanned from the same person to completely overlap, and the true matching point pairs between the two tooth point cloud matching pairs are calculated.

[0047] The specific processing process of step 2) is prior art and will not be repeated here.

[0048] 3) Dataset division.

[0049] The tooth point cloud matching pair obtained in step 2 is regarded as a sample, and all the collected and constructed samples are divided into a training set and a validation set in a ratio of 4:1 to form a complete data set. The algorithm will be trained on the training set, and the model effect will be verified on the validation set. The model with the best performance on the validation set will be selected as the final model.

[0050] 4) Construction of identity database.

[0051] After training, any one of the tooth point clouds in all sample pairs is taken out and stored in the identity database for identity information registration, and the remaining tooth point clouds are used as test set data to test the performance of the model.

[0052] like Figure 4 As shown, the color feature extraction module is used to extract the color feature information of the 3D tooth point cloud. The specific extraction process includes:

[0053] Assume that the 3D tooth point cloud is P∈R N×6 , where N represents the number of points in P, and the number 6 means that each point in P has 3D spatial coordinate information and 3D color information;

[0054] Define any point in P as the center point P center and set the default R color (such as 0.5mm) as the sphere radius, and define other points within this range as neighboring points P neighbor_1 ~P neighbor_n , the center point P center Perform difference calculation with the RGB color channel values ​​of each neighboring point: R diff =R center -R neighbor , G diff =G center -G neighbor , B diff =B center -B neighbor , and divide the calculated color difference into certain intervals, and count R diff , G diff and B diff The result (for example, divided into M color = 33 intervals, of which the 1st to 11th intervals are for statistical R diff The result is that the intervals 12-22 are for statistical G diff The result is that the interval 23-33 is for counting B diff The results of these intervals represent different levels of color difference, and the sum of the statistical results of all intervals of the three channels constitutes the center point P center The initial color frequency histogram ICH(P center ),like Figure 5 As shown, combined with the actual values ​​of the tooth color channel difference intervals, the embodiment of the present invention stipulates that the statistical value range of the three types of intervals is between -100 and 100;

[0055] Then, taking each neighboring point as the center, repeat the above operation to obtain the color frequency histogram ICH (P neighbor_1 )~ICH(P neighbor_n );

[0056] According to each neighboring point and the center point P centerThe distance d is taken as the weight, and the color frequency histogram results of all neighboring points are added to the center point P. center On the color frequency histogram, the center point P is obtained center Final color frequency histogram

[0057] The final color frequency histogram FCH(P center ) is converted to an M color dimensional vector, as the center point P center Color feature information;

[0058] Take all points in P as the center point and perform the above operation to obtain the color feature information of all points in the point cloud, which is recorded as The 3 indicates that the first three columns of information of the feature are the three-dimensional spatial coordinates of the point.

[0059] like Figure 6 As shown, the structural feature extraction module is used to extract the structural feature information of the 3D tooth point cloud. The specific extraction process includes:

[0060] Calculate the normal vector of each point in the point cloud based on the three-dimensional spatial coordinate information of each point in the point cloud and its surrounding points;

[0061] Define any point in P as the center point P center and set the default R struct (such as 0.5mm) as the sphere radius, and define other points within this range as neighboring points P neighbor_1 ~P struct_n , describing the center point P center and the geometric features of each neighboring point, where the center point P is described as follows center and P neighbor_1 The geometric structure characteristics of the two points (the center point P center and other neighboring points have similar geometrical characteristics): Let vector d = P center -P neighbor_1 , represents the center point P center and P neighbor_1 The connection vector between two points, the center point P center The normal vector is defined as P center , P neighbor_1 The normal vector is defined as P neighbor_1 , define α to represent n neighbor_1 The angle between and d is calculated as It measures the projection relationship of the normal vector of the neighborhood point in the direction of the target point; φ is defined to represent the center point P center The normal vector n centerThe angle between d and d is calculated as Reflects the neighborhood point P neighbor_1 Relative to the center point P center The normal vector n center The projection angle of n center and n neighbor_1 The angle between them is calculated as: θ = arccos(n center ·n neighbor_1 ), which measures the degree of alignment of the normal vectors of two points. These parameters together describe the center point P center With neighboring point P neighbor_1 The geometrical characteristics between

[0062] Construct the center point P center The initial structure frequency histogram ISH(P center ),like Figure 7 As shown, the initial structure frequency histogram ISH (P center )Total M struct intervals, used to count the center point P respectively center and P neighbor_1 The results of the three parameters α, φ and θ are used as feature information. The value range of α is [-1,1]. When α=1, it means that the normal vector of the neighboring point is completely parallel to the normal vector of the center point and has the same direction; when α=-1, it means that they are in opposite directions; when α=0, it means that the two are perpendicular; the value range of φ is also [-1,1]. When φ=1, the normal vector of the center point is in the same direction as the line between the center point and the neighboring point; when φ=-1, the normal vector of the center point is in the opposite direction to the line between the center point and the neighboring point; when φ=0, the normal vector of the center point is perpendicular to the line between the center point and the neighboring point; the value range of θ is [-π,π]. When θ=0, it means there is no rotation; when θ=π or θ=-π, it means a rotation of 180 degrees, reflecting a larger angular offset;

[0063] For the center point P center The other neighboring points of are then used to construct structural frequency histograms, and then added to the center point P in a weighted manner. center The initial structure frequency histogram FSH(P center ) on the center point P center Final structural frequency histogram

[0064] The center point P center The structural frequency histogram FSH(P center ) is converted to an M struct dimensional vector, as the center point P center Structural feature information;

[0065] Take all points in P as the center points and perform the above operations to obtain the structural feature information of all points in the point cloud, which can be recorded as The 3 indicates that the first three columns of information of the feature are the three-dimensional spatial coordinates of the point.

[0066] The feature fusion module extracts M from each point color dimensional color feature information and M struct The 3D structural feature information is spliced ​​with the 3D spatial coordinate information of each point, and the output point cloud contains 3+M color +M struct Fusion results of dimensional feature information

[0067] like Figure 8 As shown, the local sampling feature extraction module first performs the feature fusion module output Perform a downsampling operation to obtain a set of sparse points as neighborhood-level key points to reduce the number of points in the point cloud;

[0068] In order to calculate the features of these key points, a neighborhood with K as the neighborhood radius is established for each key point to search for original points close to these key points in the point cloud before downsampling. Then, according to the distance relationship between the searched original points and the corresponding key points, the features of these original points are aggregated to the corresponding key points as the local feature information F' of the corresponding key points, and the key points (P', F') with local feature information are output, where P' represents the three-dimensional spatial coordinates of the key points.

[0069] The features extracted by the local sampling feature extraction module are highly stable, and even if the acquisition device is changed so that the initial spatial position of the model changes, these features will not change.

[0070] like Fig. 9 As shown in FIG. 1 , the feature aggregation module relies on the attention mechanism to strengthen and aggregate the local feature information in the neighborhood-level key points, integrate the global and local context information of each point in the input key points, and enable the features of the neighborhood-level key points of the two to-be-registered point clouds to interact with each other, thereby improving the subsequent point cloud registration accuracy. The module includes two self-attention modules and one cross-attention module, and the processing process is as follows:

[0071] First, the features of the input neighborhood-level key points are enhanced by the self-attention module. The self-attention mechanism contained in the self-attention module generates the query matrix Q, the key matrix K and the value matrix V by linearly transforming the input feature matrix F':

[0072] Q=F'W Q,K=F'W K ,V=F'W V

[0073] Where W Q , W K , W V is a learnable linear transformation matrix;

[0074] Next, the attention score matrix A is obtained by calculating the dot product of the query matrix and the key matrix and normalizing it:

[0075]

[0076] where d k is the size in the key matrix, which is used for normalization to prevent the value from being too large. The attention score matrix A is used to perform weighted summation on the value matrix V to obtain the aggregated feature matrix F":

[0077] F″=A·V

[0078] In this way, the feature of each point not only contains its own information, but also aggregates the global context information related to it;

[0079] Subsequently, the result is input into the cross attention module to integrate the feature information between key points at different point cloud neighborhood levels. The feature matrices of the two point cloud key points processed by the self-attention module are F' X 'and F' Y ', the cross attention module then performs feature aggregation through the following steps:

[0080] Q X =F″ X W Q ,K Y =F″ Y W K ,V Y =F″ Y W V

[0081] Calculate F″ X and F″ Y The attention score matrix A between:

[0082]

[0083] Using the attention score matrix A X Y pair value matrix V Y Perform weighted summation to obtain the enhanced feature matrix F″′ X :

[0084] F″′ X =AX Y.V Y

[0085] Calculate F″ X and F″ Y The cross attention is F″′ Y :

[0086] F″′ Y =A Y X.V X

[0087] Through the cross attention module, the features of the two key points are mutually enhanced;

[0088] After the cross attention module, the feature representation capability is further enhanced through a self-attention calculation. Through the feature aggregation module, the features of each point in the output neighborhood-level key points not only integrate the geometric information of its neighborhood, but also integrate the features from the global context, and also interact with the key points of the point cloud to be registered with it. Finally, the key points of the registered point cloud X after feature aggregation are output. And the point cloud of the identity to be identified after feature aggregation Y Key Points

[0089] The features output by the feature aggregation module are more robust to the subsequent matching point establishment task.

[0090] like Fig.10 As shown, the coarse correspondence prediction module obtains the key points of the registered point cloud X after feature aggregation by calculating the inner product of the point cloud feature vector. And the point cloud of the identity to be identified after feature aggregation Y Key Points A similarity matrix S' between them, each element in the similarity matrix S' represents the similarity between two key points;

[0091] After obtaining the similarity matrix S', a normalization algorithm is used to normalize the similarity matrix (the purpose of normalization is to adjust these similarity values ​​to a uniform range (usually 0 to 1) to ensure that the similarity values ​​between different point clouds can be compared on the same scale, so that the similarity evaluation between all point pairs is performed under the same standard, making the difference between high and low values ​​more reasonable and not causing the matching process to be biased towards a specific range of values), and a matching probability matrix is ​​obtained by solving the problem, which represents the matching confidence of each pair of points between the key points of the neighborhood of the two point clouds;

[0092] Then, by setting a confidence threshold, key point pairs with higher confidence are screened out from the matching probability matrix as rough corresponding relationship predictions;

[0093] Finally, a neighborhood-level matching point pair containing multiple rough correspondences is output. This rough correspondence set will be further processed in the subsequent fine correspondence prediction module to obtain more accurate point pair matching results.

[0094] like Fig.11 As shown, the point cloud decoding module decodes and restores the rough neighborhood key point correspondence to the single point level, and uses the KP convolution decoder to convert the key points of the registered point cloud X after feature aggregation. And the point cloud of the identity to be identified after feature aggregation Y Key Points Decode to the same resolution as the point cloud before downsampling to obtain the decoded point cloud and This process involves a series of convolution operations to gradually restore the spatial distribution and geometric features similar to those of the point cloud before downsampling. The KP convolution decoder will gradually propagate the features of the neighborhood key points to the points after recovery. These propagated features will be combined with the geometric information of the point cloud before downsampling to generate a more fine-grained feature representation. The decoding process also involves a point-to-neighborhood level key point grouping strategy. The point cloud grouping module is used to group each point in the decoded point cloud according to its neighborhood level key points, and the points in the decoded point cloud are assigned to the nearest neighborhood level key points, and the grouping results of the registered point cloud X are output. And the grouping result of the identity point cloud Y to be identified For subsequent refined registration, the subsequent fine correspondence prediction module uses the grouping results obtained from the point cloud decoding module and the key point matching results obtained from the coarse correspondence prediction module to further refine the matching point pairs to obtain single-point level matching point pairs.

[0095] like Fig.12 As shown in FIG. 1 , the precise correspondence prediction module includes similarity matrix calculation, normalization processing and confidence screening steps. First, a set of matching point pairs is selected from the neighborhood level matching point pairs, and then the point cloud group constructed by these two points is selected from the grouping results: Grouping results of registered point cloud X and the point cloud of the identity to be identified Y The grouping results Next, the similarity matrix is ​​calculated for these two point clouds. The similarity matrix By calculating the inner product of all point feature vectors between two point cloud groups, each element in the matrix represents the similarity between two points; in the similarity matrix Afterwards, the results are normalized to obtain a matching probability matrix between the two point cloud groups, where the value of each element in the matching probability matrix represents the matching confidence of any two points between the two point cloud groups; then, the matching point pairs between the two point cloud groups are screened out by setting a confidence threshold; finally, all neighborhood-level matching point pairs and their corresponding point cloud grouping results are used to perform the above processing, and all matching point pair screening results are merged together to obtain single-point-level matching point pairs as the output result of the precise correspondence prediction module.

[0096] like Fig.13 As shown, the prediction registration module uses an optimization-based algorithm to calculate the optimal rotation and translation matrix based on the single-point level matching point pairs obtained by the precise correspondence prediction module to achieve the two tooth point clouds. and Specifically, the algorithm randomly selects a minimum number of point pairs, usually three pairs of points, from the matched point pairs to estimate a candidate rigid transformation. This transformation matrix contains rotation and translation parameters, which is used to align one point cloud to another point cloud. The estimated transformation is applied to all matched point pairs and evaluates whether all point pairs remain consistent after the transformation is applied. If the Euclidean distance between two points of a point pair is less than a predefined threshold after the transformation is applied, the feature point pair is considered to be an "inlier". The algorithm counts the number of all inliers and repeats the above steps until the set number of times is met. In each iteration, the algorithm randomly selects different point pairs for transformation estimation. After multiple iterations, the transformation with the largest number of inliers is selected as the final estimated rigid transformation matrix. The source point cloud is aligned to the target point cloud using the final estimated rigid transformation matrix. This step achieves the final registration of the two point clouds, aligning them in the same coordinate system. In general, the function of this module is to use the corresponding point pairs output by the precise correspondence prediction module to find the optimal rigid transformation matrix between the two tooth point clouds, so as to achieve the alignment between the two tooth point clouds and lay the foundation for the subsequent identity recognition function.

[0097] The main reason why the embodiments of the present invention can improve recognition efficiency and ensure recognition accuracy is that the point correspondence process from coarse to fine in the corresponding point establishment stage can quickly and accurately establish matching point pairs between two tooth point clouds, and the two tooth point clouds only need to establish matching point pairs once in the entire registration process. Subsequently, the predictive registration module uses the information of these matching point pairs to calculate the rotation and translation matrices, thereby achieving registration of the two tooth point clouds and finally obtaining the final identity recognition result. However, the registration method involved in the existing optimization-based identity recognition method usually needs to go through multiple rounds of iterative optimization processes, and each round of iterative optimization process requires re-establishing matching point pairs, and there are often mismatched point pairs in the established matching point pairs. Therefore, the recognition efficiency of the existing optimization-based identity recognition method is very poor.

[0098] The total loss function L of the training process of the tooth multi-feature fusion identity recognition backbone network is the coarse corresponding prediction loss L c And the precision corresponding prediction loss L f The weighted sum of:

[0099] L=L c +λL f

[0100] Among them, λ is the weight parameter used to balance the two loss terms. Through this combination, the model can be optimized at both coarse and fine scales to ensure that the model can learn the precise correspondence between point clouds from coarse to fine. At the coarse scale, the model is supervised to learn the reliable correspondence between the key points sampled, while at the fine scale, the model further optimizes the matching accuracy of each point at the single point level. Through the combination of these two parts, the final model can achieve high-precision point cloud registration;

[0101] The coarse corresponding prediction loss L c It is used to supervise the matching results of neighborhood-level key points. Its core idea is to use the local overlap ratio as a weighting scheme to supervise the coarse correspondence. Given a pair of neighborhood-level key points P' X (i') and P' Y (j'), their corresponding neighborhood point grouping results are and calculate Zhongyu The ratio of points with corresponding relationships is:

[0102]

[0103] in, is a known quantity, which represents the real transformation matrix that can completely align the X point cloud input into the network with the Y point cloud, τ P Indicates the distance threshold;

[0104] calculate Zhongyu The ratio of points with corresponding relationships is:

[0105]

[0106] According to the above two formulas, we get the weighted matrix W':

[0107] W'(i',j')=min(r(i',j'),r(j',i')),i'≤n'∧j'≤m'

[0108] Among them, n' and m' represent the total number of neighborhood-level key points of the two point clouds respectively, and each element in W'(i', j') represents the weighted value of the key point pair (i', j'). This weighted matrix is ​​used to reflect the geometric overlap between the key point pairs. The larger the weight, the more reliable the matching key point pair. c When , the cross entropy loss is used to measure the difference between the similarity matrix S' and the weight matrix W' output by the model:

[0109]

[0110] Among them, S'(i',j') is the confidence score of the key point pair (i',j') recorded in the similarity matrix, which indicates the similarity between the corresponding key point pairs. This loss function supervises the model learning to predict more accurate coarse correspondence by minimizing the difference between the point coarse matching result predicted by the model and the weighted matrix W'.

[0111] The precision corresponds to the prediction loss L f Used to supervise fine matching at the single point level, it optimizes based on coarse correspondences:

[0112]

[0113] in, and They represent the two point groups formed by the lth neighborhood level key point matching point pair (i', j'), is a binary matrix, indicating the correct point pair matching relationship in the two point groups. For the matching point pairs, its value is 1, otherwise it is 0. The precision correspondence prediction loss L f The cross entropy loss is also used to measure the confidence matrix of the model output With binary matrix The difference between:

[0114]

[0115] in, Represents the similarity matrix between the points in the two point groups formed by the I-th neighborhood level key point matching point pair (i', j') predicted by the model. This loss function guides the model to learn more accurate matching relationships at the single point level by minimizing the error at the single point level.

[0116] S3. Using the identity information of the 3D tooth point cloud of the registered identity that is best aligned with the 3D tooth point cloud of the identity to be identified (i.e., with the lowest alignment error) as the identity identifier of the 3D tooth point cloud of the identity to be identified.

[0117] like Fig.14 As shown, an embodiment of the present invention further provides an identity recognition system based on 3D tooth point cloud multi-feature fusion, the system comprising:

[0118] The acquisition module 1410 is used to acquire the 3D tooth point cloud of the identity to be identified;

[0119] A registration module 1420 is used to input the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity in the identity database into the trained tooth multi-feature fusion identity recognition backbone network, and output a registration result between the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity;

[0120] The tooth multi-feature fusion identity recognition backbone network includes: a color feature extraction module, a structural feature extraction module, a feature fusion module, a local sampling feature extraction module, a feature aggregation module, a coarse correspondence prediction module, a point cloud decoding module, a fine correspondence prediction module, and a prediction registration module;

[0121] The identification module 1430 is used to use the identity information of the 3D tooth point cloud of the registered identity that is best aligned with the 3D tooth point cloud of the identity to be identified as the identity identifier of the 3D tooth point cloud of the identity to be identified.

[0122] An embodiment of the present invention provides an identity recognition system based on 3D tooth point cloud multi-feature fusion, and its functional structure corresponds to an identity recognition method based on 3D tooth point cloud multi-feature fusion provided by an embodiment of the present invention, which will not be repeated here.

[0123] Fig.15It is a structural diagram of an electronic device 1500 provided in an embodiment of the present invention. The electronic device 1500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 1501 and one or more memories 1502, wherein at least one instruction is stored in the memory 1502, and the at least one instruction is loaded and executed by the processor 1501 to implement the steps of the above-mentioned identity recognition method based on 3D tooth point cloud multi-feature fusion.

[0124] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to complete the above-mentioned identity recognition method based on 3D tooth point cloud multi-feature fusion. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0125] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An identity recognition method based on 3D tooth point cloud multi-feature fusion, characterized in that: The method comprises: S1, collect 3D tooth point cloud of the identity to be identified; S2, input the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity in the identity database into the trained tooth multi-feature fusion identity recognition backbone network, and output the registration result between the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity: The tooth multi-feature fusion identity recognition backbone network includes: a color feature extraction module, a structural feature extraction module, a feature fusion module, a local sampling feature extraction module, a feature aggregation module, a coarse correspondence prediction module, a point cloud decoding module, a fine correspondence prediction module, and a prediction registration module; S3. Using the identity information of the 3D tooth point cloud of the registered identity that is best aligned with the 3D tooth point cloud of the identity to be identified as the identity identifier of the 3D tooth point cloud of the identity to be identified.

2. The method according to claim 1, characterized in that The color feature extraction module is used to extract the color feature information of the 3D tooth point cloud. The specific extraction process includes: Assume that the 3D tooth point cloud is P∈R N×6 , where N represents the number of points in P, and the number 6 means that each point in P has 3D spatial coordinate information and 3D color information; Define any point in P as the center point P center and set the default R color As the radius of the sphere, define other points within this range as neighboring points P neighbor_1 ~P neighbor_n , the center point P center Perform difference calculation with the RGB color channel values ​​of each neighboring point: R diff =R center -R neighbor , G diff =G center -G neighbor , B diff =B center -B neighbor , and divide the calculated color difference into certain intervals, and count R diff , G diff and B diff The results of these intervals represent different levels of color difference, and the sum of the statistical results of all intervals of the three channels constitutes the center point P center The initial color frequency histogram ICH(P center ); Then, taking each neighboring point as the center, repeat the above operation to obtain the color frequency histogram ICH (P neighbor_1 )~ICH(P neighbor_n ); According to each neighboring point and the center point P center The distance d is taken as the weight, and the color frequency histogram results of all neighboring points are added to the center point P. center On the color frequency histogram, the center point P is obtained center Final color frequency histogram The final color frequency histogram FCH(P center ) is converted to an M color dimensional vector, as the center point P center Color feature information; Take all points in P as the center point and perform the above operation to obtain the color feature information of all points in the point cloud, which is recorded as The 3 indicates that the first three columns of information of the feature are the three-dimensional spatial coordinates of the point.

3. The method according to claim 2, characterized in that The structural feature extraction module is used to extract the structural feature information of the 3D tooth point cloud. The specific extraction process includes: Calculate the normal vector of each point in the point cloud based on the three-dimensional spatial coordinate information of each point in the point cloud and its surrounding points; Define any point in P as the center point P center and set the default R struct As the radius of the sphere, define other points within this range as neighboring points P neighbor_1 ~P neighbor_n , describing the center point P center and the geometric features of each neighboring point, where the center point P is described as follows center and P neighbor_1 Geometric structure characteristics of two points: Let vector d = P center -P neighbor_1 , represents the center point P center and P neighbor_1 The connecting vector between two points, the center point P center The normal vector is defined as n center , P neighbor_1 The normal vector is defined as n neighbor_1 , define α to represent n neighbor_1 The angle between and d is calculated as It measures the projection relationship of the normal vector of the neighborhood point in the direction of the target point; φ is defined to represent the center point P center The normal vector n center The angle between d and d is calculated as Reflects the neighborhood point P neighbor_1 Relative to the center point P center The normal vector n center The projection angle of n center and n neighbor_1 The angle between them is calculated as: θ = arccos(n center ·n neighbor_1 ), which measures the degree of alignment of the normal vectors of two points. These parameters together describe the center point P center With neighboring point P neighbor_1 The geometrical characteristics between Construct the center point P center The initial structure frequency histogram ISH(P center ), the initial structure frequency histogram ISH (P center )Total M struct intervals, used to count the center point P respectively center and P neighbor_1 The results of the three parameters α, φ and θ are used as feature information. The value range of α is [-1, 1]. When α = 1, it means that the normal vector of the neighboring point is completely parallel to the normal vector of the center point and has the same direction; when α = -1, it means that they are in opposite directions; when α = 0, it means that the two are perpendicular; the value range of φ is also [-1, 1]. When φ = 1, the normal vector of the center point is consistent with the direction of the line between the center point and the neighboring point; when φ = -1, the normal vector of the center point is opposite to the direction of the line between the center point and the neighboring point; when φ = 0, the normal vector of the center point is perpendicular to the line between the center point and the neighboring point; the value range of θ is [-π, π]. When θ = 0, it means that there is no rotation; when θ = π or θ = -π, it means that it is rotated 180 degrees, reflecting a larger angular offset; For the center point P center The other neighboring points of are then used to construct structural frequency histograms, and then added to the center point P in a weighted manner. center The initial structure frequency histogram FSH(P center ) on the center point P center Final structural frequency histogram The center point P center The structural frequency histogram FSH(P center ) is converted to an M struct dimensional vector, as the center point P center Structural feature information; Take all points in P as the center points and perform the above operations to obtain the structural feature information of all points in the point cloud, which can be recorded as The 3 indicates that the first three columns of information of the feature are the three-dimensional spatial coordinates of the point.

4. The method according to claim 3, characterized in that The feature fusion module extracts M from each point color dimensional color feature information and M struct The 3D structural feature information is spliced ​​with the 3D spatial coordinate information of each point, and the output point cloud contains 3+M color +M struct Fusion results of dimensional feature information The local sampling feature extraction module first performs the feature fusion module output Perform a downsampling operation to obtain a set of sparse points as neighborhood-level key points to reduce the number of points in the point cloud; For each key point, a neighborhood is established with K as the neighborhood radius to search for original points close to these key points in the point cloud before downsampling. Then, according to the distance relationship between the searched original points and the corresponding key points, the features of these original points are aggregated to the corresponding key points as the local feature information F′ of the corresponding key points, and the key points (P′, F′) with local feature information are output, where P′ represents the three-dimensional spatial coordinates of the key points.

5. The method according to claim 4, characterized in that The feature aggregation module relies on the attention mechanism to strengthen and aggregate the local feature information in the neighborhood-level key points, integrate the global and local context information of each point in the input key points, and enable the features of the neighborhood-level key points of the two to-be-registered point clouds to interact, thereby improving the subsequent point cloud registration accuracy. It includes two self-attention modules and one cross-attention module, and the processing process is as follows: First, the features of the input neighborhood-level key points are enhanced by the self-attention module. The self-attention mechanism contained in the self-attention module generates the query matrix Q, the key matrix K and the value matrix V by linearly transforming the input feature matrix F′: Q=F'W Q ,K=F'W K ,V=F'W V Where W Q , W K , W V is a learnable linear transformation matrix; Next, the attention score matrix A is obtained by calculating the dot product of the query matrix and the key matrix and normalizing it: where d k is the size in the key matrix, which is used for normalization to prevent the value from being too large. The attention score matrix A is used to perform weighted summation on the value matrix V to obtain the aggregated feature matrix F″: F″=A·V In this way, the feature of each point not only contains its own information, but also aggregates the global context information related to it; Subsequently, the result is input into the cross attention module to integrate the feature information between key points at different point cloud neighborhood levels. The feature matrices of the two point cloud key points processed by the self-attention module are F″ X and F″ Y , the cross attention module then performs feature aggregation through the following steps: Q X =F″ X W Q ,K Y =F″ Y W K ,V Y =F″ Y W V Calculate F″ X and F″ Y The attention score matrix A between: Using the attention score matrix A X Y pair value matrix V Y Perform weighted summation to obtain the enhanced feature matrix F″′ X : F″′ X =A X Y·V Y Calculate F″ X and F″ Y The cross attention is F″′ Y : F″′ Y =A Y X·V X Through the cross attention module, the features of the two key points are mutually enhanced; After the cross attention module, the feature representation capability is further enhanced through a self-attention calculation. Through the feature aggregation module, the features of each point in the output neighborhood-level key points not only integrate the geometric information of its neighborhood, but also integrate the features from the global context, and also interact with the key points of the point cloud to be registered with it. Finally, the key points of the registered point cloud X after feature aggregation are output. And the point cloud of the identity to be identified after feature aggregation Y Key Points 6. The method according to claim 5, characterized in that The coarse correspondence prediction module obtains the key points of the registered point cloud X after feature aggregation by calculating the inner product of the point cloud feature vector And the point cloud of the identity to be identified after feature aggregation Y Key Points A similarity matrix S′ between them, each element in the similarity matrix S′ represents the similarity between two key points; After obtaining the similarity matrix S′, a normalization algorithm is used to normalize the similarity matrix, and a matching probability matrix is ​​obtained by solving the similarity matrix, wherein the matching probability matrix represents the matching confidence of each pair of points between the key points in the neighborhood of two point clouds; Then, by setting a confidence threshold, key point pairs with higher confidence are screened out from the matching probability matrix as rough corresponding relationship predictions; Finally, a neighborhood-level matching point pair containing multiple rough correspondences is output. This rough correspondence set will be further processed in the subsequent fine correspondence prediction module to obtain more accurate point pair matching results.

7. The method according to claim 6, characterized in that The point cloud decoding module decodes and restores the rough neighborhood key point correspondence to the single point level, and uses the KP convolution decoder to convert the key points of the registered point cloud X after feature aggregation. And the point cloud of the identity to be identified after feature aggregation Y Key Points Decode to the same resolution as the point cloud before downsampling to obtain the decoded point cloud and This process involves a series of convolution operations to gradually restore the spatial distribution and geometric features similar to those of the point cloud before downsampling. The KP convolution decoder will gradually propagate the features of the neighborhood key points to the points after recovery. These propagated features will be combined with the geometric information of the point cloud before downsampling to generate a more fine-grained feature representation. The decoding process also involves a point-to-neighborhood level key point grouping strategy. The point cloud grouping module is used to group each point in the decoded point cloud according to its neighborhood level key points, and the points in the decoded point cloud are assigned to the nearest neighborhood level key points, and the grouping results of the registered point cloud X are output. And the grouping result of the identity point cloud Y to be identified For subsequent refined registration, the subsequent fine correspondence prediction module uses the grouping results obtained from the point cloud decoding module and the key point matching results obtained from the coarse correspondence prediction module to further refine the matching point pairs to obtain single-point level matching point pairs.

8. The method according to claim 7, characterized in that The precise correspondence prediction module includes similarity matrix calculation, normalization processing and confidence screening steps. First, a set of matching point pairs is selected from the neighborhood level matching point pairs, and then the point cloud group constructed by these two points is selected from the grouping results: Grouping results of registered point cloud X and the point cloud of the identity to be identified Y The grouping results Next, the similarity matrix is ​​calculated for these two point clouds. The similarity matrix By calculating the inner product of all point feature vectors between two point cloud groups, each element in the matrix represents the similarity between two points; in the similarity matrix Afterwards, the results are normalized to obtain a matching probability matrix between the two point cloud groups, where the value of each element in the matching probability matrix represents the matching confidence of any two points between the two point cloud groups; then, the matching point pairs between the two point cloud groups are screened out by setting a confidence threshold; finally, all neighborhood-level matching point pairs and their corresponding point cloud grouping results are used to perform the above processing, and all matching point pair screening results are merged together to obtain single-point-level matching point pairs as the output result of the precise correspondence prediction module.

9. The method according to any one of claims 1 to 8, characterized in that: The total loss function L of the training process of the tooth multi-feature fusion identity recognition backbone network is the coarse corresponding prediction loss L c And the precision corresponding prediction loss L f The weighted sum of: L=L c +λL f Among them, λ is the weight parameter used to balance the two loss terms. Through this combination, the model can be optimized at both coarse and fine scales to ensure that the model can learn the precise correspondence between point clouds from coarse to fine. At the coarse scale, the model is supervised to learn the reliable correspondence between the key points sampled, while at the fine scale, the model further optimizes the matching accuracy of each point at the single point level. Through the combination of these two parts, the final model can achieve high-precision point cloud registration; The coarse corresponding prediction loss L c It is used to supervise the matching results of neighborhood-level key points. Its core idea is to use the local overlap ratio as a weighting scheme to supervise the coarse correspondence. Given a pair of neighborhood-level key points P′ X (i′) and P′ Y (j′), and their corresponding neighborhood point grouping results are and calculate Zhongyu The ratio of points with corresponding relationships is: in, is a known quantity, which represents the real transformation matrix that can completely align the X point cloud input into the network with the Y point cloud, τ P Indicates the distance threshold; calculate Zhongyu The ratio of points with corresponding relationships is: According to the above two formulas, the weighted matrix W′ is obtained: W′(i′,j′)=min(r(i′,j′),r(j′,i′)), i′≤n′∧j′≤m′ Among them, n′ and m′ represent the total number of neighborhood-level key points of the two point clouds, respectively. Each element in W′(i′, j′) represents the weighted value of the key point pair (i′, j′). This weighted matrix is ​​used to reflect the geometric overlap between the key point pairs. The larger the weight, the more reliable the matching key point pair. c When , the cross entropy loss is used to measure the difference between the similarity matrix S′ and the weight matrix W′ output by the model: Among them, S′(i′, j′) is the confidence score of the key point pair (i′, j′) recorded in the similarity matrix, which indicates the similarity between the corresponding key point pairs. This loss function supervises the model learning to predict more accurate coarse correspondence by minimizing the difference between the point coarse matching result predicted by the model and the weighted matrix W′. The precision corresponds to the prediction loss L f Used to supervise fine matching at the single point level, it optimizes based on coarse correspondences: in, and They represent the two point groups formed by the lth neighborhood level key point matching point pair (i′, j′), is a binary matrix, indicating the correct point pair matching relationship in the two point groups. For the matching point pairs, its value is 1, otherwise it is 0. The precision correspondence prediction loss L f The cross entropy loss is also used to measure the confidence matrix of the model output With binary matrix The difference between: in, Represents the similarity matrix between the points in the two point groups formed by the lth neighborhood-level key point matching point pair (i′, j′) predicted by the model. This loss function guides the model to learn more accurate matching relationships at the single-point level by minimizing the error at the single-point level.

10. An identity recognition system based on 3D tooth point cloud multi-feature fusion, characterized in that: The system comprises: The acquisition module is used to acquire the 3D tooth point cloud of the identity to be identified; A registration module is used to input the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity in the identity database into the trained tooth multi-feature fusion identity recognition backbone network, and output the registration result between the 3D tooth point cloud of the identity to be identified and the 3D tooth point cloud of each registered identity; The tooth multi-feature fusion identity recognition backbone network includes: a color feature extraction module, a structural feature extraction module, a feature fusion module, a local sampling feature extraction module, a feature aggregation module, a coarse correspondence prediction module, a point cloud decoding module, a fine correspondence prediction module, and a prediction registration module; The identification module is used to use the identity information of the 3D tooth point cloud of the registered identity that is best aligned with the 3D tooth point cloud of the identity to be identified as the identity identifier of the 3D tooth point cloud of the identity to be identified.

Images