An auricular point positioning method and system based on 3DMM

Abstract

The application relates to an ear acupoint positioning method and system based on 3DMM. The method comprises multiple steps such as data labeling, registration and coefficient optimization. First, a real human ear model and a standard human ear model are obtained, and the standard human ear model is labeled with acupoints to generate labeled acupoints; then, through center alignment and size normalization, rigid deformation registration is realized by using ICP point cloud registration; subsequently, a 3DMM model is introduced for non-rigid deformation registration, the chamfer distance between the real human ear model and the standard human ear model is calculated, and the 3DMM coefficient is optimized by using the gradient descent method, so that the registration of the 3DMM is realized; finally, based on the labeled acupoint labels, the nearest neighbor principle is used for acupoint mapping and propagation, and acupoint positioning on the personalized real auricle model is realized. The innovative method provides high accuracy and reliability for personalized three-dimensional ear acupoint positioning and can be used in the fields of traditional Chinese medicine ear diagnosis and the like.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional image processing, and in particular to a method and system for locating auricular acupoints based on 3DMM. Background Technology

[0002] Auricular acupressure is a traditional Chinese medicine therapy that involves applying pressure to specific acupoints on the ear to regulate bodily functions and treat diseases. This method originates from ancient Chinese medicine theory, which views the ear as a miniature model of the human body, containing numerous acupoints corresponding to various organs and systems. However, due to the complexity of the human body and significant individual differences, the two-dimensional method of auricular acupoint location has inherent errors and instability. Therefore, improving the accuracy and efficiency of acupoint location has always been a research hotspot in the field of traditional Chinese medicine diagnosis and treatment.

[0003] With the development of computer science and visual imaging technology, many useful models have emerged to handle real-world image feature recognition tasks. For the task of human ear feature point detection, Ke et al. proposed an automatic feature point recognition scheme based on an Active Appearance (AAM) model. Through analysis of 168 datasets (including data collected using standard equipment and ordinary movement), they demonstrated that the average distance error for 91 feature points is less than 6 pixels, which can replace the recognition work of traditional Chinese medicine auricular acupuncturists. Meanwhile, Sun et al. proposed an automatic auricular acupoint image localization method based on deep learning, and also proposed using a direction normalization module to compensate for detection errors caused by differences between the left and right ears. The validated average mean square error was 0.0514 ± 0.0023, demonstrating superiority over similar works. Although current auricular acupoint localization methods have achieved certain results, some problems and challenges remain. For example, how to improve the registration effect and accuracy of acupoints inside the auricle, and how to apply the localization results to actual diagnosis and treatment. Summary of the Invention

[0004] The first objective of this invention is to address the shortcomings of existing technologies by proposing a 3D auricular acupoint localization method based on 3DMM, which can realize automated acupoint localization of personalized 3D auricular models, thereby further improving the accuracy and efficiency of acupoint localization.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] Step 1: Obtain a real human ear model; Generate a standard human ear model using 3DMM, and annotate the standard human ear model with acupoints to obtain 72 acupoint labels;

[0007] Step 2: Center-align and normalize the dimensions of the real human ear model and the 3DMM standard human ear model;

[0008] Step 3: Perform ICP point cloud registration on the point cloud data of the real human ear model and the 3DMM standard human ear model after Step 2, so that they gradually approach and overlap.

[0009] Step 4: Introduce prior knowledge of the human ear for non-rigid deformation registration. Calculate the chamfer distance between the real human ear point cloud and the ICP point cloud after registration with the 3DMM standard human ear point cloud. Use the gradient descent method to iteratively solve the 3DMM coefficients of the 3DMM standard human ear point cloud to obtain the minimized 3DMM coefficient value, thereby obtaining the 3DMM deformed point cloud that fits the real human ear.

[0010] Step 5: Based on the acupoint labels obtained in Step 1, the coordinate values ​​of the acupoint labels are mapped to the real human ear model using the K nearest neighbor algorithm on the 3DMM deformed point cloud obtained in Step 4. The mapped points are the predicted acupoints. Then, 72 predicted acupoint locations are output from the real human ear model.

[0011] As a preferred method, the method for obtaining the real human ear model in step one is as follows: obtain point cloud data of a real human ear; perform noise reduction and downsampling on the point cloud data to obtain the real human ear model.

[0012] As a preferred option, step three, ICP point cloud registration, specifically includes the following steps:

[0013] S31. Calculate the centroid of the standard human ear point cloud and the real human ear point cloud;

[0014] S32. Convert the standard human ear point cloud and the real human ear point cloud to a centroid coordinate system with their respective centroids as the origin.

[0015] S33. Calculate the optimal transformation parameters, including the optimal rotation R* and the optimal translation t*;

[0016] S34. Repeat steps S31 to S33 until the iteration termination condition is met, and obtain a standard human ear point cloud that is registered with the real human ear point cloud.

[0017] As a preferred embodiment, step four specifically includes the following steps:

[0018] S41. Calculate the chamfer distance in three-dimensional space between the real human ear point cloud and the standard human ear point cloud after ICP point cloud registration;

[0019] S42. Optimize the 3DMM parameters iteratively using gradient descent to reduce the distance between two point clouds. By reaching the minimum value, a 3DMM deformed point cloud that fits the real human ear is obtained.

[0020] As a preferred option, step five specifically involves, based on the acupoint labels obtained in step one, calculating the three-dimensional coordinates of each acupoint on the point cloud data of the 3DMM deformed model obtained in step four, according to the vertex values ​​of each acupoint in the acupoint labels. Using the K nearest neighbor algorithm, the distance between the coordinate values ​​on the current 3DMM deformed point cloud and the K nearest neighbors in the real human ear point cloud is calculated. The average value is used for prediction to find the coordinate point in the real human ear model that is closest to its acupoint coordinates, and this point is defined as the predicted acupoint. After iterative iteration, 72 predicted acupoint locations are output from the real ear model.

[0021] The second objective of this invention is to provide a 3D acupoint localization system based on 3DMM, comprising:

[0022] The model acquisition module is used to acquire real human ear models and standard human ear models;

[0023] The rigid registration module is used to perform center alignment, size normalization, and ICP point cloud registration between a real human ear model and a standard human ear model.

[0024] The non-rigid registration module is used to iteratively solve the coefficients of the rigidly registered model to obtain a 3DMM deformation model that fits the real human ear.

[0025] The acupoint annotation module is used to map and predict acupoints on the surface of a real human ear model and output them.

[0026] A third object of the present invention is to provide an electronic device, including a processor and a memory storing a computer program; wherein the processor executes the computer program to implement the steps of the above-described method.

[0027] A fourth object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the method described above.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] This invention combines rigid deformation ICP point cloud registration with non-rigid deformation 3DMM model registration. It performs center alignment and size normalization between the real human ear and the standard human ear. After overlap, iteratively obtains a 3DMM deformation model that fits the real human ear as closely as possible. Then, according to the nearest neighbor principle, the acupoints on the 3DMM deformation model are mapped to the real human ear model. This enables automated acupoint localization of personalized three-dimensional auricle models, further improving the accuracy and efficiency of acupoint localization, and allowing the localization results to be applied to actual diagnosis and treatment.

[0030] This invention, by locating acupoints in three-dimensional space, can more accurately capture the actual location of acupoints, avoiding the positioning errors caused by factors such as angle and posture changes in two-dimensional images. Furthermore, the three-dimensional acupoint positioning method provides not only horizontal and vertical positional information but also depth information, i.e., the spatial location of the acupoint on the ear. Additionally, three-dimensional models are generally unaffected by lighting, shadows, angles, and postures because they are based on point clouds or stereoscopic information rather than images, improving the robustness of the method. Finally, the three-dimensional acupoint positioning method can typically automate the positioning process more fully, reducing reliance on professionals. Compared to two-dimensional image methods that require manual annotation, the three-dimensional method has a higher degree of automation. Attached Figure Description

[0031] Figure 1 This is a flowchart of the present invention.

[0032] Figure 2 A standardized acupoint labeling diagram.

[0033] Figure 3 A diagram illustrating manually annotated acupoints in 3DMM.

[0034] Figure 4 A map showing the distribution of acupoints for automated localization of the real human ear. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0037] like Figure 1 As shown, this invention provides a method for locating ear acupoints based on 3DMM, specifically including the following steps:

[0038] Preliminary preparations: In real-world scenarios, real human ear point cloud data was collected to create a 3D dataset of real human ears. 3DMM ear models were also collected from publicly available models. Simultaneously, professional software or tools were used to assist in the annotation process, annotating randomly generated standard ear deformation models from 3DMM to obtain... Figure 3 The acupoint labels shown include: annotations of randomly generated standard ear deformation 3D data from the 3DMM model according to the Chinese standard ear acupoint location diagram, using the MeshLab 2022.02 toolbox, to obtain acupoint labels corresponding to a standard deformed ear model. These labels correspond to 72 acupoints including the center, liver, spleen, lung, and kidney points on the Chinese standard ear acupoint location diagram. Figure 2 Each is distinguished by a different vertex, as shown in Table 1 below.

[0039] Table 1. Vertex values ​​corresponding to each acupoint.

[0040]

[0041]

[0042]

[0043]

[0044]

[0045] Rigid registration: The real human ear model and the standard deformation model generated by 3DMM are rigidly registered, including: centering the two ear point cloud data, changing the size of the ear model, and horizontally flipping the ear model to obtain the ear point cloud data after center alignment and size normalization.

[0046] Next, refined ICP rigid registration is performed. The centroids of the standard human ear point cloud and the real human ear point cloud are calculated. The standard human ear point cloud and the real human ear point cloud are transformed into a centroid coordinate system with their respective centroids as the origin. The ICP point cloud registration algorithm is introduced. By iteratively finding the closest point pair between the two sets of point clouds, the current optimal transformation parameters R* and translation t* are obtained in each iteration. The positions of the two sets of point clouds are adjusted according to these correspondences so that they gradually approach each other and eventually overlap as much as possible until the iteration termination condition is met, and a standard human ear model registered with the real human ear point cloud is obtained.

[0047] The ICP point cloud registration algorithm is as follows:

[0048] Using a standard human ear point cloud as the source point cloud and a real human ear point cloud as the target point cloud, assuming the correspondence between points is known, the matrix H is as follows:

[0049]

[0050] in This represents the centroid of the source point cloud. Represents the centroid of the target point cloud. This represents the source point cloud coordinates of the i-th 3D point. Represents the target point cloud coordinates of the i-th 3D point;

[0051] Perform SVD singular value decomposition on matrix H to obtain matrices U and V, and then obtain the optimal rotation R*:

[0052]

[0053] The optimal translation t* is calculated using the following formula:

[0054]

[0055] Non-rigid registration: The real human ear model and the standard deformation model generated by 3DMM are subjected to non-rigid registration based on 3DMM coefficient optimization. Prior knowledge of the human ear is introduced, and non-rigid deformation registration is performed based on the 3DMM deformation model. The chamfer distance d between the real human ear model and the standard deformation model of 3DMM is calculated. CD The gradient descent method is used to iteratively solve the 3DMM coefficients to minimize the 3DMM coefficient values, thereby obtaining a 3DMM deformation model that fits the real human ear as closely as possible, and obtaining its correlation coefficient and function.

[0056] The chamfer distance is calculated using the following formula:

[0057]

[0058] Where d CD (S1, S2) represents the chamfer distance between S1 and S2, where S1 represents the real human ear point cloud and S2 represents the standard human ear point cloud after ICP registration in step three. Let S1 be the sum of the minimum distances from any point x in S1 to S2. The sum of the minimum distances from any point y in S2 to S1 is represented by the double vertical bar symbol ||, which represents the Euclidean Norm of the vector.

[0059] The gradient descent method is as follows:

[0060]

[0061]

[0062] Where g(P) represents the standard human ear point cloud with a 3DMM variation coefficient of P, S i p represents the main component representing the changes in the shape of the human ear. i The i-th variation coefficient in the 3DMM variation coefficient P represents the main component S corresponding to the change in the shape of the human ear. i The weights. The main components of the changes in the shape of the human ear are the main deformation variables of 3DMM on the human ear, which are a set of coefficients given by 3DMM itself.

[0063] Acupoint label propagation: Based on the acupoint labels annotated by experts in the early stage, the nearest neighbor principle is used to map and propagate the acupoints. After point cloud registration, a 3DMM model that approximates the real human ear is obtained. On its point cloud data, the three-dimensional coordinates corresponding to the acupoints are obtained according to the vertex values ​​of the labeled acupoints. According to the nearest neighbor principle, the coordinate point in the real human ear model that is closest to the acupoint coordinates is found and defined as the labeled acupoint, thus realizing the acupoint label mapping and propagation.

[0064] Output: 72 predicted acupoint locations are obtained from a real 3D ear model. The corresponding acupoint locations are highlighted in red and enlarged, such as... Figure 4 As shown, the displayed annotations are exported as an STL file for downstream tasks, realizing automated acupoint positioning of personalized ear models. With the approval of experts, the accuracy of three-dimensional ear acupoint positioning is as high as 78% or above.

[0065] The foregoing description, in conjunction with several preferred embodiments, further explains the technical solution of the present invention. However, the experimental conditions and setting parameters described herein should not be considered as limitations on the basic technical solution of the present invention. Furthermore, the scope of protection of the present invention is not limited to the embodiments described above.

Claims

1. A three-dimensional ear acupoint localization method based on 3DMM, characterized in that, The method includes the following steps: Step 1: Obtain a real human ear model; Generate a standard human ear model using 3DMM, and annotate the standard human ear model with acupoints to obtain 72 acupoint labels; Step 2: Center-align and normalize the dimensions of the real human ear model and the 3DMM standard human ear model; Step 3: Perform ICP point cloud registration on the point cloud data of the real human ear model and the 3DMM standard human ear model after Step 2, so that they gradually approach and overlap. Step 4: Introduce prior knowledge of the human ear for non-rigid deformation registration. Calculate the chamfer distance between the real human ear point cloud and the ICP point cloud after registration with the 3DMM standard human ear point cloud. Use the gradient descent method to iteratively solve the 3DMM coefficients of the 3DMM standard human ear point cloud to obtain the minimized 3DMM coefficient value, thereby obtaining the 3DMM deformed point cloud that fits the real human ear. Step 5: Based on the acupoint labels obtained in Step 1, the coordinate values ​​of the acupoint labels are mapped to the real human ear model using the K nearest neighbor algorithm on the 3DMM deformed point cloud obtained in Step 4. The mapped points are the predicted acupoints. Then, 72 predicted acupoint locations are output from the real human ear model.

2. The three-dimensional ear acupoint localization method based on 3DMM according to claim 1, characterized in that, The method for obtaining a real human ear model in step one is as follows: obtain point cloud data of a real human ear; perform noise reduction and downsampling on the point cloud data to obtain a real human ear model.

3. The three-dimensional ear acupoint localization method based on 3DMM according to claim 1, characterized in that, Step three specifically includes the following steps: S31. Calculate the centroid of the standard human ear point cloud and the real human ear point cloud; S32. Convert the standard human ear point cloud and the real human ear point cloud to a centroid coordinate system with their respective centroids as the origin. S33. Calculate the optimal transformation parameters, including the optimal rotation R* and the optimal translation t*, and then rotate and translate the standard human ear point cloud. S34. Repeat steps S31 to S33 until the iteration termination condition is met, and obtain a standard human ear point cloud that is registered with the real human ear point cloud.

4. The three-dimensional ear acupoint localization method based on 3DMM according to claim 3, characterized in that, Step S33 is as follows: Using a standard human ear point cloud as the source point cloud and a real human ear point cloud as the target point cloud, assuming the correspondence between points is known, the matrix H is as follows: (1) in This represents the centroid of the source point cloud. Represents the centroid of the target point cloud. This represents the source point cloud coordinates of the i-th 3D point. Represents the target point cloud coordinates of the i-th 3D point; Perform SVD singular value decomposition on matrix H to obtain matrices U and V, and then obtain the optimal rotation R*: (2) The optimal translation t* is calculated using the following formula: (3)。 5. The three-dimensional ear acupoint localization method based on 3DMM according to claim 1, characterized in that, Step four specifically includes the following steps: S41. Calculate the chamfer distance in three-dimensional space between the real human ear point cloud and the standard human ear point cloud after ICP point cloud registration; S42. Optimize the 3DMM parameters iteratively using gradient descent to reduce the distance between two point clouds. By reaching the minimum value, a 3DMM deformed point cloud that fits the real human ear is obtained.

6. The three-dimensional ear acupoint localization method based on 3DMM according to claim 5, characterized in that, Step S41 is calculated using the following formula: Where S1 represents the actual human ear point cloud, and S2 represents the standard human ear point cloud after ICP registration in step three. Let S1 be the sum of the minimum distances from any point x in S1 to S2. Let S1 be the sum of the minimum distances from any point y in S2 to S1. Step S42 is calculated using the following formula: (5) (6) Where g(P) represents the standard human ear point cloud with a 3DMM variation coefficient of P, S i p represents the main component representing the changes in the shape of the human ear. i It is the i-th variation coefficient in the 3DMM variation coefficient P.

7. The three-dimensional ear acupoint localization method based on 3DMM according to claim 1, characterized in that, Step five specifically involves using the acupoint labels obtained in step one, and on the point cloud data of the 3DMM deformed model obtained in step four, calculating the three-dimensional coordinates corresponding to each acupoint based on the vertex values ​​of each acupoint in the acupoint labels. The K nearest neighbor algorithm is then used to calculate the distance between the coordinate values ​​on the current 3DMM deformed point cloud and the K nearest neighbors in the real human ear point cloud. The average value is used for prediction to find the coordinate point in the real human ear model that is closest to the acupoint coordinates, and this point is defined as the predicted acupoint. After iterative iteration, 72 predicted acupoint locations are output from the real ear model.

8. A three-dimensional ear acupoint positioning system based on 3DMM as described in any one of claims 1-7, characterized in that, include: The model acquisition module is used to acquire real human ear models and standard human ear models; The rigid registration module is used to perform center alignment, size normalization, and ICP point cloud registration between a real human ear model and a standard human ear model. The non-rigid registration module is used to iteratively solve the coefficients of the rigidly registered model to obtain a 3DMM deformation model that fits the real human ear. The acupoint annotation module is used to map and predict acupoints on the surface of a real human ear model and output them.

9. An electronic device, characterized in that, It includes a processor and a memory storing a computer program; when the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images