A tongue 3D visualization acquisition and reconstruction method, acquisition instrument and medium

Abstract

The present invention discloses a tongue phase three-dimensional visualization acquisition and reconstruction method, an acquisition instrument and a medium, including obtaining three-dimensional original point cloud data of the tongue, pre-processing the original point cloud data based on a curvature-based adaptive filtering method; performing point cloud registration of multi-view point cloud data using a point cloud registration algorithm based on spherical hashing; constructing a triangular mesh model of the tongue surface by gradually adding triangles that meet the normal vector consistency condition based on a greedy triangular reconstruction algorithm based on normal vector consistency; and performing three-dimensional reconstruction of the tongue based on the constructed triangular mesh model, and finally generating a three-dimensional shape of the tongue. The present invention constructs a triangular mesh model of the tongue surface by gradually adding triangles that meet the normal vector consistency condition based on a greedy triangular reconstruction algorithm based on normal vector consistency. It can significantly improve the reconstruction efficiency while maintaining high geometric accuracy, and is suitable for fine reconstruction of complex tongue surfaces.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a tongue three-dimensional visualization acquisition and reconstruction method, an acquisition instrument and a medium. Background Art

[0002] Tongue diagnosis, as a part of TCM visual diagnosis, plays a very important role in diagnosing diseases. Zang-Xiang in TCM refers to the image of the internal organs and the physiological phenomenon of the internal activities of the internal organs on the surface of the body. Tongue image is the external manifestation of the Zang-Xiang directly reflected on the surface of the tongue. Tongue diagnosis diagnoses diseases by observing the tongue coating, tongue quality, tongue shape, and tongue posture. The distribution, color, thickness, moisture, dryness, and greasiness of the tongue coating are closely related to the physiological and pathological information of the human body. Therefore, tongue diagnosis has a high clinical diagnostic value and plays an important role in TCM diagnosis.

[0003] With the development of science and technology, the objectification of tongue diagnosis has received more and more attention. The development of tongue diagnosis instruments has also received widespread attention and has made certain research progress. At present, there are several methods for detecting tongue phases:

[0004] Measurement methods, including tongue impedance measurement, tongue size measurement, tongue saliva content measurement, etc.;

[0005] The graphic method currently mainly uses digital cameras to shoot and integrates image processing technology to obtain tongue images. Among them, tongue imagers based on digital photography of tongue color images are the mainstream of existing tongue imager research. This method can effectively avoid the influence of human factors such as differences in physicians' subjective experience on the diagnosis results, and greatly promotes the objectivity of tongue diagnosis.

[0006] Patent CN1367454A proposes a method for analyzing tongue color, coating color, and coating thickness in traditional Chinese medicine based on a multi-class support vector machine. This method is based on digital camera image acquisition and processing technology, which segments the acquired tongue body image into regions, and counts the number of pixels in the relevant regions to perform quantitative analysis and description of the relevant coating thickness. In addition, East China University of Science and Technology proposed a tongue coating quantitative description and classification method based on the Lab color model. This method uses digital image processing and pattern recognition technology to quantitatively describe and classify different color features, uses fuzzy statistics to establish tongue coating membership functions, and classifies tongue coating through pattern recognition.

[0007] At present, the objective research on tongue coating is mostly based on digital camera shooting and image analysis and processing technology. However, this method is affected by acquisition conditions such as digital camera parameters and individual differences. Therefore, it is difficult to objectively determine the tongue phase from color or texture alone. The shape of the tongue is also an important indicator for diagnosis, but it is currently impossible to obtain detailed information such as texture, protrusions, density, smoothness and three-dimensional morphology on the tongue coating, so it has not been well applied in clinical practice.

[0008] In addition, in recent years, researchers have begun to widely use point cloud registration to achieve three-dimensional reconstruction of the tongue. Point cloud registration is an active field in computer vision research and is now widely used in target recognition, three-dimensional reconstruction, simultaneous localization and mapping (SLAM), intelligent grasping and other fields. Registration technology is a technology that converts multi-view point clouds to the same coordinate system, which mainly includes coarse registration and fine registration. Coarse registration can significantly reduce the translation and rotation errors between point clouds and prevent fine registration from falling into local optimality. Fine registration uses the Euclidean distance between point clouds as the objective function and achieves fine registration by reducing the distance between point pairs. For coarse registration, since local features are rotationally and translationally invariant, feature matching is widely used in coarse registration, and feature matching is a key step in point cloud coarse registration. Sparse point clouds need to extract robust and repeatable key points from point clouds to reduce the amount of calculation. However, the key point extraction algorithms of some existing point cloud registration algorithms mainly study the particularity of key points and ignore the uniqueness, which leads to concentrated distribution of key points and low discrimination. In addition, the research on ICP algorithm and its variants mainly focuses on optimizing the algorithm itself, while ignoring the impact of search algorithm and mismatch on algorithm accuracy and speed. There is still the problem that tongue image acquisition does not meet the clinical objective use.

[0009] In addition, point clouds usually come from scan data from different perspectives or at different times, and need to be accurately registered, that is, these point clouds need to be aligned for subsequent reconstruction. Point cloud registration involves dealing with large amounts of data, noise, and deformation, and requires the design of effective registration algorithms to achieve high-quality registration results. Point cloud data is often affected by noise interference and partial data missing, which affects the accuracy of registration and reconstruction.

[0010] In view of this, it is particularly important to propose a non-invasive tongue coating thickness objective evaluation device and algorithm that can be used in clinical practice. Summary of the invention

[0011] The purpose of the present invention is to overcome the shortcomings of the prior art. To achieve the above purpose, a tongue three-dimensional visualization acquisition and reconstruction method, an acquisition instrument and a medium are used to solve the problems raised in the above background technology.

[0012] The technical solution of the first aspect: a tongue three-dimensional visualization acquisition and reconstruction method, comprising the following steps:

[0013] Obtaining the original three-dimensional point cloud data of the tongue, and preprocessing the original point cloud data based on the curvature adaptive filtering method;

[0014] Use the point cloud registration algorithm based on spherical hashing to perform point cloud registration of multi-view point cloud data;

[0015] A greedy triangulation reconstruction algorithm based on normal vector consistency is used to construct a triangulated mesh model of the tongue surface by gradually adding triangles that meet the normal vector consistency condition.

[0016] The tongue is reconstructed in three dimensions based on the constructed triangular mesh model, and finally the three-dimensional shape of the tongue is generated.

[0017] As a further technical solution of the present invention: the specific steps of obtaining the three-dimensional original point cloud data of the tongue and preprocessing the original point cloud data based on the curvature adaptive filtering method include:

[0018] Step S11, obtaining three-dimensional original point cloud data of the tongue through structured light scanning, laser scanning or binocular stereo vision technology;

[0019] Step S12, a curvature-based adaptive filtering method, adaptively adjusting the weight of the filter according to the size of the surface curvature of each point in the point cloud, so as to retain the details of the high curvature area and perform smoothing in the low curvature area.

[0020] As a further technical solution of the present invention: the specific steps in step S12 include:

[0021] Curvature calculation:

[0022] First, the normal vector at the point to be calculated is estimated, and then a local coordinate system is established with the normal vector as the vertical axis. Finally, a quadratic surface is fitted to the spatial point in the coordinate system, and the curvature value at the point to be calculated using the fitted surface parameters.

[0023] Use KD tree to perform spatial division on point cloud data, obtain K nearest neighbor points of each point, and then use PCA method to construct the covariance matrix of the point;

[0024] For the i-th point in the point cloud data , use its k nearest neighbor points to fit a micro-cutting plane, the normal vector of the micro-cutting plane That is the point The normal vector at the point, the equation of the linear approximation of the micro-tangent plane is:

[0025] ;

[0026] In the formula, are the three-dimensional coordinates of the points approaching the micro-cutting plane; d is the distance from the origin of the coordinate system to the micro-cutting plane;

[0027] use The k nearest neighbors of a point calculate the covariance matrix N, which is expressed as:

[0028] ;

[0029] In the formula, , , ; j is The numbers of all points in the neighborhood of point k, indicating that there are k points adjacent to point i near point i. =1, 2, 3, ···, k; for The average value of the coordinates of each point in the k nearest neighborhood of the matrix N; the eigenvector corresponding to the minimum eigenvalue of the matrix N is the normal vector ;

[0030] For each point , calculate its curvature , for the point cloud data of the tongue surface, set It is a point on the surface of the tongue. is the normal vector of the point, Yes The surrounding area point set, calculate the curvature of the point , and its calculation formula is as follows:

[0031] ;

[0032] At the tip or edge of the tongue, the surface is greatly curved. is higher, while in the flat area of ​​the tongue, is lower;

[0033] Adaptive point cloud filtering:

[0034] Calculate the curvature at each point Finally, an adaptive filtering algorithm is used to smooth the point cloud while retaining the details of the area with large curvature;

[0035] Dynamically adjust the filter search radius based on curvature , the formula is as follows:

[0036] ;

[0037] in, is the base radius, is the curvature threshold, is a small positive number to avoid division by zero, To adjust the parameters, for points with large curvature, the search radius is smaller, and vice versa;

[0038] During the filtering process, points that meet the following conditions are retained:

[0039] ;

[0040] in, Yes The neighborhood point set of is the distance threshold;

[0041] Finally, the tongue point cloud data is obtained, which achieves a balance between detail preservation and noise suppression.

[0042] As a further technical solution of the present invention: the specific steps of performing point cloud registration of multi-view point cloud data using the point cloud registration algorithm based on spherical hashing include:

[0043] The Cartesian coordinates of each point cloud are converted into spherical coordinates through spherical mapping. , expressed as:

[0044] ;

[0045] ;

[0046] in, Indicates longitude, Indicates latitude, Yes The coordinates of

[0047] Then, according to the obtained spherical coordinates, the point cloud is mapped to the constructed spherical grid; the position of each point on the sphere is set to its position in the hash table to process the point cloud data;

[0048] Use a hash function to map spherical coordinates to buckets in a hash table to speed up the matching process. The hash function is defined as:

[0049] ;

[0050] in, and are the number of hash buckets for longitude and latitude respectively;

[0051] The feature descriptors of the point cloud are extracted from the hash table for rough registration of the point cloud. By comparing the point cloud features in the hash bucket, the preliminary alignment position of the point cloud data is found to complete the point cloud registration.

[0052] As a further technical solution of the present invention: the greedy triangulation reconstruction algorithm based on normal vector consistency, by gradually adding triangles that meet the normal vector consistency condition, constructs the triangulated mesh model of the tongue surface. The specific steps include:

[0053] Step S31, obtaining tongue point cloud data: ; and each point All have three-dimensional coordinates ;

[0054] Step S32: Generate a triangular mesh based on the point cloud data, and obtain a final generated surface that can most accurately represent the three-dimensional shape of the tongue. The specific steps are:

[0055] Step S321: Select three initial points from the point cloud data , forming a triangle ;

[0056] Step S322: In each iteration, select a point from the point cloud that is not added to any triangle. , and find the nearest generated edge , generate a new triangle ;

[0057] Step S323: For each generated triangle , calculate its shape quality, if the quality does not meet the preset standard, discard the triangle and reselect the point Generate new triangles;

[0058] Step S324, repeating step S322 and step S323 until all points are included in at least one triangle or a termination condition is met;

[0059] Step S33, after the point cloud is registered, three-dimensional reconstruction is performed using a greedy projection triangulation algorithm;

[0060] Step S34: adaptively adjust key parameters of the greedy projection triangulation algorithm based on normal vector consistency.

[0061] As a further technical solution of the present invention: the specific steps in step S33 include:

[0062] First, define the point set Search radius To construct a neighborhood point set , the formula is:

[0063] ;

[0064] in, yes Point around A point j in the set;

[0065] After determining the neighborhood, the triangular surface is constructed using the following conditions:

[0066] ;

[0067] in, It is composed of three points in the neighborhood The interior angles of the triangle formed by and is the preset angle threshold;

[0068] The mesh generated in the triangulation process is represented as a combination of a point set V and a face set F, and the formula is:

[0069] ;

[0070] Among them, V is the reconstructed vertex set and F is the corresponding patch index.

[0071] As a further technical solution of the present invention: the specific steps in step S34 include:

[0072] For each point in the point cloud data , its normal vector can be expressed as ; In a local area, the consistency of the normal vector is quantified by calculating the variance of the normal vector;

[0073] First, calculate the average value of all normal vectors in the local area , the formula is:

[0074] ;

[0075] Where N is the number of points in the local area;

[0076] Then calculate the angular deviation between each normal vector and the average normal vector and find its variance , the formula is:

[0077] ;

[0078] in, is the dot product between the normal vector and the mean normal vector, and are the moduli of the normal vector and the mean normal vector respectively;

[0079] Based on the consistency of the normal vector, that is, the variance , adaptively adjust the key parameters of the greedy triangle projection, including the minimum angle , maximum angle , and the maximum surface angle ;

[0080] Calculate the minimum angle , in high curvature areas with low normal vector consistency, more detailed triangles are generated through the minimum angle, and the adjustment formula is:

[0081] ;

[0082] in, is the default minimum angle, c is the adjustment factor;

[0083] Calculate the maximum angle , in areas with low normal vector consistency, larger surface angle deviations are allowed to generate larger triangles, thereby reducing redundant mesh refinement. The adjustment formula is:

[0084] ;

[0085] in, is the default maximum angle, and d is the adjustment factor;

[0086] Calculate the maximum surface angle , in areas with low normal vector consistency, a larger surface angle deviation is allowed so that the triangle can better adapt to surface changes. The adjustment formula is:

[0087] ;

[0088] in, is the default maximum surface angle and e is the adjustment factor.

[0089] The technical solution of the second aspect: a collection instrument, using a tongue phase three-dimensional visualization collection and reconstruction method as described in any one of the above items, the collection instrument comprising:

[0090] Base;

[0091] A movable seat disposed at the top of the rear end of the base;

[0092] A main collector disposed at the top of the front end of the movable seat;

[0093] A mounting frame disposed at the bottom of the main collector for adjusting the height; and

[0094] A head frame arranged at the top of the front end of the base;

[0095] The movable seat is adjusted to reciprocate along the extension direction on the base, the distance from the main collector to the head frame is adjusted, and the main collector and the head frame are adjusted to the same relative height in cooperation with the mounting frame.

[0096] As a further technical solution of the present invention: a turntable for horizontally rotating the main collector is arranged at the bottom of the main collector.

[0097] The technical solution of the third aspect: a storage medium storing processor-executable instructions, wherein the processor-executable instructions are used to implement a tongue three-dimensional visualization acquisition and reconstruction method as described in any of the above items when executed by the processor.

[0098] Compared with the prior art, the present invention has the following technical effects:

[0099] By adopting the above technical solution, the main collector and the head frame are relatively arranged so that the main collector can shoot the tongue at a specified distance and angle, and can collect multi-angle tongue phase information. By storing multiple tongue phase information, and then using dense point cloud reconstruction technology to analyze the three-dimensional tongue data, and then through the quantitative design of three-dimensional morphological features, a three-dimensional graph of the tongue morphology can be drawn, and fine information such as texture, protrusions, density, smoothness and three-dimensional morphology on the tongue coating can be obtained. Finally, by manually measuring the quantitative information of the three-dimensional features of the standard parts, comparing the data values ​​measured by the three-dimensional scanner, and analyzing the effectiveness of the three-dimensional quantitative features, it is possible to accurately and objectively obtain tongue-related information. A non-invasive objective evaluation of tongue coating thickness is achieved.

[0100] In addition, the use of a main collector and a sub-collector to collect tongue phase information can further improve the accuracy of the collected information and further increase the collection range.

[0101] Through point cloud registration technology, data from different sources can be integrated and registered to maintain data integrity and consistency. This helps to comprehensively analyze and compare tongue data from different individuals or at different time points. In addition, during the algorithm development stage, the point cloud data set of the tongue can be easily obtained to complete the algorithm development work. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] The specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings:

[0103] Figure 1 A schematic diagram of the steps of the visualization acquisition method according to the embodiment disclosed in the present application;

[0104] Figure 2 This is a schematic diagram of point cloud data before and after filtering of an embodiment disclosed in this application;

[0105] Figure 3 This is a schematic diagram of a point cloud after registration according to an embodiment disclosed in this application;

[0106] Figure 4 The three-dimensional reconstruction result of the embodiment disclosed in this application;

[0107] Figure 5 This is a schematic diagram of the first structure of the tongue phase collecting device according to the disclosed embodiment of the present application;

[0108] Figure 6 This is a schematic diagram of the second structure of the tongue phase collecting device according to the disclosed embodiment of the present application;

[0109] Figure 7 This is a schematic diagram of the third structure of the tongue phase collecting device according to the disclosed embodiment of the present application;

[0110] Figure 8This is a schematic diagram of the structure of the other side of the tongue phase collector according to the embodiment disclosed in this application.

[0111] In the figure: 1. Base; 2. Movable seat; 3. Main collector; 3.1. Mounting frame; 3.2. Turntable; 3.3. Horizontal adjustment frame; 4. Head frame; 4.1. Jaw drag; 4.2. Height adjustment frame; 5. Auxiliary frame; 5.1. Waist groove; 6. Auxiliary collector; 6.1. Mounting rod; 7. Screw rod; 8. Motor; 9. Light rod; 10. Fill light; 11. Screen. DETAILED DESCRIPTION

[0112] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0113] Please refer to Figure 1 In an embodiment of the present invention, a tongue phase three-dimensional visualization acquisition and reconstruction method includes the following steps:

[0114] Step S1, obtaining the original point cloud data of the tongue in three dimensions, and preprocessing the original point cloud data based on the curvature adaptive filtering method, the specific steps include:

[0115] Step S11, obtaining three-dimensional original point cloud data of the tongue through structured light scanning, laser scanning or binocular stereo vision technology;

[0116] In this embodiment, the point cloud data is usually obtained by structured light scanning, laser scanning or binocular stereo vision technology. Although the scanning accuracy is gradually improved, the original point cloud data obtained often contains noise and outliers, so it is very important to process the point cloud before 3D reconstruction.

[0117] In the preprocessing of 3D point cloud data, curvature is an important geometric feature that reflects the local morphology of the point cloud surface. The curvature-based adaptive filtering method adjusts the weight of the filter to retain the details of high curvature areas (such as edges and corners) while smoothing in low curvature areas to reduce noise and improve the quality of the reconstructed surface.

[0118] In three-dimensional tongue reconstruction, the geometric shape of the tongue surface is complex, especially at the tip and edge of the tongue, where the curvature is large and has obvious detail features. However, the tongue surface may also be affected by noise, such as data deviation caused by scanning equipment errors or slight patient movements. The curvature-based adaptive filtering method can effectively deal with these problems, while retaining the details of the tongue surface, suppressing noise, thereby improving the accuracy of the reconstruction results.

[0119] Step S12, a curvature-based adaptive filtering method, adaptively adjusting the weight of the filter according to the surface curvature of each point in the point cloud to retain the details of the high curvature area and perform smoothing in the low curvature area. The specific steps include:

[0120] Curvature calculation:

[0121] In this embodiment, the curvature calculation usually adopts the local surface fitting method;

[0122] First, the normal vector at the point to be calculated is estimated, and then a local coordinate system is established with the normal vector as the vertical axis. Finally, a quadratic surface is fitted to the spatial point in the coordinate system, and the curvature value at the point to be calculated using the fitted surface parameters.

[0123] Among them, the normal vector is the most basic feature descriptor in point cloud processing, and its value has a very important influence on the characterization of the morphology of the object being measured.

[0124] Use KD tree to perform spatial division on point cloud data, obtain K nearest neighbor points of each point, and then use PCA method to construct the covariance matrix of the point;

[0125] For the i-th point in the point cloud data , use its k nearest neighbor points to fit a micro-cutting plane, the normal vector of the micro-cutting plane That is the point The normal vector at the point, the equation of the linear approximation of the micro-tangent plane is:

[0126] ;

[0127] In the formula, are the three-dimensional coordinates of the points approaching the micro-cutting plane; d is the distance from the origin of the coordinate system to the micro-cutting plane;

[0128] use The k nearest neighbors of a point calculate the covariance matrix N, which is expressed as:

[0129] ;

[0130] In the formula, , , ; j is The numbers of all points in the neighborhood of point k, indicating that there are k points adjacent to point i near point i. =1, 2, 3, ···, k; for The average value of the coordinates of each point in the k nearest neighborhood of the matrix N; the eigenvector corresponding to the minimum eigenvalue of the matrix N is the normal vector ;

[0131] Specifically, the core idea of ​​the filtering method is to adaptively adjust the filter weight according to the surface curvature of each point in the point cloud, so as to achieve a balance between noise suppression and detail preservation.

[0132] For each point , calculate its curvature , for the point cloud data of the tongue surface, set It is a point on the surface of the tongue. is the normal vector of the point, Yes The surrounding area point set, calculate the curvature of the point , and its calculation formula is as follows:

[0133] ;

[0134] At the tip or edge of the tongue, the surface is greatly curved. is higher, while in the flat area of ​​the tongue, is lower; through this calculation, we can identify areas on the tongue surface with important geometric features.

[0135] Adaptive point cloud filtering:

[0136] Calculate the curvature at each point Finally, an adaptive filtering algorithm is used to smooth the point cloud while retaining the details of the area with large curvature;

[0137] Dynamically adjust the filter search radius based on curvature , the formula is as follows:

[0138] ;

[0139] in, is the base radius, is the curvature threshold, is a small positive number to avoid division by zero, To adjust the parameters, for points with large curvature, the search radius is smaller, and vice versa;

[0140] During the filtering process, points that meet the following conditions are retained:

[0141] ;

[0142] in, Yes The neighborhood point set of is the distance threshold; this condition ensures that only points within the neighborhood whose distance is less than point to further reduce noise.

[0143] In this embodiment, in the flat areas of the tongue surface, due to the small curvature, the filter tends to perform stronger smoothing on these areas to effectively remove potential noise. In areas with large curvatures such as the tip or edge of the tongue, due to the significant difference in curvature, the filter tends to retain the subtle features of these areas, thereby avoiding over-smoothing and ensuring the integrity of key geometric details.

[0144] like Figure 2 As shown, the figure is a schematic diagram of point cloud data before and after filtering;

[0145] Finally, the tongue point cloud data is obtained, which achieves a balance between detail preservation and noise suppression.

[0146] Specifically, through the above-mentioned curvature-based adaptive filtering method, the final tongue point cloud data achieves a good balance between detail preservation and noise suppression. The geometric features of key areas such as the tip of the tongue and the edge of the tongue are preserved to ensure the accuracy of the reconstructed model in these areas, while in the flat area of ​​the tongue, the noise is effectively suppressed, making the reconstructed surface smoother and more natural. This processing method is particularly important for medical applications such as tongue diagnosis, because the morphological changes in areas such as the tip of the tongue and the edge of the tongue are often closely related to health conditions, and accurately reconstructing the three-dimensional morphology of these areas is of great significance for clinical diagnosis.

[0147] Step S2: performing point cloud registration of multi-view point cloud data using a point cloud registration algorithm based on spherical hashing. The specific steps include:

[0148] Point cloud registration is a technology that converts multiple point clouds from different perspectives into the same coordinate system. It is widely used in 3D reconstruction, medical image analysis, simultaneous localization and map reconstruction, and landscape segmentation. Point cloud registration is a key task in computer vision and 3D data processing, especially when processing tongue point cloud data, accurate alignment is crucial.

[0149] In this embodiment, a point cloud registration method based on spherical hashing is used. This method improves the efficiency and accuracy of registration through spherical mapping and hash table technology.

[0150] Spherical Mapping:

[0151] The Cartesian coordinates of each point cloud are converted into spherical coordinates through spherical mapping. , expressed as:

[0152] ;

[0153] ;

[0154] in, Indicates longitude, Indicates latitude, Yes The coordinates of

[0155] Spherical mesh construction:

[0156] Then, according to the obtained spherical coordinates, the point cloud is mapped to the constructed spherical grid; the position of each point on the sphere is set to its position in the hash table to process the point cloud data;

[0157] Hash table construction and feature matching:

[0158] Use a hash function to map spherical coordinates to buckets in a hash table to speed up the matching process. The hash function is defined as:

[0159] ;

[0160] in, and are the number of hash buckets for longitude and latitude respectively; in this way, similar spherical coordinates will be mapped to the same hash bucket, thus speeding up the matching process.

[0161] The feature descriptors of the point cloud are extracted from the hash table for rough registration of the point cloud. By comparing the point cloud features in the hash bucket, the preliminary alignment position of the point cloud data is found to complete the point cloud registration.

[0162] like Figure 3 As shown, the figure is a schematic diagram of the point cloud after registration;

[0163] Step S3: A greedy triangulation reconstruction algorithm based on normal vector consistency is used to construct a triangulated mesh model of the tongue surface by gradually adding triangles that meet the normal vector consistency condition. The specific steps include:

[0164] The basic principle of greedy triangle reconstruction:

[0165] The greedy triangulation reconstruction algorithm is a 3D surface reconstruction method based on local geometric information, which is often used to generate mesh models from point cloud data. This method forms triangles by gradually connecting points in the point cloud to construct a complete surface. In the 3D reconstruction of the tongue, greedy triangulation reconstruction can effectively capture the geometric features of the tongue surface and generate a high-quality mesh model. When solving the problem, the greedy algorithm does not consider the overall global optimal point of view, but pays more attention to the local optimal solution in a certain sense. In the context of 3D point cloud surface reconstruction, the optimal triangulation network of the whole can be approximated by constructing the optimal triangulation network of the local area, because the triangulation network construction of each local area is only related to the local features of the 3D point cloud and will not affect the triangulation network construction of other areas, so the greedy algorithm is used to generate the triangulation mesh of the surface.

[0166] Step S31, obtaining tongue point cloud data: ; and each point All have three-dimensional coordinates ;

[0167] In this embodiment, the goal of greedy triangulation reconstruction is to generate a triangular mesh based on these points so that the ultimately generated surface can most accurately represent the three-dimensional shape of the tongue.

[0168] Step S32: Generate a triangular mesh based on the point cloud data, and obtain a final generated surface that can most accurately represent the three-dimensional shape of the tongue. The specific steps are:

[0169] Step S321: Select three initial points from the point cloud data , forming a triangle ;

[0170] Step S322: In each iteration, select a point from the point cloud that is not added to any triangle. , and find the nearest generated edge , generate a new triangle ;

[0171] Step S323: For each generated triangle , such as area, interior angle, etc.; calculate its shape quality. If the quality does not meet the preset standard, the triangle is discarded and the points are reselected. Generate new triangles;

[0172] Step S324, repeating step S322 and step S323 until all points are included in at least one triangle or a termination condition is met;

[0173] Step S33: After the point cloud is registered, a greedy projection triangulation algorithm is used to perform three-dimensional reconstruction. The specific steps include:

[0174] The Greedy Projection Triangulation (GP3) algorithm is used for 3D reconstruction.

[0175] First, define the search radius of the point set To construct a neighborhood point set , the formula is:

[0176] ;

[0177] in, yes Point around A point j in the set;

[0178] After determining the neighborhood, the triangular surface is constructed using the following conditions:

[0179] ;

[0180] in, It is composed of three points in the neighborhood The interior angles of the triangle formed by and A preset angle threshold; this ensures the quality of the generated triangles and avoids generating overly distorted patches.

[0181] The mesh generated in the triangulation process is represented as a combination of a point set V and a face set F, and the formula is:

[0182] ;

[0183] Among them, V is the reconstructed vertex set and F is the corresponding patch index.

[0184] Step S34: Adaptively adjust key parameters of the greedy projection triangulation algorithm based on normal vector consistency. The specific steps include:

[0185] Traditional greedy triangle projection methods usually use fixed triangulation parameters, which may not perform well when dealing with surfaces of different complexity. Various improved algorithms are currently being used to improve the quality of triangulated meshes. For example, curvature-based adaptive triangulation methods can dynamically adjust parameters according to surface curvature, but these methods may not be robust enough when dealing with high-noise data. In contrast, the normal vector consistency in this embodiment, as a direct indicator of surface smoothness, can more accurately reflect the local geometric features of the point cloud. Therefore, the adaptive method based on normal vector consistency has higher flexibility and robustness.

[0186] For each point in the point cloud data , its normal vector can be expressed as ; In a local area, the consistency of the normal vector is quantified by calculating the variance of the normal vector;

[0187] First, calculate the average value of all normal vectors in the local area , the formula is:

[0188] ;

[0189] Where N is the number of points in the local area;

[0190] Then calculate the angular deviation between each normal vector and the average normal vector and find its variance , the formula is:

[0191] ;

[0192] in, is the dot product between the normal vector and the mean normal vector, and are the moduli of the normal vector and the mean normal vector respectively;

[0193] Based on the consistency of the normal vector, that is, the variance , adaptively adjust the key parameters of the greedy triangle projection, including the minimum angle , maximum angle , and the maximum surface angle ;

[0194] Calculate the minimum angle , in high curvature areas with low normal vector consistency, more detailed triangles are generated through the minimum angle, and the adjustment formula is:

[0195] ;

[0196] in, is the default minimum angle, c is the adjustment factor;

[0197] Calculate the maximum angle , in areas with low normal vector consistency, larger surface angle deviations are allowed to generate larger triangles, thereby reducing redundant mesh refinement. The adjustment formula is:

[0198] ;

[0199] in, is the default maximum angle, and d is the adjustment factor;

[0200] Calculate the maximum surface angle , in areas with low normal vector consistency, a larger surface angle deviation is allowed so that the triangle can better adapt to surface changes. The adjustment formula is:

[0201] ;

[0202] in, is the default maximum surface angle and e is the adjustment factor.

[0203] like Figure 4 As shown, the figure shows the three-dimensional reconstruction result;

[0204] Step S4: reconstruct the tongue in three dimensions based on the constructed triangular mesh model, and finally generate the three-dimensional shape of the tongue.

[0205] Please refer to Figure 5 In an embodiment of the present invention, a collection instrument adopts a tongue phase three-dimensional visualization collection and reconstruction method as described in any one of the above items, and the collection instrument includes:

[0206] A base 1, a movable seat 2, a main collector 3 for collecting tongue phases and a head frame 4 for head positioning.

[0207] Specifically, a movable seat 2 is arranged at the top of the rear end of the base 1. In this embodiment, the movable seat 2 can move forward and backward along the base 1. The main collector 3 is installed at the top of the front end of the movable seat 2, so that the forward and backward movement of the main collector 3 can be adjusted.

[0208] In order to adjust the height of the main collector 3 to ensure that the height is flush with the head, a mounting frame 3.1 is provided at the bottom thereof. The mounting frame 3.1 in this embodiment adopts a rod structure with two ends intersecting each other, wherein the other end of one rod is respectively hinged at the bottom of the main collector 3, and the other end of the other rod is provided with a base, which can be placed on the movable seat 2, and then the height of the main collector 3 can be conveniently adjusted through the structure of the two rods being hinged, and it is simple and light. In addition, the present application is not limited to the above two-rod structure, and can also be other lifting structures, such as a bolt sub-transmission structure, a rack and pinion structure, or a worm gear mechanism.

[0209] In order to realize the corresponding arrangement of the head and the main collector 3 and ensure the shooting distance, viewing angle and shooting stability of the main collector 3, a head frame 4 is arranged at the top of the front end of the base 1, and a chin trolley 4.1 is arranged at the top of the collection frame 4, and a height adjustment frame 4.2 is arranged at the bottom of the chin trolley 4.1, which can adjust the height of the chin trolley 4.1 according to people of different heights, and is another height adjustment method in addition to adjusting the main collector 3, thereby improving the convenience of use. In this embodiment, the height adjustment mechanism of the adjustment frame 4.2 is a spiral lifting structure, which can be an electric or manual adjustment form.

[0210] When in use, first, adjust the height of the main collector 3 to a reasonable range. During collection, the person being collected sits at the front end of the base 1 and faces the rear end of the base 1, places the chin on the jaw drag 4.1, and adjusts the height according to the collector's command. At the same time, the collector adjusts the distance from the main collector 3 to the head frame 4. After determining the position, the person being collected sticks out the tongue and maintains the tongue position. Finally, the main collector 3 focuses and shoots, and performs video collection. In order to achieve accurate tongue acquisition, the tongue can be photographed from multiple angles. The specific strategy is to form an angle of about 15 degrees between the main collector 3 and the tongue, and shoot obliquely to achieve the purpose of covering a larger area of ​​the tongue. At the same time, during the shooting process, the height of the lifting frame will be appropriately adjusted to capture the side of the tongue to the greatest extent, so as to fully construct relatively complete tongue data.

[0211] On this basis, in order to further improve the adjustment convenience of the main collector 3 and achieve efficient and accurate angle shooting, the present application also provides a second structural form of the main collector, the specific structure is as follows:

[0212] like Figure 6As shown, the figure is a schematic diagram of the second structure of the tongue phase collection instrument;

[0213] A turntable 3.2 is installed at the bottom of the main collector 3, which can drive the main collector 3 to rotate horizontally.

[0214] On this basis, a lateral adjustment frame 3.2 is installed at the bottom of the turntable 3.2, which can drive the main collector 3 to move laterally along the movable seat 2. In combination with the front and rear movement of the movable seat 2 and the horizontal rotation of the movable seat 2, as well as the height adjustment, the collection direction of the main collector 3 can be adjusted more flexibly and in a wide range.

[0215] It should be noted that in this embodiment, the turntable 3.2 is driven by the control motor (stepping motor or servo motor) to perform angular displacement, and the rotation angle can be accurately adjusted. The lateral movement of the movable seat 2 is driven by the control motor through the lead screw, and the main collector 3 can also be accurately displaced. Similarly, the height of the main collector 3 can also be accurately controlled by the control motor.

[0216] On this basis, the present application also provides a third structural form of the main collector, which is as follows:

[0217] like Figure 7 As shown, the figure is a schematic diagram of the third structure of the tongue phase collection instrument;

[0218] It also includes an auxiliary frame 5, on which a plurality of sub-collectors 6 are mounted, and the plurality of sub-collectors 6 are arranged at different positions of the auxiliary frame 5 and can be arranged in different directions of the head frame 4. The strategy is to shoot at a viewing angle position that the main collector 3 cannot reach, so as to play a supplementary shooting role and make the range of the collected image larger. On the other hand, the sub-collector 6 can also obtain the viewing angle image that intersects with the main collector 3, so that comparative data can be obtained for data comparison analysis, and the single collector collection error can be eliminated to a certain extent, so that the database is more complete and the collected data is more accurate.

[0219] A plurality of waist grooves 5.1 are provided on the auxiliary frame 5, and a mounting rod 6.1 is provided at one end of the auxiliary collector 6, which can be inserted into the waist groove 5.1 and can be adjusted by a double nut. Therefore, the waist grooves 5.1 on the auxiliary frame 5 can realize convenient adjustment of a plurality of auxiliary collectors 6, so that the installation position of the auxiliary collector 6 is more extensive and the adjustment convenience is improved. In theory, the arrangement of several auxiliary collectors 6 can be realized.

[0220] like Figure 8 As shown, the figure is a schematic diagram of the structure of the other side of the tongue phase collector;

[0221] Among them, the auxiliary frame 5 in this embodiment is an arched frame structure, and the waist groove 5.1 is respectively opened at the top and side ends of the arched frame structure. In this way, while ensuring the strength of the auxiliary frame 5, the height dimension of the auxiliary collector 6 can be arranged, and the auxiliary collector 6 can be set lower than the auxiliary collector 6. In this way, the bottom of the tongue can be photographed at different angles to obtain a more complete image.

[0222] Based on the above-mentioned collector structure, in order to realize the forward and backward movement of the auxiliary frame 5, optical rods 9 are arranged on both sides of the base 1 in the forward and backward directions, and the auxiliary frame 5 is slidably installed on the optical rods 9, and the auxiliary frame 5 is used to support the auxiliary frame 5. In addition, a screw rod 7 is arranged on the other side of the base 1, and a motor 8 is arranged at the end thereof. The nut on the auxiliary frame 5 is connected with the screw rod 7. The rotation of the motor can drive the auxiliary frame 5 to move forward and backward, and then the sub-collector 6 on the auxiliary frame 5 can be driven to move forward and backward to adjust the shooting angle.

[0223] In addition, in order to improve the shooting light and avoid interference from surrounding light, a fill light 10 is provided at the end of the auxiliary frame 5 facing the head frame 4 to achieve full exposure of the tongue and make the texture and color of the tongue clearer.

[0224] In the present application, in order to improve the operation of the personnel and command the position control of the personnel being collected, a screen 11 is provided at the rear end of the movable seat 2 to display the collected tongue phase information.

[0225] In this embodiment, the main collector 3 and the auxiliary collector 6 adopt the Zividone+ model, and adopt different specifications of large, medium and small to obtain a better collection effect.

[0226] The technical solution of the third aspect: a storage medium storing processor-executable instructions, wherein the processor-executable instructions are used to implement a tongue three-dimensional visualization acquisition and reconstruction method as described in any of the above items when executed by the processor.

[0227] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and all should be included within the scope of protection of the present invention.

Claims

1. A tongue 3D visualization acquisition and reconstruction method, characterized in that: The following steps are involved: Obtaining the original three-dimensional point cloud data of the tongue, and preprocessing the original point cloud data based on the curvature adaptive filtering method; The point cloud registration algorithm based on spherical hashing is used to perform point cloud registration of multi-view point cloud data. The specific steps include: The Cartesian coordinates of each point cloud are converted into spherical coordinates through spherical mapping. , expressed as: ； ； in, Indicates longitude, Indicates latitude, Yes The coordinates of Then, according to the obtained spherical coordinates, the point cloud is mapped to the constructed spherical grid; the position of each point on the sphere is set to its position in the hash table to process the point cloud data; Use a hash function to map spherical coordinates to buckets in a hash table to speed up the matching process. The hash function is defined as: ； in, and are the number of hash buckets for longitude and latitude respectively; Extract the feature descriptor of the point cloud from the hash table for rough registration of the point cloud. By comparing the point cloud features in the hash bucket, find the preliminary alignment position of the point cloud data and complete the point cloud registration. A greedy triangulation reconstruction algorithm based on normal vector consistency is used to construct a triangulated mesh model of the tongue surface by gradually adding triangles that meet the normal vector consistency condition. The tongue is reconstructed in three dimensions based on the constructed triangular mesh model, and finally the three-dimensional shape of the tongue is generated.

2. According to claim 1, a tongue 3D visualization acquisition and reconstruction method is characterized in that: The specific steps of obtaining the original point cloud data of the tongue in three dimensions and preprocessing the original point cloud data based on the curvature adaptive filtering method include: Step S11, obtaining three-dimensional original point cloud data of the tongue through structured light scanning, laser scanning or binocular stereo vision technology; Step S12, a curvature-based adaptive filtering method, adaptively adjusting the weight of the filter according to the size of the surface curvature of each point in the point cloud, so as to retain the details of the high curvature area and perform smoothing in the low curvature area.

3. According to claim 2, a tongue 3D visualization acquisition and reconstruction method is characterized in that: The specific steps in step S12 include: Curvature calculation: First, the normal vector at the point to be calculated is estimated, and then a local coordinate system is established with the normal vector as the vertical axis. Finally, a quadratic surface is fitted to the spatial point in the coordinate system, and the curvature value at the point to be calculated using the fitted surface parameters. Use KD tree to perform spatial division on point cloud data, obtain K nearest neighbor points of each point, and then use PCA method to construct the covariance matrix of the point; For the i-th point in the point cloud data , use its k nearest neighbor points to fit a micro-cutting plane, the normal vector of the micro-cutting plane That is the point The normal vector at the point, the equation of the linear approximation of the micro-tangent plane is: ； In the formula, are the three-dimensional coordinates of the points approaching the micro-cutting plane; d is the distance from the origin of the coordinate system to the micro-cutting plane; use The k nearest neighbors of a point calculate the covariance matrix N, which is expressed as: ； In the formula, , , ; j is The numbers of all points in the neighborhood of point k, indicating that there are k points adjacent to point i near point i. =1, 2, 3, ···, k; for The average value of the coordinates of each point in the k nearest neighborhood of the matrix N; the eigenvector corresponding to the minimum eigenvalue of the matrix N is the normal vector ; For each point , calculate its curvature , for the point cloud data of the tongue surface, set It is a point on the surface of the tongue. is the normal vector of the point, Yes The surrounding area point set, calculate the curvature of the point , and its calculation formula is as follows: ； At the tip or edge of the tongue, the surface is greatly curved. is higher, while in the flat area of ​​the tongue, is lower; Adaptive point cloud filtering: Calculate the curvature at each point Finally, an adaptive filtering algorithm is used to smooth the point cloud while retaining the details of the area with large curvature; Dynamically adjust the filter search radius based on curvature , the formula is as follows: ； in, is the base radius, is the curvature threshold, is a small positive number to avoid division by zero, To adjust the parameters, for points with large curvature, the search radius is smaller, and vice versa; During the filtering process, points that meet the following conditions are retained: ； in, Yes The neighborhood point set of is the distance threshold; Finally, the tongue point cloud data is obtained, which achieves a balance between detail preservation and noise suppression.

4. According to claim 1, a tongue 3D visualization acquisition and reconstruction method is characterized in that: The greedy triangulation reconstruction algorithm based on normal vector consistency constructs the triangulated mesh model of the tongue surface by gradually adding triangles that meet the normal vector consistency condition. The specific steps include: Step S31, obtaining tongue point cloud data: ; and each point All have three-dimensional coordinates ; Step S32: Generate a triangular mesh based on the point cloud data, and obtain a final generated surface that can most accurately represent the three-dimensional shape of the tongue. The specific steps are: Step S321: Select three initial points from the point cloud data , forming a triangle ; Step S322: In each iteration, select a point from the point cloud that is not added to any triangle. , and find the nearest generated edge , generate a new triangle ; Step S323: For each generated triangle , calculate its shape quality, if the quality does not meet the preset standard, discard the triangle and reselect the point Generate new triangles; Step S324, repeating step S322 and step S323 until all points are included in at least one triangle or a termination condition is met; Step S33, after the point cloud is registered, three-dimensional reconstruction is performed using a greedy projection triangulation algorithm; Step S34: adaptively adjust key parameters of the greedy projection triangulation algorithm based on normal vector consistency.

5. According to claim 4, a tongue 3D visualization acquisition and reconstruction method is characterized in that: The specific steps in step S33 include: First, define the point set Search radius To construct a neighborhood point set , the formula is: ； in, yes Point around A point j in the set; After determining the neighborhood, the triangular surface is constructed using the following conditions: ； in, It is composed of three points in the neighborhood The interior angles of the triangle formed by and is the preset angle threshold; The mesh generated in the triangulation process is represented as a combination of a point set V and a face set F, and the formula is: ； Among them, V is the reconstructed vertex set and F is the corresponding patch index.

6. According to claim 4, a tongue 3D visualization acquisition and reconstruction method is characterized in that: The specific steps in step S34 include: For each point in the point cloud data , its normal vector can be expressed as ; In a local area, the consistency of the normal vector is quantified by calculating the variance of the normal vector; First, calculate the average value of all normal vectors in the local area , the formula is: ； Where N is the number of points in the local area; Then calculate the angular deviation between each normal vector and the average normal vector and find its variance , the formula is: ； in, is the dot product between the normal vector and the mean normal vector, and are the moduli of the normal vector and the mean normal vector respectively; Based on the consistency of the normal vector, that is, the variance , adaptively adjust the key parameters of the greedy triangle projection, including the minimum angle , maximum angle , and the maximum surface angle ; Calculate the minimum angle , in high curvature areas with low normal vector consistency, more detailed triangles are generated through the minimum angle, and the adjustment formula is: ； in, is the default minimum angle, c is the adjustment factor; Calculate the maximum angle , in areas with low normal vector consistency, larger surface angle deviations are allowed to generate larger triangles, thereby reducing redundant mesh refinement. The adjustment formula is: ； in, is the default maximum angle, and d is the adjustment factor; Calculate the maximum surface angle , in areas with low normal vector consistency, a larger surface angle deviation is allowed so that the triangle can better adapt to surface changes. The adjustment formula is: ； in, is the default maximum surface angle and e is the adjustment factor.

7. A collection instrument, characterized in that: The tongue phase three-dimensional visualization acquisition and reconstruction method according to any one of claims 1 to 6 is adopted, wherein the acquisition device comprises: Base (1); A movable seat (2) disposed at the top of the rear end of the base; A main collector (3) disposed on the top of the front end of the movable seat; A mounting frame (3.1) disposed at the bottom of the main collector for adjusting the height; and A head frame (4) arranged at the top of the front end of the base; The movable seat is adjusted to reciprocate along the extension direction on the base, the distance from the main collector to the head frame is adjusted, and the main collector and the head frame are adjusted to be at the same relative height in cooperation with the mounting frame.

8. A collector according to claim 7, characterized in that: A rotating disk (3.2) for horizontally rotating the main collector is arranged at the bottom of the main collector (3).

9. A storage medium storing instructions executable by a processor, characterized in that: The processor-executable instructions are used to implement a tongue three-dimensional visualization acquisition and reconstruction method as described in any one of claims 1-6 when executed by the processor.

Images