Medical point cloud segmentation system and method based on boundary point supervision

Abstract

The invention relates to the technical field of medical point cloud data processing and application, in particular to a medical point cloud segmentation system and method based on boundary point supervision, and the system comprises an orientation and position correction module, a boundary point search module and a loss function based on sample equalization. The orientation and position correction module unifies the position and orientation of the original point cloud, so that the orientations of different dental cast point clouds are unified; the boundary point searching module is fused with a k-nearest neighbor algorithm and a farthest point sampling algorithm to identify boundary points in a data set, and the boundary points are subjected to pointnet + + down-sampling and up-sampling; and carrying out loss calculation on the boundary points based on a loss function of sample equalization. According to the method, the oral medicine point cloud can be accurately and efficiently segmented, so that the problems that an existing segmentation algorithm cannot predict the large-angle rotating point cloud, boundary points in a three-dimensional point cloud segmentation result are poor, and the point cloud segmentation effect is poor due to the serious unbalanced class number are solved.

Description

Technical Field

[0001] The present invention relates to the field of medical point cloud data processing and application technology, and in particular to a medical point cloud segmentation system and method based on boundary point supervision. Background Art

[0002] The segmentation algorithm cannot predict point clouds with large rotation angles. For point clouds with large rotation angles, the extraction of deep semantic information of the point cloud becomes inaccurate. Specifically, it is more difficult to extract semantic features of feature points in large-angle three-dimensional rotation. Point cloud samples from different batches and manufacturers in the project have random large-angle differences on the three axes. The angle differences on the three axes make it impossible to accurately predict. The direct reason is that the receptive field of the existing segmentation algorithm has certain limitations. It is impossible to invent a global unified receptive field.

[0003] To address the problem of boundary point difference in 3D point cloud segmentation results, the existing commonly used CBL contour-based loss function is very sensitive to noise and slight shape changes. The disadvantage of the existing technology is that it is less robust under noise. If there is noise or slight changes in the image or data, it may lead to unstable results.

[0004] In order to solve the problem of severe imbalance in the number of categories, which leads to poor point cloud segmentation effect, the existing technology is to oversample (Oversampling) to copy or synthesize minority class samples to increase their number. The disadvantage of the existing technology is that it may lead to overfitting or lose important information of the majority class. Undersampling (Undersampling) reduces the number of majority class samples to balance the number of samples of each class. However, oversampling may lead to overfitting because the model will see the same minority class samples many times; undersampling may lose important information of the majority class, resulting in a decrease in model performance. In the task of oral medical point cloud segmentation, there is usually a serious problem of imbalance in the number of categories. The size of the target foreground area is often several orders of magnitude smaller than the background area. For example, the foreground area of ​​a large connective body is more than 20 times smaller than the background area with a small cell. The cross entropy loss function commonly used in segmentation has a well-known disadvantage in highly imbalanced problems, that is, it assumes that all samples and categories are equally important, which usually leads to instability in training and causes the decision boundary to be biased towards the category with a large number. For the problem of imbalance in the number of categories, a common strategy is to downsample the large number of categories to rebalance the prior distribution of the categories, but this strategy limits the use of training images. Another strategy is weighting, that is, giving greater weight to categories with fewer numbers and smaller weight to categories with more numbers. Although this method is effective for some imbalanced problems, it still has difficulties in dealing with extremely imbalanced data. The cross entropy gradient calculated on a few pixels usually contains noise. Giving a larger weight to the minority category further increases the noise, resulting in unstable training. Dice loss, another common loss function in segmentation, usually works better than CE loss in unbalanced medical point cloud segmentation problems. However, difficulties may arise when encountering very small areas. Misclassified pixels may cause a sharp decrease in loss, resulting in unstable optimization. In addition, dice loss corresponds to the harmonic mean of precision and recall. When true positives remain unchanged, false positives and false negatives are equally important. Therefore, dice loss is mainly applicable to situations where the number of these two types of errors is similar. Summary of the invention

[0005] The purpose of the present invention is to solve the three problems of the prior art mentioned above and to propose a medical point cloud segmentation system and method based on boundary point supervision, which can accurately and efficiently segment oral medical point clouds.

[0006] To achieve the above purpose, the technical solution adopted is:

[0007] A medical point cloud segmentation system based on boundary point supervision, comprising:

[0008] The orientation and position correction module unifies the position and orientation of the original point cloud, making the orientation of different dental model point clouds consistent;

[0009] The boundary point search module integrates the k-nearest neighbor algorithm and the farthest point sampling algorithm to identify boundary points in the data set. The boundary points are then downsampled and upsampled by pointnet++.

[0010] Based on the sample balanced loss function, the loss is calculated for the boundary points.

[0011] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, the working process of the orientation and position correction module is as follows:

[0012] Move the original point cloud to a coordinate system with the geometric center of the point cloud as the origin, and the point set after translation is data;

[0013] Calculate the basis vectors in the z direction, x direction and y direction, and project the point cloud to the three-coordinate plane. The normal vector of the plane with the largest area is the basis vector _ref_normal in the z direction of the local coordinate system;

[0014] The projection point set _plane_vertices of xoy is calculated by the projection matrix;

[0015] By solving the covariance matrix of the projection point set _plane_vertices, find the main direction of the point and the rotation matrix rotation to the main direction;

[0016] Calculate new direction vectors _dir0 and _dir1 in the x and y directions;

[0017] The three-dimensional rotation matrix matrix_world is obtained through the normal vector _ref_normal and the new direction vectors _dir0 and _dir1 in the x and y directions. The rotation matrix is ​​multiplied with the point set to obtain the point set with three coordinates aligned.

[0018] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, moving the original point cloud to a coordinate system with the geometric center of the point cloud as the origin includes:

[0019] Input an n*6 point cloud X, calculate the point cloud geometric center [meanY,meanX,meanZ], and get a 4*4 translation matrix The translated point set is expressed as: data = trans (X^T), where X is the point cloud with n rows * 3 columns and then padded with one column of 1 to get 4*n, and T is the matrix transpose operation.

[0020] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, calculating the basis vectors in the x-direction and the y-direction includes:

[0021] The x-direction basis vector _plane_x_axis is calculated by the perpendicular relationship between the z-direction basis vector _ref_normal and the vector [0, 0, 1];

[0022] The y-direction basis vector _plane_y_axis is calculated by the perpendicular relationship between the z-direction basis vector _ref_normal and the original x-direction basis vector _plane_x_axis.

[0023] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, the projection point set _plane_vertices of xoy is obtained by multiplying the projection matrix composed of the original x-direction basis vector _plane_x_axis and the original y-direction basis vector _plane_y_axis with the translated point set data.

[0024] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, calculating new direction vectors _dir0 and _dir1 in the x-direction and the y-direction includes:

[0025] Calculate the new direction vector in the x direction: Get the new x direction vector _dir0 by linearly combining the original basis vectors with the rotation matrix rotation. The calculation formula is: Among them, rotation[1,:] is the rotation taking the first row;

[0026] Calculate the new direction vector in the y direction: obtain the new y direction vector _dir1 by the cross product of the normal vector _ref_normal and the original x-direction basis vector _plane_x_axis. The calculation formula is: _dir1=_ref_normal×_plane_x_axis.

[0027] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, the boundary point search module first uses the k-nearest neighbor algorithm to perform preliminary boundary point identification on the input point cloud to find potential boundary points; then uses the farthest point sampling algorithm FPS to screen the preliminary identified boundary points, and FPS covers the boundary area of ​​the data by selecting the points with the farthest distance.

[0028] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, if the specified number of boundary points is greater than the actual number of boundary points, it is automatically filled to the specified number; if the specified number of boundary points is less than the actual number of boundary points, the boundary point with the farthest distance will be selected.

[0029] According to the medical point cloud segmentation system based on boundary point supervision of the present invention, further, the expression of the loss function based on sample balance is as follows:

[0030] L B =∫ Ω φ G (q)s θ (q)dq

[0031] Among them, Ω is the three-dimensional space domain, G is the foreground area, q is any point in the point set, if q belongs to G, φ G (q)=-D G (q), otherwise φ G (q) = D G (q), D G (q) means that any point q belongs to Ω and the contour The distance between the closest points on θ Indicates that the model output corresponds to the foreground area;

[0032] The total loss function expression of the point cloud segmentation network is as follows:

[0033] L total =L R (θ)+αL B (θ)

[0034] Among them, α is the weighting factor, L R (θ) is the regional loss function.

[0035] Furthermore, the present invention also provides a medical point cloud segmentation method based on boundary point supervision, comprising:

[0036] After the original point cloud is processed by the orientation and position correction module, the position and orientation of the original point cloud are unified;

[0037] The processed point cloud is passed through the boundary point search module to find the boundary points of the target, and the boundary points are then downsampled and upsampled by pointnet++;

[0038] Finally, the loss of boundary points is calculated using a loss function based on sample balance.

[0039] The beneficial effects achieved by adopting the above technical solution are:

[0040] 1. In view of the problem that the existing segmentation algorithm cannot predict the point cloud with large angle rotation, the present invention proposes a Correct Orientation and position (COP) module, which unifies the position and orientation of the point cloud. It is a point cloud preprocessing algorithm for straightening the point cloud, which makes it more conducive to the learning of semantic features during deep learning and solves the failure problem caused by the difference in the orientation of the point cloud of objects with large angle rotation. For example, the upper jaw is facing the Z axis, and the same features are easier to learn. Due to the existence of different large angle rotations, the trained model cannot generalize the dental mold point clouds of different manufacturers. Using the three-coordinate alignment of the Z axis, different large angle rotations can be unified, thus eliminating the problem of poor generalization of point clouds from different manufacturers due to the rotation angle.

[0041] 2. The current 3D point cloud segmentation method for oral medicine has a poor effect on the segmentation effect of scene boundaries, resulting in a decrease in the overall segmentation performance. To address the problem of unsatisfactory performance of 3D point cloud boundary segmentation, the present invention proposes a boundary point search module (Boundary Point Search, BPS), which combines the k-nearest neighbor algorithm and the farthest point sampling algorithm to find suitable target boundary points, and the boundary points are evenly distributed.

[0042] 3. In order to solve the serious problem of imbalance in the number of categories that leads to poor point cloud segmentation effect, the present invention proposes a loss function based on sample equilibrium (label equilibrium Loss, LE-Loss), which uses distance measurement in contour space rather than region space. The boundary loss is calculated not as an integral over the region, but as an integral over the boundary between regions. Therefore, the problem of sample imbalance segmentation can be alleviated. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0044] Figure 1 is a structural block diagram of a medical point cloud segmentation system based on boundary point supervision according to an embodiment of the present invention;

[0045] Figure 2 is a workflow diagram of the orientation and position correction module of an embodiment of the present invention;

[0046] Figure 3 is a diagram of processing results of the orientation and position correction module of an embodiment of the present invention;

[0047] Figure 4Figure 1 is a diagram of the original point cloud and the processing result of the search boundary point module, where (a) is the original point cloud (yellow and green are targets, blue, dark green and red are backgrounds), and (b) is the processing result of the search boundary point module;

[0048] Figure 5 is an overall framework diagram of a point cloud segmentation network according to an embodiment of the present invention;

[0049] Figure 6 It is a boundary effect diagram of different models of the embodiment of the present invention on the "mandibular edentulous point cloud data";

[0050] Figure 7 This is the influence of different modules of the embodiment of the present invention on the model. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.

[0052] The medical point cloud segmentation system based on boundary point supervision in this embodiment is as follows: Figure 1 As shown, it contains the following modules.

[0053] The orientation and position correction module unifies the position and orientation of the original point cloud, making the orientation of point clouds from different dental models uniform. By calculating the unique position and orientation of the point cloud, the accuracy of deep learning feature extraction of the point cloud is improved.

[0054] The boundary point search module integrates the k-nearest neighbor algorithm and the farthest point sampling algorithm to identify the boundary points in the data set. The boundary points are then downsampled and upsampled by pointnet++ to stably and efficiently learn the boundary point features.

[0055] Based on the sample balanced loss function, the loss of boundary points is calculated to further improve the learning accuracy of the samples, thereby realizing the oral three-dimensional laser point cloud segmentation.

[0056] The three modules are introduced in detail below.

[0057] (A) The working process of the Correct Orientation and Position (COP) module is as follows: Figure 2 As shown:

[0058] Step S101: Move the original point cloud to a coordinate system with the geometric center of the point cloud as the origin, and the point set after translation is data.

[0059] Input an n*6 point cloud X, calculate the point cloud geometric center [meanY,meanX,meanZ], and get a 4*4 translation matrix The translated point set is expressed as: data = trans·(X^T), where X is the point cloud with n rows * 3 columns and then padded with a column of '1' to get 4*n, and T is the matrix transpose operation.

[0060] Step S102, calculate basis vectors in the z direction, x direction and y direction.

[0061] ① Calculate the basis vector in the z direction: Project the point cloud onto the three-coordinate plane. The normal vector of the plane with the largest area is the basis vector _ref_normal in the z direction of the local coordinate system.

[0062] ② Calculate the basis vectors in the x and y directions.

[0063] X-direction basis vector: The 1*3 original x-direction basis vector _plane_x_axiss is calculated from the perpendicular relationship between the z-direction basis vector _ref_normal and the vector [0, 0, 1]. The calculation formula is: _plane_x_axis=UNIT_Z_AXIS×_ref_normal, where UNIT_Z_AXIS=[0,0,1].

[0064] Y-direction basis vector: The 1*3 original y-direction basis vector _plane_y_axis is calculated from the perpendicular relationship between the z-direction basis vector _ref_normal and the original x-direction basis vector _plane_x_axis.

[0065] Step S103: Calculate the projection point set plane_vertices of xoy using the projection matrix.

[0066] Calculate the projection point set _plane_vertices of n*2 xoy by dot multiplication, The 2*3 projection matrix composed of the original x-direction basis vector _plane_x_axis and the original y-direction basis vector _plane_y_axis data is the point set after translation.

[0067] Step S104: find the main direction of the point set and the 2*2 rotation matrix rotation to the main direction by solving the covariance matrix of the projection point set _plane_vertices.

[0068] Step S105, calculate new direction vectors _dir0 and _dir1 in the x direction and the y direction.

[0069] (1) Calculate the new direction vector in the x direction.

[0070] The linear combination of the original basis vectors by the rotation matrix rotation obtains a new x-direction vector _dir0 of 1*3. The calculation formula is: Among them, rotation[1,:] is the rotation taking the first row.

[0071] (2) Calculate the new direction vector in the y direction.

[0072] The new y-direction vector _dir1 of 1*3 is obtained by the cross product of the normal vector _ref_normal and the original x-direction basis vector _plane_x_axis. The calculation formula is: _dir1=_ref_normal×_plane_x_axis.

[0073] Step S106: Calculate the rotated point set.

[0074] Through the normal vector _ref_normal and the new direction vectors _dir0 and _dir1 in the x and y directions, we get the 4*4 three-dimensional rotation matrix matrix_world. The rotation matrix is ​​multiplied with the point set to get the three-coordinate aligned point set output=matrix_world·data.

[0075] The COP module processing results are as follows Figure 3 As shown, the left column is the original point cloud, and the right column is the point cloud processed by the COP module. In the xoy plane, in the first row, the tooth orientation of the point cloud is at an angle of approximately 100° with the x-axis (red), and the teeth of the point cloud processed by the COP module are facing the positive direction of the x-axis. In the second row, the tooth orientation of the point cloud is at an angle of approximately 210° with the x-axis (red), and the teeth of the point cloud processed by the COP module are facing the positive direction of the x-axis. In the third row, the tooth orientation of the point cloud is at an angle of approximately 100° with the x-axis (red), and the teeth of the point cloud processed by the COP module are facing the positive direction of the x-axis. Input point clouds of different directions are all processed by the COP module with the teeth facing the x-axis (red), and the geometric center is located at the origin of the coordinate axis. The experimental results verify that the COP module has obvious standardization of the direction and position of the point cloud.

[0076] (B) Boundary point search module (BPS)

[0077] By integrating the k-nearest neighbor algorithm (k-NN) and the farthest point sampling (FPS) boundary point search algorithm, boundary points in the data set can be more effectively identified and confirmed. First, the k-NN algorithm is used to perform preliminary boundary point identification on the input point cloud to find possible boundary points. The k-NN algorithm can efficiently find neighboring points and identify potential boundary points, but one of its disadvantages is that the boundary points identified by k-NN may be unevenly distributed and have some noise points, resulting in inconsistent feature scales, especially in complex or high-dimensional data sets.

[0078] In order to make up for this shortcoming of k-NN, the FPS algorithm is then used to further screen these initially identified boundary points, thereby refining and ensuring the accuracy of the boundary points. The FPS method effectively covers the boundary area of ​​the data by selecting the farthest points, ensuring that the boundary points are evenly distributed, and overcomes the shortcomings of k-NN in dealing with complex boundaries. In this way, the FPS method can avoid the problem of inconsistent feature scales caused by insufficient number of boundary points.

[0079] In addition, the fusion algorithm allows the number of boundary points to be set flexibly. If the specified number of boundary points is greater than the actual number of boundary points, it will be automatically filled to the specified number. If the specified number of boundary points is less than the actual number of boundary points, the boundary points with the farthest distance will be selected. In this way, it is ensured that the identified boundary points can accurately reflect the boundaries between classes, and the accuracy and reliability of boundary point search are improved. Overall, this fusion algorithm combines the preliminary recognition ability of k-NN and the optimization confirmation ability of FPS, and performs well in identifying and confirming boundary points in the data set. The processing results of the BPS module are shown in Figure 2. Figure 4 shown.

[0080] The structure of the point cloud segmentation network based on boundary point supervision (BS-Net) is as follows Figure 5 As shown, S1...F4 on the upper side is a commonly used backbone segmentation network, such as pointnet++. B1 is the k-NN module in the BPS module, B2 is the FPS module in the BPS module, and B3 is the LE-Loss loss function. After the BPS module, it is passed to the backbone segmentation network through B2 for boundary point feature extraction. After feature extraction and upsampling, it is passed to B3 to calculate the loss function separately, and then weighted addition is performed with the internal point loss value of the point cloud. The formula is as follows:

[0081] Loss total =α*Loss Bounding +(1-α)*Loss inter (1)

[0082] Among them, Loss Boundingis the loss function calculated separately for the boundary points. The inter point set is the point set in the foreground point set excluding the Bounding boundary points. inter is the loss function of the internal points, α = number of boundary points / total number of points.

[0083] (C) Loss function based on sample equilibrium (label equilibrium Loss, LE-Loss)

[0084] This section proposes a loss function based on sample equilibrium (label equilibrium Loss, LE-Loss), which uses a distance metric in contour space rather than region space during training. During training, the boundary loss is calculated not as an integral over the region, but as an integral over the boundary between regions, thus alleviating the problems associated with region loss in highly unbalanced segmentation problems.

[0085] Inspired by the use of discrete graph-based optimization methods to calculate the gradient flow of curve evolution, an integral method is used to calculate the changes in the target boundary point cloud, avoiding local differential calculations on the contour points. The final boundary loss is a linear function of the softmax probability of the network output area, so it can be combined with the existing regional loss.

[0086] I is a point set in the three-dimensional space domain Ω, g is the binary segmentation point set of the grouptruth real foreground of the point set, if point p belongs to the target area (foreground area) G, g(p) = 1, if point p belongs to the background area Ω\G, g(p) = 0. S_ represents the softmax probability output of the segmentation network, S θ Indicates that the model output corresponds to the foreground area, S θ ={p belongs to Ω|S_(p)>=thr}, where thr is a threshold set in advance.

[0087] Constructing the boundary loss function The loss function takes the form of the distance in the region boundary space in Ω, where is a representation of the boundary of the ground truth region G (such as the set sum of all points on the boundary), are the boundaries of the segmentation regions defined by the network output. The points on the network are represented as the network output area S θ The differentiable function of is not clear. Consider the following representation of the asymmetric L2 distance in shape space, which evaluates the distance between two adjacent boundaries. and The distance between them changes.

[0088]

[0089] where p belongs to Ω is the boundary A little bit above, It's the border The corresponding point is yes The normal at point p is Intersection point. Formula 2 cannot be used directly as a Loss function, but the differential boundary change of Formula 2 can be approximated by the integral method, which avoids the differential calculation involving the points on the contour and uses regional integrals to represent the boundary change, as follows:

[0090]

[0091] Where ΔS represents the area between the two contours, D G is a distance map relative to the boundary, D G (q) means that any point q belongs to Ω and the contour The closest point Distance between:

[0092] The following formula is obtained from formula 3:

[0093]

[0094] Where s is the binary indicator function of region S: if q belongs to the foreground, then s(q) = 1, otherwise it is 0. G It's the border The level set representation is: if q belongs to G, φ G (q)=-D G (q), otherwise φ G (q) = D G (q). For S = S θ , that is, the softmax output of the network Replacing s(q) in Formula 4, we get the boundary loss as follows:

[0095] L B =∫ Ω φ G (q)s θ (q)dq(5)

[0096] The last term in Equation 4 is removed because it contains no parameters. The level set function φ G It is directly calculated in advance based on the gt region G. The boundary loss can be combined with the commonly used region-based loss function for N-class segmentation problems.

[0097] L R (θ)+αL B (θ)(6)

[0098] In the segmentation task, in order to balance the weights of the two loss functions, the softmax output of each point q is weighted by the distance function through Formula 6. However, in the region-based loss function, this information about the distance to the boundary is ignored, and each point in the region is given the same weight regardless of its distance to the boundary. In the boundary loss, when all negative values ​​in the distance function are retained (i.e., the model's softmax prediction value for all pixels in the target region is 1) and all positive values ​​are discarded (i.e., the model's softmax prediction value for the background is 0), the boundary loss reaches the global minimum. This shows that when the model's softmax prediction value is completely consistent with the ground truth, the boundary loss is minimized, thus verifying the effectiveness of the boundary loss.

[0099] Corresponding to the above system, this embodiment also proposes a medical point cloud segmentation method based on boundary point supervision, which includes the following contents:

[0100] After the original point cloud is processed by the orientation and position correction module, the position and orientation of the original point cloud are unified.

[0101] The processed point cloud is passed through the boundary point search module to find the boundary points of the target, and the boundary points are then downsampled and upsampled by pointnet++.

[0102] Finally, the loss of boundary points is calculated using a loss function based on sample balance.

[0103] The effectiveness and superiority of the present invention are further demonstrated by experiments below.

[0104] In order to evaluate the segmentation performance, mIoU, mIoU@boundary, and mIoU@inner are used as performance evaluation indicators of the model. IoU is the ratio of the intersection and union of two sets. The mean intersection and union ratio mIoU is the average value of IoU, which can show the segmentation accuracy of each type of point cloud, as shown in Formula 7:

[0105]

[0106] Among them, Gt represents the set of points that actually belong to a certain type of label in the entire point cloud X, Pre represents the set of points that are predicted to belong to this type of label in the entire point cloud X, the numerator is the number of points with the same label, and the denominator is the number of points that actually have this label or are predicted to have this label, and finally converted into a percentage. By changing the scope of the definition domain, mIoU@boundary and mIoU@inner can be defined, as shown in formulas (8) and (9):

[0107] mIoU @boundary =mIOU(Bl) (8)

[0108] mIoU@inner = mIOU(X-Bl) (9)

[0109] Where Bl is the set of points in the boundary area, and X-B1 is the set of points in the non-boundary area.

[0110] ①Horizontal comparison experiment

[0111] Table 1 Experimental results of each model on "mandibular edentulous point cloud data"

[0112] mIOU(%) mIOU@boundary(%) mIOU@inner(%) ACC(%) Pointnet++ 78.47 63.19 79.26 85.31 pointNext 79.21 65.27 81.61 89.21 CBL 83.43 66.74 87.17 88.12 pointTransformer 82.5 65.54 86.19 90.23 Ours(BS-Net) 88.23 70.21 89.34 92.21

[0113] Table 1 shows the mIoU, mIoU@boundary, and mIoU@inner of the comparison model on the "mandibular edentulous point cloud data" dataset. Table 1 shows the experimental results of BS-Net and the comparison model on the "mandibular edentulous point cloud data" dataset. The following conclusions can be drawn from Table 1: On the "mandibular edentulous point cloud data", the improvement of mIoU of BS-Net compared with the CBL model is higher than that of the Point Transformer model. Due to the addition of contrastive learning, it is slightly better than Point Transformer, but the performance improvement is not high, which may focus on improving the boundary performance; due to the addition of the self-attention mechanism, the segmentation effect of the model is much better than other previous models, which proves the superiority of the Transformer structure for model segmentation. Compared with other methods on the "mandibular edentulous point cloud data", the improvement of the mIoU@boundary evaluation index of BS-Net is higher than that of mIoU@inner. Therefore, the experiment proves that BS-Ne can effectively improve the accuracy of boundary area segmentation.

[0114] ②BS-Net effect diagram

[0115] Figure 6 The boundary effects of each model on the "mandibular edentulous point cloud data" are visualized respectively. Among them: the COP module can effectively identify the boundary, while the Point Transformer model has no obvious effect on the identification of boundary point clouds. Since Point Transformer focuses on feature extraction and does not focus on mining boundary features, its effect on boundary identification is limited. Pointnet++ and Pointnext cannot recognize point clouds with large angles well. Since the COP module model unifies the orientation and position, the COP module gives more accurate relative position information to the points in the point cloud. During training, the model better learns deep semantic information. In the reasoning stage, the model can make better predictions and more accurate predictions, thereby improving the generalization of the model for large-angle rotation scenes.

[0116] Depend on Figure 6(d)(f) It can be seen that in the segmentation task where the boundary curvature does not change significantly, CBL cannot identify the boundary well, which shows the importance of the sample-balanced loss function for point cloud boundary recognition. Since the sample-balanced loss function comparative learning in the present invention can better distinguish boundary point samples, it can improve the point cloud segmentation performance at the boundary, and it shows that the label distribution learning proposed by the sample-balanced loss function model is beneficial for identifying more refined boundaries. In summary, the COP module + BPS module + LE-Loss has a significant improvement in the boundary point cloud segmentation effect.

[0117] ③BS-Net ablation experiment

[0118] This section mainly verifies the impact of using only a single module on the model performance of BS-Net. Figure 7 The influence of different modules on the mIoU index of the model segmentation accuracy is demonstrated. The benchmark is compared with the pointnet++ model, and the COP module is introduced on its basis. Especially in samples with large angle rotation, the original pointnet++ model is an invalid algorithm prediction, but the prediction is stable after adding the COP module. By applying the COP module, BPS module, and BE-Loss module to compare the modules, it is proved that the BS-Net model is superior to the CBL structure that only uses boundary point contrast learning and the model that only uses the Transformer structure in performance. The experimental results verify the performance improvement effect of the four structures of BS-Net. The COP module has a significant effect on improving the overall segmentation accuracy; the COP module greatly improves the boundary segmentation performance of the model, for example, the predicted edge is smoother and closer to the edge of the annotated point cloud. The reason for the above improvement is that the COP module provides a unified orientation, accurately provides the model with relative position information under a large receptive field, and the model can learn accurate semantics. The combination of BPS module + BE-Loss does not significantly improve the overall segmentation accuracy. It mainly improves the boundary segmentation performance. The boundary performance segmentation is due to the combination of BPS module + BE-Loss solving the problems of sample imbalance and small boundary point weights. The direct reason is that in the feedback propagation, the foreground points and boundary points with few samples are prone to gradient disappearance.

[0119] Therefore, the BS-Net model has a high advantage in semantic understanding and label distribution learning. By introducing the COP module, the BS-Net model further enhances the model's expressiveness and generalization capabilities. The BPS module and LE-Loss loss function can better capture the key information of the point cloud boundary, thereby improving the performance of the model.

[0120] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A medical point cloud segmentation system based on boundary point supervision, characterized in that: Include: The orientation and position correction module unifies the position and orientation of the original point cloud, making the orientation of different dental model point clouds consistent; The boundary point search module integrates the k-nearest neighbor algorithm and the farthest point sampling algorithm to identify boundary points in the data set. The boundary points are then downsampled and upsampled by pointnet++. Based on the sample balanced loss function, the loss is calculated for the boundary points.

2. The medical point cloud segmentation system based on boundary point supervision according to claim 1, characterized in that: The working process of the orientation and position correction module is as follows: Move the original point cloud to a coordinate system with the geometric center of the point cloud as the origin, and the point set after translation is data; Calculate the basis vectors in the z direction, x direction and y direction, and project the point cloud to the three-coordinate plane. The normal vector of the plane with the largest area is the basis vector _ref_normal in the z direction of the local coordinate system; The projection point set _plane_vertices of xoy is calculated by the projection matrix; By solving the covariance matrix of the projection point set _plane_vertices, find the main direction of the point and the rotation matrix rotation to the main direction; Calculate new direction vectors _dir0 and _dir1 in the x and y directions; The three-dimensional rotation matrix matrix_world is obtained through the normal vector _ref_normal and the new direction vectors _dir0 and _dir1 in the x and y directions. The rotation matrix is ​​multiplied with the point set to obtain the point set with three coordinates aligned.

3. The medical point cloud segmentation system based on boundary point supervision according to claim 2, characterized in that: Moving the original point cloud to a coordinate system with the point cloud geometric center as the origin includes: Input an n*6 point cloud X, calculate the point cloud geometric center [meanY,meanX,meanZ], and get a 4*4 translation matrix The translated point set is expressed as: data = trans (X^T), where X is the point cloud with n rows * 3 columns and then padded with one column of 1 to get 4*n, and T is the matrix transpose operation.

4. The medical point cloud segmentation system based on boundary point supervision according to claim 2, characterized in that: Calculating the x- and y-direction basis vectors involves: The x-direction basis vector _plane_x_axis is calculated by the perpendicular relationship between the z-direction basis vector _ref_normal and the vector [0, 0, 1]; The y-direction basis vector _plane_y_axis is calculated by the perpendicular relationship between the z-direction basis vector _ref_normal and the original x-direction basis vector _plane_x_axis.

5. The medical point cloud segmentation system based on boundary point supervision according to claim 4, characterized in that: The projection point set _plane_vertices of xoy is obtained by multiplying the projection matrix composed of the original x-direction basis vector _plane_x_axis and the original y-direction basis vector _plane_y_axis with the translated point set data.

6. The medical point cloud segmentation system based on boundary point supervision according to claim 4, characterized in that: Calculate the new direction vectors _dir0 and _dir1 in the x and y directions containing: Calculate the new direction vector in the x direction: Get the new x direction vector _dir0 by linearly combining the original basis vectors with the rotation matrix rotation. The calculation formula is: Among them, rotation[1,:] is the rotation taking the first row; Calculate the new direction vector in the y direction: obtain the new y direction vector _dir1 by the cross product of the normal vector _ref_normal and the original x-direction basis vector _plane_x_axis. The calculation formula is: _dir1=_ref_normal×_plane_x_axis.

7. The medical point cloud segmentation system based on boundary point supervision according to claim 1, characterized in that: The boundary point search module first uses the k-nearest neighbor algorithm to perform preliminary boundary point identification on the input point cloud to find potential boundary points; then the farthest point sampling algorithm FPS is used to filter the preliminarily identified boundary points. FPS covers the boundary area of ​​the data by selecting the farthest points.

8. The medical point cloud segmentation system based on boundary point supervision according to claim 7, characterized in that: If the specified number of boundary points is greater than the actual number of boundary points, the number will be automatically filled to the specified number; if the specified number of boundary points is less than the actual number of boundary points, the boundary point with the farthest distance will be selected.

9. The medical point cloud segmentation system based on boundary point supervision according to claim 1, characterized in that: The expression of the loss function based on sample balance is as follows: L B =∫ Ω φ G (q)s θ (q)dq Among them, Ω is the three-dimensional space domain, G is the foreground area, q is any point in the point set, if q belongs to G, φ G (q) = -D G (q), otherwise φ G (q) = D G (q), D G (q) means that any point q belongs to Ω and the contour The distance between the closest points on θ Indicates that the model output corresponds to the foreground area; The total loss function expression of the point cloud segmentation network is as follows: L total =L R (θ)+αL B (i) Among them, α is the weighting factor, L R (θ) is the regional loss function.

10. A medical point cloud segmentation method based on boundary point supervision, characterized in that: Include: After the original point cloud is processed by the orientation and position correction module, the position and orientation of the original point cloud are unified; The processed point cloud is passed through the boundary point search module to find the boundary points of the target, and the boundary points are then downsampled and upsampled by pointnet++; Finally, the loss of boundary points is calculated using a loss function based on sample balance.

Images