Medical point cloud segmentation method and system based on multi-layer boundary point comparative learning
Through the multi-layer boundary point comparison learning method and the infoNCE loss function of adaptive temperature values, the problem of inaccurate boundary region segmentation in stomatological point cloud segmentation is solved, and faster model convergence and higher segmentation accuracy are achieved.
Patent Information
- Application Number
- CN202510394091.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-29
AI Technical Summary
In the existing 3D point cloud segmentation network, the boundary area segmentation effect is poor in the stomatological point cloud segmentation task, and the temperature hyperparameters of manually setting the infoNCE loss function are inaccurate and cumbersome.
The multi-layer boundary point comparison learning method is adopted, combining the boundary point comparison module and the infoNCE loss function module of adaptive temperature values, and the sampling medical point cloud is supervised layer by layer, and the comparison learning is carried out through adaptive adjustment of temperature values.
The accuracy of boundary point segmentation and the convergence speed of the network are improved, the IOU accuracy is increased by 0.8%, and the model convergence speed is increased by 2 times, reducing the risk of overfitting.
Smart Images
Figure CN120388034A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical data processing, and particularly relates to a medical point cloud segmentation method and system based on multi-layer boundary point contrast learning. Background Art
[0002] In the oral medicine point cloud segmentation task, high segmentation accuracy and accurate prediction of the edge are required. For ordinary 3D point cloud segmentation networks, the segmentation effect in the central region is good. However, the segmentation effect in the boundary region is not satisfactory, that is, the prediction effect of the part adjacent to the target and the background is inaccurate. Objectively, due to the high similarity of features at the junction of the target and the background, it is difficult to distinguish the target boundary from the background. For example, in some samples, due to the very small curvature change, the target edge and the background features are very similar. Ordinary 3D point cloud segmentation networks do not "targetedly" supervise and learn the point set at the junction, and the boundary point prediction is inaccurate.
[0003] The temperature value of the infoNCE loss function is very important, which directly affects the convergence efficiency and the final accuracy. The temperature value of the infoNCE loss function is an unknown real number between 0.1 and 1.0, and the result of the value can only be known after the entire training is completed. Manually setting the temperature hyperparameter of INFO is inaccurate and very cumbersome, and the manual setting efficiency is very low. Summary of the Invention
[0004] Aiming at the problems of inaccurate boundary point prediction in the existing point cloud segmentation network and inaccurate and cumbersome manual setting of the temperature hyperparameter of infoNCE, in the face of oral medicine point clouds, the present invention proposes a medical point cloud segmentation method and system based on multi-layer boundary point contrast learning.
[0005] To achieve the above object, the present invention adopts the following technical solutions:
[0006] On the one hand, the present invention proposes a medical point cloud segmentation method based on multi-layer boundary point contrast learning, including:
[0007] Segmenting the medical point cloud by a point cloud segmentation network based on multi-layer boundary point contrast learning; the point cloud segmentation network based on multi-layer boundary point contrast learning includes a boundary point contrast module and an infoNCE loss function module with an adaptive temperature value; the boundary point contrast module is used to obtain boundary point features and neighbor point features of boundary points based on the upsampled point cloud coordinates and features of each upsampling layer; the infoNCE loss function module with an adaptive temperature value is used to obtain a loss function value based on the adaptively adjusted temperature value, the boundary point features and neighbor point features of boundary points obtained by the boundary point contrast module, so as to perform contrast learning on the downsampling results of each upsampling layer.
[0008] Further, the point cloud segmentation network for multi-layer boundary point contrast learning further includes a plurality of downsampling layers and a plurality of upsampling layers.
[0009] Further, the boundary point contrast module is arranged in one-to-one correspondence with the upsampling layer, and the output of each upsampling layer is used as the input of a boundary point contrast module.
[0010] Further, the boundary point contrast module is specifically configured to:
[0011] Obtain the medical point cloud coordinates and features of each upsampling layer;
[0012] Generate labels for the medical point cloud, and generate boundary points for each layer based on the labels;
[0013] Use KNN query to find the nearest neighbor point index of each point in the medical point cloud;
[0014] Remove the self-loop in the nearest neighbor point index of each point to obtain the neighbor index corresponding to each point;
[0015] Obtain the labels and features of the neighbors according to the neighbor index, and then obtain the boundary point features.
[0016] Further, the generation of labels for the medical point cloud includes:
[0017] Downsample the medical point cloud data obtained for each upsampling layer to obtain the corresponding downsampled point set, find the nearest kr neighbor points for each point in the downsampled point set, and assign the mean value of the label values of the kr neighbor points of each point to the point as the label of the point.
[0018] Further, in the infoNCE loss function module with an adaptive temperature value, the temperature value is adaptively adjusted in the following manner:
[0019]
[0020] where τ t+1 is the temperature value of the next round, τt is the temperature value of the current round, Loss pre is the loss value of the previous iteration, Loss t is the loss value of the current iteration, and β is the adjustment factor.
[0021] Further, the Loss pre and Loss t are obtained according to the infoNCE loss function calculation formula, and the temperature value in the infoNCE loss function calculation formula is adaptively adjusted according to Formula 1.
[0022] On the other hand, the present invention proposes a medical point cloud segmentation system based on multi-layer boundary point contrast learning, and the system is specifically configured to:
[0023] Segment the medical point cloud with a point cloud segmentation network based on multi-layer boundary point contrast learning; the point cloud segmentation network based on multi-layer boundary point contrast learning includes a boundary point contrast module and an infoNCE loss function module with an adaptive temperature value; the boundary point contrast module is used to obtain boundary point features and neighbor point features of boundary points based on the upsampled point cloud coordinates and features of each upsampling layer; the infoNCE loss function module with an adaptive temperature value is used to obtain a loss function value based on the adaptively adjusted temperature value, the boundary point features and neighbor point features of boundary points obtained by the boundary point contrast module, so as to perform contrast learning on the downsampling results of each upsampling layer.
[0024] The present invention also proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the medical point cloud segmentation method based on multi-layer boundary point contrast learning described in any one of the above.
[0025] The present invention also proposes a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the medical point cloud segmentation method based on multi-layer boundary point contrast learning described in any one of the above.
[0026] Compared with the prior art, the beneficial effects of the present invention are:
[0027] The present invention adds a boundary point contrast module to supervise the sampling of medical point clouds (such as oral medical point clouds) layer by layer. This module can extract more and more accurate deep semantic features. Transmit richer and more accurate deep semantic information to the upsampling layer, making the representation of the upsampling more accurate. This structure increases the network convergence speed to 2 times the original, and the IOU accuracy increases by 0.8%.
[0028] The present invention proposes an adaptive temperature value adjustment function (Formula 1). The temperature value is set efficiently and accurately through this function. Specifically, this function automatically obtains the temperature value of the next round based on the loss value of the previous iteration, the loss value of the current iteration, the temperature value of the current round, and the adjustment factor, rather than manually setting the temperature value. The model convergence is accelerated by adaptively adjusting the temperature value. Description of the Drawings
[0029] Figure 1 It is a logical schematic diagram of the point cloud segmentation network based on multi-layer boundary point contrast learning provided by the embodiment of the present invention;
[0030] Figure 2 It is a structural schematic diagram of the point cloud segmentation network based on multi-layer boundary point contrast learning provided by the embodiment of the present invention;
[0031] Figure 3Internal structure schematic diagram of the boundary point comparison module provided by the embodiment of the present invention;
[0032] Figure 4 Example of the result diagram of the boundary point comparison module provided by the embodiment of the present invention. Detailed implementation manners
[0033] The following further explains and illustrates the present invention in conjunction with the accompanying drawings and specific embodiments:
[0034] A medical point cloud segmentation method based on multi-layer boundary point contrast learning of the present invention includes:
[0035] Segmenting the medical point cloud by a multi-layer boundary point contrast learning point cloud segmentation network (Multi-layer Boundary Points Contrastive Learning network, MBCL); the multi-layer boundary point contrast learning point cloud segmentation network includes a boundary point contrast (point_contrast, PC) module and an Adaptive Temperature infoNCE Loss (AT-INFO) module; the boundary point contrast module is used to obtain boundary point features and neighbor point features of boundary points based on the upsampled point cloud coordinates and features of each upsampling layer; the Adaptive Temperature infoNCE loss function module is used to obtain a loss function value based on the adaptively adjusted temperature value, the boundary point features and neighbor point features obtained by the boundary point contrast module, so as to perform contrast learning on the upsampling results of each upsampling layer.
[0036] As an implementable manner, the medical point cloud uses oral three-dimensional laser point cloud.
[0037] The logic diagram of the medical point cloud segmentation network with multi-layer boundary point contrast learning is as Figure 1 shown. Specifically, first, the boundary point contrast module finds the neighbor points of the downsampled points of each layer (upsampling layer), and calculates the point label by the mean value of the neighbor labels. Then, the boundary points of each layer are found through the sampling point labels of each layer. By calculating the loss function through the features of the boundary points themselves and the neighbor features, contrast learning can be performed on the downsampling process of each layer (upsampling layer), obtaining more semantic information, and finally enhancing the accuracy of boundary point segmentation. Finally, the Adaptive Temperature infoNCE loss function module further improves the convergence efficiency of the AI network (MBCL) by efficiently calculating the temperature function. In this way, the segmentation of the oral three-dimensional laser point cloud can be realized.
[0038] The following specifically introduces each module.
[0039] 1) Boundary point contrast module
[0040] The specific implementation of the point cloud segmentation network (MBCL) for multi-layer boundary point contrast learning is as follows Figure 2 shown. The boundary point contrast module is connected to PointNet / PointNet++. First, Figure 2 the sa1-sa4 downsampling modules (layers) and fp1-fp4 upsampling modules (layers) in it are the original network frameworks of the PointNet / PointNet++ semantic segmentation network. In the present invention, the boundary point contrast module is connected to each upsampling layer (Point Feature Propagation, FP). Specifically, fp1-fp4 are respectively connected to pc1-pc4. Then, the inputs of the multiple pc1-pc4 modules are the upsampled points and features of fp1-fp4, and the outputs of the pc1-pc4 modules are the boundary point features and the neighbor point features of the boundary points.
[0041] The input of the boundary point contrast module is the upsampled points and features of each layer of FP; the output is the boundary point features and the features of the neighbor points of the boundary points. The boundary point contrast module is mainly divided into five parts, and the flowchart is as Figure 3 shown. Specifically, it is described as follows:
[0042] S1. Obtain the input point cloud and features: Obtain the point cloud coordinates and features of the FP upsampling layer.
[0043] S2. Calculate the (pseudo) label of each point: The input of step S2 is the n-row * 3-column (n is the number of original points) original point cloud p_from of each FP upsampling layer, the current layer's original point cloud, and the m-row * 3-column (m is the number of sampled points) downsampled point set p_to and the number kr of neighbors to be examined. The output of step S2 is the m * cls (m cls-dimensional labels, cls is the result after one-hot encoding, essentially m labels) labels of the boundary points. Specifically: Find the known m-row * 3-column downsampled point set p_to in the known n-row * 3-column original point cloud p_from of this layer, and find the nearest kr neighbors for each point in the downsampled point set. Assign the mean value of the kr neighbor labels of each point to this point. The m * cls labels of the m points are equal to the mean value of the m * kr * cls labels of the m * kr neighbors of the m points (m points, each point has kr neighbors, and each neighbor label is a number with cls being 1) along the kr axis (due to one-hot encoding, it is m * cls). For example, for point p0, there are 4 neighbors, so the label value of p0 is the mean value of the 4 neighbor labels. Why must pseudo labels be used? The reason is that there are no labels for multiple downsamplings and upsamplings, but there are labels for the original point cloud. Therefore, the pseudo label technology based on the original point cloud (the mean value of neighbor points is used as the label value of this point) is used.
[0044] S3, KNN query: Use KNN query to find the indices of the nearest neighbor points for each point.
[0045] S4, Remove self-loops: Remove the self-loops (i.e., itself) in the neighbor indices. Among the neighbors obtained according to the distance, the point itself will be included and should be removed.
[0046] S5, Obtain neighbor labels and features: Obtain the labels and features of the neighbors according to the neighbor indices. Principle and implementation: For the neighbor set of each point, find the points whose labels are different from that of itself. For example, for a point p0 with a label value of 1, if there is a point with a label value of 0 in the numerous neighbor sets, then the point p0 is a boundary point.
[0047] As an example, Figure 4 For the result graph obtained by the boundary point comparison module, the downsampled boundary points of the first layer (upsampling layer) to the fourth layer (upsampling layer) are the upper left, upper right, lower left, and lower right in turn.
[0048] The advantages of the boundary point comparison module are as follows:
[0049] 1. Eliminate manual annotation. In step S2, the label of each point after upsampling is calculated, and step S2 is the premise of layer-by-layer boundary point comparison learning. By calculating the boundary points of the labeled points in each sampling layer through S3 - S5 and using the calculated labels instead of manual annotation, deep learning can be carried out without manual annotation. This improves efficiency, reduces errors, and at the same time, can also improve the model prediction accuracy.
[0050] 2. Enhance the feature representation ability: The layer-by-layer boundary point comparison module prompts the model to learn more robust and discriminative feature representations. By training on positive sample pairs (boundary point feature - boundary point feature) and negative sample pairs (boundary point feature - feature of the boundary point's neighbor), the distance between different category features is increased, and the distance between the same category features is reduced, making the boundary point prediction more accurate. The layer-by-layer boundary point comparison module can maintain a high recognition accuracy under different perspectives, noise interference, etc.
[0051] 3. Improve the generalization ability: Since the boundary point comparison module performs supervised learning on each downsampled layer, better extracts and represents the deep semantic information of the point cloud, introducing the layer-by-layer boundary point comparison module helps to improve the generalization ability of the model, making it perform more stably when facing unseen data. This is because contrastive learning encourages the model to learn the essential features of the data rather than the surface details, thus reducing the risk of overfitting.
[0052] 2) InfoNCE loss function module with adaptive temperature value
[0053] According to Loss t / Loss preAdjust the temperature according to the (rate of change of the loss value). If the loss decreases slowly, a larger adjustment may be required; conversely, the adjustment amplitude gradually decreases.
[0054]
[0055] In Formula 1 (the function for adaptively adjusting the temperature value), τ t+1 is the temperature value for the next round, and τ t is the temperature value for the current round. Loss pre is the loss value of the previous iteration; Loss t is the loss value of the current iteration. β is the adjustment factor.
[0056] It should be noted that the said Loss pre and Loss t are obtained according to the calculation formula of the infoNCE loss function, and the temperature value in the calculation formula of the infoNCE loss function is adaptively adjusted according to Formula 1.
[0057] In Formula 1 of the present invention, the temperature is dynamically adjusted by the change of the rate of change of the loss value. It controls the attenuation or growth of the temperature according to the change of the loss. Usually, the temperature is adjusted according to the decrease or increase of the loss. The above strategy has its specific advantages as follows:
[0058] 1. Intelligent adjustment: As the training process progresses, the change of the loss usually shows regularity (for example, the loss gradually decreases). By dynamically adjusting the temperature according to the change of the loss, the model can automatically adapt to the training stage without too much manual intervention. Avoid premature temperature reduction: If the temperature drops prematurely, it may cause the model to lack sufficient exploration and converge prematurely. Adjusting the temperature according to the change of the loss allows the model to have more flexibility in the initial stage of training and gradually find a suitable learning strategy.
[0059] 2. Accelerate convergence. Reduce the temperature when the loss decreases: When the loss decreases, the model gradually finds better feature representations. At this time, reducing the temperature can help the model focus more on fine feature learning and avoid excessive "exploration". Increase the temperature when the loss increases: When the loss increases, the increase in temperature helps to enhance the fault tolerance of the model, avoid getting stuck in bad local minima during the training process, and provide more "exploration" opportunities.
[0060] 3. Reduce the risk of overfitting: Dynamic temperature adjustment helps to balance the exploration and exploitation of the model. When the loss decreases slowly, the increase in temperature prompts the model to maintain a certain degree of exploration, which helps to avoid overfitting.
[0061] 4. Smoother training process: The adjustment of temperature can smooth the training process according to the loss fluctuation, making the optimization process more stable and avoiding overly drastic gradient updates or temperature mutations.
[0062] Based on the above embodiments, the present invention further provides a medical point cloud segmentation system based on multi-layer boundary point contrast learning, which is specifically used for:
[0063] Segmenting the medical point cloud by a point cloud segmentation network based on multi-layer boundary point contrast learning; the point cloud segmentation network based on multi-layer boundary point contrast learning includes a boundary point contrast module and an infoNCE loss function module with an adaptive temperature value; the boundary point contrast module is used to obtain boundary point features and neighbor point features of boundary points based on the upsampled point cloud coordinates and features of each upsampling layer; the infoNCE loss function module with an adaptive temperature value is used to obtain a loss function value based on the adaptively adjusted temperature value, the boundary point features and neighbor point features of boundary points obtained by the boundary point contrast module, so as to perform contrast learning on the downsampling results of each upsampling layer.
[0064] Based on the above embodiments, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the medical point cloud segmentation method based on multi-layer boundary point contrast learning described in any one of the above.
[0065] Based on the above embodiments, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the medical point cloud segmentation method based on multi-layer boundary point contrast learning described in any one of the above.
[0066] In summary, through the boundary point contrast module, the present invention does not need to label and calculate the tag value and determine the boundary point, enhances the feature representation ability, and improves the generalization ability. Through the adaptive temperature value function of AT-INFO, intelligent adjustment is carried out, the convergence is accelerated, the risk of overfitting is reduced, and the training is smoother.
[0067] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.
Claims
1. A medical point cloud segmentation method based on multi-layer boundary point contrast learning, characterized in that Including: Segmenting medical point clouds using a point cloud segmentation network based on multi-layer boundary point contrast learning; the point cloud segmentation network based on multi-layer boundary point contrast learning includes a boundary point contrast module and an infoNCE loss function module with an adaptive temperature value; the boundary point contrast module is used to obtain boundary point features and neighbor point features of boundary points based on the upsampled point cloud coordinates and features of each upsampling layer; the infoNCE loss function module with an adaptive temperature value is used to obtain a loss function value based on the adaptively adjusted temperature value, the boundary point features and neighbor point features of boundary points obtained by the boundary point contrast module, so as to perform contrast learning on the downsampling results of each upsampling layer.
2. The medical point cloud segmentation method based on multi-layer boundary point contrast learning according to claim 1, wherein The point cloud segmentation network based on multi-layer boundary point contrast learning further includes a plurality of downsampling layers and a plurality of upsampling layers.
3. The medical point cloud segmentation method based on multi-layer boundary point contrast learning according to claim 2, wherein The boundary point contrast module is set in one-to-one correspondence with the upsampling layer, and the output of each upsampling layer is used as the input of a boundary point contrast module.
4. The medical point cloud segmentation method based on multi-layer boundary point contrast learning according to claim 1, wherein Specifically, the boundary point contrast module is used for: Obtaining the medical point cloud coordinates and features of each upsampling layer; Generating labels for the medical point cloud, and generating boundary points for each layer based on the labels; Using KNN query to find the nearest neighbor point index of each point in the medical point cloud; Removing the self-loop in the nearest neighbor point index of each point to obtain the neighbor index corresponding to each point; Obtaining the labels and features of neighbors according to the neighbor index, and further obtaining boundary point features.
5. The medical point cloud segmentation method based on multi-layer boundary point contrast learning according to claim 4, characterized in that, The generating labels for the medical point cloud includes: Downsampling the medical point cloud data obtained for each upsampling layer to obtain a corresponding downsampled point set, finding the nearest kr neighbor points for each point in the downsampled point set, and assigning the average value of the label values of the kr neighbor points of each point to this point as the label of this point.
6. The medical point cloud segmentation method based on multi-layer boundary point contrast learning according to claim 1, characterized in that, In the infoNCE loss function module with an adaptive temperature value, the temperature value is adaptively adjusted in the following manner: Among them, τ t+1 is the temperature value of the next round, and τ t is the temperature value of the current round. Loss pre is the loss value of the previous iteration, and Loss t is the loss value of the current iteration. β is the adjustment factor.
7. The medical point cloud segmentation method based on multi-layer boundary point contrast learning according to claim 6, wherein The said Loss pre and Loss t are obtained according to the calculation formula of the infoNCE loss function, and the temperature value in the infoNCE loss function calculation formula is adaptively adjusted according to Formula 1.
8. A medical point cloud segmentation system based on multi-layer boundary point contrast learning, characterized in that, Specifically, the system is used for: Segmenting medical point clouds using a point cloud segmentation network based on multi-layer boundary point contrast learning; the point cloud segmentation network based on multi-layer boundary point contrast learning includes a boundary point contrast module and an infoNCE loss function module with an adaptive temperature value; the boundary point contrast module is used to obtain boundary point features and neighbor point features of boundary points based on the upsampled point cloud coordinates and features of each upsampling layer; the infoNCE loss function module with an adaptive temperature value is used to obtain a loss function value based on the adaptively adjusted temperature value, the boundary point features and neighbor point features of boundary points obtained by the boundary point contrast module, so as to perform contrast learning on the downsampling results of each upsampling layer.
9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the medical point cloud segmentation method based on multi-layer boundary point contrast learning according to any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the medical point cloud segmentation method based on multi-layer boundary point contrast learning according to any one of claims 1 to 7.