Rib three-dimensional reconstruction method and system based on point cloud upsampling

Abstract

The application discloses a rib three-dimensional reconstruction method and system based on point cloud upsampling, and the method comprises the following steps: step 1, rib point cloud data preparation; step 2, rib point cloud data denoising; step 3, rib point cloud data semantic segmentation; step 4, rib point cloud data upsampling; and step 5, rib point cloud data three-dimensional reconstruction. The three-dimensional reconstruction of the point cloud can capture the details and morphology of an object, the data fusion of multiple perspectives can improve the reconstruction accuracy and robustness, thereby reducing the error in the reconstruction process, and fine adjustment can be performed in the point cloud data processing and reconstruction process, which is beneficial to generating a high-precision rib three-dimensional model.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a method and system for three-dimensional reconstruction of ribs based on point cloud upsampling. Background Technology

[0002] The ribs are a vital and indispensable part of the human skeletal structure. Their primary function is to protect vital internal organs such as the heart, lungs, and liver. Beyond protecting these organs from external injury, the ribs also participate in various physiological functions, including respiration, body support, and movement. Three-dimensional rib reconstruction uses computer-aided design and imaging techniques to transform two-dimensional images of the ribs into three-dimensional models, enabling a better understanding of human anatomy and disease diagnosis. This technology has significant clinical and research value, helping doctors better understand the structure and function of the ribs and improving the efficiency and accuracy of diagnosis, treatment, and research.

[0003] Existing 3D rib reconstruction methods first acquire multi-layered, multi-angle rib images using CT scans to obtain slice images of the body's interior. These slice data are then preprocessed to improve the accuracy and quality of the 3D reconstruction, and the slice images are segmented to obtain the target region separated from the background. Finally, a voxel reconstruction method based on the slice images is used to convert the 2D slice images into 3D voxel data, forming a 3D model. While existing methods provide relatively intuitive and comprehensive rib information, aiding medical diagnosis and treatment, they have several drawbacks: First, the process requires multiple steps such as preprocessing, segmentation, and registration, relying heavily on the operator's technical experience, increasing the instability of 3D rib reconstruction. Second, this method requires significant time and cost, thus limiting its application. Finally, the complex rib morphology and significant interference from surrounding tissues create a bottleneck in the accuracy of 3D rib reconstruction.

[0004] Based on the above situation, this invention designs a novel three-dimensional reconstruction method for ribs. This method can obtain stable three-dimensional reconstruction results for ribs, and the reconstruction inference time and reconstruction accuracy can meet the current level of medical diagnosis and treatment, which can greatly improve the quality and effect of current medical treatment involving ribs. Summary of the Invention

[0005] The existing voxel reconstruction methods based on rib slice images have limitations due to their low resolution and inability to directly obtain true 3D geometric information. Furthermore, they require manual annotation or segmentation of rib medical images. To address these limitations, this invention proposes a rib 3D reconstruction method and system based on point cloud upsampling. This invention captures the details and shape of the object through point cloud 3D reconstruction, improves reconstruction accuracy and robustness through multi-view data fusion, thereby reducing errors during reconstruction. Fine-tuning can be performed during point cloud data processing and reconstruction, facilitating the generation of high-precision rib 3D models. Compared to traditional image reconstruction, point cloud 3D reconstruction has advantages such as smaller data volume and simpler algorithms. It avoids the inefficiency of voxel data containing position, size, and attributes, and eliminates the need for multiple steps such as image segmentation, feature extraction, and 3D reconstruction required by traditional image reconstruction, thus rapidly generating 3D visualization models. This method is more efficient than traditional 3D reconstruction techniques.

[0006] The present invention adopts the following technical solution:

[0007] A 3D rib reconstruction method based on point cloud upsampling, comprising the following steps:

[0008] Step 1: Preparation of rib point cloud data;

[0009] Step 2: Denoising rib point cloud data;

[0010] Step 3: Semantic segmentation of rib point cloud data;

[0011] Step 4: Upsampling of rib point cloud data;

[0012] Step 5: 3D reconstruction of rib point cloud data.

[0013] Preferably, step 1: rib point cloud data preparation

[0014] First, based on the publicly available CT datasets RibFrac and RibSeg, the RibSeg dataset, which has unified rib markings, was manually labeled with different voxel values ​​for different ribs, thus achieving independent labeling of 24 ribs. Then, the 3D CT dataset was initially filtered for non-rib voxels by setting the HU threshold range to [200, 1000], resulting in a voxel mesh containing rib voxels. Finally, the rib data with complex geometry was processed using a point cloud generation algorithm to obtain point cloud data with high accuracy and completeness. The mask was then saved as point cloud labels point by point to prepare data for subsequent point cloud segmentation model training. At the same time, the point cloud data was divided into spatial coordinate regions to form a correspondence between the original point cloud and sparse point cloud data, preparing data for subsequent point cloud upsampling model training.

[0015] The detailed steps of the further optimized point cloud generation algorithm are as follows:

[0016] 1. The voxel mesh is converted into surface mesh data using mesh generation methods commonly used in the field of 3D graphics processing.

[0017] 2. Perform topology processing on the surface mesh to remove unnecessary mesh data and retain the topology of the surface mesh.

[0018] 3. Traverse each vertex of the surface mesh and convert the vertices into point cloud data. Specifically, the following methods can be used:

[0019] a. For each vertex, calculate its normal vector on the surface mesh.

[0020] b. Translate the vertex a certain distance along the direction of the normal vector to obtain a point in the point cloud data.

[0021] c. Repeat step b to obtain multiple points, thus forming point cloud data.

[0022] Preferably, step 2: denoising the rib point cloud data

[0023] The rib point cloud data was obtained from CT scans, and this data contains noise and errors. To improve the accuracy, visualization, and analysis efficiency of the subsequent rib model, denoising processing is necessary to obtain high-quality rib point cloud data. First, for each point, the classic machine learning algorithm KNN (K-Nearest Neighbors) is used to adaptively select neighboring points. The point density and average distance to each neighboring point are calculated by counting the number of neighboring points. Then, point density and average distance thresholds are set, and each point is checked to see if it simultaneously meets both the thresholds. If the conditions are not met, the point is identified as noise and deleted. Finally, the point cloud is smoothed using a Gaussian kernel smoothing method widely used in point cloud processing, resulting in smooth point cloud data.

[0024] Preferably, step 3: semantic segmentation of rib point cloud data

[0025] Step 3.1: Design a rib segmentation model based on point clouds

[0026] The high-quality point cloud data obtained after point cloud denoising is used to extract the features of the input point cloud data for rib segmentation, based on the Point Transformer point cloud model which is widely used in natural language processing and other sequence data processing tasks. This further removes noise points and achieves more accurate rib morphology analysis.

[0027] The Point Transformer point cloud model includes the Point Transformer module, the Transition Down module, and the Transition Up module.

[0028] The Point Transformer module integrates a self-attention layer, enabling dimensionality reduction and accelerating linear projection and residual connections. The input is a set of feature vectors with associated 3D coordinates. The Point Transformer module facilitates information exchange between these local feature vectors, generating new feature vectors as its output for all data points. Information aggregation adapts to both the content of the feature vectors and their layout in 3D.

[0029] The Transition Down module's key function is to reduce the number of points. First, it performs FPS (Farthest Point Sampling) on ​​the input point set P1 to obtain a well-distributed subset P2. Then, it uses KNN (K-Nearest Neighbors) to pool the feature vectors from P1 into P2, with each input feature undergoing a linear transformation. Finally, it performs batch normalization and ReLU (Reactive Luteinization), max-pooling the K neighbors of P2 to each point in P2.

[0030] The Transition Up module, which connects successive stages in the decoder, primarily maps features from the downsampled input point set P2 onto its superset P1. Each input point undergoes a linear layer, followed by batch normalization and ReLU, and then trilinear interpolation to map the P2 features onto the higher-resolution point set P1. These interpolated features from the previous decoder stage are combined with features from the corresponding encoder stage via skip connections.

[0031] Step 3.2: Design the training loss function

[0032] To train the Point Transformer model, a Cross Entropy loss function is defined to measure the difference between each point in the predicted point cloud and each point in the real point cloud. The smaller the loss, the closer the two point clouds are. The network parameters are optimized by minimizing the training loss function until the model converges, thereby obtaining the optimal training weights. The Cross Entropy loss function is shown in Equation (1).

[0033]

[0034] Where y is the rib label value. This represents the predicted value for rib segmentation.

[0035] Preferably, step 4: rib point cloud data upsampling

[0036] Step 4.1: Design a rib point cloud data upsampling model

[0037] The rib point cloud data obtained in step 3 has a low resolution. Therefore, the PUTransformer point cloud upsampling model based on transformer is selected to obtain more accurate high-resolution point cloud data and improve the visualization effect of the point cloud data.

[0038] The PU Transformer point cloud model includes the Transformer Encoder module, the Shuffle module, and the MLP module.

[0039] The Transformer Encoder module primarily consists of a Position Fusion module and a Shifted Channel Multi-head Self-Attention module. Position Fusion aggregates point cloud coordinate information P and point cloud feature information F separately to obtain local geometric information, then performs MLP encoding operations, followed by max pooling operations on the neighborhood to fuse two different types of contextual information. This module can not only encode the positional information of a set of unordered points for Transformer processing but also aggregate comprehensive local details to achieve accurate point cloud upsampling. The Shifted Channel Multi-head Self-Attention module overcomes the shortcomings of MSA, which only calculates point-wise correlations in each independent head and lacks the integration of channel-related information for shuffling-based upsampling.

[0040] The Shuffle module is used to rearrange point cloud data, increasing the diversity and generalization ability of the point cloud data.

[0041] The MLP module consists of multiple fully connected layers, each transforming the input vector into another vector. The MLP extracts feature information from the point cloud for subsequent point cloud processing tasks.

[0042] Step 4.2: Design the training loss function

[0043] To train the PU Transformer model, a Chamfer Distance loss function is defined to measure the distance between each point in the predicted point cloud and each point in the real point cloud. The smaller the loss, the better the match between the two point clouds. The network parameters are optimized by minimizing the training loss function until the model converges, thereby obtaining the optimal training weights. The Chamfer Distance loss function is shown in Equation (2).

[0044]

[0045] Where P is the predicted point cloud set and G is the actual point cloud set data.

[0046] Preferably, step 5: 3D reconstruction of rib point cloud data

[0047] High-resolution rib point cloud data is obtained through a point cloud upsampling model, enabling 3D reconstruction of the ribs for easy visualization of rib morphology. First, surface point sets are extracted from the point cloud data. Then, these surface point sets are transformed into a continuous triangular network model and optimized to obtain a smoother 3D model. Finally, post-processing operations such as stitching and stitching are performed on the generated 3D model to make it more realistic.

[0048] Preferably, the present invention further includes step 6: rib prediction and obtaining three-dimensional reconstruction results.

[0049] The CT scans of a single test set are converted into point cloud data. First, all point cloud data undergoes point cloud denoising processing. Then, 20K randomly sampled point clouds are fed into the point cloud semantic segmentation model with optimal weights loaded in step 3 to obtain point cloud predictions for the rib area. All partial point cloud predictions are then merged to form the point cloud prediction for the entire rib. A majority voting method is used for each prediction to improve accuracy. Next, a uniform number of point clouds are sequentially fed into the point cloud upsampling model with optimal weights loaded in step 4 to obtain high-resolution rib point cloud data. Finally, the rib point cloud predictions are processed by the 3D reconstruction algorithm in step 5 to obtain high-quality rib reconstruction results.

[0050] In practical applications, rib point cloud data is obtained from test CT (same as step 1), the point cloud data is denoised (same as step 2), and then the point cloud semantic segmentation is performed through the model verified in step 3 to obtain the rib segmentation result. The model verified in step 4 is used to upsample the point cloud to obtain high-resolution rib point cloud data. Finally, the refined and high-precision rib 3D reconstruction result is obtained through step 5.

[0051] This invention also discloses a rib three-dimensional reconstruction system based on point cloud upsampling using the above method, which includes the following modules:

[0052] Data preparation module: used for preparing rib point cloud data;

[0053] Data denoising module: used for denoising rib point cloud data;

[0054] Semantic segmentation module: used for semantic segmentation of rib point cloud data;

[0055] Data upsampling module: used for upsampling rib point cloud data;

[0056] 3D Reconstruction Module: Used for 3D reconstruction of rib point cloud data.

[0057] The beneficial effects of this invention are as follows: This invention designs a rib 3D reconstruction method and system based on point cloud upsampling. Firstly, the 3D reconstruction method of this invention can improve reconstruction accuracy and robustness through multi-view data fusion, utilizing the accurate positional information of point cloud data for reconstruction, rather than just pixel information, thus achieving higher reconstruction accuracy. Secondly, point cloud 3D reconstruction provides better interactivity; users can observe various angles and details of the object by rotating and zooming the point cloud data. Finally, because point cloud data only contains the positional information of each point, while traditional image reconstruction requires processing data including pixel color, brightness, and other information, the point cloud 3D reconstruction algorithm is simpler than traditional image reconstruction algorithms, thus making this 3D reconstruction method more efficient. This invention is convenient to use and easy to apply in clinical practice. Attached Figure Description

[0058] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. The accompanying drawings constitute a part of the specification and are used together with the embodiments of the present invention to explain the present invention, but do not constitute a limitation on the present invention.

[0059] Figure 1 The flowchart illustrates a preferred embodiment of the present invention for a rib three-dimensional reconstruction method based on point cloud upsampling.

[0060] Figure 2 This is a flowchart of the point cloud denoising algorithm in an embodiment of the present invention.

[0061] Figure 3 This is a schematic diagram of the Point Transformer point cloud semantic segmentation model structure used in this embodiment of the invention.

[0062] Figure 4 This is a schematic diagram of the PU Transformer point cloud upsampling model structure used in this embodiment of the invention.

[0063] Figure 5 This is a visualization of the rib point cloud data at various stages in an embodiment of the present invention.

[0064] Figure 6 This is a block diagram of a rib three-dimensional reconstruction system based on point cloud upsampling, which is a preferred embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with embodiments.

[0066] This embodiment presents a rib 3D reconstruction method based on point cloud upsampling, with the following specific steps:

[0067] Step 1: Preparation of Rib Point Cloud Data

[0068] First, based on the publicly available CT datasets RibFrac and RibSeg, the RibSeg dataset, which has unified rib labeling, was manually labeled with different voxel values ​​for different ribs, thus achieving independent labeling of all 24 ribs. Then, CT was used to initially filter non-rib voxels by setting the HU threshold range to [200, 1000], resulting in a voxel mesh containing rib voxels. Finally, a point cloud generation algorithm was used to effectively process the rib data with complex geometries, obtaining point cloud data with high accuracy and completeness. The mask was then saved point-by-point as point cloud labels, preparing data for subsequent point cloud segmentation model training. Simultaneously, the point cloud data was divided into spatial coordinate regions, corresponding to the original point cloud and sparse point cloud data, preparing data for subsequent point cloud upsampling model training. Figure 5 As shown in (a), the rib point cloud data obtained based on the point cloud generation algorithm contains all the information about the rib morphology, but there are also many noise points that need to be removed.

[0069] In this embodiment, the point cloud generation algorithm proceeds as follows: First, the voxel mesh is converted into surface mesh data using a mesh generation method. Then, topological processing is performed on the surface mesh to remove unnecessary mesh data and retain the topological structure of the surface mesh. Finally, each vertex of the surface mesh is traversed, and the vertex is converted into point cloud data.

[0070] Step 2: Denoising Rib Point Cloud Data

[0071] The rib point cloud data was obtained from CT scans, and this data contains noise and errors. To improve the accuracy, visualization, and analysis efficiency of subsequent rib models, denoising processing is required to obtain high-quality rib point cloud data. For example... Figure 2As shown, firstly, neighboring points are adaptively selected for each point using the KNN method. The point density is obtained by calculating the number of neighboring points, and the average distance to each neighboring point is calculated. Then, point density thresholds and average distance thresholds are set. For each point, it is determined whether it simultaneously meets the conditions of being greater than the point density threshold and less than the average distance threshold. If the conditions are not met, it is identified as a noise point and deleted. Finally, the point cloud is smoothed using a Gaussian kernel smoothing method to obtain smooth point cloud data. Figure 5 As shown in (b), after the rib point cloud data is processed by the denoising algorithm, a large number of noise points are removed, and the outline of the ribs can be clearly observed. However, there are still a small number of noise points and non-rib bone point clouds of the human body.

[0072] Step 3: Semantic segmentation of rib point cloud data, as detailed below:

[0073] Step 3.1: Design a rib segmentation model based on point clouds

[0074] like Figure 3 As shown, given a rib bone input point cloud, where N is the cardinality of the input point cloud, features are first learned through five stages of a feature encoder. The first stage consists of an MLP and a Point Transformer module, and the following four stages consist of a Transition Down module and a Point Transformer module. Each stage operates on a progressively decreasing set of sampled points. Then, feature extraction is performed through an MLP and a Point Transformer block, followed by four Transition Up blocks and Point Transformer blocks to recover a higher-resolution point set and generate a feature vector for each point. Finally, an MLP is used to obtain the final semantic segmentation result. Figure 5 As shown in (c), there are a small number of noisy points and point cloud data of non-rib bones of the human body. After the semantic segmentation model, the non-rib point cloud data is removed, and the neat rib data can be clearly displayed, but there is a discontinuity of ribs.

[0075] Step 3.2: Design the training loss function

[0076] By setting the AdaBound optimizer to optimize model parameters and the preheating cosine annealing learning rate update method, the Point Transformer model is trained. The Cross Entropy loss function is defined to measure the error between the model's predictions and the actual results, guiding model training and optimization, and evaluating model performance. The network parameters are optimized until the model converges by minimizing the training loss function, thereby obtaining the optimal training weights. The Cross Entropy loss function is shown in Equation (1).

[0077]

[0078] Where y is the rib label value. This represents the predicted value for rib segmentation.

[0079] Step 4: Upsampling of rib point cloud data

[0080] Step 4.1: Design a rib point cloud data upsampling model

[0081] like Figure 4 As shown, given a rib bone input point cloud, where N is the cardinality of the input point cloud, it first passes through an MLP module to generate an Nx16 feature map. Then, it passes through five cascaded Transformer encoders, with each output channel size as follows: 32→64→128→256. Each Positional Fusion module aggregates comprehensive local details to achieve accurate point cloud upsampling. Each Shifted Channel Multi-head Self-Attention module effectively changes the channels of each point feature to represent the new point without introducing additional parameters. Finally, the encoder output of size Nx256 undergoes a shuffling operation to boost the channels and construct a dense feature map of size rNx256 / r, where r is the upsampling scale. Another MLP is then applied to estimate the 3D coordinates of the upsampled point cloud (rNx3). Figure 5 As shown in (d), point cloud data with discontinuous rib morphology is obtained by using a point cloud upsampling model to obtain denser point cloud data with complete and continuous rib morphology, but a small number of noise points are introduced.

[0082] Step 4.2: Design the training loss function

[0083] The PUTransformer model is trained by optimizing model parameters using the RAdam optimizer and the preheating cosine annealing learning rate update method. A Chamfer Distance loss function is defined to measure the error between the model's predictions and the actual results, guiding model training and optimization, and evaluating model performance. The network parameters are optimized until the model converges by minimizing the training loss function, thus obtaining the optimal training weights. The Chamfer Distance loss function is shown in equation (2).

[0084]

[0085] Where P is the predicted point cloud set and G is the actual point cloud set data.

[0086] Step 5: 3D Reconstruction of Rib Point Cloud Data

[0087] High-resolution rib point cloud data is obtained through a point cloud upsampling model, enabling 3D reconstruction of the ribs for easy visualization of rib morphology. First, surface point sets are extracted from the point cloud data. Then, these surface point sets are transformed into a continuous triangular network model and optimized to obtain a smoother 3D model. Finally, post-processing operations such as stitching and stitching are performed on the generated 3D model to make it more realistic. Figure 5 As shown in (e), point cloud data with complete and coherent rib morphology and a small number of noise points are processed by three-dimensional reconstruction post-processing to remove the noise points and retain point cloud data with complete and refined rib morphology.

[0088] This embodiment also includes step 6: rib prediction and obtaining three-dimensional reconstruction results.

[0089] In practical applications, such as Figure 1 As shown, the test CT scan is converted into rib point cloud data (same as step 1), the point cloud data is denoised (same as step 2), and then the model verified in step 3 is used for point cloud semantic segmentation to obtain high-quality rib point cloud data. The model verified in step 4 obtains high-resolution rib point cloud data, and finally, step 5 yields a refined, high-precision rib 3D reconstruction result. Experiments show that running two GeForce 3090Ti GPUs on a Linux operating system for rib prediction requires only 15 seconds of inference time, while the existing voxel reconstruction method based on rib slice images requires 41 seconds of inference time, fully demonstrating the efficiency of the rib 3D reconstruction method based on point cloud upsampling of this invention.

[0090] like Figure 6 As shown, this embodiment discloses a rib three-dimensional reconstruction system based on point cloud upsampling using the above method, which includes the following modules:

[0091] Data preparation module: used for preparing rib point cloud data;

[0092] Data denoising module: used for denoising rib point cloud data;

[0093] Semantic segmentation module: used for semantic segmentation of rib point cloud data;

[0094] Data upsampling module: used for upsampling rib point cloud data;

[0095] 3D Reconstruction Module: Used for 3D reconstruction of rib point cloud data.

[0096] The above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for 3D reconstruction of ribs based on point cloud upsampling, characterized in that: Follow these steps: Step 1: Preparation of rib point cloud data; Step 2: Denoising rib point cloud data; Step 3: Semantic segmentation of rib point cloud data; Step 4: Upsampling of rib point cloud data; Step 5: 3D reconstruction of rib point cloud data; Step 1 is as follows: S1.1 Using the RibSeg public dataset with unified rib labeling, different voxel values ​​are labeled for different ribs, achieving independent labeling of 24 ribs; S1.

2. After initially filtering non-rib voxels by setting the HU threshold range of the 3D CT to [200, 1000], a voxel mesh containing rib voxels is obtained. S1.3 Process the rib data through the point cloud generation algorithm to obtain high-precision and complete point cloud data, and save the rib mask as point cloud labels by point correspondence. At the same time, divide the point cloud data into spatial coordinate regions to form the original point cloud and sparse point cloud data corresponding to each other. The point cloud generation algorithm steps are as follows: S1.3.

1. Convert the voxel mesh into surface mesh data using a mesh generation method; S1.3.

2. Perform topology processing on the surface mesh to remove unnecessary mesh data and retain the topological structure of the surface mesh; S1.3.

3. Traverse each vertex of the surface mesh and convert the vertices into point cloud data; Step S1.3.3 is as follows: a. For each vertex, calculate its normal vector on the surface mesh; b. Translate the vertex a certain distance along the direction of the normal vector to obtain a point in the point cloud data; c. Repeat step b to obtain multiple points and form point cloud data.

2. The rib three-dimensional reconstruction method based on point cloud upsampling as described in claim 1, characterized in that, Step 2 is as follows: S2.1 Adaptively select neighboring points using the KNN method. Obtain the point density by calculating the number of neighboring points within the distance range based on Euclidean distance. Calculate the Euclidean distance between the point and each neighboring point and then calculate the average distance. S2.2 Set a point density threshold and an average distance threshold. For each point, determine whether it simultaneously meets the conditions of being greater than the point density threshold and less than the average distance threshold. If the conditions are not met, it is judged as a noise point and deleted. S2.

3. The point cloud is smoothed using the Gaussian kernel smoothing method to obtain smooth point cloud data.

3. The rib three-dimensional reconstruction method based on point cloud upsampling as described in claim 2, characterized in that, Step 3 is as follows: S3.1 Design a rib segmentation model based on point cloud. The point cloud data obtained after the point cloud denoising step is used to select the PointTransformer point cloud model based on the neural network model transformer to extract the features of the input point cloud data for rib segmentation. The Point Transformer point cloud model includes the following modules: The Point Transformer module integrates a self-attention layer for dimensionality reduction and acceleration of linear projection and residual connections. It takes a set of feature vectors with relevant 3D coordinates as input, and facilitates information exchange between these local feature vectors to generate new feature vectors for all data points as its output. The Transition Down module is used to reduce the number of points. It performs FPS on the input point set P1 to obtain a well-distributed subset P2. The feature vectors of the subset P2 are then pooled from P1 to P2 using KNN. Each input feature undergoes a linear transformation. Finally, batch normalization and ReLU are performed to max pool the K neighbors of P2 to each point of P2. The Transition Up module connects successive stages in the decoder, mapping features from the downsampled input point set P2 onto its superset P1. Each input point is processed through a linear layer, then batch normalized and ReLU is performed, and then trilinear interpolation maps the P2 features onto the higher-resolution point set P1. Interpolated features from the previous decoder level are combined with features from the corresponding encoder level through skip connections. S3.2 Design the training loss function To train the Point Transformer point cloud model, a Cross Entropy loss function is defined to measure the difference between each point in the predicted point cloud and each point in the real point cloud. The smaller the loss, the closer the two point clouds are. The network parameters are optimized by minimizing the training loss function until the model converges, thereby obtaining the optimal training weights. The Cross Entropy loss function is shown in Equation (1). in, For rib label values, This represents the predicted value for rib segmentation.

4. The rib three-dimensional reconstruction method based on point cloud upsampling as described in claim 3, characterized in that, Step 4 is as follows: S4.1 Design a rib point cloud data upsampling model Select the PU Transformer point cloud upsampling model based on transformers, which includes the following modules: The Transformer Encoder module includes a Position Fusion module and a Shifted Channel Multi-head Self-Attention module. Position Fusion is used to aggregate point cloud coordinate information P and point cloud feature information F to obtain local geometric information. Through MLP encoding operation, max pooling is used to fuse two different types of context information in the neighborhood. The Shuffle module is used to rearrange point cloud data; The MLP module consists of multiple fully connected layers, each of which transforms the input vector into another vector. The MLP module extracts feature information from the point cloud for subsequent point cloud processing tasks. S4.2 Design the training loss function To train the PU Transformer point cloud upsampling model, a Chamfer Distance loss function is defined to measure the distance between each point in the predicted point cloud and each point in the real point cloud. The smaller the loss, the better the match between the two point clouds. The network parameters are optimized by minimizing the training loss function until the model converges, thereby obtaining the optimal training weights. The Chamfer Distance loss function is shown in Equation (2). Where P is the predicted point cloud set and G is the actual point cloud set data.

5. The rib three-dimensional reconstruction method based on point cloud upsampling as described in claim 4, characterized in that, Step 5 is as follows: S5.1 Extract the surface point set from the point cloud data to obtain the surface point set; S5.2 Transform the surface point set into a continuous triangular network model and optimize it to obtain a smoother 3D model; S5.3 Perform post-processing operations on the generated 3D model.

6. The rib three-dimensional reconstruction method based on point cloud upsampling as described in claim 5, characterized in that, It also includes step 6: rib prediction and obtaining three-dimensional reconstruction results.

7. The rib three-dimensional reconstruction method based on point cloud upsampling as described in claim 6, characterized in that, Step 6 is as follows: S6.

1. Convert the CT sample of a single test set into point cloud data, perform point cloud denoising processing on all point cloud data, and input 20K randomly sampled point cloud data into the point cloud semantic segmentation model with the optimal weights loaded in step 3 to obtain the point cloud prediction of the rib part. Merge all partial point cloud predictions into the point cloud prediction of the entire rib. At the same time, the majority voting method is used for each prediction to improve the prediction accuracy. S6.

2. Feed the point cloud data into the point cloud upsampling model with the optimal weights loaded in step 4 in a uniform manner to obtain high-resolution rib point cloud data. S6.

3. The rib point cloud prediction is used to obtain the rib reconstruction result through step 5.

8. A rib three-dimensional reconstruction system based on point cloud upsampling according to any one of claims 1-7, characterized in that: Includes the following modules: Data preparation module: used for preparing rib point cloud data; Data denoising module: used for denoising rib point cloud data; Semantic segmentation module: used for semantic segmentation of rib point cloud data; Data upsampling module: used for upsampling rib point cloud data; 3D Reconstruction Module: Used for 3D reconstruction of rib point cloud data.

Images