A spine segmentation method based on three-dimensional images

Abstract

The application discloses a spine segmentation method based on three-dimensional images, comprising the following steps: obtaining a posteroanterior digital reconstructed radiograph containing a spine; segmenting the posteroanterior digital reconstructed radiograph to obtain each spine region image and a non-spine region image; obtaining the minimum circumscribed rectangle of each spine in the spine region image, and obtaining the spatial key point of each spine in the three-dimensional image according to the center point; taking the spatial key point of each spine as the center, and segmenting and cutting the three-dimensional image along the spine height direction in the form of a set column to obtain each spine segment cut image; and splicing each spine segment cut image to obtain the segmentation result of the spine in the three-dimensional image. The application combines a 2D digital reconstructed radiograph as input, is more accurate in final result accuracy, greatly reduces prediction time consumption, and is faster in algorithm reasoning speed under the same hardware configuration.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for spine segmentation based on three-dimensional images. Background Technology

[0002] Currently, common medical image segmentation is mainly accomplished using traditional algorithms, such as thresholding and region growing. Traditional algorithms have high requirements for feature design, are difficult to model, have poor robustness, and are time-consuming.

[0003] In medical spinal surgery robots, accurate navigation for intraoperative spinal punctures requires precise segmentation of the regions containing each vertebral segment, and each segment needs unique instance information. Therefore, 3D segmentation of the human spine is necessary, but the following difficulties exist:

[0004] (1) Medical images are complex, the boundaries between vertebrae are not clear, and the vertebrae are relatively similar, which greatly interferes with the detection.

[0005] (2) When segmenting each vertebral segment in 3D, the boundaries must be strictly separated and there should be no overlap. The segmentation accuracy is required to be high.

[0006] (3) Each vertebral segment is required to meet accurate classification information. Since each vertebral segment is similar, classification is quite difficult. Summary of the Invention

[0007] Purpose of the invention: To address the above-mentioned shortcomings, this invention proposes a spine segmentation method based on three-dimensional images, which can greatly improve segmentation efficiency and accuracy.

[0008] Technical solution:

[0009] A spine segmentation method based on 3D images, comprising the following steps:

[0010] Obtain anteroposterior and lateral digitally reconstructed radiographic images containing the spine;

[0011] The anterolateral digital reconstructed radiographic images are segmented to obtain images of each spinal segment and non-spinal segment images.

[0012] Obtain the minimum bounding rectangle of the spine in the image of each vertebral segment, and obtain the spatial key points of each vertebral segment in the three-dimensional image based on its center point.

[0013] Using the spatial key points of each vertebral segment as the center, the three-dimensional image is segmented along the height direction of the spine in the form of a set columnar body to obtain segmented slice images of each vertebral segment.

[0014] The segmented images of each vertebra are stitched together to obtain the segmentation result of the spine in the three-dimensional image.

[0015] The specific steps for obtaining the minimum bounding rectangle of the spine in each spinal segment region image are as follows:

[0016] Based on the coordinates of each pixel in the segmented lateral digital reconstructed radiographic images of each vertebral region, the maximum coordinate difference between the two coordinate axes of the corresponding image's two-dimensional coordinate system for that vertebral segment is used as the length and width of the minimum bounding rectangle, thereby obtaining the minimum bounding rectangle of the spine in each vertebral region image.

[0017] Based on the center point of the minimum bounding rectangle, the spatial key points of each vertebra in the three-dimensional image are calculated using the least squares method.

[0018] Obtaining the coordinates of each pixel in the segmented anteroposterior and lateral digitally reconstructed radiographic images of each spinal segment involves the following steps:

[0019] Pixel segmentation is performed based on the pixel level of the images of each spinal segment and the images of non-spinal segments. Then, the resulting images of each spinal segment are subjected to binarization masking to obtain the pixel matrix of each spinal segment image and the coordinates of each pixel in each spinal segment image.

[0020] The segmentation of the anterolateral digital reconstructed radiographic images is performed using a 2D neural network model, which is trained using several 2D spinal images.

[0021] The defined column is a cuboid, and the length, width, and height of the cuboid correspond to the left-right, front-back, and height directions of the human body, respectively, and are all larger than the maximum dimension of each vertebra in the corresponding direction.

[0022] After segmenting the 3D image along the height of the spine, the obtained segmented slice images of each spine segment are binarized to obtain a 3D mask of the single spine slice image. The 3D masks of the obtained single spine slice images are then stitched together to obtain the segmentation result of the spine in the 3D image.

[0023] The 3D segmentation model is used to train the segmented slice images of each vertebra obtained from the 3D image segmentation.

[0024] The acquisition of anteroposterior and lateral digital reconstructed radiographic images containing the spine is performed in the following manner:

[0025] The system can generate anteroposterior and lateral digital reconstructed radiographs by inputting a 3D image containing the spine, or directly input an anteroposterior and lateral digital reconstructed radiograph containing the spine.

[0026] Beneficial effects:

[0027] (1) This invention employs multimodal input, including 2D digital reconstructed radiographic images (DRR images) and sliced ​​images of 3D images, resulting in higher accuracy of the final result. Specifically, using 2D digital reconstructed radiographic images as input, the invention detects the center point of the reconstructed images of each vertebra in the anteroposterior and lateral digital images and the outer contour of each vertebra, achieving high localization and segmentation accuracy and strong robustness. Using sliced ​​images of 3D images as input, the invention performs semantic segmentation on each vertebra in the sliced ​​images of 3D images, achieving high localization and segmentation accuracy and strong robustness.

[0028] (2) The present invention combines 2D digital reconstruction radiographic images as input, which greatly reduces the prediction time and the algorithm reasoning speed is faster on the same hardware configuration. Attached Figure Description

[0029] Figure 1 This is a flowchart of the present invention;

[0030] Figure 2 This is a schematic diagram of a three-dimensional image;

[0031] Figure 3 A schematic diagram of a digitally reconstructed radiographic image;

[0032] Figure 4 Schematic diagram of lateral digital reconstructed radiographic image;

[0033] Figure 5 This is a schematic diagram of segmenting a digitally reconstructed radiographic image from the front and lateral views.

[0034] Figure 6 To obtain a schematic diagram of the key spatial points of each vertebral segment;

[0035] Figure 7 This is a schematic diagram of each slice of a 3D image;

[0036] Figure 8 A schematic diagram of generating masks for each slice of a 3D image;

[0037] Figure 9 This is the final three-dimensional segmentation result of the spine. Detailed Implementation

[0038] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0039] The present invention provides a spinal segmentation method based on three-dimensional images, as follows: Figure 1 As shown, the steps include:

[0040] (1) Obtain anteroposterior and lateral digital reconstructed radiographic images containing the spine;

[0041] In this invention, anteroposterior and lateral digitally reconstructed radiographic images can be generated from input three-dimensional images containing the spine. The input three-dimensional images are original clinical CT slice DICOM data containing the spine, such as... Figure 2 As shown, based on the characteristics of CT slice DICOM data, the DRR process can be simulated to automatically generate anteroposterior and lateral CT DRR images;

[0042] In this invention, the aforementioned frontal and lateral digital reconstructed radiographic images can also be directly input;

[0043] (2) The anteroposterior and lateral digital reconstructed radiographic images obtained in step (1) are segmented to obtain images of each spinal region and non-spinal region.

[0044] In this invention, a pre-trained 2D segmentation model is used to segment anteroposterior and lateral digitally reconstructed radiographic images. The pre-trained 2D segmentation model is a neural network model, which is trained using labeled vertebrae in several 2D spinal images as samples. The neural network used can be a 2D convolutional neural network. In this invention, the 2D segmentation model can greatly improve the accuracy and prediction speed of key point detection, and increase the feasibility of algorithm implementation.

[0045] (3) Use the softmax function to perform binary classification on the pixel level of the images of each spinal region and the images of non-spine regions to perform pixel segmentation of the images of each spinal region and the images of non-spine regions, and perform binarization masking on the segmented spinal regions to obtain the pixel matrix of each spinal region.

[0046] (4) Based on the pixel matrix of each spinal segment obtained in step (3), calculate the coordinates of each pixel point in each spinal segment in the anteroposterior and lateral digital reconstructed radiographs. Use the maximum coordinate difference between the two coordinate axes of the corresponding two-dimensional coordinate system of the digital reconstructed radiographs for each spinal segment as the length and width of the minimum bounding rectangle of that spinal segment. Use this minimum bounding rectangle to select the spinal segment in the anteroposterior and lateral digital reconstructed radiographs of the patient. Figure 3 , 4 As shown;

[0047] (5) The center point of the smallest bounding rectangle of each vertebral segment is used as the center point of the corresponding vertebral segment in the frontal and lateral digital reconstructed radiographs, such as... Figure 5 As shown, for example, the center point A of a certain vertebra can be detected, and its anterior center point A1(x,y) and lateral center point A2(z,y) can be detected respectively.

[0048] Based on the obtained center points, the spatial positions of each vertebra in the three-dimensional image slice are calculated using the least squares method, such as... Figure 6As shown, the spatial key points of each segment of the spine are obtained.

[0049] (6) Using the spatial key points of each vertebra in the three-dimensional image slice image obtained in step (5) as the center, the three-dimensional image slice image is segmented along the direction of the spine height in a set cuboid as the unit to obtain the segmented slice image of each vertebra, as shown in 7.

[0050] The length, width, and height of the cuboid are set to correspond to the left-right, front-back, and height directions of the human body, respectively, and are all larger than the maximum size of each vertebra in the corresponding direction. The cuboid is used to cut the body into segments. In addition to the target vertebra, other vertebrae may also be within the cutting range.

[0051] (7) The segmented slice images of each spine obtained in step (6) are segmented and binarized using a pre-trained 3D segmentation model to obtain a 3D mask for each spine slice image, such as... Figure 8 As shown;

[0052] In this invention, the 3D segmentation model can segment a single vertebra. By using the labeled vertebrae in the 3D spinal image as samples for training, interference from other vertebrae images or impurities can be eliminated, resulting in higher accuracy and robustness.

[0053] To improve the prediction performance of the 3D segmentation model, this invention adopts several mainstream evaluation metrics, namely Dice Similarity Coefficient (DSC), Accuracy, and Recall.

[0054] Let A and B represent the truth value and the inference result of the semantic segmentation model, respectively. Then the formulas for these evaluation criteria are as follows:

[0055]

[0056]

[0057]

[0058] The Dice similarity coefficient (DSC), accuracy, and recall values ​​range from [0,1]. A higher Dice similarity coefficient (DSC) indicates a better segmentation effect. Recall reflects the positive sample count after segmentation. A higher recall indicates a higher number of correctly segmented targets. A higher accuracy indicates a higher proportion of correctly segmented targets.

[0059] (8) The 3D mask of the single spine slice image obtained in step (7) is stitched together to obtain the spine segmentation result of the 3D slice image, that is, the 3D segmentation mask result of the spine in the three-dimensional image.

[0060] This invention employs multimodal input, including 2D digitally reconstructed radiographic images (DRR images) and sliced ​​images of 3D images, resulting in higher accuracy of the final results. Furthermore, by combining 2D DRR images as input, prediction time is significantly reduced, and with the same hardware configuration, the algorithm's inference speed is faster, leading to higher accuracy of the final results.

[0061] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical solution of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.

Claims

1. A spine segmentation method based on three-dimensional images, characterized in that: Including the following steps: Obtain anteroposterior and lateral digitally reconstructed radiographic images containing the spine; The anterolateral digital reconstructed radiographic images are segmented to obtain images of each spinal segment and non-spinal segment images. Obtain the minimum bounding rectangle of the spine in the image of each vertebral segment, and obtain the spatial key points of each vertebral segment in the three-dimensional image based on its center point. Using the spatial key points of each vertebral segment as the center, the three-dimensional image is segmented along the height direction of the spine in the form of a set columnar body to obtain the segmented slice image of each vertebral segment. The length, width, and height directions of the set columnar body correspond to the left-right, front-back, and height directions of the human body, respectively, and are all greater than the maximum size of each vertebral segment in the corresponding direction. The segmented images of each vertebral segment are segmented and binarized to obtain a 3D mask of the single vertebral segment image. The 3D masks of the obtained vertebral segment images are then stitched together to obtain the segmentation result of the spine in the three-dimensional image.

2. The spine segmentation method based on three-dimensional images according to claim 1, characterized in that: The specific steps for obtaining the minimum bounding rectangle of the spine in each spinal segment region image are as follows: Based on the coordinates of each pixel in the segmented lateral digital reconstructed radiographic images of each vertebral region, the maximum coordinate difference between the two coordinate axes of the corresponding image's two-dimensional coordinate system for that vertebral segment is used as the length and width of the minimum bounding rectangle, thereby obtaining the minimum bounding rectangle of the spine in each vertebral region image.

3. The spine segmentation method based on three-dimensional images according to claim 1 or 2, characterized in that: Based on the center point of the minimum bounding rectangle, the spatial key points of each vertebra in the three-dimensional image are calculated using the least squares method.

4. The spine segmentation method based on three-dimensional images according to claim 2, characterized in that: Obtaining the coordinates of each pixel in the segmented anteroposterior and lateral digitally reconstructed radiographic images of each spinal segment involves the following steps: Pixel segmentation is performed based on the pixel level of the images of each spinal segment and the images of non-spinal segments. The resulting images of each spinal segment are then subjected to binarization masking to obtain the pixel matrix of each spinal segment image. Based on this matrix, the coordinates of each pixel in each spinal segment image are obtained.

5. The spine segmentation method based on three-dimensional images according to claim 1, characterized in that: The segmentation of the anterolateral digital reconstructed radiographic images is performed using a 2D neural network model, which is trained using several 2D spinal images.

6. The spine segmentation method based on three-dimensional images according to claim 1, characterized in that: The 3D segmentation model is used to train the segmented slice images of each vertebra obtained from the 3D image segmentation.

7. The spine segmentation method based on three-dimensional images according to claim 1, characterized in that: The acquisition of anteroposterior and lateral digital reconstructed radiographic images containing the spine is performed in the following manner: The system can generate anteroposterior and lateral digital reconstructed radiographs by inputting a 3D image containing the spine, or directly input an anteroposterior and lateral digital reconstructed radiograph containing the spine.

Images