CT image conversion method and application

Abstract

The method for converting the CT image of the lumbar fracture into the MR image comprises the following steps that S1, the CT image and the MR image of the lumbar fracture are processed, and image processing comprises image cutting, unification of the brightness and the contrast of the CT image and the MR image of different patients and unified pairing of the CT image and the MR image of the patients; s2, inputting the processed picture into a Self-pix model for training and learning for multiple rounds; s3, after model training is completed, the lumbar fracture CT image is input into the Self-pix, the generator can convert styles according to trained parameters, and the lumbar fracture CT image is converted into an MR image. The cutting requirements of CT and MR images are as follows: the front edge is the foremost edge of the vertebral body of the waist 4 or the waist 5, the rear edge is the rearmost direction of the spinous process, the upper edge is the uppermost part of the thoracic vertebra 12, and the lower edge is the lowermost part of the sacral vertebrae 1. The method is used for solving the problem that CT and DR cannot distinguish whether the lumbar fracture is fresh or old.

Description

[0001] This invention is an invention patent divisional application with application number CN202410216297.9, and its invention name is a method for converting CT images of lumbar fractures into MR images. The original application date is February 27, 2024. Technical Field

[0002] The invention relates to a method for converting a lumbar fracture CT image into an MR image. Background Art

[0003] Lumbar fracture is a relatively common type of spinal trauma, usually caused by external force acting on the spine, which may cause severe pain, limited movement and even nerve damage. In severe cases, it may even affect life functions. Lumbar fractures are divided into fresh fractures and old fractures. The treatment plans and prognosis of the two are completely different, so early diagnosis and treatment are crucial. MRI is the gold standard for diagnosing lumbar fractures, but it has the following defects:

[0004] 1. Long inspection time;

[0005] 2. The patient needs to maintain a relatively fixed posture, which is not suitable for patients with severe trauma;

[0006] 3. People with metal implants, such as pacemakers, or certain special diseases, such as claustrophobia, are not suitable for MRI examinations.

[0007] 4. The cost of MRI is relatively high (about 500 for one part), which creates a financial burden for the examiner.

[0008] Based on the above reasons, CT and DR are commonly used methods for the preliminary diagnosis of lumbar fractures, which have the advantages of being fast, convenient and economical, but cannot distinguish between fresh and old lumbar fractures, and the treatment plans for fresh and old lumbar fractures are completely different. Therefore, the present invention proposes a new deep learning method to convert lumbar fracture CT images into MR images, thereby combining the advantages of both CT and MR, which is beneficial for spinal surgeons to quickly diagnose lumbar fractures and accurately select treatment plans, and is beneficial to patients' postoperative recovery and long-term prognosis. Summary of the invention

[0009] The purpose of the present invention is to provide a method for converting CT images of lumbar fractures into MR images, so as to solve the problem that CT and DR cannot distinguish whether a lumbar fracture is fresh or old.

[0010] In order to solve the above problems, the technical solution of the present invention is:

[0011] The method for converting lumbar fracture CT images into MR images comprises the following steps:

[0012] S1: CT image and MR image processing of lumbar fractures, including image cropping, unifying the brightness and contrast of CT and MR images of different patients, and pairing the CT and MR images of patients;

[0013] S2: Input the processed images into the Self-pix model for multiple rounds of training;

[0014] S3: After the model training is completed, the lumbar fracture CT image is input into Self-pix. The generator will convert the style according to the trained parameters and convert the lumbar fracture CT image into an MR image.

[0015] The cropping requirements for CT and MR images are as follows: the anterior edge is the anterior edge of the L4 or L5 vertebra, the posterior edge is the posteriormost part of the spinous process, the upper edge is the uppermost part of the 12th thoracic vertebra, and the lower edge is the lowest part of the sacral vertebra.

[0016] The generator of the Self-pix model uses a residual network, which adopts a skip connection method.

[0017] The residual network is composed of multiple residual blocks, and 9 residual blocks are used in the Self-pix model.

[0018] The structure of the residual block includes the input passing through a convolution layer, instance normalization and activation function ReLU, then passing through another convolution layer and instance normalization, and finally adding the input and output and passing through the ReLU activation function again, and using a 1x1 convolution kernel in each residual block.

[0019] The Self-pix model uses spectral normalization on the discriminator in Pix2pix, where the generator of Pix2pix uses the U-net network. In the pix2pix model, the U-net downsampling structure is that the input first passes through a convolution layer and then a pooling layer, and the upsampling is performed through a deconvolution layer and a jump connection, using feature fusion to generate the output image.

[0020] Pix2pix's discriminator uses patchGAN, which adopts a multi-layer convolutional layer structure and adds the activation function Leaky ReLU after each convolution layer. It extracts features from each small patch of the input image through different convolution kernels and outputs a probability value between 0 and 1, indicating the probability that the patch is a real image.

[0021] The beneficial effects of the present invention are as follows: the present invention greatly improves the diagnostic efficiency of lumbar fractures, which is of great benefit to both the Department of Imaging and Spine Surgery. It not only improves the MR imaging efficiency of the Department of Imaging, but also assists spinal surgeons in the diagnosis and treatment selection of lumbar fractures. In severe cases where the spine is compressed by explosive lumbar fracture fragments, the literature reports that there are two golden decompression periods, 6-8 hours after trauma, and 24 hours after trauma. Due to the limitations of actual conditions, it is extremely difficult to carry out decompression within 6-8 hours. However, with the support and application of the present invention, the speed of diagnosis and treatment can be greatly improved, making it possible to complete decompression 6-8 hours after trauma. And the synthetic MR images of the present invention also show extremely high accuracy (97.97%) for the selection of diagnosis and treatment methods, which can assist spinal surgeons in deciding treatment plans and selecting the most suitable treatment plan for different patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below in conjunction with the accompanying drawings:

[0023] Figure 1 It is a schematic diagram of the Unet structure of Pix2pix of the present invention,

[0024] Figure 2 is a schematic diagram of the structure of the Self-pix of the present invention,

[0025] Figure 3 is the preprocessed image.

[0026] Figure 4 is the converted MR image. DETAILED DESCRIPTION

[0027] like Figures 1 to 4 As shown, the method for converting lumbar fracture CT images into MR images comprises the following steps:

[0028] S1: CT image and MR image processing of lumbar fractures, including image cropping, unifying the brightness and contrast of CT and MR images of different patients, and pairing the CT and MR images of patients;

[0029] S2: Input the processed images into the Self-pix network model for multiple rounds of training. Self-pix will continuously learn the features of lumbar fractured vertebrae and non-fractured vertebrae, and continuously optimize the generator and discriminator in this process.

[0030] S3: After the model training is completed, the lumbar fracture CT image is input into Self-pix. The generator will convert the style according to the trained parameters and convert the lumbar fracture CT image into an MR image.

[0031] The cropping requirements for CT and MR images are as follows: the anterior edge is the anterior edge of the L4 or L5 vertebra, the posterior edge is the posteriormost part of the spinous process, the upper edge is the uppermost part of the 12th thoracic vertebra, and the lower edge is the lowest part of the sacral vertebra.

[0032] The generator of the Self-pix model uses a residual network (ResNet) structure, which introduces residual blocks to solve the gradient vanishing and gradient exploding problems in deep network training. In traditional deep neural networks, information needs to be transmitted layer by layer, and each layer needs to learn appropriate feature representations. However, as the number of network layers increases, the stacking of convolutional layers will cause the gradient to gradually decrease with back propagation, causing the network performance to saturate or decline.

[0033] To solve this problem, the residual network uses a skip connection. Skip connections allow some layers in the network to skip a part of the layer directly and add the input of the previous layer directly to the output of the next layer. In this way, the network can improve the feature representation by learning the residual, rather than relying entirely on layer-by-layer learning. The residual network is composed of a series of residual blocks, and 9 residual blocks are used in the Self-pix model.

[0034] The structure of the residual block includes the input passing through a convolution layer, instance normalization and activation function ReLU, and then passing through another convolution layer and instance normalization. Finally, the input and output are added and passed through the ReLU activation function again. Through the skip connection of the residual block, the residual network can better transfer the gradient and enable the network to learn more meaningful feature representations. In addition, the residual network also adopts a design called "bottleneck structure", that is, a 1x1 convolution kernel is used in each residual block to reduce the computational complexity.

[0035] Among them, the generator of the Self-pix model first undergoes two downsamplings, and the downsampling structure is convolution layer-instance normalization layer-activation function LeakyReLU, then passes through 9 residual blocks, and finally passes through upsampling. The upsampling structure is deconvolution layer-instance normalization layer-activation function ReLU.

[0036] The Self-pix model uses spectral normalization on the discriminator in Pix2pix (adversarial production network). Spectral normalization is mainly applied to the weight matrix of the discriminator, and reduces the instability in training by limiting the spectral norm of the weight matrix. Through spectral normalization, the training stability of the generative adversarial network can be effectively improved, making it easier for the training process to converge to the ideal state.

[0037] The generator of Pix2pix uses the U-net network, which adopts "convolution-batch normalization-nonlinearity". In the pix2pix model, the U-net downsampling structure is that the input first passes through a convolution layer and then a pooling layer. This structure is used to extract features, and the upsampling is done through deconvolution layers and jump connections, using feature fusion to generate the output image.

[0038] like Figure 1 As shown in the figure, a jump connection is added between the i-th layer and the ni-th layer of Unet, where n is the total number of layers. Each jump connection only connects all channels of the i-th layer to the channels of the ni-th layer, which can bypass the information bottleneck. First, input an image from the leftmost side of the above figure, and after two convolutions, it becomes 64 channels. Then, it passes through a pooling layer without changing the number of channels, but only reduces the size of the image to half of the original size. Then the operation is similar, until it becomes the bottom 1×512×32×32 type (batchsize=1, the number of channels is 512, and the image size is 32×32), and then passes through two convolution layers again to become 1×1024×28×28.

[0039] Pix2pix's discriminator uses patchGAN. PatchGAN uses the same network module as the generator, with a multi-layer convolutional layer structure and an activation function Leaky ReLU after each convolution layer. It extracts features from each small patch of the input image through different convolution kernels and outputs a probability value between 0 and 1, indicating the probability that the patch is a real image. Through the structure of PatchGAN, local areas of the generated image can be finely evaluated. This can help the generator better learn to generate details and texture information in real images.

[0040] Transformed into MR images Figure 4 As shown in the figure, the synthetic MR (left) and the real MR (right) of lumbar fractures, groups A to E show L1 to L5 fractures, and groups E and F show multiple fractures. It can be seen that the synthetic images show good similarity in both non-fractured vertebrae and non-fractured vertebrae. Among them, in the fractured vertebrae, the degree of vertebral compression and soft tissue injuries such as bleeding and edema near the fractured vertebrae are well demonstrated. Soft tissue injuries cannot be well displayed in CT images, but they appear as signal changes near the fractured vertebrae in MR, which can be used to distinguish fresh fractures from old fractures, and further guide doctors' treatment choices.

[0041] Both subjective and objective evaluation methods were used to evaluate the quality of MR images. The objective evaluation used structural similarity (SSIM), and the value range of SSIM was 0-1. The closer to 1, the higher the image similarity quality. The fractured vertebra in MR was defined as the region of interest (ROI).

[0042] In the objective evaluation results, the SSIM of Self-pix's synthetic MR and real MR reached 0.771, and the SSIM of Self-pix's synthetic MR-ROI and real MR-ROI reached 0.832. In the subjective evaluation results, the sensitivity of Self-pix's synthetic MR in diagnosing fresh fractures reached 84.36%, and the specificity reached 96.65%. The accuracy of Self-pix's synthetic MR in selecting treatment plans reached 97.97%.

[0043] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A CT image conversion method, characterized in that: Select CT images of lumbar fractures and convert them into MR images.

2. The method according to claim 1, characterized in that: The conversion into MR images comprises the following steps: S1: CT image and MR image processing of lumbar fractures, including image cropping, unifying the brightness and contrast of CT and MR images of different patients, and pairing the CT and MR images of patients; S2: Input the processed images into the Self-pix model for multiple rounds of training; S3: After the model training is completed, the lumbar fracture CT image is input into Self-pix. The generator will convert the style according to the trained parameters and convert the lumbar fracture CT image into an MR image.

3. The method according to claim 2, characterized in that The cropping requirements for CT and MR images are as follows: the anterior edge is the anterior edge of the L4 or L5 vertebra, the posterior edge is the posteriormost part of the spinous process, the upper edge is the uppermost part of the 12th thoracic vertebra, and the lower edge is the lowest part of the sacral vertebra.

4. The method according to claim 2, characterized in that: The generator of the Self-pix model uses a residual network, which adopts a skip connection method.

5. The method according to claim 2, characterized in that: The residual network is composed of multiple residual blocks, and 9 residual blocks are used in the Self-pix model.

6. The method according to claim 2, characterized in that The structure of the residual block includes the input passing through a convolution layer, instance normalization and activation function ReLU, then passing through another convolution layer and instance normalization, and finally adding the input and output and passing through the ReLU activation function again, and using a 1x1 convolution kernel in each residual block.

7. The method according to any one of claims 1 to 6, used for distinguishing fresh or old lumbar fractures for non-disease diagnosis purposes.

Images