Lumbar intervertebral disc protrusion image recognition and conversion method and system

Abstract

The invention provides a lumbar disc herniation image recognition and conversion method and system. The method comprises the steps that firstly, a lumbar disc herniation CT image and a lumbar disc herniation MRI image of a patient are collected; preprocessing the acquired lumbar disc herniation CT image and the acquired lumbar disc herniation MRI image, and constructing a training set; then, an ITAD-pix model is constructed; inputting the training set into an ITAD-pix model for training; and finally, inputting the lumbar disc herniation CT image which does not participate in training into the trained ITAD-pix model, and converting the lumbar disc herniation CT image which does not participate in training into a target lumbar disc herniation MRI image through the ITAD-pix model. The CT image of the lumbar disc herniation is converted into the MRI image, so that the advantages of CT and MRI are combined, a spine surgeon can quickly diagnose the lumbar disc herniation and accurately select a treatment scheme, and postoperative recovery and long-term prognosis of a patient are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and system for identifying and transforming lumbar disc herniation images. Background Art

[0002] Lumbar disc herniation refers to the displacement of the content (nucleus pulposus) of the intervertebral disc through its outer membrane (annulus fibrosus), usually occurring in the posterolateral region of the intervertebral disc. Depending on the volume of the herniated mass, it may compress and stimulate the lumbar nerve roots and the dural sac. Lumbar disc herniation is the most common diagnosis in lumbar degenerative abnormalities and is also the main cause of adult spinal surgery. Typical clinical manifestations include initial low back pain, which then progresses to progressive sciatica. For patients with lumbar disc herniation, the diagnosis is completed through magnetic resonance imaging (MRI), computed tomography (CT) combined with clinical symptoms and physical examination. As the gold standard for diagnosis, MRI has many limitations; early and accurate diagnosis and active treatment are of significant importance for improving the prognosis of patients.

[0003] Intervertebral disc degeneration is the fundamental cause of lumbar disc herniation. As people age, the intervertebral disc gradually degenerates, the water content of the annulus fibrosus and the nucleus pulposus gradually decreases, the nucleus pulposus loses its elasticity, and the annulus fibrosus gradually develops fissures. On the basis of degeneration, under the accumulation of strain and the action of external forces, the intervertebral disc ruptures, and the nucleus pulposus, annulus fibrosus, and even the endplate protrude backward, and in severe cases, it compresses the nerve and causes symptoms. Cumulative injury is the main cause of intervertebral disc degeneration. Repeated bending, twisting and other movements are most likely to cause intervertebral disc injury, so this disease is related to occupation to a certain extent. During pregnancy, the entire ligament system is in a relaxed state, and the lumbosacral region bears greater stress than usual, increasing the risk of lumbar disc herniation. Among teenagers under 20 years old, about 32% have a positive family history. Congenital developmental abnormalities of the lumbosacral region such as lumbarization of the sacrum, sacralization of the lumbar vertebrae, and asymmetry of the articular processes cause abnormal stress on the lower lumbar vertebrae, all of which will increase the damage to the intervertebral disc.

[0004] Lumbar disc herniation is common in patients aged 20 - 50, and the male - to - female ratio is about (4 - 6):1. Most patients have a history of bending labor or long - term sitting work, and the first onset often occurs during the process of holding heavy objects in a semi - bent position or suddenly twisting the waist. The following are the possible symptoms and characteristics:

[0005] Low back pain: Most patients with lumbar disc herniation have low back pain. The low back pain can occur before the leg pain, or at the same time or after the leg pain. The reason for the occurrence of low back pain is that the herniated intervertebral disc stimulates the sinuvertebral nerve fibers in the outer annulus fibrosus and the posterior longitudinal ligament.

[0006] Sciatica: Since about 95% of lumbar disc herniation occurs in the L4 - L5 and L5 - S1 intervertebral spaces, it is often accompanied by sciatica. Sciatica usually develops gradually, and the pain is radiating, from the buttocks, the posterior outer side of the thigh, the outer side of the calf to the heel or the dorsum of the foot.

[0007] Cauda equina syndrome: Severe lumbar disc herniation may lead to changes in the spinal column morphology, presenting obvious deformities or malformations. Movements during daily activities may exacerbate the deformation, and in the long term, it may lead to lumbar lordosis or kyphosis.

[0008] The treatment of lumbar disc herniation is generally divided into non-surgical treatment and surgical treatment.

[0009] Indications for non-surgical treatment: (1) Patients with the first onset and a short course of disease; (2) Those whose symptoms can relieve spontaneously after rest; (3) Those who cannot undergo surgery due to systemic diseases or local skin diseases; (4) Those who do not agree to surgery. It can include conservative treatment, rest and activity restriction, and other treatments. Conservative treatment is commonly used for patients without obvious neurological function impairment, without obvious spinal instability, or other serious complications, aiming to relieve pain, improve function, and promote the patient's recovery. Patients may need to strictly restrict activities at the initial stage of treatment, stay in bed strictly for 3 weeks, and gradually get out of bed with a lumbar corset. Other treatments usually include pelvic traction therapy and physical therapy.

[0010] Indications for surgical treatment: (1) Patients with severe low back and leg pain symptoms that recur frequently and are ineffective after more than half a year of non-surgical treatment, and the condition is gradually worsening, affecting work and life; (2) Those with cauda equina syndrome and sphincter dysfunction due to central herniation should undergo emergency surgery; (3) Those with obvious manifestations of nerve involvement. Traditional open surgeries include total laminectomy and discectomy, hemilaminectomy and discectomy, and laminotomy and discectomy. Microscopic lumbar discectomy uses a microscope to assist the surgery for discectomy. Minimally invasive discectomy surgeries include microendoscopic discectomy (MED), percutaneous endoscopic lumbar discectomy (PELD), and discectomy under unilateral biportal endoscopy (UBE), etc. In recent years, percutaneous spinal endoscopy techniques represented by PELD have developed rapidly and are increasingly widely used in clinical practice due to their advantages such as less injury and faster recovery. The indications for artificial disc replacement are still controversial, and this surgery must be chosen with caution.

[0011] The diagnosis of lumbar disc herniation requires the use of different imaging methods, including X-rays, CT scans, and MRIs. These methods have their own imaging principles, advantages and disadvantages, and imaging manifestations, which can provide in-depth understanding of lumbar disc herniation and its imaging features. In addition, the diagnosis is also combined with the patient's physical signs and physical examinations.

[0012] Among them, X-ray imaging is through the radiation penetrating human tissue and being absorbed to varying degrees, and then captured by the detector to form an image. Bone tissue absorbs more X-rays and appears as a bright area, while soft tissue absorbs less and appears as a dark area. X-rays are usually used as routine examinations. For patients with lumbar disc herniation, the appearance of lumbar spine X-rays can be completely normal. The following are common imaging manifestations: lumbar scoliosis can be seen on the frontal film, and the physiological lordosis can be reduced or disappeared on the lateral film, and the intervertebral space is narrowed.

[0013] Advantages of X-ray imaging: X-ray imaging is fast (can be completed within one minute), convenient, and low-cost. Disadvantages: It cannot show the condition of soft tissue, cannot assess the compression of the spinal cord, and cannot assess the location and size of lumbar disc herniation. DR requires low-dose radiation and may not be suitable for certain groups such as pregnant women.

[0014] The main clinical application of X-rays is that degenerative manifestations such as annular fibrosis calcification, bone hyperplasia, articular process hypertrophy and sclerosis can be seen on X-ray films.

[0015] CT scanning is an imaging technique that uses a rotating X-ray source and detector. The specific imaging principle is as follows: X-ray source: The rotating X-ray source irradiates the patient's body at different angles. Detector: The detector receives X-ray signals absorbed and attenuated by body tissues at a relative position. Computer reconstruction: The computer uses a large amount of data collected, and uses mathematical algorithms to calculate and reconstruct to generate cross-sectional (fault) images. Multi-planar reconstruction: CT scanning not only provides the appearance of lumbar disc herniation on CT images, but also the appearance of the disc in the protruding part. Depending on the severity, the disc may appear lower in height, which is shown as a narrowing of the intervertebral space. The water content in the disc may be reduced on CT, which is shown as a low-density area. The disc may appear elliptical or curved. The disc may appear to protrude backward or laterally on the CT image. If there is a prolapsed disc tissue, it may appear as a low-density image on the CT image. Under long-term pressure, bone hyperplasia (bone spurs) may occur, which is shown as a jagged protrusion on the edge of the vertebral body on the CT image. Bone marrow edema may occur around the vertebral body or near the disc.

[0016] Advantages of CT imaging: CT provides high-resolution three-dimensional images, showing the lumbar spine in more detail, with higher sensitivity and specificity in diagnosing lumbar disc herniation. It can better display the details of the spinal bony structure. The manifestations of lumbar disc herniation on CT include deformed protrusion of the posterior margin of the intervertebral disc, compression and deformation of the dural sac, displacement of epidural fat, soft tissue density shadow in the epidural space, and compression and displacement of the nerve root sheath. Three-dimensional anatomy display: Compared with DR, CT can perform multi-planar reconstruction, showing three-dimensional anatomical structures at multiple angles such as cross-section, coronal plane, and sagittal plane, which helps to evaluate the position and size of the protrusion. In addition, the CT scanning speed is fast (it can be scanned within one minute), and the imaging time is short, which is very useful for rapid diagnosis in emergency situations. Disadvantages: Compared with DR, the radiation dose of CT scanning is relatively high, and long-term and frequent examinations may have a certain impact on the patient's health. It cannot show the soft tissue conditions and cannot evaluate the spinal cord compression. High cost: Compared with X-ray examination, the cost of CT scanning is relatively high, which may increase the medical expenses.

[0017] The main clinical application of CT lies in: CT scanning plays an important role in the diagnosis and treatment of lumbar disc herniation. CT can clearly show the morphological changes of the lumbar intervertebral disc and its relationship with surrounding tissues (such as nerve roots and spinal cord), thus helping doctors confirm the presence of disc herniation, as well as the specific location and degree of the herniation.

[0018] The imaging principle of MRI mainly utilizes the magnetic resonance phenomenon. MRI uses a strong magnetic field and lossless radio waves to obtain images. Under the action of the magnetic field, atomic nuclei in the human body (such as hydrogen atomic nuclei) will generate resonance signals. The imaging process includes the following steps. Exciting the atomic nuclei to resonate: The patient is placed in a strong magnetic field, and the atomic nuclei change between different energy levels, absorbing and releasing energy. Signal reception: The probe generates radio waves to capture the signals released by the atomic nuclei. Signal processing and image reconstruction: The computer processes the received signals to generate images, showing the detailed conditions of the tissue structure.

[0019] MRI can clearly show the images of the human anatomical structure, which is of great help in the diagnosis of lumbar disc herniation. MRI can be used to comprehensively observe the degeneration of each intervertebral disc. A healthy intervertebral disc usually shows a relatively high signal (T2-weighted image) on MRI, while a herniated or degenerated intervertebral disc shows a lower signal, appearing dim, which usually indicates a reduction in the water content in the intervertebral disc. It can also understand the degree and position of the nucleus pulposus herniation, clearly show the herniation site of the intervertebral disc, and help to evaluate the type of herniation (such as central herniation, lateral herniation). MRI can evaluate whether the herniated disc compresses the nerve root or spinal cord. The compressed nerve usually shows signal changes or swelling on the image, showing the changes in the nerve. It can also show the inflammation or edema around the herniated disc, and these manifestations are often accompanied by reactive changes in the muscles or membranes; and distinguish whether there are other space-occupying lesions in the spinal canal.

[0020] Advantages of MRI imaging: High soft tissue resolution. MRI shows soft tissue structures very well and can clearly display structures such as paravertebral tissues and the spinal cord. Multi-planar imaging: It can perform imaging in the sagittal, coronal, and transverse planes, providing the possibility of observing lumbar disc herniation from multiple angles. Radiation-free: MRI imaging is radiation-free, avoiding the impact of X-ray radiation on patients, and is particularly suitable for the examination of special populations such as pregnant women and children. Disadvantages: Long examination time: MRI imaging requires a long examination time (usually more than half an hour), and MRI examination cannot be performed in case of emergency, such as in the case of night emergency. The patient needs to maintain a relatively fixed position, which poses a great challenge to some patients with severe lumbar disc herniation; Not applicable to all patients: For some patients with metal implants such as cardiac pacemakers and certain special diseases such as claustrophobia, they may not be suitable for MRI examination. The cost of MRI is relatively high (about 500 for one part), causing an economic burden on the examiner.

[0021] The clinical application of MRI mainly lies in: The manifestation of MRI in lumbar disc herniation can not only accurately evaluate the damage of the intervertebral disc and nerve compression, but also provide detailed images of the surrounding tissues. Since there is no radiation, it also has advantages in repeatedly evaluating the patient's condition. It provides important references for the formulation of treatment plans.

[0022] Imaging detection methods can also be used. Methods such as myelography, epidural contrast imaging, and discography can indirectly show the presence and degree of lumbar disc herniation. Since these methods are invasive operations, some have complications and some are technically complex, so they are currently less used in clinical applications and are only carefully performed when the general diagnostic methods cannot be clear.

[0023] This application provides a lumbar disc herniation image recognition and conversion method and system under the above background, which converts lumbar disc herniation CT images into MRI images, so as to combine the advantages of CT and MRI, which is beneficial for spinal surgeons to quickly diagnose lumbar disc herniation, accurately select treatment plans, and is beneficial for the postoperative recovery and long-term prognosis of patients. Summary of the Invention

[0024] The purpose of the present invention is to provide a lumbar disc herniation image recognition and conversion method and system to solve the problems existing in the above-mentioned prior art.

[0025] To achieve the above purpose, the present invention provides the following solutions:

[0026] The present invention provides a lumbar disc herniation image recognition and conversion method, including the following steps:

[0027] S1. Collect CT images and MRI images of lumbar disc herniation of patients;

[0028] S2. Preprocess the collected CT images and MRI images of lumbar disc herniation to construct a training set;

[0029] S3. Construct the ITAD-pix model;

[0030] S4. Input the training set into the ITAD-pix model for training;

[0031] S5. Input the CT images of lumbar disc herniation that did not participate in the training into the trained ITAD-pix model, and convert the CT images of lumbar disc herniation that did not participate in the training into target MRI images of lumbar disc herniation through the ITAD-pix model.

[0032] Preferably, in step S2, the preprocessing includes:

[0033] S21. Crop the collected CT images and MRI images of lumbar disc herniation;

[0034] S22. Denoise some of the images, and unify the brightness, grayscale, and contrast;

[0035] S23. Pair the unified CT images and MRI images of lumbar disc herniation to align the structures.

[0036] Preferably, in step S21, the criteria for cropping the images include: For sagittal images: Crop the upper part at the mid-lower edge of the second lumbar vertebra, the lower part at the upper edge of the first sacral vertebra, the front part at the anterior edge of the fourth or fifth lumbar vertebra, and the rear part at the mid-rear of the lumbar spinous process; For transverse images: Crop the upper part at the anterior edge of the lumbar disc, the rear part at the vertebral arch plate, and on the left and right at the left and right of the lumbar disc respectively.

[0037] Preferably, step S3 includes:

[0038] S31. Construct the model of ITAD-pix based on the residual network structure;

[0039] S32. Generate the AdaIN layer;

[0040] S33. Add the AdaIN layer to each sampling layer in the generator and discriminator to optimize the ITAD-pix model.

[0041] Preferably, step S32 includes: Adopt an encoder-decoder architecture, where the encoder f is fixed in the first few layers of the pre-trained VGG-19. After encoding the content and style images in the feature space, both feature maps are fed into the AdaIN layer, which aligns the mean and variance of the content feature map with the mean and variance of the style feature map, thereby generating the target feature map t:

[0042] t = AdaIN(f(c), f(s));

[0043] The randomly initialized decoder g is trained to map t back to the image space to generate the stylized image T(c, s):

[0044] T(c, s) = g(t).

[0045] Preferably, step S33 includes: First, input the lumbar disc herniation CT image to be detected into the generator. The structure of each layer of downsampling in the generator is convolution, IN regularization, and the Leaky ReLU activation function. Before downsampling, an image enhancement operation is performed first, and the ReflectionPad2d layer is introduced. After the first downsampling, the obtained relevant features are acquired and used as the input for the next downsampling. At this time, the data becomes 64×256×256. After the second downsampling, the data becomes 128×128×128. After the third downsampling, the data becomes 256×64×64. At this time, it reaches the bottom. Nine residual modules are introduced between downsampling and upsampling to deepen the network and enhance the data at the same time. Then, an upsampling operation is performed. The structure of upsampling is transposed convolution, IN normalization, and the ReLU activation function to restore the size of the image, and the ReflectionPad2d layer is also used for data enhancement. The first upsampling receives the data from the Resnet_block and changes the data to 256×64×64. At this time, the AdaIN layer is introduced to learn and transform the local context information. The second upsampling changes the data to 128×128×128. After the third upsampling, the data becomes 64×256×256; the discriminator takes the data transmitted from the generator as the input, passes it through the convolutional layer and LeakyReLU to change the data to 64×128×128, and through three convolutions, IN regularization, and the Leaky ReLU activation function, the data is changed from 64×128×128 to 128×64×64, then to 256×32×32, and finally to 512×31×31, and then the data is changed to 1×30×30 through a convolutional layer.

[0046] Preferably, the code for iteration is added in the final generation step so that the generated MRI image is generated iteratively.

[0047] Preferably, step S4 is trained by an alternating training method.

[0048] Preferably, step S4 includes:

[0049] S41. Training of the generator: The given input image is passed through the generator to generate the corresponding output image, and the generated image and the target image are sent to the discriminator for training. The training process first passes the input image through the generator to generate the corresponding output image; then calculates the loss function between the generated image and the target image, and backpropagates the error to update the parameters of the generator; finally, during the training process, the parameters of the generator are continuously updated in a stochastic gradient descent manner.

[0050] S42. Training of the discriminator: The discriminator discriminates the generated image and the target image to determine whether these images are real. The training process first randomly selects a set of real images and a set of generated images; then inputs these images into the discriminator, and calculates the probabilities that the real images and the generated images belong to the real data respectively; calculates the loss functions corresponding to the real images and the generated images, as well as the overall loss function of the discriminator, and backpropagates the error to update the parameters of the discriminator; finally, during the training process, the parameters of the discriminator are continuously updated in a stochastic gradient descent manner.

[0051] The present invention also provides a lumbar disc herniation image recognition and conversion system, including:

[0052] An image acquisition module, configured to acquire lumbar disc herniation CT images and lumbar disc herniation MRI images of patients;

[0053] An image preprocessing module, configured to preprocess the acquired lumbar disc herniation CT images and lumbar disc herniation MRI images to construct a training set;

[0054] A model construction module, configured to construct an ITAD-pix model;

[0055] A model training module, configured to input the training set into the ITAD-pix model for training;

[0056] An image recognition and conversion module, configured to input the lumbar disc herniation CT images that have not participated in the training into the trained ITAD-pix model, and convert the lumbar disc herniation CT images that have not participated in the training into target lumbar disc herniation MRI images through the ITAD-pix model.

[0057] The present invention has achieved the following beneficial technical effects compared with the prior art:

[0058] A method and system for image recognition and conversion of lumbar disc herniation provided by the present invention first collect CT images and MRI images of lumbar disc herniation of patients; then preprocess the collected CT images and MRI images of lumbar disc herniation to construct a training set; then construct an ITAD-pix model; input the training set into the ITAD-pix model for training; finally, input the CT images of lumbar disc herniation that have not participated in the training into the trained ITAD-pix model, and convert the CT images of lumbar disc herniation that have not participated in the training into target MRI images of lumbar disc herniation through the ITAD-pix model; by converting CT images of lumbar disc herniation into MRI images, the advantages of both CT and MRI are combined, which is beneficial to the rapid diagnosis of lumbar disc herniation by spinal surgeons, accurate selection of treatment plans, and the postoperative recovery and long-term prognosis of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0060] Figure 1 It is the image after preprocessing of the present invention;

[0061] Figure 2 It is the structural diagram of the ITAD-pix model of the present invention;

[0062] Figure 3 It is the effect diagram after recognition and conversion of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0064] The purpose of the present invention is to provide a method and system for image recognition and conversion of lumbar disc herniation to solve the problems existing in the prior art.

[0065] The present invention is a specific application of deep learning in medical imaging. Deep learning is a machine learning technique that enables a computer to automatically learn and understand data by mimicking the structure and function of the human brain's neural network. It processes complex data through multi-layer neural network models, performs high-level abstraction and understanding of the data, and thus achieves efficient processing and analysis of data such as images, speech, and text. Common neural network structures in deep learning include: convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), long short-term memory network (LSTM), autoencoder (AE), attention mechanism (AM), etc.

[0066] Deep learning has the following advantages:

[0067] Automatic feature learning: Deep learning models can automatically learn features from data without the need for manual feature extraction, making them more adaptable.

[0068] Processing large-scale data: It can efficiently process large-scale data, and its learning ability improves with the increase in the amount of data, making it suitable for the large amount of medical imaging data in the medical field.

[0069] Improving prediction accuracy: In medical image analysis, deep learning models can provide prediction results with higher accuracy and precision.

[0070] Potential generality: It is applicable to different types of medical imaging data, such as X-rays, MRI, CT, etc., and has a certain degree of generality.

[0071] Interpretability: Some deep learning models have a certain degree of interpretability, which can help doctors understand the diagnostic process.

[0072] Currently, the applications of deep learning in medical imaging mainly include:

[0073] Lesion segmentation: In medical imaging, deep learning can be used to locate and segment lesion areas. For example, there are papers reporting the segmentation of normal or diseased vertebral bodies in spinal surgery.

[0074] Image reconstruction: Deep learning in medical imaging can be used to reconstruct three-dimensional images from planar images. For example, there are papers reporting that lumbar three-dimensional CT can be reconstructed from lumbar DR in spinal surgery.

[0075] Image simulation: Currently, there are papers reporting the realization of single-modal or multi-modal imaging image conversion simulation. For example, converting non-contrast CT images of the carotid artery and aorta into contrast CT images, and converting normal lumbar cross-sectional CT into MRI images.

[0076] Disease-assisted diagnosis: Deep learning can be used in medical imaging for diagnosing tumors, lung diseases, cardiovascular diseases, etc. For example, it is used for the analysis of mammogram images for breast cancer screening, the auxiliary detection of lung nodules, and the tumor detection and classification for CT or MRI scans.

[0077] The objective of the present invention is to convert CT images of lumbar disc herniation into MRI images and perform auxiliary diagnosis. CT and MRI are images with two different imaging principles. To achieve the conversion between them, it is naturally suitable to apply GAN for learning and training.

[0078] The GAN network consists of two parts: a generator and a discriminator. Among them, the generator is responsible for converting a random noise vector into an image or other types of data similar to the training data, while the discriminator is responsible for distinguishing the differences between the images generated by the generator and the real data. The generator and the discriminator gradually optimize their network parameters through repeated games until they reach a balanced state - the generator can generate highly realistic data, and the discriminator cannot accurately distinguish whether the generated data is real data. During the training process of GAN, the generator and the discriminator confront each other and continuously optimize their performance in the game. Specifically, the generator tries to deceive the discriminator so that it cannot accurately distinguish the generated data from the real data, while the discriminator tries to identify the generated data and the real data as accurately as possible. In this way, during the process of backpropagating the error, the generator can adjust its strategy of generating data according to the feedback of the discriminator, thereby gradually generating more realistic data. The objective function in the GAN network is shown in the following figure:

[0079]

[0080] Among them and represent the probabilities of real data and generated data. It can be seen from the above loss function that the calculation of the loss function is generated in D (the discriminator). Since the output of D is generally a True / Fake judgment, the binary cross-entropy function is generally adopted as a whole. The left side contains two parts: minG and maxD. For maxD, for 1 - D(G(z)), it is hoped that it can be as close as possible to 1, that is, the discriminator can accurately distinguish the generated images. For minG, because it is desired that the generated data can pass the discrimination of the discriminator, it is necessary to make D(G(z)) close to 1, that is, the generated data must pass the discriminator. Therefore, for it is necessary to be as small as possible. This reflects the idea of game learning.

[0081] The following is a detailed introduction to the operating principle of GAN:

[0082] Generator:

[0083] The generator is a neural network that takes random noise (latent space vector) as input and attempts to generate realistic data. It tries to generate data that can "fool" the discriminator by repeatedly learning the characteristics of the data distribution.

[0084] Discriminator:

[0085] The discriminator is another neural network, similar to a binary classifier, which takes real data and fake data generated by the generator as input and attempts to correctly classify them. The goal of the discriminator is to accurately distinguish between real data and fake data generated by the generator.

[0086] Adversarial Training:

[0087] The generator and the discriminator compete against each other. During the training process, the generator tries to generate data that is realistic enough to "fool" the discriminator, while the discriminator tries to learn to accurately distinguish between real and fake data. This adversarial training process is iterated until the data generated by the generator is realistic enough to be indistinguishable by the discriminator.

[0088] The training process of GAN includes:

[0089] 1) Initialize network parameters:

[0090] The weights of the generator and the discriminator are usually randomly initialized.

[0091] 2) Adversarial training loop:

[0092] In each training iteration, the generator and the discriminator are alternately trained. For each training iteration: The generator receives random noise and generates data samples. The discriminator receives real data and fake data generated by the generator and makes classification judgments respectively. The discriminator calculates the loss (the accuracy of distinguishing between real and fake data), and the generator calculates the loss (trying its best to "fool" the discriminator). The weights of the generator and the discriminator are updated through the loss function to improve the generation ability of the generator and the discrimination ability of the discriminator.

[0093] 3) Convergence and generation:

[0094] As the training progresses, the generator gradually learns the characteristics of the data distribution, and the generated data will become more and more realistic. When the data generated by the generator is realistic enough to be accurately distinguished by the discriminator, the model reaches a convergence state.

[0095] After convergence, the generator that has learned sufficient distribution features has reached its optimal state. We input the original image into the generator, and the generator will output the target image using the learned features.

[0096] The objective of the present invention is to convert CT images of lumbar disc herniation into MRI images, which is naturally suitable for the training and image generation of GAN. However, the learning of GAN is too generalized and will widely learn the features of each vertebral body and paravertebral area, which is not conducive to clinical needs because we need to focus on the location and size of lumbar disc herniation to assist in clinical diagnosis and treatment selection. Therefore, in the present invention, we have innovated a new enhanced GAN model, in which an iterative instance normalization generation mechanism is introduced to specifically enhance the manifestation of lumbar disc herniation.

[0097] Iterative instance normalization generation is a method of optimizing the data scale and distribution through multiple iterations. Its purpose is to ensure that the input data remains within a reasonable range (for example, mean of 0 and variance of 1) in each iteration, thereby accelerating model convergence and improving training stability. By optimizing the distribution of the input data through multiple iterations, it becomes more stable during the training process. Instance normalization adopts a novel adaptive instance normalization layer (AdaIN), which matches the mean and variance of the content features with the mean and variance of the style features, achieving a perfect fit in image style transfer.

[0098] The instance normalization residual layer decoder is mainly a mirror image of the encoder, and all pooling layers are replaced by the nearest upsampling to reduce the checkerboard effect. We use reflection padding in f and g to avoid boundary artifacts. Another important architectural choice is whether the decoder should use instance layers, batch layers, or no normalization layers.

[0099] In image-class style transfer, the model can extract the features of the content map and style map through multi-channel fusion, and then apply iterative normalization to ensure that each feature remains stable in each iteration, thereby generating higher-quality images. In image restoration, signals from different feature extractors (such as color information, texture information, etc.) are fused, and at the same time, iterative normalization is used to maintain stability during the image restoration process, which is crucial for the restoration effect.

[0100] The above are the principles and methods on which the present invention is based. To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0101] Example 1:

[0102] This embodiment provides a method for identifying and converting images of lumbar disc herniation, including the following steps:

[0103] S1. Collect the CT images and MRI images of patients with lumbar disc herniation;

[0104] S2. Preprocess the collected CT images and MRI images of lumbar disc herniation to construct a training set;

[0105] To exclude the interference of surrounding soft tissues and enable the model to more accurately learn the characteristics of lumbar disc herniation, we preprocessed the CT and MRI images of lumbar disc herniation. The preprocessing includes:

[0106] S21. Crop the collected CT images and MRI images of lumbar disc herniation;

[0107] S22. Denoise some of the images and unify the brightness, grayscale, and contrast;

[0108] S23. Pair the unified CT images and MRI images of lumbar disc herniation to align the structures. Figure 1 The figure shows the images after preprocessing;

[0109] S3. Construct the ITAD-pix model;

[0110] The ITAD-pix model in this embodiment adopts the Pix2Pix principle. Pix2Pix is a common adversarial generative network for image style conversion. The present invention proposes a new network model ITAD-pix on this basis;

[0111] The objective function of Pix2Pix is based on the conditional adversarial generative network (CGAN). The loss selection of CGAN is different from that of GAN. It makes certain changes on the basis of GAN, that is, it adds the conditional feature and has an additional L1 loss to make the images in the source domain and the target domain as close as possible. The objective function of CGAN can be expressed as:

[0112]

[0113] The objective of CGAN is similar to that of GAN, but GAN generates images based on the random noise z while CGAN generates based on the random noise z and the input image x; the difference between CGAN and GAN is that in addition to generating images that can deceive D, the G of CGAN also needs to be as close as possible to the image y in the target domain; regarding the loss selection, CGAN selects the L1 loss, which is expressed as follows:

[0114]

[0115] L1 loss is selected instead of L2 loss to ensure less blur. L1 loss can better retain detail information. L1 loss has a higher tolerance to noise and outliers than L2 loss, which makes the generated image clearer and can better retain the details of the original image. L1 loss can better solve the problem of mode collapse. L2 loss will have the problem of mode collapse in the generated image, that is, the generated image is too smooth and loses some features of the original image. L1 loss can better avoid this problem, so that the generated image is more diversified. The final objective function is:

[0116]

[0117] The Pix2Pix model generator uses the Unet structure. As the name suggests, its overall network structure is like a capital letter U, including a contraction path and an expansion path, also known as an encoder-decoder structure, and these two paths are connected by jump connections; the generator uses the "convolution-Batch Normalization-Non-Linear" (convolution-BatchNorm-ReLu) network module. Image translation needs to be from one high-resolution image to another high-resolution image, that is, although the input and output are different in surface details, they have exactly the same underlying model structure, so the input and output need to be roughly aligned to make the model converge better; for many image translation related problems, Pix2Pix proposes a novel jump connection method, adding a jump connection between the i-th layer and the ni-th layer of Unet, where n is the total number of layers, and each jump connection only connects all channels of the i-th layer with the channels of the ni-th layer, which can bypass the information bottleneck.

[0118] First, input an image, which becomes 64 channels after two convolutions. Then it passes through a pooling layer without changing the number of channels, but only reduces the image size to half of the original size. The operation is similar, until it becomes the bottom 1×512×32×32 type (batchsize=1, the number of channels is 512, and the image size is 32×32), and then passes through two convolution layers again to become 1×1024×28×28.

[0119] The discriminator of Pix2Pix uses PatchGAN, which adopts the same network module as the generator. The difference between it and the discriminator of a general GAN network is that PatchGAN will output an N*N matrix (N = 70 is selected in Pix2Pix). All elements in this matrix have only two values, True or False, which will increase the receptive field of the model for the original image, thus making the image generated by the model more stable. The author states that for low-frequency information, selecting the L1 loss can achieve good reconstruction; while for high-frequency information, the PatchGAN redesigned by the author is needed. It only judges whether an N*N patch is true or false, so it can better discriminate high-frequency information. Even if N is much smaller than the size of the original image, PatchGAN can still produce good results. It has fewer parameters, runs more quickly, and can be applied to images of any size. One of the advantages of PatchGAN is that a discriminator with a fixed size can be used for images of any size during testing, and the generator convolution can be applied to an image larger than the trained generator. For example, an image with a pixel size of 256×256 is selected during the training process, and finally an image with a pixel size of 512×512 is used during testing.

[0120] Step S3 includes:

[0121] S31. Construct the ITAD-pix model based on the residual network structure; the generator of the ITAD-Pix2pix model adopts the residual network (ResNet) structure. The residual network proposed the concept of shortcut connection, which greatly increases the depth of the network and effectively solves the problem of gradient disappearance caused by excessive depth; in the traditional convolutional neural network, the output of the network layer is directly transformed through the activation function and then passed to the next layer; while in ResNet, the input of each network layer will not only be passed to the next layer, but also directly passed to several subsequent layers through the shortcut connection, forming a residual block. The advantage of this is that even if the number of network layers increases, due to the direct path, the information of the original input can be more easily propagated to the subsequent layers, avoiding the rapid attenuation of gradients in the deep network and the problem of gradient explosion; the residual block in ResNet consists of two or three convolutional layers, including an identity mapping and a non-linear activation function (such as ReLU). The identity mapping directly passes the input to the output, while the non-linear activation function introduces non-linear transformation. This design allows the network to learn the identity mapping when needed, that is, directly pass the input to the output, so as to better adapt to tasks with different complexities and difficulties. ITAD-pix uses 9 residual blocks, deepening the network depth and being able to better capture the features and details of the image;

[0122] S32. Generate the AdaIN layer; Adaptive Instance Normalization (AdaIN) takes the content image c and any style image s as inputs and synthesizes an output image that recombines the content of the former and the style of the latter; adopt a simple encoder-decoder architecture, where the encoder f is fixed in the first few layers of the pre-trained VGG-19. After encoding the content and style images in the feature space, both feature maps are fed into the AdaIN layer, which aligns the mean and variance of the content feature map with the mean and variance of the style feature map, thus generating the target feature map t:

[0123] t = AdaIN(f(c), f(s));

[0124] The randomly initialized decoder g is trained to map t back to the image space to generate the stylized image T(c, s):

[0125] T(c, s) = g(t);

[0126] The decoder is mainly a mirror image of the encoder. All pooling layers are replaced by the nearest upsampling to reduce the checkerboard effect; reflection padding is used in f and g to avoid boundary artifacts; another important architectural choice is whether the decoder should use instance layers, batch layers, or no normalization layers.

[0127] S33. Add the AdaIN layer to each sampling layer in the generator and discriminator to optimize the ITAD-pix model; to find the model with the best optimization effect, we add the AdaIN layer to each sampling layer in the generator and discriminator to improve the conversion performance and imaging effect of the model for different instance-style images. Figure 2 Specifically shows the structure of ITAD-Pix;

[0128] First, input the CT image to be detected into the generator. The structure of each layer of downsampling in the generator is convolution, IN regularization, and Leaky ReLU activation function. Before downsampling, an image enhancement operation is performed first. The ReflectionPad2d layer is introduced to symmetrically expand the image along the edges in all directions to increase the image resolution. After the first downsampling, the relevant features obtained are used as the input for the next downsampling. At this time, the data becomes 64×256×256. After the second downsampling, the data becomes 128×128×128. After the third downsampling, the data becomes 256×64×64. At this time, it reaches the bottom. Nine residual modules are introduced between downsampling and upsampling to deepen the network and enhance the data at the same time. Then, the upsampling operation is performed. The role of upsampling is to magnify and restore the extracted features. The structure of upsampling is transposed convolution, IN normalization, and ReLU activation function to restore the size of the image, and the ReflectionPad2d layer is also used for data enhancement. The first upsampling receives data from the Resnet_block and changes the data to 256×64×64. At this time, Adaptive Instance Normalization is introduced to learn and transform the local context information, which can better generate images that conform to MRI features. The second upsampling changes the data to 128×128×128. After the third upsampling, the data becomes 64×256×256;

[0129] The discriminator takes the data transmitted from the generator as input and changes the data to 64×128×128 through the convolutional layer and LeakyReLU. Through three convolutions, IN regularization, and Leaky ReLU activation function, the data is changed from 64×128×128 to 128×64×64, then to 256×32×32, and finally to 512×31×31. Then, the data is changed to 1×30×30 through a convolutional layer;

[0130] In the final generation step, iterative corresponding code is added to make the generated MRI images be repeatedly generated, compensating for the deficiencies in the previous round of generation to generate MRI images with better effects and performance;

[0131] S4. Input the training set into the ITAD-pix model for training;

[0132] For the two different networks of the discriminator and the generator, an alternating training method is used, that is, the entire training process is divided into two parts: the training of the generator and the training of the discriminator;

[0133] Training of the generator. In the training of the generator, the given input image is passed through the generator to generate the corresponding output image, and the generated image and the target image are sent to the discriminator for training together; the training process is first to pass the input image through the generator to generate the corresponding output image; then calculate the loss function between the generated image and the target image, and backpropagate the error to update the parameters of the generator; finally, during the training process, the parameters of the generator are continuously updated in a random gradient descent manner, so as to gradually improve the performance of the generator;

[0134] Training of the discriminator. In the training of the discriminator, the discriminator needs to discriminate between the generated images and the target images to determine whether these images are real; the training process is first to randomly select a set of real images and a set of generated images; then these images are input into the discriminator, and the probabilities that the real images and the generated images respectively belong to the real data (i.e., the output values of the discriminator) are calculated; by calculating the loss functions corresponding to the real images and the generated images, as well as the overall loss function of the discriminator, and backpropagating the error to update the parameters of the discriminator; finally, during the training process, the parameters of the discriminator are also continuously updated in a random gradient descent manner to improve its performance;

[0135] During the alternating training process, the training of the generator and the discriminator is repeatedly carried out, and their parameters are continuously optimized until the model converges or reaches the preset number of training times. This alternating training method can effectively balance the performance of the generator and the discriminator during the training process, so that the final model can better complete the image translation task;

[0136] S5. Input the lumbar disc herniation CT images that have not participated in the training into the trained ITAD-pix model, and the generator will convert the style according to the trained parameters to convert the lumbar disc herniation CT images into MRI images.

[0137] Figure 3 The effect of the ITAD-pix synthesized images is shown. The synthesized images show good similarity in lumbar disc herniation.

[0138] Both subjective and objective evaluations were adopted to assess the quality of our images. For the objective evaluation, structural similarity (SSIM) was used. The value range of SSIM is 0 - 1, and the closer it is to 1, the higher the similarity quality of the images. For the subjective evaluation, senior spinal surgeons were invited to participate in the evaluation, and the lumbar disc herniation area in the MRI was defined as the region of interest (ROI).

[0139] In the objective evaluation results, the SSIM between the generated sagittal MRI and the real MRI of ITAD - pix reached 0.782, and the SSIM between the synthetic MRI - ROI and the real MRI - ROI of ITAD - pix reached 0.878. The SSIM between the generated transverse MRI and the real MRI reached 0.721, and the SSIM between the synthetic MRI - ROI and the real MRI - ROI of ITAD - pix reached 0.758. In the subjective evaluation results, the sensitivity of the synthetic MRI in diagnosing lumbar disc herniation reached 85.28%, and the specificity reached 95.56%. The accuracy of using the MRI synthesized by ITAD - pix to select treatment plans reached 96.38%.

[0140] This embodiment also provides a lumbar disc herniation image recognition and conversion system, including:

[0141] An image acquisition module for acquiring CT images and MRI images of lumbar disc herniation of patients;

[0142] An image pre - processing module for pre - processing the acquired CT images and MRI images of lumbar disc herniation to construct a training set;

[0143] A model construction module for constructing an ITAD - pix model;

[0144] A model training module for inputting the training set into the ITAD - pix model for training;

[0145] An image recognition and conversion module for inputting the CT images of lumbar disc herniation that did not participate in the training into the trained ITAD - pix model, and converting the CT images of lumbar disc herniation that did not participate in the training into target lumbar disc herniation MRI images through the ITAD - pix model.

[0146] A lumbar disc herniation image recognition and conversion method and system provided by the present invention have the following positive significances:

[0147] As the gold standard for diagnosing lumbar disc herniation, MRI has various inconveniences: high cost, long time consumption, being in a closed space, and the patient needs to remain still, etc. Although CT is fast, convenient, and efficient, it cannot provide soft tissue-related information for spinal surgeons to observe the size and location of the disc herniation. Therefore, this patent develops the ITAD-pix model to combine the advantages of CT and MRI examinations for convenient clinical use.

[0148] MRI can provide high-resolution images of the lumbar region, including bones, soft tissues, nerves, and blood vessels, etc. This provides doctors with a clear view of the anatomical structure; effectively identifies morphological changes of the intervertebral disc, such as disc herniation, extrusion, bulging, etc. These changes are manifested as high signals and shape changes of the intervertebral disc in MRI images. Through MRI, doctors can check whether the disc herniation compresses the surrounding spinal nerve roots or the spinal cord. This is crucial for evaluating the causes of symptoms such as pain and numbness. It can help rule out other lesions that may cause similar symptoms, such as spinal stenosis, tumors, fractures, etc. It shows the location and degree of the disc herniation, helping doctors formulate more effective treatment plans. It brings important imaging evidence to the clinic, enabling doctors to accurately evaluate the condition and formulate corresponding treatment plans.

[0149] This invention greatly improves the diagnostic efficiency of lumbar disc herniation, which is of great benefit to both the imaging department and the spinal surgery department. It not only improves the MRI imaging efficiency of the imaging department but also assists spinal surgeons in the diagnosis and treatment selection of lumbar disc herniation. It can assist spinal surgeons in making decisions on treatment plans and selecting the most suitable treatment plan for different patients.

[0150] This invention elaborates on the principle and implementation mode of the invention with specific examples. The description of the above embodiments is only used to help understand the method and its core idea of the invention; at the same time, for those of ordinary skill in the art, according to the idea of the invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be construed as a limitation to the invention.

Claims

1. A method for recognizing and transforming lumbar disc herniation images, characterized in that: The following steps are involved: S1. Collect CT images and MRI images of lumbar disc herniation of patients; S2. preprocessing the collected lumbar disc herniation CT images and lumbar disc herniation MRI images to construct a training set; S3. Construct ITAD-pix model; S4. Input the training set into the ITAD-pix model for training; S5. The untrained lumbar disc herniation CT image is input into the trained ITAD-pix model, and the untrained lumbar disc herniation CT image is converted into a target lumbar disc herniation MRI image through the ITAD-pix model.

2. The method for recognizing and converting lumbar disc herniation images according to claim 1, characterized in that: In step S2, the preprocessing includes: S21. Crop the acquired lumbar disc herniation CT image and lumbar disc herniation MRI image; S22. performing noise reduction processing on part of the image, and unifying the brightness, grayscale and contrast; S23. Pair the unified lumbar disc herniation CT image and lumbar disc herniation MRI image to align the structures.

3. The method for recognizing and converting lumbar disc herniation images according to claim 2, characterized in that: In step S21, the standards for cropping the image include: cropping for sagittal image: cropping at the top with the middle and lower edge of the second lumbar vertebra, at the bottom with the upper edge of the first sacral vertebra, at the front with the front edge of the fourth or fifth lumbar vertebra, and at the back with the middle and back of the lumbar spinous process; cropping for transverse image: cropping at the top with the front edge of the lumbar intervertebral disc, at the back with the vertebral arch plate, and at the left and right sides of the lumbar intervertebral disc respectively.

4. The method for recognizing and transforming lumbar disc herniation images according to claim 1, characterized in that: Step S3 includes: S31. Construct ITAD-pix model based on residual network structure; S32. Generate AdaIN layer; S33. The ITAD-pix model is optimized by adding AdaIN layers to each sampling layer in the generator and discriminator.

5. The method for recognizing and converting lumbar disc herniation images according to claim 4, characterized in that: Step S32 includes: adopting an encoder-decoder architecture, where the encoder f is fixed to the first few layers of the pre-trained VGG-19, after encoding the content and style images in the feature space, both feature maps are fed to the AdaIN layer, which aligns the mean and variance of the content feature map with the mean and variance of the style feature map, thereby producing a target feature map t: t = AdaIN(f(c),f(s)); The randomly initialized decoder g is trained to map t back to image space, generating a stylized image T(c, s): T(c,s)=g(t).

6. The method for recognizing and converting lumbar disc herniation images according to claim 4, characterized in that: Step S33 includes: first, inputting the CT image of lumbar disc herniation to be detected into the generator, the structure of each downsampling layer of the generator is convolution, IN regularization and Leaky ReLU activation function, before downsampling, a step of image enhancement operation is performed, and the ReflectionPad2d layer is introduced. After the first downsampling, the relevant features are obtained and used as the input of the next downsampling. At this time, the data becomes 64×256×256, the data becomes 128×128×128 after the second downsampling, and the data becomes 256×64×64 after the third downsampling. At this time, it reaches the bottom, and 9 residual modules are introduced between downsampling and upsampling to deepen the network and enhance the data at the same time, and then the upsampling operation is performed. The upsampling structure is deconvolution, IN normalization, ReLU The activation function is used to restore the size of the image, and the ReflectionPad2d layer is also used for data enhancement. The first upsampling receives the data from Resnet_block and converts the data to 256×64×64. At this time, the AdaIN layer is introduced to learn and transform the local context information. The second upsampling converts the data to 128×128×128, and the third upsampling converts the data to 64×256×256; the discriminator uses the data from the generator as input and converts the data to 64×128×128 through the convolution layer and LeakyReLU. Through three convolutions, IN regularization and Leaky ReLU activation function, the data is converted from 64×128×128 to 128×64×64, then to 256×32×32 and finally to 512×31×31, and then a convolution layer converts the data to 1×30×30.

7. The method for recognizing and transforming lumbar disc herniation images according to claim 6, characterized in that: In the final generation step, the corresponding iterative code is added so that the generated MRI image is generated repeatedly.

8. The method for recognizing and transforming lumbar disc herniation images according to claim 1, characterized in that: Step S4 uses an alternating training method to perform training.

9. The method for recognizing and transforming lumbar disc herniation images according to claim 1, characterized in that: Step S4 includes: S41. Generator training: a given input image is passed through the generator to generate a corresponding output image, and the generated image and the target image are sent to the discriminator for training. The training process first passes the input image through the generator to generate a corresponding output image; then the loss function between the generated image and the target image is calculated, and the error is back-propagated to update the parameters of the generator; finally, during the training process, the parameters of the generator are continuously updated by using stochastic gradient descent; S42. Training of the discriminator. The discriminator discriminates between the generated images and the target images to determine whether these images are real. The training process first randomly selects a group of real images and a group of generated images; then these images are input into the discriminator, and the probabilities that the real images and the generated images belong to the real data are calculated; the parameters of the discriminator are updated by calculating the loss function corresponding to the real images and the generated images, as well as the overall loss function of the discriminator, and back-propagating the error; finally, during the training process, the parameters of the discriminator are continuously updated by using stochastic gradient descent.

10. A lumbar disc herniation image recognition and conversion system, characterized by: include: An image acquisition module, used for acquiring CT images and MRI images of lumbar disc herniation of patients; An image preprocessing module is used to preprocess the collected lumbar disc herniation CT images and lumbar disc herniation MRI images to construct a training set; Model building module, used to build ITAD-pix model; Model training module, used to input the training set into the ITAD-pix model for training; The image recognition conversion module is used to input the untrained lumbar disc herniation CT image into the trained ITAD-pix model, and convert the untrained lumbar disc herniation CT image into the target lumbar disc herniation MRI image through the ITAD-pix model.

Images