Device for detecting scoliosis on medical image and electronic equipment
By designing a device that includes units such as image enhancement, boundary extraction and area of interest detection, it automatically detects scoliosis in medical images, solving the problems of long manual measurement time and insufficient accuracy in automatic evaluation in the prior art, and achieving efficient and accurate scoliosis detection.
Patent Information
- Application Number
- CN202311734318.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-15
- Publication Date
- 2025-06-17
AI Technical Summary
The prior art has problems such as long manual measurement time, large energy consumption, large observer differences, and insufficient accuracy and universality of automatic evaluation methods when detecting scoliosis.
Design a device and electronic device to automatically detect scoliosis in medical images through image enhancement and preprocessing, boundary extraction, area of interest detection and maximum Cobb angle recognition and other units, calculate the Cobb angle and determine the type and degree of curvature.
It realizes fully automated scoliosis detection, improves detection efficiency and accuracy, reduces errors in manual measurement by doctors, is suitable for a variety of clinical scenarios, and provides intuitive and easy-to-understand output results.
Smart Images
Figure CN120163757A_ABST
Abstract
Description
[0001] In this patent application, nouns and pronouns related to people are not limited to a specific gender. Technical Field
[0002] The present invention relates to the detection of scoliosis. Background Art
[0003] Lateral spinal deformity is called scoliosis. The sites of scoliosis are mostly in the thoracic and lumbar vertebrae. There are many causes of scoliosis: structural scoliosis caused by bone, muscle, nerve lesions, etc.; non-structural scoliosis caused by pain, inflammation, etc. Its incidence rate in China is about 2.4%, and it shows an increasing trend year by year. According to the deviation situation, it can be divided into S-shaped and C-shaped scoliosis. The S-shaped variation is called double scoliosis: The S-shaped scoliosis is becoming more and more frequent, and it may deteriorate rapidly, so it is a more dangerous disease. The C-shaped variation is also called the overall curve, which is divided into right scoliosis and left scoliosis. If the scoliosis angle is less than 10 degrees, it is called spinal asymmetry, and if the scoliosis angle is greater than 10 degrees, it is called scoliosis. In addition to causing an unattractive body appearance, if this kind of deformity is severe, it may cause cardiopulmonary failure due to compression of internal organs such as the heart.
[0004] Currently, most doctors' measurements of scoliosis are based on visual inspection. Doctors will first visually judge whether the scoliosis degree is less than 10°. If it is less than 10°, it will be regarded as a negative case. If doctors visually judge that the scoliosis degree is approximately greater than or equal to 10°, doctors will manually measure the Cobb angle to quantitatively evaluate the bending curvature, so as to determine the severity of scoliosis. Figure 1 Schematic diagram for measuring the Cobb angle. As Figure 1 shown, doctors need to visually find a pair (C-shaped scoliosis) or two pairs (S-shaped scoliosis) of vertebral bodies with the largest inclination degree towards the concave side of the scoliosis among the scoliosis, then draw a parallel horizontal line on the vertebral margins of a pair of upper and lower end vertebrae respectively, and finally draw the perpendicular lines of these two parallel lines and manually measure the angle.
[0005] However, manually measuring spinal curvature requires a relatively long time and a considerable amount of energy, and there are also related problems such as differences between observers. In addition, the variability range of manually measuring the Cobb angle is 3° to 10°, and there will also be relatively large errors in manual accuracy.
[0006] In the past five years, a large number of methods for automatically evaluating the Cobb angle have been proposed. Since it is not necessary to manually extract features like traditional algorithms (such as machine learning algorithms), algorithms based on deep learning have shown great advantages. However, there are still problems with the accuracy, universality, stability, and robustness of the model.
[0007] The methods for predicting the Cobb angle based on deep learning can be roughly divided into two categories.
[0008] One type is the regression task, which aims to learn the shape features of the original image or vertebrae and establish an association with the corresponding Cobb angle. The method of calculating the Cobb angle using only a simple regression task does not consider clinical rationality, resulting in the calculated Cobb angle not conforming to clinical rules. In principle, a Cobb angle can only contain one spinal curvature, so a simple regression task can only generate one Cobb angle. For complex cases such as S-shaped scoliosis, Cobb angles with two curvatures need to be generated to enable doctors to make a more complete judgment on the severity of the patient. Additionally, this basic regression method also results in a high false positive rate, thus hindering its popularization in real clinical screening scenarios.
[0009] Another type is the method using vertebral key point detection. The automatic evaluation method based on vertebral key points calculates the slope of the curve formed by three key points to calculate the vertebral inclination and indirectly estimate the Cobb angle. Its evaluation quality of the Cobb angle highly depends on the accurate positioning of the key points, which means that a tiny prediction error in the key point coordinates may lead to a huge deviation in the final result. And due to the problem of organ tissue overlap and occlusion in X-ray images, vertebral key points may be obscured, and it is usually inevitable to have errors in predicting the key point coordinates. Therefore, there is currently a lack of an effective and reliable workflow for diagnosing scoliosis in X-ray images to adapt to a wide range of clinical scenarios. Summary of the Invention
[0010] In view of this, the present invention proposes a device and an electronic device for detecting scoliosis in medical images.
[0011] According to a first aspect of the present invention, there is provided a device for detecting scoliosis in a medical image, where the medical image is anteroposterior or posteroanterior, and includes: an image enhancement and preprocessing unit, which performs image enhancement and preprocessing on the medical image; a boundary extraction unit, which extracts the upper and lower boundaries of each vertebra on the medical image; a region of interest detection unit, which detects one or a plurality of regions of interest on the medical image that may have scoliosis; a maximum Cobb angle recognition unit, which calculates a plurality of Cobb angles for each region of interest and identifies the largest Cobb angle as the maximum Cobb angle of the region of interest; and a scoliosis detection unit, which determines the type and degree of scoliosis based on the number and size of the maximum Cobb angles.
[0012] In an embodiment, the image enhancement and preprocessing unit includes an image enhancement unit and a preprocessing unit.
[0013] In one embodiment, the image enhancement unit is configured to: scale the medical image with a 50% probability using a random non-linear histogram; rotate the medical image with a 50% probability randomly between -20 and +20 degrees; rotate the medical image with a 50% probability along the y-axis, where the y-axis is the height direction of the medical image; scale the medical image with a 50% probability randomly between 0.8 times and 1.2 times; add random noise to the medical image such that I = I + aN, where I represents the image intensity, N is uniformly random noise generated within the range min , I max , and I min , I max are the minimum intensity and the maximum intensity of the medical image respectively.
[0014] In one embodiment, the preprocessing unit is configured to: resample the medical image to 1024×1024 using bilinear interpolation; normalize the medical image according to the following formula: where I represents the image intensity, and I min , I max are the minimum intensity and the maximum intensity of the medical image respectively.
[0015] In one embodiment, the boundary extraction unit includes: a mask determination unit that determines a segmentation mask of the visible spine in the medical image; a boundary determination unit that determines the upper and lower boundaries of the segmentation mask using the minimum bounding rectangle method.
[0016] In one embodiment, the region of interest detection unit is configured to detect one or more regions of interest in the medical image that may have scoliosis through a vision transformer.
[0017] In one embodiment, the maximum Cobb angle recognition unit is configured to: in each region of interest, traverse the upper boundary of each vertebra and the lower boundary of its lower vertebra, calculate the Cobb angle of each pair of upper and lower boundaries, and identify the maximum Cobb angle among them as the maximum Cobb angle of the region of interest.
[0018] In one embodiment, the medical image is one of an X-ray image, a CT image, and an MR image.
[0019] According to a second aspect of the present invention, there is provided an electronic device including the apparatus for detecting scoliosis on a medical image described above.
[0020] The device for detecting scoliosis on medical images and related electronic devices according to the present invention can optimize the workflow of spinal screening based on anteroposterior or posteroanterior medical images, and automatically assist doctors in detecting scoliosis (judging whether there is scoliosis) and abnormal judgment of scoliosis (C-shaped, S-shaped, others). In addition, the present invention can automatically provide spinal segmentation results, automatically extract the upper and lower edges of the spine, and automatically provide the Cobb angle of the abnormal spinal segment, so as to assist doctors in making a more comprehensive assessment of the final judgment of scoliosis, effectively preventing radiologists from missing the diagnosis of scoliosis or misjudging the abnormal classification of the spinal curvature degree, which may delay the patient's condition and subsequent treatment. The present invention also has the following advantages:
[0021] 1. Short output result time.
[0022] 2. Good universality, applicable to various clinical scenarios.
[0023] 3. The output display is intuitive and easy to understand, and the generated quantitative and qualitative results can assist doctors in judging the condition.
[0024] 4. The measurement results are stable and the repeatability is relatively strong.
[0025] 5. Good ease of use. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that those of ordinary skill in the art can more clearly understand the above and other features and advantages of the present invention. In the drawings:
[0027] Figure 1 It is a schematic diagram for measuring the Cobb angle.
[0028] Figure 2 It is a schematic structural block diagram of the device for detecting scoliosis on medical images according to an embodiment of the present invention.
[0029] Figure 3 For Figure 2 The schematic structural block diagram of the image enhancement and preprocessing unit of the device for detecting scoliosis on medical images.
[0030] Figure 4 For Figure 2 The schematic structural block diagram of the boundary extraction unit of the device for detecting scoliosis on medical images.
[0031] Figure 5 For Figure 4 The schematic diagram of the boundary extraction unit for determining the segmentation mask of the spine.
[0032] Figure 6 For Figure 2Schematic diagram of the region of interest detection unit of the device for detecting scoliosis on medical images, which detects one or more regions of interest that may have scoliosis on the medical images through a vision transformer.
[0033] Figure 7 For Figure 2 Schematic diagram of the device for detecting scoliosis on medical images showing the maximum Cobb angle on the region of interest.
[0034] In the above figures, the reference numerals used are as follows:
[0035] 100 Device for detecting scoliosis on medical images 110 Scoliosis detection unit
[0036] 102 Image enhancement and preprocessing unit 112 Image enhancement unit
[0037] 104 Boundary extraction unit 114 Preprocessing unit
[0038] 106 Region of interest detection unit 116 Mask determination unit
[0039] 108 Maximum Cobb angle recognition unit 118 Boundary determination unit Detailed implementation manners
[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the following examples are given to further elaborate on the present invention in detail.
[0041] Figure 2 Schematic structural block diagram of the device 100 for detecting scoliosis on medical images according to an embodiment of the present invention. The medical image is anteroposterior or posteroanterior, and can be one of an X-ray image, a CT image, an MR image, etc. In this embodiment, the medical image is an X-ray image. The device 100 for detecting scoliosis on medical images includes an image enhancement and preprocessing unit 102, a boundary extraction unit 104, a region of interest detection unit 106, a maximum Cobb angle recognition unit 108, and a scoliosis detection unit 110.
[0042] The image enhancement and preprocessing unit 102 performs image enhancement and preprocessing on the medical image. Figure 3 For Figure 2 Schematic structural block diagram of the image enhancement and preprocessing unit 102 of the device 100 for detecting scoliosis on medical images. As Figure 3 shown, the image enhancement and preprocessing unit 102 includes an image enhancement unit 112 and a preprocessing unit 114.
[0043] In this embodiment, the image enhancement unit 112 is configured to: scale the medical image randomly with a non - linear histogram with a 50% probability; rotate the medical image randomly between - 20 and + 20 degrees with a 50% probability; rotate the medical image along the y - axis (the y - axis is the height direction of the medical image, i.e., the up - and - down direction of the spine. In the present invention, the "up" and "down" of the spine, vertebra, and vertebral body respectively correspond to the directions close to the head and hip) with a 50% probability; scale the medical image randomly between 0.8 times and 1.2 times with a 50% probability; add random noise to the medical image such that I = I + aN, where I represents the image intensity, N is a uniformly random noise generated within the range of min , I max , and I min , I max are the minimum intensity and the maximum intensity of the medical image respectively.
[0044] In this embodiment, the pre - processing unit 114 is configured to: resample the medical image to 1024×1024 using bilinear interpolation; normalize the medical image according to the following formula: where I represents the image intensity, and I min , I max are the minimum intensity and the maximum intensity of the medical image respectively.
[0045] The boundary extraction unit 104 extracts the upper and lower boundaries of each vertebral body on the medical image. Figure 4 FIG. Figure 2 is a schematic structural block diagram of the boundary extraction unit 104 of the scoliosis detection device 100 on the medical image. As Figure 4 shown, in this embodiment, the boundary extraction unit 104 includes a mask determination unit 116 and a boundary determination unit 118. The mask determination unit 116 determines the segmentation mask of the visible vertebrae in the medical image. Figure 5 FIG. Figure 4Schematic diagram of the segmentation mask of the spine determined by the boundary extraction unit 104. Before processing medical images, the image enhancement and preprocessing unit 102 of the device 100 can perform image enhancement and preprocessing on a large number of other medical images, and then the mask determination unit 116 performs deep learning on them, for example, through the UNet algorithm. The UNet model has an encoder-decoder structure. The backbone based on the convolutional neural network can be used to construct the encoder. Each encoder consists of four layers, arranged in the following order: convolutional layer, BN layer, ReLU activation layer, and max pooling layer. Each encoder (downsampling) includes two 3*3 convolutions, BN, and leaky ReLU, as well as a max pooling layer with a stride of 2. Finally, the downsampling path is shared and updated jointly by the two tasks of contour and mask information. The multi-task model considering the boundary is proved to be able to effectively enhance the robustness and suppress outliers. The formula of the loss function designed in the present invention is: where Pred represents the prediction result and Target represents the actual value. In order to make the model pay more attention to the boundary information, a boundary-aware attention mechanism is also designed in this embodiment. Finally, the segmentation mask (mask) and segmentation boundary of all visible spines (vertebrae) in the X-ray image can be obtained (the present invention may not utilize the segmentation boundary).
[0046] The purpose of fine spine segmentation is to extract the upper and lower boundaries of each vertebra in order to calculate the Cobb angle. The boundary determination unit 118 determines the upper and lower boundaries of the segmentation mask (corresponding to the vertebra) by the minimum bounding rectangle method (MBR). The upper and lower boundaries of each minimum matrix are the upper boundary (BDup) and lower boundary (BDdown) of each vertebra.
[0047] The region of interest detection unit 106 detects one or more regions of interest in the medical image that may have scoliosis. Before processing the medical image, the image enhancement and preprocessing unit 102 of the device 100 can perform image enhancement and preprocessing on a large number of other medical images, and then the region of interest detection unit 106 performs deep learning on them. In this embodiment, an object detection algorithm based on vision transformer is adopted, and this algorithm has unique advantages for detecting large lesions and can pay more attention to global features. Figure 6 For Figure 2 Schematic diagram of the region of interest detection unit 106 of the device 100 for detecting scoliosis on a medical image detecting one or more regions of interest (ROIs) in the medical image that may have scoliosis by vision transformer. As Figure 6As shown, the object detection algorithm includes three main components: a CNN backbone for extracting a compact feature representation; an encoder-decoder acting as a transformer; and a simple feed-forward network (FFN) for predicting the final possible scoliosis ROI (i.e., the abnormal ROI, which can be marked with an ROI box on the image).
[0048] The maximum Cobb angle recognition unit 108 calculates a plurality of Cobb angles for each region of interest and identifies the largest Cobb angle among them as the maximum Cobb angle of the region of interest. The maximum Cobb angle recognition unit 108 is configured to: in each region of interest, traverse the upper boundary of each vertebra and the lower boundary of its lower vertebra, calculate the Cobb angle of each pair of upper and lower boundaries, and identify the largest Cobb angle among them as the maximum Cobb angle of the region of interest. Each abnormal ROI box can be represented by Broi n , where n represents the ordinal number of the ROI box. Each ROI box contains m(n) (m is a function of n, hereinafter only represented by m) segmentation masks (mask m ), as well as m upper boundaries of the vertebrae (BDup m ) and m lower boundaries of the vertebrae (BDdown m ). Therefore, in the abnormal ROI box Broin, theoretically Cobb angles can be obtained, and the largest Cobb angle among them can be used as the maximum Cobb angle of the region of interest. The set of the maximum Cobb angles can be represented by α Cobb .
[0049] The scoliosis detection unit 110 determines the type and degree of scoliosis based on the number and size of the maximum Cobb angles. For a certain region of interest, if the maximum Cobb angle is less than 10°, it is determined to be a normal spinal curvature, and the Cobb angle may not be displayed on the image. Otherwise, the maximum Cobb value can be output near the region of interest on the medical image for doctors' reference. Figure 7 For Figure 2 Figure 100 shows a schematic diagram of the device for detecting scoliosis on a medical image, which displays the maximum Cobb angle on the region of interest.
[0050] Regarding the type of scoliosis:
[0051] If there is only one maximum Cobb angle greater than or equal to 10°, it can be determined as type C scoliosis;
[0052] If there are two maximum Cobb angles greater than or equal to 10°, it can be determined as type S scoliosis;
[0053] If the number of maximum Cobb angles greater than or equal to 10° exceeds two, it can be determined as other types of scoliosis.
[0054] Regarding the degree of scoliosis:
[0055] If no abnormal ROI is detected, or max(α cobb ) is less than 10°, it is determined as normal spinal curvature;
[0056] If 10° ≤ max(α cobb ) < 20°, it is determined as mild scoliosis;
[0057] If 20° ≤ max(α cobb ) < 40°, it is determined as moderate scoliosis;
[0058] If max(α cobb ) ≥ 40°, it is determined as severe scoliosis.
[0059] The type and degree of scoliosis determined by the scoliosis detection unit 110 can be used as a reference for doctors, and doctors can make a diagnosis in combination with clinical conditions.
[0060] Part or all of the device 100 for detecting scoliosis on medical images can be deployed in the cloud, or the pattern of the unit involving deep learning after learning can be packaged locally and an interface can be provided for calling.
[0061] According to another aspect of the present invention, an electronic device is provided, which includes the device 100 for detecting scoliosis on medical images.
[0062] The device for detecting scoliosis on medical images and the related electronic device of the present invention can optimize the spinal screening workflow based on anteroposterior or posteroanterior medical images, and automatically help doctors detect scoliosis (judge whether there is scoliosis) and make abnormal judgments on scoliosis (C type, S type, others). In addition, the present invention can automatically provide the spinal segmentation result, automatically extract the upper and lower edges of the spine, and automatically provide the cobb angle of the abnormal spinal segment, so as to help doctors make a more comprehensive assessment for the final judgment of scoliosis, effectively preventing radiologists from missing the diagnosis of scoliosis or misjudging the abnormal classification of the spinal curvature degree, which may delay the patient's condition and subsequent treatment. The present invention also has the following advantages:
[0063] 1. The output result time is short.
[0064] 2. It has good universality and is applicable to various clinical scenarios.
[0065] 3. The output display is intuitive and easy to understand, and the generated quantitative and qualitative results can assist doctors in judging the condition.
[0066] 4. The measurement result is stable and the repeatability is relatively strong.
[0067] 5. It has better usability.
[0068] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.
Claims
1. An apparatus for detecting scoliosis on medical images, wherein the medical images are anteroposterior or posteroanterior, comprising: An image enhancement and preprocessing unit (102) that performs image enhancement and preprocessing on the medical image; A boundary extraction unit (104) that extracts the upper and lower boundaries of each vertebra on the medical image; A region of interest detection unit (106) that detects one or more regions of interest on the medical image that may have scoliosis; A maximum Cobb angle recognition unit (108) that calculates a plurality of Cobb angles for each region of interest and identifies the largest Cobb angle among them as the maximum Cobb angle of the region of interest; A scoliosis detection unit (110) that determines the type and degree of scoliosis based on the number and magnitude of the maximum Cobb angles.
2. The apparatus according to claim 1, characterized in that, The image enhancement and preprocessing unit (102) includes an image enhancement unit (112) and a preprocessing unit (114).
3. The apparatus according to claim 2, characterized in that, The image enhancement unit (112) is configured to: Scale the medical image with a 50% probability using a random non-linear histogram; Rotate the medical image with a 50% probability between -20 and +20 degrees; Rotate the medical image with a 50% probability along the y-axis, where the y-axis is the height direction of the medical image; Scale the medical image with a 50% probability between 0.8 times and 1.2 times; Add random noise to the medical image such that I = I + aN, where I represents the image intensity, N is uniform random noise generated within the range of min , I max , a is an attenuation factor randomly selected within the range of [0, 0.15], and I min , I max are the minimum intensity and the maximum intensity of the medical image, respectively.
4. The apparatus according to claim 2, characterized in that, The preprocessing unit (114) is configured to: Resample the medical image to 1024×1024 using bilinear interpolation; Normalize the medical image according to the following formula: where I represents the image intensity, and I min , I max are the minimum intensity and the maximum intensity of the medical image, respectively.
5. The apparatus according to claim 1, characterized in that, The boundary extraction unit (104) includes: A mask determination unit (116) that determines a segmentation mask of the visible vertebrae in the medical image; A boundary determination unit (118) that determines the upper and lower boundaries of the segmentation mask using the minimum bounding matrix method.
6. The apparatus according to claim 1, characterized in that, The region of interest detection unit (106) is configured to detect one or more regions of interest on the medical image that may have scoliosis through a vision transformer.
7. The apparatus according to claim 1, characterized in that, The maximum Cobb angle recognition unit (108) is configured to: In each region of interest, traverse the upper boundary of each vertebra and the lower boundary of its lower vertebra, calculate the Cobb angle of each pair of upper and lower boundaries, and identify the largest Cobb angle among them as the maximum Cobb angle of the region of interest.
8. The apparatus according to claim 1, characterized in that, The medical image is one of an X-ray image, a CT image, and an MR image.
9. An electronic device, comprising the apparatus for detecting scoliosis on medical images according to any one of claims 1 to 8.