Apparatus and method for determining risser grade
The CNN-based MMRotate deep learning model for pelvic image extraction from spinal radiographs addresses subjective re-grade determination issues, enhancing accuracy and efficiency in scoliosis assessment.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- SILLASYST CO LTD
- Filing Date
- 2024-11-19
- Publication Date
- 2026-05-28
AI Technical Summary
Conventional re-grade determination in scoliosis patients relies on subjective judgment, leading to evaluation errors and inefficiencies in measuring axial vertebral rotation and predicting the progression of scoliosis.
An apparatus and method using an object detection model, specifically a CNN-based MMRotate deep learning model, to extract pelvic images from spinal radiographs and determine re-grade by learning from a training dataset that includes right and left pelvic images, gender, and age, with sample augmentation through SMOTE, and employing multiple learning modules for accurate determination.
Provides accurate and efficient re-grade determination, reducing errors in surgical planning and improving the precision of spinal treatment by objectively assessing spinal curvature and rotation.
Smart Images
Figure KR2024018252_28052026_PF_FP_ABST
Abstract
Description
Rese grade determination device and method
[0001] The present invention relates to a rese-grade determination device and method, and more specifically, to a rese-grade determination device and method that extracts an image using an object detection model.
[0002] Meanwhile, the present invention (result) is the result of the Local Government-University Cooperation-based Regional Innovation Project conducted in 2024 with funding from the Ministry of Education and support from the National Research Foundation of Korea (2022RIS-006).
[0003] Scoliosis refers to a condition in which the spine is abnormally curved sideways, rotated vertically, or twisted. In this case, accurately measuring the curvature of the spine and its three-dimensional rotation is an important factor in determining the prognosis and treatment plan for scoliosis.
[0004] In clinical practice, the severity of scoliosis is measured by evaluating spinal curvature using Cobb's Angle. Cobb's Angle is determined by identifying the upper and lower vertebrae that are most inclined along the curve from the frontal view.
[0005] In addition, several studies emphasize the importance of measuring the severity of axial vertebral rotation (AVR) in addition to the Cobb angle in establishing treatment plans for scoliosis. Inaccurate measurement of axial vertebral rotation severity can lead to incorrect placement of pedicle screws during surgery, which can result in a risk of spinal cord injury.
[0006] One method for evaluating axial rotation of the spine is the Nash and Moe Method, introduced by Nash and Moe in 1969. This method identifies the vertebrae and measures the angle deviation and rotation angle based on pedicle asymmetry; it is widely used because of its simple execution method.
[0007] In addition, the Risser Grade is used to predict the progression of scoliosis based on the patient's pelvic X-ray.
[0008] Rese Grade is used as an indicator to evaluate a patient's growth, predict the progression of scoliosis based on growth, and determine the direction of treatment.
[0009] However, because conventional re-grade determination relies on the subjective judgment of the performer, there are problems of evaluation errors and inefficiency associated with manual measurement.
[0010] The present invention aims to solve the above-mentioned problems by providing an apparatus and method for extracting a pelvic image of a scoliosis patient using an object detection model and determining the regrade of the extracted image.
[0011] According to one embodiment of the present invention for achieving the above-mentioned purpose, a riser grade determination device is provided, comprising: an image extraction unit that extracts an entire pelvic region from a spinal radiograph and extracts a right pelvic image and a left pelvic image from the extracted entire pelvic region; a learning unit that generates riser grade determination information by learning a training dataset, in which a training image including a right pelvic image and a left pelvic image received from the image extraction unit is input, through a CNN model; and a judgment unit that determines the riser grade of a newly input right pelvic image and a left pelvic image from the image extraction unit through the determination information.
[0012] In addition, a resemblance grade determination device according to one embodiment of the present invention is characterized in that the training dataset includes a right pelvis image, a left pelvis image, gender, and age.
[0013] In addition, a resurgrade determination device according to one embodiment of the present invention is characterized in that the image extraction unit is an MMRotate-based deep learning model including a ReDet (Rotation-Equivariant Detector).
[0014] In addition, a re-grade determination device according to one embodiment of the present invention is characterized by comprising a first learning module that generates re-grade determination information by learning a training dataset, in which a training image including a right pelvis image is input, through a CNN model, and a second learning module that generates re-grade determination information by learning a training dataset, in which a training image including a left pelvis image is input, through a CNN model.
[0015] In addition, a re-grade determination device according to one embodiment of the present invention is characterized in that a determination unit is configured to first determine the re-grade through determination information of a first learning module and secondarily determine the re-grade through determination information of a second learning module.
[0016] In addition, a re-grade judgment device according to one embodiment of the present invention is characterized by further including a sample augmentation unit for inputting a training dataset of a learning unit by augmenting a training image generated from an image extraction unit through SMOTE.
[0017] Meanwhile, according to one embodiment of the present invention, a method for determining a lower grade is provided, comprising: a first image processing step for extracting an entire pelvic region from a spinal radiograph; a second image processing step for extracting a right pelvic image and a left pelvic image from the extracted entire pelvic region; a learning step for generating determination information for a lower grade by training a training dataset, which includes a training image including the right pelvic image and the left pelvic image extracted through the second image processing step, through a CNN model; and a determination step for determining a lower grade of the newly extracted right pelvic image and left pelvic image through the second image processing step using the determination information.
[0018] In addition, a resemblance grade determination method according to one embodiment of the present invention is characterized in that the training dataset includes a right pelvis image, a left pelvis image, gender, and age.
[0019] In addition, a regrade determination method according to one embodiment of the present invention is characterized in that the learning step comprises a first learning step of generating regrade determination information by learning a training dataset, in which a training image including a right pelvis image is input, through a CNN model, and a second learning step of generating regrade determination information by learning a training dataset, in which a training image including a left pelvis image is input, through a CNN model.
[0020] In addition, a re-grade determination method according to one embodiment of the present invention is characterized by determining the re-grade primarily through the determination information of the first learning step and determining the re-grade secondarily through the determination information of the second learning step in the determination step.
[0021] According to one embodiment of the present invention, an apparatus and method for extracting a pelvic image of a scoliosis patient using an object detection model and determining the regrade of the extracted image can be provided.
[0022] FIG. 1 is a drawing showing a resor grade determination device according to one embodiment of the present invention.
[0023] FIG. 2 is a diagram showing the step of extracting a pelvic image from a spinal radiograph in a rese-grade determination method according to one embodiment of the present invention.
[0024] Figure 3 is a diagram showing the division of pelvic regions and classification of grades in a rese grade determination method according to one embodiment of the present invention.
[0025] Figure 4 is a diagram showing the degree of bone growth graded according to the position of the pelvic growth plate in a resurgrade determination method according to one embodiment of the present invention.
[0026] FIGS. 5 and 6 are diagrams showing the detailed configurations of the right model and the left model, respectively, in the learning step of generating decision information for regrade according to an embodiment of the present invention.
[0027] FIG. 7 is a flowchart illustrating a re-grade determination method according to one embodiment of the present invention.
[0028] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings.
[0029] Additionally, identical or corresponding components are assigned the same or similar reference numbers regardless of drawing symbols, and redundant descriptions thereof are omitted; furthermore, for the convenience of explanation, the size and shape of each illustrated component may be exaggerated or reduced.
[0030] FIG. 1 is a drawing showing a resergrade determination device according to an embodiment of the present invention, FIG. 2 is a drawing showing a step of extracting a pelvic image from a spinal radiograph in a resergrade determination method according to an embodiment of the present invention, FIG. 3 is a drawing showing a pelvic region divided and grade classified in a resergrade determination method according to an embodiment of the present invention, and FIG. 4 is a drawing showing the degree of bone growth graded according to the location of the pelvic growth plate in a resergrade determination method according to an embodiment of the present invention.
[0031] In addition, FIGS. 5 and 6 are diagrams showing the detailed configurations of the right 0 and the left model, respectively, in the learning step of generating re-grade judgment information according to an embodiment of the present invention, and FIG. 7 is a flowchart showing a re-grade judgment method according to an embodiment of the present invention.
[0032] Referring to FIGS. 1 to 7, a re-grade determination device (100) according to one embodiment of the present invention may include an image extraction unit (110) that extracts a whole pelvic area from a spinal radiograph and extracts a right pelvic image and a left pelvic image from the extracted whole pelvic area, a learning unit (120) that generates re-grade determination information by learning a training dataset in which a training image including a right pelvic image and a left pelvic image received from the image extraction unit (110) is input through a CNN model, and a determination unit (130) that determines a re-grade using the determination information for a newly input right pelvic image and a left pelvic image from the image extraction unit (110).
[0033] An image extraction unit (110) according to one embodiment of the present invention may be an MMRotate-based deep learning model including a ReDet (Rotation-Equivariant Detector). In this case, the object detection deep learning model of the image extraction unit (110) may include a CNN (Convolutional Neural Network) algorithm.
[0034] CNN can be defined as a representative deep learning model that offers significant advantages in image recognition and classification. CNN can find features in image data that humans cannot recognize or detect.
[0035] Additionally, the image extraction unit (110) can receive spinal radiographs through at least one of an external radiographic unit (200) and a database (300).
[0036] The image extraction unit (110) can extract the entire pelvic area from the input spinal radiograph and can separate and extract the right pelvic image and the left pelvic image.
[0037] At this time, the re-grade judgment device (100) may include a sample augmentation unit (140) that generates new synthetic data from an image extracted from an image extraction unit (110). Specifically, the sample augmentation unit (140) can input the training image generated from the image extraction unit into the training dataset of the learning unit (120) by augmenting it through SMOTE (Synthetic Minority Over-sampling Technique).
[0038] Meanwhile, the training dataset used in the learning unit (120) may include right pelvic image, left pelvic image, gender, and age.
[0039] The learning unit (120) may include a first learning module that generates judgment information of the riser grade by learning a training dataset in which a training image including a right pelvis image is input through a CNN model, and a second learning module that generates judgment information of the riser grade by learning a training dataset in which a training image including a left pelvis image is input through a CNN model.
[0040] The first training module can be trained using data from the training dataset that includes right pelvic images, gender, and age.
[0041] In addition, the second learning module can be trained using data including left pelvic images, gender, and age from the training dataset.
[0042] Meanwhile, the judgment unit (130) can determine the re-grade primarily through the judgment information of the first learning module and secondarily determine the re-grade through the judgment information of the second learning module.
[0043] Meanwhile, a re-grade determination method according to one embodiment of the present invention may include a first image processing step (S100) for extracting an entire pelvic area from a spinal radiograph, a second image processing step (S200) for extracting a right pelvic image and a left pelvic image from the extracted entire pelvic area, a learning step for generating re-grade determination information by training a training dataset input with a CNN model that includes a training image including the right pelvic image and the left pelvic image extracted through the second image processing step (S200), and a determination step (S500) for determining a re-grade using the determination information for the newly extracted right pelvic image and left pelvic image from the second image processing step (S200).
[0044] In the first image processing step (S100), a spinal radiograph (a in FIG. 2) may be input through at least one of an external radiographic unit (200) and a database (300). At this time, the entire pelvic area (b in FIG. 2) may be extracted from the spinal radiograph by the object detection model of the image extraction unit (110).
[0045] Additionally, in the second image processing step (S200), the object detection model of the image extraction unit (110) can extract a right pelvis image (top of Fig. 2c) and a left pelvis image (bottom of Fig. 2c) from the entire pelvis area extracted in the first image processing step.
[0046] At this time, the object detection models used in the first image processing step (S100) and the second image processing step (S200) may be provided as different models.
[0047] Meanwhile, in the learning phase, the right pelvic image and the left pelvic image extracted in the second image processing step (S200) can be prepared as a training dataset. At this time, the training dataset may include the right pelvic image, the left pelvic image, gender, and age.
[0048] Additionally, the learning step may include a first learning step (S300) that generates judgment information for a lower grade by learning a training dataset in which a training image including a right pelvis image is input through a CNN model, and a second learning step (S400) that generates judgment information for a lower grade by learning a training dataset in which a training image including a left pelvis image is input through a CNN model.
[0049] Additionally, in the judgment step (S500), the judgment unit (130) may be configured to first determine the regrade through the judgment information of the first learning step (S300) and secondarily determine the regrade through the judgment information of the second learning step (S400).
[0050] In the image processing step according to an embodiment of the present invention, the object detection model may include a Convolutional Neural Network (CNN) algorithm.
[0051] Convolutional neural networks are representative deep learning models that have significant advantages in image recognition and classification, and can find features in image data that humans cannot recognize or detect.
[0052] The deep learning model according to an embodiment of the present invention was developed using an algorithm based on standing anterior-posterior full spinal radiographs of a patient. The photographs used in the deep learning model according to an embodiment of the present invention are of patients aged 9 to 19, and photographs with interference from external objects or spinal implants were excluded.
[0053] In an embodiment of the present invention, a deep learning model based on MMRotate, an open-source toolbox designed to detect rotated objects using the PyTorch 2 framework, was used. MMRotate includes algorithms such as ReDet, faster-rcnn, and oriented-reppoints.
[0054] The research grade decision model was developed using Python 3.8, PyTorch 2.0, and CUDA 11.7. The model was trained using full training and transfer learning techniques, and the AdamW optimizer was used for network optimization. Various optimizers, including the SGD optimizer, can be used instead of the AdamW optimizer.
[0055] The training dataset consisted of a total of 3,318 images, with 1,659 images each of the right pelvic ROI (Region of Interest) and the left pelvic ROI; 1,295 images (80%) of the right and left sides were used for training, and 364 images (20.0%) were used for validation. The Random Flip augmentation method was used to optimize the training process for all three axes (horizontal, vertical, and diagonal).
[0056] In addition, to improve the performance of the research grade reading model, an ensemble model can be constructed by training multiple CNN models and a CNN combined with clinical data model, and then hard voting and soft voting techniques can be applied.
[0057] Meanwhile, Resur Grade provides a standardized clinical protocol for evaluating patient growth. This method assesses grades based on the maturity of the iliac crest, assigning a grade according to the extent of the epiphyses covering the iliac crest.
[0058] At this time, since it is difficult to distinguish between Grade 0 and Grade 5 in the image, the patient's gender and age data may be used together. For example, if the patient's gender is female and she is 16 years old, the judgment unit (130) can determine the patient's re-grade as Grade 5.
[0059] The preferred embodiments of the present invention described above are disclosed for illustrative purposes only, and various modifications and variations of the technical concept of the present invention are possible by those skilled in the art to which the present invention pertains, and such modifications and variations will fall within the scope of protection of the present invention.
Claims
1. An image extraction unit that extracts the entire pelvic region from a spinal radiograph and extracts a right pelvic image and a left pelvic image from the extracted entire pelvic region; A learning unit that generates Risser Grade judgment information by training a training dataset, which is input with training images including a right pelvis image and a left pelvis image received from the image extraction unit, through a CNN model; and A re-grade determination device comprising a judgment unit that determines a re-grade using the judgment information for the newly input right pelvic image and left pelvic image from the image extraction unit.
2. In Paragraph 1, The above training dataset is a Rese Grade judgment device including right pelvic image, left pelvic image, gender, and age.
3. In Paragraph 1, The above image extraction unit is a re-grade determination device that is an MMRotate-based deep learning model including ReDet (Rotation-Equivariant Detector).
4. In Paragraph 1, The above learning unit comprises: a first learning module that generates Risser Grade judgment information by learning a training dataset, into which a training image including a right pelvis image is input, through a CNN model; and A re-grade judgment device comprising a second learning module that generates re-grade judgment information by training a training dataset, into which a training image including a left pelvis image is input, through a CNN model.
5. In Paragraph 4, A re-grade determination device configured such that the above-mentioned judgment unit first determines the re-grade through the judgment information of the first learning module and secondarily determines the re-grade through the judgment information of the second learning module.
6. In Paragraph 1, A re-grade judgment device further comprising a sample augmentation unit for inputting a training dataset of a learning unit by augmenting a training image generated from the above image extraction unit through SMOTE.
7. A method for determining a re-grade using a re-grade determination device according to claim 1, A first image processing step for extracting the entire pelvic area from a spinal radiograph; A second image processing step for extracting a right pelvis image and a left pelvis image from the entire extracted pelvic region; A learning step for generating re-grade judgment information by training a training dataset, into which training images including a right pelvis image and a left pelvis image extracted through the second image processing step are input, through a CNN model; and A method for determining a regrade, comprising a judgment step for determining a regrade using the judgment information for the newly extracted right pelvic image and left pelvic image through a second image processing step.
8. In Paragraph 7, The training dataset is a Research Grade judgment method including right pelvic images, left pelvic images, gender, and age.
9. In Paragraph 7, The above learning step comprises: a first learning step of generating re-grade judgment information by training a training dataset, into which a training image including a right pelvis image is input, through a CNN model; and A regrade determination method comprising a second training step of generating regrade determination information by training a training dataset, in which training images including left pelvic images are input, through a CNN model.
10. In Paragraph 9, A method for determining a regrade in the above-mentioned judgment step, wherein the regrade is determined primarily through the judgment information of the first learning step and the regrade is determined secondarily through the judgment information of the second learning step.