A method for scoliosis detection based on deep learning

Abstract

The present invention relates to a method for detecting scoliosis based on deep learning, and the method includes: S1, obtaining a spinal X-ray image; S2, inputting the spinal X-ray image into a deep learning model to predict the center point of the spine and the positioning points for positioning; S3, filtering the predicted center point of the spine; S4, fitting a spinal curve based on the filtered center point of the spine; S5, determining the curved segments of the spine according to the fitted spinal curve, and determining the end vertebrae of each curved segment; S6, classifying the scoliosis of each curved segment; S7, calculating the Cobb angle of each curved segment based on the end vertebrae of each curved segment. Compared with the prior art, the detection efficiency, accuracy and consistency of the detection results of the present invention are high.

Description

Technical Field

[0001] The present invention relates to a method for detecting scoliosis, and more particularly to a method for detecting scoliosis based on deep learning. Background Art

[0002] Scoliosis is a common disease that occurs frequently in teenagers and children. Generally, a lateral curvature of the spine greater than 10 degrees shown on a spinal X-ray can be diagnosed as scoliosis. Scoliosis can not only cause an unattractive body shape such as hunchback, pelvic tilt, and body distortion; it can also lead to thoracic deformity and a decrease in thoracic volume, affecting cardiopulmonary function; in severe cases, it will compress the spinal nerves and blood vessels, hindering nerve transmission and blood supply, resulting in reduced attention and intelligence, low mood, and affected daily life of the patient. In China, the screening project for abnormal spinal curvature has been included in the physical examination content for each school year or new students.

[0003] Traditional methods for determining the degree of scoliosis mainly rely on manual operation. First, obtain the X-ray film of the patient's spine, and then the doctor determines the type and location of the scoliosis according to personal experience, and at the same time calibrates and draws lines on the X-ray film, and uses a protractor to measure and calculate the Cobb angle. Such a measurement method usually takes several minutes or dozens of minutes to complete the detection of a spinal X-ray film, and due to the different professional knowledge levels of different detectors, there will be certain differences in the obtained measurement results, that is, it is easily affected by the personal subjective factors of the measurer. Therefore, this manual method is time-consuming, inefficient, and has poor accuracy, and at the same time, the consistency of the scoliosis type, location, and Cobb angle is poor.

[0004] Early detection and treatment of scoliosis are of great significance to prevent the development of severe deformities. In recent years, with the increasing attention to adolescent scoliosis, early screening of scoliosis will gradually become popular. For the initially detected scoliosis cases, X-ray film detection is required.

[0005] Due to the time-consuming, low-efficiency, and consistency problems of the manual measurement method, there is currently a need for a method that can quickly and accurately detect scoliosis, improving the efficiency, accuracy, and consistency of the detection results. Summary of the Invention

[0006] The purpose of the present invention is to overcome the above-mentioned defects existing in the prior art and provide a method for detecting scoliosis based on deep learning, thereby improving the efficiency, accuracy, and consistency of the detection results.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A method for detecting scoliosis based on deep learning, the method comprising:

[0009] S1. Obtain spinal X-ray images;

[0010] S2. Input the spinal X-ray images into a deep learning model to predict the spinal center points and the positioning points for localization;

[0011] S3. Filter the predicted spinal center points;

[0012] S4. Fit the spinal curve based on the filtered spinal center points;

[0013] S5. Determine the spinal curvature segments according to the fitted spinal curve, and determine the end vertebrae of each curvature segment;

[0014] S6. Classify the scoliosis of each curvature segment;

[0015] S7. Calculate the Cobb angle of each curvature segment based on the end vertebrae of each curvature segment.

[0016] Preferably, the deep learning model includes a feature extraction network for extracting features of spinal X-ray images and a segmentation network for medical segmentation.

[0017] Preferably, the spinal center points predicted by the deep learning model include the spinal center points of the cervical vertebrae, thoracic vertebrae, and lumbar vertebrae, and the positioning points include 2 groups of positioning points located at the first group of collarbones and hip bones respectively, and each group includes two positioning points located on both sides of the spinal line.

[0018] Preferably, the deep learning model is trained using the method of transfer learning, including: first training the feature extraction network until the model converges; then, freezing and protecting the weights of the previously trained feature extraction network, and training the entire deep learning model; finally, unfreezing the weights of the frozen feature extraction network and training the entire deep learning model.

[0019] Preferably, the feature extraction network includes a MobileNet network, and the segmentation network includes a U-Net network.

[0020] Preferably, step S3 includes:

[0021] The predicted cervical vertebra center points can be filtered according to the four predicted positioning points;

[0022] For the noise points with distances close to the accurately predicted center points, the noise points are filtered by the predicted scores of the pixel points within a certain pixel range of each predicted point;

[0023] For the noise points with larger distances, calculate the absolute value of the difference between the abscissa of each point and the average value of the abscissas of all predicted points. If it exceeds a given threshold, it is considered a noise point and the noise point is filtered.

[0024] Preferably, step S5 is specifically as follows: The spinal curve is divided into several bending segments according to the inflection points obtained by taking the second derivative of the fitted spinal curve, and the end vertebra is determined at each inflection point position.

[0025] Preferably, step S6 is specifically as follows: For each bending segment, obtain the type of the vertebra in the bending segment that is farthest from the mid-perpendicular line of the hip bone, and determine the scoliosis classification.

[0026] Preferably, step S7 is specifically as follows:

[0027] For each bending segment, obtain the upper end vertebra and the lower end vertebra corresponding to the bending segment;

[0028] Based on the fitted spinal curve, determine the normal vectors of the intervertebral discs at the positions of the upper end vertebra and the lower end vertebra respectively;

[0029] Correct the two normal vectors;

[0030] Obtain the included angle between the two normal vectors as the cobb angle of the current bending segment.

[0031] Preferably, the method for normal vector correction is as follows:

[0032] Calculate the normal vector correction angle:

[0033]

[0034] where i represents the index of the spinal center point, d i is the normal vector slope at the i-th intervertebral disc on the fitted spinal curve, α is the correction factor, and Δ i is the normal vector correction angle at the i-th spinal center point.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] (1) The present invention can accurately predict the spinal center point through the deep learning model and calculate the cobb angle, which can be realized by machine without manual calculation, thereby improving the detection efficiency, accuracy and consistency of the detection results.

[0037] (2) Before calculating the cobb angle in the present invention, the two normal vectors are corrected to improve the accuracy of the result. Description of the Drawings

[0038] Figure 1 is the flowchart of a scoliosis detection method based on deep learning according to the present invention;

[0039] Figure 2 is the schematic diagram of the key points predicted by the trained deep learning model in the example of the present invention;

[0040] Figure 3Schematic diagram of key points predicted by the deep learning model in the embodiments of the present invention after noise filtering;

[0041] Figure 4 Schematic diagram of fitting the spinal curve with key points after noise filtering in the embodiments of the present invention;

[0042] Figure 5 Schematic diagram of the detection result detected by the scoliosis detection method based on deep learning in the embodiments of the present invention. Detailed implementation manners

[0043] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the following description of the embodiments is only illustrative in nature, and the present invention is not intended to limit the objects or uses to which it is applicable, nor is the present invention limited to the following embodiments.

[0044] Embodiment

[0045] The diagnosis of scoliosis uses the Cobb angle as an evaluation index. The Cobb angle is determined by the end vertebrae, which are the vertebrae with the greatest degree of inclination at the scoliosis of the spine. At present, the measurement of the Cobb angle mainly requires the measurer to first determine the end vertebrae, draw lines at the end vertebrae, and then measure the angle between the two end vertebrae of the drawn lines with a protractor. It can be seen that such a measurement method is inefficient and greatly affected by the professional level and subjective experience of the measurer. At the same time, with the increasing attention to adolescent idiopathic scoliosis in recent years, the preliminary screening diagnosis of scoliosis in adolescents will gradually become popular. When scoliosis is found during screening, further X-ray diagnosis is required. Obviously, the traditional manual measurement method is time-consuming, laborious and inefficient.

[0046] Therefore, the present application uses the technologies of image processing and deep learning, and proposes a scoliosis detection method based on deep learning in view of the defects of the traditional manual measurement of the Cobb angle method, such as large workload, poor accuracy and consistency.

[0047] As Figure 1 shown, the present embodiment provides a scoliosis detection method based on deep learning, and the method includes:

[0048] S1. Obtain a spinal X-ray image;

[0049] S2. Input the spinal X-ray image into a deep learning model to predict the center point of the spine and the positioning points for positioning;

[0050] S3. Filter the predicted center point of the spine;

[0051] S4. Fit the spinal curve based on the filtered center point of the spine;

[0052] S5. Determine the spinal curvature segments based on the fitted spinal curve and determine the end vertebrae of each curvature segment.

[0053] S6. Classify the scoliosis type for each curvature segment.

[0054] S7. Calculate the Cobb angle of each curvature segment based on the end vertebrae of each curvature segment.

[0055] In the example of this application, the deep learning model mentioned in step S2 is composed of the lightweight network MobileNet and the U-Net network for medical segmentation. Since MobileNet has a small number of parameters and the feature extraction effect is close to that of the same type of network, it is used as the backbone of the deep learning model in this solution, that is, the feature extraction network. The U-Net mainly consists of a contracting path and an expanding path. The contracting path will continuously reduce the resolution of the feature layer during the process of extracting information, while the expanding path can retain the high-level abstract semantic information to the high-resolution feature layer, that is, the expanding path plays a more accurate positioning role.

[0056] In the example of this application, the training of the deep learning model mentioned in the above step S2 uses the method of transfer learning, which is mainly divided into three steps: First, train the adopted feature extraction network MobileNet until the model converges; second, train the entire deep learning model, and in this step, freeze and protect the weights of the previously trained feature extraction network, that is, only train the weights of the feature layers other than the feature extraction network; third, unfreeze the weights of the frozen feature extraction network and train the entire model. The spinal center points predicted by the deep learning model include the spinal center points of the cervical vertebrae, thoracic vertebrae, and lumbar vertebrae, and the positioning points include 2 groups of positioning points located at the first group of collarbones and hip bones respectively, and each group includes two positioning points located on both sides of the spinal line.

[0057] The deep learning model is trained using the method of transfer learning, including: First, train the feature extraction network until the model converges; then, freeze and protect the weights of the previously trained feature extraction network and train the entire deep learning model; finally, unfreeze the weights of the frozen feature extraction network and train the entire deep learning model.

[0058] In the example of this application, the annotation of the dataset used for training the deep learning model is divided into two parts: one is the spinal center points, including the cervical vertebrae, thoracic vertebrae, and lumbar vertebrae; the other is four positioning points, and the annotation positions are located at the first group of collarbones and hip bones respectively. The four positioning points play a role in filtering the cervical vertebra center points. Figure 2 The example shows the spinal center points and four positioning points predicted by the model.

[0059] In the example of this application, during the prediction process of the above deep learning model, due to the influence of image quality, there may be several noise points. There are mainly two types of noise points: one type is the points that are similar to the characteristics of the spine center point to be predicted for the model, usually far from the spine; the other type is that two points are predicted near the spine center. For the first type of noise points, the filtering method is to calculate the absolute value of the difference between the abscissa of each point and the average value of the abscissas of all predicted points. If it exceeds the given threshold, it represents a noise point and needs to be filtered. For the second type of noise points, they can be filtered by taking the maximum value of the prediction scores of the pixel points within a certain pixel range of each predicted point. Figure 3 The example shows 17 central points of thoracic and lumbar vertebrae obtained after filtering.

[0060] In step S4, a 7th-degree polynomial is used to fit the 17 predicted central points of the spine. Figure 4 The example shows the result of fitting the spine curve of the spine X-ray in the final user graphical interface of this application example.

[0061] Step S5 is specifically: the spine curve is divided into several bending segments according to the inflection points obtained by taking the second derivative of the fitted spine curve, and the position of each inflection point is determined as the end vertebra.

[0062] Step S6 is specifically: for each bending segment, obtain the type of the vertebra in this bending segment that is farthest from the mid-perpendicular line of the hip bone, and determine the scoliosis classification, that is, determine whether this bend is an upper thoracic bend, a thoracic bend, a thoracolumbar bend or a lumbar bend according to the type of the vertebra in each bending segment that is farthest from the mid-perpendicular line of the sacrum.

[0063] Since there are interaction forces between the vertebrae, the normal line at the end vertebra of the spine fitting curve is not parallel to the end face of the vertebral body, that is, there is an error between the angle between the normal lines at the two end vertebrae calculated directly and the true Cobb angle. Therefore, step S7 is specifically:

[0064] For each bending segment, obtain the upper end vertebra and the lower end vertebra corresponding to this bending segment;

[0065] Based on the fitted spine curve, determine the normal lines of the intervertebral discs at the positions of the upper end vertebra and the lower end vertebra respectively;

[0066] Correct the two normal lines;

[0067] Obtain the included angle between the two normal lines as the Cobb angle of the current bending segment.

[0068] The method of normal line correction is:

[0069] Calculate the normal line correction angle:

[0070]

[0071] Among them, i represents the spine center point index, di is the normal slope at the i-th intervertebral disc on the fitted spinal curve, α is the correction factor, Δ i is the normal correction angle at the center point of the i-th vertebra.

[0072] In the above correction process, the angle between the normal and the positive direction of the X-axis is corrected. When Δ i is positive, the angle between the normal and the positive direction of the X-axis minus Δ i is the angle between the corrected normal and the positive direction of the X-axis. When Δ i is a negative angle, the angle between the normal and the positive direction of the X-axis plus |Δ i | is the angle between the corrected normal and the positive direction of the X-axis.

[0073] This embodiment also provides a spinal scoliosis detection system based on deep learning. This system implements the above-mentioned spinal scoliosis detection method based on deep learning and finally presents it in a user graphical interface. By opening the X-ray image to be detected in this interface program, spinal scoliosis detection can be automatically performed, and the scoliosis type and cobb angle can be given. For the usage effect reference of the interface program, see Figure 5 , that is, there are 3 types of spinal scoliosis in this embodiment, namely upper thoracic curve, thoracic curve, and lumbar curve (main lumbar curve), and the cobb angles of the curves are 27.87°, 24.67°, and 10.2° in sequence.

[0074] The above embodiments are only examples and do not represent limitations on the scope of the present invention. These embodiments can also be implemented in various other ways and can be subject to various omissions, substitutions, and changes without departing from the technical idea of the present invention.

Claims

1. A method for detecting scoliosis based on deep learning, characterized in that, The method includes: S1. Obtain a spinal X-ray image; S2. Input the spinal X-ray image into a deep learning model to predict the spinal center point and the positioning points for positioning; S3. Filter the predicted spinal center point; S4. Fit the spinal curve based on the filtered spinal center point; S5. Determine the spinal curvature segments according to the fitted spinal curve, and determine the end vertebrae of each curvature segment; S6. Classify the scoliosis of each curvature segment; S7. Calculate the Cobb angle of each curvature segment based on the end vertebrae of each curvature segment, including: For each curvature segment, obtain the upper end vertebra and the lower end vertebra corresponding to the curvature segment; Based on the fitted spinal curve, determine the normal vectors of the intervertebral discs at the positions of the upper end vertebra and the lower end vertebra respectively; Correct the two normal vectors, wherein the way of normal vector correction is: calculate the normal vector correction angle: where i represents the index of the center point of the spine, and d i is the normal slope at the i-th intervertebral disc on the fitted spinal curve, α is the correction factor, and Δ i is the normal correction angle at the i-th spine center point; Obtain the included angle between the two normal vectors as the Cobb angle of the current curvature segment.

2. The method for detecting scoliosis based on deep learning according to claim 1, wherein, The deep learning model includes a feature extraction network for spinal X-ray image feature extraction and a segmentation network for medical segmentation.

3. A method for detecting scoliosis based on deep learning according to claim 1, characterized in that, The spinal center points predicted by the deep learning model include the spinal center points of the cervical vertebrae, thoracic vertebrae, and lumbar vertebrae. The positioning points include 2 groups of positioning points located at the first group of collarbones and hip bones respectively, and each group includes two positioning points on both sides of the spinal line.

4. The method for detecting scoliosis based on deep learning according to claim 2, wherein The deep learning model is trained by using the method of transfer learning, including: first training the feature extraction network until the model converges; then, freezing and protecting the weights of the previously trained feature extraction network, and training the entire deep learning model; finally, unfreezing the weights of the frozen feature extraction network and training the entire deep learning model.

5. A method for detecting scoliosis based on deep learning according to claim 2, wherein The feature extraction network includes a MobileNet network, and the segmentation network includes a U-Net network.

6. The method for detecting scoliosis based on deep learning according to claim 3, characterized in that, Step S3 includes: Filter the predicted cervical vertebra center point according to the four predicted positioning points; For the noise points with a distance close to the accurately predicted center point, filter the noise points by the predicted scores of the pixel points within the preset pixel range of each predicted point; For the noise points with a large distance, calculate the absolute value of the difference between the abscissa of each point and the average value of the abscissas of all predicted points. If it exceeds the given threshold, it is considered a noise point and the noise point is filtered.

7. A spinal curvature detection method based on deep learning according to claim 1, characterized in that, Step S5 is specifically: Divide the spinal curve into several curvature segments according to the inflection points obtained by taking the second derivative of the fitted spinal curve, and determine the end vertebrae at each inflection point position.

8. The method for detecting scoliosis based on deep learning according to claim 3, wherein Step S6 is specifically: For each curvature segment, obtain the type of the vertebra that deviates the farthest from the mid-perpendicular line of the hip bone in the curvature segment, and determine the scoliosis classification.

Images