X-ray film-based evaluation system for assisting diagnosis of juvenile idiopathic scoliosis

Abstract

The application discloses a kind of based on X-ray film's adolescent idiopathic scoliosis auxiliary diagnosis evaluation system, it is related to medical instrument technical field. Including: image preprocessing module, key point detection module, Cobb angle measurement module and Lenke classification module.The application of based on X-ray film's adolescent idiopathic scoliosis auxiliary diagnosis evaluation system, realizes the key point detection of standing position, Bending position and lateral position spine X-ray film, automatic Cobb angle measurement and Lenke classification, including bending type, lumbar revision type and thoracic vertebra sagittal plane kyphosis classification, significantly improve the accuracy, efficiency and reliability of auxiliary diagnosis, provide comprehensive support for the clinical diagnosis and treatment decision of scoliosis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical devices, and particularly relates to an adolescent idiopathic scoliosis auxiliary diagnosis and evaluation system based on X-ray films. BACKGROUND

[0002] Scoliosis is a common three-dimensional deformity of the spine, which refers to the curvature of the spine along the longitudinal axis to one side, and is mainly characterized by lateral deviation of the spine and loss of normal sagittal curvature. The gold standard for its diagnosis is Cobb angle > 10°. Under normal circumstances, the human spine is a straight line, but when there is scoliosis, the spine will present a "C" or "S" shape in the plane. This disease usually develops in children and adolescents, among which adolescent idiopathic scoliosis (AIS) is the most common, accounting for more than 80% of scoliosis. AIS has a hidden onset, and if not discovered and treated in time, it may cause body asymmetry, abnormal posture, back pain and other problems, and even affect the heart and lung function in more serious cases. Therefore, early detection, early diagnosis and early intervention can effectively prevent the progression of scoliosis, which is crucial for ensuring physical health.

[0003] The existing diagnosis of scoliosis mainly includes physical examination, X-ray examination and other imaging examinations. Among them, X-ray examination is the most commonly used imaging examination method for evaluating scoliosis in clinical medicine. By taking X-ray films of the spine in the frontal position and left and right bending positions, the Cobb angle can be measured, that is, the included angle between the initial vertebra and the terminal vertebra. Cobb angle was first proposed by John Robert Cobb, an American doctor, in 1948, and is used to evaluate the severity of scoliosis. Cobb angle is an important indicator for judging the degree of scoliosis and is the basis for Lenke classification.

[0004] The existing Cobb angle measurement techniques mainly include: 1) manual measurement: the traditional Cobb angle measurement method is for a doctor to manually select the most severely inclined vertebrae on the concave side of the scoliosis in the spine as the upper and lower end vertebrae, and then use a protractor to measure the included angle between the upper and lower end vertebrae. 2) semi-automatic measurement of Cobb angle: Zhang based on neural network technology proposed a semi-automatic computer-aided measurement algorithm for Cobb angle, which automatically estimates the corresponding slope of the vertebrae after manually selecting the region of interest block of the upper and lower end vertebrae, so as to obtain the Cobb angle. 3) fully automatic measurement of Cobb angle: Tuo proposed a Cobb angle automatic measurement algorithm based on traditional machine learning for X-ray films. This method uses a segmentation algorithm to segment the vertebrae and extract the center points of each vertebra, and then uses a six-degree polynomial to fit the center point set to obtain the spine curve, and then automatically calculates the Cobb angle using the six-degree polynomial.

[0005] Although the existing Cobb angle measurement techniques assist in the diagnosis of scoliosis to some extent, they still have some significant shortcomings. Manual measurement of the Cobb angle relies on the manual selection of the starting and ending vertebrae by the doctor, is easily affected by personal experience and drawing habits, and has a high degree of subjectivity. Semi-automatic measurement of the Cobb angle still requires the doctor to participate in the key steps and verify and adjust the results, which cannot completely eliminate the subjective influence. Fully automatic measurement of the Cobb angle is usually based on image processing, machine learning and deep learning technologies, and can automatically identify, segment and measure the spinal curvature in the image. However, this method also has some shortcomings and challenges: 1) segmentation accuracy: the segmentation of the spinal skeleton is a key step in the fully automatic measurement method. However, due to the morphological and density changes of the spinal skeleton in the image, noise interference, and mutual occlusion of soft tissue and bone, the accuracy of segmentation may be affected. Especially in complex cases or low-contrast images, the segmentation algorithm may have difficulty in accurately identifying and separating the spinal skeleton, resulting in inaccuracy of the measurement results. 2) Inaccurate key point detection: due to the image quality, contrast and complexity of the anatomical structure, the existing technology is not accurate in automatically detecting the key points of the vertebrae. 3) Poor adaptability to different posture X-ray films: most existing technologies are only applicable to standing X-ray films, and have limited analysis capability for Bending and lateral X-ray films, which limits the application in different clinical scenarios. 4) Lack of automated Lenke classification: doctors need to manually classify according to the measurement results, which increases the complexity and time cost of the diagnosis process. SUMMARY

[0006] To solve the above problems in the prior art, the present application provides an X-ray film-based adolescent idiopathic scoliosis auxiliary diagnosis and evaluation system. The technical problem to be solved by the present application is solved by the following technical scheme:

[0007] The present application provides an X-ray film-based adolescent idiopathic scoliosis auxiliary diagnosis and evaluation system, comprising:

[0008] An image preprocessing module for ROI region detection of the detected spinal X-ray film to obtain a cropped image containing the spine, and for size adjustment and normalization processing of the cropped image to obtain a preprocessed image;

[0009] A key point detection module for detecting the preprocessed image using a pre-set trained key point detection model to obtain the coordinates of the key points of each vertebra in the preprocessed image;

[0010] A Cobb angle measurement module for calculating the standing Cobb angle, Bending Cobb angle and lateral angle according to the coordinates of the key points of each vertebra in the preprocessed image;

[0011] A Lenke classification module is configured to classify the spine according to the X-ray film to be detected, the standing Cobb angle, the bending Cobb angle and the lateral angle, and obtain a Lenke classification result.

[0012] Compared with the prior art, the present application has the following beneficial effects:

[0013] 1. The X-ray film-based adolescent idiopathic scoliosis auxiliary diagnosis and evaluation system of the present application realizes key point detection of standing, bending and lateral spine X-ray films, automatic Cobb angle measurement and Lenke classification, including bending type, lumbar revision type and thoracic sagittal plane kyphosis type, significantly improving the accuracy, efficiency and reliability of auxiliary diagnosis, and providing comprehensive support for clinical diagnosis and treatment decision of scoliosis.

[0014] 2. The X-ray film-based adolescent idiopathic scoliosis auxiliary diagnosis and evaluation system of the present application can accurately identify the key points of the vertebrae in various body positions including standing, left and right bending and lateral positions through deep learning and key point detection technology, thereby calculating more accurate Cobb angles and significantly improving the accuracy of Cobb angle measurement results.

[0015] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the content of the specification can be implemented, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a structural block diagram of an X-ray film-based adolescent idiopathic scoliosis auxiliary diagnosis and evaluation system provided by an embodiment of the present application;

[0017] Figure 2 is a processing flowchart of an image preprocessing module provided by an embodiment of the present application;

[0018] Figure 3 is a processing flowchart of a key point detection module provided by an embodiment of the present application;

[0019] Figure 4 is a structural schematic diagram of a key point detection model provided by an embodiment of the present application;

[0020] Figure 5 is a calculation flowchart of a standing Cobb angle provided by an embodiment of the present application;

[0021] Figure 6is a Bending Cobb angle calculation flowchart provided by an embodiment of the present application;

[0022] Figure 7 is a bending type classification flowchart provided by an embodiment of the present application;

[0023] Figure 8 is a lumbar correction type classification flowchart provided by an embodiment of the present application;

[0024] Figure 9 is a CSVL and pedicle relationship example diagram provided by an embodiment of the present application;

[0025] Figure 10 is a lumbar correction type classification schematic diagram provided by an embodiment of the present application.

[0026] Figure 11 is a thoracic sagittal kyphosis classification flowchart provided by an embodiment of the present application;

[0027] Figure 12 is a structure block diagram of another juvenile idiopathic scoliosis auxiliary diagnosis and evaluation system based on X-ray films provided by an embodiment of the present application;

[0028] Figure 13 is a standing Cobb angle measurement interface diagram provided by an embodiment of the present application;

[0029] Figure 14 is a Bending Cobb angle measurement interface diagram provided by an embodiment of the present application;

[0030] Figure 15 is a lateral angle and Lenke classification interface diagram provided by an embodiment of the present application;

[0031] Figure 16 is an interactive flowchart of a visualization software provided by an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined purposes, the following will be described in detail in combination with the drawings and specific embodiments, and a juvenile idiopathic scoliosis auxiliary diagnosis and evaluation system based on X-ray films according to the present application will be described in detail.

[0033] The foregoing and other technical contents, features and effects of the present application can be clearly presented in the following detailed description of specific embodiments in combination with the drawings. Through the description of the specific embodiments, the technical means and effects taken by the present application to achieve the predetermined purposes can be more deeply and specifically understood. However, the attached drawings are provided for reference and illustration only, and are not used to limit the technical solutions of the present application.

[0034] The embodiment of the present application provides a kind of based on X-ray film's adolescent idiopathic scoliosis auxiliary diagnosis evaluation system, please see Figure 1 , Figure 1 It is the structural diagram of a kind of based on X-ray film's adolescent idiopathic scoliosis auxiliary diagnosis evaluation system provided in the embodiment of the present application.As Figure 1 Shown, the based on X-ray film's adolescent idiopathic scoliosis auxiliary diagnosis evaluation system of the embodiment, it includes: image preprocessing module, key point detection module, Cobb angle measurement module and Lenke classification module.The based on X-ray film's adolescent idiopathic scoliosis auxiliary diagnosis evaluation system of the embodiment of the present application, by detecting the key point of the vertebra of the to-be-detected spine X-ray film, realize the automatic measurement of coronal plane Cobb angle, calculate the kyphosis angle of corresponding side image simultaneously, and judge the structural nature of main bend and secondary bend according to the size of Cobb angle and kyphosis angle, to carry out Lenke classification to scoliosis.

[0035] Further, each module of the based on X-ray film's adolescent idiopathic scoliosis auxiliary diagnosis evaluation system of the embodiment is described in detail.

[0036] Among them, image preprocessing module is used to detect the ROI (Region of Interest, region of interest) region of the to-be-detected spine X-ray film, obtain the intercepted image containing the spine, adjust the size of the intercepted image and carry out normalization processing, obtain preprocessed image.

[0037] In the embodiment, the to-be-detected spine X-ray film includes: standing position spine X-ray film, left bending position spine X-ray film, right bending position spine X-ray film and side position spine X-ray film.

[0038] Please see Figure 2 , Figure 2 It is the processing flow chart of an image preprocessing module provided in the embodiment of the present application, as Figure 2 Shown, first select the to-be-detected spine X-ray film, carry out ROI region detection, intercept the intercepted image containing the spine, then adjust the intercepted image to the input size specified by the key point detection model in the subsequent key point detection module, for example 640x640, then carry out normalization processing, i.e. the pixel value of the intercepted image is scaled from [0, 255] to [0, 1], to keep the same statistical characteristics as the training data of the key point detection model, which helps the stable convergence of model and improves performance.

[0039] The image preprocessing module of the embodiment, the spine region in X-ray film is identified and cropped using ROI region detection technology, which can reduce interference information and improve the accuracy of subsequent processing.

[0040] The key point detection module is configured to detect the preprocessed image by using a preset trained key point detection model to obtain coordinates of key points of each vertebra in the preprocessed image.

[0041] In this embodiment, the key point detection model is a key point detection model based on heat map regression with a channel attention mechanism. The coordinates of the key points of each vertebra in the preprocessed image are obtained by the key point heat map technology. The key points include the upper left point, the upper right point, the lower left point and the lower right point of the vertebra.

[0042] Referring to Figure 4 , Figure 4 is a structural diagram of a key point detection model provided by an embodiment of the present application, as Figure 4 indicated, the key point detection model uses ResNet34 as the network backbone to extract high-level semantic features, and combines deep and shallow features through a skip connection to utilize high-level semantic information and low-level detailed information of the image. Four corner landmark points of each vertebra are located by constructing a heat map, a center offset and an angle offset, and finally the key points of all the vertebrae, i.e., 17x4 key points of the vertebrae T1-T12, L1-L5 from top to bottom, are detected. In this embodiment, an attention mechanism is added after the deep and shallow features are combined through the skip connection (the yellow part in Figure 4 indicates the feature map optimized after the channel attention mechanism is introduced, and a schematic diagram of the channel attention mechanism is shown in the lower part of Figure 4 ), the purpose is to make the model pay more attention to the spine region to be detected, and the improved model has better recognition ability for the vertebrae themselves and the relationship between the upper and lower vertebrae.

[0043] In this embodiment, the key point detection model is trained by using a doctor-labeled spine X-ray training set with 17x4 key points, wherein the spine X-ray training set includes: a standing position spine X-ray, a left bending position spine X-ray, a right bending position spine X-ray and a lateral position spine X-ray.

[0044] Referring to Figure 3 , Figure 3 is a processing flowchart of a key point detection module provided by an embodiment of the present application, as Figure 3 indicated, taking the standing position spine X-ray as an example, the preprocessed image of the standing position spine X-ray is input into the trained key point detection model for key point detection. First, 17 vertebra center points are detected based on the heat map regression method, i.e., the 17 points correspond to 17 vertebrae, completing the positioning of the vertebrae, and then the four key points around the vertebra center points are located by using the learned angle offset, the purpose of which is to make the detected key points of the vertebrae more accurate.

[0045] The key point detection module of the embodiment introduces a channel attention mechanism to improve the key point detection model, realizes key point detection of the spine X-ray film, and the introduced channel attention mechanism enhances the response of the model to image features, especially the key point detection performance in low-contrast or blurred images.

[0046] It can be understood that in addition to the key point detection technology mentioned in the embodiment, other automatic measurement methods can also be used, such as using a segmentation network in deep learning, such as U-Net or its variants, to segment the vertebral body edge to extract the accurate contour of the vertebra, thereby realizing subsequent automatic measurement of the Cobb angle. Or apply pattern recognition algorithms to identify and classify different scoliosis types to assist in subsequent Cobb angle measurement. Or use multi-task learning to simultaneously predict segmentation and key points of the vertebra, improving the efficiency and accuracy of the overall subsequent Cobb angle measurement process.

[0047] The Cobb angle measurement module is configured to calculate the standing Cobb angle, the Bending Cobb angle, and the lateral angle according to the coordinates of the key points of each vertebra in the preprocessed image.

[0048] In an optional embodiment, the Cobb angle measurement module includes a standing Cobb angle measurement unit, a Bending Cobb angle measurement unit, and a lateral angle measurement unit.

[0049] In the embodiment, the standing Cobb angle measurement unit is configured to determine a first tilt vector of each vertebra according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the standing spine X-ray film, calculate the included angle of the two vertebrae with the largest angle among all the vertebrae as the standing Cobb1 angle according to the first tilt vector, and calculate the standing Cobb2 angle and the standing Cobb3 angle according to the scoliosis type of the spine X-ray film to be detected and the vertebra position corresponding to the standing Cobb1 angle.

[0050] Specifically, the coordinates of the key points of each vertebra in the preprocessed image corresponding to the standing spine X-ray film can be obtained by the key point detection module, i.e., 17x4 key point coordinates. The left and right midpoints of each vertebra can be found by the coordinates of the left upper point, the left lower point, the right upper point, and the right lower point of each vertebra. Then, the right midpoint coordinates are subtracted from the left midpoint coordinates to obtain a vector, i.e., the first tilt vector, and the direction of the vector represents the tilt direction of the vertebra. After obtaining 17 first tilt vectors corresponding to 17 vertebrae, a 17x17 matrix can be used to calculate the included angle between two vertebrae, find the two vertebrae with the largest angle and their included angle, i.e., the standing Cobb1 angle. The included angle between the two vertebrae is obtained by the dot product of the corresponding vectors of the two vertebrae.

[0051] In the embodiment, according to the type of scoliosis of the spine X-ray film to be detected and the vertebra position corresponding to the standing Cobb1 angle, the standing Cobb2 angle and the standing Cobb3 angle are calculated, which can include the following steps:

[0052] Step 1: According to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the standing spine X-ray film, the type of scoliosis of the spine X-ray film to be detected is determined.

[0053] The type of scoliosis includes C-type and S-type, which can be determined by vector calculation method to determine the left and right deviation of the spine and determine whether it can be offset to determine the type of scoliosis. The specific determination process is: according to the coordinates of the key points of the 17 vertebrae, the coordinates of the left and right midpoints of each vertebra are obtained. For each point, calculate the difference in the Y-axis direction, i.e. the vertical direction and the last point, and standardize it as the difference relative to the first point and the last point. For each point, calculate the difference in the X-axis direction, i.e. the horizontal direction and the last point, and standardize it as the difference relative to the first point and the last point. In this way, the standardized coordinate difference of each point in the X-axis and Y-axis is obtained. For each point, subtract the X-axis difference from the standardized Y-axis difference to finally obtain a list of difference values for all points. Let a be the sum of the difference matrix of all points, and b be the sum of its absolute value. If the difference between a and b is close to 0 (less than 1e-4), it is considered that the type of scoliosis is C-type, otherwise it is S-type.

[0054] That is, by comparing the horizontal and vertical differences of the cone midpoints, the shape of the spine is determined. If the curvature of the spine is relatively stable in a certain direction, i.e. C-type, the difference between a and b will be small, otherwise if the curvature is complex in two directions, i.e. S-type, the difference will be significantly larger.

[0055] Step 2: If the type of scoliosis is C-type, then according to the first inclined vector, the included angle between the upper vertebra of the standing Cobb1 angle and the vertebra T1 is calculated as the standing Cobb2 angle, and the included angle between the lower vertebra of the standing Cobb1 angle and the vertebra L5 is calculated as the standing Cobb3 angle.

[0056] Step 3: If the type of scoliosis is S-type, it is determined whether the standing Cobb1 angle is an upper thoracic curve.

[0057] In the embodiment, the bending type is determined according to the position of the top vertebra, wherein the top vertebra is at the vertebra T2-T5 for upper thoracic bending, the top vertebra is at the vertebra T5-T11 for main thoracic bending, the top vertebra is at the vertebra T12 for thoracolumbar bending, the top vertebra is at the vertebra L1-L4 for lumbar bending, and the top vertebra is the vertebra farthest from the central sacral vertical line in the scoliosis. Then, whether the standing Cobb1 angle is upper thoracic bending is determined after finding the top vertebra of the upper thoracic bending.

[0058] Specifically, the search for the top vertebra is performed according to the following steps: first, the coordinates of the left and right midpoints of the 17 vertebrae are fitted into a quintic polynomial curve, representing the overall shape of the spine. Then, the derivative of the fitted polynomial is obtained by derivation, and the zero point of the derivative is the extreme point of the curve. Secondly, the extreme points of the spine curve are obtained by the zero point of the derivative, indicating that the shape of the spine changes at these points. Finally, the range of the extreme points is limited to ensure that the analysis focuses on the middle segment of the spine and excludes the interference of the boundary region.

[0059] Step 4: If the standing Cobb1 angle is upper thoracic bending, then according to the first inclination vector, the maximum angle between the lower vertebra of the standing Cobb1 angle and all the vertebrae below the lower vertebra of the standing Cobb1 angle is calculated as the standing Cobb2 angle, and the maximum angle between the lower vertebra of the standing Cobb2 angle and all the vertebrae below the lower vertebra of the standing Cobb2 angle is calculated as the standing Cobb3 angle.

[0060] Step 5: If the standing Cobb1 angle is not upper thoracic bending, then according to the first inclination vector, the maximum angle between the upper vertebra of the standing Cobb1 angle and all the vertebrae above the upper vertebra of the standing Cobb1 angle is calculated as the standing Cobb2 angle, and the maximum angle between the lower vertebra of the standing Cobb1 angle and all the vertebrae below the lower vertebra of the standing Cobb1 angle is calculated as the standing Cobb3 angle.

[0061] Exemplarily, the calculation process of the standing Cobb angle is as shown in Figure 5 , Figure 5 is a calculation process diagram of the standing Cobb angle provided by the embodiment of the present application.

[0062] In the embodiment, the Bending position Cobb angle measurement unit is used to calculate the Bending position Cobb1 angle, the Bending position Cobb2 angle and the Bending position Cobb3 angle according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the left bending position X-ray film, the coordinates of the key points of each vertebra in the preprocessed image corresponding to the right bending position X-ray film, and the positions of the vertebrae corresponding to the standing position Cobb1 angle, the standing position Cobb2 angle and the standing position Cobb3 angle.

[0063] Specifically, the following steps can be included:

[0064] Step 1: determining the third inclination vector of each vertebra according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the left bending position X-ray film, and calculating the left bending position Cobb1 angle, the left bending position Cobb2 angle and the left bending position Cobb3 angle by using the third inclination vector according to the positions of the vertebrae corresponding to the standing position Cobb1 angle, the standing position Cobb2 angle and the standing position Cobb3 angle.

[0065] Step 2: determining the fourth inclination vector of each vertebra according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the right bending position X-ray film, and calculating the right bending position Cobb1 angle, the right bending position Cobb2 angle and the right bending position Cobb3 angle by using the fourth inclination vector according to the positions of the vertebrae corresponding to the standing position Cobb1 angle, the standing position Cobb2 angle and the standing position Cobb3 angle.

[0066] Step 3: selecting the smaller value of the left bending position Cobb1 angle and the right bending position Cobb1 angle as the Bending position Cobb1 angle, selecting the smaller value of the left bending position Cobb2 angle and the right bending position Cobb2 angle as the Bending position Cobb2 angle, and selecting the smaller value of the left bending position Cobb3 angle and the right bending position Cobb3 angle as the Bending position Cobb3 angle.

[0067] In the embodiment, the calculation process of the third inclination vector and the fourth inclination vector is similar to the calculation process of the first inclination vector, which is not described here. Exemplarily, taking the calculation of the Cobb angle of the left bending position as an example, the calculation process is as shown in Figure 6 Figure 6 is a calculation flowchart of the Bending position Cobb angle provided by the embodiment of the present application.

[0068] ​In the embodiment, the lateral angle measuring unit is configured to determine a second inclination vector of each vertebra according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the lateral spine X-ray film, and calculate the angle between the vertebra T2 and the vertebra T5 as the first lateral angle, the angle between the vertebra T10 and the vertebra L2 as the second lateral angle, and the angle between the vertebra T5 and the vertebra T12 as the third lateral angle according to the second inclination vector.

[0069] The Lenke classification module is configured to perform Lenke classification according to the spine X-ray film to be detected, the standing Cobb angle, the bending Cobb angle, and the lateral angle, and obtain a Lenke classification result.

[0070] In an optional embodiment, the Lenke classification module includes a first classification unit, a second classification unit, and a third classification unit. The first classification unit is configured to implement classification of the curve type, the second classification unit is configured to implement classification of the lumbar revision type, and the third classification unit is configured to implement classification of the thoracic sagittal kyphosis.

[0071] For example, a classification flowchart of the curve type is shown in Figure 7 , Figure 7 which is a classification flowchart of the curve type provided by the embodiment of the present application. In the embodiment, the first classification unit is configured to determine whether the Cobb angle of the standing main thoracic curve is the largest according to the standing Cobb angle. If yes, the first classification condition and the second classification condition are sequentially determined for the bending Cobb angle and the lateral angle. If both the first classification condition and the second classification condition are satisfied, the curve type is determined to be Type 4. If the first classification condition is satisfied and the second classification condition is not satisfied, the curve type is determined to be Type 2. If the first classification condition is not satisfied and the second classification condition is satisfied, the curve type is determined to be Type 3. If both the first classification condition and the second classification condition are not satisfied, the curve type is determined to be Type 1. If no, the third classification condition is determined for the bending Cobb angle and the lateral angle. If the third classification condition is satisfied, the curve type is determined to be Type 6. If the third classification condition is not satisfied, the curve type is determined to be Type 5.

[0072] The first classification condition is that the Cobb angle of the upper thoracic curve in the bending position is not less than 25° or the first lateral angle is not less than 20°. The second classification condition is that the Cobb angle of the lumbar curve in the bending position is not less than 25° or the second lateral angle is not less than 20°. The third classification condition is that the Cobb angle of the main thoracic curve in the bending position is not less than 25° or the second lateral angle is not less than 20°.

[0073] For example, a classification flowchart of the lumbar revision type is shown in Figure 8 , Figure 8is a lumbar revision type classification flowchart provided by an embodiment of the present application. In this embodiment, the second classification unit is used to determine the central sacral vertical line according to the standing position spine X-ray film, and determine the two pedicles of the lumbar apex according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the standing position spine X-ray film. If the central sacral vertical line is between the two pedicles, the lumbar revision type is A type. If the central sacral vertical line is on any pedicle, the lumbar revision type is B type. If the central sacral vertical line is outside the two pedicles, the lumbar revision type is C type.

[0074] Specifically, the central sacral vertical line (CSVL) is a line extending vertically upward from the center of the sacrum, which is commonly used to assess the overall balance of the spine. When classifying the lumbar revision type, first, the two vertebral body angles of the first sacral upper edge are located according to the standing position spine X-ray film, and then the average of the horizontal coordinates of the two vertebral body angles is taken as the S1, i.e. the midpoint of the first sacral upper edge. The vertical line passing through this point and parallel to the edge of the X-ray film is the central sacral vertical line. This median line is not a straight line perpendicular to the upper edge of S1, but a line perpendicular to the lower edge of the X-ray film, that is, a straight line of X = S1 midpoint horizontal coordinate. Then, according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the standing position spine X-ray film, the coordinates of the left and right midpoints of the 17 vertebrae are obtained. The curve of the 17 vertebrae midpoint coordinates is fitted by a fifth-order function, and the solution of the first derivative of the curve equal to zero in [13-17] is the lumbar apex. After determining the apex, the coordinates of the four key points of the vertebra can be used to intercept the cone image, which is sent to the image segmentation network U-Net for pedicle detection. U-Net is an "encoder-decoder" structure that down-samples the input image, extracts deep features through convolution and pooling operations, gradually up-samples, restores the spatial resolution of the image, and combines the skip connection to transfer the feature map information in the encoder to the decoder to help more accurate boundary positioning. The final output is a prediction mask with the same size as the input image, where the value of each pixel represents whether the pixel belongs to the pedicle. Finally, the classification of the lumbar revision type is performed according to the positional relationship between the central sacral vertical line and the pair of pedicles of the detected lumbar apex.

[0075] Please refer to Figure 9 , Figure 9 is a relationship example diagram between the central sacral vertical line (CSVL) and the pedicle provided by an embodiment of the present application. As Figure 9As shown, a pair of pedicles A and B of the top vertebra of the lumbar bend are detected, wherein the range of the pedicle A on the X axis is (X1, X2), the range of the pedicle B on the X axis is (X3, X4), and the CSLV is represented as X0 on the X axis. Then the relationship between the CSLV and the pedicle includes: the CSLV is on either pedicle, that is, X1≤X0≤X2 or X3≤X0≤X4; the CSLV is between the two pedicles, that is, X2<X0<X3; and the CSLV is outside the pedicle, that is, X0<X1 or X0>X4. According to the relationship between the CSLV and the pedicle, the corresponding lumbar revision type can be determined, such as Figure 10 As shown, Figure 10 is a lumbar revision type classification schematic diagram provided by an embodiment of the present application.

[0076] Exemplarily, a thoracic kyphosis classification flowchart is as shown in Figure 11 As shown, Figure 11 is a thoracic kyphosis classification flowchart provided by an embodiment of the present application. In the embodiment, the third classification unit is used to determine whether the third lateral angle is less than 10° according to the lateral angle, if yes, the thoracic kyphosis is classified as a negative type, if not, it is determined whether the third lateral angle is less than 40°, if yes, the thoracic kyphosis is classified as a normal type, if not, the thoracic kyphosis is classified as a positive type.

[0077] The X-ray-based adolescent idiopathic scoliosis auxiliary diagnosis evaluation system provided by the embodiment of the present application realizes key point detection of standing position, Bending position and lateral position X-ray, automatic Cobb angle measurement and Lenke classification, including bending type, lumbar revision type and thoracic kyphosis classification, and significantly improves the accuracy, efficiency and reliability of auxiliary diagnosis, and provides comprehensive support for clinical diagnosis and treatment decision of scoliosis. Through deep learning and key point detection technology, the key points of vertebrae in various body positions including standing position, left and right bending position and lateral position can be accurately recognized, so that more accurate Cobb angle can be calculated, and the accuracy of Cobb angle measurement result is significantly improved.

[0078] Further, please refer to Figure 12 , Figure 12 is a structure block diagram of another X-ray-based adolescent idiopathic scoliosis auxiliary diagnosis evaluation system provided by an embodiment of the present application, as shown in Figure 12 The X-ray-based adolescent idiopathic scoliosis auxiliary diagnosis evaluation system provided by the embodiment of the present application further includes a visualization module, and the visualization module is used to display standing position Cobb angle, Bending position Cobb angle, lateral angle and Lenke classification result.

[0079] In the present embodiment, the Cobb angle in standing position, the Cobb angle in bending position, the lateral angle and the Lenke classification result can be visualized by using the visualization software, please refer to Figures 13-15 , Figure 13 is a measurement interface diagram of the Cobb angle in standing position provided by the present embodiment; Figure 14 is a measurement interface diagram of the Cobb angle in bending position provided by the present embodiment; Figure 15 is an interface diagram of the lateral angle and the Lenke classification provided by the present embodiment. In the visualization processing, different colors can be used to distinguish the Cobb angle according to the size and the vertebrae where the Cobb angle is located. In this way, the doctor can directly see which two vertebrae the Cobb angle corresponds to. At the same time, the calculated Cobb angles of the upper thoracic curve, the main thoracic curve and the thoracolumbar curve / lumbar curve are output in the software interface, so that the doctor can clearly understand the numerical values of these angles. In addition, in order to better guide the clinical judgment, the grade division of the Cobb angle is also given to help the doctor evaluate the severity of the scoliosis, so as to select the corresponding treatment plan. Through the above visualization processing, the angle and the Lenke classification of the scoliosis can be directly understood, which further provides support and reference for the diagnosis and treatment of the doctor.

[0080] Please refer to Figure 16 , Figure 16 is an interactive flowchart of the visualization software provided by the present embodiment, as shown in Figure 16 , the Cobb angle in standing position is input in the measurement interface of the Cobb angle in standing position, and the size of the Cobb angle can be viewed in the result box by clicking the detection. If it is necessary to further measure the Cobb angle or the lateral angle of the Cobb angle in bending position, the Cobb angle in bending position or the lateral Cobb angle can be obtained by clicking the bending position detection or the lateral detection button, inputting the left and right Cobb angles in bending position or the lateral Cobb angle in the measurement interface, and clicking the detection. Finally, the Lenke classification is clicked, and the classification result is obtained by automatically combining the Cobb angles in standing position and bending position and the three angles in lateral position. Through the software, the doctor can more conveniently measure the Cobb angle of the scoliosis, simplify the work process, and improve the work efficiency and accuracy. At the same time, the doctor can quickly perform the Lenke classification. The software can also be functionally extended and optimized according to actual needs to meet the different needs of the doctor.

[0081] It is to be understood that the terminology used herein such as first and second, and the like, is only to distinguish one from another without prejudice to either and is not necessarily used in a sequence. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. The terms "connected" and "coupled" as well as "linking" or variations thereof are not restricted to a physical or mechanical connections or relationships. Rather, any of these terms can mean any connection or relationship between components, systems, or modules, which is capable of being either a direct connection, an indirect connection through one or more other components or modules, or an indirect connection through some type of medium.

[0082] The above description is further to the application in conjunction with specific preferred embodiments and should not be construed as limiting the application to these descriptions. Those skilled in the art who are familiar with the technical field of the application can make some simple deductions or substitutions without departing from the concept of the application, and all of these should be considered within the protection scope of the application.

Claims

1. An X-ray-based auxiliary diagnosis and evaluation system for adolescent idiopathic scoliosis, characterized by: include: An image preprocessing module is configured to perform ROI detection on the spinal X-ray film to be inspected, obtain a cut-off image containing the spine, and perform size adjustment and normalization processing on the cut-off image to obtain a preprocessed image; the spinal X-ray film to be inspected includes: a standing spinal X-ray film, a left bending spinal X-ray film, a right bending spinal X-ray film, and a lateral spinal X-ray film; A key point detection module, configured to detect the pre-processed image using a preset trained key point detection model to obtain the coordinates of the key points of each vertebra in the pre-processed image; A Cobb angle measurement module is configured to calculate the standing Cobb angle, the bending Cobb angle, and the lateral Cobb angle based on the coordinates of the key points of each vertebra in the preprocessed image; the Cobb angle measurement module includes: A standing Cobb angle measurement unit is configured to determine a first tilt vector for each vertebra based on the coordinates of a key point of each vertebra in a preprocessed image corresponding to the standing spinal X-ray film, calculate the angle between the two vertebrae with the largest angle among all vertebrae based on the first tilt vector as the standing Cobb1 angle, and calculate the standing Cobb2 angle and the standing Cobb3 angle based on the type of scoliosis in the spinal X-ray film to be detected and the position of the vertebra corresponding to the standing Cobb1 angle; wherein the calculation of the standing Cobb2 angle and the standing Cobb3 angle based on the type of scoliosis in the spinal X-ray film to be detected and the position of the vertebra corresponding to the standing Cobb1 angle includes: Determining the type of scoliosis in the spinal X-ray to be detected based on the coordinates of the key points of each vertebra in the preprocessed image corresponding to the spinal X-ray in the standing position; If the scoliosis type is type C, then based on the first tilt vector, the angle between the upper vertebra and vertebra T1 obtained from the standing Cobb1 angle is calculated as the standing Cobb2 angle, and the angle between the lower vertebra and vertebra L5 obtained from the standing Cobb1 angle is calculated as the standing Cobb3 angle; If the scoliosis type is S-type, determining whether the standing Cobb1 angle is an upper thoracic curve; If the standing Cobb1 angle is an upper thoracic curve, then based on the first tilt vector, the maximum angle between the lower vertebra of the standing Cobb1 angle and all the vertebrae below the lower vertebra of the standing Cobb1 angle is calculated as the standing Cobb2 angle, and the maximum angle between the lower vertebra of the standing Cobb2 angle and all the vertebrae below the lower vertebra of the standing Cobb2 angle is calculated as the standing Cobb3 angle; If the standing Cobb1 angle is not an upper thoracic curve, then based on the first tilt vector, the maximum angle between the upper vertebra of the standing Cobb1 angle and all vertebrae above the upper vertebra of the standing Cobb1 angle is calculated as the standing Cobb2 angle, and the maximum angle between the lower vertebra of the standing Cobb1 angle and all vertebrae below the lower vertebra of the standing Cobb1 angle is calculated as the standing Cobb3 angle; A bending position Cobb angle measurement unit is configured to calculate the bending position Cobb1 angle, the bending position Cobb2 angle, and the bending position Cobb3 angle based on the coordinates of the key points of each vertebra in the preprocessed image corresponding to the left bending position spinal X-ray film, the coordinates of the key points of each vertebra in the preprocessed image corresponding to the right bending position spinal X-ray film, and the vertebral positions corresponding to the standing position Cobb1 angle, the standing position Cobb2 angle, and the standing position Cobb3 angle; comprising: Determine a third tilt vector for each vertebra based on the coordinates of a key point of each vertebra in a preprocessed image corresponding to the left bending spinal X-ray film, and calculate the left bending Cobb1 angle, the left bending Cobb2 angle, and the left bending Cobb3 angle based on the vertebral positions corresponding to the standing Cobb1 angle, the standing Cobb2 angle, and the standing Cobb3 angle using the third tilt vector; Determine a fourth tilt vector for each vertebra based on the coordinates of a key point of each vertebra in a preprocessed image corresponding to the right bending spinal X-ray film, and calculate the right bending Cobb1 angle, the right bending Cobb2 angle, and the right bending Cobb3 angle based on the vertebral positions corresponding to the standing Cobb1 angle, the standing Cobb2 angle, and the standing Cobb3 angle using the fourth tilt vector; The smaller value between the left bending position Cobb1 angle and the right bending position Cobb1 angle is selected as the bending position Cobb1 angle, the smaller value between the left bending position Cobb2 angle and the right bending position Cobb2 angle is selected as the bending position Cobb2 angle, and the smaller value between the left bending position Cobb3 angle and the right bending position Cobb3 angle is selected as the bending position Cobb3 angle; a lateral angle measurement unit, configured to determine a second inclination vector for each vertebra based on the coordinates of a key point of each vertebra in a preprocessed image corresponding to the lateral spinal X-ray film, and to calculate, based on the second inclination vector, an angle between vertebrae T2 and T5 as a first lateral angle, an angle between vertebrae T10 and L2 as a second lateral angle, and an angle between vertebrae T5 and T12 as a third lateral angle; The Lenke classification module is used to perform Lenke classification based on the spinal X-ray film to be tested, the standing Cobb angle, the bending Cobb angle and the lateral angle to obtain a Lenke classification result.

2. The X-ray-based auxiliary diagnosis and evaluation system for adolescent idiopathic scoliosis according to claim 1, characterized in that: The key point detection model is a key point detection model based on heat map regression that introduces a channel attention mechanism.

3. The X-ray-based auxiliary diagnosis and evaluation system for adolescent idiopathic scoliosis according to claim 1, characterized in that: The key points include the upper left point, the upper right point, the lower left point and the lower right point of the vertebra.

4. The X-ray-based auxiliary diagnosis and evaluation system for adolescent idiopathic scoliosis according to claim 1, characterized in that: The Lenke classification module includes: a first classification unit, configured to determine whether the Cobb angle of the main thoracic curve in the standing position is the largest based on the standing Cobb angle; if so, sequentially performing a first classification condition and a second classification condition on the Bending Cobb angle and the lateral angle; if both the first classification condition and the second classification condition are satisfied, determining the curvature type as Type 4; if the first classification condition is satisfied but the second classification condition is not satisfied, determining the curvature type as Type 2; if the first classification condition is not satisfied but the second classification condition is satisfied, determining the curvature type as Type 3; if neither the first classification condition nor the second classification condition is satisfied, determining the curvature type as Type 1; if not, performing a third classification condition on the Bending Cobb angle and the lateral angle; if the third classification condition is satisfied, determining the curvature type as Type 6; if the third classification condition is not satisfied, determining the curvature type as Type 5; a second classification unit, configured to determine a central sacral vertical line according to the standing spinal X-ray film, and determine two pedicles of the apical vertebra of the lumbar curve according to the coordinates of the key points of each vertebra in the preprocessed image corresponding to the standing spinal X-ray film, wherein if the central sacral vertical line is between the two pedicles, the lumbar vertebrae are classified as type A; if the central sacral vertical line is on any one of the pedicles, the lumbar vertebrae are classified as type B; and if the central sacral vertical line is outside the two pedicles, the lumbar vertebrae are classified as type C; The third classification unit is used to determine whether the third lateral angle is less than 10° based on the lateral angle. If so, the thoracic sagittal kyphosis classification is negative; if not, determine whether the third lateral angle is less than 40°. If so, the thoracic sagittal kyphosis classification is normal; otherwise, the thoracic sagittal kyphosis classification is positive.

5. The X-ray-based auxiliary diagnosis and evaluation system for adolescent idiopathic scoliosis according to claim 4, characterized in that: The first classification condition is: the Cobb angle of the thoracic curve in the bending position is not less than 25° or the first lateral angle is not less than 20°; The second classification condition is: the Cobb angle of the bending lumbar curve is not less than 25° or the second lateral angle is not less than 20°; The third classification condition is that the Cobb angle of the main thoracic curve in the bending position is not less than 25° or the second lateral angle is not less than 20°.

6. The X-ray-based auxiliary diagnosis and evaluation system for adolescent idiopathic scoliosis according to claim 1, characterized in that: The X-ray-based auxiliary diagnosis and evaluation system for adolescent idiopathic scoliosis also includes a visualization module for displaying the standing Cobb angle, the bending Cobb angle, the lateral angle and the Lenke classification result.

Images