Spinal full-length cobb angle detection device, method, apparatus, and storage medium

Abstract

The application discloses a spine full-length film Cobb angle detection device, method, equipment and storage medium, which comprises a training sample acquisition module, which is used for acquiring multiple different orientation spine full-length film images with labeled vertebra key points as training samples; a model construction module, which is used for constructing a spine detection model composed of a spine ROI positioning model and a vertebra key point model; a model training module, which is used for training the spine detection model using the training samples; a coordinate output module, which is used for inputting different orientation spine full-length film images to be detected into the trained model and outputting position coordinates of the vertebra key points in different orientations; and a Cobb angle calculation module, which is used for obtaining Cobb angle sizes of upper thoracic bends, thoracic bends and thoracolumbar / lumbar bends corresponding to different orientations according to the position coordinates. In this way, the algorithm process is realized to be interpretable, manual operation is saved, and the detection precision is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a device, method, equipment, and storage medium for detecting the Cobb angle of a full-length spinal radiograph. Background Technology

[0002] Spinal deformities are common spinal disorders, with idiopathic scoliosis (IS) being the most prevalent. The cause of idiopathic scoliosis is unknown. The Scoliosis Research Society defines scoliosis as a lateral curvature of the spine in the coronal plane exceeding 10 degrees as measured by the Cobb angle. The severity of scoliosis is mostly assessed by the angle of lateral curvature, with the Cobb angle measurement method being commonly used clinically.

[0003] Currently, such as Figure 1 As shown, radiologists typically measure the Cobb angle by manually selecting the vertebrae with the most severe tilt towards the concave side of the scoliosis as the upper and lower vertebrae, and then using a goniometer to measure the angle between them. The accuracy of Cobb angle measurement largely depends on the radiologist's subjective experience. Manually selecting the upper and lower vertebrae for Cobb angle measurement can introduce significant errors, greatly impacting the diagnosis and treatment of scoliosis patients.

[0004] Therefore, how to solve the problem of large errors in Cobb angle measurement caused by the subjective experience of radiologists is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a device, method, equipment, and storage medium for detecting the Cobb angle of a full-length spinal radiograph, which enables interpretability of the algorithm flow, eliminates manual operation, and improves detection accuracy. The specific solution is as follows:

[0006] A full-length spinal Cobb angle detection device, comprising:

[0007] The training sample acquisition module is used to acquire multiple full-length spinal images from different orientations that have been labeled with key points of the vertebral body as training samples.

[0008] The model building module is used to construct a spine detection model consisting of a spine ROI localization model and a vertebral body key point model. The spine ROI localization model takes a full-length spine image as input and the spine ROI image as output. The vertebral body key point model takes a spine ROI image as input and the position coordinates of the vertebral body key points as output.

[0009] a model training module, configured to train the spine detection model using the training samples until network convergence;

[0010] a coordinate output module, configured to input different orientation full-length spine images to be detected into the spine detection model, and output position coordinates of vertebral key points of the different orientation full-length spine images to be detected;

[0011] a Cobb angle calculation module, configured to obtain Cobb angle sizes of different orientations corresponding to upper thoracic curves, thoracic curves and thoracolumbar / lumbar curves according to the position coordinates of the vertebral key points of the different orientation full-length spine images to be detected.

[0012] Preferably, in the Cobb angle detection device for full-length spine images provided in the embodiment of the present application, the Cobb angle calculation module is configured to calculate Cobb angle sizes of the upper thoracic curves, the thoracic curves and the thoracolumbar / lumbar curves corresponding to the frontal position and the position of the end vertebrae according to the position coordinates of the vertebral key points of the full-length spine image to be detected in the frontal position; and calculate Cobb angle sizes of the upper thoracic curves, the thoracic curves and the thoracolumbar / lumbar curves corresponding to the left lateral flexion and the right lateral flexion according to the position coordinates of the vertebral key points of the full-length spine images to be detected in the left lateral flexion and the right lateral flexion and the position of the end vertebrae calculated.

[0013] Preferably, in the Cobb angle detection device for full-length spine images provided in the embodiment of the present application, the model training module is configured to input the training samples into the spine ROI positioning model; when training the spine ROI positioning model, generate a vertebral mask region of the spine according to the labeled vertebral key points, and calculate a maximum bounding rectangle of the mask region to obtain a spine ROI image; and input the obtained spine ROI image into the vertebral key point model for training to generate a heat map for representing position coordinates of the vertebral key points.

[0014] Preferably, in the Cobb angle detection device for full-length spine images provided in the embodiment of the present application, the spine ROI positioning model adopts a U-net network structure.

[0015] The backbone network of the U-net network structure is a MobileNet-v2; and the MobileNet-v2 includes a linear bottleneck layer and an inverse residual module.

[0016] The linear bottleneck layer is arranged at the output end of each convolution module and adopts a linear activation function.

[0017] The inverse residual module adopts a 1x1 convolution layer to increase the dimension of the feature map in the residual module.

[0018] Preferably, in the spinal full-length film Cobb angle detection device provided by the embodiment of the present application, the vertebral body key point model adopts a cascaded hourglass network.

[0019] The cascaded hourglass network takes a residual network as a basic module to perform down-sampling and up-sampling on a picture to extract multi-scale features.

[0020] Preferably, in the spinal full-length film Cobb angle detection device provided by the embodiment of the present application, the cascaded hourglass network is formed by cascading a plurality of hourglass networks; two outputs of a previous hourglass network and an original input picture are added to serve as an input of a next hourglass network.

[0021] The hourglass network is used to obtain feature maps of various scales to capture information contained in pictures of different scales.

[0022] Preferably, in the spinal full-length film Cobb angle detection device provided by the embodiment of the present application, the cascaded hourglass network is trained using relay supervision to integrate type and position information of vertebral body key points into an output feature map to obtain a heat map of each key point.

[0023] The embodiment of the present application also provides a detection method of the spinal full-length film Cobb angle detection device provided by the embodiment of the present application, which comprises the following steps:

[0024] Obtaining a plurality of spinal full-length film images of different orientations, which have been labeled with vertebral body key points, as training samples;

[0025] Constructing a spinal detection model composed of a spinal ROI positioning model and a vertebral body key point model; the spinal ROI positioning model takes a spinal full-length film image as an input and outputs a spinal ROI image; the vertebral body key point model takes a spinal ROI image as an input and outputs position coordinates of vertebral body key points;

[0026] Training the spinal detection model using the training samples until the network converges;

[0027] Inputting a to-be-detected spinal full-length film image of different orientations into the spinal detection model to output position coordinates of vertebral body key points of the to-be-detected spinal full-length film image of different orientations;

[0028] According to the position coordinates of the vertebral body key points of the to-be-detected spinal full-length film image of different orientations, Cobb angles of upper thoracic curves, thoracic curves and thoracolumbar / lumbar curves corresponding to different orientations are obtained.

[0029] The embodiment of the present application also provides an electronic device, comprising a processor and a memory, wherein the processor implements the detection method of the above-mentioned spine full-length film Cobb angle detection device provided by the embodiment of the present application when executing the computer program stored in the memory.

[0030] The embodiment of the present application also provides a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the detection method of the above-mentioned spine full-length film Cobb angle detection device provided by the embodiment of the present application.

[0031] From the above technical solution, it can be seen that the spine full-length film Cobb angle detection device provided by the present application comprises: a training sample acquisition module, configured to acquire a plurality of different orientation spine full-length film images labeled with vertebral key points as training samples; a model construction module, configured to construct a spine detection model composed of a spine ROI positioning model and a vertebral key point model; the spine ROI positioning model takes the spine full-length film image as input and takes the spine ROI image as output; the vertebral key point model takes the spine ROI image as input and takes the position coordinates of the vertebral key points as output; a model training module, configured to train the spine detection model using the training samples until the network converges; a coordinate output module, configured to input the different orientation to-be-detected spine full-length film images into the spine detection model and output the position coordinates of the vertebral key points of the different orientation to-be-detected spine full-length film images; and a Cobb angle calculation module, configured to obtain the Cobb angle sizes of the upper thoracic bend, the thoracic bend and the thoracolumbar / lumbar bend corresponding to different orientations according to the position coordinates of the vertebral key points of the different orientation to-be-detected spine full-length film images.

[0032] In the above-mentioned spine full-length film Cobb angle detection device provided by the present application, the spine detection model composed of the spine ROI positioning model and the vertebral key point model is constructed and trained through the interaction of the above-mentioned five modules, the spine ROI image is acquired by using the spine ROI positioning model, the position coordinates of the vertebral key points are predicted by using the vertebral key point model, then the Cobb angles of different regions corresponding to the different orientation to-be-detected spine full-length film images can be calculated by using the predicted position coordinates of the key points, and the algorithm flow is further realized to be interpretable, so that the detection method is simplified, the manual operation is saved, and the detection accuracy is improved.

[0033] In addition, the present application also provides a corresponding method, device and computer readable storage medium for the spine full-length film Cobb angle detection device, which further makes the above-mentioned device more practical, and the method, device and computer readable storage medium have corresponding advantages. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description only constitute a part of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0035] Figure 1 A schematic diagram of the existing Cobb angle measurement method;

[0036] Figure 2 A structural schematic diagram of the spinal full-length film Cobb angle detection device provided by the embodiment of the present application;

[0037] Figure 3 A schematic diagram of the spinal full-length film Cobb angle detection device provided by the embodiment of the present application in the detection process;

[0038] Figure 4 A structural schematic diagram of the spinal ROI positioning model provided by the embodiment of the present application;

[0039] Figure 5 A structural schematic diagram of the vertebral key point model provided by the embodiment of the present application;

[0040] Figure 6 A structural schematic diagram of the cascade hourglass network provided by the embodiment of the present application;

[0041] Figure 7 A structural schematic diagram of the hourglass network provided by the embodiment of the present application;

[0042] Figure 8 A flow chart of the detection method of the spinal full-length film Cobb angle detection device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0044] The present application provides a spinal full-length film Cobb angle detection device, as shown in Figure 2 , comprising:

[0045] The training sample acquisition module 21 is configured to acquire a plurality of spinal full-length film images of different orientations with labeled vertebral key points as training samples.

[0046] In practical applications, a professional orthopedic surgeon can be used to mark the key points of the vertebral body. The key points of the vertebral body include the four vertices of the vertebral body. Since there are a total of 17 vertebral bodies from top to bottom, 68 key points can be marked. The full-length spine images in different orientations include the full-length spine anteroposterior (AP) image, the full-length spine left bending (Lbending) image and the full-length spine right bending (Rbending) image. This step can obtain 200 data containing AP, Lbending and Rbending orientations.

[0047] The model construction module 22 is configured to construct a spine detection model composed of a spine ROI positioning model and a vertebral body key point model; the spine ROI positioning model takes a full-length spine image as input and outputs a spine ROI image; and the vertebral body key point model takes the spine ROI image as input and outputs the position coordinates of the vertebral body key points.

[0048] The model training module 23 is configured to train the spine detection model using training samples until the network converges.

[0049] The coordinate output module 24 is configured to input the full-length spine images to be detected in different orientations into the spine detection model and output the position coordinates of the vertebral body key points of the full-length spine images to be detected in different orientations.

[0050] The Cobb angle calculation module 25 is configured to obtain the Cobb angle sizes of the proximal thoracic (PT), main thoracic (MT) and thoracolumbar / lumbar (TL / L) bends corresponding to different orientations according to the position coordinates of the vertebral body key points of the full-length spine images to be detected in different orientations.

[0051] In the above-mentioned spine full-length film Cobb angle detection device provided by the embodiment of the present application, the spine detection model composed of the spine ROI positioning model and the vertebral body key point model can be constructed and trained through the interaction of the above-mentioned five modules. The spine ROI positioning model is used to obtain the spine ROI image, and the vertebral body key point model is used to predict the position coordinates of the vertebral body key points. Then, the position coordinates of the predicted key points can be used to calculate the Cobb angles of different regions corresponding to the full-length spine images to be detected in different orientations, and the algorithm flow is further made interpretable, the detection method is simplified, manual operation is saved, and the detection accuracy is improved.

[0052] In specific implementation, in the above-mentioned device for detecting the Cobb angle of the full-length spine sheet, the model training module 23 can be specifically configured to input the training sample into the spine ROI positioning model; when training the spine ROI positioning model, the vertebra mask region of the spine is generated according to the labeled vertebra key points, and the maximum bounding rectangle of the mask region is calculated to obtain the spine ROI image; and the obtained spine ROI image is input into the vertebra key point model for training to generate a heat map for representing the position coordinates of the vertebra key points.

[0053] Specifically, in the process of training the spine ROI positioning model, the vertebra mask region of the spine can be first generated according to the labeled vertebra key points; then the pixel mean and variance of the labeled data are calculated; the network input size of the spine ROI positioning model is 3*512*256 (h*w), after normalization, the mean and variance use the values calculated in the previous step, and the network output size is 2*512*256 (h*w). Preferably, the network uses the imagenet pre-training model of mobilenet-v2, the optimizer uses SGD, the learning rate lr=0.0003, the momentum when optimizing the gradient momentum=0.96, the weight decay weight_decay=0.00005, and the loss function uses Dice Loss, and the formula is as follows:

[0054]

[0055] wherein N represents the number of pixels, C represents the number of categories, g i represents the labeled data, s i represents the network prediction value.

[0056] In addition, in the process of training the vertebra key point model, 68 heat maps representing the positions of the key points can be first generated according to the labeled vertebra key points, and the coordinates (x k ,y k ) of the kth key point are encoded using the heat map m k , wherein k∈{1,…,K}, and here K=68.

[0057] The heat map m k is generated by taking the coordinates (x k ,y T ) of the key points as the mean of the Gaussian distribution, and the formula is as follows:

[0058]

[0059] where (x, y) represents the coordinates in the heat map space, and the hyperparameter s represents the standard deviation of the predefined Gaussian distribution. After defining the heat map of the key points, a specific spine key point estimation target loss function can be given, and the formula is as follows:

[0060]

[0061] Then, the pixel mean and variance of the labeled data are calculated. The network input size of the vertebral key point model is 3x512x256 (h*w), and after normalization, the mean and variance use the values calculated in the previous step. The network output size is 2x512x256 (h*w). The optimizer of the network uses Adam, the learning rate lr=0.000125, the momentum of gradient optimization momentum=0.96, the weight decay weight_decay=0.00001, and the loss function uses Mean-Squared Error (MSE). Data augmentation includes rotation (such as ±30°), scaling (such as 0.75-1.25).

[0062] Further, in the specific implementation, in the above-mentioned Cobb angle detection device for the full-length spine film provided by the embodiment of the present application, the Cobb angle calculation module 25 can be specifically used to calculate the Cobb angle size of the upper thoracic bend, thoracic bend, and thoracolumbar / lumbar bend corresponding to the ortho position and the end vertebra index corresponding to the Cobb angle according to the position coordinates of the vertebral key points of the ortho position image of the full-length spine to be measured; and calculate the Cobb angle size of the upper thoracic bend, thoracic bend, and thoracolumbar / lumbar bend corresponding to the left lateral flexion and right lateral flexion according to the position coordinates of the vertebral key points of the left lateral flexion image and the right lateral flexion image of the full-length spine to be measured, and the position of the calculated end vertebra index. In this way, the vertebral position of the full-length spine film ortho position is used for Cobb angle prediction in the left lateral flexion and right lateral flexion films, which can further simplify the measurement method and improve the measurement accuracy.

[0063] The functions of the coordinate output module 24 and the Cobb angle calculation module 25 in the above-mentioned Cobb angle detection device for the full-length spine film provided by the embodiment of the present application will be described below with a specific example:

[0064] Firstly, the three full-length spine film images in different directions can be input to the spine ROI positioning model through the coordinate output module 24 for spine positioning, and three spine ROI images are obtained respectively; as shown in the formula: Figure 3 As shown in the formula, the full-length spine ortho position film, full-length spine left lateral flexion film, and full-length spine right lateral flexion film of the same patient are input to the spine ROI positioning model for spine positioning, and three different direction vertebral ROI images are obtained respectively;

[0065] Then, the three different direction vertebral body ROI images can be input to the vertebral body key point model through the coordinate output module 24 for vertebral body key point detection, and the vertebral body key points are obtained respectively;

[0066] After that, for the front and rear images, the Cobb angle calculation module 25 uses the key point positions to obtain the Cobb angle sizes of PT, MT and TL / L and the key point coordinates and vertebral body indexes of the upper and lower end vertebrae corresponding to the Cobb angles according to the "Cobb angle calculation method";

[0067] Finally, for the vertebral body key points of the left and right bending positions, the Cobb angle calculation module 25 uses the obtained vertebral body index positions to calculate the Cobb angle sizes of PT, MT and TL / L of the left and right bending positions respectively.

[0068] Through the interaction of the coordinate output module 24 and the Cobb angle calculation module 25, the Cobb angle sizes of three different views can be finally obtained, and the Cobb angle can be measured more efficiently and quickly.

[0069] In the present application, the "Cobb angle calculation method" can specifically include the following steps:

[0070] Firstly, in the full-length anteroposterior film of the to-be-measured spine, the horizontal direction middle line segment of each vertebral body, i.e. the middle points on the left and right sides, are calculated as the starting coordinates, and the vector of the starting coordinates is recorded as vec_m, and the vector dimension is 17*2;

[0071] Secondly, the matrix dot_v is calculated, dot_v=vec_m*vec_m T , and the matrix size is 17*17; wherein, T represents the quasi-set;

[0072] Thirdly, the modulus of the vector vec_m is calculated and recorded as_mod_v, and the vector dimension is 17*1:

[0073]

[0074] Then, the modulus matrix of vec_m is calculated and recorded as mod_v, mod_v=mod_v*mod_v T , and the matrix size is 17*17;

[0075] Fourthly, the Cobb angle candidate area of each vertebral body and other vertebral bodies is calculated according to the cosine formula, and the vector dimension is 17*17:

[0076]

[0077] Fifthly, the index position corresponding to the maximum Cobb angle is calculated;

[0078] Step 6, calculate the coordinates of the center points of the upper and lower edges of each vertebra, a total of 34 points, recorded as mid_p, and the center point of the lower edge of the lowest vertebra is recorded as PC34, the height-width distance of each point in mid_p to point PC34 is calculated from top to bottom, and the proportion of the height-width distance to the height and width of the vertebra is calculated, and the difference between the two is calculated to form a 34x1 vector, recorded as vec_dist_wh, and a matrix mat_dist_wh is calculated, with a size of 34x34:

[0079] mat_dist_wh = vec_dist_wh * vec_dist_wh T

[0080] The sum of mat_dist_wh and the sum of its absolute value are calculated, and the difference between the two is calculated; if the difference is greater than 0.01, it indicates that the spine is S-shaped, otherwise the spine is straight;

[0081] If the spine is straight, it indicates that the maximum Cobb angle is in the middle of the spine, which is the MT region Cobb angle, then the PT region Cobb angle is calculated along the upper region of the maximum angle, and the TL / L region Cobb angle is calculated along the lower region of the maximum angle.

[0082] If the spine is S-shaped, calculate whether the maximum CObb angle region is in the upper part of the spine or in the lower part of the spine. If in the upper part, the maximum Cobb angle is the PT region, and then the Cobb angles of the MT, TL / L regions are calculated in turn; if in the lower part, the maximum Cobb angle is the TL / L region, and then the Cobb angles of the MT, PT regions are calculated in turn.

[0083] In specific implementation, in the above-mentioned spine full-length film Cobb angle detection device provided by the embodiment of the application, as shown in Figure 4 The spine ROI positioning model adopts a U-net network structure; the backbone network of the U-net network structure is MobileNet-v2, MobileNet-v2 is a lightweight real-time segmentation network, and coarse-grained segmentation; MobileNet-v2 includes a linear bottleneck layer and an inverse residual module; the linear bottleneck layer is set at the output end of each convolution module using a linear activation function; and the inverse residual module uses a 1x1 convolution layer to upgrade the feature map in the residual module.

[0084] It should be noted that the U-net network structure shape is similar to U type, which is a convolutional neural network with nine layers of depth, which can combine the information of the bottom layer and the high layer, and the low resolution information obtained after multiple downsampling can provide the semantic information of the segmentation target. In order to further improve the recognition accuracy of MobileNet network, linear bottleneck layer and inverse residual module are introduced on the basis of MobileNet-v1, and MobileNet-v2 is proposed. The main reason for introducing the linear bottleneck layer is that considering that the output feature dimension is low after the depth separable convolution is adopted in MobileNet-v1, the effective feature information is easy to be lost by using the nonlinear activation function, thereby affecting the final recognition effect, therefore, the linear activation function is adopted at the output end of each convolution module in MobileNet-v2, so as to retain more information. Different from the traditional residual module which introduces a bottleneck layer to reduce the dimension of the feature map and improve the sparsity of the network, the inverse residual module uses a 1x1 convolution layer to increase the dimension of the feature map in the residual module, so as to make up for the problem of insufficient feature complexity after the depth separable convolution is adopted in MobileNet-v1, and after the operation of the connection layer, the high resolution information can be obtained from the encoder to the decoder, that is, more detailed features are provided for segmentation. Medical images have fuzzy boundaries, complex gradients, require high resolution information, and the structure inside the human body is relatively fixed, and the distribution of the segmentation target in the image has a regularity, while the low resolution can provide this information.

[0085] In specific implementation, in the above-mentioned Cobb angle detection device for full-length spine film provided by the embodiment of the application, as shown in Figure 5 , the vertebral body key point model adopts a stacked hourglass network; the stacked hourglass network uses a convolutional neural network to detect a given single RGB image and outputs the coordinate position of the human body key point. The stacked hourglass network uses a residual network as a basic module to downsample and upsample the whole picture to extract multi-scale features.

[0086] In specific implementation, in the above-mentioned Cobb angle detection device for full-length spine film provided by the embodiment of the application, as shown in Figure 6 , the stacked hourglass network is formed by cascading a plurality of hourglass networks; two outputs of the previous hourglass network and the original input picture are added as the input of the next hourglass network. The structure of each hourglass network is the same, Figure 7 , which shows a structural schematic diagram of the hourglass network. The purpose of the hourglass network is to obtain feature maps of various scales to capture information contained in pictures of different scales. Preferably, the stacked hourglass network is trained using relay supervision at the end of the hourglass network, and the type and position information of the vertebral body key point is integrated into the output feature map to obtain a heat map of each key point.

[0087] It should be noted that the present application repeatedly down-sampling and up-sampling multiple times to construct a cascade hourglass network, which means that the entire network has multiple top-down and bottom-up flows to capture information of different scales. And with the increase of the number of cascade networks, the information of the body key points is continuously enhanced, and the cascade hourglass network can gradually adjust the key points. In addition, since the position distribution of the spine is rectangular and curved, a method based on heat map position point regression is adopted, and the accuracy and robustness of the cascade hourglass network in human pose estimation are used to construct an efficient key point regression algorithm.

[0088] Based on the same inventive concept, the present application also provides a detection method of the spinal full-length film Cobb angle detection device. Since the principle of solving the problem of the method is similar to the above-mentioned spinal full-length film Cobb angle detection device, the implementation of the method can be referred to the implementation of the spinal full-length film Cobb angle detection device, and the repeated parts will not be described again.

[0089] In specific implementation, the detection method of the spinal full-length film Cobb angle detection device provided by the present application embodiment, as shown in Figure 8 , specifically includes the following steps:

[0090] S801, acquiring multiple spinal full-length film images of different orientations which have been labeled with vertebral key points as training samples;

[0091] S802, constructing a spinal detection model composed of a spinal ROI positioning model and a vertebral key point model; the spinal ROI positioning model takes the spinal full-length film image as input and outputs the spinal ROI image; the vertebral key point model takes the spinal ROI image as input and outputs the position coordinates of the vertebral key points;

[0092] S803, training the spinal detection model using the training samples until the network converges;

[0093] S804, inputting the different orientations of the to-be-detected spinal full-length film images into the spinal detection model, and outputting the position coordinates of the vertebral key points of the different orientations of the to-be-detected spinal full-length film images;

[0094] S805, obtaining the Cobb angles of the upper thoracic bend, thoracic bend, and thoracolumbar / lumbar bend corresponding to different orientations according to the position coordinates of the vertebral key points of the different orientations of the to-be-detected spinal full-length film images.

[0095] In the detection method of the spinal full-length film Cobb angle detection device provided in the embodiment of the present application, first, the spinal detection model composed of the spinal ROI positioning model and the vertebral key point model is constructed and trained, the spinal ROI image is acquired by using the spinal ROI positioning model, the position coordinates of the vertebral key points are predicted by using the vertebral key point model, then the Cobb angles of different regions corresponding to the different orientations of the to-be-detected spinal full-length film image are calculated by using the predicted position coordinates of the key points, and the interpretability of the algorithm process is further realized, the detection mode is simplified, manual operation is saved, and the detection accuracy is improved.

[0096] The more specific working processes of the above steps can refer to the corresponding contents disclosed in the foregoing embodiments, and will not be described here.

[0097] Correspondingly, the embodiment of the present application also discloses an electronic device comprising a processor and a memory; wherein the processor implements the detection method of the spinal full-length film Cobb angle detection device disclosed in the foregoing embodiments when executing the computer program stored in the memory.

[0098] The more specific processes of the above method can refer to the corresponding contents disclosed in the foregoing embodiments, and will not be described here.

[0099] Further, the present application also discloses a computer readable storage medium for storing a computer program; the computer program is executed by a processor to implement the detection method of the spinal full-length film Cobb angle detection device disclosed above.

[0100] The more specific processes of the above method can refer to the corresponding contents disclosed in the foregoing embodiments, and will not be described here.

[0101] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other. For the method, device and storage medium disclosed in the embodiment, since it corresponds to the device disclosed in the embodiment, the description is relatively simple, and the related parts can refer to the device part description.

[0102] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0103] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0104] In summary, the Cobb angle detection device for full-length spinal radiographs provided by this invention includes: a training sample acquisition module, used to acquire multiple full-length spinal radiographs from different orientations with vertebral key points annotated as training samples; a model construction module, used to construct a spinal detection model composed of a spinal ROI localization model and a vertebral key point model; the spinal ROI localization model takes the full-length spinal radiographs as input and outputs the spinal ROIs; the vertebral key point model takes the spinal ROIs as input and outputs the position coordinates of the vertebral key points; a model training module, used to train the spinal detection model using the training samples until the network converges; a coordinate output module, used to input the full-length spinal radiographs of the subject from different orientations into the spinal detection model and output the position coordinates of the vertebral key points in the full-length spinal radiographs of the subject from different orientations; and a Cobb angle calculation module, used to obtain the Cobb angles of the upper thoracic curve, thoracic curve, and thoracolumbar / lumbar curves corresponding to different orientations based on the position coordinates of the vertebral key points in the full-length spinal radiographs of the subject from different orientations. This approach leverages the interaction of the five modules mentioned above to first construct and train a spine detection model composed of a spine ROI localization model and a vertebral keypoint model. The spine ROI localization model is used to acquire spine ROI images, and the vertebral keypoint model is used to predict the position coordinates of vertebral keypoints. Then, using the predicted keypoint coordinates, the Cobb angles of different regions corresponding to the full-length spine image from different orientations can be calculated. This further enhances the interpretability of the algorithm, simplifies the detection method, eliminates manual operation, and improves detection accuracy. Furthermore, this invention also provides a corresponding method, device, and computer-readable storage medium for detecting the Cobb angle of a full-length spine image, further enhancing the practicality of the device. The method, device, and computer-readable storage medium offer corresponding advantages.

[0105] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be used to distinguish one entity or action from another entity or action, without necessarily requiring or implying that there is any such actual relationship or order between these entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover non-exclusive inclusions, so that a process, method, article, or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0106] The above describes the full-length spine Cobb angle detection device, method, equipment and storage medium provided by the application in detail, the principle and implementation mode of the application are described by applying specific examples in this document, and the above example is only used to help understand the method and core idea of the application; meanwhile, for those skilled in the art, according to the idea of the application, the specific implementation mode and application range will be changed, and the above description should not be understood as a limitation of the application.

Claims

1. A device for detecting the Cobb angle of a full-length spinal radiograph, characterized in that, The application relates to a method for detecting vertebral key points of a spine, and belongs to the technical field of medical image processing. The method comprises the following steps: a training sample acquisition module is used to acquire multiple different-orientation spine full-length film images with labeled vertebral key points as training samples; a model construction module is used to construct a spine detection model composed of a spine ROI positioning model and a vertebral key point model; the spine ROI positioning model takes a spine full-length film image as input and outputs a spine ROI image; the vertebral key point model takes a spine ROI image as input and outputs the position coordinates of a vertebral key point; a model training module is used to train the spine detection model by using the training samples until the network converges; wherein the training samples are input into the spine ROI positioning model; in the process of training the spine ROI positioning model, a vertebral mask region of the spine is generated according to the labeled vertebral key points, and the maximum outer rectangle of the mask region is calculated to obtain a spine ROI image; the network of the spine ROI positioning model adopts an imagenet pre-training model of mobilenet-v2, an optimizer adopts SGD, and a loss function adopts Dice Loss; the obtained spine ROI image is input into the vertebral key point model, in the process of training the vertebral key point model, a heat map for representing the position coordinates of a vertebral key point is generated according to the labeled vertebral key points, and the heat map is used to encode the coordinates of the vertebral key point; the optimizer of the vertebral key point model adopts Adam, and the loss function adopts MSE; The Cobb angle calculation module is used to obtain the Cobb angles (PT, MT, TL / L) for different orientations based on the coordinates of key vertebral points in the full-length radiographs of the spine under test from different orientations. Here, PT represents the upper thoracic curve, MT represents the thoracic curve, and TL / L represents the thoracolumbar / lumbar curve. For the full-length anteroposterior radiograph of the spine under test, the Cobb angle calculation module uses the positions of key vertebral points to obtain the Cobb angles (PT, MT, TL / L) and the coordinates of the key vertebrae above and below each Cobb angle, as well as the vertebral index. For the key vertebral points in the left and right flexion radiographs of the spine under test, the Cobb angle calculation module uses the vertebral index positions to calculate the Cobb angles (PT, MT, TL / L) for the left and right flexion radiographs of the spine under test. In the full-length anteroposterior radiograph of the spine under test, the horizontal midline segment of each vertebra, i.e., the midpoint of the left and right sides, is calculated and used as the starting coordinates. The vector for calculating the starting coordinates is denoted as... The vector dimension is 17×2; calculate the vector matrix. , The matrix size is 17×17; T denotes the quasi-position; calculate the vector. The length of the module is recorded as Vector dimension 17×1; calculation The modulus matrix is ​​denoted as , The matrix size is 17×17; based on the vector matrix and Modulus matrix The ratio of the two vertebrae is used in conjunction with the cosine formula to calculate the candidate region for the Cobb angle formed by each vertebra and other vertebrae; the index position corresponding to the largest Cobb angle is calculated; the coordinates of the center points of the upper and lower edges of each vertebra are calculated, totaling 34 points, denoted as [missing information]. Let PC34 be the center point of the lower edge of the lowest vertebral body, and calculate from top to bottom. The ratio of the height and width distance from each point to point PC34 to the height and width of the cone is calculated. The difference between the two ratios is used to form a 34×1 vector, denoted as . Calculate the matrix The size is 34×34: ;calculate and Calculate the sum of the absolute values ​​of the numbers. The sum and The difference between the absolute values ​​and the sum of the values; if the difference is greater than the set threshold, it indicates that the spine is S-shaped; otherwise, the spine is straight. a coordinate output module is used to input different-orientation to-be-detected spine full-length film images into the spine detection model, and output the position coordinates of the vertebral key points of the different-orientation to-be-detected spine full-length film images; the different-orientation to-be-detected spine full-length film images comprise a to-be-detected spine full-length frontal film, a to-be-detected spine full-length left lateral bending film and a to-be-detected spine full-length right lateral bending film; the to-be-detected spine full-length frontal film, the to-be-detected spine full-length left lateral bending film and the to-be-detected spine full-length right lateral bending film are input into the spine ROI positioning model for spine positioning, and three vertebral ROI images in different directions are obtained respectively; the three vertebral ROI images in different directions are input into the vertebral key point model for vertebral key point detection, and vertebral key points are obtained respectively; if the spine is straight, the maximum Cobb angle is in the middle of the spine, that is, the MT region Cobb angle, the PT region Cobb angle is calculated along the upper region of the maximum angle, and the Cobb angle of the TL / L region is calculated along the lower region of the maximum angle; 2. The full length spinal column Cobb angle detection device of claim 1, wherein, if the spine is S-shaped, the maximum Cobb angle region is calculated; if the maximum Cobb angle region is in the upper part of the spine, the maximum Cobb angle is the PT region, and the Cobb angles of the MT and TL / L regions are calculated in sequence from top to bottom; if the maximum Cobb angle region is in the lower part of the spine, the maximum Cobb angle is the TL / L region, and the Cobb angles of the MT and PT regions are calculated in sequence from top to bottom. The spine ROI positioning model adopts a U-net network structure. The backbone network of the U-net network structure is MobileNet-v2; the MobileNet-v2 includes a linear bottleneck layer and an inverse residual module; The linear bottleneck layer is arranged at the output end of each convolution module and adopts a linear activation function; The inverse residual module adopts a 1x1 convolution layer to increase the dimension of the feature map in the residual module.

3. The full length spinal column Cobb angle detection device of claim 1, wherein, The vertebral key point model adopts a cascaded hourglass network; The cascaded hourglass network takes a residual network as a basic module to down-sample and up-sample a picture to extract multi-scale features.

4. The full length spinal column Cobb angle detection device of claim 3, wherein, The cascaded hourglass network is formed by cascading a plurality of hourglass networks; two outputs of a previous hourglass network and an original input picture are added to serve as an input of a next hourglass network. The hourglass network is used to obtain feature maps of various scales to capture information contained in pictures of different scales.

5. The full length spinal column Cobb angle detection device of claim 4, wherein, The cascaded hourglass network is trained by using relay supervision to integrate type and position information of vertebral key points into an output feature map to obtain a heat map of each key point.

6. A detection method using the full-length spinal Cobb angle detection device according to any one of claims 1 to 5, characterized in that, The method comprises the following steps: a plurality of different orientation spine full-length film images labeled with vertebral key points are acquired as training samples; a spine detection model composed of a spine ROI positioning model and a vertebral key point model is constructed; the spine ROI positioning model takes a spine full-length film image as an input and outputs a spine ROI image; the vertebral key point model takes a spine ROI image as an input and outputs position coordinates of vertebral key points; the spine detection model is trained using the training samples until the network converges; different orientation to-be-detected spine full-length film images are input into the spine detection model, and position coordinates of vertebral key points of the different orientation to-be-detected spine full-length film images are output; Cobb angles of upper thoracic bends, thoracic bends and thoracolumbar / lumbar bends corresponding to different orientations are obtained according to the position coordinates of the vertebral key points of the different orientation to-be-detected spine full-length film images.

7. An electronic device, comprising: A processor and a memory are included, wherein the processor implements the detection method of the spine full-length film Cobb angle detection device of claim 6 when executing a computer program stored in the memory.

8. A computer-readable storage medium, characterized in that, A computer program is stored, wherein the computer program is executed by a processor to implement the detection method of the spine full-length film Cobb angle detection device of claim 6.

Images