A method for detecting and identifying key points in X-ray spine images

Abstract

The present invention belongs to the field of medical image processing, and discloses a method for detecting and identifying key points of X-ray spine images. The method comprises: firstly obtaining and annotating an X-ray spine image, then using the image and key point coordinates to train a spine key point detection network, then using the image and name labels to train a spine recognition network, then cutting out a single vertebra image, using the single vertebra image and key point coordinates to train a single vertebra key point detection network, and then using the trained model to make predictions, and after the predictions, post-processing is required, and then a vertebra connection graph is constructed, the longest path is obtained to obtain the most likely vertebra in the graph, and finally the vertebra names of the coordinates of the key points in the spine image are integrated. The present invention sorts out the key points and identification of X-ray spine images. The key point detection strategy of coarse first and fine later is used to improve the accuracy, and a small number of cases are taken into account, so that it has better generalization.

Description

Technical Field

[0001] The invention belongs to the field of medical image processing, and in particular relates to a method for detecting and identifying key points of X-ray spine images. Background Art

[0002] The spine is an important structure that supports the human body. The spine is composed of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum and coccyx connected by ligaments, joints and intervertebral discs. With the development of society, spinal diseases are becoming younger due to some bad habits. In the diagnosis process, taking X-rays is an important auxiliary means. At present, the identification of the spine in X-rays is mostly done by manual annotation, which has many problems, such as long reading time, high noise in X-rays, and easy errors. Therefore, there is a need for a method to automatically annotate the names of vertebrae.

[0003] Surgical robots are a way to treat spinal diseases. To implement surgical robots, the spine needs to be automatically positioned during the operation. One way is to take a CT scan before the operation and use X-rays to position the spine during the operation. This involves CT-X-ray registration, and one way is to find the key points of the CT and the key points of the X-ray for registration. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for detecting and identifying key points of X-ray spine images.

[0005] The technical solution provided by the present invention is:

[0006] (1) Obtain X-ray spinal images, annotate the images, mark the coordinates and names of the key points of each vertebra, and then divide the images in the dataset into training set, validation set, and test set according to the proportion. After the division, expand the dataset for the usage scenario where the number of vertebrae in the image is uncertain;

[0007] (2) using the spinal image as input and the key point coordinates of each vertebra as labels to train the spinal key point detection network to obtain a first network model;

[0008] (3) Using the spinal image as input and the name of each vertebra as a label to train the spinal recognition network, a second network model is obtained;

[0009] (4) Based on step (2), in order to obtain more precise key point coordinates, a single vertebrae image is cropped, and the single vertebrae image is used as input. The key point coordinates of the vertebrae are used as labels to train a single vertebrae key point detection network, thereby obtaining a third network model;

[0010] (5) Based on step (2), the spinal image is input into the trained first network model, and after post-processing, the coordinates of key points in the image are predicted;

[0011] (6) Based on step (4) and step (5), using the key point coordinates of each vertebra predicted in step (5), a single vertebra is cropped, and the image of the single vertebra is input into the trained third network model. After post-processing, more accurate key point coordinates of the single vertebra are predicted, and then these key point coordinates need to be transformed back to the original image;

[0012] (7) Based on step (3), the spinal image is input into the trained second network model to predict the probability of existence of each vertebra in the image;

[0013] (8) Based on steps (6) and (7), in order to solve the problem that the number of vertebrae of a few people is arranged differently, a vertebra connection diagram is constructed, and the most accurate name of each vertebra in the image is obtained by finding the longest path;

[0014] (9) Based on step (6) and step (8), draw the key points of each predicted vertebra in the spinal image and mark the name of each predicted vertebra.

[0015] Furthermore, for the key points of each vertebra marked in step (1), regardless of whether the data set uses an anteroposterior or lateral X-ray of the entire spine, the four corner points of each vertebra are selected as key points, and their coordinates x, y are recorded.

[0016] Furthermore, the data set expansion in step (1) includes: starting from 1, the number of vertebrae in the image is k, and then each vertebra in the image is taken as the first vertebra in the image is v n , the second vertebra is v n+1 , you can extract the content containing {v n ,v n+1 ,…,v n+k-1}, the interception method is to find the minimum and maximum x-coordinate values ​​and y-coordinate values ​​according to the coordinates of the key points of all vertebrae, and then expand these values ​​to a certain distance in all directions, so as to obtain an image containing only part of the vertebrae. Accordingly, the key points also need to be adjusted.

[0017] Furthermore, the construction of the spine key point detection network model described in step (2) specifically includes:

[0018] (2.1) Expand the input spine image into a square;

[0019] (2.2) Perform online data augmentation on the images in the training set, such as random stretching, random brightness and contrast transformation, random horizontal mirror flipping, and random rotation. Correspondingly, the coordinates of the key points should also be transformed;

[0020] (2.3) The model is an encoder-decoder structure. The downsampling part uses Resnet34. In order to speed up the training speed and generalization ability of the model, the transfer learning technology is used to fine-tune the parameters that have been trained on ImageNet. The upsampling part first uses bilinear interpolation to upsample the feature map, and then uses a 3×3 convolution kernel with a step size of 1. It is then normalized using BatchNorm and connected to a ReLU activation function. At the same time, skip connections are used for feature reuse, so that the downsampled features assist in upsampling learning. Before skip connections are made, an Attention Gate, assuming that the feature map of the jump connection of the upsampling part is g, and the corresponding feature map of the downsampling part is x, first perform a 1×1 convolution operation on g and x respectively to reduce the dimension of the feature map to half, then add the two obtained feature maps and perform a ReLU operation, then use a 1×1 convolution to reduce the dimension to 1, then perform sigmoid, and finally multiply it with x to assign the attention weight to the original feature map. The result of Attention is then concatenated with the result of upsampling and a 1×1 convolution is used to reduce the dimension to the same as the original, so that the attention mechanism is implemented for the extracted features;

[0021] (2.4) The features are sent to three branches respectively, and continuous convolution is performed using 3×3 and 1×1 convolution kernels. Finally, a heat map with a channel number of 1 and a center point coordinate offset with a dimension of 2 is obtained. For the coordinate offset of the four corners relative to the center point, continuous convolution is performed using 3×3 and 7×7 convolution kernels to obtain the coordinate offset of the four corners relative to the center point with a dimension of 8. Among them, the heat map also requires sigmoid to facilitate the calculation of loss and post-processing;

[0022] (2.5) Based on the real key point coordinates of each spine image, the ground truth Gaussian heat map is generated. First, the bounding box is calculated based on the real key point coordinates. The radius R of the Gaussian circle is calculated based on the size of the bounding box of each vertebra. Finally, on the heat map, the center point of the bounding box is used as the circle center and the radius is R. The IOU critical value is overlap. The size of the bounding box is h×w. The formula for calculating R is:

[0023]

[0024] Among them, the values ​​of a, b, and c are:

[0025]

[0026] Take the coordinates of the center point of the bounding box as (p x ,p y ), then the Gaussian function is:

[0027]

[0028] Among them, the value of σ is

[0029] For heat maps, the improved focal loss is used as the loss function, and the specific calculation method is:

[0030]

[0031] Where N represents the number of key points in the image, Y xy represents the true value, Represents the predicted value. For the offset of the center point coordinates and the coordinate offset of the four corners relative to the center point, L1 loss is used as the loss function. The total loss function is the sum of the calculation results of the three loss functions.

[0032] (2.6) After training, the first network model can be obtained.

[0033] Furthermore, the construction of the spine recognition network model described in step (3) specifically includes:

[0034] (3.1) Expand the input spine image into a square;

[0035] (3.2) Perform online data augmentation on the images in the training set, such as random stretching, random brightness and contrast transformation, random horizontal mirror flipping, and random rotation;

[0036] (3.3) Build the ResNeXt50 model, load the weight parameters of the ResNeXt50 that has been pre-trained on ImageNet into the model, and use transfer learning to freeze all the parameters of the first three layers and only train the parameters after the fourth layer of the model;

[0037] (3.4) Add the CSRA attention module at the end of the network. The CSRA attention module has two branches, namely Spatial pooling and Average pooling, and then perform weighted fusion of the two:

[0038] F i =G i +λ*M i (5)

[0039] Among them, F i represents the probability of the i-th category, G i represents the probability of the i-th category after average pooling, M i Represents the probability of the i-th category after spatial pooling;

[0040] (3.5) Because spine recognition can be regarded as a multi-label classification problem, a sigmoid activation function is used for each output neuron to map the prediction results to the interval (0,1);

[0041] (3.6) In view of the possible category imbalance in the data set, focal loss is used as the loss function.

[0042] (3.7) After training, the second network model can be obtained.

[0043] Furthermore, the method for constructing the single vertebra key point detection network described in step (4) is the same as step (2), except that the image input to the network is different, that is, a single vertebra image is used as input, and the third network model can be obtained after training.

[0044] Furthermore, the post-processing process described in step (5) includes: finding the maximum point for the heat map, then extracting the confidence score, index and center point coordinates of the center point with a confidence greater than a threshold, and then using the predicted offset of the center point coordinates and the coordinate offset of the four corners relative to the center point to obtain the coordinates of the four key points; for the post-processing process of a single vertebra, no threshold is required, and the center point with the maximum confidence can be directly obtained.

[0045] Furthermore, the construction of the vertebra connection graph described in step (8) includes: using the name of each vertebra as a node in the graph, connecting vertebrae that may be adjacent to each other according to actual conditions, and using the probability of each vertebra as the weight of each edge.

[0046] Compared with the prior art, the advantages of the present invention are:

[0047] (1) The fully automatic workflow for key point and recognition of X-ray spine images was sorted out, which can realize key point detection and recognition of cervical, thoracic and lumbar vertebrae.

[0048] (2) The accuracy of key point detection is higher. A coarse-first-fine-later key point detection strategy is adopted. Key point detection is performed on the entire image first, and then fine key point detection is performed on a single vertebra.

[0049] (3) Better generalization: In view of the fact that a small number of people have different spinal alignment orders, a plan has been developed that takes these situations into consideration. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the working principle of the key point detection and recognition of X-ray spine images described in the present invention.

[0051] Figure 2 This is a structural diagram of the spine key point detection network described in the present invention.

[0052] Figure 3 This is the spine connection diagram described in the present invention. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present invention will be fully described below in conjunction with the drawings in the embodiments of the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work based on the embodiments of the present invention shall fall within the scope of protection of the present invention.

[0054] Example:

[0055] like Figure 1 FIG. 1 is an embodiment of the present invention, which provides a method for detecting and identifying key points of an X-ray spine image, comprising the following steps:

[0056] (1) Obtain X-ray spinal images, annotate the key points of these images and the names of the vertebrae in the images. Put the key point coordinates and vertebrae names of each spinal image into a dictionary type variable, and then write this variable into an npy file, that is, each image has a label file with the extension npy.

[0057] Since the model has hyperparameters that need to be adjusted, it is necessary to divide the training set, validation set, and test set and then put them into different subfolders.

[0058] Because we need to expand the data set, we start enumerating the number of vertebrae in the image from 1 as k, and then let each vertebra in the image be the first vertebra in the image as v n , the second vertebra is v n+1 , you can extract the content containing {v n ,v n+1 ,…,v n+k-1 The interception method is to find the minimum and maximum x- and y-coordinate values ​​based on the coordinates of the key points of all vertebrae, and then expand these values ​​by 15 pixels in all directions to obtain an image that only contains part of the vertebrae. Accordingly, the key points also need to be adjusted. In this way, the training set, validation set, and test set are expanded respectively.

[0059] (2) Using the spinal image as input and the key point coordinates of each vertebra as labels, the spinal key point detection network is trained to obtain the first network model.

[0060] First, the spine image is expanded into a square. The expansion method is to take the longer side of the length and width as the side length of the square, and fill the rest with black. The original spine image should be kept on the left or top of the expanded square, so that the original key points can be correct after expansion. Then the images in the data set are resized to the same size of 512×512.

[0061] In order to improve the robustness of the model, it is necessary to use data enhancement technology to perform some transformations on the training set. This embodiment uses data enhancement strategies such as random stretching, random brightness and contrast transformation, random horizontal mirror flipping and random rotation. Of course, when performing random stretching, random horizontal mirror flipping and random rotation transformations, the coordinates of the key points must also be transformed accordingly.

[0062] When building the model, the overall network is an encoder-decoder architecture, and the network structure is as follows Figure 2 The downsampling part uses Resnet34 as the backbone network. In order to speed up the training speed and generalization ability of the model, the transfer learning technology is used to fine-tune the parameters that have been trained on ImageNet.

[0063] The upsampling part performs 3 layers of operations, with the number of image channels set to 256, 128, and 64 respectively. At the same time, skip connections are used to reuse features, allowing the downsampled features to assist upsampling learning. Before connecting the features on the encoder with the corresponding features in the decoder, an Attention Gate is used to readjust the output features of the encoder. This module generates a gating signal to control the importance of features at different spatial locations. A specific structure is that the three layers first use bilinear interpolation to upsample the feature map, then use a 3×3 convolution kernel with a step size of 1 to operate, and then use BatchNorm for normalization, connect a ReLU activation function, and pass through an Attention Gate before performing a jump connection. Assuming that the feature map of the jump connection of the upsampling part is g, and the corresponding feature map of the downsampling part is x, first perform a 1×1 convolution operation on g and x respectively to reduce the dimension of the feature map to half, then add the two obtained feature maps and perform a ReLU operation, then use a 1×1 convolution to reduce the dimension to 1, then perform a sigmoid, and finally multiply it with x to assign the attention weight to the original feature map. The result of Attention is then concatenated with the result of upsampling, and a 1×1 convolution is used to reduce the dimension to the same as the original.

[0064] After that, the feature map is sent to three branches respectively to obtain the heat map, the offset of the center point coordinates, and the coordinate offset of the four corners relative to the center point. A specific construction is that in order to obtain the heat map and the offset of the center point coordinates, 3×3 and 1×1 convolution kernels are used for continuous convolution, and finally a heat map with a channel number of 1 and a size of 128×128 and a center point coordinate offset with a dimension of 2 and a size of 128×128 is obtained. For the coordinate offset of the four corners relative to the center point, 3×3 and 7×7 convolution kernels are used for continuous convolution to obtain a dimension of 8 and a size of 128×128 for the coordinate offset of the four corners relative to the center point. Among them, the heat map also needs sigmoid to facilitate the calculation of loss and post-processing.

[0065] In this embodiment, it is necessary to generate a real Gaussian heat map. The specific generation method is to first scale the key points to a scale of 128×128, calculate the size of the bounding box according to the key point coordinates, then calculate the coordinates of the center point of the bounding box and round it to point, and calculate the radius of the Gaussian circle according to the size of the bounding box of each vertebra, and set it to R. Finally, on the heat map, fill the Gaussian function calculation value with point as the center and R as the radius. Among them, the IOU critical value is overlap, the size of the bounding box is h×w, and the formula for calculating R is:

[0066]

[0067] Among them, the values ​​of a, b, and c are:

[0068]

[0069] Take the coordinates of the center point of the bounding box as (p x ,p y ), then the Gaussian function is:

[0070]

[0071] Among them, the value of σ is

[0072] In this embodiment, the loss function of the heat map uses the improved focal loss:

[0073]

[0074] Where N represents the number of key points in the image, Y xy represents the true value, In this embodiment, the hyperparameter α is 2 and β is 4.

[0075] The loss function for the offset of the center point coordinates and the offset of the four corners relative to the center point uses L1loss. The final loss is the sum of the three.

[0076] (3) The spine image is used as input and the name of each vertebra is used as a label to train the spine recognition network to obtain the second network model.

[0077] First, the spine image is expanded into a square by taking the longer side of the length and width as the side length of the square and filling the rest with black. Then the images in the dataset are resized to the same size of 512×512.

[0078] Similarly, in order to use data enhancement technology to perform some transformations on the training set, this embodiment uses data enhancement strategies such as random stretching, random brightness and contrast transformation, random horizontal mirror flipping, and random rotation.

[0079] The network backbone uses ResNeXt50. The weight parameters of ResNeXt50 that have been pre-trained on ImageNet are loaded into the model, and transfer learning is used to freeze all the parameters of the first three layers, and only train the parameters after the fourth layer of the model.

[0080] At the end of the network, a CSRA attention module is added. The CSRA attention module has two branches, namely Spatialpooling and Average pooling, and then the two are weighted fused. In this embodiment, a more specific structure is to perform a 1×1 convolution operation and then flatten it to obtain a linear feature map with 26 channels, representing 7 cervical vertebrae, 12 thoracic vertebrae, 5 lumbar vertebrae, and possible extra thoracic vertebrae and extra lumbar vertebrae. The feature map is then sent to two branches, one for average pooling and the other for max pooling, and then the two results are weighted fused:

[0081] F i =G i +λ*M i (5)

[0082] Among them, F i represents the probability of the i-th category, G i represents the probability of the i-th category after average pooling, M i represents the probability of the i-th category obtained after max pooling. In this embodiment, the hyperparameter λ is set to 0.2.

[0083] Because spine recognition can be viewed as a multi-label classification problem, a sigmoid activation function is used for each output neuron to map the prediction results to the interval (0, 1).

[0084] The loss function used in this embodiment is focal loss to alleviate the problem of class imbalance that may exist in the data set.

[0085] (4) On the basis of step (2), in order to obtain more precise key point coordinates, the image of a single vertebra is cropped, and the single vertebra image is used as input. The key point coordinates of the vertebra are used as labels to train the single vertebra key point detection network to obtain the third network model.

[0086] The network used in this step is the same as that in step (2) and will not be described further.

[0087] The training images are obtained by finding the maximum and minimum values ​​of the x,y coordinates of the four key points of each vertebra for each spinal image, and then expanding 15 pixels outward on all sides to obtain the image of a single vertebra.

[0088] (5) Based on step (2), the spinal image is input into the trained first network model, and after post-processing, the coordinates of the key points in the image are predicted.

[0089] In the post-processing process, the maximum point is first found for the heat map. In this embodiment, a 3×3 maxpooling is used to retain the maximum point, and then the value of the maximum point is retained, and the non-maximum point is set to 0. Then the confidence score, index and center point coordinates of the center point with a confidence greater than the threshold are extracted, and the coordinates of the four key points are obtained by using the predicted offset of the center point coordinates and the offset of the four corners relative to the center point. In this embodiment, the threshold is set to 0.3.

[0090] (6) Based on step (4) and step (5), the key point coordinates of each vertebra predicted in step (5) are used to crop a single vertebra, and the image of the single vertebra is input into the trained third network model. After post-processing, more accurate key point coordinates of the single vertebra are predicted, and then these key point coordinates need to be transformed back to the original image.

[0091] The process of cropping out a single vertebra is to obtain the maximum and minimum values ​​of the x, y coordinates of the four key points of each vertebra predicted in step (5), and then expand outward and then outward in all directions by 15 pixels to obtain the image of the single vertebra.

[0092] The post-processing process is the same as the post-processing process in step (5) except that only the coordinates of the center point with the maximum confidence level need to be extracted.

[0093] The key point transformation process is to record the coordinates of the four corners of the cropped single vertebra image in the original image, and then add the offset value of the cropped image on the original image to the predicted coordinates of the single vertebra.

[0094] (7) Based on step (3), the spinal image is input into the trained second network model to predict the probability of the existence of each vertebra in the image.

[0095] (8) Based on steps (6) and (7), in order to solve the problem that a few people have different numbers of vertebrae, a vertebra connection diagram is constructed, and the most accurate names of each vertebra in the image are obtained by finding the shortest path.

[0096] Vertebral connection diagram Figure 3 As shown in the figure, the number of layers of the connection graph is the number of vertebrae in the predicted spinal image. Each layer has 26 nodes, representing 7 cervical vertebrae, 12 thoracic vertebrae, 5 lumbar vertebrae, and possible extra thoracic vertebrae and extra lumbar vertebrae. Then the negative number of the predicted probability of each category is used as the weight of the edge. The reason for using negative numbers is that this can transform the longest path problem into the shortest path problem.

[0097] The problem of different arrangement of the number of vertebrae of a few people is specifically that there may be one more thoracic vertebra, one less thoracic vertebra, or one more lumbar vertebra. For the problem of one more thoracic vertebra or one less lumbar vertebra, in this embodiment, these two vertebrae are directly considered as two more categories, and for the problem of one less thoracic vertebra, in this embodiment, an edge is connected between node T11 and node L1.

[0098] A super source and a super sink should be established on both sides to simplify the problem and find the shortest path.

[0099] The method to find the shortest path is the SPFA algorithm (queue optimization algorithm of the Bellman-Ford algorithm), because there are negative weight edges in the graph. At the same time, the best path must be maintained, and finally the most likely vertebra in the image can be obtained.

[0100] (9) Based on step (6) and step (8), draw the key points of each predicted vertebra in the spinal image and mark the name of each predicted vertebra.

[0101] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, shall fall within the protection scope of the present invention.

Claims

1. A method for detecting and identifying key points of an X-ray spine image, characterized in that: The following steps are involved: S1: Obtain X-ray spinal images, annotate the images, mark the key point coordinates and names of each vertebra, and then divide the images in the dataset into training set, validation set and test set according to the proportion. After the division, expand the dataset for the use scenario where the number of vertebrae in the image is uncertain; S2: Using the spinal image as input and the key point coordinates of each vertebra as labels to train the spinal key point detection network to obtain a first network model; the construction of the spinal key point detection network model specifically includes: S21: Expand the input spine image into a square; S22: Perform online data enhancement on the images in the training set, specifically random stretching, random brightness and contrast transformation, random horizontal mirror flipping and random rotation. Correspondingly, the coordinates of the key points should also be transformed; S23: The model is an encoder-decoder structure. The downsampling part uses Resnet34. In order to speed up the training speed and generalization ability of the model, the transfer learning technology is used to fine-tune the parameters that have been trained on ImageNet. The upsampling part first uses bilinear interpolation to upsample the feature map, and then uses a 3×3 convolution kernel with a step size of 1. It is normalized with BatchNorm and connected to a ReLU activation function. At the same time, skip connections are used for feature reuse, so that the downsampled features assist upsampling learning. Before skip connections are performed, a AttentionGate: Assume that the feature map of the jump connection of the upsampling part is g, and the corresponding feature map of the downsampling part is x. First, perform a 1×1 convolution operation on g and x respectively to reduce the dimension of the feature map to half. Then add the two obtained feature maps and perform a ReLU operation. Then use a 1×1 convolution to reduce the dimension to 1, perform a sigmoid operation, and finally multiply it with x to assign the attention weight to the original feature map. The result of Attention is then concatenated with the result of upsampling and a 1×1 convolution is used to reduce the dimension to the same as the original. In this way, the attention mechanism is implemented for the extracted features. S24: The features are sent to three branches respectively, and continuous convolution is performed using 3×3 and 1×1 convolution kernels. Finally, a heat map with a channel number of 1 and a center point coordinate offset with a dimension of 2 is obtained. For the coordinate offset of the four corners relative to the center point, continuous convolution is performed using 3×3 and 7×7 convolution kernels to obtain the coordinate offset of the four corners relative to the center point with a dimension of 8. The heat map also requires sigmoid to facilitate the calculation of loss and post-processing; S25: Generate a Gaussian heat map of the ground truth based on the real key point coordinates of each spine image. First, find the bounding box based on the real key point coordinates, and calculate the radius R of the Gaussian circle based on the size of the bounding box of each vertebra. Finally, fill the Gaussian function calculation value on the heat map with the center point of the bounding box as the center and the radius R. Take the IOU critical value as overlap. The size of the bounding box is h×w, then the formula for finding R is: Among them, the values ​​of a, b, and c are: Take the coordinates of the center point of the bounding box as (p x ,p y ), then the Gaussian function is: Among them, the value of σ is For heat maps, the improved focal loss is used as the loss function, and the specific calculation method is: Where N represents the number of key points in the image, Y xy represents the true value, Represents the predicted value. For the offset of the center point coordinates and the offset of the four corners relative to the center point, L1 loss is used as the loss function. The total loss function is the sum of the calculation results of the three loss functions. S26: After training, the first network model can be obtained; S3: Using the spinal image as input and the name of each vertebra as a label to train the spinal recognition network to obtain a second network model; the construction of the spinal recognition network model specifically includes: S31: expanding the input spine image into a square; S32: Perform online data augmentation on the images in the training set, specifically random stretching, random brightness and contrast transformation, random horizontal mirror flipping, and random rotation; S33: Build the ResNeXt50 model, load the weight parameters of the ResNeXt50 pre-trained on ImageNet into the model, and use transfer learning to freeze all the parameters of the first three layers and only train the parameters after the fourth layer of the model; S34: Add the CSRA attention module at the end of the network. The CSRA attention module has two branches, namely Spatial pooling and Average pooling, and then perform weighted fusion of the two: F i =G i +λ*M i (5) Among them, F i represents the probability of the i-th category, G i represents the probability of the i-th category after average pooling, M i Represents the probability of the i-th category after spatial pooling; S35: Since spine recognition can be regarded as a multi-label classification problem, a sigmoid activation function is used for each output neuron to map the prediction results to the interval (0,1); S36: In view of the imbalance of categories in the data set, focal loss is used as the loss function; S37: After training, the second network model can be obtained; S4: Based on step S2, in order to obtain more precise key point coordinates, a single vertebrae image is cropped, and the single vertebrae image is used as input. The key point coordinates of the vertebrae are used as labels to train a single vertebrae key point detection network to obtain a third network model. S5: Based on step S2, the spine image is input into the trained first network model, and after post-processing, the coordinates of key points in the image are predicted; S6: Based on step S4 and step S5, using the key point coordinates of each vertebra predicted in step S5, a single vertebra is cut out, and the image of the single vertebra is input into the trained third network model. After post-processing, more accurate key point coordinates of the single vertebra are predicted, and then these key point coordinates need to be transformed back to the original image; S7: Based on step S3, the spinal image is input into the trained second network model to predict the probability of existence of each vertebra in the image; S8: Based on step S6 and step S7, in order to solve the problem that the number of vertebrae of a few people is arranged differently, a vertebra connection diagram is constructed, and the most accurate name of each vertebra in the image is obtained by finding the longest path method; S9: Based on step S6 and step S8, draw the key points of each predicted vertebra in the spinal image and mark the name of each predicted vertebra.

2. The method for detecting and identifying key points of an X-ray spine image according to claim 1, characterized in that: In step S1, the key points of each vertebra are marked. Regardless of whether the data set uses an X-ray of the entire spine in the anteroposterior or lateral view, the four corners of each vertebra are selected as key points, and their coordinates x, y are recorded.

3. The method for detecting and identifying key points of an X-ray spine image as claimed in claim 1, characterized in that: The data set expansion in step S1 includes: starting from 1, enumerating the number of vertebrae in the image as k, and then letting each vertebra in the image be the first vertebra in the image as v n , the second vertebra is v n+1 , extract the content containing {v n ,v n+1 ,…,v n+k-1 }, the interception method is to find the minimum and maximum x-coordinate values ​​and y-coordinate values ​​according to the coordinates of the key points of all vertebrae, and then expand these values ​​to a certain distance in all directions to obtain an image that only contains part of the vertebrae. Accordingly, the key points also need to be adjusted.

4. The method for detecting and identifying key points of an X-ray spine image according to claim 1, characterized in that: The method for constructing the single vertebra key point detection network described in step S4 is the same as that in step S2, except that the image input to the network is different, that is, a single vertebra image is used as input, and the third network model can be obtained after training.

5. The method for detecting and identifying key points of X-ray spine images as claimed in claim 1, characterized in that: The post-processing process described in step S5 includes: finding the maximum point for the heat map, then extracting the confidence score, index and center point coordinates of the center point with a confidence greater than a threshold, and then using the predicted center point coordinate offset and the four corners relative to the center point coordinate offset to obtain the coordinates of the four key points; for the post-processing process of a single vertebra, no threshold is required, and the center point with the maximum confidence can be directly obtained.

6. The method for detecting and identifying key points of X-ray spine images as claimed in claim 1, characterized in that: The construction of the vertebra connection graph described in step S8 includes: taking the name of each vertebra as a node in the graph, connecting adjacent vertebrae according to actual conditions, and taking the probability of each vertebra as the weight of each edge.

Images