Method for automatically measuring image indexes after hip replacement

Abstract

The invention discloses an automatic measurement method for image indexes after hip replacement, and belongs to the technical field of medical image processing. Marking to obtain a segmentation data set and a key point detection data set, and performing data enhancement and data preprocessing on the image data; processing the segmented data set by using the segmentation model to obtain a region of interest; detecting key points in the image of the region of interest by using the key point detection model in combination with a Gaussian thermodynamic diagram regression method; and finally, calculating four parameters of the patient after hip replacement according to the positions of the key points so as to accurately evaluate the condition of the patient after hip replacement. According to the invention, some important indexes are rapidly, accurately and repeatedly and automatically measured by using a deep learning measurement method, the problem of pain points of manual labeling is solved, manpower and time are saved, and the method has excellent reliability and accuracy and has application value for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for automatically measuring image indicators after hip replacement surgery. Background Art

[0002] The hip joint is a complex ball-and-socket joint, one of the heaviest-loaded joints in the body. Due to movement and biomechanics, the hip joint is easily damaged, and severe damage requires hip replacement surgery. After more than 30 years of clinical testing, artificial hip replacement has become an effective means of treating end-stage hip degenerative diseases, but complications such as prosthesis loosening and dislocation still occur from time to time.

[0003] After hip replacement surgery, doctors evaluate the surgical results based on X-ray images and measure imaging parameters to develop a personalized rehabilitation plan. They also provide timely intervention and treatment for possible complications, reducing the probability of a second surgery and alleviating pain and financial burden.

[0004] At present, the main postoperative evaluation method is manual measurement through various tools. The process of measuring image anatomical parameters is complicated, highly professional, time-consuming and laborious. Therefore, it is urgent to develop a fast, accurate and highly repeatable deep learning measurement method to automatically measure some important indicators. To this end, an automatic measurement method for imaging indicators after hip replacement is proposed. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to solve the problem of low accuracy, time-consuming and labor-intensive manual measurement in the prior art, and provide an automatic measurement method for imaging indicators after hip replacement.

[0006] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:

[0007] S1: Obtain multiple X-ray images after hip replacement and perform preprocessing to form an original data set;

[0008] S2: Annotate the image data in the original data set to obtain a segmentation data set and a key point detection data set, and perform data enhancement and data preprocessing on the image data in the segmentation data set and the key point detection data set;

[0009] S3: using the segmentation model to process the segmentation data set to obtain the region of interest, and obtain an image of the region of interest;

[0010] S4: Detect key points in the image of the region of interest using the key point detection model combined with the Gaussian heat map regression method, output a Gaussian heat map, and then obtain the position of each key point;

[0011] S5: Finally, the four parameters of acetabular abduction angle, leg length difference, vertical distance of rotation center and horizontal distance of rotation center after hip replacement surgery are calculated according to the positions of key points to accurately evaluate the condition of patients after hip replacement surgery.

[0012] Furthermore, in step S1, after obtaining the X-ray image after hip replacement surgery, all RGB images and DICOM images are converted into single-layer grayscale images in JPG format to achieve preprocessing of the X-ray image.

[0013] Furthermore, in step S2, the specific processing process is as follows:

[0014] S21: Create segmentation dataset

[0015] Use Labelme software to mark the foreground and background areas on the X-ray image, and convert the marked image into VOC format to obtain a segmentation dataset for training the segmentation model;

[0016] S22: Create a key point detection dataset

[0017] Labelme software was used to mark 8 key points on the X-ray image, and then a key point detection data set was obtained for training the key point detection model. The 8 key points were: point A: upper edge of the acetabulum on the surgical side, point O: rotation center of the hip joint on the surgical side, point B: lower edge of the acetabulum on the surgical side, point C: lower edge of the teardrop on the surgical side, point D: tip of the lesser trochanter on the surgical side, point E: rotation center of the hip joint on the non-surgical side, point F: lower edge of the teardrop on the non-surgical side, point G: tip of the lesser trochanter on the non-surgical side;

[0018] S23: Data enhancement and data preprocessing

[0019] The image data in the segmentation dataset and the key point detection dataset are subjected to data enhancement and data preprocessing. The data enhancement includes random resizing, cropping, horizontal flipping and vertical flipping operations on the segmentation training dataset, and random key point hiding, blurring, adjusting brightness, key point translation, rotation and cropping operations on the key point detection training dataset; the data preprocessing includes normalization, compression, padding and histogram equalization operations on all the data of the two datasets; among them, the compression uses a fixed aspect ratio method, and the compressed resolution is 512*512.

[0020] Furthermore, in step S3, the specific processing process is as follows:

[0021] S31: Use the segmentation model to segment the compressed X-ray image to obtain a mask;

[0022] S32: Fusing the output mask with the original image to obtain a fused image;

[0023] S33: Use a Gaussian filter to filter out the noise of the fused image to obtain an image of the region of interest.

[0024] Furthermore, in the step S31, the segmentation model is trained based on the BIU-Net network, and the BIU-Net network is improved based on the U-Net network, and the improvement content is specifically as follows: during the convolution operation, a padding 0 area is added to keep the height and width of the feature map output of the convolution unchanged, so that the output of the model is consistent with the size of the input image; in the upsampling process, bilinear interpolation is used instead of transposed convolution to double the size of the feature map, and in each group of two convolutions, the second convolution reduces the number of channels of the feature map to half, so that the feature layer after bilinear sampling has the same number of channels as the feature layer of the jump connection; for the bottom of the U-Net, in the upsampling module, the feature map of the feature layer after upsampling is padded so that the sizes of the two feature layers are matched when spliced;

[0025] The loss function during BIU-Net network training is as follows:

[0026] L=L CE +L Dice

[0027] Among them, L CE is the cross entropy loss function, L Dice is the Dice loss function, and the calculation formulas are as follows:

[0028]

[0029]

[0030] Among them, y i is the true label of the i-th sample, p i is the probability that the i-th sample is predicted to be positive, N is the number of samples, and ∈ is a small positive number.

[0031] Furthermore, in the step S4, an unbiased heat map encoding method and a DARK heat map decoding method are used to improve the key point positioning accuracy; the key point detection model is obtained based on the RCDA-ResU-Net network training, and the RCDA-ResU-Net network is improved based on the U-Net network, and the improvement content is specifically: an RFMs module is used to replace the two ordinary convolutions in the U-Net network to extract the local detail features and larger area features of the image under multiple receptive fields; in the Down series modules, the DM module, the DDSRM1 module and the DDSRM2 module, the DDSRM1 module and the DDSR The M2 module uses deep separable convolution, channel shuffle and residual structure to learn feature information, and also adds the ECA attention mechanism to focus on the channel information in the feature map; the DenseASPP module is used at the bottom of the network, which uses dilated convolutions at different dilation rates to densely connect to obtain a larger receptive field and denser sampling points, thereby improving the model's expressiveness; in the Up series modules, the UDSRM1 and UDSRM2 modules similar to those in the Down series modules are used, and also use deep separable convolution, channel shuffle and residual structure to learn feature information, and also add the CA attention mechanism to focus on the spatial information in the feature map;

[0032] The loss function during RCDA-ResU-Net network training is as follows:

[0033]

[0034] Among them, W is a hyperparameter representing the weight coefficient, a and c are hyperparameters that control the speed and position of the function curve descent, MSE Loss is the mean square error loss, L1 Loss is the absolute value loss, M is the weight mask, and the calculation formulas are as follows:

[0035]

[0036] Among them, target is the predicted value and threshold is the threshold;

[0037]

[0038] Where N is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample.

[0039] Furthermore, in step S5, when calculating the acetabular abduction angle, the key points involved include points A, B, C, and F, and the calculation formula is as follows:

[0040]

[0041] Among them, k1 is the slope of the straight line AB, k2 is the slope of the straight line CF, the angle between the straight line AB and the straight line CF is θ, that is, the acetabular abduction angle, and the angle θ in the normal state after surgery ranges from 30 degrees to 50 degrees.

[0042] Furthermore, in step S5, when calculating the leg length difference, the key points involved include points C, F, D, and G. Points C and F form a straight line CF. The difference in distance from points D and G to the straight line CF is calculated as the leg length difference. Under normal conditions after surgery, the range of the leg length difference is within 10 mm.

[0043] Furthermore, in step S5, when calculating the vertical distance of the rotation center, the key points involved include points C, F, O, and E. Points C and F form a straight line CF. The distances from points O and E to the straight line CF are calculated as the vertical distances of the rotation center. Under normal conditions after surgery, the vertical distances of the two rotation centers are the same.

[0044] Furthermore, in step S5, when calculating the horizontal distance of the rotation center, the key points involved include points C, F, O, and E. Two perpendicular lines to the straight line CF are drawn through point C and point F respectively. The distance from point O to the perpendicular line through point C and the distance from point E to the perpendicular line through point F are the horizontal distances of the rotation center. Under normal conditions after surgery, the horizontal distances of the rotation centers on both sides are the same.

[0045] Compared with the prior art, the present invention has the following advantages: the method for automatically measuring imaging indicators after hip replacement surgery uses a deep learning measurement method to automatically measure some important indicators quickly, accurately, and with high repeatability, solving the pain points of manual labeling, saving manpower and time, and has excellent reliability and accuracy, and has application value for clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a schematic diagram of annotating and segmenting a data set in an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram of a key point detection data set labeled in an embodiment of the present invention;

[0048] Figure 3a is an original X-ray image in an embodiment of the present invention;

[0049] Figure 3b is an image with a resolution of 512*512 after compression in an embodiment of the present invention;

[0050] Figure 4 is a schematic diagram of the segmentation and post-processing process of an X-ray image in an embodiment of the present invention;

[0051] Figure 5 Schematic diagram of the BIU-Net network structure in an embodiment of the present invention;

[0052] Figure 6 Schematic diagram of the RCDA-ResU-Net network structure in an embodiment of the present invention;

[0053] Figure 7 is a schematic diagram of the RFMs structure in an embodiment of the present invention;

[0054] Figure 8 is a schematic diagram of the structure of a Down module in an embodiment of the present invention;

[0055] Fig. 9 Schematic diagram of the structure of Downsampling Module (DM) in an embodiment of the present invention;

[0056] Fig.10a 1 is a schematic diagram of the structure of the Down Depth-Separable Residual Module 1 (DDSRM1) module in the Down module in an embodiment of the present invention;

[0057] Fig.10b 2 is a schematic diagram of the structure of a Down Depth-Separable Residual Module 2 (DDSRM2) module in a Down module in an embodiment of the present invention;

[0058] Fig.11 Schematic diagram of the ECA attention mechanism in an embodiment of the present invention;

[0059] Fig.12 Schematic diagram of the DenseASPP structure in an embodiment of the present invention;

[0060] Fig.13 is a schematic diagram of an Up module in an embodiment of the present invention;

[0061] Fig.14a 1 is a schematic diagram of the structure of the Up Depth-Separable Residual Module 1 (UDSRM1) module in the Up module in an embodiment of the present invention;

[0062] Fig.14b 2 is a schematic diagram of the structure of the Up Depth-Separable Residual Module 2 (UDSRM2) module in the Up module in an embodiment of the present invention;

[0063] Fig.15 Schematic diagram of the CA attention mechanism in an embodiment of the present invention;

[0064] Fig.16 is a schematic diagram of calculating the acetabulum abduction angle in an embodiment of the present invention;

[0065] Fig.17 is a schematic diagram of calculating the difference in leg length in an embodiment of the present invention;

[0066] Fig.18 is a schematic diagram of calculating the vertical distance of the rotation center in an embodiment of the present invention;

[0067] Fig.19 is a schematic diagram of calculating the vertical distance of the rotation center in an embodiment of the present invention;

[0068] Fig. 20 It is a flow chart of the method for automatically measuring imaging indicators after hip replacement surgery of the present invention. DETAILED DESCRIPTION

[0069] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0070] This embodiment provides a technical solution: a method for automatically measuring imaging indicators after hip replacement surgery, including the following main steps:

[0071] Step 1: Obtain multiple X-ray images after hip replacement and preprocess them to form an original data set;

[0072] Step 2: Annotate the image data in the original data set to obtain a segmentation data set and a key point detection data set, and perform data enhancement and data preprocessing on the image data in the segmentation data set and the key point detection data set;

[0073] Step 3: Use the segmentation model to process the segmentation data set to obtain the region of interest and obtain the region of interest image;

[0074] Step 4: Use the key point detection model combined with the Gaussian heat map regression method to detect the key points in the image of the region of interest, output the Gaussian heat map, and then obtain the position of each key point;

[0075] Step 5: Finally, the four parameters of the patient's abduction angle, leg length difference, vertical distance of the rotation center, and horizontal distance of the rotation center after hip replacement are calculated based on the key position points to accurately evaluate the patient's condition after hip replacement surgery.

[0076] The following is a further description of the detailed process of the above steps:

[0077] Step 1: Get the original dataset

[0078] After obtaining the X-ray images after hip replacement, all RGB images and DICOM images were converted into single-layer grayscale images in JPG format to reduce the computational cost, overfitting risk and redundant information.

[0079] Step 2: Make a dataset

[0080] a. Make a segmentation dataset

[0081] An X-ray image after hip replacement surgery contains a lot of information. The goal of this invention is to find the key points on the bones. The image information of the human tissue part needs to be retained, and the rest of the useless information can be cropped and turned into a background with a brightness of 0, thus reducing the adverse effects of useless information on the accuracy of key point detection. Figure 1 As shown, Labelme software is used for annotation. The area outside the red polygon is the background, the area inside is the foreground, and the area inside the green polygon is the background. The annotated image is converted into VOC format. This dataset is used for subsequent BIU-Net network training to detect the region of interest.

[0082] b. Create a key point detection dataset

[0083] like Figure 2 As shown, the Labelme software is also used to mark the eight required key points using point marking, namely A: upper edge of the acetabulum on the surgical side, O: rotation center of the hip joint on the surgical side, B: lower edge of the acetabulum on the surgical side, C: lower edge of the teardrop on the surgical side, D: tip of the lesser trochanter on the surgical side, E: rotation center of the hip joint on the non-surgical side, F: lower edge of the teardrop on the non-surgical side, G: tip of the lesser trochanter on the non-surgical side. This dataset is used for subsequent RCDA-ResU-Net network training to detect the required key points.

[0084] c. Data enhancement and data preprocessing

[0085] Data enhancement includes random resizing, cropping, horizontal flipping, and vertical flipping of the segmentation training data set, and random key point hiding, blurring, adjusting brightness, key point translation, rotation, and cropping of the key point detection training data set; data preprocessing includes normalizing, compressing, padding, and histogram equalization of all data in the two data sets. Appropriate data enhancement can improve the generalization ability of the model, prevent model overfitting, reduce the model's demand for large-scale data sets, handle unbalanced data problems, and improve robustness; appropriate data preprocessing can accelerate convergence, improve model performance, reduce model complexity, unify data scale, and improve data quality. The compression uses a fixed aspect ratio method, which compresses the resolution of X-ray images to 512*512, such as Figure 3a , Figure 3b As shown, on the left ( Figure 3a ) is the original image, and the right ( Figure 3b) is an image with a compressed resolution of 512*512, and the key point coordinates are also compressed synchronously. While retaining more image features, it has a positive effect on model learning features, improving training accuracy, reducing training time, and lowering computer performance requirements.

[0086] Step 3: Get the region of interest through image segmentation

[0087] like Figure 4 As shown in the figure, based on the segmentation dataset produced in the previous step, the improved U-Net network, BIU-Net network, is used to segment the postoperative X-ray image to obtain a mask, and the loss function (CELoss+DiceLoss) is redesigned to train the BIU-Net network, achieving high-precision segmentation, and then the output mask is fused with the original image, so that the pixels of interference information are set to 0, retaining a high-quality foreground image. Finally, a Gaussian filter is used to filter out the noise of the fused image, and the final output X-ray image is used as the dataset (region of interest dataset) for the subsequent key point detection model.

[0088] like Figure 5 As shown, the BIU-Net network is introduced below:

[0089] First, the model adds padding 0 areas during convolution operations to keep the height and width of the feature map output by the convolution unchanged, so that the output of the model is consistent with the size of the input image, both of which are (512, 512). At the same time, a BN layer (BatchNormalization) is added between the convolution and activation functions to speed up the convergence speed when training the model and prevent gradient vanishing and gradient explosion. Second, during the upsampling process, bilinear interpolation is used instead of transposed convolution to double the size of the feature map. In each group of two convolutions, the second convolution reduces the number of channels of the feature map to half. The purpose of this is to ensure that the feature layer after bilinear sampling can have the same number of channels as the feature layer of the jump connection. Third, for the bottom of U-Net, in the upsampling module, the feature map of the feature layer after upsampling needs to be padded to ensure that the sizes of the two feature layers match when splicing. This is because if the input image size is not an integer multiple of 16, the image has undergone 4 downsamplings, and the maxpooling layer will round down. In this way, when bilinear interpolation is used for upsampling, the size will be inconsistent when stitching. At this time, the difference between the two is calculated and the upsampled image is padded to the appropriate size.

[0090] The model optimizer is Adam, the initial learning rate is 0.01, and a custom learning rate update strategy is used, with a total of 300 epochs. 489 X-ray images after hip replacement, of which 394 are training sets, 50 are validation sets, and 45 are test sets, with a ratio of approximately 8:1:1.

[0091] The loss function during BIU-Net network training is as follows:

[0092] L=L CE +L Dice

[0093] Among them, L CE is the cross entropy loss function, L Dice is the Dice loss function, and the calculation formulas are as follows:

[0094]

[0095] Among them, y i is the true label of the i-th sample, p i is the probability that the i-th sample is predicted to be positive, N is the number of samples, and ∈ is a small positive number.

[0096] Step 4: Use Gaussian heat map regression method to detect key points

[0097] Based on the region of interest dataset obtained in the previous step, the unbiased heat map encoding method and the DARK heat map decoding method are used to improve the accuracy of key point positioning, and an improved U-Net network, the RCDA-ResU-Net model, is designed to detect key points in X-ray images after hip replacement surgery (such as Figure 6 As shown), we also designed the WeightedShrinkageLoss (WS Loss) loss function. The network structure is shown in Figure 6 As shown in the figure, more attention is paid to small area targets, and multi-scale dilated convolution, depth-separable convolution, channel shuffle, residual structure, DenseASPP and improved attention mechanism are used.

[0098] The traditional Gaussian heat map encoding and decoding process introduces quantization errors, which leads to a decrease in the accuracy of key point detection and the heat map predicted by the model does not meet the standard Gaussian distribution, resulting in inaccurate Argmax function prediction. Since the accuracy of key point positioning in X-ray images will directly affect the accuracy of measuring parameters, unbiased heat map encoding and DARK heat map decoding are used to improve the accuracy of key point positioning.

[0099] The RCDA-ResU-Net model consists of three parts: encoder, decoder and skip connection.

[0100] The encoder includes ReceptivefieldModules (RFMs), Down1, Down2, Down3, and DenseASPP (Dense Atrous Spatial Pyramid Pooling Module), which uses a layer-by-layer downsampling and convolutional layer method similar to the original U-Net model to extract image features, doubling the number of channels, halving the resolution, and reducing the amount of model calculation. Using an RFMs module (such as Figure 7 ) replaces the two ordinary convolutions in the U-Net network to extract local detail features and larger regional features of the image under multiple receptive fields. In order to reduce the amount of calculation and parameters of the model, in the Down (downsampling) series modules (such as Figure 8 As shown), the DM module (as Fig. 9 As shown), and the DDSRM1 module and DDSRM2 module (as shown Fig.10a , Fig.10b As shown in ), these modules use deep separable convolution, channel shuffle and residual structure to learn feature information, and also add ECA attention mechanism (as shown in Fig.11 ). The DM structure can accurately grasp the fine features of the image, while keeping the number of channels unchanged and halving the resolution. Compared with traditional convolution, depthwise separable convolution makes the model more concise and less prone to overfitting data. Channel shuffling can enhance cross-channel communication of features and extract richer and more effective feature information, thereby making the model perform better when the number of hip joint X-ray image datasets is small, achieving faster inference speed and reducing the model's requirements for the running hardware environment. The ECA attention mechanism focuses on the relationship between different channels, assigns weights to each channel, enhances important channels, and suppresses unimportant channels.

[0101] like Fig.12 As shown in the figure, the DenseASPP module uses dilated convolutions with different dilation rates to obtain a larger receptive field and denser sampling points, so that the deep feature map can be better integrated with the shallow feature map later, so that the feature map output by the previous encoder submodules can be fully learned at the "bottom" of the RCDA-ResU-Net model, and the features at different scales can be integrated to improve the model's expressiveness. In addition, the feature map size of the "bottom" is small, so using DenseASPP does not bring too much computation to the model.

[0102] like Fig.13 As shown, after the feature map is upsampled in the Up (up-sampling) series module, it is stacked with the shallow feature map transmitted by the jump connection, and the UDSRM1 module and UDSRM2 module similar to those in the Down series are used. The feature information is also learned using deep separable convolution, channel shuffling and residual structure (as shown in Fig.14a , 14bAs shown), a CA attention mechanism is also added (as shown Fig.15 As shown in the figure, we focus on the spatial information in the feature map and repeat the above operation many times until the last decoder submodule uses 3×3 convolution to adjust the number of channels and outputs a Gaussian heat map of (8, 512, 512).

[0103] The loss function of the RCDA-ResU-Net model is as follows:

[0104]

[0105] WeightedShrinkageLoss gives greater weight to foreground pixels in the Gaussian heat map, so that the model pays more attention to foreground pixels during training, improving training efficiency and key point detection accuracy. Among them, W is a hyperparameter representing the weight coefficient, a and c are hyperparameters that control the speed and position of the function curve descent, MSE Loss is the mean square error loss, L1Loss is the absolute value loss, M is the weight mask, and the calculation formulas are as follows:

[0106]

[0107] Among them, target is the predicted value and threshold is the threshold;

[0108]

[0109] Where N is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample.

[0110] The initial learning rate of the model is set to 0.001, the optimizer is Adam, the loss function is WeightedShrinkageLoss, and the epoch is 50. The X-ray image dataset after hip replacement surgery has a total of 489 images, of which 394 are training sets, 50 are validation sets, and 45 are test sets, with a ratio of approximately 8:1:1.

[0111] Step 5: Measurement of imaging parameters after hip replacement

[0112] like Fig.16 As shown, the acetabulum abduction angle: the relevant key points include A, B, C, and F. AB forms a straight line, and CF forms another horizontal reference line. The angle between the two straight lines is the abduction angle, such as Fig.16 As shown, the tangent formula can be used, that is, when the slopes of AB and CF exist, set them as k1 and k2, and the angle between the two lines is θ, then Under normal conditions, the angle ranges from 30 degrees to 50 degrees.

[0113] like Fig.17 As shown, leg length difference: the relevant key points include C, F, D, and G. A straight line is formed by CF. The difference in distance from D and G to this straight line is the leg length difference. Assuming that the coordinates of a key point are (x0, y0), the coordinates of point C are (x1, y1), and the coordinates of point F are (x2, y2), and the equation of the horizontal reference line CF is ax+by+c=0, then the distance from the key point to the straight line CF is: Among them, a = (y2-y1), b = (x1-x2), c = (x2y1-x1y2). It is common for the leg length difference after surgery to be greater than 0, and the tolerable range is within 10mm.

[0114] like Fig.18 As shown, the vertical distance of the rotation center: the relevant key points include C, F, O, and E. A straight line is formed by CF. The distances from O and E to this straight line are the vertical distances of the rotation center. The distance calculation method is as above. Ideally, the vertical distances on both sides should be the same.

[0115] like Fig.19 As shown, the horizontal distance of the rotation center: the relevant key points include C, F, O, and E. Two perpendicular lines CF are drawn through points C and F respectively. The distance from point O to the perpendicular line through point C and the distance from point E to the perpendicular line through point F are the horizontal distances of the rotation center. Therefore, the equation of the perpendicular line at a point on the horizontal reference line needs to be calculated first. Assuming that the slope of the horizontal reference line is k, the slope of the perpendicular line is Passing a point on the horizontal line (x v ,y v )’s perpendicular line equation: k v x-y+y v -k v x v = 0. According to Fig.19 Using the coordinates of points O, E, C, and F, and the equation of the perpendicular line and the distance formula from the point to the line, the horizontal distance predicted by the model can be calculated. Ideally, the horizontal distance on both sides should be the same.

[0116] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for automatically measuring imaging indicators after hip replacement surgery, characterized in that: The following steps are involved: S1: Obtain multiple X-ray images after hip replacement and perform preprocessing to form an original data set; S2: Annotate the image data in the original data set to obtain a segmentation data set and a key point detection data set, and perform data enhancement and data preprocessing on the image data in the segmentation data set and the key point detection data set; S3: using the segmentation model to process the segmentation data set to obtain the region of interest, and obtain an image of the region of interest; S4: Detect key points in the image of the region of interest using the key point detection model combined with the Gaussian heat map regression method, output a Gaussian heat map, and then obtain the position of each key point; S5: Finally, the four parameters of acetabular abduction angle, leg length difference, vertical distance of rotation center and horizontal distance of rotation center after hip replacement surgery are calculated according to the positions of key points to accurately evaluate the condition of patients after hip replacement surgery.

2. The method for automatically measuring imaging indicators after hip replacement according to claim 1, characterized in that: In step S1, after obtaining the X-ray image after hip replacement surgery, all RGB images and DICOM images are converted into single-layer grayscale images in JPG format to achieve preprocessing of the X-ray image.

3. The method for automatically measuring imaging indicators after hip replacement according to claim 2, characterized in that: In step S2, the specific processing process is as follows: S21: Create segmentation dataset Use Labelme software to mark the foreground and background areas on the X-ray image, and convert the marked image into VOC format to obtain a segmentation dataset for training the segmentation model; S22: Create a key point detection dataset Labelme software was used to mark 8 key points on the X-ray image, and then a key point detection data set was obtained for training the key point detection model. The 8 key points were: point A: upper edge of the acetabulum on the surgical side, point O: rotation center of the hip joint on the surgical side, point B: lower edge of the acetabulum on the surgical side, point C: lower edge of the teardrop on the surgical side, point D: tip of the lesser trochanter on the surgical side, point E: rotation center of the hip joint on the non-surgical side, point F: lower edge of the teardrop on the non-surgical side, point G: tip of the lesser trochanter on the non-surgical side; S23: Data enhancement and data preprocessing Perform data enhancement and data preprocessing on the image data in the segmentation dataset and key point detection dataset. Data enhancement includes random resizing, cropping, horizontal flipping and vertical flipping operations on the segmentation training dataset, and random key point hiding, blurring, adjusting brightness, key point translation, rotation and cropping operations on the key point detection training dataset. Data preprocessing includes normalization, compression, padding and histogram equalization of all data in the two data sets; compression uses a fixed aspect ratio method, and the compressed resolution is 512*512.

4. The method for automatically measuring imaging indicators after hip replacement according to claim 3, characterized in that: In step S3, the specific processing process is as follows: S31: Use the segmentation model to segment the compressed X-ray image to obtain a mask; S32: Fusing the output mask with the original image to obtain a fused image; S33: Use a Gaussian filter to filter out the noise of the fused image to obtain an image of the region of interest.

5. The method for automatically measuring imaging indicators after hip replacement according to claim 4, characterized in that: In the step S31, the segmentation model is trained based on the BIU-Net network, and the BIU-Net network is improved based on the U-Net network. The improvement content is specifically as follows: during the convolution operation, a padding 0 area is added to keep the height and width of the feature map output by the convolution unchanged, so that the output of the model is consistent with the size of the input image; in the upsampling process, bilinear interpolation is used instead of transposed convolution to double the size of the feature map. In each group of two convolutions, the second convolution reduces the number of channels of the feature map to half, so that the feature layer after bilinear sampling has the same number of channels as the feature layer of the jump connection; for the bottom of the U-Net, in the upsampling module, the feature map of the feature layer after upsampling is padded so that the sizes of the two feature layers are matched when spliced; The loss function during BIU-Net network training is as follows: L=L CE +L Dice Among them, L CE is the cross entropy loss function, L Dice is the Dice loss function, and the calculation formulas are as follows: Among them, y i is the true label of the i-th sample, p i is the probability that the i-th sample is predicted to be positive, N is the number of samples, and ∈ is a small positive number.

6. The method for automatically measuring imaging indicators after hip replacement according to claim 5, characterized in that: In the step S4, the unbiased heat map encoding method and the DARK heat map decoding method are used to improve the key point positioning accuracy; the key point detection model is obtained based on the RCDA-ResU-Net network training, and the RCDA-ResU-Net network is improved based on the U-Net network. The improvement content is as follows: an RFMs module is used to replace the two ordinary convolutions in the U-Net network to extract the local detail features and larger area features of the image under multiple receptive fields; in the Down series modules, the DM module, the DDSRM1 module and the DDSRM2 module are used, and the DDSRM1 module and the DDSRM2 module are used. The block uses deep separable convolution, channel shuffle and residual structure to learn feature information, and also adds the ECA attention mechanism to focus on the channel information in the feature map; the DenseASPP module is used at the bottom of the network, which uses dilated convolutions with different dilation rates to densely connect to obtain a larger receptive field and denser sampling points, thereby improving the expressiveness of the model; in the Up series modules, the UDSRM1 and UDSRM2 modules similar to those in the Down series modules are used, and also use deep separable convolution, channel shuffle and residual structure to learn feature information, and also add the CA attention mechanism to focus on the spatial information in the feature map; The loss function during RCDA-ResU-Net network training is as follows: Among them, W is a hyperparameter representing the weight coefficient, a and c are hyperparameters that control the speed and position of the function curve descent, MSE Loss is the mean square error loss, L1 Loss is the absolute value loss, M is the weight mask, and the calculation formulas are as follows: Among them, target is the predicted value and threshold is the threshold; Where N is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample.

7. The method for automatically measuring imaging indicators after hip replacement according to claim 6, characterized in that: In step S5, when calculating the acetabulum abduction angle, the key points involved include points A, B, C, and F, and the calculation formula is as follows: Among them, k1 is the slope of the straight line AB, k2 is the slope of the straight line CF, the angle between the straight line AB and the straight line CF is θ, that is, the acetabular abduction angle, and the angle θ in the normal state after surgery ranges from 30 degrees to 50 degrees.

8. The method for automatically measuring imaging indicators after hip replacement according to claim 7, characterized in that: In step S5, when calculating the leg length difference, the key points involved include C, F, D, and G. A straight line CF is formed by points C and F. The difference in distance from points D and G to the straight line CF is calculated as the leg length difference. Under normal conditions after surgery, the range of the leg length difference is within 10 mm.

9. The method for automatically measuring imaging indicators after hip replacement according to claim 8, characterized in that: In step S5, when calculating the vertical distance of the rotation center, the key points involved include points C, F, O, and E. Points C and F form a straight line CF. The distances from points O and E to the straight line CF are calculated as the vertical distances of the rotation center. Under normal conditions after surgery, the vertical distances of the two rotation centers are the same.

10. The method for automatically measuring imaging indicators after hip replacement according to claim 9, characterized in that: In step S5, when calculating the horizontal distance of the rotation center, the key points involved include points C, F, O, and E. Two perpendicular lines to the straight line CF are drawn through point C and point F respectively. The distance from point O to the perpendicular line through point C and the distance from point E to the perpendicular line through point F are the horizontal distances of the rotation center. Under normal conditions after surgery, the horizontal distances of the rotation centers on both sides are the same.

Images