Medical image processing method and device, computer device and storage medium

Abstract

The application relates to a medical image processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a to-be-recognized image corresponding to an initial image; the to-be-recognized image is obtained by superimposing a first image and a second image; the first image is an image on one side of a first vertical center axis of the initial image, which is determined according to a hip joint topological constraint condition; the second image is an image obtained by flipping an image on the other side of the first vertical center axis; inputting the to-be-recognized image into a pre-trained key point recognition model to obtain a key point probability graph corresponding to each target key point; the pre-trained key point recognition model is obtained by training according to the hip joint topological constraint condition; and determining a predicted coordinate of a corresponding target key point in the initial image according to each key point probability graph. The method can improve the detection accuracy of hip joint key points.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a medical image processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] As artificial intelligence technology matures, it is being applied in more and more medical imaging fields, such as lesion segmentation, registration of human tissue and organ images, and key point detection in medical images.

[0003] Medical image key point detection technology can be applied to the diagnosis of developmental dysplasia of the hip (DDH). DDH refers to various deformities that occur during hip joint development, including femoral head dysplasia, acetabular dysplasia, or hip dislocation.

[0004] Currently, in hip joint key point detection technology based on anteroposterior X-ray of the hip joint, DDH patients often have imaging manifestations such as uneven femoral head density, femoral head necrosis, and femoral head dislocation. These pathological changes increase the difficulty of key point detection, leading to inaccurate key point localization or even detection failure.

[0005] Therefore, the relevant technologies suffer from low accuracy in detecting key points of the hip joint. Summary of the Invention

[0006] Therefore, it is necessary to provide a medical image processing method, device, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of hip joint key point detection in response to the above-mentioned technical problems.

[0007] Firstly, this application provides a medical image processing method. The method includes:

[0008] Obtain the image to be identified corresponding to the initial image; the image to be identified is obtained by superimposing the first image and the second image; the first image is the image on one side of the first vertical central axis of the initial image determined according to the hip joint topological constraint conditions; the second image is the image obtained by flipping the image on the other side of the first vertical central axis.

[0009] The image to be identified is input into a pre-trained keypoint recognition model to obtain a keypoint probability map corresponding to each target keypoint; the pre-trained keypoint recognition model is trained according to the hip joint topological constraints.

[0010] The predicted coordinates of the corresponding target key points in the initial image are determined based on the probability maps of each key point.

[0011] In one embodiment, the method further includes:

[0012] Obtain a training sample set corresponding to the initial sample image; each training sample in the training sample set includes a sample image and a mask image corresponding to the sample image; the initial sample image is marked with the target coordinates corresponding to each target key point; the mask image is composed of binary mask rectangles corresponding to the target key points;

[0013] The sample image is input into the key point recognition model to be trained to obtain the prediction probability map corresponding to the sample image; the prediction probability map includes a background prediction probability map and a key point prediction probability map corresponding to each of the target key points;

[0014] The key point recognition model to be trained is trained based on the predicted probability map, the mask image, and the hip joint topological constraints to obtain the pre-trained key point recognition model.

[0015] In one embodiment, the sample image is obtained by superimposing a first sample image and a second sample image; the first sample image is an image on one side of the second vertical midline of the initial sample image determined according to the hip joint topological constraints; the second sample image is an image obtained by flipping the image on the other side of the second vertical midline; the target key points include a first key point set corresponding to the first sample image and a second key point set corresponding to the second sample image; the hip joint topological constraints include femoral head topological constraints and acetabular topological constraints; training the key point recognition model to be trained according to the predicted probability map, the mask image, and the hip joint topological constraints to obtain the pre-trained key point recognition model includes:

[0016] Based on the initial loss function, a first difference between the predicted probability map and the mask image is determined;

[0017] Based on the femoral head topological constraints, a second difference is determined between the first femoral head radius corresponding to the first set of key points and the second femoral head radius corresponding to the second set of key points.

[0018] Based on the acetabular topological constraints, a third difference is determined between the first acetabular radius corresponding to the first key point set and the second acetabular radius corresponding to the second key point set.

[0019] The key point recognition model to be trained is trained based on the first difference, the second difference, and the third difference to obtain the pre-trained key point recognition model.

[0020] In one embodiment, determining the second difference between the first femoral head radius corresponding to the first keypoint set and the second femoral head radius corresponding to the second keypoint set based on the femoral head topological constraints includes:

[0021] The first femoral head radius is obtained based on the predicted sample coordinates of the target key points in the first key point set corresponding to the initial sample image.

[0022] The second femoral head radius is obtained based on the predicted sample coordinates of the target key points in the second key point set corresponding to the initial sample image;

[0023] The length difference between the first femoral head radius and the second femoral head radius is determined as the second difference.

[0024] In one embodiment, determining the third difference between the first acetabular radius corresponding to the first keypoint set and the second acetabular radius corresponding to the second keypoint set, based on the acetabular topological constraints, includes:

[0025] Fit the first arc corresponding to the target key points in the first key point set in the initial sample image, and the second arc corresponding to the target key points in the second key point set in the initial sample image;

[0026] The radius of the first arc is determined as the radius of the first acetabulum, and the radius of the second arc is determined as the radius of the second acetabulum;

[0027] The length difference between the first acetabular radius and the second acetabular radius is determined as the third difference.

[0028] In one embodiment, the femoral head topological constraint condition is used to constrain the second difference to be less than a femoral head radius difference threshold; the acetabular topological constraint condition is used to constrain the third difference to be less than an acetabular radius difference threshold; the step of training the keypoint recognition model to be trained based on the first difference, the second difference, and the third difference to obtain the pre-trained keypoint recognition model includes:

[0029] Based on the first difference, the second difference, and the third difference, the model parameters of the key point recognition model to be trained are updated according to a preset parameter optimization algorithm.

[0030] The key point recognition model with updated model parameters is retrained. When the first difference is less than a preset difference threshold, the second difference is less than the femoral head radius difference threshold, and the third difference is less than the acetabular radius difference threshold, the trained key point recognition model is determined to meet the preset training termination condition, and the pre-trained key point recognition model is obtained.

[0031] In one embodiment, the target key points include a first target key point located in the first image and a second target key point located in the second image; the key point probability map includes a first key point probability map corresponding to each of the first target key points and a second key point probability map corresponding to each of the second target key points; determining the predicted coordinates of the corresponding target key points in the initial image based on each of the key point probability maps includes:

[0032] For any of the aforementioned key point probability maps, binarization processing is performed on the aforementioned key point probability map according to a preset probability threshold to obtain the corresponding binarized mask image;

[0033] The target connected component is obtained by determining the connected component with the largest area in the corresponding binary mask image.

[0034] For any of the first key point probability maps, determine the coordinates of the geometric center of the target connected component, which are used as the predicted coordinates of the first target key point corresponding to any of the first key point probability maps in the initial image;

[0035] For any second key point probability map, the coordinates of the geometric center of the target connected region are flipped along the first vertical central axis to obtain the predicted coordinates of the second target key point corresponding to any second key point probability map in the initial image.

[0036] Secondly, this application also provides a medical image processing apparatus. The apparatus includes:

[0037] The acquisition module is used to acquire the image to be identified corresponding to the initial image; the image to be identified is obtained by superimposing the first image and the second image; the first image is the image on one side of the first vertical central axis of the initial image determined according to the hip joint topological constraint conditions; the second image is the image obtained by flipping the image on the other side of the first vertical central axis.

[0038] The input module is used to input the image to be recognized into a pre-trained keypoint recognition model to obtain a keypoint probability map corresponding to each target keypoint; the pre-trained keypoint recognition model is trained according to the hip joint topological constraints.

[0039] The determination module is used to determine the predicted coordinates of the corresponding target key points in the initial image based on the probability maps of each key point.

[0040] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0041] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0042] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0043] The aforementioned medical image processing method, apparatus, computer equipment, storage medium, and computer program product acquire an image to be identified corresponding to an initial image; the image to be identified is obtained by superimposing a first image and a second image; the first image is an image on one side of the first vertical midline of the initial image determined according to the hip joint topological constraints; the second image is an image obtained by flipping the image on the other side of the first vertical midline; the image to be identified is input into a pre-trained keypoint recognition model to obtain a keypoint probability map corresponding to each target keypoint; the pre-trained keypoint recognition model is trained according to the hip joint topological constraints; the predicted coordinates of the corresponding target keypoints in the initial image are determined according to the keypoint probability maps; thus, the image to be identified can be... The first and second images in the image are input as a group into a pre-trained keypoint recognition model. The pre-trained keypoint recognition model is trained using hip joint topological constraints. The first image is the image on one side of the first vertical central axis of the initial image determined by the hip joint topological constraints, and the second image is the image obtained by flipping the image on the other side of the first vertical central axis. Thus, the robustness of the keypoint recognition model for locating target keypoints in the symmetrical initial image can be enhanced by the hip joint topological constraints. This can improve the accuracy of the keypoint probability map output by the keypoint recognition model, so as to accurately determine the predicted coordinates of the target keypoints in the initial image based on the keypoint probability map, thereby improving the detection accuracy of hip joint keypoints. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating a medical image processing method in one embodiment;

[0045] Figure 2This is a schematic diagram showing the location of a target key point in an initial sample image in one embodiment;

[0046] Figure 3 This is a flowchart illustrating a medical image processing method in another embodiment;

[0047] Figure 4 This is a flowchart illustrating another medical image processing method in one embodiment;

[0048] Figure 5 This is a schematic diagram showing the position of a central edge angle in a pelvic X-ray image in one embodiment.

[0049] Figure 6 This is a schematic diagram showing the position of the acetabular roof tilt angle in an anteroposterior pelvic X-ray image in one embodiment;

[0050] Figure 7 This is a schematic diagram showing the position of the acetabular angle in a pelvic X-ray image in one embodiment.

[0051] Figure 8 This is a structural block diagram of a medical image processing device in one embodiment;

[0052] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0055] In one embodiment, such as Figure 1As shown, a medical image processing method is provided, applied to a computer device. In practical applications, the computer device can be a user terminal, or it can be implemented using a standalone server or a server cluster composed of multiple servers. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. In this embodiment, the method includes the following steps:

[0056] Step S110: Obtain the image to be recognized corresponding to the initial image.

[0057] The initial image is an image containing the hip joint of the target object. In practical applications, the initial image can be an anteroposterior X-ray image of the hip joint or an anteroposterior X-ray image of the pelvis of the target object.

[0058] The image to be identified is obtained by superimposing the first image and the second image.

[0059] The first image is an image of one side of the first vertical midline of the initial image determined according to the hip joint topological constraints.

[0060] The second image is obtained by flipping the image on the other side of the first vertical central axis.

[0061] The hip joint topological constraints are derived from the symmetry of the hip joint's anatomical structure.

[0062] The anatomical structure of the hip joint is bilaterally symmetrical.

[0063] In practice, the computer device acquires an initial image containing the hip joint of the target object. It then uses the image on one side of the first vertical central axis of the initial image, determined according to the hip joint topological constraints, as the first image. The image on the other side of the first vertical central axis is flipped along that axis and used as the second image. The computer device then superimposes the first and second images to form an image with two channels, which becomes the image to be recognized and serves as the input to a pre-trained keypoint recognition model.

[0064] Step S120: Input the image to be recognized into the pre-trained key point recognition model to obtain the key point probability map corresponding to each target key point.

[0065] The first and second images include key points of the target to be identified.

[0066] In this image, the target key points in the first image correspond one-to-one with the target key points in the second image.

[0067] The pre-trained keypoint recognition model was trained based on hip joint topological constraints.

[0068] In practice, the computer device can acquire a pre-trained keypoint recognition model, which is trained based on hip joint topological constraints. The computer device can input the image to be recognized into the pre-trained keypoint recognition model, which is a multi-channel output model. This model utilizes hip joint topological constraints derived from the left-right symmetry of the hip joint anatomy to detect target keypoints in a first image and a second image obtained based on the first vertical central axis of the initial image. It then outputs keypoint probability maps corresponding to each target keypoint in the first image and the second image, respectively.

[0069] Step S130: Determine the predicted coordinates of the corresponding target key points in the initial image based on the probability map of each key point.

[0070] In practice, the computer device can determine the target connected region with the largest area in each key point probability map, and then determine the predicted coordinates of the target key points in the initial image based on the target connected regions in each key point probability map.

[0071] In the aforementioned medical image processing method, the image to be identified is obtained by overlaying the first image and the second image. The first image is an image on one side of the first vertical midline of the initial image, determined according to the hip joint topological constraints. The second image is an image obtained by flipping the image on the other side of the first vertical midline. The image to be identified is input into a pre-trained keypoint recognition model to obtain keypoint probability maps corresponding to each target keypoint. The pre-trained keypoint recognition model is trained according to the hip joint topological constraints. The predicted coordinates of the corresponding target keypoints in the initial image are determined based on the keypoint probability maps. Thus, the first image and the second image in the image to be identified can be compared. Two images are input as a group into a pre-trained keypoint recognition model, which is trained using hip joint topological constraints. The first image is an image of one side of the first vertical central axis of the initial image determined by the hip joint topological constraints, and the second image is an image obtained by flipping the image of the other side of the first vertical central axis. Thus, the robustness of the keypoint recognition model for locating target keypoints in the symmetrical initial image can be enhanced by the hip joint topological constraints, thereby improving the accuracy of the keypoint probability map output by the keypoint recognition model. Based on the keypoint probability map, the predicted coordinates of the target keypoints in the initial image can be accurately determined, thus improving the detection accuracy of hip joint keypoints.

[0072] In one embodiment, the method further includes: obtaining a training sample set corresponding to the initial sample image; inputting the sample image into the key point recognition model to be trained to obtain a predicted probability map corresponding to the sample image; and training the key point recognition model to be trained based on the predicted probability map, the mask image, and the hip joint topological constraints to obtain a pre-trained key point recognition model.

[0073] Each training sample in the training sample set includes a sample image and a mask image corresponding to the sample image.

[0074] The initial sample image is labeled with the target coordinates corresponding to each target key point.

[0075] The target coordinates corresponding to the target key points are obtained by labeling the target key points in the initial sample image using annotation software.

[0076] The initial sample image is a sample image containing the hip joint. In practical applications, the initial sample image can be an anteroposterior X-ray image of the hip joint or an anteroposterior X-ray image of the pelvis.

[0077] The mask image consists of binary mask rectangles corresponding to each target key point, meaning the number of binary mask rectangles is equal to the number of target key points.

[0078] In this context, the center point of each binarized mask rectangle represents the target coordinates of each target key point in the initial sample image.

[0079] The prediction probability map includes a background prediction probability map and a key point prediction probability map corresponding to each target key point.

[0080] The key point recognition model to be trained can be a convolutional neural network model.

[0081] Among them, convolutional neural network models can be fully convolutional neural networks (FCN), semantic segmentation networks (U-Net), image segmentation networks (V-Net), etc.

[0082] In the specific implementation, the computer device can acquire N initial sample images containing the hip joint. Each initial sample image is marked with the target coordinates corresponding to the target key point. Based on the second vertical central axis of each initial sample image, the corresponding sample image is obtained, resulting in N sample images, which are used as input to the key point recognition model to be trained. The sample images are marked with the target coordinates corresponding to the target key point.

[0083] Simultaneously, the computer equipment can determine the target coordinates corresponding to each target keypoint in the initial sample image, and generate a mask image corresponding to each initial sample image based on the target coordinates corresponding to each target keypoint in each initial sample image, resulting in N mask images. Each mask image consists of I binary mask rectangles of size L×H, centered at the target coordinates corresponding to each target keypoint; where I represents the total number of target keypoints; L represents the length of the binary mask rectangle; and H represents the height of the binary mask rectangle.

[0084] In this way, N sample images and N mask images are paired to form the training sample set corresponding to the initial sample images.

[0085] Then, the computer device can input the sample image into the constructed key point recognition model to be trained to obtain the prediction probability map corresponding to the sample image. The prediction probability map includes the corresponding key point prediction probability map output by the key point recognition model for each target key point, and the corresponding background prediction probability map output by the key point recognition model for the background in the sample image. Among them, the pixel value corresponding to each pixel in the prediction probability map is the probability that the pixel at each position belongs to the corresponding target key point or the background.

[0086] Finally, the computer equipment can train the key point recognition model to be trained based on the predicted probability map corresponding to the sample image, the mask image corresponding to the sample image, and the hip joint topological constraints, to obtain the pre-trained key point recognition model.

[0087] In this embodiment, the key point recognition model can output a key point prediction probability map corresponding to each target key point during training, realizing the output of multiple sets of target key point position predictions at one time, which can improve the recognition efficiency of target key points. Furthermore, the key point recognition model to be trained is further constrained according to the hip joint topological constraints, which can enhance the robustness of the model for target key point localization in the image to be recognized, and improve the overall recognition accuracy of target key points.

[0088] In one embodiment, a keypoint recognition model to be trained is trained based on a predicted probability map, a mask image, and hip joint topological constraints to obtain a pre-trained keypoint recognition model. This includes: determining a first difference between the predicted probability map and the mask image based on an initial loss function; determining a second difference between the first femoral head radius corresponding to a first set of keypoints and the second femoral head radius corresponding to a second set of keypoints based on femoral head topological constraints; determining a third difference between the first acetabular radius corresponding to the first set of keypoints and the second acetabular radius corresponding to the second set of keypoints based on acetabular topological constraints; and training the keypoint recognition model to be trained based on the first, second, and third differences to obtain the pre-trained keypoint recognition model.

[0089] The sample image is obtained by overlaying the first sample image and the second sample image.

[0090] The first sample image is an image on one side of the second vertical central axis of the initial sample image determined according to the hip joint topological constraints.

[0091] The second sample image is the image obtained by flipping the image on the other side of the second vertical central axis along the second vertical central axis.

[0092] The target key points include a first set of key points corresponding to the first sample image and a second set of key points corresponding to the second sample image.

[0093] Among them, each target key point in the first key point set corresponds one-to-one with each target key point in the second key point set.

[0094] Among them, the hip joint topological constraints include the femoral head topological constraints and the acetabulum topological constraints.

[0095] The initial loss function may include, but is not limited to, functions such as set similarity measurement function (Dice loss) and focal loss function.

[0096] In practical applications, the femoral head topological constraint condition can be named the left and right femoral head topological constraint condition.

[0097] In practical applications, the acetabular topological constraint condition can be named the left and right acetabular topological constraint condition.

[0098] Among them, the femoral head topological constraint condition is used to constrain the radius of the femoral head corresponding to the target key point in the sample image to meet the first preset condition.

[0099] Among them, the acetabular topological constraint condition is used to constrain the acetabular radius corresponding to the target key point in the sample image to meet the second preset condition.

[0100] In specific implementation, during the process of obtaining sample images corresponding to each initial sample image based on the second vertical central axis of each initial sample image, the computer device can use the image on one side of the second vertical central axis of the initial sample image determined according to the hip joint topological constraints as the first sample image, and the image obtained by flipping the image on the other side of the second vertical central axis along the second vertical central axis as the second sample image. The target key points are distributed on both sides of the second vertical central axis. Specifically, the target key points distributed on one side of the second vertical central axis, i.e., distributed in the first sample image, are used as the first key point set corresponding to the first sample image; the target key points distributed on the other side of the second vertical central axis, i.e., distributed in the second sample image, are used as the second key point set corresponding to the second sample image. The first and second sample images are superimposed to form an image with two channels, resulting in a sample image marked with the target coordinates corresponding to the target key points.

[0101] In the process of training the keypoint recognition model to be trained based on the predicted probability map corresponding to the sample image, the mask image corresponding to the sample image, and the topological constraints of the hip joint, the computer equipment can determine the difference between the predicted probability map corresponding to the sample image and the mask image corresponding to the sample image based on the initial loss function, as the first difference; and based on the femoral head topological constraints, determine the first femoral head radius corresponding to the first keypoint set in the first sample image, and the second femoral head radius corresponding to the second keypoint set in the second sample image, and take the difference between the first femoral head radius and the second femoral head radius as the second difference; at the same time, the computer equipment can also determine the first acetabular radius corresponding to the first keypoint set in the first sample image, and the second acetabular radius corresponding to the second keypoint set in the second sample image based on the acetabular topological constraints, and take the difference between the first acetabular radius and the second acetabular radius as the third difference.

[0102] In this way, the computer device can train the key point recognition model to be trained based on the first difference, the second difference and the third difference mentioned above. When the trained key point recognition model meets the preset training termination condition, the pre-trained key point recognition model is obtained.

[0103] In this embodiment, the technical solution trains the key point recognition model based on the difference between the predicted probability map corresponding to the sample image and the mask image corresponding to the sample image during the training process. This effectively improves the recognition accuracy of the key point recognition model for target key points. Simultaneously, the key point recognition model can be trained based on the femoral head topological constraints, utilizing the left-right symmetry of the femoral head and acetabular structures of the human hip joint. By adding femoral head and acetabular topological constraints to the initial loss function, the model training is further guided, enhancing its robustness in locating target key points in the image to be recognized and further improving the recognition accuracy of target key points.

[0104] In one embodiment, determining a second difference between the first femoral head radius corresponding to the first set of keypoints and the second femoral head radius corresponding to the second set of keypoints, based on the femoral head topological constraints, includes: obtaining the first femoral head radius based on the predicted sample coordinates of the target keypoints in the first set of keypoints in the initial sample image; obtaining the second femoral head radius based on the predicted sample coordinates of the target keypoints in the second set of keypoints in the initial sample image; and determining the length difference between the first and second femoral head radii as the second difference.

[0105] The key target points include the upper acetabulum rim (UAR), the femoral head center (FHC), and the top acetabulum rim (TAR).

[0106] Among them, the target key points in the first key point set include the first lateral superior edge of the acetabulum (named L_uar), the first femoral head rotation center point (named L_fhc), and the first acetabular weight-bearing zone inner edge point (named L_tar).

[0107] Among them, the target key points in the second key point set include the second lateral superior edge of the acetabulum (named R_uar), the second femoral head rotation center point (named R_fhc), and the second acetabular weight-bearing zone inner edge point (named R_tar).

[0108] For the ease of understanding of those skilled in the art, Figure 2 A schematic diagram showing the location of target key points in an initial sample image is provided. For example... Figure 2As shown, the initial sample image can be a sample image containing the hip joint, such as an anteroposterior X-ray sample image of the hip joint. The first keypoint set is distributed on one side of the second vertical central axis of the initial sample image; the second keypoint set is distributed on the other side of the vertical central axis of the initial sample image; each target keypoint in the first keypoint set corresponds one-to-one with each target keypoint in the second keypoint set.

[0109] In specific implementation, during the process of determining the second difference between the first femoral head radius corresponding to the first keypoint set and the second femoral head radius corresponding to the second keypoint set based on the femoral head topological constraints, the computer device can determine the distance between the first femoral head rotation center point and the first acetabular upper lateral edge point as the first distance, named r11, based on the predicted sample coordinates output by the keypoint recognition model during training for the first femoral head rotation center point, the first acetabular upper lateral edge point, and the first acetabular weight-bearing zone inner edge point. The computer device can also determine the distance between the first femoral head rotation center point and the first acetabular weight-bearing zone inner edge point as the second distance, named r12. Thus, the computer device can determine the average of the first and second distances, obtaining the average distance between the first and second distances, which is used as the first femoral head radius. A schematic diagram of r11 and r12 is shown below. Figure 2 As shown.

[0110] Simultaneously, the computer device can also determine the distance between the second femoral head rotation center point and the second lateral upper edge of the acetabulum based on the predicted sample coordinates output in the initial sample image for the second femoral head rotation center point, the predicted sample coordinates output in the initial sample image for the second lateral upper edge of the acetabulum, and the predicted sample coordinates output in the initial sample image for the second acetabulum weight-bearing zone inner edge point during the training process of the key point recognition model. This third distance is named r21; the fourth distance is determined based on the distance between the second femoral head rotation center point and the second acetabulum weight-bearing zone inner edge point. This fourth distance is named r22. Thus, the computer device can determine the average value between the third and fourth distances, obtaining the average distance between the three distances, which is used as the radius of the second femoral head. A schematic diagram of r21 and r22 is shown below. Figure 2 As shown.

[0111] Thus, the computer device can determine the length difference between the first femoral head radius and the second femoral head radius as the second difference.

[0112] In the technical solution of this embodiment, during the model training process, the lengths of the first femoral head radius and the second femoral head radius can be accurately determined, thereby accurately determining the length difference between the first and second femoral head radii. This enables the key point recognition model training to be further guided based on the symmetry of the femoral head structure, improving the recognition accuracy of the key point recognition model for each target key point.

[0113] In one embodiment, determining a third difference between the first acetabular radius corresponding to the first set of key points and the second acetabular radius corresponding to the second set of key points, based on acetabular topological constraints, includes: fitting a first arc corresponding to the target key points in the first set of key points in the initial sample image, and a second arc corresponding to the target key points in the second set of key points in the initial sample image; determining the radius of the first arc as the first acetabular radius, and determining the radius of the second arc as the second acetabular radius; and determining the length difference between the first acetabular radius and the second acetabular radius as the third difference.

[0114] The target key points include the upper acetabulum rim (UAR), the femoral head center (FHC), the top acetabulum rim (TAR), and the lower teardrop (LT).

[0115] Among them, the target key points in the first key point set include the first acetabular outer upper edge point (named L_uar), the first femoral head rotation center point (named L_fhc), and the first acetabular weight-bearing zone inner edge point (named L_tar), as well as the first teardrop lower edge point (named L_lt).

[0116] Among them, the target key points in the second key point set include the second acetabulum outer upper edge point (named R_uar), the second femoral head rotation center point (named R_fhc), the second acetabulum weight-bearing zone inner edge point (named R_tar), and the second teardrop lower edge point (named R_lt).

[0117] For ease of understanding by those skilled in the art, the positions of the first teardrop lower edge point and the second teardrop lower edge point in the initial sample image are as follows: Figure 2 As shown.

[0118] In specific implementation, during the process of determining the third difference between the first acetabular radius corresponding to the first key point set and the second acetabular radius corresponding to the second key point set based on the acetabular topological constraints, the computer device can determine the positions of the first teardrop lower edge point, the first acetabular upper edge point, and the first acetabular weight-bearing zone inner edge point in the initial sample image based on the predicted sample coordinates output by the key point recognition model during training, for the first teardrop lower edge point, the first acetabular upper edge point, and the first acetabular weight-bearing zone inner edge point in the initial sample image, respectively. Then, using the least squares method, the computer device fits the first arc corresponding to the first teardrop lower edge point, the first acetabular upper edge point, and the first acetabular weight-bearing zone inner edge point in the initial sample image; wherein, the first teardrop lower edge point, the first acetabular upper edge point, and the first acetabular weight-bearing zone inner edge point are located on the first arc.

[0119] Simultaneously, the computer equipment can determine the corresponding positions of the second teardrop lower edge point, the second acetabulum upper edge point, and the second acetabulum weight-bearing zone inner edge point in the initial sample image based on the predicted sample coordinates output by the key point recognition model during training. This is done for the second teardrop lower edge point, the second acetabulum upper edge point, and the second acetabulum weight-bearing zone inner edge point in the initial sample image. Then, using the least squares method, the computer can fit the corresponding second arc in the initial sample image. The second teardrop lower edge point, the second acetabulum upper edge point, and the second acetabulum weight-bearing zone inner edge point are located on the second arc.

[0120] Thus, the computer device can determine the radius of the first arc (named R1) as the first acetabular radius based on the predicted sample coordinates of the lower edge of the first teardrop, the upper outer edge of the first acetabulum, and the inner edge of the first weight-bearing zone of the first acetabulum in the initial sample image; and determine the radius of the second arc (named R2) as the second acetabular radius based on the predicted sample coordinates of the lower edge of the second teardrop, the upper outer edge of the second acetabulum, and the inner edge of the weight-bearing zone of the second acetabulum in the initial sample image. A schematic diagram of R1 and R2 is shown below. Figure 2 As shown.

[0121] Thus, the computer device can determine the length difference between the first acetabular radius and the second acetabular radius as a third difference.

[0122] In this embodiment, the technical solution can accurately determine the length of the first acetabular radius and the length of the second acetabular radius during model training. This allows for accurate determination of the length difference between the first and second acetabular radii, enabling further guidance of the key point recognition model training based on the symmetry of the acetabular structure and improving the recognition accuracy of the key point recognition model for each target key point.

[0123] In one embodiment, training the keypoint recognition model to be trained based on the first difference, the second difference, and the third difference to obtain a pre-trained keypoint recognition model includes: updating the model parameters of the keypoint recognition model to be trained according to the first difference, the second difference, and the third difference using a preset parameter optimization algorithm; retraining the keypoint recognition model with updated model parameters; and determining that the trained keypoint recognition model meets the preset training termination condition when the first difference is less than a preset difference threshold, the second difference is less than a femoral head radius difference threshold, and the third difference is less than a acetabular radius difference threshold, thereby obtaining the pre-trained keypoint recognition model.

[0124] Among them, the femoral head topological constraint is used to constrain the second difference to be less than the femoral head radius difference threshold.

[0125] Among them, the acetabular topological constraint is used to constrain the third difference to be less than the acetabular radius difference threshold.

[0126] The preset parameter optimization algorithm can be, but is not limited to, stochastic gradient descent (SGD), adaptive moment estimation (Adam), and momentum algorithm.

[0127] In practice, during the process of training the keypoint recognition model to be trained based on the first difference, the second difference, and the third difference to obtain the pre-trained keypoint recognition model, the computer device can feed back the first difference, the second difference, and the third difference as training errors to the keypoint recognition model to be trained, and update the model parameters of the keypoint recognition model to be trained according to the preset parameter optimization algorithm. After that, the computer device retrains the keypoint recognition model with updated model parameters until the first difference is less than the preset difference threshold, the second difference is less than the femoral head radius difference threshold, and the third difference is less than the acetabular radius difference threshold. At this point, it is determined that the trained keypoint recognition model meets the preset training termination condition, and the pre-trained keypoint recognition model is obtained.

[0128] The first difference being less than a preset difference threshold can characterize the difference between the predicted probability map corresponding to the sample image and the mask image corresponding to the sample image as being minimized or the difference tending to stabilize.

[0129] Among them, the second difference being less than the femoral head radius difference threshold can indicate that the first femoral head radius and the second femoral head radius are approximately equal.

[0130] Among them, the third difference being less than the acetabular radius difference threshold can indicate that the first acetabular radius and the second acetabular radius are approximately equal.

[0131] In practical applications, even in cases of hip dysplasia where pathological dislocation of the femoral head occurs, resulting in a difference in the height of the left and right femoral heads, the dimensions of the left and right femoral heads should generally still be similar and symmetrical. When the target keypoints corresponding to the left and right femoral heads (the first keypoint set for the left femoral head and the second keypoint set for the right femoral head) are correctly detected, the radii of the left and right femoral heads should be approximately equal, that is, the radius of the first femoral head (named r1) and the radius of the second femoral head (named r2) should be approximately equal. Therefore, the topological constraint condition for the left and right femoral heads (i.e., the femoral head topological constraint condition) is defined as ||r1-r2||1; where ||·|| represents the L1 norm.

[0132] Meanwhile, regarding hip dysplasia, even if pathological deformities of the acetabulum occur, causing differences in the height of the left and right acetabulums, the dimensions of the left and right acetabulums should generally still be similar and symmetrical. When the target key points corresponding to the left and right acetabulums (the left femoral head corresponds to the first key point set, and the right femoral head corresponds to the second key point set) are correctly detected, the radii of the left and right acetabulums should be approximately equal, that is, the first acetabular radius (named R1) and the second acetabular radius (named R2) are approximately equal. Therefore, the topological constraint condition of the left and right acetabulums (i.e., the acetabular topological constraint condition) is defined as: ||R1-R2||1.

[0133] Therefore, the target loss function of the key point recognition model during training can be expressed as follows:

[0134] Loss function = initial loss function + α||r1-r2||1+β||R1-R2||1

[0135] The initial loss function can be, but is not limited to, functions such as the set similarity measure function (Dice loss) and the focal loss function; α represents the weight parameter of the femoral head topological constraint with respect to the target loss function; β represents the weight parameter of the acetabular topological constraint with respect to the target loss function.

[0136] The technical solution of this embodiment utilizes the left-right symmetry of the femoral head and acetabulum structures of the human hip joint, adds femoral head and acetabulum topological constraints to the initial loss function, and further guides model training with prior knowledge and hip joint anatomy, thereby enhancing the robustness of the model in locating target key points in pathological images.

[0137] In one embodiment, determining the predicted coordinates of the corresponding target key point in the initial image based on each key point probability map includes: for any key point probability map, performing binarization processing on any key point probability map according to a preset probability threshold to obtain a corresponding binarized mask image; determining the connected component with the largest area in the corresponding binarized mask image to obtain the target connected component; and determining the coordinates of the geometric center of the target connected component to obtain the predicted coordinates of the target key point corresponding to any key point probability map in the initial image.

[0138] The binarized mask image contains at least one connected component.

[0139] The coordinates of the geometric center of each connected domain are used as candidate coordinates for the corresponding target key points.

[0140] In the specific implementation, during the process of determining the predicted coordinates of the corresponding target key points in each key point probability map, the computer device can obtain a preset probability threshold for any key point probability map, and perform binarization processing on any key point probability map according to the probability threshold to obtain the binarized mask image corresponding to any key point probability map; the corresponding binarized mask image has at least one connected component, and the coordinates of the geometric center of each connected component are used as the candidate coordinates of the corresponding target key point.

[0141] When there are at least two candidate coordinates for the corresponding target key point, in order to prevent multiple candidate points from being output and to select the predicted coordinates that most accurately represent the actual position of the target key point from the candidate coordinates, the computer device can determine the connected component with the largest area in the corresponding binarized mask image, obtain the target connected component, determine the coordinates of the geometric center of the target connected component, so as to obtain the predicted coordinates of the target key point in the initial image corresponding to any key point probability map.

[0142] In this embodiment, when there are multiple connected components in the binarized mask image, that is, when there are multiple candidate coordinates for the target key point corresponding to the binarized mask image, the coordinates of the geometric center of the largest connected component in the binarized mask image are used as the predicted coordinates of the corresponding target key point. This avoids the situation of outputting multiple candidate coordinates for the corresponding target key point and effectively improves the accuracy of the predicted coordinates of the target key point.

[0143] In one embodiment, determining the coordinates of the geometric center of the target connected region to obtain the predicted coordinates of the target key point in the initial image corresponding to any key point probability map includes: for any first key point probability map, determining the coordinates of the geometric center of the target connected region as the predicted coordinates of the first target key point in the initial image corresponding to any first key point probability map; for any second key point probability map, flipping the coordinates of the geometric center of the target connected region along a first vertical central axis to obtain the predicted coordinates of the second target key point in the initial image corresponding to any second key point probability map.

[0144] The target key points include the first target key points located in the first image and the second target key points located in the second image.

[0145] Among them, the first target key point is the target key point in the first key point set, which is distributed on one side of the first vertical central axis, that is, distributed in the first image.

[0146] Among them, the second target key point is the target key point in the second key point set, which is distributed on the other side of the first vertical central axis, that is, in the second image.

[0147] The key point probability map includes a first key point probability map corresponding to each first target key point and a second key point probability map corresponding to each second target key point.

[0148] Each key point of the first objective corresponds one-to-one with each key point of the second objective.

[0149] In specific implementation, during the process of determining the coordinates of the geometric center of the target connected region and obtaining the predicted coordinates of the target key point in the initial image corresponding to any key point probability map, the computer device can determine the coordinates of the geometric center of the target connected region for any first key point probability map, which serves as the predicted coordinates of the first target key point in the initial image corresponding to that first key point probability map. For any second key point probability map, the computer device can flip the coordinates of the geometric center of the target connected region along a first vertical central axis to obtain the predicted coordinates of the second target key point in the initial image corresponding to that second key point probability map.

[0150] In this embodiment, since the first image is an unflipped image located on one side of the first vertical central axis, the coordinates of the geometric center of the target connected region corresponding to the first key point probability map are directly used as the predicted coordinates of the first target key point in the initial image, thus accurately determining the predicted coordinates of the first target key point in the initial image. Since the second image is an image obtained by flipping the image located on the other side of the first vertical central axis along the first vertical central axis, the predicted coordinates of the second target key point in the initial image can be accurately determined by flipping the coordinates of the geometric center of the target connected region corresponding to the second key point probability map along the first vertical central axis.

[0151] In another embodiment, such as Figure 3 As shown, a medical image processing method is provided, and the method is illustrated using an application to a computer device as an example. The method includes the following steps:

[0152] Step S302: Obtain the training sample set corresponding to the initial sample image.

[0153] Step S304: Input the sample image into the key point recognition model to be trained to obtain the predicted probability map corresponding to the sample image.

[0154] Step S306: Determine the first difference between the predicted probability map and the mask image based on the initial loss function.

[0155] Step S308: Based on the femoral head topological constraints, determine the second difference between the first femoral head radius corresponding to the first keypoint set and the second femoral head radius corresponding to the second keypoint set.

[0156] Step S310: Based on the acetabular topological constraints, determine the third difference between the first acetabular radius corresponding to the first key point set and the second acetabular radius corresponding to the second key point set.

[0157] Step S312: Based on the first difference, the second difference, and the third difference, update the model parameters of the key point recognition model to be trained according to the preset parameter optimization algorithm.

[0158] Step S314: Retrain the key point recognition model with updated model parameters. When the first difference is less than the preset difference threshold, the second difference is less than the femoral head radius difference threshold, and the third difference is less than the acetabular radius difference threshold, the trained key point recognition model is determined to meet the preset training termination condition, and the pre-trained key point recognition model is obtained.

[0159] Step S316: Obtain the image to be recognized corresponding to the initial image.

[0160] Step S318: Input the image to be recognized into the pre-trained key point recognition model to obtain the key point probability map corresponding to each target key point.

[0161] Step S320: Determine the predicted coordinates of the corresponding target key points in the initial image based on the probability map of each key point.

[0162] It should be noted that the specific limitations of the above steps can be found in the specific limitations of a medical image processing method described above.

[0163] In this embodiment, the technical solution utilizes the left-right symmetry of the femoral head and acetabulum structures of the human hip joint during the keypoint recognition model training process. Femoral head topological constraints are added to the initial loss function to further guide model training and obtain a pre-trained keypoint recognition model. Therefore, when the first and second images from the image to be recognized are input as a group into the pre-trained keypoint recognition model, the robustness of the keypoint recognition model for locating target keypoints in the symmetrical initial image can be enhanced based on the femoral head and acetabulum topological constraints. This improves the accuracy of the keypoint probability map output by the keypoint recognition model, allowing for accurate determination of the predicted coordinates of the target keypoints in the initial image based on the keypoint probability map.

[0164] In one embodiment, the method further includes: constructing a key point recognition model to be trained; dividing the training sample set to obtain a training set, a validation set, and a test set; and training the key point recognition model to be trained based on the training set, the validation set, and the test set to obtain a pre-trained key point recognition model.

[0165] The training set is used to train the keypoint recognition model; the validation set is used to evaluate the current performance of the keypoint recognition model; and the test set is used to test the generalization performance of the keypoint recognition model.

[0166] In practice, the computer device can construct a keypoint recognition model to be trained, such as a convolutional neural network model, and set the hyperparameters of the keypoint recognition model to be trained. These hyperparameters include the number of network layers, convolutional kernels, learning rate, parameter initialization, number of training epochs, and batch size. In the initial sample image, since the target keypoints are distributed on both sides of the second vertical central axis, each target keypoint in the first keypoint set corresponds one-to-one with each target keypoint in the second keypoint set. If I represents the total number of target keypoints, then the total number of target keypoints in the first keypoint set corresponding to the first sample image is I / 2; the total number of target keypoints in the second keypoint set corresponding to the second sample image is I / 2. Therefore, when constructing the keypoint recognition model to be trained, the input channels of the keypoint recognition model can be 2, corresponding to sample images with two channels; the output channels are 2×(I / 2+1), used to output the prediction probability map, that is, to output the corresponding keypoint prediction probability map for I / 2 target keypoints corresponding to each input channel, and to output the corresponding background prediction probability map for the background corresponding to each input channel; that is, the output channels are used to output the corresponding keypoint prediction probability map for I / 2 target keypoints in the first sample image, and to output the corresponding background prediction probability map for the background in the first sample image; to output the corresponding keypoint prediction probability map for I / 2 target keypoints in the second sample image, and to output the corresponding background prediction probability map for the background in the second sample image.

[0167] Thus, when performing target keypoint recognition on the image to be recognized based on the pre-trained keypoint recognition model, the first keypoint set is distributed on one side of the first vertical central axis, i.e., distributed in the first image, and the second keypoint set is distributed on the other side of the first vertical central axis, i.e., distributed in the second image. The pre-trained keypoint recognition model can output the corresponding keypoint probability map for I / 2 target keypoints in the first image and the corresponding background probability map for the background in the first image; it can also output the corresponding keypoint probability map for I / 2 target keypoints in the second image and the corresponding background probability map for the background in the second image.

[0168] In practical applications, the number of target key points I can be 8, namely the first lateral upper edge of the acetabulum, the first femoral head rotation center point, the first acetabular weight-bearing zone inner edge point, the first teardrop lower edge point, the second lateral upper edge of the acetabulum, the second femoral head rotation center point, the second acetabular weight-bearing zone inner edge point, and the second teardrop lower edge point.

[0169] In practical applications, the keypoint recognition model to be trained can use an adjusted V-net network. The adjusted V-net network has 2 input channels and 2×(4+1) output channels. The adjusted V-net network model includes an input module, two downsampling modules, two upsampling modules, and an output module. Except for the output module, all other modules use batch normalization layers and the ReLU nonlinear activation function, while the nonlinear activation function in the output module is replaced with the Softmax function (normalized exponential function), whose output value is in the (0,1) interval. The softmax function of the output module is calculated across each output channel. Thus, the sum of the pixel values ​​of corresponding pixels in each predicted probability map output by the output module is 1. The pixel values ​​in each predicted probability map represent the probability that the pixel at the current position in the input sample image belongs to the corresponding target keypoint or background.

[0170] Before training the keypoint recognition model, the computer device can divide the training sample set into a training set (named X1), a validation set (named X2), and a test set (named X3). The training set, validation set, and test set are independent of each other, with sizes n1, n2, and n3, respectively; where n1, n2, and n3 are natural numbers, and n1 + n2 + n3 = N, n1 ≥ 1 / 2N (N is the number of initial sample images). The training set X1 is used to train the keypoint recognition model, the validation set X2 is used to evaluate the current performance of the keypoint recognition model, and the test set X3 is used to test the generalization performance of the keypoint recognition model.

[0171] During the training of the keypoint recognition model, the training set X1 is repeatedly input into the keypoint recognition model in multiple batches for multiple training rounds. Simultaneously, the first, second, and third differences are fed back to the keypoint recognition model as training errors, and the parameters of the keypoint recognition model are updated through a preset parameter optimization algorithm. After each batch of training, the validation set X2 is used to perform performance testing on the keypoint recognition model. Once the performance test indicators stabilize, the training of the keypoint recognition model is considered complete, and the pre-trained keypoint recognition model is obtained. The performance test indicators are the first, second, and third differences. The performance test indicators stabilize when the first difference is less than a preset difference threshold, the second difference is less than a femoral head radius difference threshold, and the third difference is less than a acetabular radius difference threshold.

[0172] In practical applications, N can be 1000; the number of training sets n1 can be 500; the number of validation sets n2 can be 200; and the number of test sets n3 can be 300.

[0173] In practical applications, the training set X1 can be repeatedly input into the keypoint recognition model in 100 batches for 100 rounds of training; simultaneously, the focal loss function can be used as the initial loss function. The target loss function formula is then:

[0174] Loss function=Focal loss+α||r1-r2||1+β||R1-R2||1

[0175] The technical solution in this embodiment uses a multi-channel deep learning network as its basic framework to achieve multi-channel output based on instance segmentation. This allows for the simultaneous output of predicted locations corresponding to multiple sets of target key points, thereby improving the efficiency of target key point recognition. Furthermore, by dividing the training sample set into training, validation, and test sets for training the key point recognition model, a key point recognition model with high accuracy can be obtained.

[0176] For the ease of understanding of those skilled in the art, Figure 4 A flowchart of another medical image processing method is provided, with an example of its application in a computer device. The computer device first acquires an initial image. Then, it crops the initial image along a first vertical central axis to obtain an image on one side of the first vertical central axis, which is the first image, and obtains an image on the other side of the first vertical central axis. The image on the other side is then flipped along the first vertical central axis to obtain the second image. After that, the first image and the second image are superimposed to form an image with two channels, which is the image to be identified.

[0177] Then, the computer device acquires the key point recognition model and determines whether the key point recognition model has been trained. If not, the key point recognition model is trained. For the specific training process, please refer to the above description of the specific limitations of a medical image processing method. If yes, the image to be recognized is input into the trained key point recognition model to obtain the key point probability map corresponding to each target key point, and the predicted coordinates of each target key point in the initial image are determined based on the key point probability map.

[0178] Finally, the computer equipment can calculate the measurement indicators required for the diagnosis of hip dysplasia based on the predicted coordinates of each target key point in the initial image.

[0179] When the initial image is a pelvic X-ray image, the required measurement indicators for diagnosing hip dysplasia include the CE angle (center-edge angle), the Tonnis angle (acetabular roof tilt angle, also known as the acetabular index (AI)), and the Sharp angle (acetabular angle).

[0180] The CE angle is defined as follows: it is the angle formed by the line connecting the superior lateral margin of the acetabulum (uar) and the center of rotation of the femoral head (fhc), and the perpendicular line from the human body passing through the center of rotation of the femoral head (fhc). It can be used to evaluate defects in the superior lateral margin of the acetabulum and femoral head dislocation. A CE angle less than 20° on a pelvic X-ray (anteroposterior view) is sufficient for a diagnosis of hip dysplasia. For ease of understanding by those skilled in the art, Figure 5 A schematic diagram of the CE angle position in a pelvic X-ray anteroposterior image is provided. Here, L_uar and R_uar represent the points on the lateral superior margin of the acetabulum located on either side of the vertical midline of the pelvic X-ray anteroposterior image; L_fhc and R_fhc represent the centers of rotation of the femoral head located on either side of the vertical midline of the pelvic X-ray anteroposterior image; Left CE and Right CE represent the CE angles located on either side of the vertical midline of the pelvic X-ray anteroposterior image.

[0181] The Tonnis angle is defined as follows: it is the angle formed by the line connecting the inner edge of the weight-bearing area of ​​the acetabulum (tar, the upper edge of the fossa ovalis) and the outer upper edge of the acetabulum (uar), and the horizontal line (the line connecting the lower edges of the teardrops on both sides of the vertical midline of the pelvic X-ray image). It is used to assess the inclination of the acetabular roof. The normal Tonnis angle ranges from 0° to 10°. When this angle is greater than 10°, hip dysplasia is diagnosed. For ease of understanding by those skilled in the art, Figure 6 A schematic diagram of the Tonnis angle position in a pelvic X-ray anteroposterior image is provided. Here, L_tar and R_tar represent the inner edges of the acetabular weight-bearing area located on either side of the vertical midline of the pelvic X-ray anteroposterior image; L_lt and R_lt represent the lower edges of the teardrop located on either side of the vertical midline of the pelvic X-ray anteroposterior image.

[0182] The Sharp angle is defined as follows: it is the angle between the line connecting the upper outer edge of the acetabulum (uar) and the lower edge of the teardrop (lt), and the horizontal line (the line connecting the lower edges of the teardrop on both sides of the vertical midline of the pelvic X-ray image). It reflects the acetabular opening and tilt. The normal value of the Sharp angle is less than 40°. When the angle is greater than 40°, hip dysplasia is diagnosed. For ease of understanding by those skilled in the art, Figure 7 A schematic diagram of the position of the Sharp angle in an anteroposterior X-ray image of the pelvis is provided. Left Sharp and Right Sharp represent the Sharp angles located on either side of the vertical midline of the anteroposterior X-ray image of the pelvis.

[0183] In the process of calculating the measurement indicators required for the diagnosis of hip dysplasia based on the predicted coordinates of each target key point in the initial image, the computer equipment can generate two spatial vectors corresponding to the auxiliary straight lines according to the above measurement indicator definition, using the predicted coordinates corresponding to each target key point, and output the angle between the vectors. The acute angle of this angle is taken as the output measurement indicator.

[0184] Specifically, the method for automatically measuring the CE angle is as follows: First, connect the lower edge points L_lt and R_lt on both sides of the vertical midline to form the horizontal vector V of the human body. Then, connect the upper outer edge point uar of the acetabulum and the center of rotation point fhc of the femoral head on both sides of the vertical midline to form vector V. CE1 Draw a perpendicular vector V to vector V through the center of rotation point fhc of the femoral head. CE2 Calculate vector V CE1 With V CE2 The angle between the two sides is taken as the acute angle, and the CE angle of that side is taken as the acute angle. When the CE angle is less than 20°, hip dysplasia can be diagnosed on that side.

[0185] The method for automatically measuring the Tonnis angle is as follows: First, connect the lower edge points L_lt and R_lt of the teardrop located on both sides of the vertical midline to form the horizontal vector V of the human body. Then, connect the inner edge point tar of the weight-bearing area of ​​the acetabulum and the upper outer edge point uar of the acetabulum on both sides of the vertical midline to form vector V. T Calculate vector V T The angle between the angle and V is taken as the acute angle as the Tonnis angle on that side. When the Tonnis angle is greater than 10°, hip dysplasia can be diagnosed on that side.

[0186] The method for automatically measuring the Sharp angle is as follows: First, connect the lower edge points L_lt and R_lt of the teardrop located on both sides of the vertical midline to form the horizontal vector V of the human body. Then, connect the upper outer edge point uar of the acetabulum and the lower edge point lt of the teardrop on both sides of the vertical midline to form vector V. S Calculate vector V S The angle between the hip joint and V is taken as the acute angle of that side. When the acute angle is greater than 40°, hip dysplasia can be diagnosed on that side.

[0187] This allows for the automated calculation of measurement parameters in the initial image to achieve automated diagnosis of hip dysplasia, and provides a visual interface to improve the efficiency of hip dysplasia diagnosis.

[0188] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0189] Based on the same inventive concept, this application also provides a medical image processing apparatus for implementing the aforementioned medical image processing method. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in the one or more medical image processing apparatus embodiments provided below can be found in the above-described limitations of the medical image processing method, and will not be repeated here.

[0190] In one embodiment, such as Figure 8 As shown, a medical image processing device is provided, including: an acquisition module 810, an input module 820, and a determination module 830, wherein:

[0191] The acquisition module 810 is used to acquire the image to be identified corresponding to the initial image; the image to be identified is obtained by superimposing the first image and the second image; the first image is the image on one side of the first vertical central axis of the initial image determined according to the hip joint topological constraint conditions; the second image is the image obtained by flipping the image on the other side of the first vertical central axis.

[0192] The input module 820 is used to input the image to be recognized into a pre-trained key point recognition model to obtain a key point probability map corresponding to each target key point; the pre-trained key point recognition model is trained according to the hip joint topological constraints.

[0193] The determination module 830 is used to determine the predicted coordinates of the corresponding target key points in the initial image based on the probability maps of each key point.

[0194] In one embodiment, the apparatus further includes: a sample acquisition module, configured to acquire a training sample set corresponding to an initial sample image; each training sample in the training sample set includes a sample image and a mask image corresponding to the sample image; the initial sample image is marked with target coordinates corresponding to each of the target key points; the mask image is composed of binary mask rectangles corresponding to the target key points; a sample input module, configured to input the sample image into a key point recognition model to be trained, to obtain a prediction probability map corresponding to the sample image; the prediction probability map includes a background prediction probability map and a key point prediction probability map corresponding to each of the target key points; and a training module, configured to train the key point recognition model to be trained according to the prediction probability map, the mask image, and the hip joint topological constraints, to obtain the pre-trained key point recognition model.

[0195] In one embodiment, the sample image is obtained by superimposing a first sample image and a second sample image; the first sample image is an image on one side of the second vertical central axis of the initial sample image determined according to the hip joint topological constraints; the second sample image is an image obtained by flipping the image on the other side of the second vertical central axis; the target key points include a first key point set corresponding to the first sample image and a second key point set corresponding to the second sample image; the hip joint topological constraints include femoral head topological constraints and acetabular topological constraints; the training module is specifically used to determine a first difference between the predicted probability map and the mask image according to an initial loss function; determine a second difference between the first femoral head radius corresponding to the first key point set and the second femoral head radius corresponding to the second key point set according to the femoral head topological constraints; determine a third difference between the first acetabular radius corresponding to the first key point set and the second acetabular radius corresponding to the second key point set according to the acetabular topological constraints; and train the key point recognition model to be trained according to the first difference, the second difference, and the third difference to obtain the pre-trained key point recognition model.

[0196] In one embodiment, the training module is specifically configured to obtain the first femoral head radius based on the predicted sample coordinates of the target key points in the first key point set corresponding to the initial sample image; obtain the second femoral head radius based on the predicted sample coordinates of the target key points in the second key point set corresponding to the initial sample image; and determine the length difference between the first femoral head radius and the second femoral head radius as the second difference.

[0197] In one embodiment, the training module is specifically configured to fit a first arc corresponding to the target key points in the first key point set in the initial sample image, and a second arc corresponding to the target key points in the second key point set in the initial sample image; determine the radius of the first arc as the first acetabular radius, and determine the radius of the second arc as the second acetabular radius; and determine the length difference between the first acetabular radius and the second acetabular radius as the third difference.

[0198] In one embodiment, the femoral head topological constraint condition is used to constrain the second difference to be less than the femoral head radius difference threshold; the acetabular topological constraint condition is used to constrain the third difference to be less than the acetabular radius difference threshold; the training module is specifically used to update the model parameters of the key point recognition model to be trained according to the first difference, the second difference, and the third difference, and according to a preset parameter optimization algorithm; retrain the key point recognition model with updated model parameters, and when the first difference is less than the preset difference threshold, the second difference is less than the femoral head radius difference threshold, and the third difference is less than the acetabular radius difference threshold, determine that the trained key point recognition model meets the preset training termination condition, and obtain the pre-trained key point recognition model.

[0199] In one embodiment, the target key points include a first target key point located in the first image and a second target key point located in the second image; the key point probability map includes a first key point probability map corresponding to each of the first target key points and a second key point probability map corresponding to each of the second target key points; the determining module 830 is specifically used to, for any key point probability map, perform binarization processing on the key point probability map according to a preset probability threshold to obtain a corresponding binarized mask image; determine the connected component with the largest area in the corresponding binarized mask image to obtain a target connected component; for any first key point probability map, determine the coordinates of the geometric center of the target connected component as the predicted coordinates of the first target key point corresponding to the first key point probability map in the initial image; for any second key point probability map, flip the coordinates of the geometric center of the target connected component along the first vertical central axis to obtain the predicted coordinates of the second target key point corresponding to the second key point probability map in the initial image.

[0200] Each module in the aforementioned medical image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0201] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a medical image processing method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad located on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0202] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0203] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0204] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0205] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0206] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0207] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A medical image processing method, characterized in that, The method includes: Obtain the image to be identified corresponding to the initial image; the image to be identified is obtained by superimposing the first image and the second image; the first image is the image on one side of the first vertical central axis of the initial image determined according to the hip joint topological constraints; the second image is the image obtained by flipping the image on the other side of the first vertical central axis; the initial image is an image containing the hip joint of the target object. The image to be identified is input into a pre-trained keypoint recognition model to obtain a keypoint probability map corresponding to each target keypoint. The pre-trained keypoint recognition model is trained based on the hip joint topological constraints. The hip joint topological constraints are obtained based on the left-right symmetry of the hip joint's anatomical structure. The hip joint topological constraints include femoral head topological constraints and acetabular topological constraints. The femoral head topological constraints indicate that the dimensions of the left and right femoral heads should still be similar and symmetrical. The acetabular topological constraints indicate that the dimensions of the left and right acetabula should be similar and symmetrical. The predicted coordinates of the corresponding target key points in the initial image are determined based on the probability maps of each key point.

2. The method according to claim 1, characterized in that, The method further includes: Obtain a training sample set corresponding to the initial sample image; each training sample in the training sample set includes a sample image and a mask image corresponding to the sample image; the initial sample image is marked with the target coordinates corresponding to each target key point; the mask image is composed of binary mask rectangles corresponding to the target key points; The sample image is input into the key point recognition model to be trained to obtain the prediction probability map corresponding to the sample image; the prediction probability map includes a background prediction probability map and a key point prediction probability map corresponding to each of the target key points; The key point recognition model to be trained is trained based on the predicted probability map, the mask image, and the hip joint topological constraints to obtain the pre-trained key point recognition model.

3. The method according to claim 2, characterized in that, The sample image is obtained by superimposing the first sample image and the second sample image; the first sample image is an image of one side of the second vertical midline of the initial sample image determined according to the hip joint topological constraint conditions; The second sample image is an image obtained by flipping the image on the other side of the second vertical central axis; the target key points include a first key point set corresponding to the first sample image and a second key point set corresponding to the second sample image; the hip joint topological constraints include femoral head topological constraints and acetabular topological constraints; training the key point recognition model to be trained based on the predicted probability map, the mask image, and the hip joint topological constraints to obtain the pre-trained key point recognition model includes: Based on the initial loss function, a first difference between the predicted probability map and the mask image is determined; Based on the femoral head topological constraints, a second difference is determined between the first femoral head radius corresponding to the first set of key points and the second femoral head radius corresponding to the second set of key points. Based on the acetabular topological constraints, a third difference is determined between the first acetabular radius corresponding to the first key point set and the second acetabular radius corresponding to the second key point set. The key point recognition model to be trained is trained based on the first difference, the second difference, and the third difference to obtain the pre-trained key point recognition model.

4. The method according to claim 3, characterized in that, The step of determining the second difference between the first femoral head radius corresponding to the first keypoint set and the second femoral head radius corresponding to the second keypoint set based on the femoral head topological constraints includes: The first femoral head radius is obtained based on the predicted sample coordinates of the target key points in the first key point set corresponding to the initial sample image. The second femoral head radius is obtained based on the predicted sample coordinates of the target key points in the second key point set corresponding to the initial sample image; The length difference between the first femoral head radius and the second femoral head radius is determined as the second difference.

5. The method according to claim 3, characterized in that, The step of determining the third difference between the first acetabular radius corresponding to the first keypoint set and the second acetabular radius corresponding to the second keypoint set based on the acetabular topological constraints includes: Fit the first arc corresponding to the target key points in the first key point set in the initial sample image, and the second arc corresponding to the target key points in the second key point set in the initial sample image; The radius of the first arc is determined as the radius of the first acetabulum, and the radius of the second arc is determined as the radius of the second acetabulum; The length difference between the first acetabular radius and the second acetabular radius is determined as the third difference.

6. The method according to claim 3, characterized in that, The femoral head topological constraint condition is used to constrain the second difference to be less than the femoral head radius difference threshold; the acetabular topological constraint condition is used to constrain the third difference to be less than the acetabular radius difference threshold; the step of training the key point recognition model to be trained based on the first difference, the second difference, and the third difference to obtain the pre-trained key point recognition model includes: Based on the first difference, the second difference, and the third difference, the model parameters of the key point recognition model to be trained are updated according to a preset parameter optimization algorithm. The key point recognition model with updated model parameters is retrained. When the first difference is less than a preset difference threshold, the second difference is less than the femoral head radius difference threshold, and the third difference is less than the acetabular radius difference threshold, the trained key point recognition model is determined to meet the preset training termination condition, and the pre-trained key point recognition model is obtained.

7. The method according to claim 1, characterized in that, The target key points include a first target key point located in the first image and a second target key point located in the second image; the key point probability map includes a first key point probability map corresponding to each of the first target key points and a second key point probability map corresponding to each of the second target key points. Determining the predicted coordinates of the corresponding target key points in the initial image based on the probability maps of each key point includes: For any of the aforementioned key point probability maps, binarization processing is performed on the aforementioned key point probability map according to a preset probability threshold to obtain the corresponding binarized mask image; The target connected component is obtained by determining the connected component with the largest area in the corresponding binary mask image. For any of the first key point probability maps, determine the coordinates of the geometric center of the target connected component, which are used as the predicted coordinates of the first target key point corresponding to any of the first key point probability maps in the initial image; For any second key point probability map, the coordinates of the geometric center of the target connected region are flipped along the first vertical central axis to obtain the predicted coordinates of the second target key point corresponding to any second key point probability map in the initial image.

8. A medical image processing device, characterized in that, The device includes: The acquisition module is used to acquire the image to be identified corresponding to the initial image; the image to be identified is obtained by superimposing the first image and the second image; the first image is an image on one side of the first vertical central axis of the initial image determined according to the hip joint topological constraint conditions; the second image is an image obtained by flipping the image on the other side of the first vertical central axis; the initial image is an image containing the hip joint of the target object. An input module is used to input the image to be recognized into a pre-trained keypoint recognition model to obtain a keypoint probability map corresponding to each target keypoint. The pre-trained keypoint recognition model is trained based on the hip joint topological constraints. The hip joint topological constraints are obtained based on the left-right symmetry of the anatomical structure of the hip joint. The hip joint topological constraints include femoral head topological constraints and acetabular topological constraints. The femoral head topological constraints indicate that the dimensions of the left and right femoral heads should still be similar and symmetrical. The acetabular topological constraints indicate that the dimensions of the left and right acetabula should be similar and symmetrical. The determination module is used to determine the predicted coordinates of the corresponding target key points in the initial image based on the probability maps of each key point.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images