Lower extremity force line determination method and device, computer equipment and storage medium

Abstract

The application relates to a lower limb force line determination method and device, computer equipment and a storage medium, comprising: performing feature extraction on a bone X-ray film image through an encoder in a trained multi-task detection model to obtain image features of the bone X-ray film image; inputting the image features into a first decoder in the trained multi-task detection model to obtain key point position information in the bone X-ray film image, and inputting the image features into a second decoder in the trained multi-task detection model to obtain bone segmentation part position information in the bone X-ray film image; and determining a lower limb force line in the bone X-ray film image based on the key point position information and the bone segmentation part position information. The method can perform key point detection and bone part segmentation at one stage, which can improve the efficiency and the accuracy of key point detection and bone part segmentation.

Description

Technical Field

[0001] This application relates to the field of medical testing technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for determining lower limb force lines. Background Technology

[0002] Knee osteoarthritis is a disease based on degenerative pathological changes, often treated with clinical surgeries such as total knee replacement and high tibial osteotomy. The success of these surgeries and the evaluation of postoperative outcomes rely heavily on accurate measurement of lower limb alignment.

[0003] The lower limb alignment refers to the axis passing through the centers of the hip, knee, and ankle joints. Traditionally, healthcare professionals determine the lower limb alignment by manually drawing lines on the patient's X-ray films. However, this method heavily relies on the experience of healthcare professionals, is highly subjective, and makes it difficult to guarantee the accuracy of the results. Summary of the Invention

[0004] Therefore, it is necessary to address the technical problems of the above-mentioned methods for determining lower limb alignment, which heavily rely on the experience of medical personnel, are highly subjective, and make it difficult to guarantee the accuracy of the results. A method, device, computer equipment, computer-readable storage medium, and computer program product for determining lower limb alignment should be provided.

[0005] Firstly, this application provides a method for determining the force line of the lower limbs. The method includes:

[0006] The encoder in the trained multi-task detection model is used to extract features from the bone X-ray image to obtain the image features of the bone X-ray image.

[0007] The image features are input into the first decoder of the trained multi-task detection model to obtain the key point location information in the bone X-ray image; and the image features are input into the second decoder of the trained multi-task detection model to obtain the bone segmentation location information in the bone X-ray image.

[0008] Based on the key point location information and the bone segmentation location information, the lower limb force line in the bone X-ray image is determined.

[0009] In one embodiment, the step of inputting the image features into the first decoder of the trained multi-task detection model to obtain the key point location information in the skeletal X-ray image includes:

[0010] The image features are input into the first decoder of the trained multi-task detection model to obtain a probability distribution map corresponding to the skeletal X-ray image; the probability distribution map includes the probability that each pixel in the skeletal X-ray image belongs to a key point;

[0011] Based on the probability corresponding to each pixel, multiple key point regions are extracted from the probability distribution map as key point location information in the skeletal X-ray image.

[0012] In one embodiment, the step of extracting multiple key point regions from the probability distribution map based on the probability corresponding to each pixel includes:

[0013] Based on the probability corresponding to each pixel, target pixels with a probability value greater than a probability threshold are extracted from each pixel.

[0014] Perform connectivity analysis on the target pixels to form connected components from the target pixels that have connectivity relationships;

[0015] The identified connected domains are used as the multiple key point regions.

[0016] In one embodiment, the key point location information includes multiple key point regions and the probability that each pixel in each key point region belongs to a key point;

[0017] The step of determining the lower limb force line in the bone X-ray image based on the key point location information and the bone segmentation location information includes:

[0018] For each key point region in the key point location information, based on the probability that each pixel in the key point region belongs to a key point, the weighted centroid of the key point region is determined as the key point in the key point region;

[0019] Based on the key points in each key point region of the key point location information and the location information of the bone segmentation parts, the lower limb force line in the bone X-ray image is determined.

[0020] In one embodiment, determining the lower limb force line in the bone X-ray image based on key points in each key point region of the key point location information and the bone segmentation location information includes:

[0021] Detect the positional relationship between the key points in each key point region and the bone segmentation region corresponding to the bone segmentation region position information;

[0022] When the key points in each key point region are located in the bone segmentation region corresponding to the bone segmentation location information, the lower limb force line in the bone X-ray image is determined based on the key points in each key point region of the key point location information.

[0023] In one embodiment, the multi-task detection model is trained in the following manner:

[0024] Obtain a sample dataset, which includes sample bone X-ray images and key point location labels and bone segmentation location labels in the sample bone X-ray images;

[0025] Using the sample bone X-ray image as input, and the key point location labels and the bone segmentation location labels as supervision information, the multi-task detection model to be trained is trained to obtain the trained multi-task detection model.

[0026] In one embodiment, the step of training the multi-task detection model to be trained using the sample bone X-ray image as input and the key point location labels and the bone segmentation location labels as supervision information to obtain the trained multi-task detection model includes:

[0027] The sample bone X-ray image is input into the encoder of the multi-task detection model to be trained to obtain the sample image features of the sample bone X-ray image. The sample image features are input into the first decoder of the multi-task detection model to be trained to obtain the predicted key point location information. The sample image features are input into the second decoder of the multi-task detection model to be trained to obtain the predicted bone segmentation location information.

[0028] Based on the first difference information between the predicted key point location information and the key point location label, and the second difference information between the predicted bone segmentation location information and the bone segmentation location label, the multi-task detection model to be trained is trained to obtain the trained multi-task detection model.

[0029] In one embodiment, the key point location labels include positive sample labels and negative sample labels; the first difference information includes positive sample difference information and negative sample difference information;

[0030] The multi-task detection model to be trained is trained based on the first difference information between the predicted keypoint location information and the keypoint location label, and the second difference information between the predicted bone segmentation location information and the bone segmentation location label, to obtain a trained multi-task detection model, including:

[0031] The multi-task detection model to be trained is trained with the optimization objectives of reducing the positive sample difference information, increasing the negative sample difference information, and reducing the second difference information, so as to obtain the trained multi-task detection model.

[0032] In one embodiment, inputting the sample bone X-ray image into the multi-task detection model to be trained to obtain predicted key point location information and predicted bone segmentation location information includes:

[0033] The sample bone X-ray images are preprocessed to obtain preprocessed sample bone X-ray images; the preprocessing includes normalization, resampling, and cropping of the resampled data.

[0034] The preprocessed sample bone X-ray image is input into the multi-task detection model to be trained to obtain the predicted key point location information and the predicted bone segmentation location information.

[0035] Secondly, this application also provides a lower limb force line determination device. The device includes:

[0036] The extraction module is used to extract features from the bone X-ray image through the encoder in the trained multi-task detection model to obtain the image features of the bone X-ray image;

[0037] The prediction module is used to input the image features into the first decoder in the trained multi-task detection model to obtain the key point location information in the bone X-ray image, and to input the image features into the second decoder in the trained multi-task detection model to obtain the bone segmentation location information in the bone X-ray image.

[0038] The determination module is used to determine the lower limb force line in the bone X-ray image based on the key point location information and the bone segmentation location information.

[0039] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0040] The encoder in the trained multi-task detection model is used to extract features from the bone X-ray image to obtain the image features of the bone X-ray image.

[0041] The image features are input into the first decoder of the trained multi-task detection model to obtain the key point location information in the bone X-ray image; and the image features are input into the second decoder of the trained multi-task detection model to obtain the bone segmentation location information in the bone X-ray image.

[0042] Based on the key point location information and the bone segmentation location information, the lower limb force line in the bone X-ray image is determined.

[0043] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0044] The encoder in the trained multi-task detection model is used to extract features from the bone X-ray image to obtain the image features of the bone X-ray image.

[0045] The image features are input into the first decoder of the trained multi-task detection model to obtain the key point location information in the bone X-ray image; and the image features are input into the second decoder of the trained multi-task detection model to obtain the bone segmentation location information in the bone X-ray image.

[0046] Based on the key point location information and the bone segmentation location information, the lower limb force line in the bone X-ray image is determined.

[0047] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0048] The encoder in the trained multi-task detection model is used to extract features from the bone X-ray image to obtain the image features of the bone X-ray image.

[0049] The image features are input into the first decoder of the trained multi-task detection model to obtain the key point location information in the bone X-ray image; and the image features are input into the second decoder of the trained multi-task detection model to obtain the bone segmentation location information in the bone X-ray image.

[0050] Based on the key point location information and the bone segmentation location information, the lower limb force line in the bone X-ray image is determined.

[0051] The aforementioned method, apparatus, computer equipment, storage medium, and computer program product for determining lower limb alignment extract features from skeletal X-ray images using an encoder in a trained multi-task detection model. These features are then input into the first and second decoders of the multi-task detection model to obtain keypoint location information and bone segmentation location information from the skeletal X-ray images. Based on these keypoint and segmentation location information, the lower limb alignment in the skeletal X-ray images is further determined. This method improves the efficiency of skeletal X-ray image processing by training a multi-task detection model, which can perform keypoint detection and bone segmentation simultaneously in one stage. Furthermore, multi-task training of the detection model enhances the accuracy of its shared encoder, thereby improving the accuracy of keypoint detection and bone segmentation performed by the multi-task detection model, and ultimately increasing the accuracy of the lower limb alignment determined from the keypoint detection and bone segmentation results. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a method for determining the force line of the lower limbs in one embodiment;

[0053] Figure 2 This is a schematic diagram of the model structure of a multi-task detection model in one embodiment;

[0054] Figure 3a This is a schematic diagram of key points in a bone X-ray image in one embodiment;

[0055] Figure 3b This is a schematic diagram of segmented regions in a bone X-ray image in one embodiment;

[0056] Figure 3c This is a schematic diagram of key point location labels in a bone X-ray image in one embodiment;

[0057] Figure 4 This is a flowchart illustrating a multi-task detection model training method in one embodiment;

[0058] Figure 5 This is a schematic diagram of the model structure of a multi-task detection model in another embodiment;

[0059] Figure 6 This is a structural block diagram of a lower limb force line determination device in one embodiment;

[0060] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0063] In related technologies, medical staff determine the lower limb alignment by manually drawing lines on the patient's X-ray film. However, this method relies heavily on the experience of medical staff, is highly subjective, and makes it difficult to guarantee the accuracy of the results.

[0064] An effective way to solve this problem is to automatically extract lower limb force lines from X-ray images, reducing errors caused by manual drawing by medical staff while ensuring accuracy. Currently, some methods for automatically extracting lower limb force lines include: using object detection algorithms to first extract regions of interest, and then using ResNet to detect key points within those regions; or using deep learning algorithms based on convolutional neural networks to first detect coarse points in the femoral head, knee, and ankle joints, and then extracting six regions from both legs for key point detection; or using a deep neural network to first segment the femoral head, knee, and ankle joint regions, and then fitting key points within the segmented regions.

[0065] Although the above algorithms can automatically extract the force lines of the lower limbs, they are all two-stage algorithms, which consume a lot of computer resources and have a long inference time. At the same time, the location of key points is closely related to the location of the bones in the lower limbs, but none of these algorithms utilize the information of the entire lower limb skeleton.

[0066] This application considers that the tasks of key point detection on bone X-ray films and the segmentation of bones and prostheses are mutually influential and mutually reinforcing. Therefore, it proposes a deep neural network model that can simultaneously perform key point detection and segmentation of bones, prostheses, and other parts in one stage. This can improve both efficiency and accuracy of key point detection and bone segmentation, thereby improving the accuracy and efficiency of the lower limb force line determined by the key point detection results and bone segmentation results.

[0067] refer to Figure 1This is a flowchart illustrating a method for determining lower limb force lines, provided as an exemplary embodiment. This embodiment uses the application of this method to a terminal as an example for illustration. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster composed of multiple servers. In this embodiment, the method includes the following steps:

[0068] Step S110: The encoder in the trained multi-task detection model is used to extract features from the bone X-ray image to obtain the image features of the bone X-ray image.

[0069] In this application, the multi-task detection model refers to the key point detection task and the skeletal segmentation task.

[0070] Among them, image features can be high-order semantic information extracted from bone X-ray images.

[0071] In practice, a multi-task detection model can be pre-trained using a sample dataset, such as... Figure 2 The diagram shown is a schematic of the model structure of a multi-task detection model in an exemplary embodiment. The multi-task detection model includes an encoder, a first decoder, and a second decoder. In practical applications, the bone X-ray image to be processed can be input into the encoder of the trained multi-task detection model through the input layer. The encoder performs feature extraction processing on the bone X-ray image to obtain the image features of the bone X-ray image.

[0072] Step S120: Input the image features into the first decoder of the trained multi-task detection model to obtain the key point location information in the bone X-ray image; and input the image features into the second decoder of the trained multi-task detection model to obtain the bone segmentation location information in the bone X-ray image.

[0073] Here, the key point location information represents the location information of preset skeletal key points in the skeletal X-ray image. (Reference) Figure 3aThis is a schematic diagram of key points in a skeletal X-ray image, illustrating an embodiment. Taking the right leg as an example, the key points detected in the lower limb force line example may include, but are not limited to, the following 10 key points: A: femoral head; B: greater trochanter; C: lateral femoral condyle; D: medial femoral condyle; E: intercondylar fossa; F: lateral tibial condyle; G: medial tibial condyle; H: intercondylar eminence; I: lateral point of the ankle joint (azimuth line); J: medial point of the ankle joint (azimuth line).

[0074] The bone segmentation location information refers to the position of the preset segmentation region in the bone X-ray image. (Reference) Figure 3b This is a schematic diagram of the segmented parts in a bone X-ray image according to an embodiment. The segmented parts in this application may include: femur, tibia, fibula and prosthesis, etc.

[0075] The first decoder is used for keypoint analysis, and the second decoder is used for skeletal segmentation.

[0076] The first decoder and the second decoder have different functions but the same structure. Both the first decoder and the second decoder can include 4 decoder layers and 4 attention modules, and the 4 decoder layers and the 4 attention modules are connected at intervals.

[0077] In the specific implementation, after the encoder in the multi-task detection model extracts the image features of the bone X-ray image, these image features can be input into the first decoder and the second decoder in the multi-task detection model respectively. The first decoder processes the image features to obtain the key point location information of the bone in the bone X-ray image, and the second decoder processes the image features to obtain the location information of the segmented bone parts in the bone X-ray image.

[0078] Step S130: Based on the key point location information and the bone segmentation location information, determine the lower limb force line in the bone X-ray image.

[0079] In practice, after determining the location information of key points and bone segmentation parts in the bone X-ray image, the center of the hip joint, the center of the knee joint, and the center of the ankle joint in the bone X-ray image can be determined by combining the location information of key points and bone segmentation parts. A connecting line is drawn between the center of the hip joint, the center of the knee joint, and the center of the ankle joint as the force line of the lower limb in the bone X-ray image.

[0080] In the aforementioned method for determining lower limb alignment, the encoder in a trained multi-task detection model extracts features from skeletal X-ray images to obtain image features. These image features are then input into the first and second decoders of the multi-task detection model to obtain keypoint location information and bone segmentation location information from the skeletal X-ray images. Based on these keypoint and bone segmentation location information, the lower limb alignment in the skeletal X-ray images is further determined. This method improves the efficiency of skeletal X-ray image processing by training a multi-task detection model, which can perform keypoint detection and bone segmentation simultaneously in one stage. Furthermore, multi-task training of the detection model improves the accuracy of its shared encoder, thereby enhancing the accuracy of keypoint detection and bone segmentation performed by the multi-task detection model, and ultimately improving the accuracy of the lower limb alignment determined from the keypoint detection and bone segmentation results.

[0081] In an exemplary embodiment, the step S120 above, in which image features are input into the first decoder of the trained multi-task detection model to obtain key point location information in the skeletal X-ray image, can be specifically implemented through the following steps:

[0082] Step S121: Input the image features into the first decoder of the trained multi-task detection model to obtain the probability distribution map corresponding to the skeletal X-ray image; the probability distribution map includes the probability that each pixel in the skeletal X-ray image belongs to a key point;

[0083] Step S122: Based on the probability corresponding to each pixel, extract multiple key point regions from the probability distribution map as key point location information in the skeletal X-ray image.

[0084] Among them, the key points can be as follows: Figure 3a The preset key points shown include the femoral head, greater trochanter, lateral femoral condyle, medial femoral condyle, intercondylar fossa, lateral tibial condyle, medial tibial condyle, intercondylar eminence, lateral point of the ankle joint (orientation line), and medial point of the ankle joint (orientation line).

[0085] In the specific implementation, after training the multi-task detection model using a sample dataset, a first decoder is obtained that can be used for keypoint detection. Therefore, after the image features extracted by the encoder in the multi-task detection model are input into the first decoder, the first decoder can extract features that affect the keypoint detection results from these image features. Through feature analysis, a probability distribution map corresponding to the skeletal X-ray image is obtained. Then, the probability distribution map is post-processed to obtain the keypoint location information in the skeletal X-ray image. More specifically, the probability distribution map can determine the probability that each pixel in the skeletal X-ray image belongs to a preset keypoint. Based on the probability that each pixel belongs to a preset keypoint, multiple keypoint regions can be extracted from the probability distribution map as the keypoint location information in the skeletal X-ray image.

[0086] In this embodiment, the probability distribution map corresponding to the bone X-ray image is first obtained through the first decoder. Since the probability distribution map represents the probability that each pixel in the bone X-ray image belongs to a key point, some noise areas can be filtered out based on this probability, and multiple key point areas can be extracted as key point location information in the bone X-ray image, thereby ensuring the accuracy of the obtained key point location information.

[0087] Furthermore, in an exemplary embodiment, step S122 above, based on the probability corresponding to each pixel, extracts multiple key point regions from the probability distribution map, including:

[0088] Step S122A: Based on the probability corresponding to each pixel, extract the target pixels whose probability value is greater than the probability threshold from each pixel.

[0089] Step S122B: Perform connectivity analysis on the target pixels and form connected components from the target pixels that have connectivity relationships.

[0090] Step S122C: The identified multiple connected components are used as multiple key point regions.

[0091] In practice, a higher probability that a pixel belongs to a keypoint indicates a higher likelihood of that pixel being a keypoint; conversely, a lower probability indicates a lower likelihood of that pixel being a keypoint. The purpose of determining the keypoint region is to identify the keypoints themselves; therefore, the probability corresponding to keypoints within the keypoint region should be relatively high. Thus, a probability threshold can be set, and the probability of each pixel can be compared with this threshold. Based on the comparison results, pixels with probability values ​​greater than the threshold are selected from the pixels of the skeletal X-ray image as target pixels.

[0092] Furthermore, connectivity analysis can be performed on the selected target pixels to determine whether they are adjacent. If two target pixels are adjacent (e.g., target pixel A is above target pixel B), then these two target pixels are considered to be connected and can be grouped into the same connected component. Based on the connectivity analysis results of each target pixel, multiple connected components can be obtained. Each connected component is taken as a key point region, thus obtaining multiple key point regions as key point location information in the skeletal X-ray image.

[0093] In this embodiment, target pixels with a probability value greater than a probability threshold are extracted from each pixel based on the probability corresponding to each pixel. By performing connectivity analysis on the target pixels, target pixels with connectivity relationships are formed into connected components. The determined connected components are used as multiple key point regions to facilitate the subsequent determination of key points from each key point region.

[0094] In one exemplary embodiment, the key point location information includes multiple key point regions and the probability that each pixel in each key point region belongs to a key point;

[0095] In step S130 above, the lower limb force line in the bone X-ray image is determined based on the key point location information and the bone segmentation location information. This can be achieved through the following steps:

[0096] Step S131: For each key point region in the key point location information, based on the probability that each pixel in the key point region belongs to a key point, determine the weighted centroid of the key point region as the key point in the key point region.

[0097] Step S131: Based on the key points in each key point region of the key point location information and the location information of the bone segmentation parts, determine the lower limb force line in the bone X-ray image.

[0098] In the specific implementation, since each keypoint region contains multiple pixels, to ensure the confidence level of the determined keypoints, for each keypoint region in the keypoint location information, the pixels within that region can be weighted, and the weighted centroid at the determined location can be used as the keypoint of that keypoint region. More specifically, the probability of each pixel belonging to a keypoint in the keypoint region can be used as the weight corresponding to each pixel. The weight is then weighted with the position coordinates of each pixel in the keypoint region to obtain the weighted centroid, which is used as the keypoint of this keypoint region. Similarly, the keypoints in each keypoint region can be obtained using this method. Furthermore, based on the keypoints in each keypoint region and the location information of the bone segmentation parts, the center of the hip joint, the center of the knee joint, and the center of the ankle joint in the bone X-ray image can be determined. A connecting line is drawn between the centers of the hip joint, the center of the knee joint, and the center of the ankle joint as the force line of the lower limb in the bone X-ray image.

[0099] In this embodiment, the weighted centroid of the key point region is determined by the probability that each pixel in the key point region belongs to a key point, and this centroid is used as the key point in the key point region, which can further improve the accuracy of the determined key points.

[0100] Furthermore, in an exemplary embodiment, step S131, determining the lower limb force line in the skeletal X-ray image based on the key points in each key point region of the key point location information and the bone segmentation location information, further includes: detecting the positional relationship between the key points in each key point region and the bone segmentation location region corresponding to the bone segmentation location information; and, when the key points in each key point region are located in the bone segmentation location region corresponding to the bone segmentation location information, determining the lower limb force line in the skeletal X-ray image based on the key points in each key point region of the key point location information.

[0101] In practice, the key points corresponding to the key point location information should be located within the bone segmentation region corresponding to the bone segmentation region location information. This ensures the accuracy of the determined lower limb force line. Therefore, in this embodiment, before determining the lower limb force line based on the key point location information and the bone segmentation region location information, the positional relationship between the key points in each key point region and the bone segmentation region corresponding to the bone segmentation region location information is first detected. If a key point is not located within the corresponding bone segmentation region, it indicates a deviation in the detection result, and a prompt message can be output for staff to handle. If the key point is located within the corresponding bone segmentation region, the center of the hip joint, the center of the knee joint, and the center of the ankle joint can be determined based on the key points in each key point region of the key point location information. A connecting line is then drawn between the centers of the hip joint, the center of the knee joint, and the center of the ankle joint as the lower limb force line in the bone X-ray image.

[0102] In this embodiment, before determining the lower limb force line, the positional relationship between the key points in each key point region and the bone segmentation region corresponding to the bone segmentation region position information is first detected. When the key point is located in the corresponding bone segmentation region, the lower limb force line is determined based on the key point to ensure the accuracy of the determined lower limb force line.

[0103] In an exemplary embodiment, the multi-task detection model is trained in the following manner:

[0104] Step S210: Obtain the sample dataset, which includes sample bone X-ray images and key point location labels and bone segmentation location labels in the sample bone X-ray images.

[0105] Step S220: Using sample bone X-ray images as input and key point location labels and bone segmentation location labels as supervision information, train the multi-task detection model to be trained to obtain the trained multi-task detection model.

[0106] In practice, a subset of qualified skeletal X-ray images can be selected from historically captured images and used as sample skeletal X-ray images. These sample images are pre-labeled with keypoint locations and bone segmentation locations, allowing them to be used as supervisory information. The multi-task detection model is trained using these sample skeletal X-ray images as input. After obtaining the sample dataset, it can be divided into training and validation sets. The multi-task detection model is trained using the training set and validated using the validation set, adjusting the hyperparameters of the model: the number of training epochs and the learning rate.

[0107] Further, in an exemplary embodiment, step S220 further includes: inputting the sample bone X-ray image into the encoder of the multi-task detection model to be trained to obtain sample image features of the sample bone X-ray image; inputting the sample image features into the first decoder of the multi-task detection model to be trained to obtain predicted key point location information; inputting the sample image features into the second decoder of the multi-task detection model to be trained to obtain predicted bone segmentation location information; and training the multi-task detection model to be trained based on the first difference information between the predicted key point location information and the key point location label, and the second difference information between the predicted bone segmentation location information and the bone segmentation location label, to obtain the trained multi-task detection model.

[0108] In practice, during training, each frame of the sample bone X-ray image is sequentially input into the encoder of the multi-task detection model to be trained to obtain sample image features. These features are then input into the first decoder of the model to obtain predicted keypoint location information, and into the second decoder to obtain predicted bone segmentation location information. Thus, for each frame, both predicted keypoint location information and predicted bone segmentation location information can be obtained. Furthermore, the prediction results can be compared with the supervision information: the predicted keypoint location information is compared with the keypoint location labels of the frame to obtain first difference information, and the predicted bone segmentation location information is compared with the bone segmentation location labels to obtain second difference information. Based on the first and second difference information, a total loss value is obtained. The model parameters of the multi-task detection model are adjusted using this total loss value, and the model is trained to obtain the completed multi-task detection model.

[0109] Furthermore, the method for obtaining the total loss value based on the first and second difference information can be as follows: obtain the first loss value based on the first difference information; obtain the second loss value based on the second difference information; and obtain the total loss value based on the first and second loss values. Here, the loss value is used to measure the difference between the model's prediction and the actual region. The smaller the loss value, the better the model's prediction. The model training process is essentially the process of optimizing the loss value. In this embodiment, to optimize the loss value calculation between the keypoint detection task and the skeletal segmentation task, the Pareto optimization method is used during training. This means that the loss value of at least one task is improved without increasing the loss value of either task.

[0110] In practical applications, the training loss function used for skeletal segmentation tasks can be a weighted sum of multiple loss functions, such as Focal loss, Dice loss, and Lovasz-softmax loss. Since keypoint detection tasks are only used in the block-based regions, the loss function used can be Focal loss.

[0111] In this embodiment, the multi-task detection model is trained by predicting the first difference information between key point location information and key point location labels, and the second difference information between bone segmentation location information and bone segmentation location labels. This allows the trained multi-task detection model to perform bone segmentation while performing key point detection in one stage, thereby improving the processing efficiency of bone X-ray images. At the same time, training the detection model through multiple tasks can improve the accuracy of its shared encoder, thereby improving the accuracy of key point detection and bone segmentation performed by the multi-task detection model, and further improving the accuracy of the lower limb force line determined by the key point detection results and bone segmentation results.

[0112] In one exemplary embodiment, the key point location label includes positive sample label and negative sample label; the first difference information includes positive sample difference information and negative sample difference information;

[0113] In step S230 above, the multi-task detection model to be trained is trained based on the first difference information between the predicted key point location information and the key point location label, and the second difference information between the predicted bone segmentation location information and the bone segmentation location label, to obtain the trained multi-task detection model. This includes: training the multi-task detection model to be trained with the optimization objectives of reducing positive sample difference information, increasing negative sample difference information, and reducing the second difference information, to obtain the trained multi-task detection model.

[0114] In the specific implementation, considering that key points are pixel-sized points, numerous in X-ray films, and prone to errors, higher precision is required. Therefore, this embodiment employs a method of training with both positive and negative labels. More specifically, the key point detection task can be transformed into a multi-label segmentation task. The specific steps are as follows: using the x and y coordinates of the key point as the center point, the corresponding small blocks are extracted and used as positive sample labels. The image after block extraction is shown below. Figure 3cAs shown in the diagram. Simultaneously, small patches of the same size are randomly extracted from the bone X-ray images as negative sample labels. Both positive and negative sample labels are used together as keypoint location labels for training. During training, since the positive sample labels represent accurate keypoint location labels and the negative sample labels represent inaccurate keypoint location labels, the predicted keypoint location information obtained through the multi-task learning model should have the smallest possible difference from the accurate positive sample labels and the largest possible difference from the inaccurate negative sample labels. This ensures the prediction accuracy of the trained model. The second difference information is the difference between the predicted bone segmentation location information and the bone segmentation location label. Since the bone segmentation location label is an accurate label, the second difference information should be minimized. Therefore, when training the multi-task detection model, the optimization objectives should be to reduce the positive sample difference information, increase the negative sample difference information, and reduce the second difference information. This results in a fully trained multi-task detection model.

[0115] In this embodiment, the idea of ​​contrastive learning is used to train the key point location detection task, which can improve the key point detection effect of the trained multi-task detection model, reduce the dependence on training data, and improve the generalization of the model.

[0116] In an exemplary embodiment, step S220 above, which involves inputting sample bone X-ray images into the multi-task detection model to be trained to obtain predicted keypoint location information and predicted bone segmentation location information, includes:

[0117] Step S221: Preprocess the sample bone X-ray image to obtain the preprocessed sample bone X-ray image; the preprocessing includes normalization, resampling, and cropping of the resampled data.

[0118] Step S222: Input the preprocessed sample bone X-ray image into the multi-task detection model to be trained to obtain the predicted key point location information and the predicted bone segmentation location information.

[0119] In practice, after obtaining the sample dataset, it is necessary to preprocess it. This involves normalizing, resampling, and cropping the resampled bone X-ray images. The preprocessed bone X-ray images are then input into the multi-task detection model to be trained to obtain predicted keypoint locations and predicted bone segmentation locations.

[0120] More specifically, the preprocessing process for the sample bone X-ray images is as follows: the sample bone X-ray images are normalized to between 0 and 1 using a maximum-minimum normalization algorithm, and then resampled to a preset spatial size (e.g., 0.5mm*0.5mm). The resampled sample bone X-ray images are then randomly cropped, and the cropped sample bone X-ray images are used as the preprocessed sample bone X-ray images and input into the multi-task detection model to be trained for training.

[0121] In this embodiment, the sample bone X-ray images are preprocessed before training the multi-task detection model. This can improve the quality of the sample data input to the multi-task detection model, accelerate the convergence speed during model training, make the model training process more stable, and avoid gradient explosion or gradient vanishing. At the same time, it can simplify the information in the bone X-ray images to adapt to the training and learning of the multi-task detection model.

[0122] In one embodiment, such as Figure 4 The diagram illustrates a flowchart of a training method for a multi-task detection model, including the following steps:

[0123] (1) Collect sample datasets, including collecting sample bone X-ray images and determining key point location labels and bone segmentation location labels for sample bone X-ray images.

[0124] (2) Preprocess the sample bone X-ray images in the sample dataset to obtain the preprocessed sample bone X-ray images.

[0125] (3) Divide the sample dataset into a training set and a validation set. Train the multi-task detection model using the training set. During the training process, save the multi-task detection model whose loss value meets the conditions.

[0126] (4) Validate the multi-task detection model trained in step (3) using a validation set, and adjust the hyperparameters of the multi-task detection model, i.e., adjust the number of training rounds and the learning rate of the model, so as to obtain the multi-task detection model with optimal hyperparameters, which is then used as the trained multi-task detection model.

[0127] (5) After obtaining the probability distribution map and skeletal segmentation results output by the multi-task detection model, the probability distribution map is further post-processed to obtain the key point detection results.

[0128] (6) Determine the accuracy of the multi-task detection model based on the key point detection results and the skeletal segmentation results.

[0129] refer to Figure 5 This is a schematic diagram of the structure of a multi-task detection model shown in another embodiment, such as... Figure 5As shown, the encoder module in the multi-task detection model includes four encoder layers, which can extract features from skeletal X-ray images and obtain high-order semantic information.

[0130] The decoder module consists of four decoder layers and four attention modules. Each task has its own corresponding decoder; specifically, the keypoint detection task corresponds to the first decoder, and the skeletal segmentation task corresponds to the second decoder. The main function of the decoder is to collect high-order semantic information from the encoder and map this information to its respective task objective. In this model, the two decoders have identical structures.

[0131] The attention module in the decoder module can be composed of a two-layer convolutional neural network and a sigmoid activation function. By processing the multi-scale high-order semantic information extracted by the encoder, it learns features suitable for each task at each scale, and then applies them to the corresponding decoder for decoding.

[0132] The decoder module can also include a feature extraction module with multiple convolutional neural networks, which is used to extract features from the semantic information of the encoder and decoder at each scale.

[0133] Specifically, the processing procedures of each module in the multi-task detection model are described as follows:

[0134] Input layer: For the input skeletal X-ray image, shallow features are initially extracted through a convolutional neural network, followed by batch normalization and ReLU activation function.

[0135] Encoder Layer 1: For the shallow features extracted from the input layer, downsampling is performed through a convolutional neural network layer, followed by feature extraction through a feature extraction module, and the extracted features are added to the downsampled features as residual connections.

[0136] Encoder layers 2, 3, and 4: For each layer, the features of the previous layer are downsampled through a convolutional neural network layer, then the feature extraction module extracts the features, and the extracted features are added to the downsampled features to form a residual connection.

[0137] The attention module corresponding to layer 4 of the decoder: For the features of layer 4 of the encoder, they are first upsampled through a convolutional neural network layer. The upsampled features and the output features of layer 3 of the encoder are then processed by the attention module to obtain the attention features of layer 4 of the decoder.

[0138] Decoder Layer 4: The attention features of the decoder layer 4 are concatenated with the features of the encoder layer 3 to obtain fused features. Then, the features are extracted by the feature extraction module and added to the fused features as residual connections.

[0139] The attention module corresponding to layer 3 of the decoder: For the features of layer 4 of the decoder, they are first upsampled through a convolutional neural network layer. The upsampled features and the output features of layer 2 of the encoder are then processed by the attention module to obtain the attention features of layer 3 of the decoder.

[0140] Decoder Layer 3: The attention features of the decoder layer 3 are concatenated with the output features of the encoder layer 2 to obtain fused features. These features are then extracted by the feature extraction module and added to the fused features as residual connections.

[0141] The attention module corresponding to layer 2 of the decoder: For the features of layer 3 of the decoder, they are first upsampled through a convolutional neural network layer. The upsampled features and the output features of layer 1 of the encoder are then processed by the attention module to obtain the attention features of layer 2 of the decoder.

[0142] Decoder Layer 2: The attention features of the decoder layer 3 are concatenated with the features of the encoder layer 2 to obtain fused features. Then, the features are extracted by the feature extraction module and added to the fused features as residual connections.

[0143] The attention module corresponding to layer 1 of the decoder: For the features of layer 2 of the decoder, they are first upsampled through a convolutional neural network layer. The upsampled features and the features of the output layer are then processed by the attention module to obtain the attention features of layer 1 of the decoder.

[0144] Decoder Layer 1: The attention features of the decoder layer 1 are concatenated with the features of the output layer to obtain fused features. Then, the features are extracted by the feature extraction module and added to the fused features as residual connections.

[0145] Segmentation Output Layer: The features output by the segmentation decoder are processed by a convolutional neural network layer to output data for segmentation.

[0146] Keypoint Detection Output Layer: The features output by the keypoint detection decoder are processed by a convolutional neural network layer to output data for keypoint detection.

[0147] This application discloses a fully automated skeletal X-ray image recognition model based on a convolutional neural network. This model can undergo multi-task learning and training, meaning it can simultaneously perform keypoint detection and bone segmentation tasks, effectively improving the efficiency of both. Furthermore, training the detection model through multiple tasks enhances the accuracy of its shared encoder, thereby improving the accuracy of keypoint detection and bone segmentation performed by the multi-task detection model. In the example of lower limb alignment, this application enables medical personnel to conveniently and quickly measure lower limb alignment and identify corresponding bones, greatly improving diagnostic efficiency and facilitating widespread adoption.

[0148] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0149] Based on the same inventive concept, this application also provides a lower limb force line determination device for implementing the lower limb force line determination method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the lower limb force line determination device provided below can be found in the limitations of the lower limb force line determination method described above, and will not be repeated here.

[0150] In one embodiment, such as Figure 6 As shown, a lower limb force line determination device is provided, including: an extraction module 610, a prediction module 620, and a determination module 630, wherein:

[0151] The extraction module 610 is used to extract features from the bone X-ray image through the encoder in the trained multi-task detection model to obtain the image features of the bone X-ray image.

[0152] The prediction module 620 is used to input image features into the first decoder of the trained multi-task detection model to obtain key point location information in the bone X-ray image, and to input image features into the second decoder of the trained multi-task detection model to obtain bone segmentation location information in the bone X-ray image.

[0153] The determination module 630 is used to determine the lower limb force line in the bone X-ray image based on the key point location information and the bone segmentation location information.

[0154] In one embodiment, the prediction module 620 is specifically used to input image features into the first decoder in the trained multi-task detection model to obtain a probability distribution map corresponding to the skeletal X-ray image; the probability distribution map includes the probability that each pixel in the skeletal X-ray image belongs to a key point; based on the probability corresponding to each pixel, multiple key point regions are extracted from the probability distribution map as key point location information in the skeletal X-ray image.

[0155] In one embodiment, the prediction module 620 is further configured to extract target pixels with a probability value greater than a probability threshold from each pixel based on the probability corresponding to each pixel; perform connectivity analysis on the target pixels to form connected components from the target pixels with connected relationships; and use the determined connected components as multiple key point regions.

[0156] In one embodiment, the key point location information includes multiple key point regions and the probability that each pixel in each key point region belongs to a key point; the determination module 630 is specifically used to determine the weighted centroid of each key point region in the key point location information as a key point in the key point region based on the probability that each pixel in the key point region belongs to a key point; and to determine the lower limb force line in the bone X-ray image based on the key points in each key point region in the key point location information and the bone segmentation location information.

[0157] In one embodiment, the determining module 630 is further configured to detect the positional relationship between the key points in each key point region and the bone segmentation region corresponding to the bone segmentation location information; and, when the key points in each key point region are located in the bone segmentation region corresponding to the bone segmentation location information, determine the lower limb force line in the bone X-ray image based on the key points in each key point region in the key point location information.

[0158] In one embodiment, the above-mentioned apparatus further includes a training module for acquiring a sample dataset, which includes sample bone X-ray images and key point location labels and bone segmentation location labels in the sample bone X-ray images; using the sample bone X-ray images as input, and the key point location labels and bone segmentation location labels, the multi-task detection model to be trained is trained to obtain the trained multi-task detection model.

[0159] In one embodiment, the training module is further configured to input sample bone X-ray images into the encoder of the multi-task detection model to be trained to obtain sample image features of the sample bone X-ray images; input the sample image features into the first decoder of the multi-task detection model to be trained to obtain predicted key point location information; input the sample image features into the second decoder of the multi-task detection model to be trained to obtain predicted bone segmentation location information; and train the multi-task detection model to be trained based on the first difference information between the predicted key point location information and the key point location labels, and the second difference information between the predicted bone segmentation location information and the bone segmentation location labels, to obtain the trained multi-task detection model.

[0160] In one embodiment, the key point location label includes positive sample label and negative sample label; the first difference information includes positive sample difference information and negative sample difference information; the training module is further configured to train the multi-task detection model to be trained with the optimization objectives of reducing positive sample difference information, increasing negative sample difference information, and reducing second difference information, so as to obtain the trained multi-task detection model.

[0161] In one embodiment, the training module is further configured to preprocess the sample bone X-ray images to obtain preprocessed sample bone X-ray images; the preprocessing includes normalization processing, resampling processing, and cropping processing of the resampled data; the preprocessed sample bone X-ray images are input into the multi-task detection model to be trained to obtain predicted key point location information and predicted bone segmentation location information.

[0162] Each module in the aforementioned lower limb force line determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0163] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for determining lower limb force lines. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0164] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0165] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0166] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0167] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0169] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0170] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0171] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining the force line of the lower limb, characterized in that, The method includes: The encoder in the trained multi-task detection model extracts features from the bone X-ray image to obtain the image features of the bone X-ray image; the multi-task in the multi-task detection model includes a key point detection task that affects each other and a segmentation task for bone parts. The image features are input into the first decoder of the trained multi-task detection model to obtain a probability distribution map corresponding to the skeletal X-ray image. Multiple keypoint regions are extracted based on the probability distribution map as keypoint location information. Each pixel in the keypoint region has a probability greater than a probability threshold that it belongs to a keypoint. The process of determining the multiple keypoint regions includes: extracting target pixels with a probability value greater than a probability threshold from each pixel based on the probability corresponding to each pixel in the probability distribution map; performing connectivity analysis on the target pixels to form connected components from the connected target pixels; and using the determined connected components as the multiple keypoint regions. Furthermore, the image features are input into the second decoder in the trained multi-task detection model to obtain the location information of the bone segmentation parts in the bone X-ray image; For each key point region in the key point location information, the probability of each pixel in the key point region belonging to a key point is used as the weight of each pixel. The weight is weighted by the position coordinates of each pixel to obtain a weighted centroid. When the weighted centroid is located in the bone segmentation region corresponding to the bone segmentation location information, the weighted centroid is used as the key point constituting the lower limb force line to determine the lower limb force line in the bone X-ray image.

2. The method according to claim 1, characterized in that, The first decoder has the same structure as the second decoder.

3. The method according to claim 1, characterized in that, The method further includes: Detect the positional relationship between the weighted centroid point in each key point region and the bone segmentation region corresponding to the bone segmentation region position information; When the weighted centroid of each key point region is located in the bone segmentation region corresponding to the bone segmentation location information, the lower limb force line in the bone X-ray image is determined based on the weighted centroid of each key point region in the key point location information.

4. The method according to claim 1, characterized in that, The multi-task detection model is trained in the following manner: Obtain a sample dataset, which includes sample bone X-ray images and key point location labels and bone segmentation location labels in the sample bone X-ray images; Using the sample bone X-ray image as input, and the key point location labels and the bone segmentation location labels as supervision information, the multi-task detection model to be trained is trained to obtain the trained multi-task detection model.

5. The method according to claim 4, characterized in that, The process involves using the sample bone X-ray images as input, and the key point location labels and bone segmentation location labels as supervision information to train the multi-task detection model, resulting in the trained multi-task detection model, including: The sample bone X-ray image is input into the encoder of the multi-task detection model to be trained to obtain the sample image features of the sample bone X-ray image. The sample image features are input into the first decoder of the multi-task detection model to be trained to obtain the predicted key point location information. The sample image features are input into the second decoder of the multi-task detection model to be trained to obtain the predicted bone segmentation location information. Based on the first difference information between the predicted key point location information and the key point location label, and the second difference information between the predicted bone segmentation location information and the bone segmentation location label, the multi-task detection model to be trained is trained to obtain the trained multi-task detection model.

6. The method according to claim 5, characterized in that, The key point location labels include positive sample labels and negative sample labels; the first difference information includes positive sample difference information and negative sample difference information; The multi-task detection model to be trained is trained based on the first difference information between the predicted keypoint location information and the keypoint location label, and the second difference information between the predicted bone segmentation location information and the bone segmentation location label, to obtain a trained multi-task detection model, including: The multi-task detection model to be trained is trained with the optimization objectives of reducing the positive sample difference information, increasing the negative sample difference information, and reducing the second difference information, so as to obtain the trained multi-task detection model.

7. A device for determining the force line of a lower limb, characterized in that, The device includes: The extraction module is used to extract features from the bone X-ray image through the encoder in the trained multi-task detection model to obtain the image features of the bone X-ray image; the multi-task in the multi-task detection model includes a key point detection task that affects each other and a segmentation task for bone parts. The prediction module is used to input the image features into the first decoder of the trained multi-task detection model to obtain a probability distribution map corresponding to the skeletal X-ray image, extract multiple key point regions based on the probability distribution map as key point location information, wherein the probability of a pixel in each key point region belonging to a key point is greater than a probability threshold; and input the image features into the second decoder of the trained multi-task detection model to obtain the location information of the bone segmentation parts in the skeletal X-ray image; further, it is used to extract target pixels with a probability value greater than a probability threshold from each pixel based on the probability corresponding to each pixel in the probability distribution map; perform connectivity analysis on the target pixels to form connected components of target pixels with connected relationships; and use the determined multiple connected components as the multiple key point regions. The determination module is used to, for each key point region in the key point location information, use the probability that each pixel in the key point region belongs to a key point as the weight of each pixel; and to obtain a weighted centroid by weighting the weights with the position coordinates of each pixel. When the weighted centroid is located in the bone segmentation region corresponding to the bone segmentation location information, the weighted centroid is used as the key point constituting the lower limb force line to determine the lower limb force line in the bone X-ray image.

8. The apparatus according to claim 7, characterized in that, The first decoder has the same structure as the second decoder.

9. The apparatus according to claim 7, characterized in that, The device further includes a training module for acquiring a sample dataset, which includes sample bone X-ray images and key point location labels and bone segmentation location labels in the sample bone X-ray images. Using the sample bone X-ray image as input, and the key point location labels and the bone segmentation location labels as supervision information, the multi-task detection model to be trained is trained to obtain the trained multi-task detection model.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for determining the lower limb force line as described in any one of claims 1 to 6.

Images