Dynamic knee radiography image segmentation method, system and radiography machine

Abstract

The disclosure provides a dynamic knee X-ray image segmentation method, system and X-ray machine, and relates to the technical field of dynamic knee X-ray image segmentation. The method comprises the following steps: constructing a multi-segmentation region weight loss function according to a mask label image corresponding to a set dynamic knee X-ray image training set; training a plurality of set segmentation networks by using the set dynamic knee X-ray image training set, the mask label image corresponding to the set dynamic knee X-ray image training set and the multi-segmentation region weight loss function, to obtain a plurality of dynamic knee X-ray image segmentation models; evaluating the plurality of dynamic knee X-ray image segmentation models to determine a corresponding optimal dynamic knee X-ray image segmentation model; and segmenting one or more of a patella, a femur and a tibia and a patellar tendon of a dynamic knee X-ray image to be segmented based on the optimal dynamic knee X-ray image segmentation model.

Description

Technical Field

[0001] This disclosure relates to the field of dynamic knee joint X-ray image segmentation technology, and in particular to a dynamic knee joint X-ray image segmentation method, system and X-ray camera. Background Technology

[0002] The knee joint is one of the largest and most complex joints in the human body, serving as a primary support point for body weight. It plays a crucial role in maintaining stability and normal motor function when standing, walking, or running. For example, during walking, the flexion and extension of the knee joint work in conjunction with the leg muscles to propel the gait. Therefore, knee health is essential for daily life, requiring attention to appropriate exercise, weight control, avoiding overuse, and timely treatment of pain or discomfort.

[0003] The patella, femur, and tibia have a specific positional relationship within the knee joint, collectively forming its bony structural basis. Furthermore, the patellar tendon, as the primary soft tissue connecting the patella and tibia, works in conjunction with these three bone structures to enable basic knee joint movements, such as flexion and extension. In addition, it plays a crucial role in stabilizing the joint's bony structure during these movements. Therefore, the normal function of the patella, femur, tibia, and patellar tendon is essential for the stability of knee joint movement, making objective radiographic evaluation of these structures crucial.

[0004] Compared to computed tomography (CT), magnetic resonance imaging (MRI), and other imaging modalities, X-rays are better suited for the initial assessment of degenerative osteoarthritis and pain from car accident injuries. As the most widely used basic imaging method in orthopedics, X-ray imaging has become the preferred imaging device for initial knee examinations (such as detecting fractures, dislocations, and other abnormalities) and assessing clinical conditions such as osteoarthritis and bone destruction due to its widespread availability, low cost, and rapid convenience. However, X-rays, CT, and MRI are not conflicting but complementary for knee imaging. In particular, because X-rays are an overlapping imaging modality, there are blind spots in the visualization of intra-articular structures, and they remain insufficient in detecting subtle or hidden fractures. Therefore, minor fractures or bone injuries can be detected from three-dimensional (3D) CT images of the knee. From 3D CT images of the knee, bone injuries and tumors around the knee joint can be observed from multiple angles. However, 3D CT images of the knee joint show lower diagnostic sensitivity for changes in the muscles and ligaments around the knee joint, especially when cartilage changes or bone hyperplasia have not yet occurred. Similar to CT, MR imaging is also multi-parameter, multi-planar, and multi-directional, with higher resolution for soft tissues than for bone. Clinical examination of cartilage, meniscus, or muscle and ligament injuries, synovitis, and joint effusion can be performed using 3D knee MR images. However, MRI is expensive, time-consuming, and the equipment is complex to operate. Furthermore, it is currently not possible to obtain information about knee joint movement function from MRI.

[0005] Compared to dynamic knee radiography, static knee X-ray images captured at a single moment lack information about knee joint movement, which is detrimental to assessing knee joint function. Dynamic knee radiography can capture the movement trajectory of the knee joint and holds promise for analyzing knee joint function. However, for dynamic knee radiography, the main task of knee joint motion analysis is to accurately and automatically segment the patella, femur, tibia, and patellar tendon from multi-moment dynamic knee (knee / lower limb) X-ray images. Summary of the Invention

[0006] This disclosure proposes a dynamic knee joint X-ray image segmentation method, system, and corresponding technical solution for X-ray cameras.

[0007] According to one aspect of this disclosure, a method for segmenting dynamic knee X-ray images is provided, comprising: constructing a multi-segmentation region weight loss function based on mask label images corresponding to a set of dynamic knee X-ray images; wherein the mask label images include one or more of the following: patellar mask label, femoral mask label, tibial mask label, and patellar tendon mask label; training multiple set segmentation networks using the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to obtain multiple dynamic knee X-ray image segmentation models; evaluating the multiple dynamic knee X-ray image segmentation models to determine the corresponding optimal dynamic knee X-ray image segmentation model; and segmenting the dynamic knee X-ray image to be segmented by one or more of the patella, femur, tibia, and patellar tendon based on the optimal dynamic knee X-ray image segmentation model.

[0008] Preferably, the step of constructing a multi-segmentation region weight loss function based on the mask label image corresponding to the set of dynamic knee X-ray images includes: determining the areas of N mask segmentation regions corresponding to the mask label image in the set of dynamic knee X-ray images; calculating N ratios between the area of ​​each of the N mask segmentation regions and the total area of ​​the mask segmentation regions corresponding to the N mask segmentation regions; and determining the loss function weights of each mask segmentation region corresponding to the set segmentation network based on the N different products corresponding to any N-1 mask segmentation regions among the N mask segmentation regions.

[0009] Preferably, determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas includes: calculating N products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas; and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N products and the sum of the products corresponding to the N products.

[0010] Preferably, in determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas, determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N products and the sum of the products corresponding to the N products includes: extracting the product of the mask segmentation region areas that do not contain the weight to be determined from the N different products; calculating the ratio of the product of the mask segmentation region areas that contain the weight to be determined to the sum of the products corresponding to the N different products, and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network.

[0011] Preferably, determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas includes: multiplying the N masked segmentation region areas by a plurality of corresponding weight variables to be optimized to obtain N masked segmentation region weight areas; setting the N masked segmentation region weight areas equal to a set value to obtain the N weight variables to be optimized based on the set value and the N masked segmentation region areas; and determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the set value and the N masked segmentation region areas.

[0012] Preferably, in determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas, the determination of the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the defined value and the N masked segmentation region areas includes: calculating the sum of the weight variables to be optimized corresponding to the defined value and the N masked segmentation region areas to obtain the weight variables to be optimized and their expressions; and calculating the ratio between each of the N weight variables to be optimized and their expressions to determine the loss function weights for each masked segmentation region corresponding to the defined segmentation network.

[0013] Preferably, the evaluation of the plurality of dynamic knee X-ray image segmentation models to determine the corresponding optimal dynamic knee X-ray image segmentation model includes: determining multiple evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models under a set dynamic knee X-ray training set and a multi-segmentation region weight loss function, and multiple evaluation scales corresponding to multiple evaluation scales on a set dynamic knee X-ray test set, wherein the values ​​of these scales are positively proportional or positively correlated with the performance of the segmentation models, and multiple evaluation scales are inversely proportional or negatively correlated with the performance of the segmentation models; determining the first score vector corresponding to the plurality of dynamic knee X-ray image segmentation models under the multiple first evaluation scales and the second score vector corresponding to the plurality of dynamic knee X-ray image segmentation models under the multiple second evaluation scales; and evaluating the plurality of dynamic knee X-ray image segmentation models based on the first score vector and the second score vector to determine the corresponding optimal dynamic knee X-ray image segmentation model.

[0014] Preferably, the step of determining multiple evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models of multiple defined segmentation networks on a defined dynamic knee X-ray image training set and a defined loss function, and multiple evaluation scales corresponding to multiple evaluation scales on a defined dynamic knee X-ray image test set, wherein the values ​​of the scales are positively proportional or positively correlated with the performance of the segmentation models, and multiple evaluation scales are inversely proportional or negatively correlated with the performance of the segmentation models, includes: calculating multiple evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models of multiple defined segmentation networks on a defined dynamic knee X-ray image test set; determining the first set of evaluation scales corresponding to the multiple dynamic knee X-ray image segmentation models on the defined dynamic knee X-ray image test set, wherein the values ​​of the scales are positively proportional or positively correlated with the performance of the segmentation models, and the second set of evaluation scales are inversely proportional or negatively correlated with the performance of the segmentation models; and determining multiple first evaluation scales and multiple second evaluation scales based on the first set of evaluation scales and the second set of evaluation scales, respectively.

[0015] Preferably, determining the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple first comprehensive evaluation scales and the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple second comprehensive evaluation scales includes: sorting the multiple dynamic knee X-ray image segmentation models according to the set order of the multiple first comprehensive evaluation scales to determine the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple first comprehensive evaluation scales; and sorting the multiple dynamic knee X-ray image segmentation models according to the set order of the multiple second comprehensive evaluation scales to determine the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple second comprehensive evaluation scales.

[0016] Preferably, training multiple segmentation networks using the set of dynamic knee X-ray images and their corresponding masked labels, and the multi-segmentation region weight loss function includes: obtaining a set of dynamic knee X-ray images and the corresponding masked labels, and setting a maximum number of iterations; if the number of iterations of the segmentation network is less than the maximum number of iterations, the loss value between the set of dynamic knee X-ray images and their corresponding masked labels is less than a set loss value, and the first average intersection-union ratio (IUU) between the set of dynamic knee X-ray images and their corresponding masked labels at the first set number of iterations is greater than the first average IUU set value, then the second average IUU ratio of the smallest masked segmentation region among the N masked segmentation regions corresponding to the masked labels and the smallest masked segmentation region in the set of dynamic knee X-ray images is greater than the second average IUU set value, and then the segmentation network is controlled to stop training.

[0017] Preferably, the method further includes: if the number of iterations of the defined segmentation network reaches the defined maximum number of iterations, then controlling the defined segmentation network to stop training.

[0018] Preferably, the method further includes: if the number of iterations of the defined segmentation network is less than the set maximum number of iterations, and the loss value between the defined dynamic knee X-ray image training set and its corresponding mask label image is less than the set loss value, and the first average intersection-union ratio (IUU) between the defined dynamic knee X-ray image validation set and its corresponding mask label image at the first set number of iterations is greater than the set value of the first average IUU, and the second average IUU between the smallest mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest mask segmentation region in the defined dynamic knee X-ray image validation set is greater than the set value of the second average IUU, then calculate the third average IUU between the defined dynamic knee X-ray image validation set and its corresponding mask label image within the second set number of iterations; if the third average IUU is less than or equal to the set fluctuation value, then control the defined segmentation network to stop training.

[0019] According to one aspect of this disclosure, a dynamic knee X-ray image segmentation system is provided, comprising: a construction unit, configured to construct a multi-segmentation region weight loss function based on mask label images corresponding to a set of dynamic knee X-ray images; wherein the mask label images include one or more of patellar mask labels, femoral mask labels, tibial mask labels, and patellar tendon mask labels; a training unit, configured to train multiple set segmentation networks using the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to obtain multiple dynamic knee X-ray image (multi-time dynamic knee X-ray image) segmentation models; a determination unit, configured to evaluate the multiple dynamic knee X-ray image segmentation models and determine the corresponding optimal dynamic knee X-ray image segmentation model; and a segmentation unit, configured to segment the dynamic knee X-ray images to be segmented based on the optimal dynamic knee X-ray image segmentation model. The method comprises: segmenting knee X-ray images into one or more of the patella, femur, tibia, and patellar tendon; or, comprising: an electronic device; the electronic device being configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic knee X-ray image segmentation method; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic knee X-ray image segmentation method; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above-described dynamic knee X-ray image segmentation method; or, comprising: a computer program product, the computer program product being configured with a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the above-described dynamic knee X-ray image segmentation method.

[0020] According to one aspect of this disclosure, an X-ray camera is provided, comprising: the dynamic knee joint X-ray image segmentation system as described above; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the dynamic knee joint X-ray image segmentation method described above; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing the dynamic knee joint X-ray image segmentation method when executed by a processor; or, comprising: a computer program product configured with computer programs / instructions implementing the dynamic knee joint X-ray image segmentation method when executed by a processor.

[0021] In this disclosure, a dynamic knee joint X-ray imaging image segmentation method, system, and corresponding X-ray camera are proposed to solve the technical problem that the prior art cannot accurately and automatically segment one or more of the patella, femur, and tibia, as well as the patellar tendon.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0023] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0025] Figure 1 A flowchart illustrating a dynamic knee joint X-ray image segmentation method according to an embodiment of the present disclosure is shown. Detailed Implementation

[0026] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0027] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0028] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0030] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0031] In addition, this disclosure also provides a dynamic knee joint X-ray image segmentation device or system, electronic device, computer-readable storage medium, and program, all of which can be used to implement any of the dynamic knee joint X-ray image segmentation methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0032] Figure 1 A flowchart illustrating a dynamic knee joint X-ray image segmentation method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the dynamic knee X-ray image segmentation method includes: Step S101: Constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a set of dynamic knee X-ray images; wherein, the mask label images include one or more of the following: patellar mask label, femur mask label, tibia mask label, and patellar tendon mask label; Step S102: Training multiple set segmentation networks using the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to obtain multiple dynamic knee X-ray image (multi-time dynamic knee X-ray image) segmentation models; Step S103: Evaluating the multiple dynamic knee X-ray image segmentation models to determine the corresponding optimal dynamic knee X-ray image segmentation model; Step S104: Based on the optimal dynamic knee X-ray image segmentation model, segmenting the dynamic knee X-ray image to be segmented into one or more of the patella, femur, and tibia, and the patellar tendon. This invention aims to address the technical problem that existing technologies cannot accurately and automatically segment one or more of the patella, femur, and tibia, as well as the patellar tendon.

[0033] Step S101: Based on the mask label images corresponding to the set of dynamic knee joint X-ray images for training, construct a multi-segmentation region weight loss function; wherein, the mask label images include: one or more of the following: patellar mask label, femoral mask label, tibia mask label, and patellar tendon mask label. This addresses the technical problem that the uneven area ratio of the N mask segmentation regions corresponding to the mask label images causes smaller mask segmentation regions to not be sufficiently trained, thereby improving the segmentation performance of smaller mask segmentation regions.

[0034] For example, in a training set of multi-time dynamic knee X-ray images (multi-time dynamic knee X-ray imaging images), there is a technical problem that the area ratios of the patellar, femoral, tibial, and patellar tendon masked segments corresponding to the N masked segmentation areas of the masked label images are unbalanced, resulting in the smaller patellar tendon masked segmentation area not being sufficiently trained.

[0035] In embodiments of this disclosure, the step of constructing a multi-segmentation region weight loss function based on the mask label image corresponding to the set of dynamic knee X-ray images includes: determining the areas of N mask segmentation regions corresponding to the mask label image in the set of dynamic knee X-ray images; wherein N is greater than 1 or greater than or equal to 2, and N is a positive integer; calculating N ratios between the area of ​​each of the N mask segmentation regions and the total area of ​​the mask segmentation regions corresponding to the N mask segmentation regions; and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N different products corresponding to any N-1 mask segmentation regions among the N mask segmentation regions.

[0036] In embodiments of this disclosure, determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas includes: calculating N products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas; and determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on the N products and the sum of the products corresponding to the N products.

[0037] In the embodiments of this disclosure, in determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas, determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N products and the sum of the products corresponding to the N products includes: extracting the product of the mask segmentation region areas that do not contain the weight to be determined from the N different products; calculating the ratio of the product of the mask segmentation region areas that contain the weight to be determined to the sum of the products corresponding to the N different products, and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network.

[0038] For example, the areas of the N masked segmentation regions are configured as A, B, C, and D; calculate N ratios a (a=A / (A+B+C+D)), b (b=B / (A+B+C+D)), c (c=C / (A+B+C+D)), and d (d=D / (A+B+C+D)) for each of the N masked segmentation regions A, B, C, and D, respectively, to the total area of ​​the N masked segmentation regions (A+B+C+D); calculate N different products a*b*c, a*c*d, a*b*d, and b*c*d corresponding to any N-1 masked segmentation region areas among the N masked segmentation regions. Extract the products b*c*d, a*c*d, a*b*d, and a*b*c of the areas of the masked regions A, B, C, and D that do not contain the weight to be determined from the N different products a*b*c, a*c*d, a*b*d, and b*c*d, respectively. Then, calculate the sum of the products b*c*d, a*c*d, a*b*d, and a*b*c of the areas of the masked regions whose weights are to be determined, and the sum of the products of the N different products (a) and b*c*d. The ratio of *b*c+a*c*d+a*b*d+b*c*d is used to determine the loss function weights b*c*d / (a*b*c+a*c*d+a*b*d+b*c*d), a*c*d / (a*b*c+a*c*d+a*b*d+b*c*d), a*b*d / (a*b*c+a*c*d+a*b*d+b*c*d), and a*b*c / (a*b*c+a*c*d+a*b*d+b*c*d) for each mask segmentation region corresponding to the set segmentation network.

[0039] In embodiments of this disclosure, determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas includes: multiplying the N masked segmentation region areas by a plurality of corresponding weight variables to be optimized to obtain N masked segmentation region weight areas; setting the N masked segmentation region weight areas equal to a set value to obtain the N weight variables to be optimized based on the set value and the N masked segmentation region areas; and determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the set value and the N weight variables to be optimized based on the N masked segmentation region areas.

[0040] In the embodiments of this disclosure, when determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas, the determination of the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the defined value and the N masked segmentation region areas includes: calculating the sum of the weight variables to be optimized corresponding to the defined value and the N masked segmentation region areas to obtain the weight variables to be optimized and their expressions; and calculating the ratio between each of the N weight variables to be optimized and their expressions to determine the loss function weights for each masked segmentation region corresponding to the defined segmentation network.

[0041] For example, the areas of the N mask segmentation regions are configured as A, B, C, and D; the areas of the N mask segmentation regions are multiplied by the corresponding multiple weight variables to be optimized ε1, ε2, ε3, and ε4 respectively to obtain the weight areas of the N mask segmentation regions ε1*A, ε2*B, ε3*C, and ε4*D; the weight areas of the N mask segmentation regions ε1*A, ε2*B, ε3*C, and ε4*D are set to equal a predetermined value ε, resulting in the N weight variables to be optimized ε1=ε / A, ε2=ε / B, ε3=ε / C, and ε4=ε / D, represented by the predetermined value and the areas of the N mask segmentation regions; Calculate the sum of the N weight variables to be optimized, represented by the set value and the areas of the N masked segmentation regions, to obtain the weight variables to be optimized and the expression (ε1+ε2+ε3+ε4=ε / A+ε / B+ε / C+ε / D); calculate the ratio ε1 / (ε1+ε2+ε3+ε4) between each of the N weight variables to be optimized represented by the set value and the areas of the N masked segmentation regions and the weight variables to be optimized and the expression. )=(ε / A) / (ε / A+ε / B+ε / C+ε / D), ε2 / (ε1+ε2+ε3+ε4)=(ε / B) / (ε / A+ε / B+ε / C+ε / D), ε3 / (ε1+ε2+ε3+ε4)=(ε / C) / (ε / A+ε / B+ε / C+ε / D), ε4 / (ε1+ε2+ε3+ε4)=(ε / D) / (ε / A+ε / B+ε / C+ε / D), determine the loss function weight of each mask segmentation area corresponding to the set segmentation network.

[0042] Step S102: Using the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, train multiple set segmentation networks to obtain multiple dynamic knee X-ray images (multi-time dynamic knee X-ray images) segmentation models.

[0043] In embodiments of this disclosure, the plurality of segmentation networks are configured as one or more of FCN, UNet, UPerNet, SegFormer, PSPNet, DeepLabV3, or a combination thereof.

[0044] In the embodiments of this disclosure, the training set of the set dynamic knee X-ray images and their corresponding mask label images, and the weight loss function of the multi-segmentation region are used to train one or more of the following or a combination of multiple set segmentation networks: FCN, UNet, UPerNet, SegFormer, PSPNet, and DeepLabV3, to obtain multiple dynamic knee X-ray image (multi-time dynamic knee X-ray image) segmentation models.

[0045] Step S103: Evaluate the multiple dynamic knee X-ray image segmentation models and determine the corresponding optimal dynamic knee X-ray image segmentation model.

[0046] In the embodiments of this disclosure, the evaluation of the plurality of dynamic knee X-ray image segmentation models to determine the corresponding optimal dynamic knee X-ray image segmentation model includes: determining multiple first comprehensive evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models under a set dynamic knee X-ray training set and a multi-segmentation region weight loss function (set loss function), and their values ​​on a set dynamic knee X-ray test set, where the values ​​of these scales are directly proportional or positively correlated with the performance of the segmentation model (the larger the value, the better the performance of the segmentation model). Multiple second comprehensive evaluation scales that are inversely proportional or negatively correlated (smaller values ​​indicate better segmentation model performance) are used. First score vectors and second score vectors are determined for each of the multiple dynamic knee X-ray imaging segmentation models under the multiple first comprehensive evaluation scales and under the multiple second comprehensive evaluation scales, respectively. Based on the first and second score vectors, the multiple dynamic knee X-ray imaging segmentation models are evaluated, and the optimal dynamic knee X-ray imaging segmentation model is determined. This addresses the technical problem that improvements to dynamic knee X-ray imaging segmentation models often fail to achieve improvements across all evaluation scales, hindering the selection of appropriate dynamic knee X-ray imaging segmentation models. This approach aims to improve the performance of dynamic knee X-ray imaging segmentation models.

[0047] In this embodiment of the disclosure, the plurality of evaluation metrics are configured as any of the following: IoU (Intersection over Union), Dice, precision, recall, Hausdorff distance (HD) or median 95th Hausdorff distance (HD95), and average symmetric surface distance (ASSD); and / or, the plurality of first comprehensive evaluation metrics are configured as any one or more of IoU, Dice, precision, and recall; and / or, the plurality of second comprehensive evaluation metrics are any one or more of Hausdorff distance or median 95th Hausdorff distance and average symmetric surface distance.

[0048] In this embodiment of the disclosure, determining multiple evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models of multiple defined segmentation networks on a defined dynamic knee X-ray image training set and a defined loss function, and multiple evaluation scales corresponding to multiple evaluation scales on a defined dynamic knee X-ray image test set, wherein the values ​​of the scales are positively proportional or positively correlated with the performance of the segmentation models, and multiple evaluation scales are inversely proportional or negatively correlated with the performance of the segmentation models, includes: calculating multiple evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models of multiple defined segmentation networks on a defined dynamic knee X-ray image test set; determining the first set of evaluation scales corresponding to the multiple dynamic knee X-ray image segmentation models on the defined dynamic knee X-ray image test set, wherein the values ​​of the scales are positively proportional or positively correlated with the performance of the segmentation models, and the second set of evaluation scales are inversely proportional or negatively correlated with the performance of the segmentation models; and determining multiple first evaluation scales and multiple second evaluation scales based on the first set of evaluation scales and the second set of evaluation scales, respectively.

[0049] In this embodiment of the disclosure, determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively includes: summing the first set of evaluation scales corresponding to the multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray image test sets to determine multiple first comprehensive evaluation scales corresponding to the multiple dynamic knee X-ray image segmentation models; and summing the second set of evaluation scales corresponding to the multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray image test sets to determine multiple second comprehensive evaluation scales corresponding to the multiple dynamic knee X-ray image segmentation models.

[0050] In this embodiment of the disclosure, the step of determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively further includes: determining a first number corresponding to the evaluation scales in the first set of evaluation scales and a second number corresponding to the evaluation scales in the second set of evaluation scales respectively; averaging the multiple first comprehensive evaluation scales corresponding to the multiple dynamic knee joint X-ray image segmentation models using the first number respectively to obtain the final multiple first comprehensive evaluation scales; and averaging the multiple second comprehensive evaluation scales corresponding to the multiple dynamic knee joint X-ray image segmentation models using the second number respectively to obtain the final multiple second comprehensive evaluation scales.

[0051] In this embodiment of the disclosure, determining the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple first comprehensive evaluation scales and the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple second comprehensive evaluation scales includes: sorting the multiple dynamic knee X-ray image segmentation models according to the set order of the multiple first comprehensive evaluation scales (from largest to smallest or from smallest to largest) to determine the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple first comprehensive evaluation scales; and sorting the multiple dynamic knee X-ray image segmentation models according to the set order of the multiple second comprehensive evaluation scales (from largest to smallest or from smallest to largest) to determine the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple second comprehensive evaluation scales.

[0052] In this embodiment of the disclosure, the step of determining the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple first comprehensive evaluation scales and the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple second comprehensive evaluation scales further includes: determining the maximum score value among the first score vector and the second score vector; determining the score corresponding to the dynamic knee X-ray image segmentation model with the largest first comprehensive evaluation scale among the multiple first comprehensive evaluation scales as the maximum score value; and sorting the multiple dynamic knee X-ray image segmentation models according to a predetermined order (from largest to smallest or from smallest to largest) corresponding to the multiple first comprehensive evaluation scales based on the maximum score value and a predetermined score difference between any two adjacent scores in the first score vector. A dynamic knee X-ray image segmentation model is scored and assigned a first score vector under multiple first comprehensive evaluation scales. The score corresponding to the dynamic knee X-ray image segmentation model with the smallest second comprehensive evaluation scale among multiple second comprehensive evaluation scales is determined as the maximum score value. Based on the maximum score value and the set score difference between any two adjacent scores in the second score vector, the multiple dynamic knee X-ray image segmentation models, after being sorted according to a set order (from largest to smallest or from smallest to largest) corresponding to the multiple second comprehensive evaluation scales, are scored and assigned a second score vector under multiple second comprehensive evaluation scales.

[0053] In this embodiment of the disclosure, determining the maximum score value among the first score vector and the second score vector includes: determining the maximum score value among the first score vector and the second score vector based on the number corresponding to the plurality of dynamic knee X-ray image segmentation models.

[0054] For example, the maximum score value in the first and second score vectors is configured to be 50; or, the number of the multiple dynamic knee joint X-ray image segmentation models is configured to be 50, then the maximum score value in the first and second score vectors is configured to be 50. In this case, the set score difference between any two adjacent scores in the first and second score vectors is configured to be 1, and the determined first and second score vectors are configured as [50, 49, 48, …, 3, 2, 1] or [1, 2, 3, …, 48, 49, 50], respectively.

[0055] In this embodiment of the disclosure, the evaluation of the plurality of dynamic knee X-ray image segmentation models based on the first scoring vector and the second scoring vector includes: constructing a correlation relationship between the first scoring vector and the second scoring vector according to the plurality of dynamic knee X-ray image segmentation models; calculating the sum of the first correlation score and the second correlation score of the same dynamic knee X-ray image segmentation model under the correlation relationship, and the corresponding multiple joint scores; and evaluating the plurality of dynamic knee X-ray image segmentation models based on the multiple joint scores.

[0056] In this embodiment of the disclosure, the step of constructing the association between the first scoring vector and the second scoring vector based on the plurality of dynamic knee X-ray image segmentation models includes: extracting the first score and the second score corresponding to the same dynamic knee X-ray image segmentation model from the first scoring vector and the second scoring vector, respectively, to construct the association between the first scoring vector and the second scoring vector; or, determining the association labels of the first score in the first scoring vector and the second score in the second scoring vector corresponding to the same dynamic knee X-ray image segmentation model in the plurality of dynamic knee X-ray image segmentation models; and constructing the association between the first scoring vector and the second scoring vector based on the association labels.

[0057] In this embodiment of the disclosure, the evaluation of the multiple dynamic knee X-ray image segmentation models based on the multiple joint scores includes: sorting the multiple joint scores to obtain a sorted joint score vector; and evaluating the multiple dynamic knee X-ray image segmentation models based on the sorted joint score vector; wherein the joint score in the sorted joint score vector is proportional to the performance of the dynamic knee X-ray image segmentation model.

[0058] In this embodiment of the disclosure, determining multiple dynamic knee X-ray image segmentation models corresponding to multiple defined segmentation networks includes: determining target segmentation regions corresponding to the multiple dynamic knee X-ray image segmentation models; if the number of target segmentation regions is greater than 1, determining the areas of multiple masked segmentation regions corresponding to the masked label images in the training set of dynamic knee X-ray images corresponding to the multiple defined segmentation networks; if the difference or ratio between the largest and smallest masked segmentation region areas among the multiple masked segmentation region areas is greater than a set difference or a set ratio, optimizing the defined loss function corresponding to the multiple defined segmentation networks to obtain an optimized loss function; and training the multiple defined segmentation networks based on the dynamic knee X-ray image training set and the optimized loss function to determine multiple dynamic knee X-ray image segmentation models corresponding to the multiple defined segmentation networks.

[0059] In this embodiment, optimizing the set loss function corresponding to the plurality of set segmentation networks to obtain an optimized loss function includes: determining the areas of N masked segmentation regions corresponding to the masked label images in the set dynamic knee joint X-ray image training set corresponding to the set segmentation networks; wherein N is greater than 1 or greater than or equal to 2, and N is a positive integer; calculating N ratios of each masked segmentation region area to the total area of ​​the masked segmentation regions corresponding to the N masked segmentation regions; and determining the loss function weights of each masked segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation regions, thereby obtaining the optimized loss function. This addresses the technical problem that the unbalanced proportion of the N masked segmentation region areas corresponding to the masked label images causes smaller masked segmentation regions to not be sufficiently trained, thereby improving the segmentation performance of smaller masked segmentation regions.

[0060] In this embodiment of the disclosure, training multiple segmentation networks using the set of dynamic knee X-ray images and their corresponding masked labels, and the multi-segmentation region weight loss function, includes: obtaining a set of dynamic knee X-ray images and the masked labels corresponding to the set of dynamic knee X-ray images, and setting a maximum number of iterations; if the number of iterations of the segmentation network is less than the maximum number of iterations, and the loss value between the set of dynamic knee X-ray images and their corresponding masked labels is less than a set loss value, and the first average intersection-union ratio (IUU) between the set of dynamic knee X-ray images and their corresponding masked labels at the first set number of iterations is greater than a set first average IUU value, then the second average IUU value between the smallest masked segmentation region in the N masked segmentation regions corresponding to the masked labels and the smallest masked segmentation region (patellar tendon) in the set of dynamic knee X-ray images is greater than a set second average IUU value, then the segmentation network is controlled to stop training. To address at least one technical problem in multi-objective region segmentation models, such as overfitting, wasted computational resources and time, difficulty in capturing the optimal state of the multi-objective region segmentation model, and poor robustness of the multi-objective region segmentation model.

[0061] In this embodiment of the disclosure, the Intersection over Union (IoU) ratio is the ratio of the intersection area to the union area of ​​two regions, and is used to measure the similarity between the predicted segmentation result and the actual segmentation result.

[0062] In this embodiment of the disclosure, it further includes: if the number of iterations of the set segmentation network reaches the set maximum number of iterations, then controlling the set segmentation network to stop training.

[0063] In this embodiment of the disclosure, the method further includes: if the number of iterations of the defined segmentation network is less than a defined maximum number of iterations, and the loss value between the defined dynamic knee X-ray image training set and its corresponding mask label image is less than a defined loss value, and the first average intersection-union ratio (IUU) between the defined dynamic knee X-ray image validation set and its corresponding mask label image at the first defined number of iterations is greater than a defined first average IUU value, and the second average IUU between the smallest mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest mask segmentation region in the defined dynamic knee X-ray image validation set is greater than a defined second average IUU value, then a third average IUUU between the defined dynamic knee X-ray image validation set and its corresponding mask label image is calculated within the second defined number of iterations; if the third average IUUU is less than or equal to a defined fluctuation value, then the defined segmentation network is controlled to stop training.

[0064] In this embodiment of the disclosure, the method further includes: constructing a multi-segmentation region weight loss function based on the loss function weights of each of the N masked segmentation regions; and training the set segmentation network using the set dynamic knee joint X-ray image training set and its corresponding masked label images, as well as the multi-segmentation region weight loss function.

[0065] In this embodiment of the disclosure, the step of constructing a multi-segmentation region weighted loss function based on the loss function weights of each masked segmentation region includes: obtaining the base loss function corresponding to each masked segmentation region; multiplying the loss function weights of each masked segmentation region by their respective base loss functions and summing the results to construct the multi-segmentation region weighted loss function; wherein the base loss function corresponding to the loss function weights is configured as a cross-entropy loss function or a weighted cross-entropy loss function.

[0066] In this embodiment, the method further includes: summing the boundary loss functions corresponding to each masked segmentation region, or summing the corresponding multi-segmentation region boundary loss function after multiplying the loss function weights of each masked segmentation region by their respective boundary loss functions; summing the corresponding DICE loss functions corresponding to each masked segmentation region, or summing the corresponding multi-segmentation region DICE loss function after multiplying the loss function weights of each masked segmentation region by their respective DICE loss functions; constructing a multivariate loss function based on one or two of the segmentation region weight loss function, multi-segmentation region boundary loss function, or multi-segmentation region DICE loss function; and training the defined segmentation network using the defined dynamic knee joint X-ray image training set and its corresponding mask label images, and the multivariate loss function. By using the multivariate loss function corresponding to the multivariate loss function and one or two of the multivariate loss functions (multi-segmentation region weight loss function and boundary loss function or multi-segmentation region DICE loss function), the parameter training direction of the defined segmentation network is guided, thereby solving the technical problem that a univariate loss function cannot comprehensively train the defined segmentation network.

[0067] For example, firstly, one of the basic criteria for stopping network training is that the loss value between the knee multi-object segmentation images (set as a training set of dynamic knee X-ray images) and the ground truth (GT) (mask label / mask label image) in its training set is less than 0.2 (loss setting). Secondly, the overall segmentation performance of the knee multi-object segmentation model in four object regions (segmentation regions corresponding to the patella, femur, tibia, and patellar tendon) is evaluated every 50 iterations (first set iteration number) using a validation set and the knee multi-object segmentation model. Based on the loss value between the knee multi-object segmentation images and the GT in its training set being less than 0.2, a first average intersection-over-union (IoU) ratio greater than 0.8 (first average IoU setting) between the knee multi-object segmentation images and the GTs in its validation set is set as the second basic criterion for stopping network training. Thirdly, due to the small area of ​​the patellar tendon (the smallest mask segmentation region), it is difficult to achieve good performance. To provide more comprehensive training for the patellar tendon, a third criterion for stopping training is set: an average IoU greater than 0.85 (the second average intersection-union ratio setting) based on the patellar tendon object segmentation images and their ground truth (GT). Finally, after meeting these three criteria, network training can be stopped if the total average IoU between the four object segmentation regions and their GTs in the validation set remains unchanged for 10 consecutive iterations (the second set number of iterations). This means the relative fluctuation of the total average IoU between the four object segmentation regions and their GTs in the ten validation sets is less than or equal to the set fluctuation value of 0.001. If these three criteria are not met, network training stops when the maximum number of iterations is reached. After network training is complete, the segmentation model that performs best on the validation set is selected for testing on the test set.

[0068] In this embodiment of the disclosure, the method further includes: dividing the learning rate decay of the optimizer corresponding to the specified segmentation network into an initial learning rate decay phase, a stable learning rate decay period, and a fine-tuning learning rate decay phase; wherein, the initial learning rate decay phase uses warm-up steps to ensure rapid gradient descent in the initial stage of training the specified segmentation network while maintaining the stability of the specified segmentation network; the stable learning rate decay period uses polynomial decay to maintain a relatively stable learning rate; and the fine-tuning learning rate decay phase uses cosine learning rate decay.

[0069] For example, in network training, the learning rate is a key hyperparameter determining model convergence. Dynamic learning rates, by adaptively adjusting the learning rate during training, can achieve rapid convergence early on and fine-tune later, resulting in better model performance. In this study, the AdamW optimizer or a modified AdamW optimizer was used with an initial learning rate of 1e-4. The AdamW optimizer's learning rate decay includes an initial learning rate decay phase, a stable learning rate decay phase, and a fine-tuning learning rate decay phase. The initial learning rate decay phase, based on the initial learning rate, uses warmup steps, which provides rapid gradient descent in the initial stages of network training while maintaining stability. The stable learning rate decay phase uses polynomial decay, maintaining a relatively stable learning rate during long-term network training. The fine-tuning learning rate decay phase uses cosine learning rate decay, applied at the end of training.

[0070] In this embodiment of the disclosure, the multiple dynamic knee joint X-ray image segmentation models corresponding to the multiple set segmentation networks are configured as multi-target segmentation regions that are consistent with the N mask segmentation regions corresponding to the mask label image.

[0071] In this embodiment of the disclosure, the multi-target segmentation region that corresponds to the N mask segmentation regions of the mask label image is configured as two or more of the following: patella, femur, tibia and patellar tendon.

[0072] Step S104: Based on the optimal dynamic knee X-ray image segmentation model, segment the dynamic knee X-ray image to be segmented or processed into one or more of the patella, femur, tibia, and patellar tendon.

[0073] In this embodiment of the disclosure, before training multiple segmentation networks using a set of dynamic knee X-ray images and their corresponding mask labels to obtain multiple dynamic knee X-ray image segmentation models, the method includes: extracting each dynamic knee X-ray image from the set of dynamic knee X-ray images; wherein, each dynamic knee X-ray image includes: multiple knee X-ray images at multiple times corresponding to the same subject; calculating the similarity between multiple knee X-ray images corresponding to multiple preset labeled knee X-ray images in each dynamic knee X-ray image and the multiple preset labeled knee X-ray images; if the multiple image similarity is less than or equal to a preset image similarity, then the knee X-ray image corresponding to the preset image similarity is determined as a dynamic knee X-ray image to be labeled. To address the challenges of X-ray images with similarity levels below a preset threshold hindering the training of segmentation networks and preventing the acquisition of highly generalized segmentation models, the following measures are proposed: Firstly, determining the X-ray images to be labeled in the dynamic X-ray image training set according to preset time intervals introduces a high degree of subjectivity. Secondly, labeling all images in the dynamic X-ray image training set presents not only the technical problem of X-ray images with similarity levels below a preset threshold hindering the training of segmentation networks and preventing the acquisition of highly generalized segmentation models, but also presents at least one of the technical challenges inherent in the heavy physician annotation task. These measures aim to improve the generalization of the segmentation model.

[0074] In this embodiment of the disclosure, the image similarity can be configured as the mean square error corresponding to the average of the squared differences of the corresponding pixel values ​​of two images, extracting key points of the image and their descriptors, and calculating one or more of the similarity based on features, hash-based similarity, and statistical similarity (histogram similarity or structural similarity index) by matching descriptors.

[0075] In this embodiment of the disclosure, the first (first or first) preset labeled X-ray image among the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to the initial moment in each dynamic X-ray image; and / or, the last (last) preset labeled X-ray image among the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to the last moment in each dynamic X-ray image.

[0076] In this embodiment of the disclosure, it further includes: if the initial preset labeled X-ray image corresponding to the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to a non-initial time in each example of dynamic X-ray images, then the similarity between the initial preset labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the plurality of times before the initial preset labeled X-ray image is also calculated.

[0077] In this embodiment of the disclosure, it further includes: if the last preset-labeled X-ray image corresponding to the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to a non-last moment in each example of dynamic X-ray images, then the similarity between the last preset-labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the times after the last preset-labeled X-ray image is also calculated.

[0078] In this embodiment of the disclosure, the preset dynamic X-ray image training set is configured as a dynamic knee X-ray image training set or a dynamic chest X-ray image training set. Specifically, each dynamic X-ray image in the dynamic knee X-ray image training set or the dynamic chest X-ray image training set is a multi-time knee X-ray image or chest X-ray image of the same subject.

[0079] In the embodiments disclosed herein, those skilled in the art can configure the preset image similarity based on actual needs. For example, the preset image similarity can be configured to 80% or other values ​​less than 100%.

[0080] In this embodiment of the disclosure, the method further includes: determining dynamic X-ray images to be labeled in a set of dynamic X-ray imaging training images using the dynamic X-ray image labeling method described above; labeling the regions to be segmented in the dynamic X-ray images to obtain corresponding mask label images of the regions to be segmented (mask label images); and training a set segmentation network based on the dynamic X-ray images to be labeled and their corresponding mask label images of the regions to be segmented to obtain a segmentation model.

[0081] This disclosure proposes a method for determining the bilateral boundary lines of the femur, comprising: extracting a femoral mask boundary image corresponding to the femoral mask in a femoral region mask image; and determining one or both of the first and second boundary lines among the bilateral boundary lines of the femur or femoral cortex based on the non-zero coordinates of the femoral mask boundary in the femoral mask boundary image. This addresses at least one of the following technical problems: the lack of an effective quantitative method for determining the bilateral boundary lines and femoral midline, which hinders the auxiliary diagnosis of fractures and dislocations, assessment of joint development and deformities, guidance of surgical planning and reduction, and monitoring of treatment effects and rehabilitation progress; the inherent subjectivity and difficulty in ensuring objectivity in manually determining the bilateral boundary lines and femoral midline; and the increased workload for radiologists due to the already demanding nature of the procedure.

[0082] In this embodiment of the disclosure, the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image includes: eroding the femoral mask in the femoral region mask image to obtain a femoral mask erosion image; subtracting the femoral mask erosion image from the femoral region mask image to obtain the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image.

[0083] In this embodiment of the disclosure, determining one or both of the first and second boundary lines among the boundary lines on both sides of the femur based on the non-zero coordinates of the femur mask boundary of the femur mask boundary image includes: using polar coordinate Hough transform to determine the first boundary line on one side of the femur corresponding to the first straight line (first polar coordinate equation) based on the first polar coordinate equation corresponding to the first preset distance range and the first preset angle range, and the non-zero coordinates of the first side femur mask boundary corresponding to the femur mask boundary image; and / or, using polar coordinate Hough transform to determine the second boundary line on the other side of the femur corresponding to the second straight line (second polar coordinate equation) based on the second polar coordinate equation corresponding to the second preset distance range and the second preset angle range, and the non-zero coordinates of the second side femur mask boundary corresponding to the femur mask boundary image.

[0084] In this embodiment of the disclosure, determining the first preset distance range includes: using the diagonal length of the femoral mask boundary image to determine the first preset distance range corresponding to the first distance from the origin of the rectangular coordinate system corresponding to the first polar coordinate parameter in the first polar coordinate equation to the first straight line.

[0085] In this embodiment of the disclosure, determining the second preset distance range includes: using the diagonal length of the femoral mask boundary image to determine the second preset distance range corresponding to the second distance from the origin of the rectangular coordinate system corresponding to the third polar coordinate parameter in the second polar coordinate equation to the second straight line.

[0086] In this embodiment of the disclosure, the first preset included angle range is configured to be 0 to 180 degrees; the first preset distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs, respectively.

[0087] In this embodiment of the disclosure, the second preset included angle range is configured to be 0 to 180 degrees and / or, and the second preset distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs, respectively.

[0088] In this embodiment of the disclosure, determining one or both of the first and second boundary lines among the boundary lines on both sides of the femur based on the non-zero coordinates of the femur mask boundary in the femur mask boundary image includes: determining a first distance range corresponding to the first distance from the origin of the rectangular coordinate system corresponding to the first polar coordinate parameter in the first polar coordinate equation to the first straight line using the diagonal length of the femur mask boundary image; configuring a first angle range corresponding to the first angle between the perpendicular line from the origin of the rectangular coordinate system corresponding to the second polar coordinate parameter in the first polar coordinate equation to the first straight line and the x-axis; and using the polar coordinate Hough transform to determine the boundary lines based on the first distance range and the first angle range corresponding to the first polar coordinate equation and the non-zero coordinates of the first side femur mask boundary corresponding to the femur mask boundary image. First, determine the first boundary line on one side of the femur corresponding to the first straight line (first polar coordinate equation); and / or, determine the second distance range corresponding to the second distance from the origin of the rectangular coordinate system corresponding to the third polar coordinate parameter in the second polar coordinate equation to the second straight line using the diagonal length of the femoral mask boundary image; configure the second angle range corresponding to the second included angle between the perpendicular line from the origin of the rectangular coordinate system corresponding to the fourth polar coordinate parameter in the second polar coordinate equation to the second straight line and the x-axis; based on the second distance range and the second included angle range corresponding to the second polar coordinate equation and the non-zero coordinates of the second side femoral mask boundary corresponding to the femoral mask boundary image, determine the second boundary line on the other side of the femur corresponding to the second straight line (second polar coordinate equation) using polar coordinate Hough transform.

[0089] In this embodiment of the disclosure, the first included angle range is configured to be 0 to 180 degrees; and / or, the first distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs respectively; and / or, the second included angle range is configured to be 0 to 180 degrees; and / or, the second distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs respectively.

[0090] In this embodiment of the disclosure, before extracting the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image, the method includes: using a knee X-ray imaging image segmentation model to perform femoral segmentation on the knee X-ray imaging image to be processed, thereby obtaining the femoral region mask image corresponding to the knee X-ray imaging image to be processed.

[0091] In this embodiment of the disclosure, determining the knee X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset knee X-ray image training set or a preset dynamic knee X-ray image training set; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more mask labels such as a femoral mask label, a patellar mask label, a tibial mask label, and a patellar tendon mask label; and training a preset segmentation network using the preset knee X-ray image training set or the preset dynamic knee X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain the knee X-ray image segmentation model or the dynamic knee X-ray image segmentation model.

[0092] In this embodiment of the disclosure, determining the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset knee joint X-ray image training set or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more mask labels such as a femoral mask label, a patellar mask label, a tibial mask label, and a patellar tendon mask label; and using the preset knee joint X-ray image training set or the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function, performing multiple preset segmentation networks... Training is performed to obtain multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models; the multiple knee joint X-ray image segmentation models or the multiple dynamic knee joint X-ray image segmentation models are evaluated to determine the corresponding optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model; based on the optimal knee joint X-ray image segmentation model or the optimal dynamic knee joint X-ray image segmentation model, the knee joint X-ray image or dynamic knee joint X-ray image to be processed is segmented into one or more segments, such as femoral segmentation or segmentation of the femur and patella, tibia, and patellar tendon.

[0093] This disclosure also proposes a method for determining the femoral midline, comprising: using the above-described method for determining the boundary lines on both sides of the femur to determine a first boundary line and a second boundary line among the boundary lines on both sides of the femur or the femoral cortex; and determining the femoral midline based on the first boundary line and the second boundary line.

[0094] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line and the second boundary line includes: determining a first direction vector corresponding to the first boundary line and a second direction vector corresponding to the second boundary line; performing dot products on the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector; if the first product is less than a first preset value, inverting the first direction vector in the direction; otherwise, not processing the first direction vector; if the second product is less than a second preset value, inverting the second direction vector in the direction; otherwise, not processing the second direction vector; adding the first direction vector corresponding to the inverted or unprocessed and the second direction vector corresponding to the inverted or unprocessed to obtain the bisecting line direction vector; and determining the femoral midline based on the first boundary line, the second boundary line, and the bisecting line direction vector.

[0095] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line, the second boundary line, and the direction vector of the bisector includes: if the first boundary line and the second boundary line intersect, then determining the femoral midline corresponding to the third polar coordinate equation based on the intersection point, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector; otherwise, determining the femoral midline corresponding to the third polar coordinate equation based on the midpoint of the perpendicular segment between the first boundary line and the second boundary line, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector.

[0096] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the direction vector corresponding to the femoral region mask image.

[0097] In this embodiment of the disclosure, the values ​​corresponding to the first preset value and the second preset value are the same or different; and / or, the values ​​corresponding to the first preset value and the second preset value are respectively configured to 0.

[0098] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector, the method includes: calculating the femoral principal component direction vector corresponding to the non-zero point coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the femoral principal component direction vector to obtain a corrected femoral principal component direction vector; and performing dot products of the first direction vector and the second direction vector with the corrected femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector.

[0099] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the femoral mask in the femoral region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the femoral principal component direction vector of the corresponding non-zero point coordinate to obtain multiple projection values ​​of the femoral principal component direction vector in the current principal component direction; determining the first non-zero point coordinate corresponding to the minimum projection value and the second non-zero point coordinate corresponding to the maximum projection value among the multiple projection values; calculating the first distance and the second distance between the center point of the femoral region mask image and the first non-zero coordinate and the second non-zero coordinate respectively; determining the direction corresponding to the femoral principal component direction vector based on the first distance and the second distance; and correcting the femoral principal component direction vector based on the direction to obtain the corrected femoral principal component direction vector.

[0100] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the first distance and the second distance includes: if the first distance is less than the second distance, the direction is from the femoral knee joint to the hip joint; otherwise, the direction is from the hip joint to the femoral knee joint.

[0101] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector based on the direction to obtain a corrected femoral principal component direction vector includes: if the direction is from the femoral knee joint to the hip joint, then the femoral principal component direction vector is not corrected; if the direction is from the hip joint to the femoral knee joint, then the direction vector is inverted in the direction to obtain a corrected femoral principal component direction vector.

[0102] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the femoral principal component direction vector corresponding to the femoral region mask image.

[0103] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the modified femoral principal component direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the modified femoral principal component direction vector corresponding to the femoral region mask image.

[0104] This disclosure proposes a femoral direction vector correction method, comprising: calculating the femoral principal component direction vector corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero coordinates; and correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector. This addresses at least one of the following technical problems: the lack of femoral direction vector correction during the automatic determination of the femoral or femoral cortex boundary lines results in insufficient accuracy for femoral or femoral cortex boundary line detection; the need to avoid femoral midline detection failure; the high subjectivity of manually annotating the tibial boundary line and tibial midline; and the need for further improvement in the accuracy of current automatic detection of tibial boundary lines and tibial midline.

[0105] In this embodiment of the disclosure, calculating the femoral principal component direction vector corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image includes: processing the non-zero coordinates of the femoral mask in the femoral region mask image using a principal component analysis algorithm to obtain the femoral principal component direction vector corresponding to the non-zero coordinates.

[0106] In this embodiment of the disclosure, calculating the center point corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image includes: calculating the average value of the abscissa of the non-zero abscissa and the average value of the ordinate of the non-zero ordinate corresponding to the femoral mask, and determining the center point corresponding to the non-zero coordinates of the femoral mask.

[0107] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the femoral mask in the femoral region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the femoral principal component direction vector of the corresponding non-zero point coordinate to obtain multiple projection values ​​of the femoral principal component direction vector in the current principal component direction; determining the first non-zero point coordinate corresponding to the minimum projection value and the second non-zero point coordinate corresponding to the maximum projection value among the multiple projection values; calculating the first distance and the second distance between the center point of the femoral region mask image and the first non-zero coordinate and the second non-zero coordinate respectively; determining the direction corresponding to the femoral principal component direction vector based on the first distance and the second distance; and correcting the femoral principal component direction vector based on the direction to obtain the corrected femoral principal component direction vector.

[0108] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the first distance and the second distance includes: if the first distance is less than the second distance, the direction is from the femoral knee joint to the femoral hip joint; otherwise, the direction is from the femoral hip joint to the femoral knee joint.

[0109] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector based on the direction to obtain a corrected femoral principal component direction vector includes: if the direction is from the femoral knee joint to the femoral hip joint, then the femoral principal component direction vector is not corrected; if the direction is from the femoral hip joint to the femoral knee joint, then the femoral principal component direction vector is inverted in the direction to obtain a corrected femoral principal component direction vector.

[0110] In this embodiment of the disclosure, before calculating the femoral principal component direction vector corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero coordinates, the method includes: using a knee X-ray imaging image segmentation model to perform femoral segmentation on the knee X-ray imaging image to be processed, thereby obtaining the femoral region mask image corresponding to the knee X-ray imaging image to be processed.

[0111] In this embodiment of the disclosure, determining the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more of the following mask labels: femoral mask label, patellar mask label, tibia mask label, and patellar tendon mask label; and training a preset segmentation network using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain the dynamic knee joint X-ray image segmentation model.

[0112] In this embodiment of the disclosure, determining the knee joint X-ray image segmentation model includes: determining the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set corresponding to mask label images; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more mask labels such as a femoral mask label and a patellar mask label, a tibial mask label, and a patellar tendon mask label; using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, the multi-segmentation region weight loss function... The system trains multiple preset segmentation networks to obtain multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models; it evaluates the multiple knee X-ray image segmentation models or the multiple dynamic knee X-ray image segmentation models to determine the optimal knee X-ray image segmentation model or the optimal dynamic knee X-ray image segmentation model; based on the optimal knee X-ray image segmentation model or the optimal dynamic knee X-ray image segmentation model, it performs femoral segmentation or one or more segmentations of the femur, patella, tibia, and patellar tendon on the knee X-ray image or dynamic knee X-ray image to be segmented.

[0113] This disclosure also proposes a method for processing X-ray images of the knee joint, comprising: determining one or two boundary lines, namely a first boundary line and a second boundary line, in the boundary lines on both sides of the femur or femoral cortex, based on the non-zero coordinates of the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image; determining a modified femoral principal component direction vector corresponding to the femoral principal component direction vector using the femoral direction vector correction method described above; and determining the femoral midline based on the modified femoral principal component direction vector, the first boundary line, and the second boundary line.

[0114] In this embodiment of the disclosure, determining the femoral midline based on the modified femoral principal component direction vector, the first boundary line, and the second boundary line includes: determining a first direction vector corresponding to the first boundary line and a second direction vector corresponding to the second boundary line; performing dot products on the first direction vector and the second direction vector with the modified femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector; if the first product is less than a first preset value, inverting the first direction vector in the direction; otherwise, not processing the first direction vector; if the second product is less than a second preset value, inverting the second direction vector in the direction; otherwise, not processing the second direction vector; adding the first direction vector corresponding to the inverted or unprocessed and the second direction vector corresponding to the inverted or unprocessed to obtain the bisecting line direction vector; and determining the femoral midline based on the first boundary line, the second boundary line, and the bisecting line direction vector.

[0115] Calculate the direction vectors (principal component direction vectors) corresponding to the non-zero coordinates of the femoral mask (femurmask) in the femoral region mask image or femoral region mask image. Specifically, this includes: extracting the non-zero coordinates of the femoral mask (femurmask), calculating the direction vectors (femoral principal component direction vectors) corresponding to the non-zero coordinates of the femoral mask (femoral principal component direction vectors) using the principal component analysis (PCA) algorithm, and normalizing the direction vectors to obtain the normalized direction vector (femoral normalized direction vector) PCA_femurvec. (Note that the direction vector or normalized direction vector PCA_femurvec corresponding to the non-zero coordinates obtained at this time may point from the side of the femur near the patella to the side near the hip joint, or it may point from the side of the femur near the hip joint to the side of the femur near the patella.)

[0116] Calculate the center point center_femur of the femoral mask. Specifically, this includes: calculating the average of the x-coordinates of all non-zero points on the femoral mask and the average of the y-coordinates of all non-zero points on the femoral mask, to obtain the x-coordinate and y-coordinate of the center point center_femur.

[0117] Correcting the direction vectors corresponding to the non-zero coordinates of the femurmask. Specifically, this includes: subtracting the center point `center_femur` from each non-zero coordinate of the femurmask to obtain the relative vector of each non-zero coordinate relative to the center point `center_femur`; multiplying each relative vector by the direction vector or normalized direction vector `PCA_femurvec` of the corresponding non-zero coordinate to obtain multiple projection values ​​`projs_pts` of the direction vector in the current principal component direction; finding the first and second non-zero coordinates of the femurmask corresponding to the minimum and maximum projection values ​​among the multiple projection values ​​`projs_pts`, denoted as `end1_pos` (first non-zero coordinate) and `end2_pos` (second non-zero coordinate). Take the center point `imgpoint_center` of the image to be segmented or the segmentation mask image corresponding to the image to be segmented; calculate the first distance `dist1` between the first non-zero point coordinate `end1_pos` and `imgpoint_center`, and the second distance `dist2` between the second non-zero point coordinate `end2_pos` and `imgpoint_center`; if the first distance `dist1` is less than the second distance `dist2`, then the direction of the direction vector or normalized direction vector `PCA_femurvec` is from the femoral knee joint to the femoral hip joint; if the first distance `dist1` is greater than the second distance `dist2`, then invert the direction of the direction vector or normalized direction vector `PCA_femurvec` (add a negative sign), that is, set the direction vector or normalized direction vector `PCA_femurvec` = -PCA_femurvec. After the above steps, the direction vector corresponding to the non-zero point coordinates of the femoral mask `femurmask` is corrected, and the corrected direction vector is obtained.

[0118] Extract the set of non-zero coordinates of the femoral mask boundary image (femurmask_edge) corresponding to the femoral mask boundary image (femurmask_edge). Specifically, this includes: eroding the femoral mask to obtain the eroded femoral mask image femurmaskerodeimg; subtracting the eroded femoral mask image femurmaskerodeimg from the femoral mask to extract the femoral mask boundary image femurmask_edge; and extracting the set of non-zero coordinates of the femoral mask boundary image femurmask_edge femurmask_edgeptrvec.

[0119] Based on the non-zero coordinate set femurmask_edgeptrvec of the femoral mask boundary, fit the first polar coordinate equation of the first straight line corresponding to the two sides of the femur or femoral cortex and the second polar coordinate equation of the second straight line opposite to the first straight line (the first straight line and the second straight line constitute the boundary of the two sides of the femur). Specifically, this includes: based on the diagonal length of the image to be segmented or the segmentation mask image corresponding to the image to be segmented being max_rho, determining the range of the first polar coordinate parameter (the first distance from the origin of the rectangular coordinate system to the first straight line) ρ in the polar coordinate equation to be from -max_rho to max_rho; configuring the range of the second polar coordinate parameter (the first angle between the perpendicular from the origin of the rectangular coordinate system to the first straight line and the x-axis) θ in the polar coordinate equation to be from 0 to 180 degrees; based on the polar coordinate equations corresponding to the distance range and the angle range and the set of non-zero coordinates of the femoral mask boundary (the set of non-zero coordinates of the first femoral mask boundary and the set of non-zero coordinates of the second femoral mask boundary), using the polar coordinate Hough (line) transformation (the line from a point in polar coordinates to the Hough space), determining the two most likely straight lines, respectively denoted as the first straight line Linerleft (the first boundary line of the femoral side) corresponding to one side of the femur and the second straight line Lineright (the second boundary line of the femoral side) corresponding to the other side of the femur. In this system, each straight line is expressed in polar coordinates as x*cosθ+y*sinθ=ρ, where x and y are the variables corresponding to the polar coordinate equation. Specifically, the first polar coordinate equation for the first straight line Linerleft on one side of the femur is configured as x*cosθ1+y*sinθ1=ρ1, and the second polar coordinate equation for the second straight line Lineright on the other side of the femur is configured as x*cosθ2+y*sinθ2=ρ2. Here, θ1 and θ2 represent the first angle between the perpendicular from the origin of the rectangular coordinate system to the first straight line and the x-axis, and the second angle between the perpendicular from the origin of the rectangular coordinate system to the second straight line and the x-axis, respectively. ρ1 and ρ2 represent the first distance from the origin of the rectangular coordinate system to the first straight line and the second distance from the origin of the rectangular coordinate system to the second straight line, respectively. The first straight line is represented using the first polar coordinate equation; the second straight line is represented using the corresponding second polar coordinate equation.

[0120] Based on the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur, determine the polar coordinate equation corresponding to the femoral midline. Specifically, this includes: based on the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur, determine the first direction vector vec1 of the two straight lines as (-sinθ1, cosθ1) and the second direction vector vec2 as (-sinθ2, cosθ2).

[0121] Normalize the first direction vector vec1 and the second direction vector vec2 respectively to obtain a first normalized direction vector and a second normalized direction vector. Then, perform a dot product between the first normalized direction vector, the first normalized direction vector, the second normalized direction vector, or the direction vector corresponding to the non-zero coordinates of the second direction vector and the modified femoral mask (femurmask) to obtain a first product of the first normalized direction vector and a second product of the second normalized direction vector. If the first product is less than 0, invert the first direction vector or the first normalized direction vector in the direction (add a negative sign); if the second product is less than 0, invert the second direction vector or the second normalized direction vector in the direction (add a negative sign).

[0122] After the above steps, ensure that the directions of the first and second direction vectors of the two straight lines (the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur) are consistent with the direction of the femoral direction vector or the normalized femoral direction vector PCA_femurvec. After determining the directions, add the first direction vector vec1 and the second direction vector vec2 to obtain the bisector direction vector (i.e., the direction vector of the middle bisector corresponding to the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur). Normalize the bisector direction vector to obtain the normalized bisector direction vector femurvec3, which is (-sinθ3, cosθ3). The polar coordinate equation of the third straight line corresponding to the bisector direction vector or the normalized bisector direction vector femurvec3 is configured as x*cosθ3+y*sinθ3=ρ3. Where θ3 represents the third angle between the perpendicular line from the origin of the rectangular coordinate system to the third line and the x-axis, and ρ3 represents the third distance from the origin of the rectangular coordinate system to the third line. Taking the intersection point (x, y) of the first line Linerleft and the second line Lineright, the third distance ρ3 from the origin of the rectangular coordinate system to the third line can be calculated based on the intersection point (x, y), the bisector direction vector, and the third polar coordinate equation. If the first line Linerleft and the second line Lineright do not intersect, the third distance ρ3 from the origin of the rectangular coordinate system to the third line is determined based on the midpoint (x, y) of the perpendicular segment between the first line Linerleft and the second line Lineright, the bisector direction vector, and the third polar coordinate equation. Therefore, the third polar coordinate equation corresponding to the femoral midline is configured as x*cosθ3 + y*sinθ3 = ρ3. This disclosure proposes a method for determining the tibial boundary line, comprising: extracting a tibial mask boundary image corresponding to the tibial mask from a tibial region mask image; and determining one or two boundary lines, namely a third boundary line and a fourth boundary line, among the boundary lines on both sides of the tibia or tibial cortex, based on the tibial mask boundary image and the non-zero coordinates of the tibial mask boundary image. This addresses at least one of the technical problems of currently using manual annotation to determine the tibial boundary line and tibial midline, which are highly subjective, and the need for further improvement in the accuracy of automatic detection of the tibial boundary line and tibial midline.

[0123] In this embodiment of the disclosure, determining one or two boundary lines of the third and fourth boundary lines among the boundary lines on both sides of the tibia or tibial cortex based on the tibial mask boundary image and the non-zero coordinates of the tibial mask boundary image includes: performing a polar coordinate Hough transform on the tibial mask boundary image to determine the most likely third and fourth corrected boundary lines on the outermost side of the tibia or tibial cortex; determining whether the third and fourth corrected boundary lines are parallel; if they are not parallel and the intersection of the third and fourth corrected boundary lines is within the tibial mask image, then correcting the third and fourth corrected boundary lines based on the tibial mask boundary image to obtain one or two boundary lines of the third and fourth boundary lines.

[0124] In this embodiment of the disclosure, the step of correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary image to obtain one or both of the third and fourth boundary lines includes: determining a mirror line based on the X-ray image of the knee joint to be processed corresponding to the tibial mask boundary image; performing mirror processing on the tibial mask boundary in the tibial mask boundary image with the mirror line as the axis of symmetry to obtain a mirror image of the tibial mask boundary; and correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary and the mirror image of the tibial mask boundary to obtain one or both of the third and fourth boundary lines.

[0125] In this embodiment of the disclosure, the step of correcting the third boundary line to be corrected and the fourth boundary line based on the tibial mask boundary and the mirror image of the tibial mask boundary to obtain one or two boundary lines of the third boundary line and the fourth boundary line includes: performing a polar coordinate Hough transform on the tibial mask boundary and the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are parallel, then any one or two of the parallel straight lines are configured as one or two boundary lines of the third boundary line and the fourth boundary line.

[0126] In this embodiment of the disclosure, the step of correcting the third boundary line to be corrected and the fourth boundary line based on the tibial mask boundary and the mirror image of the tibial mask boundary to obtain one or two boundary lines of the third boundary line and the fourth boundary line further includes: if any two lines among the plurality of most probable lines are not parallel and the intersection point of any two non-parallel lines is outside the tibial mask image, then any one or two of the two lines corresponding to the intersection point of any two non-parallel lines outside the tibial mask image are configured as one or two boundary lines of the third boundary line and the fourth boundary line.

[0127] In this embodiment of the disclosure, the step of determining one or two boundary lines of the third and fourth boundary lines among the boundary lines on both sides of the tibia or tibial cortex based on the tibial mask boundary image and the non-zero coordinates of the tibial mask boundary of the tibial mask boundary image further includes: if they are parallel, then the third boundary line to be corrected and the fourth boundary line to be corrected are not corrected, and the third boundary line to be corrected and the fourth boundary line to be corrected are respectively configured as one or two boundary lines of the third and fourth boundary lines; if they are not parallel and the intersection of the third boundary line to be corrected and the fourth boundary line to be corrected is outside the tibial mask image, then the third boundary line to be corrected and the fourth boundary line to be corrected are not corrected, and the third boundary line to be corrected and the fourth boundary line to be corrected are respectively configured as one or two boundary lines of the third and fourth boundary lines.

[0128] In this embodiment of the disclosure, before correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary image, the method includes: determining the tibial length based on the tibial mask image or the tibial mask boundary image corresponding to the tibial mask image; if the tibial length is less than a preset tibial length, then correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary image; otherwise, not correcting the third boundary line to be corrected and the fourth boundary line to be corrected, and configuring the third boundary line to be corrected and the fourth boundary line to be corrected as one or both of the third and fourth boundary lines.

[0129] In this embodiment of the disclosure, determining the mirror line based on the X-ray image of the knee joint to be processed corresponding to the tibial mask boundary image includes: determining the anterior vertex and posterior vertex of the tibial plateau corresponding to the X-ray image of the knee to be processed; and determining the mirror line based on the anterior vertex and posterior vertex of the tibial plateau.

[0130] In this embodiment of the disclosure, determining the anterior and posterior tibial plateau vertices corresponding to the knee X-ray image to be processed (the knee joint X-ray image to be processed) includes: using a keypoint detection model corresponding to a keypoint detection network to detect the anterior and posterior tibial plateau vertices of the knee X-ray image to be processed, and determining the anterior and posterior tibial plateau vertices corresponding to the knee X-ray image to be processed.

[0131] In this embodiment of the disclosure, constructing a keypoint detection model corresponding to the keypoint detection network includes: acquiring anterior and posterior tibial plateau labels corresponding to multiple knee X-ray images; and training a preset keypoint detection network using the anterior and posterior tibial plateau labels to obtain a keypoint detection model.

[0132] In this embodiment of the disclosure, before training the preset keypoint detection network using the anterior and posterior tibial plateau vertices labels, the process includes: pre-training the preset keypoint detection network using multiple preset pose points corresponding to human pose images and / or multiple preset facial feature points corresponding to facial images to obtain a pre-trained keypoint detection model; then, re-training the pre-trained keypoint detection model using the anterior and posterior tibial plateau vertices labels to obtain a keypoint detection model.

[0133] In this embodiment of the disclosure, the keypoint detection network can be configured as one or more of the following deep learning networks: OpenPose, HRNet (High-Resolution Network), Hourglass Network, DEKR (Distributed Keypoint Regression), YOLO-KP, CenterNet, Cornernet, TokenPose, PoseTransformer, Mask R-CNN, etc.

[0134] In this embodiment of the disclosure, before extracting the tibial mask boundary image corresponding to the tibial mask in the tibial region mask image, the method includes: using a knee X-ray image segmentation model corresponding to a preset segmentation network based on deep learning to perform tibial segmentation on the knee X-ray image to be processed, thereby obtaining a tibial region mask image.

[0135] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; and training a preset segmentation network using the preset dynamic knee joint X-ray image training set and its corresponding mask label images to obtain a dynamic knee joint X-ray image segmentation model.

[0136] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set corresponding to mask label images; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; training multiple preset segmentation networks using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain multiple dynamic knee joint X-ray image segmentation models; evaluating the multiple dynamic knee joint X-ray image segmentation models to determine the corresponding optimal dynamic knee joint X-ray image segmentation model.

[0137] In this embodiment of the disclosure, the step of extracting the tibial mask boundary image corresponding to the tibial mask in the tibial region mask image includes: eroding the tibial mask in the tibial region mask image to obtain a tibial mask erosion image; and subtracting the tibial mask erosion image from the tibial region mask image to obtain the tibial mask boundary image corresponding to the tibial mask in the tibial region mask image.

[0138] In this embodiment of the disclosure, a method for determining the tibial midline is also proposed, comprising: determining a third boundary line and a fourth boundary line among the boundary lines on both sides of the tibia or the tibial cortex using the tibial boundary line determination method described above; and determining the tibial midline based on the third boundary line and the fourth boundary line.

[0139] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line and the fourth boundary line includes: determining a fourth direction vector corresponding to the third boundary line and a fifth direction vector corresponding to the fourth boundary line; performing dot products on the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain a third product corresponding to the fourth direction vector and a fourth product corresponding to the fifth direction vector; if the third product is less than a third preset value, inverting the fourth direction vector in the direction; otherwise, not processing the fourth direction vector; if the fourth product is less than a fourth preset value, inverting the fifth direction vector in the direction; otherwise, not processing the fifth direction vector; adding the inverted or unprocessed fourth direction vector and the inverted or unprocessed fifth direction vector to obtain the bisector direction vector; and determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector.

[0140] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector includes: if the third boundary line and the fourth boundary line intersect, then determining the tibial midline corresponding to the sixth polar coordinate equation based on the intersection point, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector; otherwise, determining the tibial midline corresponding to the sixth polar coordinate equation based on the midpoint of the perpendicular segment between the third boundary line and the fourth boundary line, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector.

[0141] In this embodiment of the disclosure, before performing dot products on the fourth direction vector and the fifth direction vector with the direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: normalizing the fourth direction vector, the fifth direction vector, and the direction vector corresponding to the tibial region mask image; wherein the values ​​corresponding to the third preset value and the fourth preset value are the same or different; and the values ​​corresponding to the third preset value and the fourth preset value are respectively configured to 0.

[0142] In this embodiment of the disclosure, before performing dot products of the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: calculating the tibial principal component direction vector corresponding to the non-zero point coordinates of the tibial mask in the tibial region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the tibial principal component direction vector to obtain the corrected tibial principal component direction vector; and performing dot products of the fourth direction vector and the fifth direction vector with the corrected tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector.

[0143] In this embodiment of the disclosure, the step of correcting the tibial principal component direction vector using the center point to obtain a corrected tibial principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the tibial mask in the tibial region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the corresponding non-zero point coordinate's tibial principal component direction vector to obtain multiple projection values ​​of the tibial principal component direction vector in the current principal component direction; determining the third non-zero point coordinate corresponding to the smallest projection value and the fourth non-zero point coordinate corresponding to the largest projection value among the multiple projection values; calculating the third distance and the fourth distance corresponding to the center point of the tibial region mask image and the third and fourth non-zero coordinates respectively; determining the direction corresponding to the tibial principal component direction vector based on the third and fourth distances; and correcting the tibial principal component direction vector based on the direction to obtain the corrected tibial principal component direction vector.

[0144] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the third distance and the fourth distance includes: if the third distance is less than the fourth distance, then the direction points from the tibia-knee joint to the ankle joint; otherwise, the direction points from the ankle joint to the tibia-knee joint; and / or, correcting the tibial principal component direction vector based on the direction to obtain a corrected tibial principal component direction vector includes: if the direction points from the tibia-knee joint to the ankle joint, then the tibial principal component direction vector is not corrected; if the direction points from the ankle joint to the tibia-knee joint, then the direction vector is inverted in the direction to obtain a corrected tibial principal component direction vector.

[0145] In this embodiment, a tibial mask image processing method is also proposed, comprising: determining a mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed (X-ray image of the knee joint to be processed); using the mirror line as the axis of symmetry, performing mirror processing on the tibial mask boundary image corresponding to the X-ray image of the knee to be processed to obtain a mirror image of the tibial mask boundary; and constructing a tibial mask image to be processed based on the tibial mask boundary image and the mirror image of the tibial mask boundary.

[0146] In this embodiment of the disclosure, the knee X-ray image to be processed can be manually outlined to determine the anterior and posterior vertices of the tibial plateau corresponding to the knee X-ray image to be processed; at the same time, the knee X-ray image to be processed can also be automatically processed using image processing algorithms to determine the corresponding anterior and posterior vertices of the tibial plateau.

[0147] In this embodiment of the disclosure, the step of constructing a tibial mask image to be processed based on the tibial mask boundary image and the mirror image of the tibial mask boundary includes: stitching the tibial mask boundary image and the mirror image of the tibial mask boundary along the mirror line to construct the tibial mask image to be processed.

[0148] In this embodiment of the disclosure, determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed includes: using a keypoint detection model corresponding to a keypoint detection network to detect the anterior and posterior vertices of the tibial plateau in the X-ray image of the knee to be processed, and determining the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed.

[0149] In this embodiment of the disclosure, constructing a keypoint detection model corresponding to the keypoint detection network includes: acquiring anterior and posterior tibial plateau labels corresponding to multiple knee X-ray images; and training a preset keypoint detection network using the anterior and posterior tibial plateau labels to obtain a keypoint detection model.

[0150] In this embodiment of the disclosure, before training the preset keypoint detection network using the anterior and posterior tibial plateau vertices labels, the process includes: pre-training the preset keypoint detection network using multiple preset pose points corresponding to human pose images and / or multiple preset facial feature points corresponding to facial images to obtain a pre-trained keypoint detection model; then, re-training the pre-trained keypoint detection model using the anterior and posterior tibial plateau vertices labels to obtain a keypoint detection model.

[0151] In this embodiment of the disclosure, the preset keypoint detection network can be configured as one or more of the following deep learning networks: OpenPose, HRNet (High-Resolution Network), Hourglass Network, DEKR (Distributed Keypoint Regression), YOLO-KP, CenterNet, Cornernet, TokenPose, PoseTransformer, Mask R-CNN, etc.

[0152] In this embodiment of the disclosure, before determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the knee X-ray image to be processed, the method includes: determining the tibial length based on the tibial mask image or the tibial mask boundary image corresponding to the tibial mask image; if the tibial length is less than a preset tibial length, then determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the knee X-ray image to be processed; otherwise, the tibial mask boundary is not mirrored. This addresses at least one of the technical problems, such as the tibia being too short in the knee X-ray image or the tibia in its corresponding mask image being too short in the knee X-ray image, which easily leads to errors in the detection of the tibial or tibial cortex boundary lines, thereby affecting the accuracy of the tibial midline detection.

[0153] In this embodiment of the disclosure, determining the mirror line based on the anterior and posterior tibial plateau vertices corresponding to the X-ray image of the knee to be processed includes: constructing a corresponding vertex line based on the anterior and posterior tibial plateau vertices corresponding to the X-ray image of the knee to be processed; and configuring the vertex line as a fixed mirror line corresponding to the anterior and posterior tibial plateau vertices.

[0154] In this embodiment of the disclosure, determining the tibial mask boundary image corresponding to the knee X-ray image to be processed includes: using a knee joint X-ray image segmentation model corresponding to a preset segmentation network based on deep learning to perform tibial segmentation on the knee X-ray image to be processed to obtain a tibial region mask image; and extracting the boundary of the tibial mask in the tibial region mask image to obtain a tibial mask boundary image.

[0155] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; and training a preset segmentation network using the preset dynamic knee joint X-ray image training set and its corresponding mask label images to obtain a dynamic knee joint X-ray image segmentation model.

[0156] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set corresponding to mask label images; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; training multiple preset segmentation networks using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain multiple dynamic knee joint X-ray image segmentation models; evaluating the multiple dynamic knee joint X-ray image segmentation models to determine the corresponding optimal dynamic knee joint X-ray image segmentation model.

[0157] In this embodiment of the disclosure, the step of extracting the boundary of the tibia mask in the tibia region mask image to obtain a tibia mask boundary image includes: eroding the tibia mask in the tibia region mask image to obtain a tibial mask erosion image; and subtracting the tibial mask erosion image from the tibia region mask image to obtain a tibia mask boundary image corresponding to the tibia mask in the tibia region mask image.

[0158] In this embodiment of the disclosure, a method for processing X-ray images of the knee joint is also proposed, comprising: using the tibial mask image processing method described above to determine the tibial mask image corresponding to the X-ray image of the knee to be processed (X-ray image of the knee joint to be processed); and determining one or two boundary lines, namely a third boundary line and a fourth boundary line, in the boundary lines of the tibia or tibial cortex on both sides of the X-ray image of the knee to be processed, based on the tibial mask image.

[0159] In this embodiment of the disclosure, determining one or both of the third and fourth boundary lines among the tibial or tibial cortex boundary lines on both sides of the knee X-ray image to be processed based on the tibial mask image to be processed includes: performing a polar coordinate Hough transform on the tibial mask boundary and the tibial mask image to be processed corresponding to the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are parallel, then any one or two of the parallel straight lines are configured as one or both of the third and fourth boundary lines.

[0160] In this embodiment of the disclosure, determining one or both of the third and fourth boundary lines among the tibial or tibial cortex boundary lines of the knee X-ray image to be processed based on the tibial mask image to be processed includes: performing a polar coordinate Hough transform on the tibial mask boundary and the tibial mask image to be processed corresponding to the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are not parallel and the intersection point of any two non-parallel straight lines is outside the tibial mask image, then any one or two of the two straight lines corresponding to the intersection point of any two non-parallel straight lines outside the tibial mask image are configured as one or both of the third and fourth boundary lines.

[0161] In this embodiment of the disclosure, the method further includes: determining the tibial midline based on the third boundary line and the fourth boundary line.

[0162] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line and the fourth boundary line includes: determining a fourth direction vector corresponding to the third boundary line and a fifth direction vector corresponding to the fourth boundary line; performing dot products on the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain a third product corresponding to the fourth direction vector and a fourth product corresponding to the fifth direction vector; if the third product is less than a third preset value, inverting the fourth direction vector in the direction; otherwise, not processing the fourth direction vector; if the fourth product is less than a fourth preset value, inverting the fifth direction vector in the direction; otherwise, not processing the fifth direction vector; adding the inverted or unprocessed fourth direction vector and the inverted or unprocessed fifth direction vector to obtain the bisector direction vector; and determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector.

[0163] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector includes: if the third boundary line and the fourth boundary line intersect, then determining the tibial midline corresponding to the sixth polar coordinate equation based on the intersection point, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector; otherwise, determining the tibial midline corresponding to the sixth polar coordinate equation based on the midpoint of the perpendicular segment between the third boundary line and the fourth boundary line, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector.

[0164] In this embodiment of the disclosure, before performing dot products on the fourth direction vector and the fifth direction vector with the direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: normalizing the fourth direction vector, the fifth direction vector and the direction vector corresponding to the tibial region mask image, respectively.

[0165] In this embodiment of the disclosure, the values ​​corresponding to the third preset value and the fourth preset value are the same or different; and / or, the values ​​corresponding to the third preset value and the fourth preset value are respectively configured to 0.

[0166] In this embodiment of the disclosure, before performing dot products of the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: calculating the tibial principal component direction vector corresponding to the non-zero point coordinates of the tibial mask in the tibial region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the tibial principal component direction vector to obtain the corrected tibial principal component direction vector; and performing dot products of the fourth direction vector and the fifth direction vector with the corrected tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector.

[0167] In this embodiment of the disclosure, the step of correcting the tibial principal component direction vector using the center point to obtain a corrected tibial principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the tibial mask in the tibial region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the corresponding non-zero point coordinate's tibial principal component direction vector to obtain multiple projection values ​​of the tibial principal component direction vector in the current principal component direction; determining the third non-zero point coordinate corresponding to the smallest projection value and the fourth non-zero point coordinate corresponding to the largest projection value among the multiple projection values; calculating the third distance and the fourth distance corresponding to the center point of the tibial region mask image and the third and fourth non-zero coordinates respectively; determining the direction corresponding to the tibial principal component direction vector based on the third and fourth distances; and correcting the tibial principal component direction vector based on the direction to obtain the corrected tibial principal component direction vector.

[0168] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the third distance and the fourth distance includes: if the third distance is less than the fourth distance, then the direction points from the tibia-knee joint to the ankle joint; otherwise, the direction points from the ankle joint to the tibia-knee joint; and / or, correcting the tibial principal component direction vector based on the direction to obtain a corrected tibial principal component direction vector includes: if the direction points from the tibia-knee joint to the ankle joint, then the tibial principal component direction vector is not corrected; if the direction points from the ankle joint to the tibia-knee joint, then the direction vector is inverted in the direction to obtain a corrected tibial principal component direction vector.

[0169] Specifically, the non-zero coordinates of the tibial mask (tibiamask) in the tibial region mask image or tibial region mask image are extracted, and the principal component direction vector PCA_tibiavec of the tibia is calculated using the principal component analysis (PCA) algorithm. This includes calculating the principal component direction vectors corresponding to the non-zero coordinates of the tibial mask (tibiamask) in the tibial region mask image or tibial region mask image. Specifically, this includes: extracting the non-zero coordinates of the tibial mask tibiamask, using the principal component analysis (PCA) algorithm to calculate the principal component direction vector (tibial principal component direction vector) corresponding to the non-zero coordinates of the tibial mask tibiamask, and normalizing the tibial direction vector to obtain the normalized direction vector (tibial normalized principal component direction vector) PCA_tibiavec. (Note that the direction vector or normalized direction vector PCA_tibiavec corresponding to the non-zero coordinates obtained at this time may point from the side of the tibia near the patella to the side near the ankle joint, or it may point from the side of the tibia near the ankle joint to the side of the tibia near the patella.)

[0170] Calculating the coordinates of the center point cener_tibia of the tibial mask tibiamask includes: calculating the average of the abscissas of all non-zero points of the tibial mask tibiamask and the average of the ordinates of all non-zero points of the tibial mask tibiamask, to obtain the abscissa and ordinate of the center point cener_tibiar.

[0171] Correct the direction vector corresponding to the non-zero coordinates of the tibial mask femurmask. Specifically, this includes: extracting the non-zero coordinates of the tibial mask tibiamask; calculating the principal component direction vector PCA_tibiavec of the tibia using the principal component analysis (PCA) algorithm; determining the direction of the principal component direction vector of the tibia, and the coordinates of the center point cener_tibia of the tibial mask. Specifically, this includes: subtracting the center point cener_tibia of the tibial mask from each non-zero coordinate of the tibial mask tibiamask to obtain the relative vector of each non-zero coordinate of the tibial mask tibiamask relative to the center point cener_tibia of the tibial mask; multiplying each relative vector by the direction vector or normalized direction vector PCA_femurvec of the corresponding non-zero coordinate to obtain multiple projection values ​​projs_pts of the direction vector in the current principal component direction; finding the third and fourth non-zero coordinates of the tibial mask tibiamask corresponding to the minimum and maximum projection values ​​among the multiple projection values ​​projs_pts, denoted as end1_pos (third non-zero coordinate) and end2_pos (fourth non-zero coordinate). Take the center point `imgpoint_center` of the image to be segmented or the corresponding segmentation mask image. Calculate the third distance `dist3` between the third non-zero point coordinates `end3_pos` and `imgpoint_center`, and the fourth distance `dist4` between the fourth non-zero point coordinates `end4_pos` and `imgpoint_center`. If the third distance `dist3` is less than the fourth distance `dist4`, then the direction vector or normalized direction vector `PCA_tibiavec` is from the tibia-knee joint to the tibia-ankle joint. If the third distance `dist3` is greater than the fourth distance `dist4`, then the direction vector or normalized direction vector `PCA_tibiavec` is inverted (negated), i.e., `PCA_tibiavec = -PCA_tibiavec`. After the above steps, the tibia direction vector corresponding to the non-zero point coordinates of the tibia mask `tibiamask` is corrected, resulting in the corrected tibia direction vector.

[0172] Extract the tibial mask boundary image (tibiamask edge image) from the tibial mask image. Specifically, this includes: performing tibial mask erosion on the tibial mask image to obtain the eroded tibial mask image tibiamaskerodeimg, and subtracting the eroded tibial mask image tibiamaskodeimg from the tibial mask image to extract the tibial mask boundary image tibiamask_edge.

[0173] The third and fourth boundary lines corresponding to the outermost two boundary lines of the tibia or tibial cortex are determined. Specifically, this includes performing a polar coordinate Hough transform on the tibial mask boundary image tibiamask_edge to determine the two most likely equations of the lines to be corrected, which are the fourth and fifth polar coordinate equations corresponding to the two outermost edge lines of the tibia or tibial cortex. The two outermost edge lines of the tibia or tibial cortex are respectively configured as the third and fourth boundary lines corresponding to the outermost two boundary lines of the tibia or tibial cortex. The third boundary line is represented using the fourth polar coordinate equation; the fourth boundary line is represented using the corresponding fifth polar coordinate equation. It is then determined whether the two lines are parallel or whether the intersection of the two lines is outside the tibial mask image. Specifically, this includes: determining whether the third and fourth boundary lines to be corrected corresponding to the outermost boundary lines of the tibia or tibial cortex are parallel; if not parallel, determining the intersection point of the third and fourth boundary lines to be corrected corresponding to the outermost boundary lines of the tibia or tibial cortex; determining whether the intersection point is outside the tibial mask image; if the two lines to be corrected are parallel or the intersection point is outside the tibial mask image, then these two lines to be corrected are the equations (fourth polar coordinate equation and fifth polar coordinate equation) of the two outermost edge lines to be corrected of the tibial cortex. If the intersection point of the two lines to be corrected is inside the tibial mask image, the following steps are performed: The keypoint detection model corresponding to the HRnet network was used to detect keypoints in the X-ray image of the knee joint to be processed corresponding to the tibial mask image, obtaining the anterior vertex A and posterior vertex B of the tibial plateau. Based on the anterior vertex A and posterior vertex B of the tibial plateau, a straight line AB (a mirror line) was determined. Using the straight line AB as the axis of symmetry, the tibial mask boundary in the tibiamask_edge image was symmetrically copied to the other side to obtain the copied tibial mask boundary tibiamask_edgecopy.

[0174] A polar coordinate Hough transform is performed on both the copied tibial mask boundary (tibiamask_edgecopy) and the tibial mask boundary to determine the sixth polar coordinate equation (the first modified polar coordinate equation corresponding to the fourth polar coordinate equation), the seventh polar coordinate equation (the second modified polar coordinate equation corresponding to the fifth polar coordinate equation), and the eighth polar coordinate equation (interference polar coordinate equation) corresponding to the three most likely straight lines (multiple most likely straight lines). The third and fourth boundary lines corresponding to the outermost two boundary lines of the tibia or tibial cortex, and the interference boundary lines are denoted as Line1, Line2, and Line3, respectively. Each straight line is expressed in polar coordinates as x*cosθ+y*sinθ=ρ, where x and y are variables.

[0175] Based on the sixth polar equation (the first modified polar equation corresponding to the fourth polar equation), the seventh polar equation (the second modified polar equation corresponding to the fifth polar equation), and the eighth polar equation (interference polar equation) corresponding to the three most likely straight lines, the equations of the two outermost edge lines of the tibia or tibial cortex are determined. Specifically, this includes: determining whether any two of the three most likely straight lines are parallel; if they are parallel, then the two parallel lines are determined as the third and fourth boundary lines corresponding to the two outermost boundary lines of the tibia or tibial cortex; if any two of the three most likely straight lines are not parallel, then determine whether the intersection point of any two non-parallel lines is outside the tibial mask image; if it is outside the tibial mask image, then the two lines corresponding to the intersection point outside the tibial mask image are determined as the third and fourth boundary lines corresponding to the two outermost boundary lines of the tibia or tibial cortex.

[0176] Specifically, the fourth polar coordinate equation of the fourth straight line corresponding to the third boundary line on one side of the tibia is configured as x*cosθ4+y*sinθ4=ρ4, and the fifth polar coordinate equation of the fifth straight line corresponding to the fourth boundary line on the other side of the tibia is configured as x*cosθ5+y*sinθ5=ρ5. Here, θ4 and θ5 represent the fourth angle between the perpendicular from the origin of the rectangular coordinate system to the fourth boundary line and the x-axis, and the fifth angle between the perpendicular from the origin of the rectangular coordinate system to the fourth boundary line and the x-axis, respectively. ρ4 and ρ5 represent the fourth distance from the origin of the rectangular coordinate system to the fourth boundary line and the fifth distance from the origin of the rectangular coordinate system to the fourth boundary line, respectively. The third boundary line is represented using the fourth polar coordinate equation; the fourth boundary line is represented using the corresponding fifth polar coordinate equation.

[0177] Based on the third boundary line corresponding to one side of the tibia and the fourth boundary line corresponding to the other side of the tibia, the polar coordinate equation corresponding to the tibial midline is determined. This includes: based on the third boundary line Linerleft corresponding to one side of the tibia and the fourth boundary line Lineright corresponding to the other side of the tibia, determining the third direction vector vec3 as (-sinθ4, cosθ4) and the fourth direction vector vec4 as (-sinθ5, cosθ5) of the two straight lines. The third direction vector vec3 and the fourth direction vector vec4 are normalized to obtain the third normalized direction vector and the fourth normalized direction vector; the third direction vector or the third normalized direction vector, the fourth direction vector or the fourth normalized direction vector, and the tibial correction direction vector corresponding to the non-zero coordinates of the corrected tibial mask tibiamask are then multiplied by a dot to obtain the third product corresponding to the third normalized direction vector or the third direction vector and the fourth product corresponding to the fourth normalized direction vector or the fourth direction vector. If the third product is less than 0, the direction of the third direction vector or the third normalized direction vector is inverted (negated); if the fourth product is less than 0, the direction of the fourth direction vector or the fourth normalized direction vector is inverted (negated). After the above steps, it is ensured that the directions of the third normalized direction vector or the third direction vector and the fourth normalized direction vector or the fourth direction vector of the two lines (the third boundary line Linerleft corresponding to one side of the tibia and the fourth boundary line Lineright corresponding to the other side of the tibia) are consistent with the direction of the tibia direction vector or the normalized tibia direction vector PCA_tibiavec.

[0178] After determining the direction, the third direction vector vec3 and the fourth direction vector vec4 are added together to obtain the tibial bisector direction vector (i.e., the direction vector of the middle bisector corresponding to the third boundary line Linerleft on one side of the tibia and the fourth boundary line Lineright on the other side of the tibia). The tibial bisector direction vector is then normalized to obtain the tibial normalized bisector direction vector tibiavec6, which is (-sinθ6, cosθ6). The sixth polar coordinate equation of the sixth line corresponding to the tibial bisector direction vector or the tibial normalized bisector direction vector tibiavec6 is configured as x*cosθ6+y*sinθ6=ρ6. Here, θ6 represents the sixth angle between the perpendicular line from the origin of the rectangular coordinate system to the sixth line and the x-axis, and ρ6 represents the sixth distance from the origin of the rectangular coordinate system to the sixth line. By taking the intersection point (x, y) of the fourth line Linerleft and the fifth line Lineright, the sixth distance ρ6 from the origin of the rectangular coordinate system corresponding to the tibial bisector direction vector or the tibial normalized bisector direction vector femurvec6 can be calculated based on the intersection point (x, y), the tibial normalized bisector direction vector, and the sixth polar coordinate equation of the sixth line. If the fourth line Linerleft and the fifth line Lineright do not intersect, the sixth distance ρ6 from the origin of the rectangular coordinate system corresponding to the tibial bisector direction vector or the tibial normalized bisector direction vector femurvec6 can be determined based on the midpoint (x, y) of the perpendicular segment between the fourth line Linerleft and the fifth line Lineright, the tibial bisector direction vector, and the sixth polar coordinate equation of the sixth line. Furthermore, the sixth polar coordinate equation of the sixth line corresponding to the tibial midline is configured as x*cosθ6 + y*sinθ6 = ρ6.

[0179] In this embodiment of the disclosure, the method further includes: determining the joint angle between the femoral midline and the tibial midline based on the tibial midline and the femoral midline corresponding to the X-ray image of the knee joint to be processed.

[0180] In this embodiment of the disclosure, determining the joint angle between the femoral midline and the tibial midline based on the tibial midline and the femoral midline corresponding to the X-ray image of the knee joint to be processed includes: establishing a system of polar coordinate equations based on the femoral midline and the tibial midline corresponding to the X-ray image of the knee joint to be processed; and using the system of polar coordinate equations to determine the joint angle between the femoral midline and the tibial midline.

[0181] In this embodiment of the disclosure, determining the joint angle point between the femoral midline and the tibial midline using the polar coordinate equations includes: if the polar coordinate equations have a unique solution, determining whether the coordinates corresponding to the unique solution are within the X-ray image of the knee joint to be processed; if they are within the X-ray image of the knee joint to be processed, determining the coordinates corresponding to the unique solution as the joint angle point between the femoral midline and the tibial midline; if they are not within the X-ray image of the knee joint to be processed or the polar coordinate equations have no solution, then based on the femoral midline and the femoral mask boundary image... The first coordinate point corresponding to the joint angle is determined by using the first non-zero coordinate corresponding to the boundary of the femoral mask, the coordinates of the femoral midline, and the center point of the X-ray image of the knee joint to be processed; the second coordinate point corresponding to the joint angle is determined by using the second non-zero coordinate corresponding to the boundary of the tibial mask in the image of the tibial midline and the coordinates of the center point of the X-ray image of the knee joint to be processed; the midpoint between the first coordinate point and the second coordinate point is calculated, and the midpoint is determined as the joint angle between the femoral midline and the tibial midline.

[0182] In this embodiment of the disclosure, determining the femoral midline includes: extracting the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image corresponding to the knee joint X-ray image to be processed; determining the first boundary line and the second boundary line among the boundary lines on both sides of the femur or femoral cortex based on the non-zero coordinates of the femoral mask boundary image; and determining the femoral midline based on the first boundary line and the second boundary line.

[0183] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line and the second boundary line includes: determining a first direction vector corresponding to the first boundary line and a second direction vector corresponding to the second boundary line; performing dot products on the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector; if the first product is less than a first preset value, inverting the first direction vector in the direction; otherwise, not processing the first direction vector; if the second product is less than a second preset value, inverting the second direction vector in the direction; otherwise, not processing the second direction vector; adding the first direction vector corresponding to the inverted or unprocessed and the second direction vector corresponding to the inverted or unprocessed to obtain the bisecting line direction vector; and determining the femoral midline based on the first boundary line, the second boundary line, and the bisecting line direction vector.

[0184] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line, the second boundary line, and the direction vector of the bisector includes: if the first boundary line and the second boundary line intersect, then determining the femoral midline corresponding to the third polar coordinate equation based on the intersection point, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector; otherwise, determining the femoral midline corresponding to the third polar coordinate equation based on the midpoint of the perpendicular segment between the first boundary line and the second boundary line, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector.

[0185] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the direction vector corresponding to the femoral region mask image.

[0186] In this embodiment of the disclosure, the values ​​corresponding to the first preset value and the second preset value are the same or different; and / or, the values ​​corresponding to the first preset value and the second preset value are respectively configured to 0.

[0187] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector, the method includes: calculating the femoral principal component direction vector corresponding to the non-zero point coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the femoral principal component direction vector to obtain a corrected femoral principal component direction vector; and performing dot products of the first direction vector and the second direction vector with the corrected femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector.

[0188] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the femoral mask in the femoral region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the femoral principal component direction vector of the corresponding non-zero point coordinate to obtain multiple projection values ​​of the femoral principal component direction vector in the current principal component direction; determining the first non-zero point coordinate corresponding to the minimum projection value and the second non-zero point coordinate corresponding to the maximum projection value among the multiple projection values; calculating the first distance and the second distance between the center point of the femoral region mask image and the first non-zero coordinate and the second non-zero coordinate respectively; determining the direction corresponding to the femoral principal component direction vector based on the first distance and the second distance; and correcting the femoral principal component direction vector based on the direction to obtain the corrected femoral principal component direction vector.

[0189] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the first distance and the second distance includes: if the first distance is less than the second distance, the direction is from the femoral knee joint to the hip joint; otherwise, the direction is from the hip joint to the femoral knee joint.

[0190] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector based on the direction to obtain a corrected femoral principal component direction vector includes: if the direction is from the femoral knee joint to the hip joint, then the femoral principal component direction vector is not corrected; if the direction is from the hip joint to the femoral knee joint, then the direction vector is inverted in the direction to obtain a corrected femoral principal component direction vector.

[0191] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the femoral principal component direction vector corresponding to the femoral region mask image.

[0192] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the modified femoral principal component direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the modified femoral principal component direction vector corresponding to the femoral region mask image.

[0193] This disclosure also proposes a method for determining the trajectory of joint angle changes, comprising: determining the femoral midline and tibial midline corresponding to the initial joint angle among the multiple time-point joint angles; configuring the initial time joint angle corresponding to the initial time joint angle as an axis point; configuring the femoral midline or tibial midline corresponding to the initial time joint angle as a fixed axis line corresponding to the axis point; wherein the coordinate positions of the fixed axis line and the axis point remain unchanged; and rotating the fixed axis line to the tibial midline or femoral midline corresponding to the other time-point angle based on the multiple time-point angles or other time-point angles besides the initial time-point angle, thereby determining the trajectory of joint angle changes. This addresses the current lack of an effective quantitative method for determining the trajectory of knee joint angle changes, which hinders the auxiliary diagnosis of knee joint diseases, assessment of joint degeneration, optimization of athletic performance, guidance of rehabilitation training, and elucidation of joint movement mechanisms.

[0194] For example, the initial joint angle point corresponding to the initial joint angle is configured as the pivot point; the femoral midline corresponding to the initial joint angle point is configured as the fixed axis corresponding to the pivot point; the coordinate positions of the fixed axis and the pivot point are kept unchanged; based on the multiple time angles or other time angles in the multiple time angles except the initial time angle, the fixed axis is rotated to the tibial midline corresponding to other time angles using the pivot point to determine the joint angle change trajectory corresponding to the examinee.

[0195] For example, the initial joint angle point corresponding to the initial joint angle is configured as the pivot point; the tibial midline corresponding to the initial joint angle point is configured as the fixed axis corresponding to the pivot point; the coordinate positions of the fixed axis and the pivot point are kept unchanged; based on the multiple time angles or other time angles among the multiple time angles excluding the initial time angle, the fixed axis is rotated to the femoral midline corresponding to other time angles using the pivot point to determine the joint angle change trajectory corresponding to the examinee.

[0196] This disclosure proposes a dynamic knee X-ray image segmentation method, comprising: constructing a multi-segmentation region weight loss function based on a set of multiple knee X-ray images or a set of dynamic knee X-ray images corresponding to a mask label image; wherein the mask label image includes one or more of the following: patellar mask label, femoral mask label, tibial mask label, and patellar tendon mask label; and using the set of multiple knee X-ray images or the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to perform segmentation on multiple defined segmentation networks. Training is performed to obtain multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image (multi-time dynamic knee X-ray image) segmentation models. These multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models are evaluated to determine the corresponding optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model. Based on the optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model, the patella, femur, tibia, and patellar tendon are segmented from the dynamic knee joint X-ray image to be segmented. This addresses the technical problem that existing technologies cannot accurately and automatically segment one or more of the patella, femur, and tibia, and the patellar tendon.

[0197] In this embodiment, a multi-segmentation region weight loss function is constructed based on a training set of multiple knee X-ray images or a training set of dynamic knee X-ray images corresponding to masked label images. The masked label images include one or more of the following: patellar mask label, femoral mask label, tibial mask label, and patellar tendon mask label. This addresses the technical problem of unbalanced area ratios among the N masked segmentation regions corresponding to the masked label images, resulting in insufficient training of smaller masked segmentation regions, thereby improving the segmentation performance of smaller masked segmentation regions.

[0198] For example, in a training set of multi-time dynamic knee X-ray images (multi-time dynamic knee X-ray imaging images), there is a technical problem that the area ratios of the patellar, femoral, tibial, and patellar tendon masked segments corresponding to the N masked segmentation areas of the masked label images are unbalanced, resulting in the smaller patellar tendon masked segmentation area not being sufficiently trained.

[0199] In embodiments of this disclosure, the step of constructing a multi-segmentation region weight loss function based on a set of multiple knee X-ray images or a set of dynamic knee X-ray images corresponding to a masked label image includes: determining the areas of N masked segmentation regions corresponding to the masked label image in the dynamic knee X-ray image training set; wherein N is greater than 1 or greater than or equal to 2, and N is a positive integer; calculating N ratios between the area of ​​each of the N masked segmentation regions and the total area of ​​the N masked segmentation regions; and determining the loss function weights of each masked segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 masked segmentation regions among the N masked segmentation regions.

[0200] In embodiments of this disclosure, determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas includes: calculating N products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas; and determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on the N products and the sum of the products corresponding to the N products.

[0201] In the embodiments of this disclosure, in determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas, determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N products and the sum of the products corresponding to the N products includes: extracting the product of the mask segmentation region areas that do not contain the weight to be determined from the N different products; calculating the ratio of the product of the mask segmentation region areas that contain the weight to be determined to the sum of the products corresponding to the N different products, and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network.

[0202] For example, the areas of the N masked segmentation regions are configured as A, B, C, and D; calculate N ratios a (a=A / (A+B+C+D)), b (b=B / (A+B+C+D)), c (c=C / (A+B+C+D)), and d (d=D / (A+B+C+D)) for each of the N masked segmentation regions A, B, C, and D, respectively, to the total area of ​​the N masked segmentation regions (A+B+C+D); calculate N different products a*b*c, a*c*d, a*b*d, and b*c*d corresponding to any N-1 masked segmentation region areas among the N masked segmentation regions. Extract the products b*c*d, a*c*d, a*b*d, and a*b*c of the areas of the masked regions A, B, C, and D that do not contain the weight to be determined from the N different products a*b*c, a*c*d, a*b*d, and b*c*d, respectively. Then, calculate the sum of the products b*c*d, a*c*d, a*b*d, and a*b*c of the areas of the masked regions whose weights are to be determined, and the sum of the products of the N different products (a) and b*c*d. The ratio of *b*c+a*c*d+a*b*d+b*c*d is used to determine the loss function weights b*c*d / (a*b*c+a*c*d+a*b*d+b*c*d), a*c*d / (a*b*c+a*c*d+a*b*d+b*c*d), a*b*d / (a*b*c+a*c*d+a*b*d+b*c*d), and a*b*c / (a*b*c+a*c*d+a*b*d+b*c*d) for each mask segmentation region corresponding to the set segmentation network.

[0203] In embodiments of this disclosure, determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas includes: multiplying the N masked segmentation region areas by a plurality of corresponding weight variables to be optimized to obtain N masked segmentation region weight areas; setting the N masked segmentation region weight areas equal to a set value to obtain the N weight variables to be optimized based on the set value and the N masked segmentation region areas; and determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the set value and the N weight variables to be optimized based on the N masked segmentation region areas.

[0204] In the embodiments of this disclosure, when determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas, the determination of the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the defined value and the N masked segmentation region areas includes: calculating the sum of the weight variables to be optimized corresponding to the defined value and the N masked segmentation region areas to obtain the weight variables to be optimized and their expressions; and calculating the ratio between each of the N weight variables to be optimized and their expressions to determine the loss function weights for each masked segmentation region corresponding to the defined segmentation network.

[0205] For example, the areas of the N mask segmentation regions are configured as A, B, C, and D; the areas of the N mask segmentation regions are multiplied by the corresponding multiple weight variables to be optimized ε1, ε2, ε3, and ε4 respectively to obtain the weight areas of the N mask segmentation regions ε1*A, ε2*B, ε3*C, and ε4*D; the weight areas of the N mask segmentation regions ε1*A, ε2*B, ε3*C, and ε4*D are set to equal a predetermined value ε, resulting in the N weight variables to be optimized ε1=ε / A, ε2=ε / B, ε3=ε / C, and ε4=ε / D, represented by the predetermined value and the areas of the N mask segmentation regions; Calculate the sum of the N weight variables to be optimized, represented by the set value and the areas of the N masked segmentation regions, to obtain the weight variables to be optimized and the expression (ε1+ε2+ε3+ε4=ε / A+ε / B+ε / C+ε / D); calculate the ratio ε1 / (ε1+ε2+ε3+ε4) between each of the N weight variables to be optimized represented by the set value and the areas of the N masked segmentation regions and the weight variables to be optimized and the expression. )=(ε / A) / (ε / A+ε / B+ε / C+ε / D), ε2 / (ε1+ε2+ε3+ε4)=(ε / B) / (ε / A+ε / B+ε / C+ε / D), ε3 / (ε1+ε2+ε3+ε4)=(ε / C) / (ε / A+ε / B+ε / C+ε / D), ε4 / (ε1+ε2+ε3+ε4)=(ε / D) / (ε / A+ε / B+ε / C+ε / D), determine the loss function weight of each mask segmentation area corresponding to the set segmentation network.

[0206] In this embodiment of the disclosure, multiple segmentation networks are trained using the set of multiple knee X-ray images or the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to obtain multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image (multi-time dynamic knee X-ray images) segmentation models.

[0207] In embodiments of this disclosure, the plurality of segmentation networks are configured as one or more of FCN, UNet, UPerNet, SegFormer, PSPNet, DeepLabV3, or a combination thereof.

[0208] In the embodiments of this disclosure, the training set of the set dynamic knee X-ray images and their corresponding mask label images, and the weight loss function of the multi-segmentation region are used to train one or more of the following, or a combination of them, of multiple set segmentation networks: FCN, UNet, UPerNet, SegFormer, PSPNet, and DeepLabV3, to obtain multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image (multi-time dynamic knee X-ray image) segmentation models.

[0209] In this embodiment of the disclosure, the plurality of knee joint X-ray image segmentation models or the plurality of dynamic knee joint X-ray image segmentation models are evaluated to determine the corresponding optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model.

[0210] In the embodiments of this disclosure, the evaluation of the plurality of dynamic knee X-ray image segmentation models to determine the corresponding optimal knee X-ray image segmentation model or optimal dynamic knee X-ray image segmentation model includes: determining the plurality of knee X-ray image segmentation models corresponding to the plurality of set segmentation networks under a set dynamic knee X-ray training set and a multi-segmentation region weight loss function (set loss function), or the plurality of dynamic knee X-ray image segmentation models on a set knee X-ray image test set, or the plurality of evaluation scales corresponding to the plurality of dynamic knee X-ray image test sets, whose values ​​are directly proportional or positively correlated with the performance of the segmentation model (the larger the value, the better the performance of the segmentation model), and whose values ​​are directly proportional to or positively correlated with the performance of the segmentation model. Multiple second comprehensive evaluation scales with inverse or negative correlation (the smaller the value, the better the segmentation model performance) are used. First score vectors are determined for the multiple knee X-ray image segmentation models or the multiple dynamic knee X-ray image segmentation models under the multiple first comprehensive evaluation scales, and second score vectors are determined for the multiple dynamic knee X-ray image segmentation models under the multiple second comprehensive evaluation scales. Based on the first score vectors and the second score vectors, the multiple knee X-ray image segmentation models or the multiple dynamic knee X-ray image segmentation models are evaluated, and the corresponding optimal knee X-ray image segmentation model or optimal dynamic knee X-ray image segmentation model is determined. To address the technical problem that improvements to multiple knee X-ray image segmentation models or dynamic knee X-ray image segmentation models often fail to achieve across all evaluation scales, thus hindering the selection of appropriate models, the aim is to improve the performance of multiple knee X-ray image segmentation models or dynamic knee X-ray image segmentation models.

[0211] In this embodiment of the disclosure, the plurality of evaluation metrics are configured as any of the following: IoU (Intersection over Union), Dice, precision, recall, Hausdorff distance (HD) or median 95th Hausdorff distance (HD95), and average symmetric surface distance (ASSD); and / or, the plurality of first comprehensive evaluation metrics are configured as any one or more of IoU, Dice, precision, and recall; and / or, the plurality of second comprehensive evaluation metrics are any one or more of Hausdorff distance or median 95th Hausdorff distance and average symmetric surface distance.

[0212] In this embodiment of the disclosure, determining multiple knee X-ray image segmentation models corresponding to multiple set segmentation networks under a set of multiple knee X-ray images or a set of dynamic knee X-ray images and a set loss function, or multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray images, includes: calculating multiple knee X-ray image segmentation models corresponding to multiple set segmentation networks under a set of multiple knee X-ray images training set or a set of dynamic knee X-ray images test set, where the values ​​of these multiple evaluation metrics are positively proportional or positively correlated with the performance of the segmentation models, and multiple second comprehensive evaluation metrics where the values ​​of these metrics are inversely proportional or negatively correlated with the performance of the segmentation models. The dynamic knee X-ray image segmentation model uses multiple evaluation scales corresponding to a set of dynamic knee X-ray images. It then determines a first set of evaluation scales whose values ​​are positively proportional or correlated with the segmentation model performance, and a second set of evaluation scales whose values ​​are inversely proportional or negatively correlated with the segmentation model performance. Based on the first set of evaluation scales and the second set of evaluation scales, it determines multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales.

[0213] In this embodiment of the disclosure, determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively includes: summing the first set of evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray image test sets to determine multiple first comprehensive evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models; and summing the second set of evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray image test sets to determine multiple second comprehensive evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models.

[0214] In this embodiment of the disclosure, the step of determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively further includes: determining a first number corresponding to the evaluation scales in the first set of evaluation scales and a second number corresponding to the evaluation scales in the second set of evaluation scales respectively; averaging the multiple first comprehensive evaluation scales corresponding to the multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models using the first number respectively to obtain the final multiple first comprehensive evaluation scales; and averaging the multiple second comprehensive evaluation scales corresponding to the multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models using the second number respectively to obtain the final multiple second comprehensive evaluation scales.

[0215] In this embodiment of the disclosure, the step of determining the first score vector corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models under multiple first comprehensive evaluation scales and the second score vector corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models under multiple second comprehensive evaluation scales includes: sorting the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models according to the set order of the multiple first comprehensive evaluation scales (from largest to smallest or from smallest to largest) to determine the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple first comprehensive evaluation scales; and sorting the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models according to the set order of the multiple second comprehensive evaluation scales (from largest to smallest or from smallest to largest) to determine the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple second comprehensive evaluation scales.

[0216] In this embodiment of the disclosure, the step of determining the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple first comprehensive evaluation scales and the second score vector corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models under multiple second comprehensive evaluation scales further includes: determining the maximum score value among the first score vector and the second score vector; determining the score corresponding to the knee X-ray image segmentation model or multiple dynamic knee X-ray image segmentation model with the largest first comprehensive evaluation scale among the multiple first comprehensive evaluation scales as the maximum score value; and sorting the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models according to a predetermined order (from largest to smallest or from smallest to largest) corresponding to the multiple first comprehensive evaluation scales, based on the maximum score value and a predetermined score difference between any two adjacent scores in the first score vector. The image segmentation model is scored and assigned a first score vector to determine the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models under multiple first comprehensive evaluation scales; the score corresponding to the knee X-ray image segmentation model or multiple dynamic knee X-ray image segmentation model with the smallest second comprehensive evaluation scale among the multiple second comprehensive evaluation scales is determined as the maximum score value; based on the maximum score value and the set score difference between any two adjacent scores in the second score vector, the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models, after being sorted according to the set order (from largest to smallest or from smallest to largest) corresponding to the multiple second comprehensive evaluation scales, are scored and assigned a second score vector to determine the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models under multiple second comprehensive evaluation scales.

[0217] In this embodiment of the disclosure, determining the maximum score value among the first score vector and the second score vector includes: determining the maximum score value among the first score vector and the second score vector based on the number of the plurality of knee joint X-ray image segmentation models or the plurality of dynamic knee joint X-ray image segmentation models.

[0218] For example, the maximum score value in the first and second score vectors is configured to be 50; or, the number of the multiple knee joint X-ray image segmentation models or the multiple dynamic knee joint X-ray image segmentation models is configured to be 50, then the maximum score value in the first and second score vectors is configured to be 50. In this case, the set score difference between any two adjacent scores in the first and second score vectors is configured to be 1, and the determined first and second score vectors are configured as [50, 49, 48, …, 3, 2, 1] or [1, 2, 3, …, 48, 49, 50], respectively.

[0219] In this embodiment of the disclosure, the evaluation of the plurality of knee joint X-ray image segmentation models or the plurality of dynamic knee joint X-ray image segmentation models based on the first scoring vector and the second scoring vector includes: constructing a correlation relationship between the first scoring vector and the second scoring vector according to the plurality of knee joint X-ray image segmentation models or the plurality of dynamic knee joint X-ray image segmentation models; calculating the sum of the first correlation score and the second correlation score in the first scoring vector and the second scoring vector of the plurality of knee joint X-ray image segmentation models or the plurality of dynamic knee joint X-ray image segmentation models under the correlation relationship, corresponding to a plurality of joint scores; and evaluating the plurality of knee joint X-ray image segmentation models or the plurality of dynamic knee joint X-ray image segmentation models based on the plurality of joint scores.

[0220] In this embodiment of the disclosure, the step of constructing the association between the first scoring vector and the second scoring vector based on the plurality of knee X-ray image segmentation models or the plurality of dynamic knee X-ray image segmentation models includes: extracting the first score and the second score corresponding to the same knee X-ray image segmentation model or the same dynamic knee X-ray image segmentation model from the first scoring vector and the second scoring vector, respectively, to construct the association between the first scoring vector and the second scoring vector; or, determining the association labels of the first score in the first scoring vector and the second score in the second scoring vector corresponding to the same dynamic knee X-ray image segmentation model in the plurality of dynamic knee X-ray image segmentation models, respectively; and constructing the association between the first scoring vector and the second scoring vector based on the association labels.

[0221] In this embodiment of the disclosure, the evaluation of the multiple knee joint X-ray image segmentation models or the multiple dynamic knee joint X-ray image segmentation models based on the multiple joint scores includes: sorting the multiple joint scores to obtain a sorted joint score vector; and evaluating the multiple dynamic knee joint X-ray image segmentation models based on the sorted joint score vector; wherein the joint score in the sorted joint score vector is proportional to the performance of the dynamic knee joint X-ray image segmentation model.

[0222] In this embodiment of the disclosure, determining multiple dynamic knee X-ray image segmentation models corresponding to multiple defined segmentation networks includes: determining target segmentation regions corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models; if the number of target segmentation regions is greater than 1, determining the areas of multiple masked segmentation regions corresponding to the masked label images in the training set of dynamic knee X-ray images corresponding to the multiple defined segmentation networks; if the difference or ratio between the largest and smallest masked segmentation region areas among the multiple masked segmentation region areas is greater than a set difference or a set ratio, optimizing the defined loss function corresponding to the multiple defined segmentation networks to obtain an optimized loss function; and training the multiple defined segmentation networks based on the dynamic knee X-ray image training set and the optimized loss function to determine multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models corresponding to the multiple defined segmentation networks.

[0223] In this embodiment, optimizing the set loss function corresponding to the plurality of set segmentation networks to obtain an optimized loss function includes: determining the areas of N masked segmentation regions corresponding to the masked label images in a set of multiple knee X-ray images or a set of dynamic knee X-ray images training sets corresponding to the set segmentation networks; wherein N is greater than 1 or greater than or equal to 2, and N is a positive integer; calculating N ratios of each masked segmentation region area to the total area of ​​the N masked segmentation regions; and determining the loss function weights of each masked segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 masked segmentation region areas, thereby obtaining the optimized loss function. This addresses the technical problem that the unbalanced proportion of the N masked segmentation region areas corresponding to the masked label images prevents smaller masked segmentation regions from being adequately trained, thereby improving the segmentation performance of smaller masked segmentation regions.

[0224] In this embodiment of the disclosure, training multiple segmentation networks using the set of multiple knee X-ray images or the set of dynamic knee X-ray images and their corresponding masked images, and the multi-segmentation region weight loss function, includes: obtaining a set of dynamic knee X-ray images and the corresponding masked images of the set of dynamic knee X-ray images, and setting a maximum number of iterations; if the number of iterations of the segmentation network is less than the set maximum number of iterations and the set of multiple knee X-ray images or the set of dynamic knee X-ray images... If the loss value between the training set of X-ray images and their corresponding masked images is less than a set loss value, and the first average intersection-union ratio (IU) between the set dynamic knee X-ray image validation set and its corresponding masked images at a first set number of iterations is greater than a set first average IU, then the second average IU between the smallest masked segmentation region among the N masked segmentation regions corresponding to the masked images and the smallest masked segmentation region (patellar tendon) in the set dynamic knee X-ray image validation set is greater than a set second average IU, then the set segmentation network is controlled to stop training. This addresses at least one technical problem in multi-target region segmentation models, such as overfitting, wasted computational resources and time, difficulty in capturing the optimal state of the multi-target region segmentation model, and poor robustness of the multi-target region segmentation model.

[0225] In this embodiment of the disclosure, the Intersection over Union (IoU) ratio is the ratio of the intersection area to the union area of ​​two regions, and is used to measure the similarity between the predicted segmentation result and the actual segmentation result.

[0226] In this embodiment of the disclosure, it further includes: if the number of iterations of the set segmentation network reaches the set maximum number of iterations, then controlling the set segmentation network to stop training.

[0227] In this embodiment of the disclosure, the method further includes: if the number of iterations of the defined segmentation network is less than a defined maximum number of iterations, and the loss value between the defined training set of multiple knee X-ray images or the defined dynamic knee X-ray image training set and their corresponding mask label images is less than a defined loss value, and the first average intersection-union ratio (IUU) between the defined dynamic knee X-ray image validation set and their corresponding mask label images at the first defined number of iterations is greater than a defined first average IUU value, and the second average IUU between the smallest mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest mask segmentation region in the defined dynamic knee X-ray image validation set is greater than a defined second average IUU value, then a third average IUUU between the defined dynamic knee X-ray image validation set and their corresponding mask label images is calculated within the second defined number of iterations; if the third average IUUU is less than or equal to a defined fluctuation value, then the defined segmentation network is controlled to stop training.

[0228] In this embodiment of the disclosure, the method further includes: constructing a multi-segmentation region weight loss function based on the loss function weights of each of the N masked segmentation regions; and training the set segmentation network using the set of multiple knee X-ray images or the set of dynamic knee X-ray images and their corresponding masked label images, and the multi-segmentation region weight loss function.

[0229] In this embodiment of the disclosure, the step of constructing a multi-segmentation region weighted loss function based on the loss function weights of each masked segmentation region includes: obtaining the base loss function corresponding to each masked segmentation region; multiplying the loss function weights of each masked segmentation region by their respective base loss functions and summing the results to construct the multi-segmentation region weighted loss function; wherein the base loss function corresponding to the loss function weights is configured as a cross-entropy loss function or a weighted cross-entropy loss function.

[0230] In this embodiment, the method further includes: summing the boundary loss functions corresponding to each masked segmentation region or summing the corresponding multi-segmentation region boundary loss function after multiplying the loss function weights of each masked segmentation region by their respective boundary loss functions; summing the DICE loss functions corresponding to each masked segmentation region or summing the corresponding multi-segmentation region DICE loss function after multiplying the loss function weights of each masked segmentation region by their respective DICE loss functions; constructing a multivariate loss function based on one or two of the segmentation region weight loss function, multi-segmentation region boundary loss function, or multi-segmentation region DICE loss function; and training the specified segmentation network using the specified training set of multiple knee X-ray images or the specified dynamic knee X-ray image training set and its corresponding mask label images, and the multivariate loss function. By using the multivariate loss function corresponding to the multivariate loss function and one or two of the multivariate loss functions (multi-segmentation region weight loss function and boundary loss function or multi-segmentation region DICE loss function), the parameter training direction of the specified segmentation network is guided, thereby solving the technical problem that a univariate loss function cannot comprehensively train the specified segmentation network.

[0231] For example, firstly, one of the basic criteria for stopping network training is that the loss value between the knee multi-object segmentation images (setting a training set of multiple knee X-ray images or setting a training set of dynamic knee X-ray images) and the ground truth (GT) (mask label / mask label image) in its training set is less than 0.2 (loss setting). Secondly, the overall segmentation performance of the model in four object regions (segmentation regions corresponding to the patella, femur, tibia, and patellar tendon) is evaluated every 50 iterations (first set number of iterations) using a validation set and the knee multi-object segmentation model. Based on the loss value between the knee multi-object segmentation images and the GT in its training set being less than 0.2, a first average intersection-over-union (IoU) ratio greater than 0.8 (first average IoU setting) between the knee multi-object segmentation images and the GTs in its validation set is set as the second basic criterion for stopping network training. Thirdly, due to the small area of ​​the patellar tendon (the smallest mask segmentation region), it is difficult to achieve good performance. To provide more comprehensive training for the patellar tendon, a third criterion for stopping training is set: an average IoU greater than 0.85 (the second average intersection-union ratio setting) based on the patellar tendon object segmentation images and their ground truth (GT). Finally, after meeting these three criteria, network training can be stopped if the total average IoU between the four object segmentation regions and their GTs in the validation set remains unchanged for 10 consecutive iterations (the second set number of iterations). This means the relative fluctuation of the total average IoU between the four object segmentation regions and their GTs in the ten validation sets is less than or equal to the set fluctuation value of 0.001. If these three criteria are not met, network training stops when the maximum number of iterations is reached. After network training is complete, the segmentation model that performs best on the validation set is selected for testing on the test set.

[0232] In this embodiment of the disclosure, the method further includes: dividing the learning rate decay of the optimizer corresponding to the specified segmentation network into an initial learning rate decay phase, a stable learning rate decay period, and a fine-tuning learning rate decay phase; wherein, the initial learning rate decay phase uses warm-up steps to ensure rapid gradient descent in the initial stage of training the specified segmentation network while maintaining the stability of the specified segmentation network; the stable learning rate decay period uses polynomial decay to maintain a relatively stable learning rate; and the fine-tuning learning rate decay phase uses cosine learning rate decay.

[0233] For example, in network training, the learning rate is a key hyperparameter determining model convergence. Dynamic learning rates, by adaptively adjusting the learning rate during training, can achieve rapid convergence early on and fine-tune later, resulting in better model performance. In this study, the AdamW optimizer or a modified AdamW optimizer was used with an initial learning rate of 1e-4. The AdamW optimizer's learning rate decay includes an initial learning rate decay phase, a stable learning rate decay phase, and a fine-tuning learning rate decay phase. The initial learning rate decay phase, based on the initial learning rate, uses warmup steps, which provides rapid gradient descent in the initial stages of network training while maintaining stability. The stable learning rate decay phase uses polynomial decay, maintaining a relatively stable learning rate during long-term network training. The fine-tuning learning rate decay phase uses cosine learning rate decay, applied at the end of training.

[0234] In this embodiment of the disclosure, the multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models corresponding to the multiple set segmentation networks are configured as multi-target segmentation regions that are consistent with the N mask segmentation regions corresponding to the mask label image.

[0235] In this embodiment of the disclosure, the multi-target segmentation region that corresponds to the N mask segmentation regions of the mask label image is configured as two or more of the following: patella, femur, tibia and patellar tendon.

[0236] In this embodiment of the disclosure, based on the optimal knee X-ray image segmentation model or the optimal dynamic knee X-ray image segmentation model, the dynamic knee X-ray image to be segmented or processed is segmented into one or more of the patella, femur, tibia and patellar tendon.

[0237] In this embodiment of the disclosure, the method further includes: before training multiple segmentation networks using a set of multiple knee X-ray images or a set of dynamic knee X-ray images and their corresponding mask labels to obtain multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models, the method includes: extracting each dynamic knee X-ray image from the preset dynamic knee X-ray image training set; wherein, each dynamic knee X-ray image includes: multiple knee X-ray images at multiple times corresponding to the same subject; calculating the similarity between multiple preset labeled knee X-ray images and multiple knee X-ray images corresponding to multiple knee X-ray images in each dynamic knee X-ray image; if the multiple image similarity is less than or equal to a preset image similarity, then the knee X-ray image corresponding to the preset image similarity is determined as a dynamic knee X-ray image to be labeled. To address the challenges of X-ray images with similarity levels below a preset threshold hindering the training of segmentation networks and preventing the acquisition of highly generalized segmentation models, the following measures are proposed: Firstly, determining the X-ray images to be labeled in the dynamic X-ray image training set according to preset time intervals introduces a high degree of subjectivity. Secondly, labeling all images in the dynamic X-ray image training set presents not only the technical problem of X-ray images with similarity levels below a preset threshold hindering the training of segmentation networks and preventing the acquisition of highly generalized segmentation models, but also presents at least one of the technical challenges inherent in the heavy physician annotation task. These measures aim to improve the generalization of the segmentation model.

[0238] In this embodiment of the disclosure, the image similarity can be configured as the mean square error corresponding to the average of the squared differences of the corresponding pixel values ​​of two images, extracting key points of the image and their descriptors, and calculating one or more of the similarity based on features, hash-based similarity, and statistical similarity (histogram similarity or structural similarity index) by matching descriptors.

[0239] In this embodiment of the disclosure, the first (first or first) preset labeled X-ray image among the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to the initial moment in each dynamic X-ray image; and / or, the last (last) preset labeled X-ray image among the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to the last moment in each dynamic X-ray image.

[0240] In this embodiment of the disclosure, it further includes: if the initial preset labeled X-ray image corresponding to the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to a non-initial time in each example of dynamic X-ray images, then the similarity between the initial preset labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the plurality of times before the initial preset labeled X-ray image is also calculated.

[0241] In this embodiment of the disclosure, it further includes: if the last preset-labeled X-ray image corresponding to the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to a non-last moment in each example of dynamic X-ray images, then the similarity between the last preset-labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the times after the last preset-labeled X-ray image is also calculated.

[0242] In this embodiment of the disclosure, the preset dynamic X-ray image training set is configured as a dynamic knee X-ray image training set or a dynamic chest X-ray image training set. Specifically, each dynamic X-ray image in the dynamic knee X-ray image training set or the dynamic chest X-ray image training set is a multi-time knee X-ray image or chest X-ray image of the same subject, or multiple knee X-ray images or chest X-ray images of multiple different subjects.

[0243] This disclosure proposes a dynamic knee X-ray image segmentation method, comprising: constructing a multi-segmentation region weight loss function based on a set of multiple knee X-ray images or a set of dynamic knee X-ray images corresponding to a mask label image; wherein the mask label image includes one or more of the following: patellar mask label, femoral mask label, tibial mask label, and patellar tendon mask label; and using the set of multiple knee X-ray images or the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to perform segmentation on multiple defined segmentation networks. Training is performed to obtain multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image (multi-time dynamic knee X-ray image) segmentation models. These multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models are evaluated to determine the corresponding optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model. Based on the optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model, the patella, femur, tibia, and patellar tendon are segmented from the dynamic knee joint X-ray image to be segmented. This addresses the technical problem that existing technologies cannot accurately and automatically segment one or more of the patella, femur, and tibia, and the patellar tendon.

[0244] In this embodiment, a multi-segmentation region weight loss function is constructed based on a training set of multiple knee X-ray images or a training set of dynamic knee X-ray images corresponding to masked label images. The masked label images include one or more of the following: patellar mask label, femoral mask label, tibial mask label, and patellar tendon mask label. This addresses the technical problem of unbalanced area ratios among the N masked segmentation regions corresponding to the masked label images, resulting in insufficient training of smaller masked segmentation regions, thereby improving the segmentation performance of smaller masked segmentation regions.

[0245] For example, in a training set of multi-time dynamic knee X-ray images (multi-time dynamic knee X-ray imaging images), there is a technical problem that the area ratios of the patellar, femoral, tibial, and patellar tendon masked segments corresponding to the N masked segmentation areas of the masked label images are unbalanced, resulting in the smaller patellar tendon masked segmentation area not being sufficiently trained.

[0246] In embodiments of this disclosure, the step of constructing a multi-segmentation region weight loss function based on a set of multiple knee X-ray images or a set of dynamic knee X-ray images corresponding to a masked label image includes: determining the areas of N masked segmentation regions corresponding to the masked label image in the dynamic knee X-ray image training set; wherein N is greater than 1 or greater than or equal to 2, and N is a positive integer; calculating N ratios between the area of ​​each of the N masked segmentation regions and the total area of ​​the N masked segmentation regions; and determining the loss function weights of each masked segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 masked segmentation regions among the N masked segmentation regions.

[0247] In embodiments of this disclosure, determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas includes: calculating N products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas; and determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on the N products and the sum of the products corresponding to the N products.

[0248] In the embodiments of this disclosure, in determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas, determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N products and the sum of the products corresponding to the N products includes: extracting the product of the mask segmentation region areas that do not contain the weight to be determined from the N different products; calculating the ratio of the product of the mask segmentation region areas that contain the weight to be determined to the sum of the products corresponding to the N different products, and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network.

[0249] In this embodiment of the disclosure, multiple segmentation networks are trained using the set of multiple knee X-ray images or the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to obtain multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image (multi-time dynamic knee X-ray images) segmentation models.

[0250] In this embodiment of the disclosure, the plurality of knee joint X-ray image segmentation models or the plurality of dynamic knee joint X-ray image segmentation models are evaluated to determine the corresponding optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model.

[0251] In the embodiments of this disclosure, the evaluation of the plurality of dynamic knee X-ray image segmentation models to determine the corresponding optimal knee X-ray image segmentation model or optimal dynamic knee X-ray image segmentation model includes: determining the plurality of knee X-ray image segmentation models corresponding to the plurality of set segmentation networks under a set dynamic knee X-ray training set and a multi-segmentation region weight loss function (set loss function), or the plurality of dynamic knee X-ray image segmentation models on a set knee X-ray image test set, or the plurality of evaluation scales corresponding to the plurality of dynamic knee X-ray image test sets, whose values ​​are directly proportional or positively correlated with the performance of the segmentation model (the larger the value, the better the performance of the segmentation model), and whose values ​​are directly proportional to or positively correlated with the performance of the segmentation model. Multiple second comprehensive evaluation scales with inverse or negative correlation (the smaller the value, the better the segmentation model performance) are used. First score vectors are determined for the multiple knee X-ray image segmentation models or the multiple dynamic knee X-ray image segmentation models under the multiple first comprehensive evaluation scales, and second score vectors are determined for the multiple dynamic knee X-ray image segmentation models under the multiple second comprehensive evaluation scales. Based on the first score vectors and the second score vectors, the multiple knee X-ray image segmentation models or the multiple dynamic knee X-ray image segmentation models are evaluated, and the corresponding optimal knee X-ray image segmentation model or optimal dynamic knee X-ray image segmentation model is determined. To address the technical problem that improvements to multiple knee X-ray image segmentation models or dynamic knee X-ray image segmentation models often fail to achieve across all evaluation scales, thus hindering the selection of appropriate models, the aim is to improve the performance of multiple knee X-ray image segmentation models or dynamic knee X-ray image segmentation models.

[0252] In this embodiment of the disclosure, determining multiple knee X-ray image segmentation models corresponding to multiple set segmentation networks under a set of multiple knee X-ray images or a set of dynamic knee X-ray images and a set loss function, or multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray images, includes: calculating multiple knee X-ray image segmentation models corresponding to multiple set segmentation networks under a set of multiple knee X-ray images training set or a set of dynamic knee X-ray images test set, where the values ​​of these multiple evaluation metrics are positively proportional or positively correlated with the performance of the segmentation models, and multiple second comprehensive evaluation metrics where the values ​​of these metrics are inversely proportional or negatively correlated with the performance of the segmentation models. The dynamic knee X-ray image segmentation model uses multiple evaluation scales corresponding to a set of dynamic knee X-ray images. It then determines a first set of evaluation scales whose values ​​are positively proportional or correlated with the segmentation model performance, and a second set of evaluation scales whose values ​​are inversely proportional or negatively correlated with the segmentation model performance. Based on the first set of evaluation scales and the second set of evaluation scales, it determines multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales.

[0253] In this embodiment of the disclosure, determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively includes: summing the first set of evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray image test sets to determine multiple first comprehensive evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models; and summing the second set of evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models on a set of dynamic knee X-ray image test sets to determine multiple second comprehensive evaluation scales corresponding to the multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models.

[0254] In this embodiment of the disclosure, the step of determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively further includes: determining a first number corresponding to the evaluation scales in the first set of evaluation scales and a second number corresponding to the evaluation scales in the second set of evaluation scales respectively; averaging the multiple first comprehensive evaluation scales corresponding to the multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models using the first number respectively to obtain the final multiple first comprehensive evaluation scales; and averaging the multiple second comprehensive evaluation scales corresponding to the multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models using the second number respectively to obtain the final multiple second comprehensive evaluation scales.

[0255] The dynamic knee joint X-ray image segmentation method can be executed by a dynamic knee joint X-ray image segmentation device or system. For example, the dynamic knee joint X-ray image segmentation method can be executed by a terminal device, server, or other processing device. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc. In some possible implementations, the dynamic knee joint X-ray image segmentation method can be implemented by a processor calling computer-readable instructions stored in memory.

[0256] Those skilled in the art will understand that, in the above-described dynamic knee joint X-ray image segmentation method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0257] According to one aspect of this disclosure, a dynamic knee X-ray image segmentation system is provided, comprising: a construction unit, configured to construct a multi-segmentation region weight loss function based on mask label images corresponding to a set of dynamic knee X-ray images; wherein the mask label images include one or more of patellar mask labels, femoral mask labels, tibial mask labels, and patellar tendon mask labels; a training unit, configured to train multiple set segmentation networks using the set of dynamic knee X-ray images and their corresponding mask label images, and the multi-segmentation region weight loss function, to obtain multiple dynamic knee X-ray image (multi-time dynamic knee X-ray image) segmentation models; a determination unit, configured to evaluate the multiple dynamic knee X-ray image segmentation models and determine the corresponding optimal dynamic knee X-ray image segmentation model; and a segmentation unit, configured to segment the dynamic knee X-ray images to be segmented based on the optimal dynamic knee X-ray image segmentation model. The method comprises: segmenting knee X-ray images into one or more of the patella, femur, tibia, and patellar tendon; or, comprising: an electronic device; the electronic device being configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic knee X-ray image segmentation method; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic knee X-ray image segmentation method; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above-described dynamic knee X-ray image segmentation method; or, comprising: a computer program product, the computer program product being configured with a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the above-described dynamic knee X-ray image segmentation method.

[0258] In embodiments of this disclosure, the computer-readable storage medium stores a computer program / instructions and a bit stream thereon, wherein the computer program / instructions, when executed by a processor, implement the method described above to generate the bit stream.

[0259] According to one aspect of this disclosure, an X-ray camera is provided, comprising: the dynamic knee joint X-ray image segmentation system as described above; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the dynamic knee joint X-ray image segmentation method described above; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing the dynamic knee joint X-ray image segmentation method when executed by a processor; or, comprising: a computer program product configured with computer programs / instructions implementing the dynamic knee joint X-ray image segmentation method when executed by a processor.

[0260] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above embodiments of the dynamic knee joint X-ray imaging image segmentation method. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0261] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for segmenting dynamic knee joint X-ray images, characterized in that, include: Based on the mask label images corresponding to the set of dynamic knee joint X-ray images training set, a multi-segmentation region weight loss function is constructed; wherein, the mask label images include: one or more of the following: patellar mask label, femoral mask label, tibial mask label, and patellar tendon mask label; Using the set of dynamic knee X-ray images and their corresponding masked labels, and the multi-segmentation region weight loss function, multiple segmentation networks are trained to obtain multiple dynamic knee X-ray image segmentation models. The multiple dynamic knee joint X-ray image segmentation models are evaluated to determine the corresponding optimal dynamic knee joint X-ray image segmentation model. Based on the optimal dynamic knee X-ray image segmentation model, the dynamic knee X-ray image to be segmented is segmented into one or more of the patella, femur, tibia and patellar tendon.

2. The dynamic knee joint X-ray image segmentation method according to claim 1, characterized in that, The step of constructing a multi-segmentation region weight loss function based on the mask label image corresponding to the set of dynamic knee X-ray images includes: determining the area of ​​N mask segmentation regions corresponding to the mask label image in the set of dynamic knee X-ray images; calculating N ratios of the area of ​​each of the N mask segmentation regions to the total area of ​​the N mask segmentation regions; and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 mask segmentation regions among the N mask segmentation regions.

3. The dynamic knee joint X-ray image segmentation method according to claim 2, characterized in that, The step of determining the loss function weight for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas includes: calculating N products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas; determining the loss function weight for each masked segmentation region corresponding to the defined segmentation network based on the N products and the sum of the products corresponding to the N products; and / or, In determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas, the determination of the loss function weight of each mask segmentation region corresponding to the defined segmentation network based on the N products and the sum of the products corresponding to the N products includes: extracting the product of the mask segmentation region areas that do not contain the weight to be determined from the N different products; calculating the ratio of the product of the mask segmentation region areas that contain the weight to be determined to the sum of the products corresponding to the N different products, and determining the loss function weight of each mask segmentation region corresponding to the defined segmentation network.

4. The dynamic knee joint X-ray image segmentation method according to claim 2, characterized in that, The step of determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas from the N masked segmentation region areas includes: multiplying the N masked segmentation region areas by a plurality of corresponding weight variables to be optimized to obtain N masked segmentation region weight areas; setting the N masked segmentation region weight areas equal to a set value to obtain the N weight variables to be optimized based on the set value and the N masked segmentation region areas; determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the set value and the N masked segmentation region areas; and / or, In determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas, the loss function weights for each masked segmentation region corresponding to the defined segmentation network are determined based on the defined value and the N masked segmentation region areas. This includes: calculating the sum of the weight variables to be optimized corresponding to the defined value and the N masked segmentation region areas to obtain the weight variables to be optimized and their expressions; and calculating the ratio between each of the N weight variables to be optimized and their expressions to determine the loss function weights for each masked segmentation region corresponding to the defined segmentation network.

5. The dynamic knee joint X-ray image segmentation method according to any one of claims 1-4, characterized in that, The evaluation of the multiple dynamic knee X-ray image segmentation models to determine the corresponding optimal dynamic knee X-ray image segmentation model includes: determining multiple first comprehensive evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models under a set dynamic knee X-ray training set and a multi-segmentation region weight loss function, and multiple second comprehensive evaluation scales corresponding to multiple evaluation scales on a set dynamic knee X-ray test set, where the values ​​of these scales are positively proportional or positively correlated with the performance of the segmentation models, and the values ​​of these second comprehensive evaluation scales are inversely proportional or negatively correlated with the performance of the segmentation models; determining the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple first comprehensive evaluation scales and the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple second comprehensive evaluation scales; and evaluating the multiple dynamic knee X-ray image segmentation models based on the first score vector and the second score vector to determine the corresponding optimal dynamic knee X-ray image segmentation model.

6. The dynamic knee joint X-ray image segmentation method according to claim 5, characterized in that, The determination of multiple evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models of multiple defined segmentation networks on a defined dynamic knee X-ray image training set and a defined loss function, and multiple first comprehensive evaluation scales whose values ​​are positively proportional or positively correlated with the segmentation model performance, and multiple second comprehensive evaluation scales whose values ​​are inversely proportional or negatively correlated with the segmentation model performance, includes: calculating multiple evaluation scales corresponding to multiple dynamic knee X-ray image segmentation models of multiple defined segmentation networks on a defined dynamic knee X-ray image test set; determining the first set of evaluation scales whose values ​​are positively proportional or positively correlated with the segmentation model performance and the second set of evaluation scales whose values ​​are inversely proportional or negatively correlated with the segmentation model performance among the multiple evaluation scales corresponding to the multiple dynamic knee X-ray image segmentation models on the defined dynamic knee X-ray image test set; determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales; and / or, The step of determining the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple first comprehensive evaluation scales and the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under multiple second comprehensive evaluation scales includes: sorting the multiple dynamic knee X-ray image segmentation models according to the set order of the multiple first comprehensive evaluation scales to determine the first score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple first comprehensive evaluation scales; and sorting the multiple dynamic knee X-ray image segmentation models according to the set order of the multiple second comprehensive evaluation scales to determine the second score vector corresponding to the multiple dynamic knee X-ray image segmentation models under the multiple second comprehensive evaluation scales.

7. The dynamic knee joint X-ray image segmentation method according to any one of claims 1-6, characterized in that, The step of training multiple segmentation networks using the set of dynamic knee X-ray images and their corresponding masked images, and the multi-segmentation region weight loss function, includes: obtaining a set of dynamic knee X-ray images and the corresponding masked images, and setting a maximum number of iterations; if the number of iterations of the segmentation network is less than the maximum number of iterations, and the loss value between the set of dynamic knee X-ray images and their corresponding masked images is less than a set loss value, and the first average intersection-union ratio (IUU) between the set of dynamic knee X-ray images and their corresponding masked images at the first set number of iterations is greater than the first average IUU set value, then the second average IUU ratio of the smallest masked segmentation region in the N masked segmentation regions corresponding to the masked images and the smallest masked segmentation region in the set of dynamic knee X-ray images is greater than the second average IUU set value, then the segmentation network is stopped from training; and / or, Each dynamic X-ray image in the training set of dynamic knee X-ray images is a multi-time knee X-ray image corresponding to the same subject or multiple knee X-ray images corresponding to multiple different subjects.

8. The dynamic knee joint X-ray image segmentation method according to claim 7, characterized in that, Also includes: If the number of iterations of the defined segmentation network reaches the defined maximum number of iterations, then the defined segmentation network is controlled to stop training; And / or, It also includes: if the number of iterations of the set segmentation network is less than the set maximum number of iterations, and the loss value between the set dynamic knee X-ray image training set and its corresponding mask label image is less than the set loss value, and the first average intersection-union ratio between the set dynamic knee X-ray image validation set and its corresponding mask label image at the first set number of iterations is greater than the set first average intersection-union ratio, and the second average intersection-union ratio between the smallest mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest mask segmentation region in the set dynamic knee X-ray image validation set is greater than the set second average intersection-union ratio, then calculate the third average intersection-union ratio between the set dynamic knee X-ray image validation set and its corresponding mask label image within the second set number of iterations; If the third average crossover ratio is less than or equal to the set fluctuation value, then the set segmentation network is controlled to stop training.

9. A dynamic knee joint X-ray imaging image segmentation system, characterized in that, include: A construction unit is used to construct a multi-segmentation region weight loss function based on the mask label images corresponding to a set of dynamic knee X-ray images. The mask label images include one or more of the following: patellar mask label, femoral mask label, tibial mask label, and patellar tendon mask label. A training unit is used to train multiple segmentation networks using the set of dynamic knee X-ray images, their corresponding mask label images, and the multi-segmentation region weight loss function to obtain multiple dynamic knee X-ray image segmentation models. A determination unit is used to evaluate the multiple dynamic knee X-ray image segmentation models and determine the corresponding optimal dynamic knee X-ray image segmentation model. A segmentation unit is used to segment the dynamic knee X-ray image to be segmented based on the optimal dynamic knee X-ray image segmentation model, performing segmentation of the patella, femur, tibia, and patellar tendon into the patella. Includes: an electronic device; said electronic device is configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the dynamic knee joint X-ray imaging segmentation method according to any one of claims 1 to 8; or, Includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the dynamic knee joint X-ray imaging segmentation method according to any one of claims 1 to 8; or, Includes: a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the dynamic knee joint X-ray imaging segmentation method according to any one of claims 1 to 8; or, Includes: a computer program product configured with a computer program / instruction, which, when executed by a processor, implements the dynamic knee joint X-ray imaging image segmentation method according to any one of claims 1 to 8.

10. An X-ray camera, comprising: The dynamic knee joint X-ray imaging image segmentation system as described in claim 9; or, Includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the dynamic knee joint X-ray imaging segmentation method according to any one of claims 1 to 8; or, Includes: a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the dynamic knee joint X-ray imaging segmentation method according to any one of claims 1 to 8; or, Includes: a computer program product configured with a computer program / instruction, which, when executed by a processor, implements the dynamic knee joint X-ray imaging image segmentation method according to any one of claims 1 to 8.

Images