Dynamic x-ray image determination and segmentation method, system and x-ray camera

Abstract

The disclosure provides a dynamic X-ray image to be labeled determination and segmentation method and system and an X-ray machine, and relates to the technical field of dynamic X-ray image to be labeled. The dynamic X-ray image to be labeled determination method comprises: extracting each dynamic X-ray image in a preset dynamic X-ray image training set; wherein the each dynamic X-ray image comprises a plurality of X-ray images corresponding to the same subject at different time points; calculating a plurality of image similarities corresponding to a plurality of X-ray image pairs between a plurality of preset labeled X-ray images in the each dynamic X-ray image and the plurality of preset labeled X-ray images; and if the plurality of image similarities are less than or equal to a preset image similarity, determining an X-ray image corresponding to the preset image similarity as a dynamic X-ray image to be labeled. The embodiment of the disclosure can realize the determination of the dynamic X-ray image to be labeled.

Description

Technical Field

[0001] This disclosure relates to the field of dynamic X-ray annotation technology, and in particular to a method, system and X-ray camera for determining and segmenting dynamic X-ray images to be annotated. Background Technology

[0002] Dynamic DR, as a breakthrough upgrade of digital X-ray imaging technology, perfectly integrates high-definition static spot imaging with real-time dynamic fluoroscopy, creating a modern imaging platform with "one machine and multiple functions", bringing a new paradigm of diagnosis and treatment to multiple departments such as radiology, emergency medicine, orthopedics, and gastroenterology.

[0003] Dynamic DR has multiple core advantages: real-time dynamic fluoroscopy can clearly observe the movement of organs; millisecond-level high-definition spot images can capture subtle lesions; large-format imaging reduces the trouble of stitching; and multi-mode imaging can meet complex diagnostic needs.

[0004] For example, X-rays are better suited for the initial assessment of degenerative osteoarthritis and traffic accident injury pain compared to computed tomography (CT), magnetic resonance imaging (MRI), and other imaging modalities. As the most widely used basic imaging method in orthopedics, X-ray imaging has become the preferred imaging device for initial knee examinations (such as detecting fractures, dislocations, and other abnormalities) and assessing clinical conditions such as osteoarthritis and bone destruction due to its widespread availability, low cost, and rapid convenience. However, X-rays, CT, and MRI are not conflicting but complementary for knee imaging. In particular, because X-rays are an overlapping imaging modality, there are blind spots in the visualization of intra-articular structures, and they remain insufficient in detecting subtle or hidden fractures. Therefore, minor fractures or bone injuries can be detected from three-dimensional (3D) knee CT images. From 3D knee CT images, bone injuries and tumors around the knee joint can be observed from multiple angles. However, 3D knee CT images show lower diagnostic sensitivity for changes in muscles and ligaments around the knee joint, especially when cartilage changes or bone hyperplasia have not yet occurred. Similar to CT, MR imaging is also multi-parameter, multi-planar, and multi-directional, with higher resolution for soft tissues than for bone. Clinical examination of cartilage, meniscus, or muscle and ligament injuries, synovitis, and joint effusion can be performed using 3D knee MR images. However, MRI is expensive, time-consuming, and involves complex equipment operation. Furthermore, information on knee joint movement function cannot currently be obtained from MRI. Compared to dynamic knee radiography, static knee X-ray images taken at a single moment lack information on knee joint movement, which is detrimental to assessing knee joint function. Dynamic knee radiography can capture the movement trajectory of the knee joint and holds promise for analyzing knee joint movement function. However, for dynamic knee radiography, the primary task of knee joint movement analysis is to accurately and automatically segment the patella, femur, tibia, and patellar tendon from multi-moment dynamic knee (knee / lower limb) X-ray images.

[0005] Then, regardless of whether it corresponds to the dynamic knee X-ray image training set or the dynamic chest X-ray image training set, the X-ray images at adjacent time points have a certain degree of similarity. First, X-ray images with similarity less than the preset threshold are not conducive to training the segmentation network, and cannot obtain a segmentation model with high generalization. Second, if the X-ray images to be labeled in the dynamic X-ray image training set are determined according to the preset time interval, there is a strong problem of subjectivity. Finally, if all images in the dynamic X-ray image training set are labeled, not only is there the technical problem that X-ray images with similarity less than the preset threshold are not conducive to training the segmentation network and cannot obtain a segmentation model with high generalization, but labeling all images in the dynamic X-ray image training set also places a heavy labeling task on physicians. Summary of the Invention

[0006] This disclosure proposes a method, system, and corresponding technical solution for determining and segmenting dynamic X-ray images to be annotated, as well as an X-ray camera.

[0007] According to one aspect of this disclosure, a method for determining dynamic X-ray images to be labeled is provided, comprising: extracting each dynamic X-ray image from a preset dynamic X-ray imaging image training set; wherein, each dynamic X-ray image includes: multiple X-ray images at different times corresponding to the same subject; calculating multiple image similarities between multiple preset labeled X-ray images in each dynamic X-ray image and multiple X-ray images corresponding to the multiple preset labeled X-ray images; if the multiple image similarities are less than or equal to a preset image similarity, then the X-ray image corresponding to the preset image similarity is determined as the dynamic X-ray image to be labeled.

[0008] Preferably, the initial preset-labeled X-ray image among the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to the initial moment in each dynamic X-ray image; and / or, the last preset-labeled X-ray image among the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to the last moment in each dynamic X-ray image.

[0009] Preferably, the method further includes: if the initial preset labeled X-ray image corresponding to the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to a non-initial time in each example of dynamic X-ray images, then the similarity between the initial preset labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the plurality of times before the initial preset labeled X-ray image is also calculated.

[0010] Preferably, the method further includes: if the last preset-labeled X-ray image corresponding to the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to a non-last moment in each example of dynamic X-ray images, then the similarity between the last preset-labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the moments after the last preset-labeled X-ray image is also calculated.

[0011] Preferably, the preset dynamic X-ray image training set is configured as a dynamic knee joint X-ray image training set or a dynamic chest X-ray image training set.

[0012] According to one aspect of this disclosure, a segmentation method is provided, comprising: determining dynamic X-ray images to be labeled in a preset dynamic X-ray imaging image training set using the dynamic X-ray image labeling method described above; labeling the dynamic X-ray images to be labeled with regions to be segmented to obtain corresponding mask label images of the regions to be segmented; and training a preset segmentation network based on the dynamic X-ray images to be labeled and their corresponding mask label images of the regions to be segmented to obtain a segmentation model.

[0013] Preferably, the step of training a preset segmentation network to obtain a segmentation model based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented includes: determining the areas of N masked segmentation regions corresponding to the mask label image of the region to be segmented in a preset medical image training set corresponding to the preset segmentation network; calculating N ratios of the area of ​​each of the N masked segmentation regions to the total area of ​​the masked segmentation regions corresponding to the N masked segmentation regions; determining the loss function weights of each masked segmentation region corresponding to the preset segmentation network based on N different products corresponding to any N-1 masked segmentation regions among the N masked segmentation regions; and training the preset segmentation network to obtain a segmentation model based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, using the multi-segmentation region weight loss function corresponding to the loss function weights of each masked segmentation region.

[0014] Preferably, before training the preset segmentation network based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, using the multi-segmentation region weight loss function corresponding to the loss function weight of each mask segmentation region, to obtain the segmentation model, the process includes: obtaining the basic loss function corresponding to each mask segmentation region; and summing the loss function weights of each mask segmentation region after multiplying them by their respective basic loss functions to construct the multi-segmentation region weight loss function.

[0015] Preferably, the base loss function corresponding to the weights of the loss function is configured as a cross-entropy loss function or a weighted cross-entropy loss function.

[0016] Preferably, the method further includes: summing the boundary loss functions corresponding to each masked segmentation region or summing the corresponding multi-segmentation region boundary loss function after multiplying the loss function weights of each masked segmentation region by their respective corresponding boundary loss functions; summing the DICE loss functions corresponding to each masked segmentation region or summing the corresponding multi-segmentation region DICE loss function after multiplying the loss function weights of each masked segmentation region by their respective corresponding DICE loss functions; constructing a multivariate loss function based on one or two of the segmentation region weight loss function, multi-segmentation region boundary loss function, or multi-segmentation region DICE loss function; and training the preset segmentation network using the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, and the multivariate loss function.

[0017] Preferably, the method further includes: if the number of iterations of the preset segmentation network is less than the preset maximum number of iterations, and the loss value between the dynamic X-ray image training set and its corresponding mask label image is less than the preset loss value, and the first average intersection-union ratio (IUU) between the preset medical image verification set and its corresponding mask label image at the first preset number of iterations is greater than the preset value of the first average IUU, then the second average IUU between the smallest area mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest area mask segmentation region in the preset medical image verification set is greater than the preset value of the second average IUU, then the preset segmentation network is controlled to stop training.

[0018] Preferably, the method further includes: if the number of iterations of the preset segmentation network reaches the preset maximum number of iterations, then the preset segmentation network is controlled to stop training.

[0019] Preferably, the method further includes: if the number of iterations of the preset segmentation network is less than the preset maximum number of iterations, and the loss value between the dynamic X-ray image training set and its corresponding mask label image is less than a preset loss value, and the first average intersection-union ratio (IUU) between the dynamic X-ray image verification set and its corresponding mask label image at the first preset number of iterations is greater than the preset value of the first average IUU, and the second average IUU between the smallest area mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest area mask segmentation region in the dynamic X-ray image verification set is greater than the preset value of the second average IUU, then a third average IUU between the dynamic X-ray image verification set and its corresponding mask label image is calculated within the second preset number of iterations; if the third average IUU is less than or equal to a preset fluctuation value, then the preset segmentation network is controlled to stop training.

[0020] According to one aspect of this disclosure, a dynamic X-ray image identification system is provided, comprising: an extraction unit for extracting each dynamic X-ray image from a preset dynamic X-ray imaging image training set; wherein each dynamic X-ray image includes multiple time-based X-ray images corresponding to the same subject; a calculation unit for calculating multiple image similarities between multiple preset labeled X-ray images in each dynamic X-ray image and multiple X-ray images corresponding to the multiple preset labeled X-ray images; and a determination unit for determining the X-ray image corresponding to the multiple image similarities less than or equal to the preset image similarity as dynamic X-ray images if the multiple image similarities are less than or equal to the preset image similarity. The method includes: an X-ray image to be annotated; or, an electronic device; the electronic device being configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic X-ray image annotation determination method; or, a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic X-ray image annotation determination method; or, a computer-readable storage medium storing computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described dynamic X-ray image annotation determination method; or, a computer program product, the computer program product being configured with a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the above-described dynamic X-ray image annotation determination method.

[0021] According to one aspect of this disclosure, a segmentation system is provided, comprising: a dynamic X-ray image to be labeled system as described above, used to determine dynamic X-ray images to be labeled in a preset dynamic X-ray imaging image training set; a labeling unit used to label the dynamic X-ray images to be labeled with regions to be segmented, obtaining corresponding mask label images of the regions to be segmented; a training unit used to train a preset segmentation network based on the dynamic X-ray images to be labeled and their corresponding mask label images of the regions to be segmented, to obtain a segmentation model; or, comprising: an electronic device; the electronic device is configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above segmentation method; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above segmentation method; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above segmentation method; or, comprising: a computer program product, the computer program product being configured with a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the above segmentation method.

[0022] According to one aspect of this disclosure, an X-ray camera is provided, comprising: a dynamic X-ray image identification system as described above; or, a dynamic X-ray image identification system as described above; or, comprising: an electronic device; the electronic device being configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke instructions stored in the memory to execute the dynamic X-ray image identification method and / or execute the segmentation method according to any one of claims 3 to 7; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke instructions stored in the memory to execute the dynamic X-ray image identification method and / or execute the segmentation method; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the dynamic X-ray image identification method and / or implement the segmentation method; or, comprising: a computer program product, the computer program product being configured with a computer program / instructions, which, when executed by a processor, implement the dynamic X-ray image identification method and / or implement the segmentation method.

[0023] This disclosure provides a method, system, and corresponding technical solution for determining and segmenting dynamic X-ray images to be labeled, in order to address the following issues: X-ray images with similarity less than a preset value are not conducive to training the segmentation network, resulting in a segmentation model with high generalization; if the X-ray images to be labeled in the dynamic X-ray image training set are determined according to a preset time interval, there is a problem of strong subjectivity; if all images in the dynamic X-ray image training set are labeled, not only are X-ray images with similarity less than a preset value not conducive to training the segmentation network, resulting in a segmentation model with high generalization, but labeling all images in the dynamic X-ray image training set also presents at least one technical problem in the heavy physician annotation task, thereby improving the generalization of the segmentation model.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0025] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0027] Figure 1A flowchart illustrating a method for determining a dynamic X-ray image to be annotated according to an embodiment of the present disclosure is shown. Detailed Implementation

[0028] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0029] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0030] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0031] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0032] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further.

[0033] In addition, this disclosure also provides a device or system for determining and segmenting dynamic X-ray images to be annotated, an electronic device, a computer-readable storage medium, and a program. All of the above can be used to implement any of the methods for determining and segmenting dynamic X-ray images to be annotated provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0034] Figure 1 A flowchart illustrating a method for determining a dynamic X-ray image to be annotated according to an embodiment of the present disclosure is shown. Figure 1As shown, the method for determining dynamic X-ray images to be labeled includes: Step S101: Extracting each dynamic X-ray image from a preset dynamic X-ray imaging image training set; wherein, each dynamic X-ray image includes: multiple X-ray images at different times corresponding to the same subject; Step S102: Calculating the image similarity between multiple preset labeled X-ray images in each dynamic X-ray image and the multiple X-ray images corresponding to the multiple preset labeled X-ray images; Step S103: If the image similarity is less than or equal to a preset image similarity, then the X-ray image corresponding to the image similarity less than or equal to the preset image similarity is determined as the dynamic X-ray image to be labeled. To address the challenges of X-ray images with similarity levels below a preset threshold hindering the training of segmentation networks and preventing the acquisition of highly generalized segmentation models, the following measures are proposed: Firstly, determining the X-ray images to be labeled in the dynamic X-ray image training set according to preset time intervals introduces a high degree of subjectivity. Secondly, labeling all images in the dynamic X-ray image training set presents not only the technical problem of X-ray images with similarity levels below a preset threshold hindering the training of segmentation networks and preventing the acquisition of highly generalized segmentation models, but also presents at least one of the technical challenges inherent in the heavy physician annotation task. These measures aim to improve the generalization of the segmentation model.

[0035] Step S101: Extract each dynamic X-ray image from the preset dynamic X-ray image training set; wherein, each dynamic X-ray image includes: multiple X-ray images at different times corresponding to the same subject.

[0036] In this embodiment of the disclosure, each dynamic X-ray image in the preset dynamic X-ray image or preset dynamic X-ray image training set or preset dynamic X-ray image verification set or preset dynamic X-ray image test set is configured as one of the following: multi-time X-ray image, multi-time CT image, multi-time PET image, multi-time MR image, or multi-time ET-CT image of the same subject; the multi-time X-ray image, multi-time CT image, multi-time PET image, multi-time MR image, or multi-time PET-CT image can be configured as multi-time X-ray image, multi-time CT image, multi-time PET image, multi-time MR image, or multi-time PET-CT image of any body part. For example, multi-time X-ray images corresponding to any one of the four limbs (brain / chest / limbs), neck / spine / breast / abdomen; multi-time CT images corresponding to any one of the four limbs (brain / chest / limbs), neck / spine / breast / abdomen; multi-time PET images corresponding to any one of the four limbs (brain / chest / limbs), neck / spine / breast / abdomen; multi-time MR images corresponding to any one of the four limbs (brain / chest / limbs), neck / spine / breast / abdomen; and multi-time PET-CT images corresponding to any one of the four limbs (brain / chest / limbs), neck / spine / breast / abdomen.

[0037] Step S102: Calculate the image similarity between the multiple preset labeled X-ray images in each example of dynamic X-ray imaging and the multiple X-ray images corresponding to the multiple preset labeled X-ray images.

[0038] In this embodiment of the disclosure, the image similarity can be configured as the mean square error corresponding to the average of the squared differences of the corresponding pixel values ​​of two images, extracting key points of the image and their descriptors, and calculating one or more of the similarity based on features, hash-based similarity, and statistical similarity (histogram similarity or structural similarity index) by matching descriptors.

[0039] In this embodiment of the disclosure, the first (first or first) preset labeled X-ray image among the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to the initial moment in each dynamic X-ray image; and / or, the last (last) preset labeled X-ray image among the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to the last moment in each dynamic X-ray image.

[0040] In this embodiment of the disclosure, it further includes: if the initial preset labeled X-ray image corresponding to the plurality of preset labeled X-ray images is configured as the X-ray image corresponding to a non-initial time in each example of dynamic X-ray images, then the similarity between the initial preset labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the plurality of times before the initial preset labeled X-ray image is also calculated.

[0041] In this embodiment of the disclosure, it further includes: if the last preset-labeled X-ray image corresponding to the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to a non-last moment in each example of dynamic X-ray images, then the similarity between the last preset-labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the times after the last preset-labeled X-ray image is also calculated.

[0042] In this embodiment of the disclosure, the preset dynamic X-ray image training set is configured as a dynamic knee X-ray image training set or a dynamic chest X-ray image training set. Specifically, each dynamic X-ray image in the dynamic knee X-ray image training set or the dynamic chest X-ray image training set is a multi-time knee X-ray image or chest X-ray image of the same subject.

[0043] Step S103: If the similarity of the plurality of images is less than or equal to a preset image similarity, then the X-ray imaging image corresponding to the similarity of the preset image is determined as the dynamic X-ray image to be labeled.

[0044] In the embodiments disclosed herein, those skilled in the art can configure the preset image similarity based on actual needs. For example, the preset image similarity can be configured to 80% or other values ​​less than 100%.

[0045] This disclosure also proposes a segmentation method, comprising: determining dynamic X-ray images to be labeled in a set of dynamic X-ray imaging training images using the dynamic X-ray image labeling method described above; labeling the dynamic X-ray images to be labeled with regions to be segmented to obtain corresponding mask label images of the regions to be segmented (mask label images); and training a set segmentation network based on the dynamic X-ray images to be labeled and their corresponding mask label images of the regions to be segmented to obtain a segmentation model.

[0046] In this embodiment of the disclosure, the step of training a segmentation network based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented to obtain a segmentation model includes: determining the areas of N mask segmentation regions corresponding to the mask label image of the region to be segmented in a set of medical images corresponding to the segmentation network; wherein, N is greater than 1 or greater than or equal to 2, and N is a positive integer; calculating N ratios of the area of ​​each of the N mask segmentation regions to the total area of ​​the mask segmentation regions corresponding to the N mask segmentation regions; determining the loss function weights of each mask segmentation region corresponding to the segmentation network based on the N different products corresponding to any N-1 mask segmentation regions among the N mask segmentation regions; and training the segmentation network based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented (the label image of the region to be segmented) using the multi-segmentation region weight loss function corresponding to the loss function weights of each mask segmentation region to obtain a segmentation model.

[0047] In this embodiment of the disclosure, the areas of N masked segmentation regions corresponding to the masked label images in a set of medical images corresponding to a set segmentation network are determined. The set segmentation network is configured as one or more of FCN, UNet, UPerNet, SegFormer, PSPNet, and DeepLabV3, or one or more combinations thereof.

[0048] In this embodiment of the disclosure, the X-ray image training set, CT image training set, PET image training set, MR image training set, and PET-CT image training set corresponding to the medical image training set are configured as one of the following: X-ray image training set for brain / chest / limbs / neck / spine / breast / abdomen; CT image training set for brain / chest / limbs / neck / spine / breast / abdomen; PET image training set for brain / chest / limbs / neck / spine / breast / abdomen; MR image training set for brain / chest / limbs / neck / spine / breast / abdomen; and PET-CT image training set for brain / chest / limbs / neck / spine / breast.

[0049] In this embodiment of the disclosure, the mask label images in the medical image training set are configured as one or more mask label images of brain tissue / lung / breast / liver and their corresponding lesions, or any more mask label images of the patella, femur, tibia and patellar tendon corresponding to the knee, or any more mask label images of the ribs, clavicle, cervical vertebrae and spine corresponding to the thoracic bones.

[0050] For example, in multi-time dynamic knee (knee / lower limb) X-ray images corresponding to a set of medical images for training, there is a technical problem that the area ratios of the patellar, femoral, tibial, and patellar tendon mask segments corresponding to the N mask segmentation areas of the mask label images are unbalanced, resulting in the smaller patellar tendon mask segmentation area not being adequately trained.

[0051] In this embodiment of the disclosure, N ratios are calculated between the area of ​​each of the N masked segmentation regions and the total area of ​​the N masked segmentation regions. Specifically, calculating these ratios involves: calculating the area of ​​each of the N masked segmentation regions based on different mask values ​​corresponding to the N masked segmentation regions; calculating the total area of ​​the N masked segmentation regions; and calculating the N ratios between each masked segmentation region and the total area of ​​the N masked segmentation regions.

[0052] In this embodiment of the disclosure, the loss function weights for each masked segmentation region corresponding to the defined segmentation network are determined based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas. Specifically, determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas includes: calculating N products corresponding to any N-1 masked segmentation region areas among the N masked segmentation region areas; and determining the loss function weights for each masked segmentation region corresponding to the defined segmentation network based on the N products and the sum of the products corresponding to the N products.

[0053] In this embodiment of the disclosure, determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N products and the sum of the products corresponding to the N products includes: extracting the product of the areas of the mask segmentation regions whose weights are to be determined but do not contain the weights to be determined from the areas of the N different mask segmentation regions; calculating the ratio of the product of the areas of the mask segmentation regions whose weights are to be determined to the sum of the products corresponding to the N different products, and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network.

[0054] For example, the areas of the N masked segmentation regions are configured as A, B, C, and D; calculate N ratios a (a=A / (A+B+C+D)), b (b=B / (A+B+C+D)), c (c=C / (A+B+C+D)), and d (d=D / (A+B+C+D)) for each of the N masked segmentation regions A, B, C, and D, respectively, to the total area of ​​the N masked segmentation regions (A+B+C+D); calculate N different products a*b*c, a*c*d, a*b*d, and b*c*d corresponding to any N-1 masked segmentation region areas among the N masked segmentation regions. Extract the products b*c*d, a*c*d, a*b*d, and a*b*c of the areas of the masked regions A, B, C, and D that do not contain the weight to be determined from the N different products a*b*c, a*c*d, a*b*d, and b*c*d, respectively. Then, calculate the sum of the products b*c*d, a*c*d, a*b*d, and a*b*c of the areas of the masked regions whose weights are to be determined, and the sum of the products of the N different products (a) and b*c*d. The ratio of *b*c+a*c*d+a*b*d+b*c*d is used to determine the loss function weights b*c*d / (a*b*c+a*c*d+a*b*d+b*c*d), a*c*d / (a*b*c+a*c*d+a*b*d+b*c*d), a*b*d / (a*b*c+a*c*d+a*b*d+b*c*d), and a*b*c / (a*b*c+a*c*d+a*b*d+b*c*d) for each mask segmentation region corresponding to the set segmentation network.

[0055] In this embodiment of the disclosure, determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 mask segmentation region areas among the N mask segmentation region areas includes: multiplying the N mask segmentation region areas by a plurality of corresponding weight variables to be optimized to obtain N mask segmentation region weight areas; setting the N mask segmentation region weight areas equal to a set value to obtain the N weight variables to be optimized represented by the set value and the N mask segmentation region areas; and determining the loss function weight of each mask segmentation region corresponding to the set segmentation network based on the N weight variables to be optimized represented by the set value and the N mask segmentation region areas.

[0056] In this embodiment of the disclosure, determining the loss function weight for each mask segmentation region corresponding to the specified segmentation network based on the N weight variables to be optimized, represented by the set value and the area of ​​the N mask segmentation regions, includes: calculating the sum of the weight variables to be optimized corresponding to the N weight variables to be optimized, represented by the set value and the area of ​​the N mask segmentation regions, to obtain the weight variables to be optimized and the expression; and calculating the ratio between each of the N weight variables to be optimized and the weight variables to be optimized and the expression, to determine the loss function weight for each mask segmentation region corresponding to the specified segmentation network.

[0057] For example, the areas of the N mask segmentation regions are configured as A, B, C, and D; the areas of the N mask segmentation regions are multiplied by the corresponding multiple weight variables to be optimized ε1, ε2, ε3, and ε4 respectively to obtain the weight areas of the N mask segmentation regions ε1*A, ε2*B, ε3*C, and ε4*D; the weight areas of the N mask segmentation regions ε1*A, ε2*B, ε3*C, and ε4*D are set to equal a predetermined value ε, resulting in the N weight variables to be optimized ε1=ε / A, ε2=ε / B, ε3=ε / C, and ε4=ε / D, represented by the predetermined value and the areas of the N mask segmentation regions; Calculate the sum of the N weight variables to be optimized, represented by the set value and the areas of the N masked segmentation regions, to obtain the weight variables to be optimized and the expression (ε1+ε2+ε3+ε4=ε / A+ε / B+ε / C+ε / D); calculate the ratio ε1 / (ε1+ε2+ε3+ε4) between each of the N weight variables to be optimized represented by the set value and the areas of the N masked segmentation regions and the weight variables to be optimized and the expression. )=(ε / A) / (ε / A+ε / B+ε / C+ε / D), ε2 / (ε1+ε2+ε3+ε4)=(ε / B) / (ε / A+ε / B+ε / C+ε / D), ε3 / (ε1+ε2+ε3+ε4)=(ε / C) / (ε / A+ε / B+ε / C+ε / D), ε4 / (ε1+ε2+ε3+ε4)=(ε / D) / (ε / A+ε / B+ε / C+ε / D), determine the loss function weight of each mask segmentation area corresponding to the set segmentation network.

[0058] In this embodiment of the disclosure, before training a segmentation network based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, using a multi-segmentation region weight loss function corresponding to the loss function weight of each mask segmentation region, to obtain a segmentation model, the process includes: obtaining the basic loss function corresponding to each mask segmentation region; and summing the loss function weights of each mask segmentation region after multiplying them by their respective basic loss functions to construct a multi-segmentation region weight loss function. The basic loss function corresponding to the loss function weights is configured as a cross-entropy loss function or a weighted cross-entropy loss function.

[0059] In this embodiment of the disclosure, the method further includes: summing the boundary loss functions corresponding to each masked segmentation region or summing the corresponding multi-segmentation region boundary loss function after multiplying the loss function weights of each masked segmentation region by their respective corresponding boundary loss functions; summing the DICE loss functions corresponding to each masked segmentation region or summing the corresponding multi-segmentation region DICE loss function after multiplying the loss function weights of each masked segmentation region by their respective corresponding DICE loss functions; constructing a multivariate loss function based on one or two of the segmentation region weight loss function, multi-segmentation region boundary loss function, or multi-segmentation region DICE loss function; and training the specified segmentation network using the dynamic X-ray image to be labeled (the dynamic X-ray image to be labeled corresponding to the set of medical image training sets) and its corresponding mask label image of the region to be segmented (mask label image), and the multivariate loss function.

[0060] In this embodiment of the disclosure, the method further includes: if the number of iterations of the set segmentation network is less than the set maximum number of iterations, and the loss value between the dynamic X-ray image training set (set medical image training set) and its corresponding mask label image is less than the set loss value, and the first average intersection-union ratio (IUU) between the set medical image validation set and its corresponding mask label image at the first set number of iterations is greater than the set first average IUU value, then the second average IUU ratio of the smallest mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest mask segmentation region in the set medical image validation set is greater than the set second average IUU value, then the set segmentation network is controlled to stop training.

[0061] In this embodiment of the disclosure, the method further includes: if the number of iterations of the set segmentation network reaches the set maximum number of iterations, then the set segmentation network is controlled to stop training.

[0062] In this embodiment of the disclosure, the method further includes: if the number of iterations of the defined segmentation network is less than the set maximum number of iterations, and the loss value between the dynamic X-ray image training set and its corresponding mask label image is less than the set loss value, and the first average intersection-union ratio (IUU) between the dynamic X-ray image validation set (defined medical image validation set) and its corresponding mask label image at the first set number of iterations is greater than the set value of the first average IUU, and the second average IUU between the smallest mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest mask segmentation region in the dynamic X-ray image validation set (defined medical image validation set) is greater than the set value of the second average IUU, then a third average IUU between the dynamic X-ray image validation set (defined medical image validation set) and its corresponding mask label image is calculated within the second set number of iterations; if the third average IUU is less than or equal to the set fluctuation value, then the defined segmentation network is controlled to stop training.

[0063] In this embodiment of the disclosure, the loss function weights for each masked segmentation region corresponding to the specified segmentation network are determined using the above-described loss function weight determination method; a multi-segmentation region weight loss function is constructed based on the loss function weights for each masked segmentation region; and the specified segmentation network is trained using the specified medical image training set and its corresponding masked label images, as well as the multi-segmentation region weight loss function.

[0064] In this embodiment of the disclosure, the step of constructing a multi-segmentation region weighted loss function based on the loss function weights of each masked segmentation region includes: obtaining the basic loss function corresponding to each masked segmentation region; and summing the loss function weights of each masked segmentation region after multiplying them by their respective basic loss functions to construct the multi-segmentation region weighted loss function.

[0065] In this embodiment of the disclosure, the basic loss function corresponding to the weight of the loss function is configured as a cross-entropy loss function or a weighted cross-entropy loss function.

[0066] In this embodiment, the method further includes: summing the boundary loss functions corresponding to each masked segmentation region, or summing the corresponding multi-segmentation region boundary loss function after multiplying the loss function weights of each masked segmentation region by their respective boundary loss functions; summing the corresponding DICE loss functions corresponding to each masked segmentation region, or summing the corresponding multi-segmentation region DICE loss function after multiplying the loss function weights of each masked segmentation region by their respective DICE loss functions; constructing a multivariate loss function based on one or two of the segmentation region weight loss function, multi-segmentation region boundary loss function, or multi-segmentation region DICE loss function; and training the specified segmentation network using the specified medical image training set and its corresponding masked label images, and the multivariate loss function. By using the multivariate loss function corresponding to the multivariate loss function and one or two of the segmentation region weight loss function or multi-segmentation region DICE loss function, the parameter training direction of the specified segmentation network is guided, thereby solving the technical problem that a univariate loss function cannot comprehensively train the specified segmentation network.

[0067] In this embodiment of the disclosure, the method further includes: obtaining a set maximum number of iterations; if the number of iterations of the set segmentation network reaches the set maximum number of iterations, then controlling the set segmentation network to stop training.

[0068] In this embodiment, the method further includes: if the number of iterations of the defined segmentation network is less than a defined maximum number of iterations, calculating the loss value between the defined medical image training set and its corresponding mask label image; if the loss value is less than a set loss value, calculating the first average intersection-union ratio (IUU) between the defined medical image validation set and its corresponding mask label image at a first set number of iterations; if the first overall average IUU is greater than the first average IUU set value, controlling the defined segmentation network to stop training. This addresses at least one technical problem in multi-target region segmentation models, such as overfitting, wasted computational resources and time, difficulty in capturing the optimal state of the multi-target region segmentation model, and poor robustness of the multi-target region segmentation model.

[0069] In this embodiment of the disclosure, the method further includes: if the number of iterations of the set segmentation network is less than the set maximum number of iterations, the loss value between the set medical image training set and its corresponding mask label image is less than the loss setting value, and the first average intersection-union ratio (IUU) between the set medical image validation set and its corresponding mask label image at the first set number of iterations is greater than the first average IUU setting value, then the second average IUU ratio corresponding to the smallest mask segmentation region in the N mask segmentation regions and the smallest mask segmentation region in the set medical image validation set is calculated to be greater than the second average IUU setting value, and then the set segmentation network is controlled to stop training.

[0070] In this embodiment of the disclosure, the method further includes: if the number of iterations of the set segmentation network is less than the set maximum number of iterations, the loss value between the set medical image training set and its corresponding mask label image is less than the set loss value, the first average intersection-union ratio (IUU) between the set medical image validation set and its corresponding mask label image at the first set number of iterations is greater than the set first average IUU value, and the second average IUU between the smallest mask segmentation region in the N mask segmentation regions and the smallest mask segmentation region in the set medical image validation set is greater than the set second average IUU value, then a third average IUUU between the set medical image validation set and its corresponding mask label image is calculated within the second set number of iterations; if the third average IUUU is less than or equal to the set fluctuation value, then the set segmentation network is controlled to stop training.

[0071] For example, firstly, one of the basic criteria for stopping network training is that the loss value between the knee joint multi-object segmentation image (set as the medical image training set) and its ground truth (GT) in the training set is less than 0.2 (loss setting). Secondly, the overall segmentation performance of the model in four object regions (segmentation regions corresponding to the patella, femur, tibia, and patellar tendon) is evaluated every 50 iterations (first set iteration number) using the validation set and the knee joint multi-object segmentation model. Based on the loss value between the knee joint multi-object segmentation image and its GT in the training set being less than 0.2, a first average intersection-over-union (IoU) ratio greater than 0.8 (first average IoU setting) between the knee joint multi-object segmentation image and its GTs in the validation set is set as the second basic criterion for stopping network training. Thirdly, due to the small area of ​​the patellar tendon (the smallest masked segmentation region), it is difficult to achieve good performance. To provide more comprehensive training for the patellar tendon, a third criterion for stopping training is set: an average IoU greater than 0.85 (the second average intersection-union ratio setting) based on the patellar tendon object segmentation images and their ground truth (GT). Finally, after meeting these three criteria, network training can be stopped if the total average IoU between the four object segmentation regions and their GTs in the validation set remains unchanged for 10 consecutive iterations (the second set number of iterations). This means the relative fluctuation of the total average IoU between the four object segmentation regions and their GTs in the ten validation sets is less than or equal to the set fluctuation value of 0.001. If these three criteria are not met, network training stops when the maximum number of iterations is reached. After network training is complete, the segmentation model that performs best on the validation set is selected for testing on the test set.

[0072] In this embodiment of the disclosure, the method further includes: dividing the learning rate decay of the optimizer corresponding to the specified segmentation network into an initial learning rate decay phase, a stable learning rate decay period, and a fine-tuning learning rate decay phase; wherein, the initial learning rate decay phase uses warm-up steps to ensure rapid gradient descent in the initial stage of training the specified segmentation network while maintaining the stability of the specified segmentation network; the stable learning rate decay period uses polynomial decay to maintain a relatively stable learning rate; and the fine-tuning learning rate decay phase uses cosine learning rate decay.

[0073] For example, in network training, the learning rate is a key hyperparameter determining model convergence. Dynamic learning rates, by adaptively adjusting the learning rate during training, can achieve rapid convergence early on and fine-tune later, resulting in better model performance. In this study, the AdamW optimizer or a modified AdamW optimizer was used with an initial learning rate of 1e-4. The AdamW optimizer's learning rate decay includes an initial learning rate decay phase, a stable learning rate decay phase, and a fine-tuning learning rate decay phase. The initial learning rate decay phase, based on the initial learning rate, uses warmup steps, which provides rapid gradient descent in the initial stages of network training while maintaining stability. The stable learning rate decay phase uses polynomial decay, maintaining a relatively stable learning rate during long-term network training. The fine-tuning learning rate decay phase uses cosine learning rate decay, applied at the end of training.

[0074] In this embodiment of the disclosure, the multiple medical image segmentation models corresponding to the multiple defined segmentation networks are configured as multi-target segmentation regions that are consistent with the N mask segmentation regions corresponding to the mask label image.

[0075] In this embodiment of the disclosure, the multi-target segmentation region corresponding to the N mask segmentation regions of the mask label image is configured as two or more of the patella, femur, tibia and patellar tendon; or, the multi-target segmentation region is configured as one or more target segmentation regions of the brain tissue / lung / breast / liver mask label image and its corresponding lesion, or any of the target segmentation regions of the ribs, clavicle, cervical vertebrae and spine corresponding to the thoracic bones.

[0076] This disclosure also proposes a method for evaluating medical image segmentation models. Furthermore, it includes: determining multiple first comprehensive evaluation scales corresponding to multiple medical image segmentation models on a set of medical images training sets and loss functions, and multiple second comprehensive evaluation scales corresponding to multiple evaluation scales on a set of medical images, where the values ​​are positively proportional or positively correlated with the performance of the segmentation models (the larger the value, the better the performance); determining first score vectors corresponding to the multiple medical image segmentation models under the multiple first comprehensive evaluation scales and second score vectors corresponding to the multiple medical image segmentation models under the multiple second comprehensive evaluation scales; and evaluating the multiple medical image segmentation models based on the first score vectors and the second score vectors. This addresses the technical problem that improvements to medical image segmentation models often cannot be achieved across all evaluation scales with numerous different dimensions, thus hindering the selection of appropriate medical image segmentation models, thereby improving the performance of medical image segmentation models.

[0077] In this embodiment of the disclosure, the set of medical images for training and testing is configured as one of X-ray images, CT images, PET images, MR images, and PET-CT images; the X-ray images, CT images, PET images, MR images, and PET-CT images can be configured as X-ray images, CT images, PET images, MR images, and PET-CT images of any body part. For example, X-ray images of any limb (brain / chest / extremities), neck / spine / breast / abdomen; CT images of any limb (brain / chest / extremities), neck / spine / breast / abdomen; PET images of any limb (brain / chest / extremities), neck / spine / breast / abdomen; MR images of any limb (brain / chest / extremities), neck / spine / breast / abdomen; and PET-CT images of any limb (brain / chest / extremities), neck / spine / breast / abdomen.

[0078] In this embodiment of the disclosure, the plurality of medical image segmentation models corresponding to the plurality of defined segmentation networks are configured as multi-target segmentation regions; and / or, the plurality of defined segmentation networks are configured as one or more of FCN, UNet, UPerNet, SegFormer, PSPNet, DeepLabV3, or a combination thereof; and / or, the defined medical image training set or defined medical image test set corresponding to the plurality of medical image segmentation models is configured as X-ray images; and / or, the X-ray images corresponding to the plurality of medical image segmentation models or the X-ray image test set corresponding to the defined medical image test set are configured as knee X-ray images; and / or, the multi-target segmentation regions corresponding to the plurality of medical image segmentation models are configured as two or more of patella, femur, tibia, and patellar tendon; or, the multi-target segmentation regions are configured as one or any of the target segmentation regions of brain tissue / lung / breast / liver masked label images and their corresponding lesions, or any of the target segmentation regions of ribs, clavicle, cervical vertebrae, and spine corresponding to thoracic bones.

[0079] In this embodiment of the disclosure, the plurality of evaluation metrics are configured as any of the following: IoU (Intersection over Union), Dice, precision, recall, Hausdorff distance (HD) or median 95th Hausdorff distance (HD95), and average symmetric surface distance (ASSD); and / or, the plurality of first comprehensive evaluation metrics are configured as any one or more of IoU, Dice, precision, and recall; and / or, the plurality of second comprehensive evaluation metrics are any one or more of Hausdorff distance or median 95th Hausdorff distance and average symmetric surface distance.

[0080] In this embodiment of the disclosure, determining multiple evaluation scales corresponding to multiple medical image segmentation models of multiple defined segmentation networks on a defined medical image training set and a defined loss function, and determining multiple first comprehensive evaluation scales whose values ​​are positively proportional or positively correlated with the performance of the segmentation models on a defined medical image test set, and multiple second comprehensive evaluation scales whose values ​​are inversely proportional or negatively correlated with the performance of the segmentation models, includes: calculating multiple evaluation scales corresponding to multiple medical image segmentation models of multiple defined segmentation networks on a defined medical image test set; determining the first set of evaluation scales whose values ​​are positively proportional or positively correlated with the performance of the segmentation models and the second set of evaluation scales whose values ​​are inversely proportional or negatively correlated with the performance of the segmentation models on the defined medical image test set; and determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales, respectively.

[0081] In this embodiment of the disclosure, determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively includes: summing the multiple medical image segmentation models on the first set of evaluation scales corresponding to a set of medical images to determine multiple first comprehensive evaluation scales corresponding to the multiple medical image segmentation models; and summing the multiple medical image segmentation models on the second set of evaluation scales corresponding to the set of medical images to determine multiple second comprehensive evaluation scales corresponding to the multiple medical image segmentation models.

[0082] In this embodiment of the disclosure, the step of determining multiple first comprehensive evaluation scales and multiple second comprehensive evaluation scales based on the first set of evaluation scales and the second set of evaluation scales respectively further includes: determining a first number corresponding to the evaluation scales in the first set of evaluation scales and a second number corresponding to the evaluation scales in the second set of evaluation scales respectively; averaging the multiple first comprehensive evaluation scales corresponding to the multiple medical image segmentation models using the first number respectively to obtain the final multiple first comprehensive evaluation scales; and averaging the multiple second comprehensive evaluation scales corresponding to the multiple medical image segmentation models using the second number respectively to obtain the final multiple second comprehensive evaluation scales.

[0083] In this embodiment of the disclosure, determining the first score vector corresponding to the plurality of medical image segmentation models under a plurality of first comprehensive evaluation scales and the second score vector corresponding to the plurality of medical image segmentation models under a plurality of second comprehensive evaluation scales includes: sorting the plurality of medical image segmentation models according to a set order of the plurality of first comprehensive evaluation scales (from largest to smallest or from smallest to largest) to determine the first score vector corresponding to the plurality of medical image segmentation models under the plurality of first comprehensive evaluation scales; and sorting the plurality of medical image segmentation models according to a set order of the plurality of second comprehensive evaluation scales (from largest to smallest or from smallest to largest) to determine the second score vector corresponding to the plurality of medical image segmentation models under the plurality of second comprehensive evaluation scales.

[0084] In this embodiment of the disclosure, the step of determining the first score vector corresponding to the plurality of medical image segmentation models under a plurality of first comprehensive evaluation scales and the second score vector corresponding to the plurality of medical image segmentation models under a plurality of second comprehensive evaluation scales further includes: determining the maximum score value among the first score vector and the second score vector; determining the score corresponding to the medical image segmentation model with the largest first comprehensive evaluation scale among the plurality of first comprehensive evaluation scales as the maximum score value; and arranging the plurality of medical image segmentation models according to a predetermined order (from largest to smallest or from smallest to largest) corresponding to the plurality of first comprehensive evaluation scales based on the maximum score value and a predetermined score difference between any two adjacent scores in the first score vector. The multiple medical image segmentation models are scored and assigned values ​​to determine the first score vector corresponding to the multiple medical image segmentation models under multiple first comprehensive evaluation scales; the score corresponding to the medical image segmentation model with the smallest second comprehensive evaluation scale among the multiple second comprehensive evaluation scales is determined as the maximum score value; based on the maximum score value and the set score difference between any two adjacent scores in the second score vector, the multiple medical image segmentation models sorted according to the set order (from largest to smallest or from smallest to largest) corresponding to the multiple second comprehensive evaluation scales are scored and assigned values ​​to determine the second score vector corresponding to the multiple medical image segmentation models under the multiple second comprehensive evaluation scales.

[0085] In this embodiment of the disclosure, determining the maximum score value among the first score vector and the second score vector includes: determining the maximum score value among the first score vector and the second score vector based on the number corresponding to the plurality of medical image segmentation models.

[0086] For example, the maximum score value in the first and second score vectors is configured to be 50; or, the number of the plurality of medical image segmentation models is configured to be 50, then the maximum score value in the first and second score vectors is configured to be 50. In this case, the set score difference between any two adjacent scores in the first and second score vectors is configured to be 1, and the determined first and second score vectors are configured as [50, 49, 48, …, 3, 2, 1] or [1, 2, 3, …, 48, 49, 50].

[0087] In this embodiment of the disclosure, the evaluation of the plurality of medical image segmentation models based on the first scoring vector and the second scoring vector includes: constructing a correlation relationship between the first scoring vector and the second scoring vector according to the plurality of medical image segmentation models; calculating the sum of the first correlation score and the second correlation score of the same medical image segmentation model under the correlation relationship, and the corresponding multiple joint scores; and evaluating the plurality of medical image segmentation models based on the multiple joint scores.

[0088] In this embodiment of the disclosure, the step of constructing the association between the first scoring vector and the second scoring vector based on the plurality of medical image segmentation models includes: extracting the first score and the second score corresponding to the same medical image segmentation model from the first scoring vector and the second scoring vector respectively, so as to construct the association between the first scoring vector and the second scoring vector; or, determining the association labels of the first score in the first scoring vector and the second score in the second scoring vector corresponding to the same medical image segmentation model in the plurality of medical image segmentation models respectively; and constructing the association between the first scoring vector and the second scoring vector based on the association labels.

[0089] In this embodiment of the disclosure, the step of evaluating the multiple medical image segmentation models based on the multiple joint scores includes: sorting the multiple joint scores to obtain a sorted joint score vector; and evaluating the multiple medical image segmentation models based on the sorted joint score vector; wherein the joint score in the sorted joint score vector is proportional to the performance of the corresponding medical image segmentation model.

[0090] In this embodiment of the disclosure, determining multiple medical image segmentation models corresponding to multiple defined segmentation networks includes: determining target segmentation regions corresponding to the multiple medical image segmentation models; if the number of target segmentation regions is greater than 1, determining the areas of multiple masked segmentation regions corresponding to masked label images in the medical image training set used to train the multiple defined segmentation networks; if the difference or ratio between the largest and smallest masked segmentation region areas among the multiple masked segmentation region areas is greater than a set difference or a set ratio, optimizing the defined loss function corresponding to the multiple defined segmentation networks to obtain an optimized loss function; and training the multiple defined segmentation networks based on the medical image training set and the optimized loss function to determine the multiple medical image segmentation models corresponding to the multiple defined segmentation networks.

[0091] In this embodiment, optimizing the set loss function corresponding to the plurality of set segmentation networks to obtain an optimized loss function includes: determining the areas of N masked segmentation regions corresponding to the masked label images in the set medical image training set corresponding to the set segmentation networks; wherein N is greater than 1 or greater than or equal to 2, and N is a positive integer; calculating N ratios of each masked segmentation region area to the total area of ​​the masked segmentation regions corresponding to the N masked segmentation regions; and determining the loss function weights of each masked segmentation region corresponding to the set segmentation network based on N different products corresponding to any N-1 masked segmentation region areas among the N masked segmentation regions, thereby obtaining the optimized loss function. This addresses the technical problem that the unbalanced proportion of the N masked segmentation region areas corresponding to the masked label images causes smaller masked segmentation regions to not be sufficiently trained, thereby improving the segmentation performance of smaller masked segmentation regions.

[0092] In this embodiment of the disclosure, the set of medical image training, validation, or test sets corresponding to the plurality of medical image segmentation models are configured as multi-time dynamic knee X-ray images. In this case, the plurality of medical image segmentation models are configured as multiple multi-time dynamic knee X-ray image segmentation models. Using the optimal multi-time dynamic knee X-ray image segmentation model among the multiple multi-time dynamic knee X-ray image segmentation models, the dynamic knee X-ray image to be segmented is segmented to obtain segmentation mask images corresponding to one or more of the patella, femur, tibia, and patellar tendon.

[0093] In this embodiment of the disclosure, the dynamic medical image may be configured as a dynamic ultrasound image, a dynamic X-ray image, or other dynamic medical images. For example, a dynamic knee X-ray image or a dynamic chest X-ray image.

[0094] This disclosure proposes a method for determining the tibial boundary line, comprising: extracting a tibial mask boundary image corresponding to the tibial mask from a tibial region mask image; and determining one or two boundary lines, namely a third boundary line and a fourth boundary line, among the boundary lines on both sides of the tibia or tibial cortex, based on the tibial mask boundary image and the non-zero coordinates of the tibial mask boundary image. This addresses at least one of the technical problems of currently using manual annotation to determine the tibial boundary line and tibial midline, which are highly subjective, and the need for further improvement in the accuracy of automatic detection of the tibial boundary line and tibial midline.

[0095] In this embodiment of the disclosure, determining one or two boundary lines of the third and fourth boundary lines among the boundary lines on both sides of the tibia or tibial cortex based on the tibial mask boundary image and the non-zero coordinates of the tibial mask boundary image includes: performing a polar coordinate Hough transform on the tibial mask boundary image to determine the most likely third and fourth corrected boundary lines on the outermost side of the tibia or tibial cortex; determining whether the third and fourth corrected boundary lines are parallel; if they are not parallel and the intersection of the third and fourth corrected boundary lines is within the tibial mask image, then correcting the third and fourth corrected boundary lines based on the tibial mask boundary image to obtain one or two boundary lines of the third and fourth boundary lines.

[0096] In this embodiment of the disclosure, the step of correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary image to obtain one or both of the third and fourth boundary lines includes: determining a mirror line based on the X-ray image of the knee joint to be processed corresponding to the tibial mask boundary image; performing mirror processing on the tibial mask boundary in the tibial mask boundary image with the mirror line as the axis of symmetry to obtain a mirror image of the tibial mask boundary; and correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary and the mirror image of the tibial mask boundary to obtain one or both of the third and fourth boundary lines.

[0097] In this embodiment of the disclosure, the step of correcting the third boundary line to be corrected and the fourth boundary line based on the tibial mask boundary and the mirror image of the tibial mask boundary to obtain one or two boundary lines of the third boundary line and the fourth boundary line includes: performing a polar coordinate Hough transform on the tibial mask boundary and the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are parallel, then any one or two of the parallel straight lines are configured as one or two boundary lines of the third boundary line and the fourth boundary line.

[0098] In this embodiment of the disclosure, the step of correcting the third boundary line to be corrected and the fourth boundary line based on the tibial mask boundary and the mirror image of the tibial mask boundary to obtain one or two boundary lines of the third boundary line and the fourth boundary line further includes: if any two lines among the plurality of most probable lines are not parallel and the intersection point of any two non-parallel lines is outside the tibial mask image, then any one or two of the two lines corresponding to the intersection point of any two non-parallel lines outside the tibial mask image are configured as one or two boundary lines of the third boundary line and the fourth boundary line.

[0099] In this embodiment of the disclosure, the step of determining one or two boundary lines of the third and fourth boundary lines among the boundary lines on both sides of the tibia or tibial cortex based on the tibial mask boundary image and the non-zero coordinates of the tibial mask boundary of the tibial mask boundary image further includes: if they are parallel, then the third boundary line to be corrected and the fourth boundary line to be corrected are not corrected, and the third boundary line to be corrected and the fourth boundary line to be corrected are respectively configured as one or two boundary lines of the third and fourth boundary lines; if they are not parallel and the intersection of the third boundary line to be corrected and the fourth boundary line to be corrected is outside the tibial mask image, then the third boundary line to be corrected and the fourth boundary line to be corrected are not corrected, and the third boundary line to be corrected and the fourth boundary line to be corrected are respectively configured as one or two boundary lines of the third and fourth boundary lines.

[0100] In this embodiment of the disclosure, before correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary image, the method includes: determining the tibial length based on the tibial mask image or the tibial mask boundary image corresponding to the tibial mask image; if the tibial length is less than a preset tibial length, then correcting the third boundary line to be corrected and the fourth boundary line to be corrected based on the tibial mask boundary image; otherwise, not correcting the third boundary line to be corrected and the fourth boundary line to be corrected, and configuring the third boundary line to be corrected and the fourth boundary line to be corrected as one or both of the third and fourth boundary lines.

[0101] In this embodiment of the disclosure, determining the mirror line based on the X-ray image of the knee joint to be processed corresponding to the tibial mask boundary image includes: determining the anterior vertex and posterior vertex of the tibial plateau corresponding to the X-ray image of the knee to be processed; and determining the mirror line based on the anterior vertex and posterior vertex of the tibial plateau.

[0102] In this embodiment of the disclosure, determining the anterior and posterior tibial plateau vertices corresponding to the knee X-ray image to be processed (the knee joint X-ray image to be processed) includes: using a keypoint detection model corresponding to a keypoint detection network to detect the anterior and posterior tibial plateau vertices of the knee X-ray image to be processed, and determining the anterior and posterior tibial plateau vertices corresponding to the knee X-ray image to be processed.

[0103] In this embodiment of the disclosure, constructing a keypoint detection model corresponding to the keypoint detection network includes: acquiring anterior and posterior tibial plateau labels corresponding to multiple knee X-ray images; and training a preset keypoint detection network using the anterior and posterior tibial plateau labels to obtain a keypoint detection model.

[0104] In this embodiment of the disclosure, before training the preset keypoint detection network using the anterior and posterior tibial plateau vertices labels, the process includes: pre-training the preset keypoint detection network using multiple preset pose points corresponding to human pose images and / or multiple preset facial feature points corresponding to facial images to obtain a pre-trained keypoint detection model; then, re-training the pre-trained keypoint detection model using the anterior and posterior tibial plateau vertices labels to obtain a keypoint detection model.

[0105] In this embodiment of the disclosure, the keypoint detection network can be configured as one or more of the following deep learning networks: OpenPose, HRNet (High-Resolution Network), Hourglass Network, DEKR (Distributed Keypoint Regression), YOLO-KP, CenterNet, Cornernet, TokenPose, PoseTransformer, Mask R-CNN, etc.

[0106] This disclosure also proposes a tibial mask image processing method, including: determining a mirror line based on the anterior vertex and posterior vertex of the tibial plateau corresponding to the X-ray image of the knee to be processed (X-ray image of the knee joint to be processed); Using the mirrored line as the axis of symmetry, the tibial mask boundary in the tibial mask boundary image corresponding to the knee X-ray image to be processed is mirrored to obtain a mirrored tibial mask boundary. Based on the tibial mask boundary image and the mirrored tibial mask boundary, a tibial mask image to be processed is constructed. This addresses at least one of the technical problems, such as errors in detecting the tibial or tibial cortex boundary lines due to the tibia being too short in the knee X-ray image or its corresponding mask image being too short in the knee X-ray image, thus affecting the accuracy of tibial midline detection.

[0107] In this embodiment of the disclosure, the knee X-ray image to be processed can be manually outlined to determine the anterior and posterior vertices of the tibial plateau corresponding to the knee X-ray image to be processed; at the same time, the knee X-ray image to be processed can also be automatically processed using image processing algorithms to determine the corresponding anterior and posterior vertices of the tibial plateau.

[0108] The step of constructing a tibial mask image to be processed based on the tibial mask boundary image and the mirror image of the tibial mask boundary includes: stitching the tibial mask boundary image and the mirror image of the tibial mask boundary along the mirror line to construct the tibial mask image to be processed.

[0109] The step of determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed includes: using the key point detection model corresponding to the key point detection network to detect the anterior and posterior vertices of the tibial plateau in the X-ray image of the knee to be processed, and determining the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed.

[0110] Constructing the keypoint detection model corresponding to the keypoint detection network includes: acquiring the anterior and posterior tibial plateau labels corresponding to multiple knee X-ray images; training the preset keypoint detection network using the anterior and posterior tibial plateau labels to obtain the keypoint detection model.

[0111] Before training the preset keypoint detection network using the anterior and posterior tibial plateau labels, the process includes: pre-training the preset keypoint detection network using multiple preset pose points corresponding to human pose images and / or multiple preset facial feature points corresponding to facial images to obtain a pre-trained keypoint detection model; then, re-training the pre-trained keypoint detection model using the anterior and posterior tibial plateau labels to obtain a keypoint detection model.

[0112] Before determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed, the process includes: determining the tibial length based on the tibial mask image or the tibial mask boundary image corresponding to the tibial mask image; if the tibial length is less than a preset tibial length, then determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed; otherwise, the tibial mask boundary is not mirrored.

[0113] The step of determining the mirror line based on the anterior and posterior tibial plateau vertices corresponding to the X-ray image of the knee to be processed includes: constructing corresponding vertex lines based on the anterior and posterior tibial plateau vertices corresponding to the X-ray image of the knee to be processed; and configuring the vertex lines as fixed mirror lines corresponding to the anterior and posterior tibial plateau vertices.

[0114] Determining the tibial mask boundary image corresponding to the knee X-ray image to be processed includes: using a knee joint X-ray image segmentation model corresponding to a preset segmentation network based on deep learning to segment the knee X-ray image to be processed into a tibial region mask image; and extracting the boundary of the tibial mask in the tibial region mask image to obtain a tibial mask boundary image.

[0115] Constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on mask label images corresponding to a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; training a preset segmentation network using the preset dynamic knee joint X-ray image training set and its corresponding mask label images to obtain a dynamic knee joint X-ray image segmentation model; or, Constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; training multiple preset segmentation networks using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain multiple dynamic knee joint X-ray image segmentation models; evaluating the multiple dynamic knee joint X-ray image segmentation models to determine the corresponding optimal dynamic knee joint X-ray image segmentation model.

[0116] The step of extracting the boundary of the tibia mask in the tibia region mask image to obtain a tibia mask boundary image includes: eroding the tibia mask in the tibia region mask image to obtain a tibial mask erosion image; and subtracting the tibial mask erosion image from the tibia region mask image to obtain the tibia mask boundary image corresponding to the tibia mask in the tibia region mask image.

[0117] This disclosure also proposes a method for processing X-ray images of the knee joint, including: using the tibial mask image processing method described above to determine the tibial mask image corresponding to the X-ray image of the knee to be processed (X-ray image of the knee joint to be processed); and determining one or two boundary lines, namely the third boundary line and the fourth boundary line, of the tibial or tibial cortex boundary lines on both sides of the X-ray image of the knee to be processed based on the tibial mask image.

[0118] The embodiment of this disclosure describes determining one or both of the third and fourth boundary lines among the tibial or tibial cortex boundary lines on both sides of the knee X-ray image to be processed based on the tibial mask image to be processed, including: performing a polar coordinate Hough transform on the tibial mask boundary and the tibial mask image to be processed corresponding to the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are parallel, then any one or two of the parallel straight lines are configured as one or both of the third and fourth boundary lines.

[0119] The embodiment of this disclosure describes determining one or both of the third and fourth boundary lines among the tibial or tibial cortex boundary lines on both sides of the knee X-ray image to be processed based on the tibial mask image to be processed. This includes: performing a polar coordinate Hough transform on the tibial mask boundary and the tibial mask image to be processed corresponding to the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are not parallel and the intersection point of any two non-parallel straight lines is outside the tibial mask image, then any one or two of the two straight lines corresponding to the intersection point of any two non-parallel straight lines outside the tibial mask image are configured as one or both of the third and fourth boundary lines.

[0120] In this embodiment of the disclosure, the method further includes: determining the tibial midline based on the third boundary line and the fourth boundary line.

[0121] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line and the fourth boundary line includes: determining a fourth direction vector corresponding to the third boundary line and a fifth direction vector corresponding to the fourth boundary line; performing dot products on the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain a third product corresponding to the fourth direction vector and a fourth product corresponding to the fifth direction vector; if the third product is less than a third preset value, inverting the fourth direction vector in the direction; otherwise, not processing the fourth direction vector; if the fourth product is less than a fourth preset value, inverting the fifth direction vector in the direction; otherwise, not processing the fifth direction vector; adding the inverted or unprocessed fourth direction vector and the inverted or unprocessed fifth direction vector to obtain the bisector direction vector; and determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector.

[0122] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector includes: if the third boundary line and the fourth boundary line intersect, then determining the tibial midline corresponding to the sixth polar coordinate equation based on the intersection point, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector; otherwise, determining the tibial midline corresponding to the sixth polar coordinate equation based on the midpoint of the perpendicular segment between the third boundary line and the fourth boundary line, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector.

[0123] In this embodiment of the disclosure, before extracting the tibial mask boundary image corresponding to the tibial mask in the tibial region mask image, the method includes: using a knee X-ray image segmentation model corresponding to a preset segmentation network based on deep learning to perform tibial segmentation on the knee X-ray image to be processed, thereby obtaining a tibial region mask image.

[0124] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; and training a preset segmentation network using the preset dynamic knee joint X-ray image training set and its corresponding mask label images to obtain a dynamic knee joint X-ray image segmentation model.

[0125] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set corresponding to mask label images; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; training multiple preset segmentation networks using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain multiple dynamic knee joint X-ray image segmentation models; evaluating the multiple dynamic knee joint X-ray image segmentation models to determine the corresponding optimal dynamic knee joint X-ray image segmentation model.

[0126] In this embodiment of the disclosure, the step of extracting the tibial mask boundary image corresponding to the tibial mask in the tibial region mask image includes: eroding the tibial mask in the tibial region mask image to obtain a tibial mask erosion image; and subtracting the tibial mask erosion image from the tibial region mask image to obtain the tibial mask boundary image corresponding to the tibial mask in the tibial region mask image.

[0127] In this embodiment of the disclosure, a method for determining the tibial midline is also proposed, comprising: determining a third boundary line and a fourth boundary line among the boundary lines on both sides of the tibia or the tibial cortex using the tibial boundary line determination method described above; and determining the tibial midline based on the third boundary line and the fourth boundary line.

[0128] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line and the fourth boundary line includes: determining a fourth direction vector corresponding to the third boundary line and a fifth direction vector corresponding to the fourth boundary line; performing dot products on the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain a third product corresponding to the fourth direction vector and a fourth product corresponding to the fifth direction vector; if the third product is less than a third preset value, inverting the fourth direction vector in the direction; otherwise, not processing the fourth direction vector; if the fourth product is less than a fourth preset value, inverting the fifth direction vector in the direction; otherwise, not processing the fifth direction vector; adding the inverted or unprocessed fourth direction vector and the inverted or unprocessed fifth direction vector to obtain the bisector direction vector; and determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector.

[0129] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector includes: if the third boundary line and the fourth boundary line intersect, then determining the tibial midline corresponding to the sixth polar coordinate equation based on the intersection point, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector; otherwise, determining the tibial midline corresponding to the sixth polar coordinate equation based on the midpoint of the perpendicular segment between the third boundary line and the fourth boundary line, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector.

[0130] In this embodiment of the disclosure, before performing dot products on the fourth direction vector and the fifth direction vector with the direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: normalizing the fourth direction vector, the fifth direction vector, and the direction vector corresponding to the tibial region mask image; wherein the values ​​corresponding to the third preset value and the fourth preset value are the same or different; and the values ​​corresponding to the third preset value and the fourth preset value are respectively configured to 0.

[0131] In this embodiment of the disclosure, before performing dot products of the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: calculating the tibial principal component direction vector corresponding to the non-zero point coordinates of the tibial mask in the tibial region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the tibial principal component direction vector to obtain the corrected tibial principal component direction vector; and performing dot products of the fourth direction vector and the fifth direction vector with the corrected tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector.

[0132] In this embodiment of the disclosure, the step of correcting the tibial principal component direction vector using the center point to obtain a corrected tibial principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the tibial mask in the tibial region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the corresponding non-zero point coordinate's tibial principal component direction vector to obtain multiple projection values ​​of the tibial principal component direction vector in the current principal component direction; determining the third non-zero point coordinate corresponding to the smallest projection value and the fourth non-zero point coordinate corresponding to the largest projection value among the multiple projection values; calculating the third distance and the fourth distance corresponding to the center point of the tibial region mask image and the third and fourth non-zero coordinates respectively; determining the direction corresponding to the tibial principal component direction vector based on the third and fourth distances; and correcting the tibial principal component direction vector based on the direction to obtain the corrected tibial principal component direction vector.

[0133] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the third distance and the fourth distance includes: if the third distance is less than the fourth distance, then the direction points from the tibia-knee joint to the ankle joint; otherwise, the direction points from the ankle joint to the tibia-knee joint; and / or, correcting the tibial principal component direction vector based on the direction to obtain a corrected tibial principal component direction vector includes: if the direction points from the tibia-knee joint to the ankle joint, then the tibial principal component direction vector is not corrected; if the direction points from the ankle joint to the tibia-knee joint, then the direction vector is inverted in the direction to obtain a corrected tibial principal component direction vector.

[0134] In this embodiment, a tibial mask image processing method is also proposed, comprising: determining a mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed (X-ray image of the knee joint to be processed); using the mirror line as the axis of symmetry, performing mirror processing on the tibial mask boundary image corresponding to the X-ray image of the knee to be processed to obtain a mirror image of the tibial mask boundary; and constructing a tibial mask image to be processed based on the tibial mask boundary image and the mirror image of the tibial mask boundary.

[0135] In this embodiment of the disclosure, the knee X-ray image to be processed can be manually outlined to determine the anterior and posterior vertices of the tibial plateau corresponding to the knee X-ray image to be processed; at the same time, the knee X-ray image to be processed can also be automatically processed using image processing algorithms to determine the corresponding anterior and posterior vertices of the tibial plateau.

[0136] In this embodiment of the disclosure, the step of constructing a tibial mask image to be processed based on the tibial mask boundary image and the mirror image of the tibial mask boundary includes: stitching the tibial mask boundary image and the mirror image of the tibial mask boundary along the mirror line to construct the tibial mask image to be processed.

[0137] In this embodiment of the disclosure, determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed includes: using a keypoint detection model corresponding to a keypoint detection network to detect the anterior and posterior vertices of the tibial plateau in the X-ray image of the knee to be processed, and determining the anterior and posterior vertices of the tibial plateau corresponding to the X-ray image of the knee to be processed.

[0138] In this embodiment of the disclosure, constructing a keypoint detection model corresponding to the keypoint detection network includes: acquiring anterior and posterior tibial plateau labels corresponding to multiple knee X-ray images; and training a preset keypoint detection network using the anterior and posterior tibial plateau labels to obtain a keypoint detection model.

[0139] In this embodiment of the disclosure, before training the preset keypoint detection network using the anterior and posterior tibial plateau vertices labels, the process includes: pre-training the preset keypoint detection network using multiple preset pose points corresponding to human pose images and / or multiple preset facial feature points corresponding to facial images to obtain a pre-trained keypoint detection model; then, re-training the pre-trained keypoint detection model using the anterior and posterior tibial plateau vertices labels to obtain a keypoint detection model.

[0140] In this embodiment of the disclosure, the preset keypoint detection network can be configured as one or more of the following deep learning networks: OpenPose, HRNet (High-Resolution Network), Hourglass Network, DEKR (Distributed Keypoint Regression), YOLO-KP, CenterNet, Cornernet, TokenPose, PoseTransformer, Mask R-CNN, etc.

[0141] In this embodiment of the disclosure, before determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the knee X-ray image to be processed, the method includes: determining the tibial length based on the tibial mask image or the tibial mask boundary image corresponding to the tibial mask image; if the tibial length is less than a preset tibial length, then determining the mirror line based on the anterior and posterior vertices of the tibial plateau corresponding to the knee X-ray image to be processed; otherwise, the tibial mask boundary is not mirrored. This addresses at least one of the technical problems, such as the tibia being too short in the knee X-ray image or the tibia in its corresponding mask image being too short in the knee X-ray image, which easily leads to errors in the detection of the tibial or tibial cortex boundary lines, thereby affecting the accuracy of the tibial midline detection.

[0142] In this embodiment of the disclosure, determining the mirror line based on the anterior and posterior tibial plateau vertices corresponding to the X-ray image of the knee to be processed includes: constructing a corresponding vertex line based on the anterior and posterior tibial plateau vertices corresponding to the X-ray image of the knee to be processed; and configuring the vertex line as a fixed mirror line corresponding to the anterior and posterior tibial plateau vertices.

[0143] In this embodiment of the disclosure, determining the tibial mask boundary image corresponding to the knee X-ray image to be processed includes: using a knee joint X-ray image segmentation model corresponding to a preset segmentation network based on deep learning to perform tibial segmentation on the knee X-ray image to be processed to obtain a tibial region mask image; and extracting the boundary of the tibial mask in the tibial region mask image to obtain a tibial mask boundary image.

[0144] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; and training a preset segmentation network using the preset dynamic knee joint X-ray image training set and its corresponding mask label images to obtain a dynamic knee joint X-ray image segmentation model.

[0145] In this embodiment of the disclosure, constructing the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set corresponding to mask label images; wherein, the mask label images include at least a femoral mask label; or, the mask label images include one or more of a patellar mask label, a patellar tendon mask label, a tibial mask label, and a femoral mask label; training multiple preset segmentation networks using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain multiple dynamic knee joint X-ray image segmentation models; evaluating the multiple dynamic knee joint X-ray image segmentation models to determine the corresponding optimal dynamic knee joint X-ray image segmentation model.

[0146] In this embodiment of the disclosure, the step of extracting the boundary of the tibia mask in the tibia region mask image to obtain a tibia mask boundary image includes: eroding the tibia mask in the tibia region mask image to obtain a tibial mask erosion image; and subtracting the tibial mask erosion image from the tibia region mask image to obtain a tibia mask boundary image corresponding to the tibia mask in the tibia region mask image.

[0147] In this embodiment of the disclosure, a method for processing X-ray images of the knee joint is also proposed, comprising: using the tibial mask image processing method described above to determine the tibial mask image corresponding to the X-ray image of the knee to be processed (X-ray image of the knee joint to be processed); and determining one or two boundary lines, namely a third boundary line and a fourth boundary line, in the boundary lines of the tibia or tibial cortex on both sides of the X-ray image of the knee to be processed, based on the tibial mask image.

[0148] In this embodiment of the disclosure, determining one or both of the third and fourth boundary lines among the tibial or tibial cortex boundary lines on both sides of the knee X-ray image to be processed based on the tibial mask image to be processed includes: performing a polar coordinate Hough transform on the tibial mask boundary and the tibial mask image to be processed corresponding to the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are parallel, then any one or two of the parallel straight lines are configured as one or both of the third and fourth boundary lines.

[0149] In this embodiment of the disclosure, determining one or both of the third and fourth boundary lines among the tibial or tibial cortex boundary lines of the knee X-ray image to be processed based on the tibial mask image to be processed includes: performing a polar coordinate Hough transform on the tibial mask boundary and the tibial mask image to be processed corresponding to the mirror image of the tibial mask boundary to determine a plurality of most probable straight lines; if any two of the plurality of most probable straight lines are not parallel and the intersection point of any two non-parallel straight lines is outside the tibial mask image, then any one or two of the two straight lines corresponding to the intersection point of any two non-parallel straight lines outside the tibial mask image are configured as one or both of the third and fourth boundary lines.

[0150] In this embodiment of the disclosure, the method further includes: determining the tibial midline based on the third boundary line and the fourth boundary line.

[0151] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line and the fourth boundary line includes: determining a fourth direction vector corresponding to the third boundary line and a fifth direction vector corresponding to the fourth boundary line; performing dot products on the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain a third product corresponding to the fourth direction vector and a fourth product corresponding to the fifth direction vector; if the third product is less than a third preset value, inverting the fourth direction vector in the direction; otherwise, not processing the fourth direction vector; if the fourth product is less than a fourth preset value, inverting the fifth direction vector in the direction; otherwise, not processing the fifth direction vector; adding the inverted or unprocessed fourth direction vector and the inverted or unprocessed fifth direction vector to obtain the bisector direction vector; and determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector.

[0152] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector includes: if the third boundary line and the fourth boundary line intersect, then determining the tibial midline corresponding to the sixth polar coordinate equation based on the intersection point, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector; otherwise, determining the tibial midline corresponding to the sixth polar coordinate equation based on the midpoint of the perpendicular segment between the third boundary line and the fourth boundary line, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector.

[0153] In this embodiment of the disclosure, before performing dot products on the fourth direction vector and the fifth direction vector with the direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: normalizing the fourth direction vector, the fifth direction vector and the direction vector corresponding to the tibial region mask image, respectively.

[0154] In this embodiment of the disclosure, the values ​​corresponding to the third preset value and the fourth preset value are the same or different; and / or, the values ​​corresponding to the third preset value and the fourth preset value are respectively configured to 0.

[0155] In this embodiment of the disclosure, before performing dot products of the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector, the method includes: calculating the tibial principal component direction vector corresponding to the non-zero point coordinates of the tibial mask in the tibial region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the tibial principal component direction vector to obtain the corrected tibial principal component direction vector; and performing dot products of the fourth direction vector and the fifth direction vector with the corrected tibial principal component direction vector corresponding to the tibial region mask image to obtain the third product corresponding to the fourth direction vector and the fourth product corresponding to the fifth direction vector.

[0156] In this embodiment of the disclosure, the step of correcting the tibial principal component direction vector using the center point to obtain a corrected tibial principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the tibial mask in the tibial region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the corresponding non-zero point coordinate's tibial principal component direction vector to obtain multiple projection values ​​of the tibial principal component direction vector in the current principal component direction; determining the third non-zero point coordinate corresponding to the smallest projection value and the fourth non-zero point coordinate corresponding to the largest projection value among the multiple projection values; calculating the third distance and the fourth distance corresponding to the center point of the tibial region mask image and the third and fourth non-zero coordinates respectively; determining the direction corresponding to the tibial principal component direction vector based on the third and fourth distances; and correcting the tibial principal component direction vector based on the direction to obtain the corrected tibial principal component direction vector.

[0157] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the third distance and the fourth distance includes: if the third distance is less than the fourth distance, then the direction points from the tibia-knee joint to the ankle joint; otherwise, the direction points from the ankle joint to the tibia-knee joint; and / or, correcting the tibial principal component direction vector based on the direction to obtain a corrected tibial principal component direction vector includes: if the direction points from the tibia-knee joint to the ankle joint, then the tibial principal component direction vector is not corrected; if the direction points from the ankle joint to the tibia-knee joint, then the direction vector is inverted in the direction to obtain a corrected tibial principal component direction vector.

[0158] Specifically, the non-zero coordinates of the tibial mask (tibiamask) in the tibial region mask image or tibial region mask image are extracted, and the principal component direction vector PCA_tibiavec of the tibia is calculated using the principal component analysis (PCA) algorithm. This includes calculating the principal component direction vectors corresponding to the non-zero coordinates of the tibial mask (tibiamask) in the tibial region mask image or tibial region mask image. Specifically, this includes: extracting the non-zero coordinates of the tibial mask tibiamask, using the principal component analysis (PCA) algorithm to calculate the principal component direction vector (tibial principal component direction vector) corresponding to the non-zero coordinates of the tibial mask tibiamask, and normalizing the tibial direction vector to obtain the normalized direction vector (tibial normalized principal component direction vector) PCA_tibiavec. (Note that the direction vector or normalized direction vector PCA_tibiavec corresponding to the non-zero coordinates obtained at this time may point from the side of the tibia near the patella to the side near the ankle joint, or it may point from the side of the tibia near the ankle joint to the side of the tibia near the patella.)

[0159] Calculating the coordinates of the center point cener_tibia of the tibial mask tibiamask includes: calculating the average of the abscissas of all non-zero points of the tibial mask tibiamask and the average of the ordinates of all non-zero points of the tibial mask tibiamask, to obtain the abscissa and ordinate of the center point cener_tibiar.

[0160] Correct the direction vector corresponding to the non-zero coordinates of the tibial mask femurmask. Specifically, this includes: extracting the non-zero coordinates of the tibial mask tibiamask; calculating the principal component direction vector PCA_tibiavec of the tibia using the principal component analysis (PCA) algorithm; determining the direction of the principal component direction vector of the tibia, and the coordinates of the center point cener_tibia of the tibial mask. Specifically, this includes: subtracting the center point cener_tibia of the tibial mask from each non-zero coordinate of the tibial mask tibiamask to obtain the relative vector of each non-zero coordinate of the tibial mask tibiamask relative to the center point cener_tibia of the tibial mask; multiplying each relative vector by the direction vector or normalized direction vector PCA_femurvec of the corresponding non-zero coordinate to obtain multiple projection values ​​projs_pts of the direction vector in the current principal component direction; finding the third and fourth non-zero coordinates of the tibial mask tibiamask corresponding to the minimum and maximum projection values ​​among the multiple projection values ​​projs_pts, denoted as end1_pos (third non-zero coordinate) and end2_pos (fourth non-zero coordinate). Take the center point `imgpoint_center` of the image to be segmented or the corresponding segmentation mask image. Calculate the third distance `dist3` between the third non-zero point coordinates `end3_pos` and `imgpoint_center`, and the fourth distance `dist4` between the fourth non-zero point coordinates `end4_pos` and `imgpoint_center`. If the third distance `dist3` is less than the fourth distance `dist4`, then the direction vector or normalized direction vector `PCA_tibiavec` is from the tibia-knee joint to the tibia-ankle joint. If the third distance `dist3` is greater than the fourth distance `dist4`, then the direction vector or normalized direction vector `PCA_tibiavec` is inverted (negated), i.e., `PCA_tibiavec = -PCA_tibiavec`. After the above steps, the tibia direction vector corresponding to the non-zero point coordinates of the tibia mask `tibiamask` is corrected, resulting in the corrected tibia direction vector.

[0161] Extract the tibial mask boundary image (tibiamask edge image) from the tibial mask image. Specifically, this includes: performing tibial mask erosion on the tibial mask image to obtain the eroded tibial mask image tibiamaskerodeimg, and subtracting the eroded tibial mask image tibiamaskodeimg from the tibial mask image to extract the tibial mask boundary image tibiamask_edge.

[0162] The third and fourth boundary lines corresponding to the outermost two boundary lines of the tibia or tibial cortex are determined. Specifically, this includes performing a polar coordinate Hough transform on the tibial mask boundary image tibiamask_edge to determine the two most likely equations of the lines to be corrected, which are the fourth and fifth polar coordinate equations corresponding to the two outermost edge lines of the tibia or tibial cortex. The two outermost edge lines of the tibia or tibial cortex are respectively configured as the third and fourth boundary lines corresponding to the outermost two boundary lines of the tibia or tibial cortex. The third boundary line is represented using the fourth polar coordinate equation; the fourth boundary line is represented using the corresponding fifth polar coordinate equation. It is then determined whether the two lines are parallel or whether the intersection of the two lines is outside the tibial mask image. Specifically, this includes: determining whether the third and fourth boundary lines to be corrected corresponding to the outermost boundary lines of the tibia or tibial cortex are parallel; if not parallel, determining the intersection point of the third and fourth boundary lines to be corrected corresponding to the outermost boundary lines of the tibia or tibial cortex; determining whether the intersection point is outside the tibial mask image; if the two lines to be corrected are parallel or the intersection point is outside the tibial mask image, then these two lines to be corrected are the equations (fourth polar coordinate equation and fifth polar coordinate equation) of the two outermost edge lines to be corrected of the tibial cortex. If the intersection point of the two lines to be corrected is inside the tibial mask image, the following steps are performed: The keypoint detection model corresponding to the HRnet network was used to detect keypoints in the X-ray image of the knee joint to be processed corresponding to the tibial mask image, obtaining the anterior vertex A and posterior vertex B of the tibial plateau. Based on the anterior vertex A and posterior vertex B of the tibial plateau, a straight line AB (a mirror line) was determined. Using the straight line AB as the axis of symmetry, the tibial mask boundary in the tibiamask_edge image was symmetrically copied to the other side to obtain the copied tibial mask boundary tibiamask_edgecopy.

[0163] A polar coordinate Hough transform is performed on both the copied tibial mask boundary (tibiamask_edgecopy) and the tibial mask boundary to determine the sixth polar coordinate equation (the first modified polar coordinate equation corresponding to the fourth polar coordinate equation), the seventh polar coordinate equation (the second modified polar coordinate equation corresponding to the fifth polar coordinate equation), and the eighth polar coordinate equation (interference polar coordinate equation) corresponding to the three most likely straight lines (multiple most likely straight lines). The third and fourth boundary lines corresponding to the outermost two boundary lines of the tibia or tibial cortex, and the interference boundary lines are denoted as Line1, Line2, and Line3, respectively. Each straight line is expressed in polar coordinates as x*cosθ+y*sinθ=ρ, where x and y are variables.

[0164] Based on the sixth polar equation (the first modified polar equation corresponding to the fourth polar equation), the seventh polar equation (the second modified polar equation corresponding to the fifth polar equation), and the eighth polar equation (interference polar equation) corresponding to the three most likely straight lines, the equations of the two outermost edge lines of the tibia or tibial cortex are determined. Specifically, this includes: determining whether any two of the three most likely straight lines are parallel; if they are parallel, then the two parallel lines are determined as the third and fourth boundary lines corresponding to the two outermost boundary lines of the tibia or tibial cortex; if any two of the three most likely straight lines are not parallel, then determine whether the intersection point of any two non-parallel lines is outside the tibial mask image; if it is outside the tibial mask image, then the two lines corresponding to the intersection point outside the tibial mask image are determined as the third and fourth boundary lines corresponding to the two outermost boundary lines of the tibia or tibial cortex.

[0165] Specifically, the fourth polar coordinate equation of the fourth straight line corresponding to the third boundary line on one side of the tibia is configured as x*cosθ4+y*sinθ4=ρ4, and the fifth polar coordinate equation of the fifth straight line corresponding to the fourth boundary line on the other side of the tibia is configured as x*cosθ5+y*sinθ5=ρ5. Here, θ4 and θ5 represent the fourth angle between the perpendicular from the origin of the rectangular coordinate system to the fourth boundary line and the x-axis, and the fifth angle between the perpendicular from the origin of the rectangular coordinate system to the fourth boundary line and the x-axis, respectively. ρ4 and ρ5 represent the fourth distance from the origin of the rectangular coordinate system to the fourth boundary line and the fifth distance from the origin of the rectangular coordinate system to the fourth boundary line, respectively. The third boundary line is represented using the fourth polar coordinate equation; the fourth boundary line is represented using the corresponding fifth polar coordinate equation.

[0166] Based on the third boundary line corresponding to one side of the tibia and the fourth boundary line corresponding to the other side of the tibia, the polar coordinate equation corresponding to the tibial midline is determined. This includes: based on the third boundary line Linerleft corresponding to one side of the tibia and the fourth boundary line Lineright corresponding to the other side of the tibia, determining the third direction vector vec3 as (-sinθ4, cosθ4) and the fourth direction vector vec4 as (-sinθ5, cosθ5) of the two straight lines. The third direction vector vec3 and the fourth direction vector vec4 are normalized to obtain the third normalized direction vector and the fourth normalized direction vector; the third direction vector or the third normalized direction vector, the fourth direction vector or the fourth normalized direction vector, and the tibial correction direction vector corresponding to the non-zero coordinates of the corrected tibial mask tibiamask are then multiplied by a dot to obtain the third product corresponding to the third normalized direction vector or the third direction vector and the fourth product corresponding to the fourth normalized direction vector or the fourth direction vector. If the third product is less than 0, the direction of the third direction vector or the third normalized direction vector is inverted (negated); if the fourth product is less than 0, the direction of the fourth direction vector or the fourth normalized direction vector is inverted (negated). After the above steps, it is ensured that the directions of the third normalized direction vector or the third direction vector and the fourth normalized direction vector or the fourth direction vector of the two lines (the third boundary line Linerleft corresponding to one side of the tibia and the fourth boundary line Lineright corresponding to the other side of the tibia) are consistent with the direction of the tibia direction vector or the normalized tibia direction vector PCA_tibiavec.

[0167] After determining the direction, the third direction vector vec3 and the fourth direction vector vec4 are added together to obtain the tibial bisector direction vector (i.e., the direction vector of the middle bisector corresponding to the third boundary line Linerleft on one side of the tibia and the fourth boundary line Lineright on the other side of the tibia). The tibial bisector direction vector is then normalized to obtain the tibial normalized bisector direction vector tibiavec6, which is (-sinθ6, cosθ6). The sixth polar coordinate equation of the sixth line corresponding to the tibial bisector direction vector or the tibial normalized bisector direction vector tibiavec6 is configured as x*cosθ6+y*sinθ6=ρ6. Here, θ6 represents the sixth angle between the perpendicular line from the origin of the rectangular coordinate system to the sixth line and the x-axis, and ρ6 represents the sixth distance from the origin of the rectangular coordinate system to the sixth line. By taking the intersection point (x, y) of the fourth line Linerleft and the fifth line Lineright, the sixth distance ρ6 from the origin of the rectangular coordinate system corresponding to the tibial bisector direction vector or the tibial normalized bisector direction vector femurvec6 can be calculated based on the intersection point (x, y), the tibial normalized bisector direction vector, and the sixth polar coordinate equation of the sixth line. If the fourth line Linerleft and the fifth line Lineright do not intersect, the sixth distance ρ6 from the origin of the rectangular coordinate system corresponding to the tibial bisector direction vector or the tibial normalized bisector direction vector femurvec6 can be determined based on the midpoint (x, y) of the perpendicular segment between the fourth line Linerleft and the fifth line Lineright, the tibial bisector direction vector, and the sixth polar coordinate equation of the sixth line. Furthermore, the sixth polar coordinate equation of the sixth line corresponding to the tibial midline is configured as x*cosθ6 + y*sinθ6 = ρ6.

[0168] In this embodiment of the disclosure, the method further includes: determining the joint angle between the femoral midline and the tibial midline based on the tibial midline and the femoral midline corresponding to the X-ray image of the knee joint to be processed.

[0169] In this embodiment of the disclosure, determining the joint angle between the femoral midline and the tibial midline based on the tibial midline and the femoral midline corresponding to the X-ray image of the knee joint to be processed includes: establishing a system of polar coordinate equations based on the femoral midline and the tibial midline corresponding to the X-ray image of the knee joint to be processed; and using the system of polar coordinate equations to determine the joint angle between the femoral midline and the tibial midline.

[0170] In this embodiment of the disclosure, determining the joint angle point between the femoral midline and the tibial midline using the polar coordinate equations includes: if the polar coordinate equations have a unique solution, determining whether the coordinates corresponding to the unique solution are within the X-ray image of the knee joint to be processed; if they are within the X-ray image of the knee joint to be processed, determining the coordinates corresponding to the unique solution as the joint angle point between the femoral midline and the tibial midline; if they are not within the X-ray image of the knee joint to be processed or the polar coordinate equations have no solution, then based on the femoral midline and the femoral mask boundary image... The first coordinate point corresponding to the joint angle is determined by using the first non-zero coordinate corresponding to the boundary of the femoral mask, the coordinates of the femoral midline, and the center point of the X-ray image of the knee joint to be processed; the second coordinate point corresponding to the joint angle is determined by using the second non-zero coordinate corresponding to the boundary of the tibial mask in the image of the tibial midline and the coordinates of the center point of the X-ray image of the knee joint to be processed; the midpoint between the first coordinate point and the second coordinate point is calculated, and the midpoint is determined as the joint angle between the femoral midline and the tibial midline.

[0171] In this embodiment of the disclosure, determining the femoral midline includes: extracting the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image corresponding to the knee joint X-ray image to be processed; determining the first boundary line and the second boundary line among the boundary lines on both sides of the femur or femoral cortex based on the non-zero coordinates of the femoral mask boundary image; and determining the femoral midline based on the first boundary line and the second boundary line.

[0172] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line and the second boundary line includes: determining a first direction vector corresponding to the first boundary line and a second direction vector corresponding to the second boundary line; performing dot products on the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector; if the first product is less than a first preset value, inverting the first direction vector in the direction; otherwise, not processing the first direction vector; if the second product is less than a second preset value, inverting the second direction vector in the direction; otherwise, not processing the second direction vector; adding the first direction vector corresponding to the inverted or unprocessed and the second direction vector corresponding to the inverted or unprocessed to obtain the bisecting line direction vector; and determining the femoral midline based on the first boundary line, the second boundary line, and the bisecting line direction vector.

[0173] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line, the second boundary line, and the direction vector of the bisector includes: if the first boundary line and the second boundary line intersect, then determining the femoral midline corresponding to the third polar coordinate equation based on the intersection point, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector; otherwise, determining the femoral midline corresponding to the third polar coordinate equation based on the midpoint of the perpendicular segment between the first boundary line and the second boundary line, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector.

[0174] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the direction vector corresponding to the femoral region mask image.

[0175] In this embodiment of the disclosure, the values ​​corresponding to the first preset value and the second preset value are the same or different; and / or, the values ​​corresponding to the first preset value and the second preset value are respectively configured to 0.

[0176] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector, the method includes: calculating the femoral principal component direction vector corresponding to the non-zero point coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the femoral principal component direction vector to obtain a corrected femoral principal component direction vector; and performing dot products of the first direction vector and the second direction vector with the corrected femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector.

[0177] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the femoral mask in the femoral region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the femoral principal component direction vector of the corresponding non-zero point coordinate to obtain multiple projection values ​​of the femoral principal component direction vector in the current principal component direction; determining the first non-zero point coordinate corresponding to the minimum projection value and the second non-zero point coordinate corresponding to the maximum projection value among the multiple projection values; calculating the first distance and the second distance between the center point of the femoral region mask image and the first non-zero coordinate and the second non-zero coordinate respectively; determining the direction corresponding to the femoral principal component direction vector based on the first distance and the second distance; and correcting the femoral principal component direction vector based on the direction to obtain the corrected femoral principal component direction vector.

[0178] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the first distance and the second distance includes: if the first distance is less than the second distance, the direction is from the femoral knee joint to the hip joint; otherwise, the direction is from the hip joint to the femoral knee joint.

[0179] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector based on the direction to obtain a corrected femoral principal component direction vector includes: if the direction is from the femoral knee joint to the hip joint, then the femoral principal component direction vector is not corrected; if the direction is from the hip joint to the femoral knee joint, then the direction vector is inverted in the direction to obtain a corrected femoral principal component direction vector.

[0180] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the femoral principal component direction vector corresponding to the femoral region mask image.

[0181] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the modified femoral principal component direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the modified femoral principal component direction vector corresponding to the femoral region mask image.

[0182] This disclosure proposes a method for determining the bilateral boundary lines of the femur, comprising: extracting a femoral mask boundary image corresponding to the femoral mask in a femoral region mask image; and determining one or both of the first and second boundary lines among the bilateral boundary lines of the femur or femoral cortex based on the non-zero coordinates of the femoral mask boundary in the femoral mask boundary image. This addresses at least one of the following technical problems: the lack of an effective quantitative method for determining the bilateral boundary lines and femoral midline, which hinders the auxiliary diagnosis of fractures and dislocations, assessment of joint development and deformities, guidance of surgical planning and reduction, and monitoring of treatment effects and rehabilitation progress; the inherent subjectivity and difficulty in ensuring objectivity in manually determining the bilateral boundary lines and femoral midline; and the increased workload for radiologists due to the already demanding nature of the procedure.

[0183] In this embodiment of the disclosure, the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image includes: eroding the femoral mask in the femoral region mask image to obtain a femoral mask erosion image; subtracting the femoral mask erosion image from the femoral region mask image to obtain the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image.

[0184] In this embodiment of the disclosure, determining one or both of the first and second boundary lines among the boundary lines on both sides of the femur based on the non-zero coordinates of the femur mask boundary of the femur mask boundary image includes: using polar coordinate Hough transform to determine the first boundary line on one side of the femur corresponding to the first straight line (first polar coordinate equation) based on the first polar coordinate equation corresponding to the first preset distance range and the first preset angle range, and the non-zero coordinates of the first side femur mask boundary corresponding to the femur mask boundary image; and / or, using polar coordinate Hough transform to determine the second boundary line on the other side of the femur corresponding to the second straight line (second polar coordinate equation) based on the second polar coordinate equation corresponding to the second preset distance range and the second preset angle range, and the non-zero coordinates of the second side femur mask boundary corresponding to the femur mask boundary image.

[0185] In this embodiment of the disclosure, determining the first preset distance range includes: using the diagonal length of the femoral mask boundary image to determine the first preset distance range corresponding to the first distance from the origin of the rectangular coordinate system corresponding to the first polar coordinate parameter in the first polar coordinate equation to the first straight line.

[0186] In this embodiment of the disclosure, determining the second preset distance range includes: using the diagonal length of the femoral mask boundary image to determine the second preset distance range corresponding to the second distance from the origin of the rectangular coordinate system corresponding to the third polar coordinate parameter in the second polar coordinate equation to the second straight line.

[0187] In this embodiment of the disclosure, the first preset included angle range is configured to be 0 to 180 degrees; the first preset distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs, respectively.

[0188] In this embodiment of the disclosure, the second preset included angle range is configured to be 0 to 180 degrees and / or, and the second preset distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs, respectively.

[0189] In this embodiment of the disclosure, determining one or both of the first and second boundary lines among the boundary lines on both sides of the femur based on the non-zero coordinates of the femur mask boundary in the femur mask boundary image includes: determining a first distance range corresponding to the first distance from the origin of the rectangular coordinate system corresponding to the first polar coordinate parameter in the first polar coordinate equation to the first straight line using the diagonal length of the femur mask boundary image; configuring a first angle range corresponding to the first angle between the perpendicular line from the origin of the rectangular coordinate system corresponding to the second polar coordinate parameter in the first polar coordinate equation to the first straight line and the x-axis; and using the polar coordinate Hough transform to determine the boundary lines based on the first distance range and the first angle range corresponding to the first polar coordinate equation and the non-zero coordinates of the first side femur mask boundary corresponding to the femur mask boundary image. First, determine the first boundary line on one side of the femur corresponding to the first straight line (first polar coordinate equation); and / or, determine the second distance range corresponding to the second distance from the origin of the rectangular coordinate system corresponding to the third polar coordinate parameter in the second polar coordinate equation to the second straight line using the diagonal length of the femoral mask boundary image; configure the second angle range corresponding to the second included angle between the perpendicular line from the origin of the rectangular coordinate system corresponding to the fourth polar coordinate parameter in the second polar coordinate equation to the second straight line and the x-axis; based on the second distance range and the second included angle range corresponding to the second polar coordinate equation and the non-zero coordinates of the second side femoral mask boundary corresponding to the femoral mask boundary image, determine the second boundary line on the other side of the femur corresponding to the second straight line (second polar coordinate equation) using polar coordinate Hough transform.

[0190] In this embodiment of the disclosure, the first included angle range is configured to be 0 to 180 degrees; and / or, the first distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs respectively; and / or, the second included angle range is configured to be 0 to 180 degrees; and / or, the second distance range is configured to be -diagonal length to +diagonal length; wherein - and + represent negative and positive signs respectively.

[0191] In this embodiment of the disclosure, before extracting the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image, the method includes: using a knee X-ray imaging image segmentation model to perform femoral segmentation on the knee X-ray imaging image to be processed, thereby obtaining the femoral region mask image corresponding to the knee X-ray imaging image to be processed.

[0192] In this embodiment of the disclosure, determining the knee X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset knee X-ray image training set or a preset dynamic knee X-ray image training set; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more mask labels such as a femoral mask label, a patellar mask label, a tibial mask label, and a patellar tendon mask label; and training a preset segmentation network using the preset knee X-ray image training set or the preset dynamic knee X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain the knee X-ray image segmentation model or the dynamic knee X-ray image segmentation model.

[0193] In this embodiment of the disclosure, determining the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset knee joint X-ray image training set or a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more mask labels such as a femoral mask label, a patellar mask label, a tibial mask label, and a patellar tendon mask label; and using the preset knee joint X-ray image training set or the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function, performing multiple preset segmentation networks... Training is performed to obtain multiple knee joint X-ray image segmentation models or multiple dynamic knee joint X-ray image segmentation models; the multiple knee joint X-ray image segmentation models or the multiple dynamic knee joint X-ray image segmentation models are evaluated to determine the corresponding optimal knee joint X-ray image segmentation model or optimal dynamic knee joint X-ray image segmentation model; based on the optimal knee joint X-ray image segmentation model or the optimal dynamic knee joint X-ray image segmentation model, the knee joint X-ray image or dynamic knee joint X-ray image to be processed is segmented into one or more segments, such as femoral segmentation or segmentation of the femur and patella, tibia, and patellar tendon.

[0194] This disclosure also proposes a method for determining the femoral midline, comprising: using the above-described method for determining the boundary lines on both sides of the femur to determine a first boundary line and a second boundary line among the boundary lines on both sides of the femur or the femoral cortex; and determining the femoral midline based on the first boundary line and the second boundary line.

[0195] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line and the second boundary line includes: determining a first direction vector corresponding to the first boundary line and a second direction vector corresponding to the second boundary line; performing dot products on the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector; if the first product is less than a first preset value, inverting the first direction vector in the direction; otherwise, not processing the first direction vector; if the second product is less than a second preset value, inverting the second direction vector in the direction; otherwise, not processing the second direction vector; adding the first direction vector corresponding to the inverted or unprocessed and the second direction vector corresponding to the inverted or unprocessed to obtain the bisecting line direction vector; and determining the femoral midline based on the first boundary line, the second boundary line, and the bisecting line direction vector.

[0196] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line, the second boundary line, and the direction vector of the bisector includes: if the first boundary line and the second boundary line intersect, then determining the femoral midline corresponding to the third polar coordinate equation based on the intersection point, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector; otherwise, determining the femoral midline corresponding to the third polar coordinate equation based on the midpoint of the perpendicular segment between the first boundary line and the second boundary line, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector.

[0197] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the direction vector corresponding to the femoral region mask image.

[0198] In this embodiment of the disclosure, the values ​​corresponding to the first preset value and the second preset value are the same or different; and / or, the values ​​corresponding to the first preset value and the second preset value are respectively configured to 0.

[0199] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector, the method includes: calculating the femoral principal component direction vector corresponding to the non-zero point coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero point coordinates; using the center point to correct the femoral principal component direction vector to obtain a corrected femoral principal component direction vector; and performing dot products of the first direction vector and the second direction vector with the corrected femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector.

[0200] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the femoral mask in the femoral region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the femoral principal component direction vector of the corresponding non-zero point coordinate to obtain multiple projection values ​​of the femoral principal component direction vector in the current principal component direction; determining the first non-zero point coordinate corresponding to the minimum projection value and the second non-zero point coordinate corresponding to the maximum projection value among the multiple projection values; calculating the first distance and the second distance between the center point of the femoral region mask image and the first non-zero coordinate and the second non-zero coordinate respectively; determining the direction corresponding to the femoral principal component direction vector based on the first distance and the second distance; and correcting the femoral principal component direction vector based on the direction to obtain the corrected femoral principal component direction vector.

[0201] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the first distance and the second distance includes: if the first distance is less than the second distance, the direction is from the femoral knee joint to the hip joint; otherwise, the direction is from the hip joint to the femoral knee joint.

[0202] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector based on the direction to obtain a corrected femoral principal component direction vector includes: if the direction is from the femoral knee joint to the hip joint, then the femoral principal component direction vector is not corrected; if the direction is from the hip joint to the femoral knee joint, then the direction vector is inverted in the direction to obtain a corrected femoral principal component direction vector.

[0203] In this embodiment of the disclosure, before performing dot products of the first direction vector and the second direction vector with the direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the femoral principal component direction vector corresponding to the femoral region mask image.

[0204] In this embodiment of the disclosure, before performing dot products on the first direction vector and the second direction vector with the modified femoral principal component direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector, the method includes: normalizing the first direction vector, the second direction vector and the modified femoral principal component direction vector corresponding to the femoral region mask image.

[0205] This disclosure proposes a femoral direction vector correction method, comprising: calculating the femoral principal component direction vector corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero coordinates; and correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector. This addresses at least one of the following technical problems: the lack of femoral direction vector correction during the automatic determination of the femoral or femoral cortex boundary lines results in insufficient accuracy for femoral or femoral cortex boundary line detection; the need to avoid femoral midline detection failure; the high subjectivity of manually annotating the tibial boundary line and tibial midline; and the need for further improvement in the accuracy of current automatic detection of tibial boundary lines and tibial midline.

[0206] In this embodiment of the disclosure, calculating the femoral principal component direction vector corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image includes: processing the non-zero coordinates of the femoral mask in the femoral region mask image using a principal component analysis algorithm to obtain the femoral principal component direction vector corresponding to the non-zero coordinates.

[0207] In this embodiment of the disclosure, calculating the center point corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image includes: calculating the average value of the abscissa of the non-zero abscissa and the average value of the ordinate of the non-zero ordinate corresponding to the femoral mask, and determining the center point corresponding to the non-zero coordinates of the femoral mask.

[0208] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector using the center point to obtain a corrected femoral principal component direction vector includes: subtracting the center point from each non-zero point coordinate of the femoral mask in the femoral region mask image to obtain a relative vector of each non-zero point coordinate relative to the center point; multiplying each relative vector by the femoral principal component direction vector of the corresponding non-zero point coordinate to obtain multiple projection values ​​of the femoral principal component direction vector in the current principal component direction; determining the first non-zero point coordinate corresponding to the minimum projection value and the second non-zero point coordinate corresponding to the maximum projection value among the multiple projection values; calculating the first distance and the second distance between the center point of the femoral region mask image and the first non-zero coordinate and the second non-zero coordinate respectively; determining the direction corresponding to the femoral principal component direction vector based on the first distance and the second distance; and correcting the femoral principal component direction vector based on the direction to obtain the corrected femoral principal component direction vector.

[0209] In this embodiment of the disclosure, determining the direction corresponding to the direction vector based on the first distance and the second distance includes: if the first distance is less than the second distance, the direction is from the femoral knee joint to the femoral hip joint; otherwise, the direction is from the femoral hip joint to the femoral knee joint.

[0210] In this embodiment of the disclosure, the step of correcting the femoral principal component direction vector based on the direction to obtain a corrected femoral principal component direction vector includes: if the direction is from the femoral knee joint to the femoral hip joint, then the femoral principal component direction vector is not corrected; if the direction is from the femoral hip joint to the femoral knee joint, then the femoral principal component direction vector is inverted in the direction to obtain a corrected femoral principal component direction vector.

[0211] In this embodiment of the disclosure, before calculating the femoral principal component direction vector corresponding to the non-zero coordinates of the femoral mask in the femoral region mask image and the center point corresponding to the non-zero coordinates, the method includes: using a knee X-ray imaging image segmentation model to perform femoral segmentation on the knee X-ray imaging image to be processed, thereby obtaining the femoral region mask image corresponding to the knee X-ray imaging image to be processed.

[0212] In this embodiment of the disclosure, determining the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on the mask label images corresponding to a preset dynamic knee joint X-ray image training set; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more of the following mask labels: femoral mask label, patellar mask label, tibia mask label, and patellar tendon mask label; and training a preset segmentation network using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, and the multi-segmentation region weight loss function to obtain the dynamic knee joint X-ray image segmentation model.

[0213] In this embodiment of the disclosure, determining the knee joint X-ray image segmentation model includes: determining the knee joint X-ray image segmentation model includes: constructing a multi-segmentation region weight loss function based on a preset training set of multiple knee joint X-ray images or a preset dynamic knee joint X-ray image training set corresponding to mask label images; wherein, the mask label images include at least a femoral mask label or the mask label images include one or more mask labels such as a femoral mask label and a patellar mask label, a tibial mask label, and a patellar tendon mask label; using the preset dynamic knee joint X-ray image training set and its corresponding mask label images, the multi-segmentation region weight loss function... The system trains multiple preset segmentation networks to obtain multiple knee X-ray image segmentation models or multiple dynamic knee X-ray image segmentation models; it evaluates the multiple knee X-ray image segmentation models or the multiple dynamic knee X-ray image segmentation models to determine the optimal knee X-ray image segmentation model or the optimal dynamic knee X-ray image segmentation model; based on the optimal knee X-ray image segmentation model or the optimal dynamic knee X-ray image segmentation model, it performs femoral segmentation or one or more segmentations of the femur, patella, tibia, and patellar tendon on the knee X-ray image or dynamic knee X-ray image to be segmented.

[0214] This disclosure also proposes a method for processing X-ray images of the knee joint, comprising: determining one or two boundary lines, namely a first boundary line and a second boundary line, in the boundary lines on both sides of the femur or femoral cortex, based on the non-zero coordinates of the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image; determining a modified femoral principal component direction vector corresponding to the femoral principal component direction vector using the femoral direction vector correction method described above; and determining the femoral midline based on the modified femoral principal component direction vector, the first boundary line, and the second boundary line.

[0215] In this embodiment of the disclosure, determining the femoral midline based on the modified femoral principal component direction vector, the first boundary line, and the second boundary line includes: determining a first direction vector corresponding to the first boundary line and a second direction vector corresponding to the second boundary line; performing dot products on the first direction vector and the second direction vector with the modified femoral principal component direction vector corresponding to the femoral region mask image to obtain a first product corresponding to the first direction vector and a second product corresponding to the second direction vector; if the first product is less than a first preset value, inverting the first direction vector in the direction; otherwise, not processing the first direction vector; if the second product is less than a second preset value, inverting the second direction vector in the direction; otherwise, not processing the second direction vector; adding the first direction vector corresponding to the inverted or unprocessed and the second direction vector corresponding to the inverted or unprocessed to obtain the bisecting line direction vector; and determining the femoral midline based on the first boundary line, the second boundary line, and the bisecting line direction vector.

[0216] Calculate the direction vectors (principal component direction vectors) corresponding to the non-zero coordinates of the femoral mask (femurmask) in the femoral region mask image or femoral region mask image. Specifically, this includes: extracting the non-zero coordinates of the femoral mask (femurmask), calculating the direction vectors (femoral principal component direction vectors) corresponding to the non-zero coordinates of the femoral mask (femoral principal component direction vectors) using the principal component analysis (PCA) algorithm, and normalizing the direction vectors to obtain the normalized direction vector (femoral normalized direction vector) PCA_femurvec. (Note that the direction vector or normalized direction vector PCA_femurvec corresponding to the non-zero coordinates obtained at this time may point from the side of the femur near the patella to the side near the hip joint, or it may point from the side of the femur near the hip joint to the side of the femur near the patella.)

[0217] Calculate the center point center_femur of the femoral mask. Specifically, this includes: calculating the average of the x-coordinates of all non-zero points on the femoral mask and the average of the y-coordinates of all non-zero points on the femoral mask, to obtain the x-coordinate and y-coordinate of the center point center_femur.

[0218] Correcting the direction vectors corresponding to the non-zero coordinates of the femurmask. Specifically, this includes: subtracting the center point `center_femur` from each non-zero coordinate of the femurmask to obtain the relative vector of each non-zero coordinate relative to the center point `center_femur`; multiplying each relative vector by the direction vector or normalized direction vector `PCA_femurvec` of the corresponding non-zero coordinate to obtain multiple projection values ​​`projs_pts` of the direction vector in the current principal component direction; finding the first and second non-zero coordinates of the femurmask corresponding to the minimum and maximum projection values ​​among the multiple projection values ​​`projs_pts`, denoted as `end1_pos` (first non-zero coordinate) and `end2_pos` (second non-zero coordinate). Take the center point `imgpoint_center` of the image to be segmented or the segmentation mask image corresponding to the image to be segmented; calculate the first distance `dist1` between the first non-zero point coordinate `end1_pos` and `imgpoint_center`, and the second distance `dist2` between the second non-zero point coordinate `end2_pos` and `imgpoint_center`; if the first distance `dist1` is less than the second distance `dist2`, then the direction of the direction vector or normalized direction vector `PCA_femurvec` is from the femoral knee joint to the femoral hip joint; if the first distance `dist1` is greater than the second distance `dist2`, then invert the direction of the direction vector or normalized direction vector `PCA_femurvec` (add a negative sign), that is, set the direction vector or normalized direction vector `PCA_femurvec` = -PCA_femurvec. After the above steps, the direction vector corresponding to the non-zero point coordinates of the femoral mask `femurmask` is corrected, and the corrected direction vector is obtained.

[0219] Extract the set of non-zero coordinates of the femoral mask boundary image (femurmask_edge) corresponding to the femoral mask boundary image (femurmask_edge). Specifically, this includes: eroding the femoral mask to obtain the eroded femoral mask image femurmaskerodeimg; subtracting the eroded femoral mask image femurmaskerodeimg from the femoral mask to extract the femoral mask boundary image femurmask_edge; and extracting the set of non-zero coordinates of the femoral mask boundary image femurmask_edge femurmask_edgeptrvec.

[0220] Based on the non-zero coordinate set femurmask_edgeptrvec of the femoral mask boundary, fit the first polar coordinate equation of the first straight line corresponding to the two sides of the femur or femoral cortex and the second polar coordinate equation of the second straight line opposite to the first straight line (the first straight line and the second straight line constitute the boundary of the two sides of the femur). Specifically, this includes: based on the diagonal length of the image to be segmented or the segmentation mask image corresponding to the image to be segmented being max_rho, determining the range of the first polar coordinate parameter (the first distance from the origin of the rectangular coordinate system to the first straight line) ρ in the polar coordinate equation to be from -max_rho to max_rho; configuring the range of the second polar coordinate parameter (the first angle between the perpendicular from the origin of the rectangular coordinate system to the first straight line and the x-axis) θ in the polar coordinate equation to be from 0 to 180 degrees; based on the polar coordinate equations corresponding to the distance range and the angle range and the set of non-zero coordinates of the femoral mask boundary (the set of non-zero coordinates of the first femoral mask boundary and the set of non-zero coordinates of the second femoral mask boundary), using the polar coordinate Hough (line) transformation (the line from a point in polar coordinates to the Hough space), determining the two most likely straight lines, respectively denoted as the first straight line Linerleft (the first boundary line of the femoral side) corresponding to one side of the femur and the second straight line Lineright (the second boundary line of the femoral side) corresponding to the other side of the femur. In this system, each straight line is expressed in polar coordinates as x*cosθ+y*sinθ=ρ, where x and y are the variables corresponding to the polar coordinate equation. Specifically, the first polar coordinate equation for the first straight line Linerleft on one side of the femur is configured as x*cosθ1+y*sinθ1=ρ1, and the second polar coordinate equation for the second straight line Lineright on the other side of the femur is configured as x*cosθ2+y*sinθ2=ρ2. Here, θ1 and θ2 represent the first angle between the perpendicular from the origin of the rectangular coordinate system to the first straight line and the x-axis, and the second angle between the perpendicular from the origin of the rectangular coordinate system to the second straight line and the x-axis, respectively. ρ1 and ρ2 represent the first distance from the origin of the rectangular coordinate system to the first straight line and the second distance from the origin of the rectangular coordinate system to the second straight line, respectively. The first straight line is represented using the first polar coordinate equation; the second straight line is represented using the corresponding second polar coordinate equation.

[0221] Based on the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur, determine the polar coordinate equation corresponding to the femoral midline. Specifically, this includes: based on the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur, determine the first direction vector vec1 of the two straight lines as (-sinθ1, cosθ1) and the second direction vector vec2 as (-sinθ2, cosθ2).

[0222] Normalize the first direction vector vec1 and the second direction vector vec2 respectively to obtain a first normalized direction vector and a second normalized direction vector. Then, perform a dot product between the first normalized direction vector, the first normalized direction vector, the second normalized direction vector, or the direction vector corresponding to the non-zero coordinates of the second direction vector and the modified femoral mask (femurmask) to obtain a first product of the first normalized direction vector and a second product of the second normalized direction vector. If the first product is less than 0, invert the first direction vector or the first normalized direction vector in the direction (add a negative sign); if the second product is less than 0, invert the second direction vector or the second normalized direction vector in the direction (add a negative sign).

[0223] After the above steps, ensure that the directions of the first and second direction vectors of the two straight lines (the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur) are consistent with the direction of the femoral direction vector or the normalized femoral direction vector PCA_femurvec. After determining the directions, add the first direction vector vec1 and the second direction vector vec2 to obtain the bisector direction vector (i.e., the direction vector of the middle bisector corresponding to the first straight line Linerleft corresponding to one side of the femur and the second straight line Lineright corresponding to the other side of the femur). Normalize the bisector direction vector to obtain the normalized bisector direction vector femurvec3, which is (-sinθ3, cosθ3). The polar coordinate equation of the third straight line corresponding to the bisector direction vector or the normalized bisector direction vector femurvec3 is configured as x*cosθ3+y*sinθ3=ρ3. Where θ3 represents the third angle between the perpendicular line from the origin of the rectangular coordinate system to the third line and the x-axis, and ρ3 represents the third distance from the origin of the rectangular coordinate system to the third line. Taking the intersection point (x, y) of the first line Linerleft and the second line Lineright, the third distance ρ3 from the origin of the rectangular coordinate system to the third line can be calculated based on the intersection point (x, y), the bisector direction vector, and the third polar coordinate equation. If the first line Linerleft and the second line Lineright do not intersect, the third distance ρ3 from the origin of the rectangular coordinate system to the third line is determined based on the midpoint (x, y) of the perpendicular segment between the first line Linerleft and the second line Lineright, the bisector direction vector, and the third polar coordinate equation. Therefore, the third polar coordinate equation corresponding to the femoral midline is configured as x*cosθ3 + y*sinθ3 = ρ3.

[0224] This disclosure proposes a method for determining the joint angle between the femur and tibia, comprising: determining the joint angle between the femoral midline and the tibial midline based on the femoral midline and tibial midline corresponding to the X-ray image of the knee joint to be processed; and calculating the joint angle between the femur and tibia based on the joint angle, a third-direction vector corresponding to the femoral midline, and a sixth-direction vector corresponding to the tibial midline. This addresses the current technical problem of not being able to accurately determine the joint angle, thus affecting subsequent dynamic evaluation of the joint angle.

[0225] In this embodiment of the disclosure, annotations can be manually made on the X-ray image of the knee joint to be processed to determine the femoral midline and tibial midline corresponding to the X-ray image of the knee joint to be processed. Alternatively, a processing algorithm can be used to process the X-ray image of the knee joint to be processed to determine the femoral midline and tibial midline corresponding to the X-ray image of the knee joint to be processed.

[0226] In this embodiment of the disclosure, determining the joint angle between the femoral midline and the tibial midline based on the femoral midline and tibial midline corresponding to the X-ray image of the knee joint to be processed includes: establishing a system of polar coordinate equations based on the femoral midline and tibial midline corresponding to the X-ray image of the knee joint to be processed; and using the system of polar coordinate equations to determine the joint angle between the femoral midline and the tibial midline.

[0227] In this embodiment of the disclosure, determining the joint angle point between the femoral midline and the tibial midline using the polar coordinate equations includes: if the polar coordinate equations have a unique solution, determining whether the coordinates corresponding to the unique solution are within the X-ray image of the knee joint to be processed; if they are within the X-ray image of the knee joint to be processed, determining the coordinates corresponding to the unique solution as the joint angle point between the femoral midline and the tibial midline; if they are not within the X-ray image of the knee joint to be processed or the polar coordinate equations have no solution, then based on the femoral midline and the femoral mask boundary image... The first coordinate point corresponding to the joint angle is determined by using the first non-zero coordinate corresponding to the boundary of the femoral mask, the coordinates of the femoral midline, and the center point of the X-ray image of the knee joint to be processed; the second coordinate point corresponding to the joint angle is determined by using the second non-zero coordinate corresponding to the boundary of the tibial mask in the image of the tibial midline and the coordinates of the center point of the X-ray image of the knee joint to be processed; the midpoint between the first coordinate point and the second coordinate point is calculated, and the midpoint is determined as the joint angle between the femoral midline and the tibial midline.

[0228] In this embodiment of the disclosure, determining the first coordinate point corresponding to the joint corner point based on the first non-zero coordinates corresponding to the femoral mask boundary in the femoral midline and femoral mask boundary image, the coordinates of the center point of the femoral midline and the X-ray image of the knee joint to be processed, includes: calculating the first central axis distance from each non-zero coordinate in the first non-zero coordinates to the femoral midline based on the first non-zero coordinates corresponding to the femoral mask boundary in the femoral midline and femoral mask boundary image; determining a set of selected femoral mask boundary non-zero coordinates whose first central axis distance from each non-zero coordinate in the first non-zero coordinates to the femoral midline is within a preset pixel size length; calculating the first center distance from each selected femoral mask boundary non-zero coordinate in the set of selected femoral mask boundary non-zero coordinates to the center point coordinates of the X-ray image of the knee joint to be processed; and configuring the selected femoral mask boundary non-zero coordinate corresponding to the smallest center distance among the multiple first center distances as the first coordinate point.

[0229] In this embodiment of the disclosure, determining the second coordinate point corresponding to the joint corner point based on the second non-zero coordinates corresponding to the tibial mask boundary in the tibial midline and tibial mask boundary image, the tibial midline, and the center point coordinates of the X-ray image of the knee joint to be processed includes: calculating the second central axis distance from each non-zero coordinate in the second non-zero coordinates to the tibial midline based on the second non-zero coordinates corresponding to the tibial mask boundary in the tibial midline and tibial mask boundary image; determining a set of selected tibial mask boundary non-zero coordinates whose distance from each non-zero coordinate in the second non-zero coordinates to the tibial midline is within a preset pixel size length; calculating the second center distance from each selected tibial mask boundary non-zero coordinate in the set of selected tibial mask boundary non-zero coordinates to the center point coordinates of the X-ray image of the knee joint to be processed; and configuring the selected tibial mask boundary non-zero coordinate corresponding to the smallest center distance among multiple second center distances as the second coordinate point.

[0230] In this embodiment of the disclosure, determining the femoral midline includes: extracting the femoral mask boundary image corresponding to the femoral mask in the femoral region mask image; determining the first boundary line and the second boundary line among the boundary lines on both sides of the femur or femoral cortex based on the non-zero coordinates of the femoral mask boundary image; and determining the femoral midline based on the first boundary line and the second boundary line.

[0231] In this embodiment of the disclosure, determining the first boundary line and the second boundary line in the boundary lines on both sides of the femur or femoral cortex includes: determining the first direction vector corresponding to the first boundary line and the second direction vector corresponding to the second boundary line; performing dot products on the first direction vector and the second direction vector with the femoral principal component direction vector corresponding to the femoral region mask image to obtain the first product corresponding to the first direction vector and the second product corresponding to the second direction vector; if the first product is less than a first preset value, inverting the first direction vector in the direction; otherwise, not processing the first direction vector; if the second product is less than a second preset value, inverting the second direction vector in the direction; otherwise, not processing the second direction vector; adding the first direction vector corresponding to the inverted or unprocessed and the second direction vector corresponding to the inverted or unprocessed to obtain the bisecting line direction vector; and determining the femoral midline based on the first boundary line, the second boundary line, and the bisecting line direction vector.

[0232] In this embodiment of the disclosure, determining the femoral midline based on the first boundary line, the second boundary line, and the direction vector of the bisector includes: if the first boundary line and the second boundary line intersect, then determining the femoral midline corresponding to the third polar coordinate equation based on the intersection point, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector; otherwise, determining the femoral midline corresponding to the third polar coordinate equation based on the midpoint of the perpendicular segment between the first boundary line and the second boundary line, the direction vector of the bisector, and the third polar coordinate equation corresponding to the direction vector of the bisector.

[0233] In this embodiment of the disclosure, determining the tibial midline includes: extracting the tibial mask boundary image corresponding to the tibial mask in the tibial region mask image; determining the third boundary line and the fourth boundary line among the boundary lines on both sides of the tibia or tibial cortex based on the tibial mask boundary image and the non-zero coordinates of the tibial mask boundary of the tibial mask boundary image; and determining the tibial midline based on the third boundary line and the fourth boundary line.

[0234] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line and the fourth boundary line includes: determining a fourth direction vector corresponding to the third boundary line and a fifth direction vector corresponding to the fourth boundary line; performing dot products on the fourth direction vector and the fifth direction vector with the tibial principal component direction vector corresponding to the tibial region mask image to obtain a third product corresponding to the fourth direction vector and a fourth product corresponding to the fifth direction vector; if the third product is less than a third preset value, inverting the fourth direction vector in the direction; otherwise, not processing the fourth direction vector; if the fourth product is less than a fourth preset value, inverting the fifth direction vector in the direction; otherwise, not processing the fifth direction vector; adding the inverted or unprocessed fourth direction vector and the inverted or unprocessed fifth direction vector to obtain the bisector direction vector; and determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector.

[0235] In this embodiment of the disclosure, determining the tibial midline based on the third boundary line, the fourth boundary line, and the bisector direction vector includes: if the third boundary line and the fourth boundary line intersect, then determining the tibial midline corresponding to the sixth polar coordinate equation based on the intersection point, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector; otherwise, determining the tibial midline corresponding to the sixth polar coordinate equation based on the midpoint of the perpendicular segment between the third boundary line and the fourth boundary line, the bisector direction vector, and the sixth polar coordinate equation corresponding to the bisector direction vector.

[0236] In this embodiment of the disclosure, the step of calculating the joint angle between the femur and tibia based on the joint angle point, the third direction vector corresponding to the femoral midline, and the sixth direction vector corresponding to the tibial midline includes: calculating the joint angle between the femur and tibia based on the dot product formula using the third direction vector corresponding to the joint angle point, the femoral midline, and the sixth direction vector corresponding to the tibial midline.

[0237] In this embodiment of the disclosure, the step of calculating the joint angle between the femur and tibia based on the dot product formula, using the joint angle point, the third direction vector corresponding to the femoral midline, and the sixth direction vector corresponding to the tibial midline, includes: normalizing the third direction vector corresponding to the femoral midline and the sixth direction vector corresponding to the tibial midline to obtain a third unit direction vector and a sixth unit direction vector; and calculating the joint angle between the femur and tibia based on the dot product formula, using the joint angle point, the third unit direction vector, and the sixth unit direction vector.

[0238] In this embodiment of the disclosure, the method further includes displaying the joint angle between the femur and tibia.

[0239] In this embodiment of the disclosure, displaying the joint angle between the femur and tibia includes: taking the joint angle point as the starting point, drawing a first ray and a second ray along the third direction vector corresponding to the femoral midline and the sixth direction vector corresponding to the tibial midline, respectively; and displaying the first ray, the second ray, and the joint angle between the first ray and the second ray on the X-ray image of the knee joint to be processed.

[0240] In this embodiment of the disclosure, displaying the joint angle between the femur and tibia further includes: determining the femoral stopping point where the first ray intersects the boundary of the X-ray image of the knee joint to be processed and the tibial stopping point where the second ray intersects the boundary of the X-ray image of the knee joint to be processed; and configuring corresponding first arrows and second arrows for the femoral stopping point and the tibial stopping point, respectively.

[0241] This disclosure also proposes a method for dynamic evaluation of joint angles, comprising: determining the joint angle corresponding to each knee X-ray image in the dynamic knee X-ray images to be processed using the joint angle determination method between the femur and tibia as described above; wherein the dynamic knee X-ray images to be processed are a sequence of multiple knee X-ray images taken of the same subject in a leg movement state; and performing dynamic evaluation of the joint angles of the subject based on the joint angles corresponding to each knee X-ray image.

[0242] In this embodiment of the disclosure, the dynamic evaluation of the joint angle of the subject based on the joint angle corresponding to each knee X-ray image includes: determining the range of knee flexion angle and / or extension angle of the subject based on the maximum and minimum angles among the multiple time-series angles corresponding to the joint angles of each knee X-ray image.

[0243] The embodiment of this disclosure describes a dynamic assessment of the joint angles of the subject based on the joint angles corresponding to each knee X-ray image, which further includes: if the subject's knee flexion angle range is less than a preset joint flexion angle range, then it is determined that the subject has a flexion impairment; and / or, if the subject's knee extension angle range is less than a preset joint extension angle range, then it is determined that the subject has an extension impairment.

[0244] This disclosure also proposes a method for determining the trajectory of joint angle changes, comprising: determining the femoral midline and tibial midline corresponding to the initial joint angle among the multiple time-point joint angles; configuring the initial time joint angle corresponding to the initial time joint angle as an axis point; configuring the femoral midline or tibial midline corresponding to the initial time joint angle as a fixed axis line corresponding to the axis point; wherein the coordinate positions of the fixed axis line and the axis point remain unchanged; and rotating the fixed axis line to the tibial midline or femoral midline corresponding to the other time-point angle based on the multiple time-point angles or other time-point angles besides the initial time-point angle, thereby determining the trajectory of joint angle changes. This addresses the current lack of an effective quantitative method for determining the trajectory of knee joint angle changes, which hinders the auxiliary diagnosis of knee joint diseases, assessment of joint degeneration, optimization of athletic performance, guidance of rehabilitation training, and elucidation of joint movement mechanisms.

[0245] For example, the initial joint angle point corresponding to the initial joint angle is configured as the pivot point; the femoral midline corresponding to the initial joint angle point is configured as the fixed axis corresponding to the pivot point; the coordinate positions of the fixed axis and the pivot point are kept unchanged; based on the multiple time angles or other time angles in the multiple time angles except the initial time angle, the fixed axis is rotated to the tibial midline corresponding to other time angles using the pivot point to determine the joint angle change trajectory corresponding to the examinee.

[0246] For example, the initial joint angle point corresponding to the initial joint angle is configured as the pivot point; the tibial midline corresponding to the initial joint angle point is configured as the fixed axis corresponding to the pivot point; the coordinate positions of the fixed axis and the pivot point are kept unchanged; based on the multiple time angles or other time angles among the multiple time angles excluding the initial time angle, the fixed axis is rotated to the femoral midline corresponding to other time angles using the pivot point to determine the joint angle change trajectory corresponding to the examinee.

[0247] The execution entity of the dynamic X-ray image identification and segmentation method can be a dynamic X-ray image identification and segmentation device or system. For example, the dynamic X-ray image identification and segmentation method can be executed by a terminal device, server, or other processing device. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, wearable device, etc. In some possible implementations, the dynamic X-ray image identification and segmentation method can be implemented by a processor calling computer-readable instructions stored in memory.

[0248] Those skilled in the art will understand that in the above-described method for determining and segmenting dynamic X-ray images to be annotated in specific embodiments, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0249] According to one aspect of this disclosure, a dynamic X-ray image identification system is provided, comprising: an extraction unit for extracting each dynamic X-ray image from a preset dynamic X-ray imaging image training set; wherein each dynamic X-ray image includes multiple time-based X-ray images corresponding to the same subject; a calculation unit for calculating multiple image similarities between multiple preset labeled X-ray images in each dynamic X-ray image and multiple X-ray images corresponding to the multiple preset labeled X-ray images; and a determination unit for determining the X-ray image corresponding to the multiple image similarities less than or equal to the preset image similarity as dynamic X-ray images if the multiple image similarities are less than or equal to the preset image similarity. The method includes: an X-ray image to be annotated; or, an electronic device; the electronic device being configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic X-ray image annotation determination method; or, a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described dynamic X-ray image annotation determination method; or, a computer-readable storage medium storing computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described dynamic X-ray image annotation determination method; or, a computer program product, the computer program product being configured with a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the above-described dynamic X-ray image annotation determination method.

[0250] According to one aspect of this disclosure, a segmentation system is provided, comprising: a dynamic X-ray image to be labeled system as described above, used to determine dynamic X-ray images to be labeled in a preset dynamic X-ray imaging image training set; a labeling unit used to label the dynamic X-ray images to be labeled with regions to be segmented, obtaining corresponding mask label images of the regions to be segmented; a training unit used to train a preset segmentation network based on the dynamic X-ray images to be labeled and their corresponding mask label images of the regions to be segmented, to obtain a segmentation model; or, comprising: an electronic device; the electronic device is configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above segmentation method; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above segmentation method; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above segmentation method; or, comprising: a computer program product, the computer program product being configured with a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the above segmentation method.

[0251] According to one aspect of this disclosure, an X-ray camera is provided, comprising: a dynamic X-ray image identification system as described above; or, a dynamic X-ray image identification system as described above; or, comprising: an electronic device; the electronic device being configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke instructions stored in the memory to execute the dynamic X-ray image identification method and / or execute the segmentation method according to any one of claims 3 to 7; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke instructions stored in the memory to execute the dynamic X-ray image identification method and / or execute the segmentation method; or, comprising: a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the dynamic X-ray image identification method and / or implement the segmentation method; or, comprising: a computer program product, the computer program product being configured with a computer program / instructions, which, when executed by a processor, implement the dynamic X-ray image identification method and / or implement the segmentation method. In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above embodiments of the dynamic X-ray image to be labeled determination and segmentation method. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0252] In embodiments of this disclosure, the computer-readable storage medium stores a computer program / instructions and a bit stream thereon, wherein the computer program / instructions, when executed by a processor, implement the method described above to generate the bit stream.

[0253] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for determining a dynamic X-ray image to be annotated, characterized in that, include: Extract each dynamic X-ray image from a preset dynamic X-ray imaging training set; wherein, each dynamic X-ray image includes: multiple X-ray images at different times corresponding to the same subject; Calculate the image similarity between multiple pre-labeled X-ray images and multiple X-ray images corresponding to multiple X-ray images in each example of dynamic X-ray imaging; If the similarity of the multiple images is less than or equal to a preset image similarity, then the X-ray image corresponding to the similarity of the multiple images is determined as the dynamic X-ray image to be labeled.

2. The method for determining dynamic X-ray images to be annotated according to claim 1, characterized in that, The initial preset-labeled X-ray image among the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to the initial moment in each dynamic X-ray image; and / or, the last preset-labeled X-ray image among the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to the last moment in each dynamic X-ray image. And / or, It also includes: if the initial preset-labeled X-ray image corresponding to the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to a non-initial time in each example of dynamic X-ray images, then the similarity between the initial preset-labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the plurality of times preceding the initial preset-labeled X-ray image is further calculated; and / or, It also includes: if the last preset-labeled X-ray image corresponding to the plurality of preset-labeled X-ray images is configured as the X-ray image corresponding to a time other than the last moment in each example of dynamic X-ray images, then the similarity between the last preset-labeled X-ray image in each example of dynamic X-ray images and the plurality of X-ray images corresponding to the times after the last preset-labeled X-ray image is further calculated; and / or, The preset dynamic X-ray image training set is configured as a dynamic knee joint X-ray image training set or a dynamic chest X-ray image training set.

3. A segmentation method, characterized in that, include: Using the dynamic X-ray image identification method as described in any one of claims 1 or 2, dynamic X-ray images to be labeled in a preset dynamic X-ray imaging training set are identified. The dynamic X-ray image to be labeled is used to label the regions to be segmented, and the corresponding mask label images of the regions to be segmented are obtained. Based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, a preset segmentation network is trained to obtain a segmentation model.

4. The segmentation method according to claim 3, characterized in that, The step of training a preset segmentation network based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented to obtain a segmentation model includes: Determine the area of ​​N masked segmentation regions corresponding to the masked label images of the regions to be segmented in the preset medical image training set corresponding to the preset segmentation network. Calculate N ratios between the area of ​​each of the N masked segmentation regions and the total area of ​​the N masked segmentation regions. The loss function weights for each mask segmentation region corresponding to the preset segmentation network are determined based on N different products corresponding to any N-1 mask segmentation regions among the N mask segmentation regions. Based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, the preset segmentation network is trained using the multi-segmentation region weight loss function corresponding to the loss function weight of each mask segmentation region to obtain the segmentation model.

5. The segmentation method according to claim 4, characterized in that, Before training a pre-defined segmentation network to obtain a segmentation model based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, using the multi-segmentation region weight loss function corresponding to the loss function weight of each mask segmentation region, the process includes: obtaining the basic loss function corresponding to each mask segmentation region; multiplying the loss function weight of each mask segmentation region by its corresponding basic loss function and then summing the results to construct the multi-segmentation region weight loss function; and / or, The base loss function corresponding to the weights of the loss function is configured as either the cross-entropy loss function or the weighted cross-entropy loss function.

6. The segmentation method according to claim 4 or 5, characterized in that, Also includes: The boundary loss functions corresponding to each masked segmentation region are summed, or the boundary loss functions of the multi-segmentation regions are summed after multiplying the weights of the loss functions of each masked segmentation region by their respective boundary loss functions. The DICE loss function corresponding to each mask segmentation region is summed, or the DICE loss function of the multi-segmentation region is summed after multiplying the weight of the loss function of each mask segmentation region by its corresponding DICE loss function. A multivariate loss function is constructed based on one or two of the segmentation region weight loss function, multi-segmentation region boundary loss function, or multi-segmentation region DICE loss function. The preset segmentation network is trained using the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, as well as the multivariate loss function.

7. The segmentation method according to any one of claims 3-6, characterized in that, Also includes: If the number of iterations of the preset segmentation network is less than the preset maximum number of iterations, and the loss value between the dynamic X-ray image training set and its corresponding mask label image is less than the preset loss value, and the first average intersection-union ratio between the preset medical image verification set and its corresponding mask label image at the first preset number of iterations is greater than the preset value of the first average intersection-union ratio, then the second average intersection-union ratio between the smallest mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest mask segmentation region in the preset medical image verification set is greater than the preset value of the second average intersection-union ratio, then the preset segmentation network is controlled to stop training; And / or, It also includes: if the number of iterations of the preset segmentation network reaches the preset maximum number of iterations, then controlling the preset segmentation network to stop training; and / or, It also includes: if the number of iterations of the preset segmentation network is less than the preset maximum number of iterations, and the loss value between the training set of the dynamic X-ray image to be labeled and its corresponding mask label image is less than the preset loss value, and the first average intersection-union ratio (IU / R) between the verification set of the dynamic X-ray image to be labeled and its corresponding mask label image at the first preset number of iterations is greater than the preset value of the first average IU / R, and the second average IU / R between the smallest area mask segmentation region in the N mask segmentation regions corresponding to the mask label image and the smallest area mask segmentation region in the verification set of the dynamic X-ray image to be labeled is greater than the preset value of the second average IU / R, then calculate the third average IU / R between the verification set of the dynamic X-ray image to be labeled and its corresponding mask label image within the second preset number of iterations; if the third average IU / R is less than or equal to the preset fluctuation value, then control the preset segmentation network to stop training.

8. A dynamic X-ray image annotation determination system, characterized in that, include: An extraction unit is used to extract each dynamic X-ray image from a preset dynamic X-ray imaging training set; wherein, each dynamic X-ray image includes: multiple X-ray images at different times corresponding to the same subject; The calculation unit is used to calculate the image similarity between multiple preset labeled X-ray images in each example of dynamic X-ray imaging and the multiple X-ray images corresponding to multiple X-ray images. The determining unit is configured to, if the similarity of the plurality of images is less than or equal to a preset image similarity, determine the X-ray imaging image corresponding to the similarity being less than or equal to the preset image similarity as the dynamic X-ray image to be labeled; or, Includes: an electronic device; said electronic device is configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the dynamic X-ray image determination method according to any one of claims 1 to 2; or, Includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the dynamic X-ray image determination method according to any one of claims 1 to 2; or, Includes: a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the dynamic X-ray image determination method according to any one of claims 1 to 2; or, Includes: a computer program product, wherein the computer program product is configured with a computer program / instruction, which, when executed by a processor, implements the dynamic X-ray image to be annotated method according to any one of claims 1 to 2.

9. A segmentation system, characterized in that, include: The dynamic X-ray image identification system as described in claim 8 is used to identify dynamic X-ray images to be labeled in a preset dynamic X-ray imaging image training set. The annotation unit is used to annotate the region to be segmented in the dynamic X-ray image to be annotated, and obtain the corresponding mask label image of the region to be segmented. A training unit is used to train a preset segmentation network based on the dynamic X-ray image to be labeled and its corresponding mask label image of the region to be segmented, to obtain a segmentation model; or, it includes: an electronic device; the electronic device is configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the segmentation method of any one of claims 3 to 7; or, it includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the segmentation method of any one of claims 3 to 7; or, it includes: a computer-readable storage medium storing computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the segmentation method of any one of claims 3 to 7; or, it includes: a computer program product, the computer program product being configured with a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the segmentation method of any one of claims 3 to 7.

10. An X-ray camera, comprising: The dynamic X-ray image identification system as described in claim 8; Alternatively, the dynamic X-ray image determination system as described in claim 9; or, comprising: an electronic device; the electronic device being configured with a processor and a memory for storing processor-executable instructions; wherein the processor is configured to invoke instructions stored in the memory to execute the dynamic X-ray image determination method as described in any one of claims 1 to 2 and / or execute the segmentation method as described in any one of claims 3 to 7; or, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke instructions stored in the memory to execute the dynamic X-ray image determination method as described in any one of claims 1 to 2 and / or execute the segmentation method as described in any one of claims 3 to 7; or, comprising: a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the dynamic X-ray image determination method as described in any one of claims 1 to 2 and / or implement the segmentation method as described in any one of claims 3 to 7; or, comprising: a computer program product. The computer program product is configured with a computer program / instruction, which, when executed by a processor, implements the dynamic X-ray image determination method according to any one of claims 1 to 2 and / or the segmentation method according to any one of claims 3 to 7.

Images