Joint image segmentation method and device, electronic equipment and storage medium

Abstract

The application provides a kind of joint image segmentation method, device, electronic equipment and storage medium, it is related to medical image processing field, the method comprises: obtaining the DICOM image of knee joint;DICOM image is input into knee joint image segmentation network, obtains the knee joint segmentation image of knee joint image segmentation network output;Wherein, knee joint image segmentation network includes image segmentation model, edge segmentation network and image fusion module;Image segmentation model is used to carry out knee joint segmentation processing to DICOM image, obtains first segmentation image;Edge segmentation network is used to carry out edge prediction to first segmentation image, obtains second segmentation image;Image fusion module is used to fuse first segmentation image and second segmentation image, obtains knee joint segmentation image.The joint image segmentation method, device, electronic equipment and storage medium provided in the application can improve the accuracy of knee joint image segmentation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for joint image segmentation. Background Technology

[0002] With the development of artificial intelligence (AI) technology, deep learning-based knee joint segmentation algorithms have gradually been applied in the medical field. However, due to the diversity of knee joint diseases, such as fractures and arthritis, existing deep learning-based knee joint segmentation algorithms have low accuracy in knee joint segmentation, especially at the knee joint edges, where segmentation deviations are relatively large.

[0003] Improving the accuracy of knee joint segmentation has become an urgent problem to be solved in the industry. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention provides a joint image segmentation method, apparatus, electronic device, and storage medium.

[0005] In a first aspect, the present invention provides a joint image segmentation method, comprising:

[0006] Acquiring medical digital imaging and communication DICOM images of the knee joint;

[0007] The DICOM image is input into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network;

[0008] The knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module.

[0009] The image segmentation model is used to perform knee joint segmentation processing on the DICOM image to obtain a first segmented image;

[0010] The edge segmentation network is used to predict edges in the first segmented image to obtain a second segmented image;

[0011] The image fusion module is used to fuse the first segmented image and the second segmented image to obtain the knee joint segmented image.

[0012] Optionally, according to a joint image segmentation method provided by the present invention, the edge segmentation network includes a feature extraction unit, a feature fusion unit, and an edge prediction unit;

[0013] The feature extraction unit includes an edge convolutional layer, as well as a batch normalization and activation merging layer;

[0014] The feature fusion unit includes a 1×1 convolutional layer, a pooling layer, and a batch normalization and activation merging layer;

[0015] The edge prediction unit includes a 1×1 convolutional layer as well as a batch normalization and activation merging layer.

[0016] Optionally, in a joint image segmentation method provided by the present invention, the knee joint image segmentation network is trained based on a target loss function, which is constructed based on a segmentation loss function and an edge loss function.

[0017] Optionally, in a joint image segmentation method provided by the present invention, the target loss function is obtained by weighting the segmentation loss function and the edge loss function, and the weighting factors corresponding to the segmentation loss function and the edge loss function are determined by the lesion type of the knee joint.

[0018] Optionally, in a joint image segmentation method provided by the present invention, the segmentation loss function is any one of the following loss functions or a combination of the following loss functions:

[0019] Cross-entropy loss function, Focal loss function, and Dice loss function.

[0020] Optionally, in a joint image segmentation method provided by the present invention, the edge loss function is any one of the following loss functions or a combination of the following loss functions:

[0021] Cross-entropy loss function, distance difference loss function, distance variance loss function, and edge area difference loss function.

[0022] In a second aspect, the present invention also provides a joint image segmentation apparatus, comprising:

[0023] The acquisition module is used to acquire medical digital imaging and communication DICOM images of the knee joint;

[0024] The segmentation module is used to input the DICOM image into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network;

[0025] The knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module.

[0026] The image segmentation model is used to perform knee joint segmentation processing on the DICOM image to obtain a first segmented image;

[0027] The edge segmentation network is used to predict edges in the first segmented image to obtain a second segmented image;

[0028] The image fusion module is used to fuse the first segmented image and the second segmented image to obtain the knee joint segmented image.

[0029] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the joint image segmentation method as described in the first aspect.

[0030] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the joint image segmentation method as described in the first aspect.

[0031] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the joint image segmentation method as described in the first aspect.

[0032] The joint image segmentation method, apparatus, electronic device, and storage medium provided by this invention obtain a segmented knee joint image output by inputting a DICOM image of the knee joint into a knee joint image segmentation network. Since the knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module, after obtaining a first segmented image based on the image segmentation model, the edge segmentation network is used to predict the edges of the first segmented image to obtain a second segmented image. Finally, the first and second segmented images are fused to obtain the knee joint segmented image, thereby improving the accuracy of knee joint image segmentation. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0034] Figure 1 This is one of the flowcharts illustrating the joint image segmentation method provided by the present invention;

[0035] Figure 2 This is a schematic diagram of the edge segmentation network provided by the present invention;

[0036] Figure 3 This is the second flowchart of the joint image segmentation method provided by the present invention;

[0037] Figure 4 This is a schematic diagram of the structure of the knee joint image segmentation network provided by the present invention;

[0038] Figure 5 This is a schematic diagram of the joint image segmentation device provided by the present invention;

[0039] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0041] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, the first object can be one or more.

[0042] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to include the plural forms.

[0043] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0044] The following is in conjunction with the appendix Figure 1-6 The joint image segmentation method, apparatus, electronic device, and storage medium provided by the present invention will be described by way of example.

[0045] Figure 1 This is one of the flowcharts illustrating the joint image segmentation method provided by the present invention, such as... Figure 1 As shown, the method includes:

[0046] Step 100: Acquire digital imaging and communications in medicine (DICOM) images of the knee joint;

[0047] Step 110: Input the DICOM image into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network;

[0048] The knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module.

[0049] The image segmentation model is used to perform knee joint segmentation processing on the DICOM image to obtain a first segmented image;

[0050] The edge segmentation network is used to predict edges in the first segmented image to obtain a second segmented image;

[0051] The image fusion module is used to fuse the first segmented image and the second segmented image to obtain the knee joint segmented image.

[0052] It should be noted that the execution subject of the joint image segmentation method provided in this embodiment of the invention can be any electronic device or computer device. The following uses a joint image segmentation system for performing joint image segmentation as an example to describe the technical solution of this embodiment of the invention in detail.

[0053] Specifically, to overcome the shortcomings of existing deep learning-based knee joint segmentation algorithms, which suffer from low accuracy due to the diversity of knee joint diseases, such as fractures and arthritis, especially at the knee joint edges where segmentation deviations are significant, this invention inputs the DICOM image of the knee joint into a knee joint image segmentation network to obtain a segmented knee joint image output by the network. Since the knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module, after obtaining the first segmented image based on the image segmentation model, the edge segmentation network performs edge prediction on the first segmented image to obtain the second segmented image. Finally, the first and second segmented images are fused to obtain the knee joint segmented image, thereby improving the accuracy of knee joint image segmentation.

[0054] Optionally, medical digital imaging and communication (DICOM) images of the knee joint, such as CT image data of the knee joint, can be acquired.

[0055] Optionally, considering the correspondence between different slices of CT images, in this embodiment of the invention, 2.5D CT image data of the knee joint can be input into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network.

[0056] Optionally, to improve the accuracy of knee joint segmentation, the knee joint image segmentation network provided in this embodiment of the invention includes an image segmentation model, an edge segmentation network, and an image fusion module. The image segmentation model can be used to perform knee joint segmentation processing on the input DICOM image to obtain a first segmented image. The edge segmentation network can be used to perform edge prediction on the first segmented image to obtain a second segmented image. The image fusion module can be used to fuse the first segmented image and the second segmented image to obtain the final knee joint segmented image.

[0057] It is understood that, in the embodiments of the present invention, after the medical digital imaging and communication (DICOM) image of the knee joint is input into the knee joint image segmentation network, the input DICOM image is first processed by the image segmentation model to obtain a first segmented image. Then, the edge segmentation network performs edge prediction on the first segmented image to obtain a second segmented image. Finally, the image fusion module fuses the first segmented image and the second segmented image to obtain the final knee joint segmented image.

[0058] It is understood that, in the embodiments of the present invention, the image segmentation model and the edge segmentation network can be any deep learning network suitable for performing joint image segmentation, and the embodiments of the present invention do not specifically limit them.

[0059] Optionally, in this embodiment of the invention, the image segmentation model can use current mainstream segmentation algorithms, including but not limited to UNet and its variants, FCN (Fully Convolutional Network) series, DeepLab series, Transformer series, etc.

[0060] It is understood that, in this embodiment of the invention, before the first segmented image enters the edge segmentation network, it is necessary to extract the knee joint edge contour in the first segmented image to obtain a knee joint edge contour image corresponding to the first segmented image, and then input the knee joint edge contour image into the edge segmentation network.

[0061] Optionally, in this embodiment of the invention, the knee joint edge contour extraction can use an eight-neighbor or four-neighbor method, or use the corresponding functions of the corresponding image algorithm libraries in Python / C++ such as OpenCV and Pillow. This embodiment of the invention does not specifically limit this.

[0062] Understandably, edge segmentation networks can predict the knee joint edges in the first segmented image obtained based on an image segmentation model.

[0063] It is understood that, unlike existing joint segmentation methods, the joint image segmentation method provided in this embodiment of the invention, after obtaining a first segmented image based on an image segmentation model, also performs edge prediction on the first segmented image based on an edge segmentation network to obtain a second segmented image, and then fuses the first segmented image and the second segmented image to obtain the final knee joint segmented image, thereby improving the accuracy of knee joint image segmentation.

[0064] The joint image segmentation method provided by this invention inputs the DICOM image of the knee joint into a knee joint image segmentation network to obtain a segmented knee joint image output by the knee joint image segmentation network. Since the knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module, after obtaining the first segmented image based on the image segmentation model, the edge segmentation network is used to predict the edges of the first segmented image to obtain the second segmented image. Finally, the first segmented image and the second segmented image are fused to obtain the knee joint segmented image, which improves the accuracy of knee joint image segmentation.

[0065] Optionally, the edge segmentation network includes a feature extraction unit, a feature fusion unit, and an edge prediction unit;

[0066] The feature extraction unit includes an edge convolutional layer, as well as a batch normalization and activation merging layer;

[0067] The feature fusion unit includes a 1×1 convolutional layer, a pooling layer, and a batch normalization and activation merging layer;

[0068] The edge prediction unit includes a 1×1 convolutional layer as well as a batch normalization and activation merging layer.

[0069] Specifically, in this embodiment of the invention, the edge segmentation network includes a feature extraction unit, a feature fusion unit, and an edge prediction unit. After the first segmented image is input into the edge segmentation network, it first passes through the feature extraction unit for feature extraction, then enters the feature fusion unit to achieve feature fusion, and further enters the edge prediction unit to achieve edge prediction of the knee joint, thereby obtaining the second segmented image.

[0070] Optionally, in embodiments of the present invention, the feature extraction unit may include an edge convolutional layer (Edge Conv) and a batch normalization and activation merging layer (BN+GELU).

[0071] Optionally, in embodiments of the present invention, the feature fusion unit may include a 1×1 convolutional layer (1×1 Conv), a pooling layer, and a batch normalization and activation merging layer (BN+GELU).

[0072] Optionally, in embodiments of the present invention, the edge prediction unit may include a 1×1 convolutional layer (1×1 Conv) and a batch normalization and activation merging layer (BN+GELU).

[0073] Optionally, Figure 2 This is a schematic diagram of the edge segmentation network provided by the present invention, as shown below. Figure 2 As shown, the edge segmentation network includes a feature extraction unit, a feature fusion unit, and an edge prediction unit. The feature extraction unit includes an Edge Conv layer and a BN+GELU layer. The feature fusion unit includes a 1×1 Conv layer, an average or max pooling layer (Avg / Max Pooling), and a BN+GELU layer. The edge prediction unit includes a 1×1 Conv layer and a BN+GELU layer.

[0074] The joint image segmentation method provided by this invention uses an edge segmentation network including a feature extraction unit, a feature fusion unit, and an edge prediction unit. After the first segmented image is input into the edge segmentation network, it first passes through the feature extraction unit for feature extraction, then enters the feature fusion unit to achieve feature fusion, and further enters the edge prediction unit to achieve edge prediction of the knee joint, thereby obtaining a second segmented image, which can improve the accuracy of knee joint image segmentation.

[0075] Optionally, the knee joint image segmentation network is trained based on a target loss function, which is constructed based on a segmentation loss function and an edge loss function.

[0076] Specifically, in this embodiment of the invention, the knee joint image segmentation network is trained based on a target loss function, which is constructed based on a segmentation loss function and an edge loss function.

[0077] It is understood that, in this embodiment of the invention, before using the knee joint image segmentation network for knee joint segmentation processing, the knee joint image segmentation network can be trained based on the target loss function constructed from the segmentation loss function and the edge loss function.

[0078] Optionally, the segmentation loss function is any one of the following loss functions or a combination of the following loss functions:

[0079] Cross-entropy loss function, Focal loss function, and Dice loss function.

[0080] Optionally, the choice of which loss function or a combination of the above loss functions to use as the segmentation loss function can be determined based on the type of knee joint lesion.

[0081] Optionally, the edge loss function is any one of the following loss functions or a combination of the following loss functions:

[0082] Cross-entropy loss function, distance difference loss function, distance variance loss function, and edge area difference loss function.

[0083] Optionally, the choice of which of the above loss functions or a combination of multiple loss functions to use as the marginal loss function can be determined based on the type of knee joint lesion.

[0084] Optionally, in this embodiment of the invention, the types of lesions of the knee joint may include, but are not limited to, fractures and arthritis, and this embodiment of the invention does not specifically limit them.

[0085] Optionally, the target loss function is obtained by weighting the segmentation loss function and the edge loss function, and the weighting factors corresponding to the segmentation loss function and the edge loss function are determined by the lesion type of the knee joint.

[0086] Specifically, in this embodiment of the invention, the target loss function for training the knee joint image segmentation network is obtained by weighting the segmentation loss function and the edge loss function, and the weighting factors corresponding to the segmentation loss function and the edge loss function are determined by the lesion type of the knee joint.

[0087] It is understood that, in this embodiment of the invention, the segmentation loss function and the edge loss function can be weighted to obtain the target loss function. The weighting factors corresponding to the segmentation loss function and the edge loss function can be determined according to the lesion type of the knee joint. For example, the weighting factors corresponding to the segmentation loss function and the edge loss function are 0.5 respectively.

[0088] It is understandable that assigning different weights to the segmentation loss function and the edge loss function in different diseases and segmentation tasks, and fusing the loss functions, can make the knee joint image segmentation network trained based on the fused loss function more accurate in segmenting the knee joint.

[0089] Figure 3 This is the second flowchart illustrating the joint image segmentation method provided by the present invention, as shown below. Figure 3 As shown, the method includes: first, inputting a DICOM image of the knee joint or its downsampled image into an image segmentation model to obtain the segmentation result output by the image segmentation model; then, extracting the edge contour of the knee joint from the segmentation result to obtain the edge contour image of the knee joint; further inputting the edge contour image of the knee joint into an edge segmentation network to obtain the prediction result output by the edge segmentation network; and finally, fusing the segmentation result output by the image segmentation model and the prediction result output by the edge segmentation network to obtain the final knee joint segmentation result.

[0090] Optionally, in this embodiment of the invention, whether it is a label image or a segmentation result predicted by a segmentation model, the edge contour of the knee joint can be obtained using an eight-neighbor or four-neighbor method, or by using corresponding functions of image algorithm libraries such as OpenCV and Pillow in Python / C++. This embodiment of the invention does not impose specific limitations on this.

[0091] It is understood that the embodiments of the present invention mainly include two parts: image segmentation and edge prediction. Unlike existing segmentation algorithms, the joint image segmentation method provided in the embodiments of the present invention needs to extract the knee joint edges after obtaining the segmentation results, and then predict the knee joint edges again, which can make the knee joint segmentation results more accurate.

[0092] Optionally, Figure 4 This is a schematic diagram of the knee joint image segmentation network provided by the present invention, as shown below. Figure 4 As shown, the 2.5D knee joint image is first input into the image segmentation model (Seg Model) to obtain the first segmentation result. Then, the knee joint edge contour is extracted from the first segmentation result (Get Contours). The extracted knee joint edge contour is input into the edge segmentation network (Edge Net) to obtain the second segmentation result. The first segmentation result and the second segmentation result are then fused (Concat operation). The fused result is then convolved, batch normalized, and activated to finally obtain the segmented image of the knee joint.

[0093] It is understood that the joint image segmentation method provided in this embodiment of the invention can accurately locate, segment and identify the knee joint by utilizing deep learning and segmentation algorithms, which can assist doctors in planning knee joint surgery.

[0094] The joint image segmentation method provided by this invention inputs the DICOM image of the knee joint into a knee joint image segmentation network to obtain a segmented knee joint image output by the knee joint image segmentation network. Since the knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module, after obtaining the first segmented image based on the image segmentation model, the edge segmentation network is used to predict the edges of the first segmented image to obtain the second segmented image. Finally, the first segmented image and the second segmented image are fused to obtain the knee joint segmented image, which improves the accuracy of knee joint image segmentation.

[0095] The joint image segmentation apparatus provided by the present invention will be described below. The joint image segmentation apparatus described below can be referred to in correspondence with the joint image segmentation method described above.

[0096] Figure 5 This is a schematic diagram of the joint image segmentation device provided by the present invention, as shown below. Figure 5As shown, the device includes: an acquisition module 510 and a segmentation module 520; wherein:

[0097] The acquisition module 510 is used to acquire medical digital imaging and communication (DICOM) images of the knee joint;

[0098] The segmentation module 520 is used to input the DICOM image into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network;

[0099] The knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module.

[0100] The image segmentation model is used to perform knee joint segmentation processing on the DICOM image to obtain a first segmented image;

[0101] The edge segmentation network is used to predict edges in the first segmented image to obtain a second segmented image;

[0102] The image fusion module is used to fuse the first segmented image and the second segmented image to obtain the knee joint segmented image.

[0103] The joint image segmentation device provided by this invention obtains a knee joint segmented image by inputting a DICOM image of the knee joint into a knee joint image segmentation network. Since the knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module, after obtaining a first segmented image based on the image segmentation model, the edge segmentation network is used to predict the edges of the first segmented image to obtain a second segmented image. Finally, the first segmented image and the second segmented image are fused to obtain the knee joint segmented image, thereby improving the accuracy of knee joint image segmentation.

[0104] Optionally, the edge segmentation network includes a feature extraction unit, a feature fusion unit, and an edge prediction unit;

[0105] The feature extraction unit includes an edge convolutional layer, as well as a batch normalization and activation merging layer;

[0106] The feature fusion unit includes a 1×1 convolutional layer, a pooling layer, and a batch normalization and activation merging layer;

[0107] The edge prediction unit includes a 1×1 convolutional layer as well as a batch normalization and activation merging layer.

[0108] Optionally, the knee joint image segmentation network is trained based on a target loss function, which is constructed based on a segmentation loss function and an edge loss function.

[0109] Optionally, the target loss function is obtained by weighting the segmentation loss function and the edge loss function, and the weighting factors corresponding to the segmentation loss function and the edge loss function are determined by the lesion type of the knee joint.

[0110] Optionally, the segmentation loss function is any one of the following loss functions or a combination of the following loss functions:

[0111] Cross-entropy loss function, Focal loss function, and Dice loss function.

[0112] Optionally, the edge loss function is any one of the following loss functions or a combination of the following loss functions:

[0113] Cross-entropy loss function, distance difference loss function, distance variance loss function, and edge area difference loss function.

[0114] The joint image segmentation device provided by this invention obtains a knee joint segmented image by inputting a DICOM image of the knee joint into a knee joint image segmentation network. Since the knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module, after obtaining a first segmented image based on the image segmentation model, the edge segmentation network is used to predict the edges of the first segmented image to obtain a second segmented image. Finally, the first segmented image and the second segmented image are fused to obtain the knee joint segmented image, thereby improving the accuracy of knee joint image segmentation.

[0115] It should be noted that the joint image segmentation device provided in this embodiment of the invention can implement all the method steps implemented in the above-mentioned joint image segmentation method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0116] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the joint image segmentation methods provided by the above methods, which include:

[0117] Acquiring medical digital imaging and communication DICOM images of the knee joint;

[0118] The DICOM image is input into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network;

[0119] The knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module.

[0120] The image segmentation model is used to perform knee joint segmentation processing on the DICOM image to obtain a first segmented image;

[0121] The edge segmentation network is used to predict edges in the first segmented image to obtain a second segmented image;

[0122] The image fusion module is used to combine the first segmented image and the second segmented image. Figure 5 The images are fused together to obtain the segmented knee joint image.

[0123] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially, or rather...

[0124] The portion that contributes to the existing technology or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including if

[0125] Dry instructions are used to cause a computer device (which can be a personal computer, a server, or...) to...

[0126] Network devices, etc., execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage media include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0127] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, which, when executed by a computer, enable the computer to...

[0128] The method is capable of performing the joint image segmentation method provided by the above methods, the method including: acquiring medical digital imaging and communication DICOM images of the knee joint;

[0129] The DICOM image is input into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network;

[0130] The knee joint image segmentation network includes an image segmentation model and an edge segmentation network.

[0131] Network and image fusion module;

[0132] The image segmentation model described in section 5 is used to perform knee joint segmentation processing on the DICOM image.

[0133] Obtain the first segmented image;

[0134] The edge segmentation network is used to predict edges in the first segmented image to obtain a second segmented image;

[0135] The image fusion module is used to fuse the first segmented image and the second segmented image to obtain the knee joint segmented image.

[0136] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the joint image segmentation methods provided above, the method comprising:

[0137] Acquiring medical digital imaging and communication DICOM images of the knee joint;

[0138] The DICOM image is input into the knee joint image segmentation network to obtain the knee joint segmentation image output by the knee joint image segmentation network;

[0139] The knee joint image segmentation network includes an image segmentation model, an edge segmentation network, and an image fusion module.

[0140] The image segmentation model is used to perform knee joint segmentation processing on the DICOM image to obtain a first segmented image;

[0141] The edge segmentation network is used to predict edges in the first segmented image to obtain a second segmented image;

[0142] The image fusion module is used to fuse the first segmented image and the second segmented image to obtain the knee joint segmented image.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of segmenting an image of a joint, the method comprising: The method comprises the following steps: obtaining a medical digital imaging and communication (DICOM) image of a knee joint; inputting the DICOM image into a knee joint image segmentation network to obtain a knee joint segmentation image output by the knee joint image segmentation network; wherein the knee joint image segmentation network comprises an image segmentation model, an edge segmentation network and an image fusion module; the image segmentation model is used for knee joint segmentation processing on the DICOM image to obtain a first segmentation image; the edge segmentation network is used for edge prediction on the first segmentation image to obtain a second segmentation image; the image fusion module is used for fusing the first segmentation image and the second segmentation image to obtain the knee joint segmentation image.

2. The joint image segmentation method of claim 1, wherein, The edge segmentation network comprises a feature extraction unit, a feature fusion unit and an edge prediction unit; wherein the feature extraction unit comprises an edge convolution layer and a batch normalization and activation merging layer; the feature fusion unit comprises a 1x1 convolution layer, a pooling layer and a batch normalization and activation merging layer; the edge prediction unit comprises a 1x1 convolution layer and a batch normalization and activation merging layer.

3. The joint image segmentation method of claim 1, wherein, The knee joint image segmentation network is obtained by training based on a target loss function, and the target loss function is constructed based on a segmentation loss function and an edge loss function.

4. The joint image segmentation method of claim 3, wherein, The target loss function is obtained by weighting the segmentation loss function and the edge loss function, and the weighting factors corresponding to the segmentation loss function and the edge loss function are determined by the knee joint lesion type.

5. The joint image segmentation method of claim 3 or 4, characterized in that, The segmentation loss function is any one of the following loss functions or a combination of multiple loss functions: cross-entropy loss function, focal loss function and Dice loss function.

6. The joint image segmentation method of claim 3 or 4, wherein, The edge loss function is any one of the following loss functions or a combination of multiple loss functions: cross-entropy loss function, distance difference loss function, distance variance loss function and edge area difference loss function.

7. A joint image segmentation apparatus characterized by comprising: The method comprises the following steps: an acquisition module for acquiring a medical digital imaging and communication (DICOM) image of a knee joint; a segmentation module for inputting the DICOM image into a knee joint image segmentation network to obtain a knee joint segmentation image output by the knee joint image segmentation network; wherein the knee joint image segmentation network comprises an image segmentation model, an edge segmentation network and an image fusion module; the image segmentation model is used for knee joint segmentation processing on the DICOM image to obtain a first segmentation image; the edge segmentation network is used for edge prediction on the first segmentation image to obtain a second segmentation image; the image fusion module is used for fusing the first segmentation image and the second segmentation image to obtain the knee joint segmentation image.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the joint image segmentation method of any one of claims 1-6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the joint image segmentation method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the joint image segmentation method of any one of claims 1-6.

Images