A thoracic vertebra positioning and segmentation method and device and a storage medium

Abstract

Embodiments of the present application disclose a thoracic vertebra positioning and segmentation method, device and storage medium, wherein the thoracic vertebra positioning and segmentation method comprises the steps of: acquiring a spine medical image, segmenting a rib part of the spine medical image by using a rib segmentation model based on deep learning to obtain a rib three-dimensional image; performing projection mapping on the rib three-dimensional image to obtain a corresponding rib two-dimensional image; based on the rib two-dimensional image, positioning the thoracic vertebra according to the correspondence between rib symmetries and / or extracted rib key points to complete the positioning of the thoracic vertebra on the corresponding spine medical image; and segmenting a thoracic vertebra part of the spine medical image on which the thoracic vertebra is positioned by using a thoracic vertebra segmentation model based on deep learning to obtain a thoracic vertebra segmentation result.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method, device, and storage medium for thoracic vertebral localization and segmentation. Background Technology

[0002] In recent years, with the improvement of medical standards, surgeries such as thoracic vertebral fracture removal, foreign body removal, and thoracic vertebral curvature correction have effectively alleviated patients' pain. However, traditional thoracic spine surgery relies on the experience of physicians, and physicians with different experience levels may make different judgments, making it difficult to guarantee consistent results. To solve this problem, it is necessary to develop thoracic spine localization and segmentation methods based on digital image processing technology and deep learning to improve the accuracy of thoracic spine localization and segmentation, thereby better assisting physicians in performing thoracic spine surgery. Summary of the Invention

[0003] The purpose of this application is to provide a method, device, and storage medium for thoracic vertebral localization and segmentation, in order to solve the problem that the existing technology lacks a thoracic vertebral localization and segmentation method based on digital image processing technology and deep learning, resulting in low accuracy of thoracic vertebral localization and segmentation.

[0004] To achieve the above objectives, this application provides a method for thoracic spine localization and segmentation, comprising the steps of: acquiring a spinal medical image, segmenting the rib portion of the spinal medical image using a deep learning-based rib segmentation model, and obtaining a three-dimensional rib image.

[0005] The three-dimensional image of the ribs is projected and mapped to obtain the corresponding two-dimensional image of the ribs. Based on the two-dimensional image of the ribs, the thoracic vertebrae are located according to the symmetry of the ribs and / or the correspondence between the extracted key points of the ribs, so as to complete the thoracic vertebrae location on the corresponding medical image of the spine.

[0006] Using a deep learning-based thoracic spine segmentation model, the thoracic spine portion of the spinal medical image with completed thoracic spine localization is segmented to obtain the thoracic spine segmentation result.

[0007] Optionally, the key points of the ribs include the points where the rib portion connects to the thoracic vertebrae;

[0008] The method for locating the thoracic vertebrae also includes inferring the position of individual vertebrae based on the correspondence between key points of the ribs, so as to avoid positioning errors.

[0009] Optionally, the thoracic vertebra segmentation model includes:

[0010] The encoder and decoder are described. The encoder consists of a transformer structure and a CNN backbone network. The input spinal medical image with thoracic vertebra localization is processed using the transformer structure and the CNN backbone network respectively to obtain two feature maps. The two feature maps are then fused and the decoder generates the thoracic vertebra segmentation result.

[0011] Optionally, the method for constructing the thoracic vertebra segmentation model includes:

[0012] Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing the thoracic vertebrae as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set;

[0013] The initial neural network is trained using the training set, validation set, and / or test set to obtain the thoracic vertebra segmentation model.

[0014] Optionally, the rib segmentation model includes:

[0015] The algorithm consists of 2D and 3D algorithms. The 2D algorithm includes region growing or merging, OTU, thresholding, clustering segmentation, watershed, active contour, U-Net series, Deeplab series, transformer series, ResNet series, and / or FCN. The 3D algorithm includes point cloud segmentation, 3D convolutional neural networks, and / or 3D reinforcement learning networks.

[0016] The rib portion of the input spinal medical image is segmented using the 2D algorithm, and the 3D algorithm is used to reconstruct the segmented result in three dimensions to obtain a three-dimensional image of the rib.

[0017] Optionally, the method for constructing the rib segmentation model includes:

[0018] Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing ribs as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set.

[0019] The initial neural network is trained using the training set, validation set, and / or test set to obtain the rib segmentation model.

[0020] Optionally, the method for projecting and mapping the three-dimensional image of the rib to obtain the corresponding two-dimensional image of the rib includes:

[0021] Obtain the centroid of the three-dimensional image of the rib, and use a plane that passes through the centroid and is parallel to the plane containing the coronal axis and the vertical axis to segment the three-dimensional image of the rib. Project the segmented three-dimensional image of the thoracic vertebrae onto the plane containing the coronal axis and the vertical axis to obtain the two-dimensional image of the rib.

[0022] Optionally, after obtaining the corresponding two-dimensional image of the ribs, but before locating the thoracic vertebrae, the method further includes:

[0023] The two-dimensional image of the ribs is processed to remove burrs and / or protrusions and eliminate interference.

[0024] The image processing method includes morphological operations, contour detection, region labeling, minimum area removal, and / or minimum bounding rectangle method.

[0025] To achieve the above objectives, this application also provides a thoracic vertebra positioning and segmentation device, comprising: a memory; and

[0026] A processor connected to the memory, the processor being configured to:

[0027] A spinal medical image is acquired, and the rib portion of the spinal medical image is segmented using a deep learning-based rib segmentation model to obtain a three-dimensional rib image.

[0028] The three-dimensional image of the ribs is projected and mapped to obtain the corresponding two-dimensional image of the ribs. Based on the two-dimensional image of the ribs, the thoracic vertebrae are located according to the symmetry of the ribs and / or the correspondence between the extracted key points of the ribs, so as to complete the thoracic vertebrae location on the corresponding medical image of the spine.

[0029] Using a deep learning-based thoracic spine segmentation model, the thoracic spine portion of the spinal medical image with completed thoracic spine localization is segmented to obtain the thoracic spine segmentation result.

[0030] To achieve the above objectives, this application also provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a machine, implements the steps of the method described above.

[0031] The embodiments of this application have the following advantages:

[0032] 1. This application provides a method for thoracic vertebra localization and segmentation, comprising the following steps: acquiring a spinal medical image; segmenting the rib portion of the spinal medical image using a deep learning-based rib segmentation model to obtain a 3D rib image; projecting and mapping the 3D rib image to obtain a corresponding 2D rib image; based on the 2D rib image, locating the thoracic vertebra according to rib symmetry and / or the correspondence between extracted rib key points to complete the thoracic vertebra localization on the corresponding spinal medical image; and segmenting the thoracic vertebra portion of the spinal medical image after thoracic vertebra localization using a deep learning-based thoracic vertebra segmentation model to obtain a thoracic vertebra segmentation result.

[0033] The above method utilizes digital image processing technology and deep learning to locate and segment the thoracic vertebrae, improving the accuracy of thoracic vertebrae location and segmentation. Furthermore, since the ribs are relatively fixed in their correspondence with the thoracic vertebrae, using rib segmentation to locate the thoracic vertebrae has high safety.

[0034] 2. Furthermore, the key points of the ribs include the points where the rib portion connects to the thoracic vertebrae; the method for locating the thoracic vertebrae also includes inferring the position of individual vertebrae based on the correspondence between the key points of the ribs to avoid positioning errors.

[0035] Even if thoracic spine diseases include partial rib fractures, ectopic vertebrae, or unsatisfactory rib segmentation, the position of the thoracic spine can still be inferred and located by the relationship between characteristic points, making thoracic spine localization more controllable.

[0036] 3. Further, the thoracic vertebral segmentation model includes:

[0037] The encoder and decoder are described. The encoder consists of a transformer structure and a CNN backbone network. The input spinal medical image with thoracic vertebra localization is processed using the transformer structure and the CNN backbone network respectively to obtain two feature maps. The two feature maps are then fused and the decoder generates the thoracic vertebra segmentation result.

[0038] By combining transformer and UNet, the convolutional part and transformer part in the encoding layer are relatively independent. The reshape operation can make the convolutional part and transformer part form the same W and H feature maps. Finally, they are fused to form new encoded features. In this way, the advantages of transformer and CNN are combined to provide more accurate segmentation results. Attached Figure Description

[0039] To more clearly illustrate the embodiments of this application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a method for thoracic vertebral localization and segmentation provided in this application embodiment;

[0041] Figure 2 A schematic diagram of a rib three-dimensional image for a thoracic vertebra localization and segmentation method provided in an embodiment of this application;

[0042] Figure 3 A two-dimensional image of the ribs is provided as an embodiment of the present application for a method of thoracic vertebra localization and segmentation.

[0043] Figure 4 A schematic diagram illustrating the thoracic vertebral positioning effect of a thoracic vertebral positioning and segmentation method provided in an embodiment of this application;

[0044] Figure 5 A structural diagram of a thoracic vertebra segmentation model for a thoracic vertebra localization and segmentation method provided in an embodiment of this application;

[0045] Figure 6 A schematic diagram of the coding portion of a thoracic vertebra segmentation model for a thoracic vertebra localization and segmentation method provided in an embodiment of this application;

[0046] Figure 7 This is a block diagram of a thoracic vertebra positioning and segmentation device provided in an embodiment of this application. Detailed Implementation

[0047] The following specific embodiments illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0049] One embodiment of this application provides a method for thoracic vertebral localization and segmentation, see reference. Figure 1 , Figure 1The flowchart provided in one embodiment of this application illustrates a method for thoracic vertebral localization and segmentation. It should be understood that the method may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of this application is not limited in this respect.

[0050] In step 101, a spinal medical image is acquired, and the rib portion of the spinal medical image is segmented using a deep learning-based rib segmentation model to obtain a three-dimensional rib image. (Reference) Figure 2 .

[0051] In some embodiments, the rib segmentation model includes:

[0052] The algorithm consists of 2D and 3D algorithms. The 2D algorithm includes region growing or merging, OTU, thresholding, clustering segmentation, watershed, active contour, U-Net series, Deeplab series, transformer series, ResNet series, and / or FCN. The 3D algorithm includes point cloud segmentation, 3D convolutional neural networks, and / or 3D reinforcement learning networks.

[0053] The rib portion of the input spinal medical image is segmented using the 2D algorithm, and the 3D algorithm is used to reconstruct the segmented result in three dimensions to obtain a three-dimensional image of the rib.

[0054] Specifically, since rib segmentation primarily serves to locate the thoracic vertebrae, the segmentation results do not need to be extremely precise; only the main outlines of the ribs need to be segmented. This part can use 2D algorithms based on digital image processing and deep learning, or a combination of both, to complete the segmentation (of spinal medical images). Segmentation algorithms include region growing or merging, OTU, thresholding, clustering segmentation, watershed, active contour, U-Net series, Deeplab series, transformer series, ResNet series, FCN, etc., followed by 3D reconstruction. Similarly, this part can use 3D algorithms, such as traditional point cloud segmentation, 3D convolutional neural networks, 3D reinforcement learning networks, etc., to complete 3D reconstruction after segmentation.

[0055] In some embodiments, the method for constructing the rib segmentation model includes:

[0056] Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing ribs as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set.

[0057] The initial neural network is trained using the training set, validation set, and / or test set to obtain the rib segmentation model.

[0058] Specifically, a spinal medical image dataset is acquired, manually labeled, and labels containing rib segments are extracted as masks to establish a database. Two-dimensional cross-sectional DICOM data is converted into JPG format images (i.e., the spinal medical images used as input to the model in this embodiment), and the labeled files are converted into PNG format images. These images are then shuffled and divided into training, validation, and / or test sets in a 6:2:2 ratio. The network input has one more dimension than a 2D network, consisting of multiple stacked two-dimensional cross-sectional images; the corresponding labeled file is a block image. The initial neural network is then trained using the training, validation, and / or test sets to obtain the rib segmentation model.

[0059] In step 102, the three-dimensional image of the ribs is projected and mapped to obtain the corresponding two-dimensional image of the ribs. Based on the two-dimensional image of the ribs, the thoracic vertebrae are located according to the symmetry of the ribs and / or the correspondence between the extracted key points of the ribs, so as to complete the thoracic vertebrae location on the corresponding spinal medical image.

[0060] Specifically, this embodiment uses the segmentation results of the ribs and the correspondence between the ribs and thoracic vertebrae to accurately locate the position of the thoracic vertebrae, and then further segments the thoracic vertebrae. There is a one-to-one correspondence between the thoracic vertebrae and the ribs; the 12 thoracic vertebrae correspond to the 12 pairs of ribs.

[0061] In some embodiments, a method for projecting and mapping the three-dimensional image of the rib to obtain a corresponding two-dimensional image of the rib includes:

[0062] Obtain the centroid of the three-dimensional image of the rib, and use a plane that passes through the centroid and is parallel to the plane containing the coronal axis and the vertical axis to segment the three-dimensional image of the rib. Project the segmented three-dimensional image of the thoracic vertebrae onto the plane containing the coronal axis and the vertical axis to obtain the two-dimensional image of the rib.

[0063] Specifically, after rib segmentation is completed, the centroid of the 3D segmented portion (3D rib image) is determined, and the segmented portion is divided using a plane parallel to the XZ plane (the plane containing the coronal and vertical axes) and passing through the centroid, preserving the thoracic lateral ribs. The thoracic lateral ribs are then projected onto the XZ plane (related to the initial plane) to obtain the 2D rib segmentation result (2D rib image).

[0064] In some embodiments, after obtaining the corresponding two-dimensional image of the rib, before locating the thoracic vertebrae, the method further includes:

[0065] The two-dimensional image of the ribs is processed to remove burrs and / or protrusions and eliminate interference.

[0066] The image processing method includes morphological operations, contour detection, region labeling, minimum area removal, and / or minimum bounding rectangle method.

[0067] Specifically, digital image processing is performed on the 2D rib segmentation results, including morphological operations, contour detection, region labeling, minimum area removal, and minimum bounding rectangle, to remove burrs and protrusions, while also eliminating osteophytes and other interferences.

[0068] Subsequently, key points of the ribs are extracted, and each thoracic vertebra is accurately located based on rib symmetry or the correspondence of key points, thus completing the thoracic vertebral localization. (Reference) Figure 3 .

[0069] In some embodiments, the rib key points include the points where the rib portion connects to the thoracic vertebrae; the method for locating the thoracic vertebrae further includes inferring the position of individual vertebrae based on the correspondence between the rib key points to avoid positioning errors.

[0070] Specifically, rib key points can not only locate individual vertebrae, but also, based on the correspondence between rib key points, reasonably infer the approximate location of individual vertebrae such as fractures, displacements, and injuries, thereby avoiding incorrect localization and segmentation. Reference for thoracic vertebral localization and positioning frame effect. Figure 4 .

[0071] In step 103, a deep learning-based thoracic spine segmentation model is used to segment the thoracic spine portion of the spinal medical image after thoracic spine localization, thereby obtaining the thoracic spine segmentation result.

[0072] Specifically, existing medical image segmentation algorithms based on the Transformer family generally use pure Transformers (such as the ViT algorithm) or are combined with CNNs. The CNN-integrated part typically uses the feature maps of the backbone network directly in the encoding layer for Transformer layer operations (such as the SOTR algorithm). The combination of Transformer and UNet generally involves using a Transformer in the last layer of the UNet encoding layer (such as TransUNet) or modifying the Attention component of the Transformer.

[0073] In some embodiments, the thoracic vertebra segmentation model includes:

[0074] The encoder and decoder are described. The encoder consists of a transformer structure and a CNN backbone network. The input spinal medical image with thoracic vertebra localization is processed using the transformer structure and the CNN backbone network respectively to obtain two feature maps. The two feature maps are then fused and the decoder generates the thoracic vertebra segmentation result.

[0075] Specifically, this thoracic spine segmentation model is mainly based on the idea of ​​transformer + CNN backbone network + UNet, establishing a new network model TC-UNet. (Reference) Figure 5 .

[0076] In the encoding layer, the transformer (such as ViT, SOTR, TransUNet, UNETR, etc.) and the backbone network (such as ResNet, FCN, VGG, SENet, Darknet, EfficientNet, etc.) are processed separately, resulting in separate feature maps. The result from the Transformer layer undergoes a reshap operation to ensure that the width and height of the feature map are the same (W, H are the same), while the channel C can be the same or different. A concat operation is performed on each of the separately obtained feature maps, followed by conv+BN+reLU to fuse the transformer feature map with the corresponding CNN feature map, forming a new feature map. For the specific operation flow of the encoding layer, refer to [link to documentation]. Figure 6 .

[0077] In the decoding layer, the traditional structure of 3D-UNet is basically used. The feature map of the decoding layer is short-connected with the upsampling result, and then conv+BN+reLU operations are performed. Finally, the decoding part uses 1*1 convolution to generate the final thoracic spine segmentation result.

[0078] In some embodiments, the method for constructing the thoracic vertebra segmentation model includes:

[0079] Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing the thoracic vertebrae as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set;

[0080] The initial neural network is trained using the training set, validation set, and / or test set to obtain the thoracic vertebra segmentation model.

[0081] Specifically, a spinal medical image dataset is acquired, manually labeled, and labels containing the thoracic vertebrae are extracted as masks to establish a database. Two-dimensional cross-sectional DICOM data is converted into JPG format images (i.e., the spinal medical images used as input models in this embodiment), and the labeled files are converted into PNG format images. These images are then shuffled and divided into training, validation, and / or test sets in a 6:2:2 ratio. The network input has one more dimension than a 2D network, consisting of multiple stacked two-dimensional cross-sectional images; the corresponding labeled file is a block image. The initial neural network is then trained using the training, validation, and / or test sets to obtain the thoracic vertebra segmentation model.

[0082] The above method utilizes digital image processing technology and deep learning to locate and segment the thoracic vertebrae, improving the accuracy of thoracic vertebrae location and segmentation. Furthermore, since the ribs are relatively fixed in their correspondence with the thoracic vertebrae, using rib segmentation to locate the thoracic vertebrae has high safety.

[0083] Due to factors such as thoracic spine lesions, the degree of curvature of the thoracic spine, prostheses, the degree of fractures, or osteophyte interference, directly locating and identifying the thoracic spine will pose more risks. However, since the ribs are relatively fixed in their correspondence with the thoracic spine, locating the thoracic spine by dividing the ribs has higher safety.

[0084] The method of using rib segmentation to locate the thoracic vertebrae can not only locate the thoracic vertebrae, but also accurately classify and locate the entire spine or a non-full spine. That is, the part above the thoracic vertebrae is the cervical vertebrae, and the part below the thoracic vertebrae is the lumbar vertebrae.

[0085] The entire process of thoracic spine localization and segmentation generally uses a "white box" or "black box" approach. However, this application uses a combination of digital image processing and deep learning, which can be called a "gray box" approach. Even if the thoracic spine disease involves some rib fractures, ectopic ribs, or other conditions where the segmentation of some ribs is not ideal, the position of the thoracic spine can still be inferred and localized by the relationship between feature points. The controllability of thoracic spine localization is higher.

[0086] The thoracic spine segmentation network model uses a combination of transformer and UNet. The convolutional part and transformer part in the encoding layer are relatively independent. The reshape operation is used to make the convolutional part and transformer part form the same W and H feature maps. Finally, they are fused to form new encoded features. In this way, the advantages of transformer and CNN are combined to provide more accurate segmentation results.

[0087] Figure 7A block diagram of a thoracic vertebra positioning and segmentation device provided in an embodiment of this application. The device includes:

[0088] The memory 201; and the processor 202 connected to the memory 201, the processor 202 being configured to: acquire a spinal medical image, segment the rib portion of the spinal medical image using a deep learning-based rib segmentation model, and obtain a three-dimensional rib image.

[0089] The three-dimensional image of the ribs is projected and mapped to obtain the corresponding two-dimensional image of the ribs. Based on the two-dimensional image of the ribs, the thoracic vertebrae are located according to the symmetry of the ribs and / or the correspondence between the extracted key points of the ribs, so as to complete the thoracic vertebrae location on the corresponding medical image of the spine.

[0090] Using a deep learning-based thoracic spine segmentation model, the thoracic spine portion of the spinal medical image with completed thoracic spine localization is segmented to obtain the thoracic spine segmentation result.

[0091] In some embodiments, the processor 202 is further configured such that the thoracic vertebra segmentation model includes:

[0092] The encoder and decoder are described. The encoder consists of a transformer structure and a CNN backbone network. The input spinal medical image with thoracic vertebra localization is processed using the transformer structure and the CNN backbone network respectively to obtain two feature maps. The two feature maps are then fused and the decoder generates the thoracic vertebra segmentation result.

[0093] In some embodiments, the processor 202 is further configured to: construct the thoracic vertebra segmentation model by comprising:

[0094] Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing the thoracic vertebrae as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set;

[0095] The initial neural network is trained using the training set, validation set, and / or test set to obtain the thoracic vertebra segmentation model.

[0096] In some embodiments, the processor 202 is further configured such that the rib segmentation model includes:

[0097] The algorithm consists of 2D and 3D algorithms. The 2D algorithm includes region growing or merging, OTU, thresholding, clustering segmentation, watershed, active contour, U-Net series, Deeplab series, transformer series, ResNet series, and / or FCN. The 3D algorithm includes point cloud segmentation, 3D convolutional neural networks, and / or 3D reinforcement learning networks.

[0098] The rib portion of the input spinal medical image is segmented using the 2D algorithm, and the 3D algorithm is used to reconstruct the segmented result in three dimensions to obtain a three-dimensional image of the rib.

[0099] In some embodiments, the processor 202 is further configured to: construct the rib segmentation model by comprising:

[0100] Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing ribs as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set.

[0101] The initial neural network is trained using the training set, validation set, and / or test set to obtain the rib segmentation model.

[0102] In some embodiments, the processor 202 is further configured to: perform projection mapping on the three-dimensional image of the rib to obtain a corresponding two-dimensional image of the rib, including:

[0103] Obtain the centroid of the three-dimensional image of the rib, and use a plane that passes through the centroid and is parallel to the plane containing the coronal axis and the vertical axis to segment the three-dimensional image of the rib. Project the segmented three-dimensional image of the thoracic vertebrae onto the plane containing the coronal axis and the vertical axis to obtain the two-dimensional image of the rib.

[0104] In some embodiments, the processor 202 is further configured to: after obtaining the corresponding two-dimensional image of the rib, and before locating the thoracic vertebrae, further include:

[0105] The two-dimensional image of the ribs is processed to remove burrs and / or protrusions and eliminate interference.

[0106] The image processing method includes morphological operations, contour detection, region labeling, minimum area removal, and / or minimum bounding rectangle method.

[0107] In some embodiments, the processor 202 is further configured such that the rib key point includes the point where the rib portion connects to the thoracic vertebrae;

[0108] The method for locating the thoracic vertebrae also includes inferring the position of individual vertebrae based on the correspondence between key points of the ribs, so as to avoid positioning errors.

[0109] For specific implementation methods, please refer to the aforementioned method embodiments, which will not be repeated here.

[0110] This application may be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this application.

[0111] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0112] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0113] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0114] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0115] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0116] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0118] Note that, unless otherwise explicitly stated, all features disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by alternative features for achieving the same, equivalent, or similar purpose. Therefore, unless explicitly stated otherwise, each disclosed feature is merely one example of a set of equivalent or similar features. Where used, "further," "preferably," "even further," and "more preferably" are simple starting points for describing another embodiment based on the foregoing embodiments, the combination of which with the foregoing embodiments constitutes the complete configuration of another embodiment. Any combination of several "further," "preferably," "even further," or "more preferably" settings following the same embodiment constitutes yet another embodiment.

[0119] Although this application has been described in detail above with general descriptions and specific embodiments, some modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of this application fall within the scope of protection claimed in this application.

Claims

1. A method for locating and segmenting the thoracic vertebrae, characterized in that, Includes the following steps: A spinal medical image is acquired, and the rib portion of the spinal medical image is segmented using a deep learning-based rib segmentation model to obtain a three-dimensional rib image. The three-dimensional image of the ribs is projected and mapped to obtain a corresponding two-dimensional image of the ribs. Based on the two-dimensional image of the ribs, the thoracic vertebrae are located according to the symmetry of the ribs and / or the correspondence between the extracted key points of the ribs, so as to complete the thoracic vertebrae location on the corresponding spinal medical image. The method of projecting and mapping the three-dimensional image of the ribs to obtain the corresponding two-dimensional image of the ribs includes: obtaining the centroid of the three-dimensional image of the ribs; dividing the three-dimensional image of the ribs using a plane that passes through the centroid and is parallel to the plane containing the coronal axis and the vertical axis; and projecting and mapping the divided thoracic vertebrae side three-dimensional image towards the plane containing the coronal axis and the vertical axis to obtain the two-dimensional image of the ribs. Using a deep learning-based thoracic spine segmentation model, the thoracic spine portion of the spinal medical image with completed thoracic spine localization is segmented to obtain the thoracic spine segmentation result.

2. The method for thoracic vertebral localization and segmentation according to claim 1, characterized in that, The key points of the ribs include the points where the rib portion connects to the thoracic vertebrae. The method for locating the thoracic vertebrae also includes inferring the position of individual vertebrae based on the correspondence between key points of the ribs, so as to avoid positioning errors.

3. The method for thoracic vertebral localization and segmentation according to claim 1, characterized in that, The thoracic vertebral segmentation model includes: The encoder and decoder are described. The encoder consists of a transformer structure and a CNN backbone network. The input spinal medical image with thoracic vertebra localization is processed using the transformer structure and the CNN backbone network respectively to obtain two feature maps. The two feature maps are then fused and the decoder generates the thoracic vertebra segmentation result.

4. The method for thoracic vertebral localization and segmentation according to claim 1 or 3, characterized in that, The method for constructing the thoracic vertebra segmentation model includes: Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing the thoracic vertebrae as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set; The initial neural network is trained using the training set, validation set, and / or test set to obtain the thoracic vertebra segmentation model.

5. The method for thoracic vertebral localization and segmentation according to claim 1, characterized in that, The rib segmentation model includes: The algorithm consists of 2D and 3D algorithms. The 2D algorithm includes region growing or merging, OTU, thresholding, clustering segmentation, watershed, active contour, U-Net series, Deeplab series, transformer series, ResNet series, and / or FCN. The 3D algorithm includes point cloud segmentation, 3D convolutional neural networks, and / or 3D reinforcement learning networks. The rib portion of the input spinal medical image is segmented using the 2D algorithm, and the 3D algorithm is used to reconstruct the segmented result in three dimensions to obtain a three-dimensional image of the rib.

6. The method for thoracic vertebral localization and segmentation according to claim 1 or 5, characterized in that, The method for constructing the rib segmentation model includes: Obtain a spinal medical image dataset for model building, manually annotate it, extract labels containing ribs as annotation files, and divide the spinal medical image data converted to image format and the corresponding annotation files converted to image format into training set, validation set and / or test set. The initial neural network is trained using the training set, validation set, and / or test set to obtain the rib segmentation model.

7. The method for thoracic vertebral localization and segmentation according to claim 1, characterized in that, After obtaining the corresponding two-dimensional image of the ribs, before locating the thoracic vertebrae, the following steps are also included: The two-dimensional image of the ribs is processed to remove burrs and / or protrusions and eliminate interference. The image processing method includes morphological operations, contour detection, region labeling, minimum area removal, and / or minimum bounding rectangle method.

8. A thoracic vertebra positioning and segmentation device, characterized in that, include: Memory; as well as A processor connected to the memory, the processor being configured to: A spinal medical image is acquired, and the rib portion of the spinal medical image is segmented using a deep learning-based rib segmentation model to obtain a three-dimensional rib image. The three-dimensional image of the ribs is projected and mapped to obtain the corresponding two-dimensional image of the ribs. Based on the two-dimensional image of the ribs, the thoracic vertebrae are located according to the symmetry of the ribs and / or the correspondence between the extracted key points of the ribs, so as to complete the thoracic vertebrae location on the corresponding medical image of the spine. Using a deep learning-based thoracic spine segmentation model, the thoracic spine portion of the spinal medical image after thoracic spine localization is segmented to obtain the thoracic spine segmentation result; The method for projecting and mapping the three-dimensional image of the rib to obtain the corresponding two-dimensional image of the rib includes: obtaining the centroid of the three-dimensional image of the rib; dividing the three-dimensional image of the rib using a plane that passes through the centroid and is parallel to the plane containing the coronal axis and the vertical axis; and projecting and mapping the divided thoracic vertebral side three-dimensional image towards the plane containing the coronal axis and the vertical axis to obtain the two-dimensional image of the rib.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a machine, it implements the steps of the method as described in any one of claims 1 to 7.

Images