Bone structure segmentation method and device based on iconography, medium and program product

Through the imaging bone structure segmentation method, the spinal area is first positioned, and then the spinal bone is corroded and finely segmented, which solves the problem of difficulty and accuracy of spinal CT images segmentation, achieving high-precision and low-difficulty spinal segmentation, which is suitable for clinical applications.

CN119006495APending Publication Date: 2024-11-22VISUAL3D MEDICAL TECH DEV CO LTD

Patent Information

Application Number
CN202411146399.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art is difficult to achieve high accuracy and low difficulty when segmenting spinal CT images, especially due to the complex spinal structure and the cumbersome annotation, learning and training process under multiple views, resulting in low segmentation accuracy of neural network machine learning algorithms.

Method used

The imaging-based bone structure segmentation method is used to initially locate the spinal area, cut and corrode the vertebrae, and finely segment the spinal bones using the centroid and Gaussian distribution, reducing the difficulty of segmentation and improving accuracy.

Benefits of technology

It realizes high-precision segmentation of spinal images, reduces the requirements for spinal sequence imaging, is suitable for large-scale clinical applications, and overcomes the complexity problem during spinal and tumor imaging segmentation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006495A_ABST
    Figure CN119006495A_ABST
Patent Text Reader

Abstract

The invention provides a bone structure segmentation method and device based on iconography, a medium and a program product, and relates to the field of intelligent medical treatment. The method comprises the following steps: acquiring a three-dimensional CT image of a spine; segmenting the three-dimensional CT image to obtain an ROI (Region of Interest) image containing any one or more of a cervical vertebra, a thoracic vertebra and a lumbar vertebra; cutting the ROI region image to obtain a mask image containing ROI vertebrae; and segmenting the mask image by using a segmentation network to obtain an image containing the ROI vertebrae. According to the method, preliminary positioning is performed according to the lesion area of the patient in a targeted manner, and then fine segmentation is performed on different areas in a targeted manner, so that the segmentation difficulty is reduced while the segmentation precision is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical treatment, and more specifically, to a bone structure segmentation method, device, medium and program product based on imaging. Background Art

[0002] The commonly used imaging examinations for clinical evaluation of spinal diseases include plain X-rays, CT-myelography, magnetic resonance imaging, bone scans, CT, etc. In addition, some auxiliary examinations can also be used for the diagnosis of spinal diseases such as bone density measurement, intervertebral disc angiography, facet joint injections, etc. The use of imaging examinations should clarify the correlation between spinal morphological abnormalities and clinical conditions in order to make a correct diagnosis.

[0003] Among them, CT is the best examination for evaluating bone anatomy, and its multiple tomographic images can provide images in mutually perpendicular planes. However, CT examinations have ionizing radiation, and multiple CT examinations pose a safety hazard to patients. At present, when combining medical image processing to guide surgery, image segmentation is involved. However, due to the large area occupied by the spine, if the tumor is directly segmented according to tumor diseases, the segmentation accuracy is often difficult to meet clinical needs; in addition, due to the complex structure of the spine, which includes multiple vertebrae, pedicles, lamina, vertebral foramina, transverse processes, spinous processes, etc., it is quite difficult to segment spinal CT images for spinal surgery. Therefore, there is an urgent need for a segmentation method that can reduce the difficulty of segmentation and improve the accuracy of segmentation. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a bone structure segmentation method, device, medium and program product based on imaging; the method of the present invention first performs preliminary positioning according to the patient's lesion area and then performs fine segmentation on different areas in a targeted manner, thereby ensuring segmentation accuracy while reducing segmentation difficulty.

[0005] The first aspect of the present application discloses a bone structure segmentation method based on imaging, comprising:

[0006] 101, obtain three-dimensional CT images of the spine;

[0007] 102, segmenting the 3D CT image to obtain an ROI region image including any one or more of the cervical spine, thoracic spine, and lumbar spine;

[0008] 103, cropping the ROI region image to obtain a mask image including the ROI vertebrae;

[0009] 104. Segment the mask image using a segmentation network to obtain an image containing the ROI vertebra.

[0010] Optionally, the cropping method in 104 includes: finding the maximum and minimum values ​​of the mask in the mask image in different directions in the RAS three-dimensional space, cropping the areas smaller than the minimum value and larger than the maximum value in different directions of the three-dimensional CT image, and obtaining a mask image containing the ROI vertebra;

[0011] Optionally, the ROI area also includes: sacrum and coccyx.

[0012] In some embodiments, the mask image containing the ROI vertebra is at least a binary mask image. When it is a binary mask image, the ROI area is any one of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a ternary mask image, the ROI area is any two of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a quaternary mask image, the ROI area is any three of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a quinary mask image, the ROI area is any four of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a hexavalue mask image, the ROI area includes the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx.

[0013] In some embodiments, after 103 and before 104, the method further includes: cropping a single vertebra in the mask image containing the ROI vertebra to obtain a mask image in which adjacent single vertebrae are not connected; locating the centroid of the single vertebra in the mask image in which adjacent single vertebrae are not connected to obtain an image with a centroid; and segmenting the image with the centroid with the single centroid as the center to obtain a single vertebra image corresponding to the single centroid;

[0014] Optionally, the method of cropping the single vertebra in the mask image containing the ROI vertebra includes: gradually eroding from the edges of the single vertebra to the center in equal proportion;

[0015] Optionally, the volume of the single vertebral body after trimming in 102 is smaller than the volume of the single vertebral body before trimming;

[0016] Preferably, the ratio of the volume of the single vertebral body after trimming to the volume of the single vertebral body before trimming is 1 / 10-9 / 10, preferably 1 / 5-4 / 5, preferably 1 / 2.

[0017] In some embodiments, the method for locating the centroid of a single vertebra in a mask image where adjacent single vertebrae are not connected comprises: using a connected domain algorithm to identify at least one separate region in the N-value mask image where adjacent single vertebrae are not connected; locating the centroid of the separate region as the centroid of the single vertebra; the separate region is the region where the single vertebrae are located;

[0018] Optionally, the centroids in the image with centroids are sorted according to the z-axis coordinates of the centroids.

[0019] In some embodiments, the method for acquiring a single vertebral image corresponding to the single centroid includes: acquiring a 3-dimensional Gaussian distribution with a×b×c fixed around the single centroid; acquiring a 3-dimensional image with a×b×c fixed around the centroid; and segmenting from the 3-dimensional image, based on the 3-dimensional Gaussian distribution centered on the centroid, obtaining a single vertebral image corresponding to the single centroid around the Gaussian distribution.

[0020] In some embodiments, after 104, the method further includes 105, identifying the category of a single vertebra in the vertebral image based on the position of the center of mass; optionally, when the category of a single vertebra cannot be determined according to the method in 105, the method further includes: inputting the single vertebral image into a classifier to identify the category of the single vertebra.

[0021] The second aspect of the present application discloses a computer device, the device comprising: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method described in the first aspect of the present application.

[0022] The third aspect of the present application discloses a computer-readable storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect of the present application are implemented.

[0023] A fourth aspect of the present application discloses a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in the first aspect of the present application are implemented.

[0024] This application has the following beneficial effects:

[0025] 1. This application aims at the problem that the accuracy of the neural network model directly depends on the amount of sample data in the training set when the spine is segmented by a neural network-based machine learning algorithm. In order to achieve the segmentation effect of each component unit of the spine, each component unit of the spine in the sample data needs to be marked with a category label. However, in actual operation, it requires annotation, learning and training under multiple views of the spine. The process is cumbersome and difficult, which leads to the current situation that it is difficult to implement image segmentation using a neural network-based machine learning algorithm. An innovative bone structure segmentation method based on imaging is disclosed, which can solve the above problems well. It first makes a preliminary positioning of the spine area in the three-dimensional image, and narrows the area to ROI areas such as the cervical spine, thoracic spine or lumbar spine according to the patient's lesion area, so as to reduce the influence of the area outside the spine area on the subsequent segmentation, complete the preliminary positioning; and then perform the subsequent fine segmentation work. Since the excessive impurities at the input end of the first step have a greater impact on the fine segmentation of the subsequent second step during segmentation, this application proposes to perform preliminary positioning in the first step first, reduce impurities and further reduce the impact on the subsequent fine segmentation.

[0026] 2. The present application also discloses a centroid positioning step between the preliminary positioning and fine segmentation steps, which is performed by corroding a single vertebra in the mask image of the ROI vertebra so that adjacent vertebrae are not connected, and then using a connected domain algorithm to determine the different regions where the single vertebrae are located after corrosion, and locating the centroid of the single vertebrae in different regions as the centroid of the entire vertebra, and using a Gaussian distribution centered on the centroid to restrict the network to only segment the vertebrae at the centroid, thereby achieving fine segmentation of the spine and further obtaining the entire spine; the above method performs a bone segmentation process through a three-step method of coarse positioning, centroid positioning, and fine segmentation, thereby reducing the difficulty of segmentation while ensuring segmentation accuracy.

[0027] 3. The present application overcomes the difficulties faced in segmenting spinal images and tumor images. Unlike the direct segmentation of tumor images, the spine is a special chain structure composed of a large number of similar units, and the spinal structure is complex, including the spine and muscles. The spine is composed of arranged vertebrae, and each vertebra includes a vertebral body, pedicle, lamina, vertebral foramen, transverse process, spinous process, accessory process, superior articular process, and mastoid process. Therefore, it is quite difficult to segment spinal CT images. There are large individual differences in spinal images. Segmentation requires annotation, learning and training under multiple views of the spine. The process is cumbersome and difficult, which leads to the difficulty of implementing image segmentation based on neural network machine learning algorithms. The existing technology is difficult to achieve accurate positioning and segmentation of each unit that makes up the spine.

[0028] 4. The spinal images acquired by the segmentation method in this application have high accuracy and low requirements for spinal sequence images, which is conducive to large-scale clinical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 is a schematic diagram of a method flow chart provided by the first aspect of an embodiment of the present invention;

[0031] Figure 2 is a schematic diagram of a bone structure segmentation system based on imaging provided by an embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of a computer device provided by an embodiment of the present invention;

[0033] Figure 4 is a schematic diagram of the architecture of an exemplary computing device provided by an embodiment of the present invention;

[0034] Figure 5 is a schematic diagram of a storage medium provided by an embodiment of the present invention;

[0035] Figure 6 This is an operation flow chart of a segmentation method provided by an embodiment of the present invention, which first roughly locates the ROI region of the spine region and then uses a segmentation network for fine segmentation (a two-step method); from left to right, the first stage of rough segmentation, cropping, and the second stage of segmentation and restoration of the original CT image are respectively performed;

[0036] Figure 7 Schematic diagram of the rough segmentation process (segmenting the input three-dimensional CT image into a three-dimensional binary mask image) in the two-step method provided by an embodiment of the present invention; A is the input three-dimensional CT image, and B is the output three-dimensional binary mask image; A is outputted as result B after passing through the segrenet segmentation network;

[0037] Figure 8 : is a schematic diagram of the fine segmentation process in the two-step method provided by an embodiment of the present invention; A is a three-dimensional CT image obtained by coarse segmentation of the input, and B is an output three-dimensional binary mask image of the required multi-valued labels segmented by a multi-valued segmentation network;

[0038] Fig. 9 In the three-step method provided in the embodiment of the present invention, the input edge-cropped 3D CT image is divided into a 3D binary mask image containing only vertebrae and adjacent vertebrae are not connected; A is the input edge-cropped 3D CT image, and B is the output 3D binary mask image in which half of the volume is eliminated and only contains vertebrae; A is passed through the segrenet segmentation network to output the result B;

[0039] Fig.10 1. It is a flow chart of a method for acquiring a single vertebral image corresponding to a single centroid in the centroid positioning step of the three-step method provided in an embodiment of the present invention; A is an input 3D Gaussian distribution with a fixed size of 224x224x128 with the vertebral body as the center, B is an input cropped 3D CT image with a fixed size of 224x224x128 with the vertebral body as the center, and C is an output 3D binary mask image of the vertebral body; A and B are segmented by a segrenet network to output the result C;

[0040] Fig.11 This is a flow chart of a method for identifying the category of a single vertebra in the vertebral image based on the position of the center of mass in the process of locating the center of mass in the three-step method provided by an embodiment of the present invention; on the left is an input cropped 3D CT image of 224x224x128 fixed with the vertebra as the center, which is output as the position label class of the vertebra after passing through the DenseNet classification network

[0041] Fig.12 It is an operation flow chart of the three-step segmentation method (three-step method, first coarse positioning, then positioning the center of mass, and finally fine segmentation) after adding the step of locating the center of mass provided by an embodiment of the present invention; a is coarsely segmented and output as b, b is cropped and its corresponding three-dimensional CT image c is output, c is segmented and output as d, the center of mass of each vertebra in d is identified and output e, the single vertebra is cropped with the center of mass as the center to obtain the image f column, the image f column is ID identified to identify the category class of the single vertebra (image g column), and the image g column is synthesized to obtain the image h. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0043] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0045] Figure 1 1 is a flow chart of a bone structure segmentation method based on imaging provided by an embodiment of the present invention. Specifically, the method comprises the following steps:

[0046] 101: Obtain three-dimensional CT images of the spine;

[0047] In some embodiments, since the input image size will be much larger than the spinal region during spinal segmentation, coupled with the high similarity between spinal vertebrae, it is difficult to directly segment individual vertebrae in each spine. In order to improve the segmentation accuracy, the present application adopts a method of first coarsely locating the spinal region and then finely segmenting each unit of the spinal column to achieve accurate segmentation of the spine. Specifically, the spinal image in the three-dimensional CT image is binary, that is, the differences between the components of the spine are not distinguished in the coarse positioning step, and the entire spine is regarded as the same label. After segmentation, a "retain the maximum connected domain" post-processing operation is added; the "retain the maximum connected domain" post-processing can help reduce interference such as the skull. The purpose of coarse positioning: to find the maximum and minimum values ​​through the binary region. In the coarse positioning step, the image has impurities that have a great impact on the cropping of the fine segmentation step; however, the image lacks a certain part in the coarse positioning step, which has a relatively small impact on the cropping of the fine segmentation step.

[0048] Optionally, the ROI area also includes: sacrum and coccyx;

[0049] 102: Segment the 3D CT image to obtain an ROI region image including any one or more of the cervical spine, thoracic spine, and lumbar spine;

[0050] In some embodiments, the method of cropping the single vertebra in the mask image containing the ROI vertebra includes: gradually eroding from the edges of the single vertebra to the center in equal proportion;

[0051] Optionally, the volume of the single vertebral body after trimming in 102 is smaller than the volume of the single vertebral body before trimming;

[0052] Preferably, the ratio of the volume of a single vertebra after clipping to the volume of a single vertebra before clipping is: 1 / 10-9 / 10, preferably 1 / 5-4 / 5, preferably 1 / 2. The reason for clipping a single vertebra is: if the original size of the vertebra is predicted, there is a high probability that the vertebrae will be connected. If they are connected, the next step using the connected domain algorithm cannot distinguish them. The clipping corrosion in the present application changes the size of the vertebra to prevent the connection between the vertebrae. Each vertebra is corroded by the corrosion algorithm to retain half of the volume (because it is impossible for two half vertebrae to be connected together, the half vertebrae are not corroded). By predicting a spine with only half the volume, it is ensured that the vertebrae are not connected. Then, each vertebra is obtained by the connected domain algorithm, and the center of mass of the spine is used as the center of mass of the entire spine. Although the preferred ratio is 1 / 2, in the actual operation process, the clipping ratio is adjusted according to the shape of the subject's spine, as long as it can ensure that adjacent vertebrae are not connected. In some embodiments, the clipping process is trained using a corrosion algorithm in the training stage to obtain the area of ​​a single vertebra.

[0053] 103: cropping the ROI region image to obtain a mask image including the ROI vertebrae;

[0054] Optionally, the ROI area also includes: sacrum and coccyx.

[0055] In some embodiments, the mask image containing the ROI vertebra is at least a binary mask image. When it is a binary mask image, the ROI area is any one of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a ternary mask image, the ROI area is any two of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a quaternary mask image, the ROI area is any three of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a quinary mask image, the ROI area is any four of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a hexagram mask image, the ROI area includes the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx. Figure 6 As shown, it is a flow chart of the segmentation method operation of first roughly locating the ROI area of ​​the spine area and then finely segmenting it using the segmentation network (two-step method) provided by an embodiment of the present invention; from left to right, the first stage of rough segmentation, cropping, and the second stage of segmentation and restoration of the original CT image are respectively performed; wherein, Figure 7 Schematic diagram of the rough segmentation process in the two-step method (segmenting the input three-dimensional CT image into a three-dimensional binary mask image); Figure 8 Schematic diagram of the fine segmentation process in the two-step method;

[0056] In some embodiments, after 103 and before 104, the method further includes: cropping a single vertebra in the mask image containing the ROI vertebra to obtain a mask image in which adjacent single vertebrae are not connected; locating the centroid of the single vertebra in the mask image in which adjacent single vertebrae are not connected to obtain an image with a centroid; and segmenting the image with the centroid with the single centroid as the center to obtain a single vertebra image corresponding to the single centroid;

[0057] In some embodiments, the method for locating the centroid of a single vertebra in a mask image where adjacent single vertebrae are not connected comprises: using a connected domain algorithm to identify at least one separate region in the N-value mask image where adjacent single vertebrae are not connected; locating the centroid of the separate region as the centroid of the single vertebra; the separate region is the region where the single vertebrae are located;

[0058] Optionally, the centroids in the image with centroids are sorted according to the z-axis coordinates of the centroids.

[0059] In some embodiments, the method for acquiring a single vertebral image corresponding to the single centroid includes: acquiring a 3-dimensional Gaussian distribution with a×b×c fixed around the single centroid; acquiring a 3-dimensional image with a×b×c fixed around the centroid; and segmenting from the 3-dimensional image, based on the 3-dimensional Gaussian distribution centered on the centroid, obtaining a single vertebral image corresponding to the single centroid around the Gaussian distribution. Among them, the value of a / b / c needs to meet the following conditions: Since a / b / c is a hyperparameter, it needs to be a multiple of 16; and it needs to be used to crop a single spine, so the cropping area must include the complete spine. The larger the area, the greater the interference. Therefore, it should be as small as possible while ensuring coverage of the entire vertebra. It is preferably 224×224×128, but its specific value is variable; the value of a / b / c needs to meet the following conditions: Since a / b / c is a hyperparameter, it needs to be a multiple of 16; and if the cropping is too large, the upper and lower vertebrae may be placed in this area, preferably 224×224×128, but its specific value is variable; the 3D Gaussian and the 3D image are input together to use the Gaussian distribution for positioning, and only the spine mask of the area around the Gaussian distribution is segmented.

[0060] 104, segmenting the mask image using a segmentation network to obtain an image containing the ROI vertebrae;

[0061] In some embodiments, the cropping method in 104 includes: finding the maximum and minimum values ​​of the mask in the mask image in different directions in the RAS three-dimensional space, cropping the areas smaller than the minimum value and larger than the maximum value in different directions of the three-dimensional CT image, and obtaining a mask image containing the ROI vertebra;

[0062] In some embodiments, after 104, the method further includes 105, identifying the category of a single vertebra in the vertebral image based on the position of the centroid, and the ID corresponds to the vertebral number one by one, such as 1 for thoracic 1 and 2 for thoracic 2, which is predetermined; optionally, when the category of a single vertebra cannot be determined according to the method in 105, the method further includes: inputting the single vertebral image into a classifier to identify the category of the single vertebra. Figure 9-11 As shown, it is a specific flow chart for adding the centroid positioning step, in which Fig. 9 In the three-step method, the input edge-cropped 3D CT image is divided into a 3D binary mask image containing only vertebrae and adjacent vertebrae are not connected. Fig.10 A schematic diagram of a method for acquiring a single vertebral image corresponding to a single centroid in the centroid positioning step is shown below: Fig.11 A flow chart of a method for identifying the category of a single vertebra in the vertebral image based on the position of the centroid in the process of locating the centroid in the three-step method; Fig.12 It is an operation flow chart of the three-step segmentation method (three-step method, first coarse positioning, then centroid positioning, and finally fine segmentation) after adding the centroid positioning step provided by an embodiment of the present invention;

[0063] Optionally, the training method of the classifier includes: inputting a single vertebral image of a training set sample into the classifier to obtain a predicted classification result, comparing it with the classification label, and optimizing according to the comparison result to obtain a constructed classifier; the classifier includes any one or more of the following: decision tree, logistic regression, naive Bayes, neural network; preferably DenseNet in the neural network.

[0064] Specifically, the minimum three-dimensional box where the binary spine region of the spine image in 101 is located is input into the step of finely segmenting each unit of the spine. At this time, each vertebra represents a label, and after the segmentation is complete, a spine that distinguishes each vertebra is formed;

[0065] The principle of the minimum three-dimensional box is as follows:

[0066] inds={(i,j,k)∣data[i][j][k]=1}

[0067] x1=min(inds[0])

[0068] x2=max(inds[0])

[0069] y1=min(inds[1])

[0070] y2=max(inds[1])

[0071] z1=min(inds[1])

[0072] z2=max(inds[1])

[0073] crop_data=data[x1:x2,y1:y2,z1:z2]

[0074] Here, x1, x2, y1, y2, z1, and z2 are integer indices that define the start and end positions of the cropping region. Specifically: x1 and x2 define the start and end indices for cropping along the first dimension (usually corresponding to the length). y1 and y2 define the start and end indices for cropping along the second dimension (usually corresponding to the width). z1 and z2 define the start and end indices for cropping along the third dimension (usually corresponding to the height).

[0075] In some embodiments, in some embodiments, after 105, the method further includes 106, synthesizing at least two single vertebrae based on the categories of the single vertebrae to obtain a synthesized image containing the target vertebra.

[0076] In some embodiments, after acquiring the three-dimensional CT image and before segmenting the spinal image in 101, the method further includes: preprocessing the three-dimensional CT image to obtain a unified three-dimensional CT image; the preprocessing includes: rotating the three-dimensional CT image to the RAS direction, randomly flipping, normalizing, and randomly shearing the image to a size of 128×128×128.

[0077] Figure 3 is a schematic diagram of a computer device provided by an embodiment of the present invention, such as Figure 3 As shown, the device may include: one or more processors, and one or more memories; wherein the memories store computer-readable codes, and when the computer-readable codes are run by the one or more processors, the method described above may be executed.

[0078] The processor in this embodiment can be an integrated circuit chip with signal processing capabilities. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, operations and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X86 architecture or an ARM architecture.

[0079] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the disclosed embodiments are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0080] For example, the method or device according to the embodiment of the present disclosure may also be implemented by Figure 4 The architecture of the computing device 3000 shown in FIG. Figure 4 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as ROM 3030 or hard disk 3070, may store various data or files used for processing and / or communication of the method provided by the present disclosure and program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of a computing device are shown.

[0081] The embodiment of the present invention also provides a computer-readable storage medium, such as Figure 5As shown, it is a schematic diagram of a storage medium provided by an embodiment of the present invention, and a computer readable instruction 4010 is stored on the computer storage medium 4020. When the computer readable instruction 4010 is executed by the processor, the method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer readable storage medium in the embodiment of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0082] The embodiment of the present disclosure also provides a computer program product or system, which implements the steps of the above method when the computer program is executed by a processor;

[0083] The present embodiment discloses a bone structure segmentation system based on imaging, such as Figure 2 As shown, the system comprises:

[0084] An acquisition module 201 is used to acquire a three-dimensional CT image of the spine;

[0085] The ROI region segmentation module 202 is used to segment the 3D CT image to obtain an ROI region image containing any one or more of the cervical spine, thoracic spine, and lumbar spine;

[0086] A cropping module 203 is used to crop the ROI area image to obtain a mask image containing the ROI vertebrae;

[0087] The vertebra segmentation module 204 segments the mask image using a segmentation network to obtain an image containing the ROI vertebra.

[0088] In some embodiments, after the cropping module and before the vertebra segmentation module, the system further includes: an erosion processing module, which is used to crop the single vertebra in the mask image containing the ROI vertebra to obtain a mask image in which adjacent single vertebrae are not connected; locate the centroid of the single vertebra in the mask image in which adjacent single vertebrae are not connected to obtain an image with a centroid; and segment the image with the centroid with the single centroid as the center to obtain a single vertebra image corresponding to the single centroid;

[0089] In some embodiments, after the vertebra segmentation module, the system further comprises: a vertebra identification module for identifying the category of a single vertebra in the vertebra image based on the position of the centroid; optionally, when the category of a single vertebra cannot be determined according to the method of the vertebra identification module, the method further comprises: inputting the single vertebra image into a classifier to identify the category of the single vertebra;

[0090] In some embodiments, after the vertebra identification module, the system further includes: a vertebra synthesis module for synthesizing at least two single vertebrae based on the categories of the single vertebrae to obtain a synthesized image containing the ROI vertebrae.

[0091] Example: Segmentation based on the patient's preoperative CT images

[0092] The deep learning method swinunetr was used to segment the spine, and after training, good segmentation results were obtained; Swin UNetR combined the context modeling ability of Swin Transformer and the pixel-level prediction ability of U-Net to improve the performance of semantic segmentation tasks.

[0093] 1) Labeling Dataset

[0094] Mainstream deep learning-based segmentation methods require a large amount of data sets for training to generalize the model capabilities. First, ask doctors to mark the cervical, thoracic, and lumbar vertebrae.

[0095] 2) Deep learning algorithm training

[0096] At present, the U-shaped structure of medical image segmentation in the CNN field has achieved the most advanced performance benchmark, such as the U-Net model. Swinunetr is also based on the U-shaped structure design, including a decoder and an encoder. We know that the transformer model has shown excellent ability to capture random information in multiple fields, including natural language processing and computer vision. However, the kernel size of the convolution layer is limited, and its modeling performance for remote information is not ideal. Inspired by this, swinunetr's encoder uses transformers. This architectural design improves the algorithm's ability to capture information and reduces the algorithm's computational complexity.

[0097] 3) Implementation details

[0098] Rotate the data to the RAS direction;

[0099] Randomly flip the data;

[0100] Normalize the data;

[0101] Randomly cut the data into 128x128x128 size;

[0102] Preprocessing before segmentation, unifying the direction, constraining the data within a certain range (images have different coordinate systems), and unifying the images will result in better results during subsequent segmentation.

[0103] Specifically as attached Figure 6 As shown in FIG. 1 , it is an operation flow chart of the segmentation method provided by an embodiment of the present invention; during segmentation, binary segmentation can be performed first (first roughly locate the spinal region and reduce the region), and then multi-value segmentation, such as 25-value segmentation; or four-value segmentation can be performed first (first roughly locate the cervical (7 pieces), thoracic (12 pieces), and vertebral (5 pieces) regions, and the thoracic and lumbar vertebrae are initially located by the segmentation algorithm, so as to reduce the image outside the thoracic and lumbar vertebrae for the subsequent segmentation results), and then N-value segmentation is performed for the three different regions respectively; a centroid positioning step can also be added between the coarse positioning and the fine segmentation, and the centroid can represent a single vertebra that constitutes the spine, because the vertebrae are spaced apart from each other. They are not connected to each other, so the pedicle part is discarded during segmentation, and the remaining vertebral parts can be directly located. If adjacent vertebrae are connected, it is necessary to use corrosion algorithms or other algorithms to exploit the connected vertebrae so that the adjacent vertebrae are not connected; after locating multiple centroids (the centroid is a spatial point with three-dimensional coordinates), when locating a single vertebra according to the centroid, the size of the impact is unknown, so the image will be too large or too small during cropping. For large images, the cropped 3D CT is segmented into a single vertebra according to the 3D Gaussian distribution of a single centroid to obtain a single vertebral image, and the spine is cut. If the spine is incomplete, it is impossible to determine the specific vertebra or vertebra. It is necessary to introduce a classifier to pre-learn the shape of each vertebra in the training set sample, and then train after learning.

[0104] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the disclosed embodiments are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0106] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0107] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0108] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0110] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A bone structure segmentation method based on imaging, characterized in that: The method comprises: 101, obtain three-dimensional CT images of the spine; 102, segmenting the 3D CT image to obtain an ROI region image including any one or more of the cervical spine, thoracic spine, and lumbar spine; 103, cropping the ROI region image to obtain a mask image including the ROI vertebrae; 104. Segment the mask image using a segmentation network to obtain an image containing the ROI vertebra.

2. The bone structure segmentation method based on imaging according to claim 1, characterized in that: The cropping method in 104 includes: finding the maximum and minimum values ​​of the mask in the mask image in different directions in the RAS three-dimensional space, cropping the areas smaller than the minimum value and larger than the maximum value in different directions of the three-dimensional CT image, and obtaining a mask image containing the ROI vertebra; Optionally, the ROI area also includes: sacrum and coccyx.

3. The bone structure segmentation method based on imaging according to claim 1, characterized in that: The mask image containing the ROI vertebra is at least a binary mask image. When it is a binary mask image, the ROI area is any one of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a ternary mask image, the ROI area is any two of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a quaternary mask image, the ROI area is any three of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a quinary mask image, the ROI area is any four of the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx; when it is a hexavalue mask image, the ROI area includes the cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacrum, and coccyx.

4. The bone structure segmentation method based on imaging according to claim 1, characterized in that: After 103 and before 104, the method further includes: cropping a single vertebra in the mask image containing the ROI vertebra to obtain a mask image in which adjacent single vertebrae are not connected; locating the centroid of a single vertebra in the mask image in which adjacent single vertebrae are not connected to obtain an image with a centroid; and segmenting the image with the centroid with the single centroid as the center to obtain a single vertebra image corresponding to the single centroid; Optionally, the method of cropping the single vertebra in the mask image containing the ROI vertebra includes: gradually eroding from the edges of the single vertebra to the center in equal proportion; Optionally, the volume of the single vertebral body after trimming in 102 is smaller than the volume of the single vertebral body before trimming; Preferably, the ratio of the volume of the single vertebral body after trimming to the volume of the single vertebral body before trimming is 1 / 10-9 / 10, preferably 1 / 5-4 / 5, preferably 1 / 2.

5. The method for bone structure segmentation based on imaging according to claim 4, characterized in that: The method for locating the centroid of a single vertebra in a mask image where adjacent single vertebrae are not connected comprises: using a connected domain algorithm to identify at least one separate region in the N-value mask image where adjacent single vertebrae are not connected; locating the centroid of the separate region as the centroid of the single vertebra; the separate region is the region where the single vertebrae are located; Optionally, the centroids in the image with centroids are sorted according to the z-axis coordinates of the centroids.

6. The method for bone structure segmentation based on imaging according to claim 4, characterized in that: The method for acquiring a single vertebral image corresponding to the single centroid includes: acquiring a 3D Gaussian distribution with a×b×c fixed around the single centroid; acquiring a 3D image with a×b×c fixed around the centroid; and segmenting from the 3D image, based on the 3D Gaussian distribution centered on the centroid, to obtain a single vertebral image corresponding to the single centroid around the Gaussian distribution.

7. The method for bone structure segmentation based on imaging according to any one of claims 1 to 6, characterized in that: After 104, the method further includes 105, identifying the category of a single vertebra in the vertebral image based on the position of the center of mass; optionally, when the category of a single vertebra cannot be determined according to the method in 105, the method further includes: inputting the single vertebral image into a classifier to identify the category of the single vertebra.

8. A computer device, characterized in that: The device comprises: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Automatic segmentation and identification method for spine of X-ray film

    CN113205535A

  • Lumbar vertebra CT image space positioning method based on artificial neural network

    CN114913160A

  • Bone mass recognition method and system based on three-dimensional convolutional neural network and CT image

    CN116416428A

  • Spine image segmentation identification method, device and equipment

    CN117350932A

Cited By

  • Deep learning-based spine segmentation method, apparatus and device, and medium

    CN119624998A