Multi-stage segmentation method and device for narrow cavity of human body and storage medium

Abstract

The embodiment of the invention provides a multi-stage segmentation method and device for a narrow cavity of a human body and a storage medium. The method comprises the steps of obtaining preprocessed electronic computed tomography (CT) data; inputting the preprocessed CT data into a multi-stage segmentation model, and obtaining a segmentation result output by the multi-stage segmentation model; the multi-stage segmentation model comprises a first stage, a second stage and a third stage. According to the multi-stage segmentation method and device for the narrow cavity of the human body and the storage medium provided by the embodiment of the invention, the segmentation of the narrow cavity of the human body is performed through the multi-stage segmentation model, and each stage has respective training task and key point according to the corresponding loss function, so that the segmentation result of the narrow cavity of the human body is obtained; the accuracy, the sensitivity and the branch detection rate of the cavity can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a multi-stage segmentation method, device, and storage medium for narrow cavities in the human body. Background Art

[0002] Ideal lumen segmentation needs to meet the requirements of robustness, accuracy, completeness, and take up as little computing power and storage space as possible, but existing methods have defects in different aspects.

[0003] Traditional methods use hand-crafted features and rules for image segmentation, which makes them computationally simple and efficient, while requiring relatively little data. However, due to the lack of image semantic features, they are prone to marking a large number of leakage areas when there is noise interference and blurred boundaries in electronic computed tomography (CT), resulting in low accuracy and completeness.

[0004] Existing deep learning methods can reduce leakage and increase the length of the inspected cavities, but they are more prone to damage than traditional methods. Traditional methods typically rely on strength constraints to ensure connectivity. However, in deep learning methods, such as segmenting lung trachea, the imbalance between classes (trachea and background) and within classes (large trachea and small trachea) in lung CT images can cause gradients in some small tracheal regions to be overwhelmed during backpropagation, resulting in discontinuities in the prediction results. Furthermore, due to the complex structure of some cavities in the human body, maintaining segmentation accuracy while preserving their topological integrity is a challenging task. Summary of the Invention

[0005] In response to the above technical problems, embodiments of the present application provide a multi-stage segmentation method, device, and storage medium for narrow human cavities.

[0006] In a first aspect, embodiments of the present application provide a multi-stage segmentation method for narrow human cavities, comprising:

[0007] Acquiring pre-processed computed tomography (CT) data;

[0008] Inputting the preprocessed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model;

[0009] The multi-stage segmentation model includes a first stage, a second stage and a third stage;

[0010] The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data;

[0011] The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result;

[0012] The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

[0013] In some embodiments, the second loss function is determined based on a first static weight and a dynamic weight, wherein the first static weight is calculated based on the local imbalance of the cavity before the second stage, and the dynamic weight is calculated based on the predicted performance of the voxel in the second stage.

[0014] In some embodiments, the third loss function is determined based on a second static weight and a dynamic weight, wherein the second static weight is calculated based on the first static weight, the weight of the undetected lumen voxels in the second stage based on the distance from the center skeleton line, and the weight of the lumen voxels at the fracture in the second stage based on the lumen size.

[0015] In some embodiments, the multi-stage segmentation model is determined based on a multi-scale residual feature extraction module, which is used to coordinate image features learned from lumen regions at different scales.

[0016] In some embodiments, obtaining pre-processed computed tomography (CT) data includes:

[0017] Acquire CT data of the area where the cavity is located;

[0018] The CT values ​​are uniformly truncated at a uniform channel, and the preprocessed CT data are obtained based on normalization.

[0019] In some embodiments, obtaining the segmentation result output by the multi-stage segmentation model includes:

[0020] The segmentation result is determined from the feature map predicted by the network based on a dual threshold iterative algorithm.

[0021] In a second aspect, the present invention also provides a multi-stage segmentation device for narrow human cavities, comprising:

[0022] A first acquisition module is used to acquire pre-processed electronic computer tomography (CT) data;

[0023] A second acquisition module is used to input the preprocessed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model;

[0024] The multi-stage segmentation model includes a first stage, a second stage and a third stage;

[0025] The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data;

[0026] The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result;

[0027] The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

[0028] In some embodiments, the second loss function is determined based on a first static weight and a dynamic weight, wherein the first static weight is calculated based on the local imbalance of the cavity before the second stage, and the dynamic weight is calculated based on the predicted performance of the voxel in the second stage.

[0029] In some embodiments, the third loss function is determined based on a second static weight and a dynamic weight, wherein the second static weight is calculated based on the first static weight, the weight of the undetected lumen voxels in the second stage based on the distance from the center skeleton line, and the weight of the lumen voxels at the fracture in the second stage based on the lumen size.

[0030] In some embodiments, the multi-stage segmentation model is determined based on a multi-scale residual feature extraction module, which is used to coordinate image features learned from lumen regions at different scales.

[0031] In some embodiments, the first acquisition module includes:

[0032] The first acquisition submodule is used to acquire CT data of the area where the cavity is located;

[0033] The second acquisition submodule is used to uniformly cut the CT values ​​into a unified channel and acquire the preprocessed CT data based on normalization.

[0034] In some embodiments, the second acquisition module includes:

[0035] The first determination submodule is used to determine the segmentation result from the feature map predicted by the network based on a dual-threshold iterative algorithm.

[0036] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements any of the multi-stage segmentation methods for narrow human cavities as described above.

[0037] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the multi-stage segmentation method for narrow human cavities as described above is implemented.

[0038] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the multi-stage segmentation methods for narrow human cavities described above.

[0039] The multi-stage segmentation method, device, and storage medium for narrow human cavities provided in the embodiments of the present application perform human cavity segmentation through a multi-stage segmentation model. Each stage has its own training tasks and focus based on the corresponding loss function, thereby obtaining the segmentation results of narrow human cavities, which can improve the cavity accuracy, sensitivity, and branch detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 1 is a flow chart of a multi-stage segmentation method for a narrow human cavity provided in an embodiment of the present application;

[0042] Figure 2 2 is a schematic structural diagram of a multi-stage segmentation device for a narrow human cavity provided in an embodiment of the present application;

[0043] Figure 3 It is a schematic diagram of the physical structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] For CT-based segmentation of narrow human cavities, the common methods adopted in existing technologies are mainly divided into traditional methods and deep learning-based methods.

[0045] Traditional CT image segmentation methods mainly segment images according to physical features such as texture, shape, and structure. According to their segmentation principles, they can be roughly divided into region growing, morphology, tubular structure, etc.: 1) Region growing is a classic image processing technology based on the local features of the CT image and the similarity between pixels. By selecting a seed point in the tubular region as the starting point of growth, the growth criterion is used to gradually add adjacent pixels to the tubular region until the stopping condition is met. 2) Morphological methods are based on morphological operations. Morphological operations can adjust and grow the shape according to the shape characteristics of the cavity and the properties of the pixel neighborhood. In the two-dimensional slices of the CT image, the darker lumen is surrounded by the brighter cavity wall, forming a local extreme value region. Morphological methods can be used to detect the local extreme value region to extract the outline of the small cavity. 3) Tubular structure detection refers to the use of tubular structure filters to detect objects similar to tubular structures in the image. Since narrow cavities in the human body (such as the pulmonary trachea, blood vessels, etc.) have a natural tubular anatomical structure, the tubular structure in the image can be first detected based on the local or overall tubular structural characteristics of the cavity, and then the detection results can be finely segmented to achieve the segmentation of the human cavity.

[0046] Deep learning methods can capture more semantic features from images. Recently, more patents have applied neural networks to lumen segmentation, assisting in eliminating regional leaks or directly achieving the final segmentation results end-to-end: 1) Leak detection is modeled as a classification problem, and convolutional neural networks (CNNs) are trained using supervised learning to perform this classification task. 2) CT images are used as input, and CNNs are trained to learn the feature representations in the image. The final layer of the CNN uses a pixel-level classifier to classify each pixel, determining the probability that the pixel belongs to a human lumen and outputting the lumen segmentation label. 3) 3D U-Net-based segmentation of human lumens in CT images primarily uses a fixed-step sliding window to sample lumen images and input them into the network. Lumen locations are predicted using a symmetric encoder-decoder structure, and finally, all extracted candidate regions are combined into an integrated lumen structure. 4) 3D U-Net segmentation is combined with other algorithms and modules to improve lumen segmentation accuracy or completeness.

[0047] In response to the above technical problems, the multi-stage segmentation method for narrow human cavities provided in the embodiment of the present application inputs the preprocessed preoperative CT data into a deep network, completes multi-stage voxel-level prediction of the cavity, and post-processes to generate connected 0 / 1 binary cavity labels, thereby providing a visual cavity model for clinical surgery.

[0048] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] Figure 1 FIG. 1 is a flow chart of a multi-stage segmentation method for a narrow human cavity provided in an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a multi-stage segmentation method for narrow human cavities, comprising:

[0050] Step 101: Acquire pre-processed computed tomography (CT) data.

[0051] Step 102: Input the pre-processed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model.

[0052] The multi-stage segmentation model includes a first stage, a second stage and a third stage.

[0053] The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data.

[0054] The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, and the second loss function is used to solve the intra-class imbalance problem in the preliminary result.

[0055] The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

[0056] Specifically, a traditional image processing method is first used to extract the cavity region of interest, thereby obtaining CT data of the cavity region, and then data preprocessing is performed on the CT data to obtain preprocessed CT data.

[0057] The preprocessed CT data is then fed into a multi-stage segmentation model. The multiple stages are: In the first stage, only the Random Crop strategy is used to process the CT data and the first loss function, such as Dice Loss, is used to perform preliminary network training; in the second stage, the Hard Mining Crop and Small Airway Crop strategies are added to focus on training the segmentation of the cavities that were not extracted in the first stage, while a second loss function, such as General Union Loss, is added to address the problem of intra-class imbalance; in the third stage, the Breakage Crop strategy is added to focus on repairing the locations of cavity breaks in the second stage. A third loss function, such as a weighted Break-aware Loss, is added in conjunction with Dice Loss and General Union Loss to train the deep network, focusing on improving cavity length and branch detection rate.

[0058] During the training process at each stage, corresponding weights were designed and added to the loss function to deal with the problems of intra-class imbalance and cavity disconnection: the loss function General Union Loss combines the static prior weight based on the local imbalance of the cavity with the dynamic weight based on the difficulty and importance of voxel-level classification, focusing on training the small cavity structures that were not extracted in the first stage; the loss function Break-aware Loss uses weights based on centerline distance and weights based on cavity size, which are respectively used to repair the cavity positions and fracture positions that were not extracted in the second stage.

[0059] It is worth mentioning that the multi-stage segmentation model is a network model DMR-UNet, which is a fusion of the network model WingNet and the deformed multi-scale residual feature extraction module (MultiRes Block) structure. Several convolutional blocks of the encoder and decoder in the original network are replaced by a Residual Concatenation Module (RCM) structure, and the input data is used to complete the residual calculation in the form of Short-Middle-Long Skip Connection.

[0060] Furthermore, the backbone network of the multi-stage segmentation model can all be DMR-Unet, and the backbone network used in each stage can also be different. Specific networks such as nnUNet, VNet and Attention UNet can be designed according to the tasks of each training stage.

[0061] The multi-stage segmentation method for narrow human cavities provided in the embodiment of the present application performs human cavity segmentation through a multi-stage segmentation model. Each stage has its own training tasks and focuses according to the corresponding loss function, thereby obtaining the segmentation results of narrow human cavities, which can improve the cavity accuracy, sensitivity and branch detection rate.

[0062] In some embodiments, obtaining pre-processed computed tomography (CT) data includes:

[0063] Acquire CT data of the area where the cavity is located;

[0064] The CT values ​​are uniformly truncated at a uniform channel, and the preprocessed CT data are obtained based on normalization.

[0065] Specifically, traditional image processing methods were first used to extract the cavity region of interest, improving the efficiency of subsequent network training. Taking advantage of the fact that the cavity region in the CT scan contrasts sharply with the background, resulting in a bimodal histogram, a histogram-based threshold segmentation method was first used. Morphological knowledge was then used to optimize the binary region extraction to create a ROI mask, which was then used to clip the CT scan to its corresponding ground-truth label.

[0066] Secondly, because the data distribution of CT scans varies from person to person, the CT values ​​are uniformly truncated to a single channel and normalized to the range [0, 1]. To reduce GPU space usage and computational complexity during network training, the CT scans are cropped into small 2D / 3D blocks and fed into the network.

[0067] In addition, to improve the robustness of the segmentation model, we use random rotation and flipping for data augmentation.

[0068] It is worth mentioning that ROI extraction in data preprocessing can be processed using deep learning methods, such as UNet, FCN and other network structures.

[0069] The multi-stage segmentation method for narrow human cavities provided in the embodiments of the present application can reduce the space occupation and computational complexity of the GPU during network training and improve the robustness of the segmentation model by preprocessing the CT data.

[0070] In some embodiments, the multi-stage segmentation model is determined based on a multi-scale residual feature extraction module, which is used to coordinate image features learned from lumen regions at different scales.

[0071] Specifically, the multi-stage segmentation model in the embodiment of the present application is composed of a MultiResBlock fused with a WingNet deformation. Several convolution blocks of the encoder and decoder are replaced by an RCM. The input data is calculated as a residual in the form of a Short-Middle-LongSkip Connection. The specific structure is as follows.

[0072] WingNet consists of 18 convolutional modules (ConvBlocks), 12 of which serve as encoders and 6 as decoders. Each ConvBlock performs convolution, normalization, and activation functions on the data passed from the previous ConvBlock. The output is then passed to the next ConvBlock and upsampled to the original resolution before being added to the group-supervised feature pyramid.

[0073] To address the scale variation of certain cavities, the present embodiment incorporates a deformed MultiRes Block into the WingsNet encoder to coordinate image features learned from cavities of different scales, thereby improving the vanishing gradient problem of small structures within the cavities. The MultiRes Block is an extension of the Residual Block, inspired by the inception block. The results of the three 3x3 convolutions in the MultiRes Block are concatenated as a combined feature map, which is then added to the result of a 1x1 convolution of the input feature map.

[0074] An embodiment of the present application proposes RCM, which completes multiple convolution block calculations of its input data and splicing of the convolution results. The original CT data block is convolved with 1x1x1 in different resolutions and then added to the RCM result feature map of the corresponding resolution. That is, the original image information participates in the supplementation and understanding of the encoder information flow through the Small-Middle-Long skip connettion; the decoder only uses RCM to splice and fuse multi-scale information.

[0075] It is worth mentioning that 3D CT data can be cropped into blocks of different sizes during different stages of training. The skip connection between the encoder and decoder can enhance the channel attention mechanism and spatial attention mechanism.

[0076] The multi-stage segmentation method for narrow human cavities provided in the embodiment of the present application can introduce multi-scale information by fusing the deformed MultiRes Block structure in the feature extraction stage of WingNet, thereby improving the network's perception of features at different scales, thereby helping to alleviate the problem of gradient disappearance in small cavities.

[0077] In some embodiments, the second loss function is determined based on a first static weight and a dynamic weight. The first static weight is calculated based on the local imbalance of the cavity before the second stage, and the dynamic weight is calculated based on the prediction performance of the voxels in the second stage.

[0078] Specifically, the segmentation in the first stage of the multi-stage segmentation model is trained using the first loss function Dice Loss (DL):

[0079]

[0080] where N is the total number of voxels, p i is the predicted value of a certain voxel, and g i is the ground truth of this voxel. Since there are size differences at different positions of some cavities, if the input data patch contains both large and small cavities, using DL training will exacerbate the gradient erosion of small cavities. In the embodiments of this application, the training in the second stage adopts a mixed form of DL and the second loss function GeneralUnion Loss (GUL):

[0081]

[0082] where x and y are hyperparameters that control the degree of alleviating the intra-class imbalance. GUL is specifically:

[0083]

[0084] where w i is the weight assigned to each voxel. 0 < r < 1 achieves voxel-level focusing of the gradient ratio, and α + β = 1.

[0085] In this solution, w i is defined as two parts. One part is the static weight w s of each voxel calculated based on the local imbalance of the cavity before training, that is, the first static weight w s , and the other part is the dynamic weight w d calculated according to the prediction performance of the voxels during training.

[0086] w i = w s + w d

[0087] Among them, the first static weight w s :

[0088] w s = (1 - μ)(-log 10 FR p ) θ + μ

[0089]

[0090] where 0<μ<1, 2<θ<3 are hyperparameters, B is a predefined neighborhood of N voxels centered at p, and FR p is the local foreground rate of voxel p, which can reflect the size of the branch where the voxel is located. s The smaller the branch size, the greater the weight of the cavity.

[0091] Dynamic weight w d :

[0092] w d =-n t (1-p t ) v log(p t )

[0093]

[0094]

[0095] Where 2<ν<3 is the adjustment factor for focus loss, and n is set as the ratio of the number of non-cavity voxels to the number of cavity voxels. d Focal Loss is used to give high weights to the voxels of cavities that are difficult to classify.

[0096] The multi-stage segmentation method for narrow human cavities provided in the embodiments of the present application adopts a multi-stage deep network training process and designs appropriate weighted loss functions at each stage, so that each stage can complete its own training focus in a targeted manner. At the same time, the training results of each stage are connected to each other, and the parameters of the multi-stage segmentation model are continuously fine-tuned, which can improve the accuracy, sensitivity and branch detection rate of the cavity.

[0097] In some embodiments, the third loss function is determined based on a second static weight and a dynamic weight, wherein the second static weight is calculated based on the first static weight, the weight of the undetected lumen voxels in the second stage based on the distance from the center skeleton line, and the weight of the lumen voxels at the fracture in the second stage based on the lumen size.

[0098] Specifically, small cavities are difficult to detect and often disconnected, which affects the connectivity of the entire cavity. In this embodiment, DL, GUL, and the third loss function Break-aware Loss (BAL) are mixed to train the third stage:

[0099]

[0100]

[0101] Where C is the total number of voxels on the cavity skeleton line, is the weight assigned to each voxel.

[0102] In the third phase, It is composed of the second static weight and the dynamic weight, wherein the second static weight New components will be added:

[0103]

[0104]

[0105] Where 0<ρ<1 controls the degree of attention paid to fracture repair, w hm is the weight of the cavity voxels not detected in the second stage based on the distance from the central skeleton line, w br is the weight of the lumen voxel at the fracture in the second stage based on the lumen size.

[0106] in, The expression is:

[0107] where d i is the shortest distance from the lumen voxel to the central skeleton line, and dmax is the maximum radius of the lumen that was not detected in the second stage. hm This makes the weight of voxels closer to the center line larger in difficult-to-segment samples.

[0108] w br The expression is:

[0109]

[0110] in is the shortest distance from the cavity voxel to the cavity wall, and K is a constant. br In the fracture sample, the voxels in the cavity with thicker branches and closer to the center line have greater weights.

[0111] The method to detect the fracture situation from the difficult-to-segment samples is to perform convolution operation on the unextracted skeleton points. The points with the same convolution results belong to the broken skeleton lines.

[0112] The multi-stage segmentation method for narrow human cavities provided in the embodiments of the present application adopts a multi-stage deep network training process and designs appropriate weighted loss functions at each stage, so that each stage can complete its own training focus in a targeted manner. At the same time, the training results of each stage are connected to each other, and the parameters of the multi-stage segmentation model are continuously fine-tuned, which can improve the accuracy, sensitivity and branch detection rate of the cavity.

[0113] In some embodiments, obtaining the segmentation result output by the multi-stage segmentation model includes:

[0114] The segmentation result is determined from the feature map predicted by the network based on a dual threshold iterative algorithm.

[0115] Specifically, the segmentation result is usually determined by the predicted probability map with a threshold of 0.5. Considering the existence of some weak cavity voxels whose scores are slightly smaller than the threshold on the probability map, in order to enhance the connectivity between voxels, this scheme adopts the dual threshold iterative algorithm (DTI) to effectively binary segment the predicted probability map.

[0116] The multi-stage segmentation method for narrow human cavities provided in the embodiment of the present application performs human cavity segmentation through a multi-stage segmentation model. Each stage has its own training tasks and focuses according to the corresponding loss function, thereby obtaining the segmentation results of narrow human cavities, which can improve the cavity accuracy, sensitivity and branch detection rate.

[0117] Figure 2 Schematic diagram of the structure of the multi-stage segmentation device for narrow human cavities provided in the embodiment of the present application. Figure 2 As shown, the multi-stage segmentation device for narrow human cavities provided in the embodiment of the present application includes a first acquisition module 201 and a second acquisition module 202, wherein:

[0118] A first acquisition module 201 is used to acquire pre-processed computed tomography (CT) data;

[0119] A second acquisition module 202 is configured to input the pre-processed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model;

[0120] The multi-stage segmentation model includes a first stage, a second stage and a third stage;

[0121] The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data;

[0122] The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result;

[0123] The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

[0124] In some embodiments, the second loss function is determined based on a first static weight and a dynamic weight, wherein the first static weight is calculated based on the local imbalance of the cavity before the second stage, and the dynamic weight is calculated based on the predicted performance of the voxel in the second stage.

[0125] In some embodiments, the third loss function is determined based on a second static weight and a dynamic weight, wherein the second static weight is calculated based on the first static weight, the weight of the undetected lumen voxels in the second stage based on the distance from the center skeleton line, and the weight of the lumen voxels at the fracture in the second stage based on the lumen size.

[0126] In some embodiments, the multi-stage segmentation model is determined based on a multi-scale residual feature extraction module, which is used to coordinate image features learned from lumen regions at different scales.

[0127] In some embodiments, the first acquisition module includes:

[0128] The first acquisition submodule is used to acquire CT data of the area where the cavity is located;

[0129] The second acquisition submodule is used to uniformly cut the CT values ​​into a unified channel and acquire the preprocessed CT data based on normalization.

[0130] In some embodiments, the second acquisition module includes:

[0131] The first determination submodule is used to determine the segmentation result from the feature map predicted by the network based on a dual-threshold iterative algorithm.

[0132] Specifically, the multi-stage segmentation device for the narrow human cavity provided in the embodiment of the present application can implement all the method steps implemented in the multi-stage segmentation method embodiment of the narrow human cavity, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0133] Figure 3 is a schematic diagram of the physical structure of the electronic device provided in the embodiment of the present application, such as Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call logic instructions in the memory 330 to execute a multi-stage segmentation method for a narrow human cavity, which includes:

[0134] Acquiring pre-processed computed tomography (CT) data;

[0135] Inputting the preprocessed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model;

[0136] The multi-stage segmentation model includes a first stage, a second stage and a third stage;

[0137] The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data;

[0138] The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result;

[0139] The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

[0140] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0141] In some embodiments, the second loss function is determined based on a first static weight and a dynamic weight, wherein the first static weight is calculated based on the local imbalance of the cavity before the second stage, and the dynamic weight is calculated based on the predicted performance of the voxel in the second stage.

[0142] In some embodiments, the third loss function is determined based on a second static weight and a dynamic weight, wherein the second static weight is calculated based on the first static weight, the weight of the undetected lumen voxels in the second stage based on the distance from the center skeleton line, and the weight of the lumen voxels at the fracture in the second stage based on the lumen size.

[0143] In some embodiments, the multi-stage segmentation model is determined based on a multi-scale residual feature extraction module, which is used to coordinate image features learned from lumen regions at different scales.

[0144] In some embodiments, obtaining pre-processed computed tomography (CT) data includes:

[0145] Acquire CT data of the area where the cavity is located;

[0146] The CT values ​​are uniformly truncated at a uniform channel, and the preprocessed CT data are obtained based on normalization.

[0147] In some embodiments, obtaining the segmentation result output by the multi-stage segmentation model includes:

[0148] The segmentation result is determined from the feature map predicted by the network based on a dual threshold iterative algorithm.

[0149] Specifically, the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the method embodiment in which the execution subject is the electronic device, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0150] In another aspect, the present invention further provides a computer program product, comprising a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the multi-stage segmentation method for a narrow human cavity provided by the above methods, the method comprising:

[0151] Acquiring pre-processed computed tomography (CT) data;

[0152] Inputting the preprocessed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model;

[0153] The multi-stage segmentation model includes a first stage, a second stage and a third stage;

[0154] The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data;

[0155] The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result;

[0156] The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

[0157] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the multi-stage segmentation method for a narrow human cavity provided by the above methods is implemented. The method comprises:

[0158] Acquiring pre-processed computed tomography (CT) data;

[0159] Inputting the preprocessed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model;

[0160] The multi-stage segmentation model includes a first stage, a second stage and a third stage;

[0161] The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data;

[0162] The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result;

[0163] The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0165] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0166] It should also be noted that the terms "first," "second," and the like in the embodiments of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein. Furthermore, the objects distinguished by "first" and "second" are generally of the same type, and do not limit the number of objects. For example, the first object can be one or more.

[0167] In the embodiments of this application, the term "and / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0168] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0169] In this application, "determine B based on A" means that factor A must be considered when determining B. This is not limited to "determine B based solely on A" and should also include: "determine B based on A and C", "determine B based on A, C, and E", "determine C based on A, and further determine B based on C", etc. It can also include using A as a condition for determining B, for example, "when A meets the first condition, use the first method to determine B"; another example, "when A meets the second condition, determine B"; another example, "when A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that uses A as a factor in determining B, for example, "when A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multi-stage segmentation method for narrow human cavities, characterized in that: include: Acquiring pre-processed computed tomography (CT) data; Inputting the preprocessed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model; The multi-stage segmentation model includes a first stage, a second stage and a third stage; The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data; The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result; The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

2. The multi-stage segmentation method for narrow human cavities according to claim 1, characterized in that: The second loss function is determined based on a first static weight and a dynamic weight, wherein the first static weight is calculated based on the local imbalance of the cavity before the second stage, and the dynamic weight is calculated based on the predicted performance of the voxel in the second stage.

3. The multi-stage segmentation method for narrow human cavities according to claim 1, characterized in that: The third loss function is determined based on the second static weight and the dynamic weight, wherein the second static weight is calculated based on the first static weight, the weight of the undetected lumen voxels in the second stage based on the distance from the central skeleton line, and the weight of the lumen voxels at the fracture in the second stage based on the lumen size.

4. The multi-stage segmentation method for narrow human cavities according to claim 1, characterized in that: The multi-stage segmentation model is determined based on a multi-scale residual feature extraction module, which is used to coordinate image features learned from cavity regions at different scales.

5. The multi-stage segmentation method for narrow human cavities according to claim 1, characterized in that: The obtaining of pre-processed electronic computer tomography (CT) data comprises: Acquire CT data of the area where the cavity is located; The CT values ​​are uniformly truncated at a uniform channel, and the preprocessed CT data are obtained based on normalization.

6. The multi-stage segmentation method for narrow human cavities according to claim 1, characterized in that: The obtaining of the segmentation result output by the multi-stage segmentation model includes: The segmentation result is determined from the feature map predicted by the network based on a dual threshold iterative algorithm.

7. A multi-stage segmentation device for narrow human cavities, characterized in that: include: A first acquisition module is used to acquire pre-processed electronic computer tomography (CT) data; A second acquisition module is used to input the preprocessed CT data into a multi-stage segmentation model to obtain a segmentation result output by the multi-stage segmentation model; The multi-stage segmentation model includes a first stage, a second stage and a third stage; The first stage is used to obtain preliminary results based on a first loss function, and the first loss function is used to perform semantic segmentation on the CT data; The second stage is used to obtain an improved result based on the first loss function, the second loss function and the preliminary result, wherein the second loss function is used to solve the intra-class imbalance problem in the preliminary result; The third stage is used to obtain the segmentation result based on the first loss function, the second loss function, the third loss function and the improved result, and the third loss function is used to repair the cavity fracture in the improved result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the multi-stage segmentation method for narrow human cavities according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-stage segmentation method of a narrow cavity in the human body as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-stage segmentation method of a narrow cavity in the human body as claimed in any one of claims 1 to 6 is implemented.

Images