A method and apparatus for segmenting a medical image

Abstract

The application discloses a medical image segmentation method and device. In the case, the target cardiac medical image data to be segmented is acquired; the target detection module of the image segmentation model extracts features from the target cardiac medical image data to obtain a three-dimensional multi-scale feature map, and a plurality of candidate bounding boxes corresponding to a cardiac region are generated according to the three-dimensional multi-scale feature map; the target detection module screens a cardiac position bounding box containing a whole heart anatomical structure from the plurality of candidate bounding boxes, and obtains a target image block based on the cardiac position bounding box and by cutting a corresponding region of interest from the target cardiac medical image data; and the cardiac segmentation module of the image segmentation model segments the target image block to generate a segmentation result. The whole heart segmentation result can be obtained through one-time target detection and segmentation reasoning, without the need of cutting and splicing the whole cardiac medical image data, so that the reasoning time and the calculation cost are reduced, and the image segmentation efficiency and the segmentation precision are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for segmenting medical images. Background Technology

[0002] Whole-heart segmentation of cardiac medical imaging data (such as CT, short for Computed Tomography) is a key technical link in medical fields such as clinical diagnosis and surgical planning. Its segmentation accuracy directly affects the accuracy of clinicians' diagnosis of heart diseases, the scientific nature of disease assessment, and the rationality of surgical plan formulation. Therefore, it has irreplaceable and important application value in medical practice.

[0003] Existing whole-heart segmentation techniques for 3D cardiac CT images typically employ a tiling strategy. This involves first dividing the complete 3D medical image data into multiple image blocks, then segmenting each block separately, and finally stitching the segmentation results together to obtain the complete whole-heart segmentation. However, this method not only struggles to effectively capture the global constraints of cardiac anatomy, leading to inconsistent segmentation results and misalignments between different parts, but also wastes significant computational resources on meaningless background processing, increasing the model's computational cost and severely impacting the segmentation efficiency and accuracy of 3D cardiac medical image data. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method and apparatus for segmenting medical images, aiming to effectively improve the segmentation efficiency and accuracy of three-dimensional cardiac medical image data.

[0005] The embodiments of this application disclose the following technical solutions: In a first aspect, this application provides a method for segmenting medical images, the method comprising: Acquire the target cardiac medical image data to be segmented; the target cardiac medical image data is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart. The target detection module in the pre-trained image segmentation model is used to extract features from the target cardiac medical image data to obtain a three-dimensional multi-scale feature map. Based on the three-dimensional multi-scale feature map, multiple candidate bounding boxes corresponding to the cardiac region are generated. The three-dimensional multi-scale feature map is a cross-scale feature representation of the target cardiac medical image data from global semantics to local details. The target detection module filters out the heart location bounding box containing the whole heart anatomical structure from multiple candidate bounding boxes, and extracts the corresponding region of interest from the target heart medical image data based on the heart location bounding box to obtain the target image block; The target image block is segmented using the heart segmentation module in the image segmentation model to generate a segmentation result. The segmentation result includes a set of heart multi-class segmentation labels, which are used to identify the voxel regions where each part of the whole heart anatomical structure is located.

[0006] In an optional implementation, the target detection module is composed of an improved 3D fast region convolutional neural network, which includes a 3D backbone network, a feature pyramid network, and a region proposal network. The target detection module in the pre-trained image segmentation model is used to extract features from the target cardiac medical image data to obtain a three-dimensional multi-scale feature map, including: The target cardiac medical image data is input into the target detection module, and the three-dimensional backbone network is used to extract multiple semantic feature maps with resolution from high to low from the target cardiac medical image data layer by layer. The feature pyramid network is used to fuse the multiple semantic feature maps to generate the three-dimensional multi-scale feature map.

[0007] In an optional implementation, generating multiple candidate bounding boxes corresponding to the heart region based on the three-dimensional multi-scale feature map includes: Using the region proposal network of the target detection module, multiple anchor boxes of different sizes are set on the three-dimensional multi-scale feature map, and the target probability value and prediction offset corresponding to each anchor box are calculated; the target probability value is used to characterize the probability that the anchor box contains the whole-heart anatomical structure, and the prediction offset is used to adjust the geometric parameters of the anchor box to fit the true boundary of the whole-heart anatomical structure; For each anchor frame, the region proposal network adjusts the geometric parameters of the anchor frame based on the predicted offset corresponding to the anchor frame to generate the corresponding predicted bounding box; The region proposal network sorts all the predicted bounding boxes from largest to smallest based on the target probability values ​​corresponding to each predicted bounding box, and obtains the sorting result. The region suggestion network performs non-maximum suppression on the N predicted bounding boxes based on the target probability values ​​corresponding to the top N predicted bounding boxes in the ranking results, in order to filter out multiple candidate bounding boxes; N is a positive integer.

[0008] In an optional implementation, the region proposal network performs non-maximum suppression on the N predicted bounding boxes based on the target probability values ​​corresponding to the top N predicted bounding boxes in the ranking result, in order to filter out multiple candidate bounding boxes, including: The region proposal network is used to construct a candidate box list based on all the predicted bounding boxes; The region suggestion network performs an intersection-union ratio (IUGR) calculation operation; the IUGR calculation operation is used to take any bounding box in the candidate box list as the target bounding box, and calculate the IUGR between the target bounding box and other bounding boxes in the candidate box list, wherein the other bounding boxes are any bounding boxes in the candidate box list other than the target bounding box, and the IUGR is used to reflect the degree of overlap between the target bounding box and the other bounding boxes; If the intersection-union ratio is greater than or equal to a preset threshold, the region suggestion network performs a bounding box culling operation; the bounding box culling operation is used to compare the target probability value of the target bounding box with the target probability values ​​of other bounding boxes, and remove the bounding box with the smallest target probability value from the candidate box list. If the intersection-union ratio is less than the preset threshold, the region suggestion network performs a bounding box retention operation to retain the other bounding box in the candidate box list. The region suggestion network repeatedly performs the intersection-union calculation operation, the bounding box elimination operation, and the bounding box retention operation until all the predicted bounding boxes have been compared, thereby obtaining a target candidate box list, and each bounding box in the target candidate box list is taken as a candidate bounding box.

[0009] In an optional implementation, the step of filtering the heart location bounding box containing the whole-heart anatomical structure from a plurality of candidate bounding boxes by the target detection module includes: Using the region proposal network, multiple candidate bounding boxes are filtered based on the three-dimensional multi-scale feature map to obtain the filtering results; the filtering process is used to select bounding boxes from the candidate bounding boxes whose target probability value is greater than a preset probability threshold and completely covers the whole-heart anatomical structure. If the filtering results include only one bounding box, then that bounding box is used as the bounding box for the heart location. If the filtering results include at least two bounding boxes, then the bounding box with the highest target probability value in the filtering results shall be used as the heart location bounding box.

[0010] In an optional implementation, the step of segmenting the target image block using the heart segmentation module in the image segmentation model to generate a segmentation result includes: The target image block is input into the heart segmentation module. The encoder in the heart segmentation module extracts features from the target image block layer by layer along multiple resolution levels with decreasing spatial resolution to obtain a set of multi-scale feature maps containing semantic information at different scales. The decoder of the heart segmentation module upsamples the first feature map layer by layer, and fuses the feature map after each upsampled layer with the feature map of the corresponding scale output by the encoder to obtain a second feature map with the same size as the target image block; the first feature map is the feature map output by the last resolution level of the encoder; A multi-class probability map of the heart is generated based on the second feature map, and the class probability map is converted into a set of multi-class segmentation labels of the heart using the argmax function.

[0011] In an optional implementation, the image segmentation model is obtained through the following process: Obtain a training sample set for training an initial image segmentation model; the training sample set includes multiple training samples, each of which includes cardiac medical image data with labeled data, the label data including the actual location bounding box for identifying the cardiac region in the cardiac medical image data, and the actual cardiac multi-class segmentation label corresponding to the cardiac medical image data; Each of the training samples is sequentially input into the initial image segmentation model. The target detection module in the image segmentation model is used to extract features from the cardiac medical image data to obtain a three-dimensional multi-scale feature map of the target. Multiple target anchor boxes of different sizes are set on the three-dimensional multi-scale feature map of the target. The intersection-union ratio (CUI) between the actual location bounding box and each of the target anchor boxes is calculated using the target detection module to obtain the CUI corresponding to each of the target anchor boxes. The target detection module marks target anchor boxes with an intersection-union ratio (IU) greater than or equal to a preset threshold as positive samples, and targets with an IU less than the preset threshold as negative samples. Based on the positive samples, the negative samples, the cardiac medical image data, the target three-dimensional multi-scale feature map, the actual location bounding box, and the actual heart multi-class segmentation label, the initial image segmentation model is iteratively trained to obtain the image segmentation model.

[0012] In an optional implementation, the step of iteratively training the initial image segmentation model based on the positive samples, the negative samples, the cardiac medical image data, the target 3D multi-scale feature map, the actual location bounding box, and the actual cardiac multi-class segmentation label to obtain the image segmentation model includes: For each positive sample, the target detection module calculates the predicted offset and target probability value corresponding to the positive sample, and adjusts the geometric parameters of the corresponding target anchor box based on the predicted offset to fit the true boundary of the whole-heart anatomical structure and generate the corresponding target predicted bounding box; the target probability value corresponding to the positive sample is used to characterize the probability that the whole-heart anatomical structure is contained in the target anchor box; Using the target detection module, the actual offset between the target anchor box and the actual position bounding box is calculated, and the bounding box regression loss value is determined based on the error between the predicted offset corresponding to the positive sample and the actual offset. Using the target detection module, multiple target candidate bounding boxes are selected from all the target predicted bounding boxes based on the target probability value corresponding to each positive sample, and the multiple target candidate bounding boxes are filtered based on the target three-dimensional multi-scale feature map to obtain the target heart location bounding box; Using the target detection module, a classification loss value is determined based on the error between the bounding box of the target heart location and the actual location bounding box; Based on the bounding box regression loss value, the classification loss value, the cardiac medical image data, and the actual heart multi-class segmentation label, the initial image segmentation model is iteratively trained to obtain the image segmentation model.

[0013] In an optional implementation, the step of iteratively training the initial image segmentation model based on the bounding box regression loss value, the classification loss value, the cardiac medical image data, and the actual cardiac multi-class segmentation labels to obtain the image segmentation model includes: Using the target detection module, the corresponding region of interest is extracted from the cardiac medical image data based on the target heart location bounding box to obtain the corresponding image patch; The image block is segmented using the heart segmentation module to generate a set of heart multi-class segmentation prediction labels, and the segmentation loss value is determined based on the error between the heart multi-class segmentation prediction labels and the actual heart multi-class segmentation labels. Using the edge loss head in the initial image segmentation model, convolution operations are performed on the actual heart multi-class segmentation label and the heart multi-class segmentation prediction label to extract the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation prediction label; the edge information is used to reflect the boundary difference between the heart and the background region, as well as the boundary difference between various parts within the whole heart anatomical structure. Using the edge loss head in the initial image segmentation model, the edge loss value is determined based on the error between the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation prediction label; Based on the bounding box regression loss value, the classification loss value, the segmentation loss value, and the edge loss value, the parameters of the initial 3D image segmentation model are adjusted, and the adjusted model is iteratively trained using unused training samples from the training sample set until the training cutoff condition is met, thereby obtaining the 3D image segmentation model.

[0014] A second aspect of this application provides a medical image segmentation apparatus, the apparatus comprising: The acquisition module is used to acquire the target cardiac medical image data to be segmented; the target cardiac medical image data is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart. The target detection module is used to extract features from the target cardiac medical image data using the target detection module in the pre-trained image segmentation model, to obtain a three-dimensional multi-scale feature map, and to generate multiple candidate bounding boxes corresponding to the cardiac region based on the three-dimensional multi-scale feature map; the three-dimensional multi-scale feature map is a cross-scale feature representation of the target cardiac medical image data from global semantics to local details; The image cropping module is used by the target detection module to filter out the heart location bounding box containing the whole heart anatomical structure from multiple candidate bounding boxes, and to crop the corresponding region of interest from the target heart medical image data based on the heart location bounding box to obtain the target image block; The image segmentation module is used to segment the target image block using the heart segmentation module in the image segmentation model and generate segmentation results. The segmentation results include a set of heart multi-class segmentation labels, which are used to identify the voxel regions where each part of the whole heart anatomical structure is located.

[0015] Compared with the prior art, this application has the following beneficial effects: In this technical solution, the target cardiac medical image data to be segmented is first acquired. This target cardiac medical image data is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart. Then, the target detection module in the pre-trained image segmentation model is used to extract features from the target cardiac medical image data to obtain a cross-scale feature representation (i.e., a three-dimensional multi-scale feature map) that characterizes the target cardiac medical image data from global semantics to local details, thereby enhancing the model's ability to capture global and local information. Next, multiple candidate bounding boxes corresponding to the heart region are generated based on the three-dimensional multi-scale feature map, which can gradually narrow down the range of the area where the heart is located, improve the model's robustness to detecting heart data of different shapes, and lay a reliable foundation for subsequent accurate segmentation. Then, the object detection module accurately filters out the heart location bounding boxes containing the entire anatomical structure from multiple candidate bounding boxes, enabling the model to quickly and accurately locate the heart region. Next, based on the heart location bounding boxes, the corresponding region of interest is extracted from the target cardiac medical image data to obtain the target image patch. This allows the model to accurately obtain the target image patch containing the entire anatomical structure of the heart with only one object detection, without the need to segment the entire 3D medical image data, thus avoiding the problem of wasting a lot of computing resources on meaningless background area processing, and effectively improving the segmentation accuracy of subsequent 3D cardiac medical image data. Finally, the heart segmentation module in the image segmentation model is used to segment the target image patch, generating a segmentation result including a set of multi-class heart segmentation labels. Through these multi-class heart segmentation labels, the model can obtain the whole heart segmentation result that accurately identifies the voxel regions where each part of the entire anatomical structure is located with only one segmentation inference, avoiding the context loss problem caused by existing segmentation strategies, significantly improving the segmentation accuracy and efficiency of each anatomical boundary. At the same time, the generated multi-class segmentation labels can clearly identify the voxel positions of different structures in the whole heart, providing clinicians with structured and quantitative diagnostic basis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a medical image segmentation method provided in this application embodiment; Figure 2 A flowchart illustrating the process of generating a candidate bounding box, as provided in an embodiment of this application; Figure 3A flowchart illustrating another candidate bounding box generation process provided in this application embodiment; Figure 4 A flowchart illustrating the process of generating a heart location bounding box, as provided in an embodiment of this application; Figure 5 A flowchart illustrating a three-dimensional multi-scale feature map extraction process provided in this application embodiment; Figure 6 A flowchart illustrating the training process of an image segmentation model provided in this application embodiment; Figure 7 A flowchart illustrating the training process of another image segmentation model provided in this application embodiment; Figure 8 A flowchart illustrating the training process of another image segmentation model provided in this application embodiment; Figure 9 A schematic diagram of a 3D Sobel operator provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a medical image segmentation device provided in an embodiment of this application. Detailed Implementation

[0018] As described earlier, current 3D cardiac CT image whole-heart segmentation techniques typically employ a tiling strategy. This involves first dividing the complete 3D medical image data into multiple image blocks, then segmenting each block separately, and finally stitching the segmentation results of all blocks together to obtain the complete whole-heart segmentation. However, this method not only struggles to effectively capture the global constraints of cardiac anatomy, leading to inconsistent segmentation results and misalignments between different parts, but also wastes significant computational resources on meaningless background processing, increasing the model's computational cost and severely impacting the segmentation efficiency and accuracy of 3D cardiac medical image data.

[0019] The inventors have proposed a method and apparatus for segmenting medical images. This method first acquires the target cardiac medical image data to be segmented, which is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart. Then, using the target detection module in a pre-trained image segmentation model, features are extracted from the target cardiac medical image data to obtain a cross-scale feature representation (i.e., a three-dimensional multi-scale feature map) characterizing the target cardiac medical image data from global semantics to local details, enhancing the model's ability to capture both global and local information. Next, based on this three-dimensional multi-scale feature map, multiple candidate bounding boxes corresponding to the heart region are generated, gradually narrowing down the area where the heart is located. This improves the model's robustness in detecting heart data of different morphologies, laying a reliable foundation for subsequent accurate segmentation. Then, the object detection module accurately filters out the heart location bounding boxes containing the entire anatomical structure from multiple candidate bounding boxes, enabling the model to quickly and accurately locate the heart region. Next, based on the heart location bounding boxes, the corresponding region of interest is extracted from the target cardiac medical image data to obtain the target image patch. This allows the model to accurately obtain the target image patch containing the entire anatomical structure of the heart with only one object detection, without the need to segment the entire 3D medical image data, thus avoiding the problem of wasting a lot of computing resources on meaningless background area processing, and effectively improving the segmentation accuracy of subsequent 3D cardiac medical image data. Finally, the heart segmentation module in the image segmentation model is used to segment the target image patch, generating a segmentation result including a set of multi-class heart segmentation labels. Through these multi-class heart segmentation labels, the model can obtain the whole heart segmentation result that accurately identifies the voxel regions where each part of the entire anatomical structure is located with only one segmentation inference, avoiding the context loss problem caused by existing segmentation strategies, significantly improving the segmentation accuracy and efficiency of each anatomical boundary. At the same time, the generated multi-class segmentation labels can clearly identify the voxel positions of different structures in the whole heart, providing clinicians with structured and quantitative diagnostic basis.

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0021] Keyword definition: 3D Faster R-CNN is a 3D object detection model developed from the classic 2D object detection algorithm Faster R-CNN (Faster Region-based Convolutional Neural Network). It upgrades the algorithm, originally used to detect objects in planar photographs (2D images), to enable it to directly process 3D data with depth information (such as medical CT scans and LiDAR point clouds for autonomous driving), thereby accurately outlining the position and size of objects in 3D space.

[0022] 3DU-net is a classic three-dimensional fully convolutional neural network that evolved from the two-dimensional U-Net. It is specifically designed to process three-dimensional volumetric data with depth information, such as medical CT / MRI images and seismic data.

[0023] Method Implementation Examples

[0024] This application provides an embodiment of a method for segmenting medical images. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although the flowchart shows a logical order, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] See Figure 1 The figure is a flowchart of a medical image segmentation method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps: Step S101: Obtain the target cardiac medical image data to be segmented.

[0026] In one feasible implementation, a medical image segmentation system can serve as the execution subject of the medical image segmentation method in the embodiments of this application. For ease of description, the medical image segmentation system will be referred to as the system below.

[0027] In step S101, the target cardiac medical imaging data is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart.

[0028] In this embodiment of the application, the system can automatically collect and acquire the target cardiac medical image data to be segmented. The image data is three-dimensional voxel data (such as 3D cardiac CT image data) with the heart as the core and covering the surrounding local thoracic anatomical structure. It can completely characterize the three-dimensional spatial structure and anatomical information of the heart and adjacent thoracic tissues, and can provide standardized raw input data for subsequent cardiac region detection, region of interest truncation and whole-heart fine segmentation.

[0029] Step S102: Using the target detection module in the pre-trained image segmentation model, feature extraction is performed on the target cardiac medical image data to obtain a three-dimensional multi-scale feature map, and multiple candidate bounding boxes corresponding to the cardiac region are generated based on the three-dimensional multi-scale feature map.

[0030] In step S102, the three-dimensional multi-scale feature map is a cross-scale feature representation of the target cardiac medical image data from global semantics to local details.

[0031] In this embodiment, the pre-trained image segmentation model includes an object detection module and a heart segmentation module. The object detection module can be composed of an improved 3D Faster R-CNN, which includes a 3D backbone network, a feature pyramid network, and a Region Proposal Network (RPN). The heart segmentation module is composed of an improved 3D U-net network. The system can utilize the object detection module in the pre-trained image segmentation model to extract features from the target cardiac medical image data, obtaining a cross-scale feature representation (i.e., a 3D multi-scale feature map) that characterizes the target cardiac medical image data from global semantics to local details, effectively enhancing the model's ability to capture both global and local information.

[0032] Specifically, the system can automatically input the target cardiac medical image data (i.e., three-dimensional voxel data of the local thoracic anatomical structure centered on the heart, such as 3D cardiac CT image data) into the target detection module. Utilizing its three-dimensional backbone network, the system extracts multiple semantic feature maps from the target cardiac medical image data layer by layer, with resolutions ranging from high to low and semantic information from shallow to deep. High-resolution semantic feature maps focus on capturing local details of the cardiac anatomy (such as edges and fine tissue contours), while low-resolution semantic feature maps focus on representing the global semantic information of the heart as a whole (such as the overall morphology and spatial relationships of the cardiac anatomy). Subsequently, to fully integrate the advantages of feature maps at different scales and compensate for the limitations of single-scale feature maps in information capture, the system can use a feature pyramid network (such as FPN, short for Feature Pyramid Network) to perform deep feature fusion on the extracted semantic feature maps of different resolutions and semantic levels. Through the fusion processing of the feature pyramid network, the local details of the high-resolution feature maps can be organically combined with the global semantic information of the low-resolution feature maps, ultimately generating a three-dimensional multi-scale feature map that combines global constraints with local details. This 3D multi-scale feature map serves as the core input for the subsequent region proposal network to generate candidate bounding boxes and locate the heart region. It can provide reliable feature support for the rapid and accurate localization of the heart region, laying the foundation for subsequent fine segmentation of the whole heart.

[0033] Furthermore, the system can set preset 3D anchor boxes on the 3D multi-scale feature map through the region proposal network of the target detection module. Combining the target probability value and prediction offset corresponding to the 3D anchor box, multiple candidate bounding boxes covering the heart region are generated. This can gradually narrow down the range of the heart region, effectively avoid interference from the background region, and improve the robustness of the model to the detection of heart data with different morphologies (such as hearts of different body sizes and disease states). This ensures that the heart region can be accurately captured regardless of the differences in heart morphology, thus laying a reliable foundation for subsequent screening of location bounding boxes containing the anatomical structure of the whole heart from the candidate bounding boxes and carrying out accurate segmentation.

[0034] Specifically, see Figure 2 The figure is a flowchart of a candidate bounding box generation process provided in an embodiment of this application. The process includes the following steps: Step S1021: Using the region proposal network of the target detection module, set multiple anchor boxes of different sizes on the three-dimensional multi-scale feature map, and calculate the target probability value and prediction offset corresponding to each anchor box.

[0035] In step S1021, the target probability value is used to characterize the probability that the anchor frame contains a whole-heart anatomical structure, and the predicted offset is used to adjust the geometric parameters of the anchor frame to fit the true boundary of the whole-heart anatomical structure.

[0036] In this embodiment, the system can utilize the region proposal network of the target detection module to set multiple anchor boxes of different sizes (e.g., 43,200 3D anchor boxes) on a three-dimensional multi-scale feature map, and calculate the target probability value (i.e., dz, dy, dx, ln(dd), ln(dh), ln(dw)) and prediction offset corresponding to each anchor box. The target probability value reflects the probability that the anchor box contains the whole-heart anatomical structure. The higher the probability value, the greater the probability that the anchor box covers the whole-heart anatomical structure. The prediction offset can be used to adjust the geometric parameters of the anchor box to fit the true boundary of the whole-heart anatomical structure, thereby reducing the deviation between the anchor box and the real heart region.

[0037] Step S1022: For each anchor box, the region proposal network adjusts the geometric parameters of the anchor box based on the predicted offset corresponding to the anchor box to generate the corresponding predicted bounding box.

[0038] In this embodiment of the application, for each anchor frame, the system can use a region proposal network to adjust the geometric parameters of the anchor frame based on the predicted offset corresponding to the anchor frame, and generate the corresponding predicted bounding box [z]. i ,y i ,x i ,z j ,y j ,x jThe predicted offset serves as the basis for anchor frame adjustment, enabling targeted correction of the initial anchor frame's coordinates, dimensions, and other geometric parameters, effectively compensating for the deviation between the initial anchor frame and the true boundary of the whole-heart anatomical structure; the final generated predicted bounding box [z i ,y i ,x i ,z j ,y j ,x j [] represents the coordinates of the 3D bounding box, expressed through z. i y i x i With z j y j x j The starting and ending coordinates of the bounding box in the three-dimensional voxel space are defined respectively, and the area covered by the anchor box after adjustment is accurately defined to ensure that each predicted bounding box can fit the actual spatial position and shape of the whole anatomical structure as closely as possible.

[0039] Step S1023: The region proposal network sorts all predicted bounding boxes from largest to smallest based on the target probability values ​​corresponding to each predicted bounding box, and obtains the sorting result.

[0040] Step S1024: The region proposal network performs non-maximum suppression on the N predicted bounding boxes based on the target probability values ​​corresponding to the top N predicted bounding boxes in the ranking results, so as to filter out multiple candidate bounding boxes; N is a positive integer.

[0041] In this embodiment, the system can first use a region proposal network to sort all predicted bounding boxes from largest to smallest based on the target probability values ​​corresponding to each predicted bounding box, and obtain a sorting result. In the sorting result, the predicted bounding box with a higher target probability value is more likely to contain the whole-heart anatomical structure and is ranked higher. Then, the system can use a region proposal network to perform non-maximum suppression operation on the N predicted bounding boxes according to the target probability values ​​corresponding to the top N predicted bounding boxes in the sorting result, so as to filter out multiple candidate bounding boxes that have no obvious overlap and can fit the whole-heart anatomical structure well.

[0042] Specifically, see Figure 3 The figure is a flowchart of another candidate bounding box generation process provided in an embodiment of this application, which includes the following steps: Step S10241: Construct a candidate box list based on all predicted bounding boxes using the region proposal network.

[0043] Step S10242: The region proposal network performs the intersection-union ratio (IURR) calculation operation. The IURR calculation operation is used to take any bounding box in the candidate box list as the target bounding box and calculate the IURR between the target bounding box and other bounding boxes in the candidate box list.

[0044] In step S10242, other bounding boxes are any bounding boxes in the candidate box list other than the target bounding box, and the intersection-union ratio is used to reflect the degree of overlap between the target bounding box and other bounding boxes.

[0045] In this embodiment, the object of the intersection-over-union (IoU) calculation is the bounding boxes in the candidate box list, which is the top N predicted bounding boxes selected after sorting, as mentioned earlier. When performing the IoU calculation, the system can use a region proposal network to arbitrarily select one bounding box from the candidate box list as the target bounding box, and the remaining bounding boxes are used as other bounding boxes to be compared with the target bounding box. That is, each target bounding box will have its IoU calculated one by one with all other bounding boxes in the list to obtain the corresponding IoU, ensuring that no overlap of any set of bounding boxes is missed. The IoU is a key indicator used to quantify the degree of overlap between the target bounding box and other bounding boxes. Its calculation result is the ratio of the volume of the overlapping area of ​​the two bounding boxes to the volume of the overall covered area of ​​the two bounding boxes, ranging from 0 to 1. The closer the IoU value is to 1, the higher the degree of overlap and the stronger the redundancy of the areas covered by the two bounding boxes; the closer the IoU value is to 0, the lower the degree of overlap and the weaker the redundancy.

[0046] It should be noted that by calculating the intersection-union ratio (IUR) of the target bounding box with other bounding boxes one by one, and combining it with the preset IUR threshold (i.e., the preset threshold), it is possible to accurately identify the severely overlapping and redundant bounding boxes in the candidate box list. This provides support for retaining high-probability bounding boxes and deleting redundant bounding boxes, ensuring that the final selected candidate bounding boxes have no obvious overlap and can accurately cover the heart region, laying the foundation for the subsequent selection of bounding boxes for the heart location.

[0047] Step S10243: If the intersection-union ratio is greater than or equal to a preset threshold, the region proposal network performs a bounding box culling operation. The bounding box culling operation is used to compare the target probability value of the target bounding box with the target probability value of other bounding boxes, and remove the bounding box with the smallest target probability value from the candidate box list.

[0048] In step S10244, if the intersection-union ratio is less than a preset threshold, the region proposal network performs a bounding box retention operation to retain the other bounding box in the candidate box list.

[0049] In step S10245, the region proposal network repeatedly performs the intersection-union calculation operation, bounding box elimination operation, and bounding box retention operation until all predicted bounding boxes have been compared, resulting in a target candidate box list, and each bounding box in the target candidate box list is taken as a candidate bounding box.

[0050] For example, when the preset threshold is 0.7, and the candidate box list contains three predicted bounding boxes, denoted as A, B, and C, with target probabilities of 0.92, 0.85, and 0.78 respectively, the system can select bounding box A as the target bounding box and calculate the intersection-union ratio (IU) of A with B and A with C. If the IU of A with B is 0.75, which is greater than the preset threshold of 0.7, then since the target probability value of A (0.92) is greater than the target probability value of B (0.85), the system can remove B from the candidate box list. If the IU of A with C is 0.65, which is less than the preset threshold of 0.7, the system will perform a retention operation, keeping C in the candidate box list. After the system removes bounding box B, the candidate box list is left with A and C. The system can select bounding box C as the target bounding box and calculate the IU of C with A. If the IU of C is 0.65, which is less than the preset threshold of 0.7, the system will perform a retention operation, keeping A in the candidate box list. After the above operations, only bounding boxes A and C are retained in the final candidate box list. Both have no obvious redundancy and have a high probability of covering the heart region. This completes the bounding box screening operation in this round of non-maximum suppression, providing support for the accurate screening of bounding boxes for the heart location in the future.

[0051] In step S103, the target detection module filters out the heart location bounding box containing the whole heart anatomical structure from multiple candidate bounding boxes, and extracts the corresponding region of interest from the target heart medical image data based on the heart location bounding box to obtain the target image block.

[0052] In this embodiment, after screening using non-maximum suppression (NMS), the system can obtain multiple candidate bounding boxes with no obvious redundancy and a high probability of covering the heart region. Each candidate bounding box is an effective bounding box that fits the heart region, but further screening is still needed to determine the optimal bounding box (i.e., the heart location bounding box) that can completely contain the whole heart anatomical structure. The system can use the target detection module to combine the target probability value and three-dimensional multi-scale feature map corresponding to each candidate bounding box to screen out the heart location bounding box that contains the whole heart anatomical structure from multiple candidate bounding boxes, thus realizing the rapid and accurate localization of the heart region and providing accurate location basis for subsequent region of interest truncation and whole heart fine segmentation.

[0053] Specifically, see Figure 4 The figure is a flowchart of a process for generating a heart location bounding box according to an embodiment of this application. The process includes the following steps: Step S1031: Using a region proposal network, multiple candidate bounding boxes are filtered based on a 3D multi-scale feature map to obtain the filtering results.

[0054] In step S1031, the filtering process is used to select from the candidate bounding boxes the bounding boxes whose target probability value is greater than a preset probability threshold and which completely cover the whole heart anatomical structure.

[0055] In this embodiment of the application, the system can use a region proposal network to capture global semantic information and local detail features of the heart from a three-dimensional multi-scale feature map, and based on the features, determine whether the coverage of each candidate bounding box can completely encompass the whole heart anatomical structure, and remove candidate bounding boxes that only cover part of the heart region, do not completely contain the whole heart tissue, and have a target probability value less than a preset probability threshold, to obtain the screening results.

[0056] Step S1032: If the filtering results include only one bounding box, then use that bounding box as the heart location bounding box.

[0057] In this embodiment of the application, if the screening result contains only one bounding box, it indicates that the bounding box is the only one that simultaneously satisfies the two conditions of the target probability value being greater than the preset probability threshold and the complete coverage of the whole heart anatomical structure. At this time, the system can directly determine the bounding box as the heart location bounding box. The bounding box can accurately define the spatial location and range of the whole heart anatomical structure, providing a reliable basis for subsequent operations.

[0058] Step S1033: If the filtering results include at least two bounding boxes, then the bounding box with the highest target probability value in the filtering results shall be used as the bounding box of the heart location.

[0059] In this embodiment, if the filtering results include at least two bounding boxes, it indicates that multiple bounding boxes satisfy the dual filtering conditions, and further filtering of the optimal bounding box is required. In this case, the system can select the bounding box with the highest target probability value from the filtering results as the heart location bounding box.

[0060] It should be noted that the target probability value reflects the likelihood that the bounding box contains the whole heart anatomical structure. The higher the probability value, the more reliable the bounding box is in conforming to the real boundary of the whole heart and completely covering the whole heart structure. Therefore, using the bounding box with the highest target probability value as the bounding box of the heart location can maximize the accuracy of subsequent region of interest truncation and whole heart segmentation.

[0061] Furthermore, the system can extract the corresponding Region of Interest (ROI) from the target cardiac medical image data based on the heart location bounding box. The extracted range strictly matches the coverage of the heart location bounding box, ensuring that the extracted ROI completely includes all anatomical parts of the heart without omissions or redundant background interference. This allows the system to obtain target image blocks containing complete anatomical structures of the heart. The model only needs one target detection to accurately obtain target image blocks containing anatomical structures of the heart, without the need to segment the entire 3D medical image data. This avoids wasting a lot of computational resources on meaningless background area processing, effectively improving the segmentation accuracy of subsequent 3D cardiac medical image data, significantly reducing the model's computational cost, and improving the model's inference efficiency.

[0062] Step S104: Use the heart segmentation module in the image segmentation model to segment the target image block and generate segmentation results.

[0063] In step S104, the segmentation result includes a set of cardiac multi-class segmentation labels, which are used to identify the voxel regions where each part of the whole cardiac anatomical structure is located.

[0064] In this embodiment, the system can utilize the heart segmentation module in the image segmentation model to segment the target image block and generate a segmentation result including a set of heart multi-class segmentation labels. Through these heart multi-class segmentation labels, the model can obtain a whole-heart segmentation result that accurately identifies the voxel regions of each part of the whole-heart anatomical structure with only one segmentation inference, avoiding the context loss problem caused by existing block strategies, significantly improving the segmentation accuracy and efficiency of the boundaries of each anatomical part, and the generated multi-class segmentation labels can clearly identify the voxel positions of different structures in the whole heart, providing clinicians with structured and quantitative diagnostic basis.

[0065] Specifically, such as Figure 5 As shown, the system can input target image blocks into the heart segmentation module. The encoder in the heart segmentation module performs feature mining and extraction layer by layer along multiple resolution levels with decreasing spatial resolution, ultimately obtaining a set of multi-scale feature maps containing semantic information at different scales. Specifically, the feature maps extracted at high-resolution levels focus on capturing local details of each substructure of the heart (such as edge contours and fine tissue textures), while the feature maps extracted at low-resolution levels focus on representing the global semantic information of the heart (such as the overall morphology of the heart and the spatial relationships between its substructures), thus providing sufficient feature support for subsequent accurate segmentation.

[0066] Subsequently, the system can use the decoder of the heart segmentation module to perform layer-by-layer upsampling processing on the first feature map (i.e. the feature map output by the last resolution level of the encoder), gradually improving the spatial resolution of the feature map so that it gradually approaches the original size of the target image block. Simultaneously, after upsampling at each layer, the decoder performs deep fusion between the upsampled feature map of that layer and the corresponding scale (i.e., consistent with spatial resolution) feature map output by the encoder. By complementing features, the limitations of single-scale features are compensated for, ultimately resulting in a second feature map with the same size as the target image patch. This second feature map contains both global semantic constraints and rich local detail information, providing accurate basis for subsequent category determination. Finally, the system can generate a multi-class probability map of the heart based on the second feature map. This probability map is a three-dimensional voxel probability map, where each voxel corresponds to the probability value of each part of the whole heart anatomical structure, clearly reflecting the possibility that each voxel belongs to a different heart substructure. Subsequently, the system can process the multi-class probability map using the argmax function, selecting the category with the largest probability value for each voxel as the category to which the voxel belongs, and then converting the multi-class probability map into a set of heart multi-class segmentation labels. These segmentation labels can accurately identify the voxel regions where each part of the whole heart anatomical structure (such as the atrium, ventricle, etc.) is located, completing the entire whole heart segmentation process and providing structured and quantitative reference for clinical diagnosis.

[0067] In one feasible implementation, in order to improve the segmentation accuracy and efficiency of the image segmentation model, before performing image segmentation on the target cardiac medical image data to be segmented, the system can iteratively train the initial image segmentation model using a training sample set to obtain a high-precision image segmentation model.

[0068] Specifically, see Figure 6 The figure is a flowchart of the training process of an image segmentation model provided in an embodiment of this application. The process includes the following steps: Step S601: Obtain the training sample set for training the initial image segmentation model.

[0069] In step S601, the training sample set includes multiple training samples, each of which includes cardiac medical image data with labeled data. The label data includes the actual location bounding box used to identify the cardiac region in the cardiac medical image data, as well as the actual cardiac multi-class segmentation label corresponding to the cardiac medical image data.

[0070] Step S602: Input each training sample into the initial image segmentation model in sequence, use the target detection module in the image segmentation model to extract features from the cardiac medical image data, obtain the target three-dimensional multi-scale feature map, and set multiple target anchor boxes of different sizes on the target three-dimensional multi-scale feature map.

[0071] In this embodiment, the system utilizes a 3D backbone network in the target detection module to extract multiple semantic feature maps from cardiac medical imaging data, with resolutions ranging from high to low and semantic information from shallow to deep. High-resolution semantic feature maps focus on capturing local details of cardiac anatomy (such as edges and fine tissue contours), while low-resolution semantic feature maps focus on representing global semantic information of the heart as a whole (such as the overall morphology and spatial relationships of the cardiac anatomy). Subsequently, to fully integrate the advantages of feature maps at different scales and overcome the limitations of single-scale feature maps in information capture, the system utilizes a feature pyramid network in the target detection module to perform deep feature fusion on the extracted semantic feature maps of different resolutions and semantic levels. Through the fusion processing of the feature pyramid network, the local details of the high-resolution feature maps are organically combined with the global semantic information of the low-resolution feature maps, ultimately generating a target 3D multi-scale feature map that combines global constraints with local details. Then, the system uses the target detection module to set multiple 3D anchor boxes (i.e., target anchor boxes) of different sizes on the target 3D multi-scale feature map.

[0072] Step S603: Calculate the intersection-union ratio (IUU) between the actual location bounding box and each target anchor box using the target detection module, and obtain the IUU corresponding to each target anchor box.

[0073] Step S604: Using the target detection module, target anchor boxes with an intersection-union ratio greater than or equal to a preset threshold are marked as positive samples, and target anchor boxes with an intersection-union ratio less than the preset threshold are marked as negative samples.

[0074] Step S605: Based on positive samples, negative samples, cardiac medical image data, target three-dimensional multi-scale feature map, actual location bounding box, and actual heart multi-class segmentation label, the initial image segmentation model is iteratively trained to obtain the image segmentation model.

[0075] In this embodiment, the system can iteratively train an initial image segmentation model based on positive samples, negative samples, cardiac medical image data, target three-dimensional multi-scale feature maps, actual location bounding boxes, and actual heart multi-class segmentation labels. This training process is simple and efficient. Based on various types of sample data and standard annotations, the model parameters are continuously optimized to improve the model's localization accuracy and segmentation precision for the heart region, so as to obtain an image segmentation model that meets the needs of clinical applications.

[0076] It should be noted that negative samples only participate in classification (i.e., the background class), and their corresponding predicted offset and pixel-level segmentation mask are both 0.

[0077] See Figure 7 The figure is a flowchart of the training process of another image segmentation model provided in an embodiment of this application. The process includes the following steps: Step S6051: For each positive sample, the target detection module calculates the predicted offset and target probability value corresponding to the positive sample, and adjusts the geometric parameters of the corresponding target anchor box based on the predicted offset to fit the true boundary of the whole-heart anatomical structure and generate the corresponding target predicted bounding box.

[0078] In step S6051, the target probability value corresponding to the positive sample is used to characterize the probability that the target anchor frame contains the whole heart anatomical structure.

[0079] Step S6052: Using the target detection module, calculate the actual offset between the target anchor box and the actual location bounding box, and determine the bounding box regression loss value based on the error between the predicted offset and the actual offset corresponding to the positive sample.

[0080] In this embodiment, the system can use the target detection module to calculate the actual offset between the target anchor box and the actual location bounding box, and determine the bounding box regression loss value L based on the error between the predicted offset and the actual offset corresponding to the positive sample. box .

[0081] Step S6053: Using the target detection module, multiple target candidate bounding boxes are selected from all target prediction bounding boxes based on the target probability value corresponding to each positive sample. The multiple target candidate bounding boxes are then filtered based on the target three-dimensional multi-scale feature map to obtain the target heart location bounding box.

[0082] Step S6054: Using the target detection module, determine the classification loss value based on the error between the target heart location bounding box and the actual location bounding box.

[0083] In this embodiment, the system can use the target detection module to determine the classification loss value L based on the error between the target heart location bounding box and the actual location bounding box. cls .

[0084] Step S6055: Based on the bounding box regression loss value, classification loss value, cardiac medical image data, and actual heart multi-class segmentation labels, the initial image segmentation model is iteratively trained to obtain the image segmentation model.

[0085] In this embodiment, the system can iteratively train an initial image segmentation model based on bounding box regression loss value, classification loss value, cardiac medical imaging data, and actual cardiac multi-class segmentation labels. This training optimizes model parameters based on loss values ​​and standard annotations, improves model localization and segmentation accuracy, and obtains an image segmentation model that meets clinical application needs.

[0086] Specifically, see Figure 8The figure is a flowchart of the training process of another image segmentation model provided in the embodiment of this application, which includes the following steps: Step S60551: Using the target detection module, the corresponding region of interest is extracted from the cardiac medical image data based on the target heart location bounding box to obtain the corresponding image block.

[0087] Step S60552: The image patch is segmented using the heart segmentation module to generate a set of heart multi-class segmentation prediction labels, and the segmentation loss value is determined based on the error between the heart multi-class segmentation prediction labels and the actual heart multi-class segmentation labels.

[0088] In this embodiment, the system can use a heart segmentation module to segment image blocks, generate a set of heart multi-class segmentation prediction labels, and determine the segmentation loss value L based on the error between the heart multi-class segmentation prediction labels and the actual heart multi-class segmentation labels. seg .

[0089] Step S60553: Using the edge loss head in the initial image segmentation model, perform convolution operations on the actual heart multi-class segmentation label and the heart multi-class segmentation prediction label respectively to extract the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation prediction label.

[0090] In step S60553, edge information is used to reflect the boundary differences between the heart and the background area, as well as the boundary differences between various parts within the whole-heart anatomical structure.

[0091] To accelerate convergence and optimize edge segmentation, this application introduces an edge-loss head into the image segmentation model during training. The system can utilize this edge-loss head to apply a fixed 3D Sobel operator (i.e.,... Figure 9 The 3D Sobel kernel shown contains six directions (±x, y, and z) that perform convolution operations on the actual heart multi-class segmentation labels and the predicted heart multi-class segmentation labels, respectively. It extracts the edge information of the actual heart multi-class segmentation labels and the predicted heart multi-class segmentation labels. This edge information reflects the boundary differences between the heart and the background region, as well as the boundary differences between different parts within the whole heart anatomical structure. This forces the model to pay more attention to the edges between the heart organ and the background, thereby improving the model's segmentation accuracy.

[0092] Step S60554: Using the edge loss head in the initial image segmentation model, determine the edge loss value based on the error between the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation predicted label.

[0093] In this embodiment, the system can utilize the edge loss head in the initial image segmentation model to calculate the L2 distance (i.e., Euclidean distance) between the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation predicted label, and use this distance as the loss value L of the 3D edge loss. edge (i.e., edge loss value).

[0094] Step S60555: Based on the bounding box regression loss value, classification loss value, segmentation loss value, and edge loss value, adjust the parameters of the initial 3D image segmentation model, and iteratively train the adjusted model using unused training samples from the training sample set until the training cutoff condition is met, thereby obtaining the 3D image segmentation model.

[0095] In this embodiment of the application, the system can calculate the total loss value based on the bounding box regression loss value, classification loss value, segmentation loss value and edge loss value using the total loss function Loss shown in formula (1), and adjust the parameters of the initial three-dimensional image segmentation model based on the total loss value; then the system can iteratively train the adjusted model using unused training samples in the training sample set until the training cutoff condition is met, and obtain the three-dimensional image segmentation model.

[0096] (1)

[0097] Among them, L box L represents the bounding box regression loss value; cls L represents the classification loss value; seg L represents the segmentation loss value. edge The values ​​represent the edge loss values; w1, w2, w3, and w4 represent the weight ratios, which can be set to 2:2:2:1.

[0098] The medical image segmentation method provided in this application, through an improved 3D target detection network and an optimized 3D segmentation network, automates the entire process of cardiac medical image data from coarse-grained localization to voxel-level segmentation. The whole heart segmentation result can be obtained with only one detection and segmentation inference, without the need to cut and stitch the entire CT roll, which significantly reduces inference time and computational cost, and effectively improves image segmentation efficiency and accuracy.

[0099] Device Examples

[0100] This application provides a medical image segmentation device, wherein, Figure 10 This is a schematic diagram of the structure of a medical image segmentation device provided in an embodiment of this application, as shown below. Figure 10As shown, the device includes: an acquisition module 11, a target detection module 12, an image cropping module 13, and an image segmentation module 14. From Figure 10 You can see the connections between several modules.

[0101] Among them, the acquisition module 11 is used to acquire the target cardiac medical image data to be segmented; the target cardiac medical image data is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart. The target detection module 12 is used to extract features from the target cardiac medical image data using the target detection module in the pre-trained image segmentation model, to obtain a three-dimensional multi-scale feature map, and to generate multiple candidate bounding boxes corresponding to the cardiac region based on the three-dimensional multi-scale feature map; the three-dimensional multi-scale feature map is a cross-scale feature representation of the target cardiac medical image data from global semantics to local details; The image cropping module 13 is used to filter out the heart location bounding box containing the whole heart anatomical structure from multiple candidate bounding boxes by the target detection module, and crop the corresponding region of interest from the target cardiac medical image data based on the heart location bounding box to obtain the target image patch; Image segmentation module 14 is used to segment the target image block using the heart segmentation module in the image segmentation model and generate segmentation results. The segmentation results include a set of heart multi-class segmentation labels, which are used to identify the voxel regions where each part of the whole heart anatomical structure is located.

[0102] Optionally, the object detection module consists of an improved 3D fast region convolutional neural network, which includes a 3D backbone network, a feature pyramid network, and a region proposal network.

[0103] Optionally, the target detection module includes: The first extraction unit is used to input the target cardiac medical image data into the target detection module and use a three-dimensional backbone network to extract multiple semantic feature maps from the target cardiac medical image data layer by layer with resolution from high to low. The feature fusion unit is used to fuse multiple semantic feature maps using a feature pyramid network to generate a three-dimensional multi-scale feature map.

[0104] Optionally, the target detection module further includes: The first computational unit is used to set multiple anchor boxes of different sizes on the three-dimensional multi-scale feature map using the region proposal network of the target detection module, and to calculate the target probability value and prediction offset corresponding to each anchor box. The target probability value is used to characterize the probability that the anchor box contains the whole-heart anatomical structure, and the prediction offset is used to adjust the geometric parameters of the anchor box to fit the true boundary of the whole-heart anatomical structure. The first generation unit is used to generate a corresponding predicted bounding box for each anchor box by adjusting the geometric parameters of the anchor box based on the predicted offset of the anchor box by the region proposal network. The sorting unit is used by the region proposal network to sort all predicted bounding boxes from largest to smallest based on the target probability values ​​corresponding to each predicted bounding box, and obtain the sorting result. The first execution unit is used by the region proposal network to perform non-maximum suppression on the N predicted bounding boxes based on the target probability values ​​corresponding to the top N predicted bounding boxes in the ranking results, so as to filter out multiple candidate bounding boxes; N is a positive integer.

[0105] Optionally, the first execution unit includes: The building unit is used to construct a candidate box list based on all predicted bounding boxes using the region proposal network; The second execution unit is used to perform the intersection-union ratio (IURR) calculation operation by the region proposal network. The IURR calculation operation is used to take any bounding box in the candidate box list as the target bounding box and calculate the IURR between the target bounding box and other bounding boxes in the candidate box list. The other bounding boxes are any bounding boxes in the candidate box list other than the target bounding box. The IURR is used to reflect the degree of overlap between the target bounding box and other bounding boxes. The third execution unit is used to perform bounding box culling operation by the region proposal network if the intersection-union ratio is greater than or equal to a preset threshold. The bounding box culling operation is used to compare the target probability value of the target bounding box with the target probability value of other bounding boxes, and remove the bounding box with the smallest target probability value from the candidate box list. The fourth execution unit is used to perform a bounding box retention operation by the region proposal network if the intersection-union ratio is less than a preset threshold, so as to retain the other bounding box in the candidate box list. The fifth execution unit is used to repeatedly perform the intersection-union calculation, bounding box elimination, and bounding box retention operations by the region proposal network until all predicted bounding boxes have been compared, resulting in a list of target candidate boxes. Each bounding box in the target candidate box list is then used as a candidate bounding box.

[0106] Optionally, the image cropping module includes: The filtering unit is used to filter multiple candidate bounding boxes based on a three-dimensional multi-scale feature map using a region proposal network to obtain the filtering results. The filtering process is used to select bounding boxes from the candidate bounding boxes whose target probability value is greater than a preset probability threshold and which completely cover the whole heart anatomical structure. The first determining unit is used to determine the heart location bounding box if the filtering results include only one bounding box. The second determining unit is used to select the bounding box with the highest target probability value in the filtering results as the heart location bounding box if the filtering results include at least two bounding boxes.

[0107] Optionally, the image segmentation module includes: The second extraction unit is used to input the target image block into the heart segmentation module. Using the encoder in the heart segmentation module, the target image block is extracted layer by layer along multiple resolution levels with decreasing spatial resolution to obtain a set of multi-scale feature maps containing semantic information at different scales. The upsampling unit is used to upsample the first feature map layer by layer by the decoder of the heart segmentation module, and fuse the feature map after each upsampling layer with the feature map of the corresponding scale output by the encoder to obtain a second feature map with the same size as the target image block; the first feature map is the feature map output by the last resolution level of the encoder. The transformation unit is used to generate a cardiac multi-class probability map based on the second feature map, and to convert the class probability map into a set of cardiac multi-class segmentation labels through the argmax function.

[0108] Optionally, the medical image segmentation device further includes: The sample acquisition module is used to acquire a training sample set for training the initial image segmentation model. The training sample set includes multiple training samples, each of which includes cardiac medical image data with labeled data. The label data includes the actual location bounding box used to identify the cardiac region in the cardiac medical image data, as well as the actual multi-class segmentation label of the cardiac medical image data. The feature extraction module is used to input each training sample into the initial image segmentation model in sequence, and use the target detection module in the image segmentation model to extract features from cardiac medical image data to obtain a three-dimensional multi-scale feature map of the target, and set multiple target anchor boxes of different sizes on the three-dimensional multi-scale feature map of the target. The calculation module is used to calculate the intersection-union ratio (IU) between the actual location bounding box and each target anchor box using the target detection module, and obtain the IU corresponding to each target anchor box. The determination module is used to mark target anchor boxes with an intersection-union ratio greater than or equal to a preset threshold as positive samples and target anchor boxes with an intersection-union ratio less than the preset threshold as negative samples using the target detection module; The iterative training module is used to iteratively train the initial image segmentation model based on positive samples, negative samples, cardiac medical image data, target 3D multi-scale feature maps, actual location bounding boxes, and actual heart multi-class segmentation labels to obtain the image segmentation model.

[0109] Optionally, the iterative training module includes: The second calculation unit is used to calculate the predicted offset and target probability value corresponding to each positive sample using the target detection module, and adjust the geometric parameters of the corresponding target anchor box based on the predicted offset to fit the true boundary of the whole-heart anatomical structure and generate the corresponding target predicted bounding box; the target probability value corresponding to the positive sample is used to characterize the probability that the target anchor box contains the whole-heart anatomical structure. The bounding box regression loss value determination unit is used to calculate the actual offset between the target anchor box and the actual location bounding box using the target detection module, and determine the bounding box regression loss value based on the error between the predicted offset and the actual offset corresponding to the positive sample. The processing unit is used to use the target detection module to filter out multiple target candidate bounding boxes from all target predicted bounding boxes based on the target probability value corresponding to each positive sample, and to filter the multiple target candidate bounding boxes based on the target three-dimensional multi-scale feature map to obtain the target heart location bounding box; The classification loss value determination unit is used to determine the classification loss value based on the error between the target heart location bounding box and the actual location bounding box using the target detection module; The iterative training unit is used to iteratively train the initial image segmentation model based on the bounding box regression loss value, classification loss value, cardiac medical image data, and actual heart multi-class segmentation labels to obtain the image segmentation model.

[0110] Optionally, the iterative training unit also includes: The image cropping unit is used to use the target detection module to crop the corresponding region of interest from cardiac medical image data based on the target heart location bounding box, and obtain the corresponding image patch; The segmentation loss value determination unit is used to segment the image patch using the heart segmentation module, generate a set of heart multi-class segmentation prediction labels, and determine the segmentation loss value based on the error between the heart multi-class segmentation prediction labels and the actual heart multi-class segmentation labels. The edge information extraction unit is used to perform convolution operations on the actual heart multi-class segmentation label and the heart multi-class segmentation prediction label using the edge loss head in the initial image segmentation model, respectively, to extract the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation prediction label; the edge information is used to reflect the boundary difference between the heart and the background region, as well as the boundary difference between various parts within the whole heart anatomical structure. The edge loss value determination unit is used to determine the edge loss value based on the error between the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation predicted label, using the edge loss head in the initial image segmentation model. The parameter adjustment unit is used to adjust the parameters of the initial 3D image segmentation model based on the bounding box regression loss value, classification loss value, segmentation loss value, and edge loss value. It then iteratively trains the adjusted model using unused training samples from the training sample set until the training cutoff condition is met, thus obtaining the 3D image segmentation model.

[0111] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0112] The above is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for segmenting medical images, characterized in that, include: Acquire the target cardiac medical image data to be segmented; the target cardiac medical image data is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart. The target detection module in the pre-trained image segmentation model is used to extract features from the target cardiac medical image data to obtain a three-dimensional multi-scale feature map. Based on the three-dimensional multi-scale feature map, multiple candidate bounding boxes corresponding to the cardiac region are generated. The three-dimensional multi-scale feature map is a cross-scale feature representation of the target cardiac medical image data from global semantics to local details. The target detection module filters out the heart location bounding box containing the whole heart anatomical structure from multiple candidate bounding boxes, and extracts the corresponding region of interest from the target heart medical image data based on the heart location bounding box to obtain the target image block; The target image block is segmented using the heart segmentation module in the image segmentation model to generate a segmentation result. The segmentation result includes a set of heart multi-class segmentation labels, which are used to identify the voxel regions where each part of the whole heart anatomical structure is located.

2. The method according to claim 1, characterized in that, The target detection module is composed of an improved 3D fast region convolutional neural network, which includes a 3D backbone network, a feature pyramid network, and a region proposal network. The target detection module in the pre-trained image segmentation model is used to extract features from the target cardiac medical image data to obtain a three-dimensional multi-scale feature map, including: The target cardiac medical image data is input into the target detection module, and the three-dimensional backbone network is used to extract multiple semantic feature maps with resolution from high to low from the target cardiac medical image data layer by layer. The feature pyramid network is used to fuse the multiple semantic feature maps to generate the three-dimensional multi-scale feature map.

3. The method according to claim 2, characterized in that, The step of generating multiple candidate bounding boxes corresponding to the heart region based on the three-dimensional multi-scale feature map includes: Using the region proposal network of the target detection module, multiple anchor boxes of different sizes are set on the three-dimensional multi-scale feature map, and the target probability value and prediction offset corresponding to each anchor box are calculated; the target probability value is used to characterize the probability that the anchor box contains the whole-heart anatomical structure, and the prediction offset is used to adjust the geometric parameters of the anchor box to fit the true boundary of the whole-heart anatomical structure; For each anchor frame, the region proposal network adjusts the geometric parameters of the anchor frame based on the predicted offset corresponding to the anchor frame to generate the corresponding predicted bounding box; The region proposal network sorts all the predicted bounding boxes from largest to smallest based on the target probability values ​​corresponding to each predicted bounding box, and obtains the sorting result. The region suggestion network performs non-maximum suppression on the N predicted bounding boxes based on the target probability values ​​corresponding to the top N predicted bounding boxes in the ranking results, in order to filter out multiple candidate bounding boxes; N is a positive integer.

4. The method according to claim 3, characterized in that, The region proposal network performs non-maximum suppression on the N predicted bounding boxes based on the target probability values ​​corresponding to the top N predicted bounding boxes in the ranking result, in order to filter out multiple candidate bounding boxes, including: The region proposal network is used to construct a candidate box list based on all the predicted bounding boxes; The region suggestion network performs an intersection-union ratio (IUGR) calculation operation; the IUGR calculation operation is used to take any bounding box in the candidate box list as the target bounding box, and calculate the IUGR between the target bounding box and other bounding boxes in the candidate box list, wherein the other bounding boxes are any bounding boxes in the candidate box list other than the target bounding box, and the IUGR is used to reflect the degree of overlap between the target bounding box and the other bounding boxes; If the intersection-union ratio is greater than or equal to a preset threshold, the region suggestion network performs a bounding box culling operation; the bounding box culling operation is used to compare the target probability value of the target bounding box with the target probability values ​​of other bounding boxes, and remove the bounding box with the smallest target probability value from the candidate box list. If the intersection-union ratio is less than the preset threshold, the region suggestion network performs a bounding box retention operation to retain the other bounding box in the candidate box list. The region suggestion network repeatedly performs the intersection-union calculation operation, the bounding box elimination operation, and the bounding box retention operation until all the predicted bounding boxes have been compared, thereby obtaining a target candidate box list, and each bounding box in the target candidate box list is taken as a candidate bounding box.

5. The method according to claim 4, characterized in that, The step of filtering the heart location bounding box containing the whole heart anatomical structure from multiple candidate bounding boxes by the target detection module includes: Using the region proposal network, multiple candidate bounding boxes are filtered based on the three-dimensional multi-scale feature map to obtain the filtering results; the filtering process is used to select bounding boxes from the candidate bounding boxes whose target probability value is greater than a preset probability threshold and completely covers the whole-heart anatomical structure. If the filtering results include only one bounding box, then that bounding box is used as the bounding box for the heart location. If the filtering results include at least two bounding boxes, then the bounding box with the highest target probability value in the filtering results shall be used as the heart location bounding box.

6. The method according to claim 1, characterized in that, The step of segmenting the target image block using the heart segmentation module in the image segmentation model to generate segmentation results includes: The target image block is input into the heart segmentation module. The encoder in the heart segmentation module extracts features from the target image block layer by layer along multiple resolution levels with decreasing spatial resolution to obtain a set of multi-scale feature maps containing semantic information at different scales. The decoder of the heart segmentation module upsamples the first feature map layer by layer, and fuses the feature map after each upsampled layer with the feature map of the corresponding scale output by the encoder to obtain a second feature map with the same size as the target image block; the first feature map is the feature map output by the last resolution level of the encoder; A multi-class probability map of the heart is generated based on the second feature map, and the class probability map is converted into a set of multi-class segmentation labels of the heart using the argmax function.

7. The method according to claim 1, characterized in that, The image segmentation model is obtained through the following process: Obtain a training sample set for training an initial image segmentation model; the training sample set includes multiple training samples, each of which includes cardiac medical image data with labeled data, the label data including the actual location bounding box for identifying the cardiac region in the cardiac medical image data, and the actual cardiac multi-class segmentation label corresponding to the cardiac medical image data; Each of the training samples is sequentially input into the initial image segmentation model. The target detection module in the image segmentation model is used to extract features from the cardiac medical image data to obtain a three-dimensional multi-scale feature map of the target. Multiple target anchor boxes of different sizes are set on the three-dimensional multi-scale feature map of the target. The intersection-union ratio (CUI) between the actual location bounding box and each of the target anchor boxes is calculated using the target detection module to obtain the CUI corresponding to each of the target anchor boxes. The target detection module marks target anchor boxes with an intersection-union ratio (IU) greater than or equal to a preset threshold as positive samples, and targets with an IU less than the preset threshold as negative samples. Based on the positive samples, the negative samples, the cardiac medical image data, the target three-dimensional multi-scale feature map, the actual location bounding box, and the actual heart multi-class segmentation label, the initial image segmentation model is iteratively trained to obtain the image segmentation model.

8. The method according to claim 7, characterized in that, The process of iteratively training the initial image segmentation model based on the positive samples, the negative samples, the cardiac medical image data, the target 3D multi-scale feature map, the actual location bounding box, and the actual cardiac multi-class segmentation label to obtain the image segmentation model includes: For each positive sample, the target detection module calculates the predicted offset and target probability value corresponding to the positive sample, and adjusts the geometric parameters of the corresponding target anchor box based on the predicted offset to fit the true boundary of the whole-heart anatomical structure and generate the corresponding target predicted bounding box; the target probability value corresponding to the positive sample is used to characterize the probability that the whole-heart anatomical structure is contained in the target anchor box; Using the target detection module, the actual offset between the target anchor box and the actual position bounding box is calculated, and the bounding box regression loss value is determined based on the error between the predicted offset corresponding to the positive sample and the actual offset. Using the target detection module, multiple target candidate bounding boxes are selected from all the target predicted bounding boxes based on the target probability value corresponding to each positive sample, and the multiple target candidate bounding boxes are filtered based on the target three-dimensional multi-scale feature map to obtain the target heart location bounding box; Using the target detection module, a classification loss value is determined based on the error between the bounding box of the target heart location and the actual location bounding box; Based on the bounding box regression loss value, the classification loss value, the cardiac medical image data, and the actual heart multi-class segmentation label, the initial image segmentation model is iteratively trained to obtain the image segmentation model.

9. The method according to claim 8, characterized in that, The step of iteratively training the initial image segmentation model based on the bounding box regression loss value, the classification loss value, the cardiac medical imaging data, and the actual heart multi-class segmentation labels to obtain the image segmentation model includes: Using the target detection module, the corresponding region of interest is extracted from the cardiac medical image data based on the target heart location bounding box to obtain the corresponding image patch; The image block is segmented using the heart segmentation module to generate a set of heart multi-class segmentation prediction labels, and the segmentation loss value is determined based on the error between the heart multi-class segmentation prediction labels and the actual heart multi-class segmentation labels. Using the edge loss head in the initial image segmentation model, convolution operations are performed on the actual heart multi-class segmentation label and the heart multi-class segmentation prediction label to extract the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation prediction label; the edge information is used to reflect the boundary difference between the heart and the background region, as well as the boundary difference between various parts within the whole heart anatomical structure. Using the edge loss head in the initial image segmentation model, the edge loss value is determined based on the error between the edge information of the actual heart multi-class segmentation label and the edge information of the heart multi-class segmentation prediction label; Based on the bounding box regression loss value, the classification loss value, the segmentation loss value, and the edge loss value, the parameters of the initial 3D image segmentation model are adjusted, and the adjusted model is iteratively trained using unused training samples from the training sample set until the training cutoff condition is met, thereby obtaining the 3D image segmentation model.

10. A medical image segmentation device, characterized in that, include: The acquisition module is used to acquire the target cardiac medical image data to be segmented; the target cardiac medical image data is three-dimensional voxel data of the local thoracic anatomical structure centered on the heart. The target detection module is used to extract features from the target cardiac medical image data using the target detection module in the pre-trained image segmentation model, to obtain a three-dimensional multi-scale feature map, and to generate multiple candidate bounding boxes corresponding to the cardiac region based on the three-dimensional multi-scale feature map; the three-dimensional multi-scale feature map is a cross-scale feature representation of the target cardiac medical image data from global semantics to local details; The image cropping module is used by the target detection module to filter out the heart location bounding box containing the whole heart anatomical structure from multiple candidate bounding boxes, and to crop the corresponding region of interest from the target heart medical image data based on the heart location bounding box to obtain the target image block; The image segmentation module is used to segment the target image block using the heart segmentation module in the image segmentation model and generate segmentation results. The segmentation results include a set of heart multi-class segmentation labels, which are used to identify the voxel regions where each part of the whole heart anatomical structure is located.

Images