Three-dimensional MRI tumor segmentation method based on boundary perception and tumor diagnosis equipment

Abstract

The invention provides a three-dimensional MRI (Magnetic Resonance Imaging) tumor segmentation method based on boundary perception and tumor diagnosis equipment. The method comprises the following steps: acquiring three-dimensional MRI (Magnetic Resonance Imaging) of multiple tumor patients as a tumor data set, wherein the three-dimensional MRI comprises gadolinium-enhanced T1SC slices; performing tumor segmentation by taking the three-dimensional MRI in the tumor data set as input of a first segmentation model to obtain a category label of a tumor in the three-dimensional MRI; wherein the first segmentation model is a model based on a convolutional neural network CNN, the first segmentation model comprises a first model branch and a second model branch, the first model branch is used for predicting a category label, and the second model branch is used for predicting a surface distance field of a tumor based on boundary perception; the surface distance field is used for describing the nearest distance from each voxel to the tumor surface in the three-dimensional MRI, and training and optimization of the first segmentation model are carried out based on training and optimization of the first model branch and the second model branch.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to a three-dimensional MRI tumor segmentation method and tumor diagnosis equipment based on boundary perception. Background Art

[0002] Accurate morphological quantification of tumors, including their size, location, and type, is expected to enhance monitoring programs and optimize treatment planning strategies. In the past few years, a series of complex tumor segmentation methods based on convolution, attention, and large models have been proposed. These methods have performed well on various medical images, thanks to the availability of large-scale datasets. However, these methods still face great challenges in segmenting certain tumors (such as spinal cord tumors) and have low accuracy. Therefore, for these tumors, there is still a lack of automatic models that can accurately segment and identify tumors. Summary of the invention

[0003] The embodiments of the present application provide a three-dimensional MRI tumor segmentation method and tumor diagnosis equipment based on boundary perception.

[0004] In a first aspect, an embodiment of the present application provides a three-dimensional MRI tumor segmentation method based on boundary perception, the method comprising:

[0005] Acquire three-dimensional MRI of multiple tumor patients as a tumor data set, and the three-dimensional MRI includes gadolinium-enhanced T1SC slices;

[0006] The three-dimensional MRI in the tumor data set is used as the input of the first segmentation model to perform tumor segmentation, and obtain the category label of the tumor in the three-dimensional MRI;

[0007] Among them, the first segmentation model is a model based on a convolutional neural network CNN, and the first segmentation model includes a first model branch and a second model branch. The first model branch is used to predict category labels, and the second model branch is used to predict the surface distance field of the tumor based on boundary perception. The surface distance field is used to describe the closest distance from each voxel in the three-dimensional MRI to the tumor surface. The training and optimization of the first segmentation model are based on the training and optimization of the first model branch and the second model branch.

[0008] Optionally, the second model branch is trained and optimized based on a first boundary-aware loss function, which is used to determine the boundary-aware loss of each voxel in the three-dimensional MRI, and the boundary-aware loss of each voxel is determined based on the surface distance value of each voxel, and the surface distance value is used to represent the closest distance of each voxel to the tumor surface.

[0009] Optionally, the voxels include voxels outside the tumor and voxels inside the tumor, the surface distance values ​​of the voxels outside the tumor are represented as negative values, and the surface distance values ​​of the voxels inside the tumor are represented as positive values.

[0010] Optionally, if the absolute value of the surface distance value of the voxel outside the tumor is greater than the maximum surface distance value of the voxel inside the tumor, the surface distance value is counted as 0.

[0011] Optionally, a boundary-aware loss for each voxel is determined based on a normalized value of the surface distance value of each voxel.

[0012] Optionally, the training and optimization of the second model branch are performed according to the type of tumor.

[0013] Optionally, the first boundary-aware loss function is expressed as follows:

[0014]

[0015] Among them, l ba is used to represent the boundary perception loss, h is used to represent the number of horizontal pixels in 3D MRI, w is used to represent the number of vertical pixels in 3D MRI, d is used to represent the number of slices in 3D MRI, k is used to represent the total number of categories in 3D MRI, and f (h，w，d，k) The surface distance value of the (h, w, d, k)th voxel predicted by the second model branch is used to represent the surface distance value of the (h, w, d, k)th voxel, Used to represent the true surface distance value of the (h, w, d, k)th voxel.

[0016] Optionally, the second model branch is trained and optimized based on the average of the boundary-aware losses of all voxels in the 3D MRI.

[0017] Optionally, the first model branch is trained and optimized based on a first segmentation loss function, and the first segmentation loss function includes a cross entropy loss function and a dice loss function.

[0018] In a second aspect, an embodiment of the present application provides a tumor diagnosis device, comprising:

[0019] An acquisition module, used for acquiring three-dimensional MRI of multiple tumor patients as a tumor data set, wherein the three-dimensional MRI includes gadolinium-enhanced T1SC slices;

[0020] A segmentation module, used for performing tumor segmentation by taking the three-dimensional MRI in the tumor data set as the input of the first segmentation model, and obtaining a category label of the tumor in the three-dimensional MRI;

[0021] Among them, the first segmentation model is a model based on a convolutional neural network CNN, and the first segmentation model includes a first model branch and a second model branch. The first model branch is used to predict category labels, and the second model branch is used to predict the surface distance field of the tumor based on boundary perception. The surface distance field is used to describe the closest distance from each voxel in the three-dimensional MRI to the tumor surface. The training and optimization of the first segmentation model are based on the training and optimization of the first model branch and the second model branch.

[0022] Optionally, the second model branch is trained and optimized based on a first boundary-aware loss function, which is used to determine the boundary-aware loss of each voxel in the three-dimensional MRI, and the boundary-aware loss of each voxel is determined based on the surface distance value of each voxel, and the surface distance value is used to represent the closest distance of each voxel to the tumor surface.

[0023] Optionally, the voxels include voxels outside the tumor and voxels inside the tumor, the surface distance values ​​of the voxels outside the tumor are represented as negative values, and the surface distance values ​​of the voxels inside the tumor are represented as positive values.

[0024] Optionally, if the absolute value of the surface distance value of the voxel outside the tumor is greater than the maximum surface distance value of the voxel inside the tumor, the surface distance value is counted as 0.

[0025] Optionally, a boundary-aware loss for each voxel is determined based on a normalized value of the surface distance value of each voxel.

[0026] Optionally, the training and optimization of the second model branch are performed according to the type of tumor.

[0027] Optionally, the first boundary-aware loss function is expressed as follows:

[0028]

[0029] Among them, l ba is used to represent the boundary perception loss, h is used to represent the number of horizontal pixels in 3D MRI, w is used to represent the number of vertical pixels in 3D MRI, d is used to represent the number of slices in 3D MRI, k is used to represent the total number of categories in 3D MRI, and f (h，w，d，k) The surface distance value of the (h, w, d, k)th voxel predicted by the second model branch is used to represent the surface distance value of the (h, w, d, k)th voxel, Used to represent the true surface distance value of the (h, w, d, k)th voxel.

[0030] Optionally, the second model branch is trained and optimized based on the average of the boundary-aware losses of all voxels in the 3D MRI.

[0031] Optionally, the first model branch is trained and optimized based on a first segmentation loss function, and the first segmentation loss function includes a cross entropy loss function and a dice loss function.

[0032] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, at least one processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method as described in any one of the above-mentioned first aspects is implemented.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the above-mentioned first aspects is implemented.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any one of the methods described in the first aspect.

[0035] In the present application, the segmentation of the tumor in the three-dimensional MRI by the first segmentation model can be divided into the prediction of the tumor category by the first model branch and the prediction of the distance field of the tumor surface based on the boundary perception by the second model branch. Based on the second model branch, the three-dimensional surface of the tumor in the three-dimensional MRI can be perceived, thereby capturing the details of the tumor boundary and the surrounding area, which helps to improve the accuracy of tumor segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for describing the embodiments are briefly introduced below.

[0037] Figure 1 It is a schematic diagram of MRI of different tumors / organs provided in the embodiments of the present application.

[0038] Figure 2 It is a flowchart of a boundary-aware three-dimensional MRI tumor segmentation method provided in an embodiment of the present application.

[0039] Figure 3 It is a structural diagram of a first segmentation model provided in an embodiment of the present application.

[0040] Figure 4 It is a structural diagram of a first segmentation model provided in another embodiment of the present application.

[0041] Figure 5 This is an example diagram of the tumor surface distance field provided in an embodiment of the present application.

[0042] Figure 6 It is a structural diagram of a two-stage training baseline provided in an embodiment of the present application.

[0043] Figure 7A-7D It is a schematic diagram of the results of a tumor diagnosis experiment provided in one embodiment of the present application.

[0044] Figure 8 It is a schematic diagram of the results of a tumor diagnosis experiment provided in another embodiment of the present application.

[0045] Fig. 9 It is a schematic diagram of the structure of the tumor diagnostic device provided in the embodiment of the present application.

[0046] Fig.10 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solution of the embodiment of the present invention will be described below in conjunction with the accompanying drawings in the embodiment of the present invention.

[0048] Accurate morphological quantification of tumors, including their size, location, and type, is expected to enhance monitoring programs and optimize treatment planning strategies. In the past few years, a series of complex tumor segmentation methods based on convolution, attention, and large models have been proposed. These methods have performed well on various medical images, thanks to the availability of large-scale datasets. However, these methods still face great challenges in segmenting certain tumors (such as spinal cord tumors) and have low accuracy. Therefore, for these tumors, there is still a lack of automatic models that can accurately segment and identify tumors.

[0049] Take spinal cord tumors as an example. Spinal cord tumors are an important factor leading to neurological morbidity and mortality. The spinal cord is a key component of the central nervous system and is essential for somatic sensory and motor function. Common types of spinal cord tumors include astrocytomas, ependymomas, hemangioblastomas, and meningiomas. Spinal cord tumors are mainly identified by three-dimensional magnetic resonance imaging (MRI) scans. In the clinical field, distinguishing between different types of spinal cord tumors remains a daunting challenge, which may lead to misdiagnosis, failure to monitor tumor growth, and subsequent delays in therapeutic intervention.

[0050] The main reason why spinal cord tumors are difficult to accurately segment using the above segmentation methods is that spinal cord tumors often have small sizes, different locations and shapes. However, the above segmentation methods mainly focus on identifying shapes with relatively large shapes, such as brain tumors, left / right ventricles, abdominal organs, etc., while ignoring the challenging problem of spinal cord tumor identification. Figure 1As shown in Figure 1, common spinal cord tumors located in the long cylindrical spinal cord tend to have tiny sizes, different locations, and shape variations, while other tumors / organs show relatively uniform and larger sizes. This means that simply applying the above segmentation methods to spinal cord tumor segmentation will result in low accuracy due to the obvious anatomical differences between spinal cord tumors and other body organs / tumors.

[0051] Based on this, Figure 2 As shown, an embodiment of the present application provides a three-dimensional MRI tumor segmentation method based on boundary perception, which includes steps S210-S220.

[0052] In step S210, three-dimensional MRI of multiple tumor patients is obtained as a tumor data set. The three-dimensional MRI may include gadolinium-enhanced longitudinal relaxation time-weighted sagittal image T1SC slices. The tumor type of the tumor patient is not limited, for example, it may be a spinal cord tumor. The resolution, thickness, and number of T1SC slices in the three-dimensional MRI are not limited. Based on the T1SC slices, the patient area of ​​the tumor patient and the three-dimensional morphology of the tumor can be restored. It is worth noting that the T1SC slices in the tumor data set can be preprocessed. For example, the T1SC slices can be scaled, the scanning intensity normalized, etc.

[0053] In step S220, the three-dimensional MRI in the tumor data set is used as an input of the first segmentation model to perform tumor segmentation, and obtain a category label of the tumor in the three-dimensional MRI.

[0054] The first segmentation model may be a model based on convolutional neural networks (CNN). With the popularity of convolutional neural networks, early methods mainly used CNN-based frameworks for medical image segmentation, including U-Net, 3D-UNet, Y-Net, KiU-Net, U-Net++, U-Net 3+, nnUNet, and many other variants. Among them, nnUNet is a general segmentation framework that can automatically configure the network framework and settings to extract features at multiple scales, and has shown particularly strong performance on various medical data sets. As an example, the first segmentation model can select nnUNet as the backbone network.

[0055] The first segmentation model may include a first model branch and a second model branch, the first model branch may be used to predict a category label, and the second model branch may be used to predict a surface distance field of a tumor based on boundary perception, the surface distance field being used to describe the closest distance from each voxel in a three-dimensional MRI to a tumor surface. For example, the first model branch and the second model branch may be a convolutional neural layer in the first segmentation model, respectively.

[0056] In order to improve the segmentation performance of two-dimensional natural images, previous studies integrated boundary information into the learning process. However, they mainly focused on regressing only edge pixels, while ignoring the details of the broader background of the shape surface. Some recent studies have utilized signed distance for medical image segmentation. However, they either focus on simple two-dimensional cases or require complex deformation or transformation processes. In addition, they do not take into account boundary changes across multiple classes. However, it is worth noting that the second model branch in this application can learn the three-dimensional surface distance fields of different categories of tumors separately. Taking spinal cord tumors as an example, the second model branch can learn the three-dimensional surface distance fields of four types of spinal cord tumors separately, which helps to pay attention to boundary changes across multiple classes.

[0057] The training and optimization of the first segmentation model can be performed based on the training and optimization of the first model branch and the second model branch. For example, the first model branch and the second model branch can be trained and optimized based on corresponding loss functions, respectively. The training and optimization of the first model branch can refer to the manual tumor annotation of the three-dimensional MRI of the tumor patient by the oncologist, and the training and optimization of the second model branch can refer to the measured value of the closest distance from each voxel in the three-dimensional MRI to the tumor surface.

[0058] As an example, the architecture of the first segmentation model can be as follows Figure 3 As shown, the backbone network is the first segmentation model, the segmentation branch is the first model branch, the surface distance field branch is the second model branch, and the segmentation branch and the surface distance field branch are trained and optimized based on loss function 1 and loss function 2, respectively.

[0059] In the present application, the segmentation of the tumor in the three-dimensional MRI by the first segmentation model can be divided into the prediction of the tumor category by the first model branch and the prediction of the distance field of the tumor surface based on the boundary perception by the second model branch. Based on the second model branch, the three-dimensional surface of the tumor in the three-dimensional MRI can be perceived, thereby capturing the details of the tumor boundary and the surrounding area, which helps to improve the accuracy of tumor segmentation.

[0060] Optionally, the second model branch can be trained and optimized based on the first boundary perception loss function, which can be used to determine the boundary perception loss of each voxel in the three-dimensional MRI, and the boundary perception loss of each voxel can be determined based on the surface distance value of each voxel, which can be used to represent the closest distance of each voxel to the tumor surface. For example, for each voxel, the difference between the closest distance from the voxel to the tumor surface predicted by the second model branch and the closest distance from the voxel to the tumor surface actually measured can be used to determine its boundary perception loss. In other words, the solution of the first boundary perception loss function can be based on the difference. Based on the first boundary perception loss function, the accuracy of the second model branch in predicting the surface distance field of the tumor can be improved, thereby facilitating a more accurate perception of the details of the tumor and the surrounding area.

[0061] Optionally, the voxels in the three-dimensional MRI may include voxels outside the tumor (also referred to as background voxels) and voxels inside the tumor, the surface distance values ​​of the voxels outside the tumor may be expressed as negative values, and the surface distance values ​​of the voxels inside the tumor may be expressed as positive values. In other words, the tumor surface (i.e., the tumor boundary) is the boundary where the surface distance value is 0. Based on this, it is helpful to distinguish the voxels outside and inside the tumor, thereby accurately determining the boundary of the tumor.

[0062] Optionally, if the absolute value of the surface distance value of the voxel outside the tumor is greater than the maximum surface distance value of the voxel inside the tumor, the surface distance value of the voxel is counted as 0. That is to say, when the surface distance value of the voxel outside the tumor meets the above situation, it is truncated to 0. Usually, when the absolute value of the surface distance value of the voxel outside the tumor is greater than the maximum surface distance value of the voxel inside the tumor, it indicates that the voxel is far away from the tumor boundary and belongs to the background voxel. The background voxel is no longer of great significance for perceiving the tumor boundary, so it can be excluded from the tumor surface distance field. Based on this, more attention can be paid to the voxels around the tumor boundary, which helps to more accurately determine the boundary of the tumor.

[0063] Optionally, the boundary perception loss of each voxel can be determined based on the normalized value of the surface distance value of each voxel. As an example, the normalization of the surface distance value of each voxel can be performed based on the maximum surface distance value of all voxels in the three-dimensional MRI. For example, the normalized value of the surface distance value of each voxel = (surface distance value / maximum surface distance value+1) / 2. Based on the normalization of the surface distance value, the computational efficiency of the boundary perception loss can be improved.

[0064] Optionally, the training and optimization of the second model branch can be performed according to the tumor category. Based on the above, the second model branch can learn the three-dimensional surface distance field of different types of tumors respectively, so the training and optimization of the second model branch can also be performed according to the tumor category, thereby helping the second model branch to more accurately focus on the boundary changes across multiple categories.

[0065] Optionally, the first boundary-aware loss function can be expressed as follows:

[0066]

[0067] Among them, l ba can be used to represent boundary perception loss, h can be used to represent the number of horizontal pixels in 3D MRI, w can be used to represent the number of vertical pixels in 3D MRI, d can be used to represent the number of slices in 3D MRI, k can be used to represent the total number of categories in 3D MRI, and f (h，w，d，k) It can be used to represent the surface distance value of the (h, w, d, k)th voxel predicted by the second model branch, It can be used to represent the true surface distance value of the (h, w, d, k)th voxel.

[0068] It is worth noting that, given the predicted tumor surface distance values ​​and the corresponding true surface distance values, a simple approach is to choose As a loss function, each voxel is optimized equally. However, since spinal cord tumors only occupy a relatively small area, most of the surface distance values ​​are zero, belonging to truncated background voxels. In order to make the boundary-aware loss truly focus on the tumor boundary area, this application introduces Based on this, the model can be driven to pay more attention to voxels with non-zero distance values ​​and belonging to the boundary area of ​​interest, which helps to determine the boundary of the tumor more accurately.

[0069] Optionally, the second model branch can be trained and optimized based on the average value of the boundary-aware loss of all voxels in the three-dimensional MRI, that is, the total boundary-aware loss can be taken as the average value of the boundary-aware loss of all voxels.

[0070] Optionally, the first model branch is trained and optimized based on a first segmentation loss function, and the first segmentation loss function includes a cross entropy loss function and a Dice Loss function. For example, the cross entropy loss function can be determined based on formula (2). For another example, the Dice loss function can be determined based on formula (3).

[0071]

[0072]

[0073] Among them, among them, represents the cross entropy loss function, represents the Dice Loss function, Represents the predicted segmentation result, represents the true segmentation, Pi,k is the network's flexible maximum transfer function output, which represents the probability that the i-th voxel is classified into class k; g is a single-bit efficient code corresponding to the true segmentation map, and the dimensions of p and g are both N×K; i∈N, represents the number of voxels in a given patch or batch; k∈K, represents the total number of categories in the segmentation task; according to the true segmentation, g is considered to be the correct voxel only if the i-th voxel does belong to class k. i,k is set to 1; otherwise g i,k Set to 0.

[0074] Furthermore, the first segmentation model can be jointly trained with the above three losses (total boundary perception loss, cross entropy loss, and dice loss). For example, as shown in formula (4), the total loss of the first segmentation model can be calculated by simply making the weights of the three losses equal.

[0075] l=l ce +l dice +l ba (4).

[0076] In the following, in conjunction with the accompanying drawings, a spinal cord tumor is taken as an example to introduce the boundary-aware three-dimensional MRI tumor segmentation method of an embodiment of the present application.

[0077] Spinal Cord Tumor Dataset

[0078] Data Collection: The Spinal Cord Tumor Dataset contains 3D MRIs of 653 patients collected between October 2017 and September 2023 and is specifically designed for preoperative evaluation of spinal cord tumors that have not undergone therapeutic intervention. The dataset contains a comprehensive range of spinal manifestations (covering the cervical, thoracic, and lumbar regions) including the four main types of spinal cord tumors: 1) meningioma (247 patients), 2) ependymoma (203 patients), 3) astrocytoma (101 patients), and 4) hemangioblastoma (102 patients), as shown in Figure 2. Figure 1 The first row is shown.

[0079] Each patient’s MRI consisted of gadolinium-enhanced T1 SC slices. As shown in Table 1, the native resolution of the sagittal MRI scans in this data set varied, with in-plane resolution ranging from 0.34 to 1.06 mm and slice thickness ranging from 1.5 to 8 mm. The number of slices in the sagittal plane varied, ranging from a minimum of 9 to a maximum of 36.

[0080] Table 1 Detailed information of MRI scans in the spinal cord tumor dataset

[0081]

[0082] Tumor Annotation: Each patient’s MRI is accompanied by expertly curated manual tumor annotations. Two independent neuroradiologists manually annotate the true tumor regions on gadolinium-enhanced T1SC. If there is a dispute regarding the delineation of the tumor region, the final decision is given to a senior neuroradiologist to ensure the highest fidelity and precision of data annotation.

[0083] Data Partitioning: To make a more comprehensive and fair comparison, the spinal cord tumor dataset can be evenly divided into five different parts to enable a robust 5-fold cross-evaluation of various methods and future studies. The details of these five parts are summarized in Table 2.

[0084] Table 2 Number of subjects in the five parts

[0085]

[0086] Data preprocessing: Following the standardized protocol widely adopted in existing works, the resolution of sagittal slices was rescaled to 0.47 mm in both the anterior-posterior and superior-inferior directions and to 3.3 mm in the left-right direction. The intensity values ​​of the magnetic resonance scans were trilinearly interpolated, and tumor annotations were interpolated by nearest neighbor. The intensity of each volume was normalized by first subtracting the mean intensity value from each voxel and then dividing by the standard deviation of the scan.

[0087] Segmentation Model

[0088] like Figure 4 As shown, given an input 3D MRI scan T∈R H×W×D , where H×W represents the slice resolution and D represents the number of slices. It is input into the backbone network, i.e., the first segmentation model (e.g., nnUNet), to obtain the multi-class segmentation output S∈R for each voxel H×W×D×K , where K represents the total number of categories. In this example, K is set to 5, representing 4 types of spinal cord tumors and background categories. Let a convolutional neural layer in nnUNet be the segmentation branch (i.e., the first model branch). In parallel with the segmentation branch, another branch (i.e., the second model branch) is added to predict the tumor surface distance field F∈R H×W×D×K , where a single convolutional neural layer is applied. The segmentation branch is supervised using the widely used cross entropy and Dice losses and uses the true semantic labels. The surface distance field is supervised using a multi-class boundary-aware loss (i.e., the first boundary-aware loss function) and uses the true labels.

[0089] Tumor Surface Distance Field

[0090] like Figure 1 As shown in the first row, spinal cord tumors exhibit particularly challenging shape variations. This motivates the introduction of tumor surface distance fields to learn the complex and fine details of tumor boundaries and surrounding areas in 3D MRI scans. Figure 5 Taking the two-dimensional T1SC slice of spinal cord tumor as an example, the definition of tumor surface distance field is introduced.

[0091] like Figure 5 As shown, given a 3D MRI scan slice t∈R H× W, the slice belongs to a patient with any type of spinal cord tumor, and its corresponding true tumor mask s∈{0,1} H×W , where 1 represents foreground tumor pixels and 0 represents background. For any particular pixel p within the tumor mask i , the closest distance to the tumor mask boundary can be calculated (with a positive sign), denoted as d i For any particular pixel p outside the tumor mask j , we can also calculate the closest distance to the tumor mask boundary (with a negative sign), expressed as d j .

[0092] Since spinal cord tumors usually occupy only a relatively small area, when the absolute distance d of pixels outside the tumor area is j When it is greater than the maximum distance value within the tumor mask, the distance value can be truncated to zero. The formal definition is:

[0093]

[0094] where {d0…d i …d I} represents all distance values ​​within the tumor mask, is the indicator function, and this simple truncation will allow the network to focus on details around surface boundaries.

[0095] Finally, all untruncated distance values ​​are normalized in the range [0, 1] as shown in Equations (6) and (7), while keeping all previously truncated values ​​zero.

[0096]

[0097]

[0098] It is worth noting that the definition of the tumor surface distance field on the above two-dimensional slice can be directly extended to three-dimensional MRI. That is, the surface distance value of any voxel in three-dimensional MRI is the closest distance between the voxel and the tumor boundary in three-dimensional space.

[0099] Similarly, given a 3D MRI T and its true tumor mask S, for each voxel in the 3D MRI, its surface distance value can be calculated by measuring the closest distance to the 3D tumor boundary, and then truncated and normalized to obtain the true tumor surface distance value of the entire volume, expressed as It will supervise the newly added network branches. It is worth noting that the true surface distance value is defined per class rather than in a class-independent manner, considering that the boundary shapes of each class of spinal cord tumors are often very different.

[0100] Loss Function

[0101] Cross entropy loss function and Dice loss function: The segmentation branch is supervised by the cross entropy loss function and the Dice loss function, denoted as l ce and l dice , the calculation methods refer to formula (2) and formula (3) respectively.

[0102] Boundary-aware loss function: The predicted tumor surface distance field is supervised by the boundary-aware loss function, and the calculation method is shown in formula (4).

[0103] accomplish

[0104] All experiments were performed on a single NVIDIA 3009 graphics processing unit (GPU). The model was trained for 1000 epochs with a learning rate of 0.01 and a decay of 0.00001 per epoch. The backbone network nnUNet, when trained on the spinal cord tumor dataset based on its automatically searched configuration, contained 7 layers, each containing two convolution operations. The first two layers used a kernel size of 1×1×1, while the subsequent layers used a kernel size of 3×3×3. The stride of the first layer was configured as 1×1×1, the stride of the fourth layer was configured as 2×2×2, and the stride of the remaining layers was configured as 1×2×2. The number of feature channels for all layers was set to 32, 64, 128, 256, 320, 320, and 320.

[0105] In addition to evaluating all models on the spinal cord tumor dataset using 5-fold cross validation, all methods are also evaluated on the public KNIGHT dataset (a kidney dataset) because kidney tumors have similarities with spinal cord tumors. According to the searched configuration, when the backbone network is trained on the public KNIGHT dataset, the network contains six layers, each containing two 3×3×3 convolution operations. The stride of the first layer is configured as 1×1×1, and the stride of the remaining layers is configured as 2×2×2. The number of feature channels of all layers is set to 32, 64, 128, 256, 320, and 320.

[0106] experiment

[0107] In order to verify the effectiveness of the method of the embodiment of the present application, it can be compared with other tumor segmentation methods.

[0108] With the success of attention mechanisms and visual transformers in capturing long-range contextual information, many works have extended attention-based methods to CNN-based frameworks for better medical segmentation, including Trans U-Net, UNETR, Swin UNETR, nnFormer, 3D UX-Net, and many other variants. Recently, large segmentation base models have achieved great progress in natural images, and many subsequent works have extended them to the field of medical image segmentation. Thanks to powerful network architectures and large training datasets, these methods have achieved outstanding performance at the cost of large computational resources and careful and fast engineering tricks.

[0109] With the development of segmentation models, methods specifically for spinal cord tumor segmentation have also emerged. In one study, a dual-cascade 3D CNN was proposed for chordoma segmentation. However, the segmented tumors were located in the spinal region, they showed different intensities and sizes, and were juxtaposed with different tissue types. Therefore, this segmentation model designed for the spine is not suitable for spinal cord tumor segmentation in this case due to the inherent differences. In another study, a two-level cascade architecture consisting of two U-Net models was used to locate the spinal cord and segment tumors, respectively, but due to its weakness in discerning fine shapes, it was insufficient in determining multiple types of tumors.

[0110] Based on this, this example chooses to compare with the following two sets of baselines in medical image segmentation, and all models are trained from scratch.

[0111] End-to-end training baselines: including established state-of-the-art models nnFormer, 3D UX-Net, Swin UNETR, and nnUNet.

[0112] Two-stage training baseline: For the multi-class 3D segmentation problem at each voxel, a simple pipeline is to apply a per-voxel two-class segmentation (tumor and background) followed by a per-voxel classification model that takes as input a 3D image with / without the predicted two-class tumor mask, e.g. Figure 6As shown. For fair comparison, in stage 1, the same backbone nnUNet is selected for two-class segmentation in this example. In stage 2, two powerful models are selected in this example: 3DResNet101 and the encoder part of nnUNet (denoted as UEnc), both followed by a multilayer perceptron (MLP) layer with (1024-512-256-K) neurons. When training in stage 2, a 3D mask with or without estimated class-agnostic tumors is also input for comparison, denoted as or

[0113] In summary, there are 4 baselines in the experiment: 1) 2) 3) 4)

[0114] Metrics: Following the Multi-Centre, Multi-Vendor & Multi-Disease Cardiac Image Segmentation Challenge (M&MS Challenge), this example uses the Dice coefficient and the 95th percentile Hausdorff distance (HD) as metrics. For missing predictions, the maximum value of the Hausdorff distance is 450 mm.

[0115] Datasets: As mentioned previously, in addition to evaluating all models on the Spinal Cord Tumors dataset using 5-fold cross validation, this example also evaluates all methods on the public KNIGHT dataset. This dataset contains 400 3D computed tomography (CT) scans, divided into a training set of 300 scans and a test set of 100 scans. Each voxel is annotated with one of two tumor classes: no adjuvant therapy (NoAT) and candidate for adjuvant therapy (CanAT), or a background class. The resolution of all CT scans is rescaled to 2mm×2mm×2mm. Intensities are trilinearly interpolated and tumor annotations are interpolated via nearest neighbor interpolation. The intensity of each volume is normalized by first subtracting the mean intensity value from each voxel and then dividing by the standard deviation of the scan.

[0116] Experimental results on the spinal cord tumor dataset

[0117] Table 3 compares the quantitative results of all baselines and the methods of the present invention on the spinal cord tumor dataset, and these results are averaged over 5-fold cross validation. Tables 4-8 provide detailed quantitative results for each part. Figure 7A-7DBased on Tables 3 to 8 and Figure 7A-7D It can be seen that:

[0118] – In the first set of baselines, nnUNet performs well and achieves better Dice scores and Hausdorff distances than other attention-based methods.

[0119] – In the second set of baselines, the nnUNet encoder-based classifier achieves higher Dice scores than the ResNet101-based model.

[0120] –Compared with all these strong baselines, the method of the embodiment of the present application achieves significantly better performance in both Dice score and Hausdorff distance. It is worth noting that compared with the backbone network nnUNet, the method of the embodiment of the present application achieves a 10% higher Dice score and a 53.1mm better Hausdorff distance, demonstrating the effectiveness of the tumor surface distance field learned by boundary-aware loss proposed in this application.

[0121] – It can also be noted that the Dice score of the tumor type astrocytoma (35.2%) is significantly lower than that of the other types. A key reason for this is that astrocytomas and ependymomas are similar in appearance, leading to frequent misclassification between these two tumor types. Therefore, the Dice score of ependymomas is also relatively low (57.8%).

[0122] Table 3 Dice scores (%) and Hausdorff distances (mm) using 5-fold cross validation on the spinal cord tumor dataset

[0123]

[0124] Table 4 Dice score (%) and Hausdorff distance (mm) on the spinal cord tumor dataset Fold1

[0125]

[0126]

[0127] Table 5 Dice score (%) and Hausdorff distance (mm) on the spinal cord tumor dataset Fold2

[0128]

[0129] Table 6 Dice score (%) and Hausdorff distance (mm) on the spinal cord tumor dataset Fold3

[0130]

[0131]

[0132] Table 7 Dice score (%) and Hausdorff distance (mm) on the spinal cord tumor dataset Fold4

[0133]

[0134] Table 8 Dice score (%) and Hausdorff distance (mm) on the spinal cord tumor dataset Fold5

[0135]

[0136] Results on the KNIGHT dataset

[0137] Table 9 compares the quantitative results of all methods on the KNIGHT dataset. The qualitative results are shown in Figure 7A-7D Based on Table 9 and Figure 7A-7D You can get:

[0138] – In both baselines, although nnUNet and The performance is excellent, but the method of the embodiment of the present application surpasses them, achieving a Dice score of 40.7% and a Hausdorff distance of 226.8 mm.

[0139] – It can also be noticed that most of the two-stage methods tend to classify all subjects into the “NoAT” category. Therefore, they show high performance in the “NoAT” class but produce a Dice score close to zero in the “CanAT” class.

[0140] On two datasets, Figure 7A-7D As shown, the tumor boundaries predicted by the method of the embodiment of the present application show greater consistency with the basic truth. In contrast, other baselines often predict larger tumors, resulting in a large number of false positive voxels. In addition, some baselines tend to predict multiple sub-regions with different tumor types, while the method of the embodiment of the present application consistently generates more accurate masks with fine boundaries and correct tumor types.

[0141] Table 9 Dice score (%) and Hausdorff distance (mm) of KNIGHT dataset

[0142]

[0143] Ablation experiment

[0144] In order to verify the effectiveness of the design of the embodiment of the present application, the following groups of ablation studies were performed on the public KNIGHT dataset.

[0145] – Group 1: In order to verify the design of the distance field on the tumor surface, four different settings are selected in this example: 1) the distance field is neither truncated nor normalized; 2) the distance field is truncated but not normalized; 3) the distance field is truncated and normalized, which is the setting of the method in the embodiment of the present application; 4) the boundary-aware loss l ba The l1 term in The l2 term Replace; 5) Boundary perception loss l ba The l1 term in The cross entropy loss l ce replace.

[0146] – Group 2: For the cutoff strategy defined in formula (5), this example verifies three settings: distance d j Respectively in max{d0…d i …d I Intuitively, the wider the cutoff range, the larger the area that needs to be focused, so it may be less efficient.

[0147] – Group 3: This example further verifies the design of multi-class distance field. For comparison, this example simplifies multi-class perception into class-independent distance field. This means Figure 4 The surface distance field branch in has R H×W×D×1 The output shape of .

[0148] – Group 4: For the boundary-aware loss defined in formula (1), this example verifies The effectiveness of this one.

[0149] Tables 10-13 compare the ablation test results of groups 1 / 2 / 3 / 4, respectively. Figure 8 Qualitative results are shown. Based on Tables 10-13 and Figure 8 You can get:

[0150] – In Table 10, the truncated design of the embodiment of the present application improves the Dice score by 5.9%, and normalizing the truncated distance field further improves the Dice score by 9.5%. ba The use of l ce or e2 is slightly better.

[0151] – In Table 11, the wider the region to be truncated, the worse the results obtained. The reason is that fine details near the tumor boundary may be more important than pixels far from the boundary.

[0152] – In Table 12, the multi-class distance field in the embodiment of the present application simplifies the learning of complex boundaries, thereby providing a clearer understanding of different tumor types, and improving the Dice score by 4.3% over the class-independent setting.

[0153] – In Table 13, The term drives the model to pay more attention to voxels with non-zero distance values, such as those in the tumor boundary region of interest, which improves the Dice score by 1%. The Dice score is reduced by 2%.

[0154] Table 10 Ablation experiments of group 1 on the KNIGHT dataset

[0155]

[0156] Table 11 Ablation experiments of group 2 on the KNIGHT dataset

[0157]

[0158]

[0159] Table 12 Ablation experiments of group 3 on KNIGHT dataset

[0160]

[0161] Table 13 Ablation experiments of group 4 on the KNIGHT dataset

[0162]

[0163] Based on the above experiments, it can be determined that the method of the embodiment of the present application has a high segmentation accuracy.

[0164] They are significantly better than the existing baselines.

[0165] The method provided in the embodiment of the present application is described in detail above. The device provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0166] See also Fig. 9 , is a schematic diagram of the structure of a tumor diagnosis device provided in an embodiment of the present application. Fig. 9 The tumor diagnosis device 900 shown may include an acquisition module 910 and a segmentation module 920 .

[0167] The acquisition module 910 is used to acquire three-dimensional MRI of multiple tumor patients as a tumor data set, and the three-dimensional MRI includes gadolinium-enhanced T1SC slices;

[0168] The segmentation module 920 is used to perform tumor segmentation by using the three-dimensional MRI in the tumor data set as the input of the first segmentation model to obtain a category label of the tumor in the three-dimensional MRI;

[0169] Among them, the first segmentation model is a model based on a convolutional neural network CNN, and the first segmentation model includes a first model branch and a second model branch. The first model branch is used to predict category labels, and the second model branch is used to predict the surface distance field of the tumor based on boundary perception. The surface distance field is used to describe the closest distance from each voxel in the three-dimensional MRI to the tumor surface. The training and optimization of the first segmentation model are based on the training and optimization of the first model branch and the second model branch.

[0170] Optionally, the second model branch is trained and optimized based on a first boundary-aware loss function, which is used to determine the boundary-aware loss of each voxel in the three-dimensional MRI, and the boundary-aware loss of each voxel is determined based on the surface distance value of each voxel, and the surface distance value is used to represent the closest distance of each voxel to the tumor surface.

[0171] Optionally, the voxels include voxels outside the tumor and voxels inside the tumor, the surface distance values ​​of the voxels outside the tumor are represented as negative values, and the surface distance values ​​of the voxels inside the tumor are represented as positive values.

[0172] Optionally, if the absolute value of the surface distance value of the voxel outside the tumor is greater than the maximum surface distance value of the voxel inside the tumor, the surface distance value is counted as 0.

[0173] Optionally, a boundary-aware loss for each voxel is determined based on a normalized value of the surface distance value of each voxel.

[0174] Optionally, the training and optimization of the second model branch are performed according to the type of tumor.

[0175] Optionally, the first boundary-aware loss function is expressed as follows:

[0176]

[0177] Among them, l ba is used to represent the boundary perception loss, h is used to represent the number of horizontal pixels in 3D MRI, w is used to represent the number of vertical pixels in 3D MRI, d is used to represent the number of slices in 3D MRI, k is used to represent the total number of categories in 3D MRI, and f (h，w，d，k) The surface distance value of the (h, w, d, k)th voxel predicted by the second model branch is used to represent the surface distance value of the (h, w, d, k)th voxel, Used to represent the true surface distance value of the (h, w, d, k)th voxel.

[0178] Optionally, the second model branch is trained and optimized based on the average of the boundary-aware losses of all voxels in the 3D MRI.

[0179] Optionally, the first model branch is trained and optimized based on a first segmentation loss function, and the first segmentation loss function includes a cross entropy loss function and a dice loss function.

[0180] The present application also provides an electronic device. Fig.10 As shown, the electronic device 1000 includes: at least one processor 1001, a memory 1002, and a computer program 1003 stored in the memory 1002 and executable on the at least one processor 1001. When the processor 1001 executes the computer program 1003, the method provided in the present application is implemented.

[0181] Exemplarily, the computer program 1003 may be divided into one or more modules / units, one or more modules / units are stored in the memory 1002, and are executed by the processor 1001 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 1000.

[0182] Those skilled in the art will understand that Fig.10 It is only an example of the electronic device 1000 and does not constitute a limitation of the electronic device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 1000 may also include input and output devices, network access devices, buses, etc.

[0183] The processor 1001 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0184] The memory 1002 may be an internal storage unit of the electronic device 1000, such as a hard disk or memory of the electronic device 1000. The memory 1002 may also be an external storage device of the electronic device 1000, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1000. Further, the memory 1002 may also include both an internal storage unit of the electronic device 1000 and an external storage device. The memory 1002 is used to store computer programs and other programs and data required by the electronic device 1000. The memory 1002 may also be used to temporarily store data that has been output or is to be output.

[0185] The electronic device 1000 provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0186] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0187] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0188] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above method embodiments, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, ROM (read-only memory), RAM (random access memory), CD-ROM (compact disc read-only memory), magnetic tape, floppy disk and optical data storage device. The computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, a non-transient storage medium.

[0189] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0190] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0191] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0193] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0194] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0195] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A three-dimensional magnetic resonance imaging (MRI) tumor segmentation method based on boundary perception, characterized in that: include: Acquiring three-dimensional MRI of multiple tumor patients as a tumor data set, wherein the three-dimensional MRI includes gadolinium-enhanced longitudinal relaxation time-weighted sagittal image T1SC slices; Using the three-dimensional MRI in the tumor data set as input of a first segmentation model to perform tumor segmentation, and obtain a category label of the tumor in the three-dimensional MRI; Among them, the first segmentation model is a model based on a convolutional neural network CNN, and the first segmentation model includes a first model branch and a second model branch. The first model branch is used to predict the category label, and the second model branch is used to predict the surface distance field of the tumor based on boundary perception. The surface distance field is used to describe the closest distance from each voxel in the three-dimensional MRI to the tumor surface. The training and optimization of the first segmentation model are based on the training and optimization of the first model branch and the second model branch.

2. The method according to claim 1, characterized in that The second model branch is trained and optimized based on a first boundary-aware loss function, which is used to determine the boundary-aware loss of each voxel in the three-dimensional MRI. The boundary-aware loss of each voxel is determined based on the surface distance value of each voxel, and the surface distance value is used to represent the closest distance of each voxel to the tumor surface.

3. The method according to claim 2, characterized in that The voxels include voxels outside the tumor and voxels inside the tumor, the surface distance values ​​of the voxels outside the tumor are represented as negative values, and the surface distance values ​​of the voxels inside the tumor are represented as positive values.

4. The method according to claim 3, characterized in that If the absolute value of the surface distance value of the voxel outside the tumor is greater than the maximum surface distance value of the voxel inside the tumor, the surface distance value is calculated as 0.

5. The method according to claim 4, characterized in that The boundary perception loss of each voxel is determined based on a normalized value of the surface distance value of each voxel.

6. The method according to claim 5, characterized in that The training and optimization of the second model branch are performed according to the type of the tumor.

7. The method according to claim 6, characterized in that The first boundary perception loss function is expressed as follows: Among them, the l ba is used to represent the boundary perception loss, h is used to represent the number of horizontal pixels of the three-dimensional MRI, w is used to represent the number of vertical pixels of the three-dimensional MRI, d is used to represent the number of slices of the three-dimensional MRI, k is used to represent the total number of categories of the three-dimensional MRI, and f (h,w,d,k) used to represent the surface distance value of the (h,w,d,k)th voxel predicted by the second model branch, Used to represent the true surface distance value of the (h,w,d,k)th voxel.

8. The method according to claim 7, characterized in that The second model branch is trained and optimized based on the average value of the boundary-aware loss of all voxels in the three-dimensional MRI.

9. The method according to claim 8, characterized in that The first model branch is trained and optimized based on a first segmentation loss function, wherein the first segmentation loss function includes a cross entropy loss function and a dice loss function.

10. A tumor diagnostic device, characterized in that: include: An acquisition module, used for acquiring three-dimensional magnetic resonance imaging (MRI) of multiple tumor patients as a tumor data set, wherein the three-dimensional MRI includes gadolinium-enhanced longitudinal relaxation time-weighted sagittal image T1SC slices; a segmentation module, configured to use the three-dimensional MRI in the tumor data set as an input of a first segmentation model to perform tumor segmentation and obtain a category label of the tumor in the three-dimensional MRI; Among them, the first segmentation model is a model based on a convolutional neural network CNN, and the first segmentation model includes a first model branch and a second model branch. The first model branch is used to predict the category label, and the second model branch is used to predict the surface distance field of the tumor based on boundary perception. The surface distance field is used to describe the closest distance from each voxel in the three-dimensional MRI to the tumor surface. The training and optimization of the first segmentation model are based on the training and optimization of the first model branch and the second model branch.

Images