Automatic segmentation method and device for skull base tumor, electronic equipment and medium

Abstract

The invention provides a skull base tumor automatic segmentation method and device, electronic equipment and a medium, and the method comprises the steps: obtaining rare skull base tumor data of a plurality of patients, and constructing a data set; training the constructed skull base tumor automatic segmentation model based on the data set to obtain a trained skull base tumor automatic segmentation model; the skull base tumor automatic segmentation model is used for randomly selecting two kinds of MRI image data from the at least two kinds of MRI image data, and performing feature fusion on the second kind of MRI image data by using the first kind of MRI image data of the two kinds of MRI image data to obtain fusion features; prompting points are added in tumor areas of the at least two kinds of MRI image data, and after the prompting points are coded, the fusion features are updated based on the codes of the prompting points; and obtaining at least two kinds of MRI image data of a patient to be detected, and segmenting a tumor region in the MRI image of the patient to be detected based on the trained skull base tumor automatic segmentation model. According to the method, the rare skull base tumor region can be quickly and effectively segmented.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis and processing technology, and in particular to an automatic segmentation method, device, electronic device and medium for skull base tumors. Background Technology

[0002] Rare skull base tumors are a class of primary malignant or low-grade malignant tumors originating from the skull base bones and adjacent structures, including but not limited to chordomas, chondrosarcomas, olfactory neuroblastomas, and schwannomas. These tumors typically exhibit high invasiveness and local destruction during growth, often affecting surrounding bone, soft tissues, and important neurovascular structures. Due to the extremely complex anatomy of the skull base and its proximity to multiple key functional areas, surgical resection of rare skull base tumors is extremely challenging. Treatment strategies and prognosis largely depend on accurate preoperative assessment of the tumor's extent and its relationship to surrounding structures. Therefore, accurate, rapid, and standardized segmentation and assessment of rare skull base tumors are of significant clinical importance for surgical planning, intraoperative navigation, and postoperative radiotherapy target delineation.

[0003] However, in practice, the boundaries between tumors and surrounding normal tissues (such as bone, brain tissue, nerves, and blood vessels) are often unclear, making the interpretation and segmentation of routine MRI images highly dependent on the physician's experience. Even for professional neuroradiologists, performing a complete segmentation of a rare skull base tumor requires a significant amount of time and effort, and the consistency and reproducibility of the segmentation results are difficult to guarantee due to subjective factors. Currently, the number of experts capable of handling such challenging segmentation tasks is extremely limited, further exacerbating the limitations of manual segmentation in terms of manpower and efficiency.

[0004] To address this issue, deep learning and artificial intelligence technologies have demonstrated significant potential in automated medical image segmentation in recent years. Deep neural networks can achieve automated segmentation of complex structures to a certain extent, significantly improving efficiency and standardization. However, due to the low incidence of rare skull base tumors, high-quality image data is extremely limited. This data scarcity not only affects the training effect of the model but also limits its generalization ability, becoming a core obstacle to the further widespread application of automated segmentation. Therefore, this invention attempts to introduce a large-scale pre-trained model, such as the Segment Anything Model (SAM), to improve segmentation performance through transfer learning and domain fine-tuning.

[0005] However, the application of existing large-scale SAM models in medical image segmentation mainly focuses on single-parameter MRI sequences. In actual clinical practice, single-parameter MRI (such as using only T1WI or T2WI) often fails to fully reflect the complex anatomical structures and pathological features of rare skull base tumors. Different types of tumors exhibit different characteristics under multiparameter MRI, and the segmentation results of a single sequence often cannot accurately depict the true boundaries and extent of invasion of the tumor. Traditional automatic segmentation methods have significant limitations in capturing tumor heterogeneity and small invasive lesions, affecting subsequent diagnostic and treatment decisions. Summary of the Invention

[0006] This invention provides an automatic segmentation method, device, electronic device, and medium for skull base tumors, to address the challenge that single-parameter MRI cannot fully reflect the complex anatomical structure and pathological features of rare skull base tumors.

[0007] According to one aspect of the present invention, an automatic segmentation method for skull base tumors is provided, comprising: Data on rare skull base tumors from multiple patients were acquired and a dataset was constructed; the rare skull base tumor data included at least two types of MRI image data. The automatic segmentation model for skull base tumors is trained based on a dataset. The trained automatic segmentation model for skull base tumors is used to randomly select two types of MRI image data from at least two types of MRI image data, and to perform feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. Cue points are added to the tumor region of the at least two types of MRI image data, and after encoding the clue points, the fused features are updated based on the clue point encoding. Acquire at least two types of MRI image data from the patient to be tested, and input the at least two types of MRI image data into a trained automatic segmentation model for skull base tumors to segment the tumor region in the MRI images of the patient to be tested.

[0008] Optionally, it also includes: preprocessing the rare skull base tumor data; the preprocessing includes at least normalization, alignment, annotation, and slicing.

[0009] Optionally, the preprocessing includes: Intensity normalization was performed on at least two types of MRI image data; Using one of at least two MRI image datasets as a reference, perform affine transformations on the other datasets to achieve data alignment between the at least two MRI image datasets; Tumor regions in at least two types of MRI image data are labeled, and the MRI image data are sliced ​​according to a preset volume window to obtain slice data with a uniform volume window.

[0010] Optionally, the automatic segmentation model for skull base tumors includes a parameter selection module, an image encoder, a cue encoder, and a mask decoder; The parameter selection module is used to randomly select two MRI image data from at least two types of MRI image data. The image encoder is used to perform feature fusion on the second MRI image data using the first MRI image data of two types of MRI image data to obtain fused features; The prompt encoder is used to encode the prompt points marked by the user in the MRI image data; The mask decoder is used to take the cue point encoding and the fused features as input, and performs alternating self-attention and cross-attention processing on the cue point encoding and the fused features to achieve bidirectional updates from cue point encoding to fused features and from fused features to cue point encoding, and outputs the updated fused features, and generates a three-dimensional mask based on the updated fused features.

[0011] Optionally, it further includes: the image encoder includes a downsampling module, a feature extraction module, a master parameter selection module, a feature fusion module, a depth modeling module, and a serialization processing module; The mask decoder includes an update module, an upsampling module, and a mask generation module; The downsampling module is used to downsample the two selected MRI image data. The feature extraction module is used to extract features from the two types of MRI image data respectively; The main parameter selection module is used to select one feature of MRI image data as the main parameter and the other as an auxiliary parameter from the features of two MRI image data. The feature fusion module is used to fuse the features of auxiliary parameters into the main parameters through a multi-head self-attention mechanism to obtain fused features; The deep modeling module is used for deep modeling of the fused features; The serialization processing module is used to serialize the fused features after deep modeling through convolutional layers and normalization layers; The update module is used to take the fused features output by the cue point encoding and serialization processing module as input, and perform alternating self-attention and cross-attention processing on the cue point encoding and the fused features to achieve bidirectional updates from cue point encoding to fused features and from fused features to cue point encoding, and output the updated fused features. The upsampling module is used to upsample the fusion features output by the update module; The mask generation module is used to process the upsampled fused features using a supernetwork structure composed of three fully connected networks to generate a three-dimensional mask.

[0012] Optionally, it also includes: the depth modeling module includes a multi-layer 3D Transformer Block; the update module includes a TwoWayTransformer.

[0013] Optionally, the automatic segmentation model for skull base tumors trained and constructed based on the dataset includes: The dataset was divided into training, testing, and validation sets to train the automatic segmentation model for skull base tumors. Dice loss and cross-entropy loss were used as objective functions, and the model parameters were optimized by minimizing the loss functions.

[0014] According to another aspect of the present invention, an automatic segmentation device for skull base tumors is provided, comprising: A data acquisition unit is used to acquire rare skull base tumor data from multiple patients and construct a dataset; the rare skull base tumor data includes at least two types of MRI image data; The training unit is used to train a pre-constructed automatic segmentation model for skull base tumors based on a dataset, thereby obtaining a trained automatic segmentation model for skull base tumors. The automatic segmentation model for skull base tumors is used to randomly select two types of MRI image data from at least two types of MRI image data, and to perform feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. Cue points are added to the tumor region of the at least two types of MRI image data, and after encoding the clue points, the fused features are updated based on the clue point encoding. The detection unit is used to acquire at least two types of MRI image data from the patient to be tested, and input the at least two types of MRI image data into a trained automatic segmentation model for skull base tumors to segment the tumor region in the MRI images of the patient to be tested.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the automatic segmentation method for skull base tumors according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the automatic segmentation method for skull base tumors according to any embodiment of the present invention.

[0017] The technical solution of this invention involves acquiring rare skull base tumor data from multiple patients and constructing a dataset. This rare skull base tumor data includes at least two types of MRI image data. An automatically segmented skull base tumor model is trained based on the dataset, resulting in a trained automatic segmented skull base tumor model. This model randomly selects two types of MRI image data from the at least two types, and performs feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. This approach fully considers the information differences and complementarities of MRI with different parameters. By comprehensively utilizing MRI data with multiple parameters from the same patient, the versatility and robustness of the rare skull base tumor segmentation model are improved. Furthermore, for complex examples, this invention can input more parameter MRI to assist the model segmentation. Under conditions of limited resources or missing parameter MRI, it can reduce the types of input parameter MRI, greatly improving the versatility and automation level of the rare skull base tumor segmentation model.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of an automatic segmentation method for skull base tumors provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the architecture of an automatic segmentation model for skull base tumors provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an automatic skull base tumor segmentation device according to Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the automatic segmentation method for skull base tumors according to embodiments of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] Example 1 Figure 1 This is a flowchart illustrating an automatic segmentation method for skull base tumors according to Embodiment 1 of the present invention. Figure 1 As shown, the method includes: S101. Obtain rare skull base tumor data from multiple patients and construct a dataset; the rare skull base tumor data includes at least two types of MRI image data.

[0024] In this embodiment, historical data of multiple patients with skull base tumors can be obtained and a dataset can be constructed for subsequent training of an automatic segmentation model for skull base tumors.

[0025] Data for rare skull base tumors includes at least two types of MRI images, such as MRI data from different equipment manufacturers, including three sequences: T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and contrast-enhanced T1-weighted imaging (CE-T1WI). The specific data acquisition process includes: first, the patient is placed in a supine position with their head fixed in a special head coil to minimize the impact of motion artifacts on image quality; throughout the image acquisition process, the patient is instructed to remain still and coordinate with their breathing rhythm to avoid blurring due to movement; T1WI is acquired without contrast agent injection, which highlights the low-signal or isointense areas of the tumor, aiding in the localization of the tumor body and some tissue characteristics; next, T2WI is acquired, which clearly shows cystic changes in the tumor, surrounding edema, and the boundary between the tumor and surrounding soft tissues, improving tissue contrast and boundary interpretation; finally, CE-T1WI is acquired after intravenous injection of gadolinium contrast agent, which effectively enhances the contrast between the lesion and surrounding fat, bone marrow, and other structures, helping to clarify the tumor's enhancement pattern, invasive range, and small lesions.

[0026] S102. The automatic segmentation model for skull base tumors is trained and constructed based on the dataset to obtain the trained automatic segmentation model for skull base tumors. The automatic segmentation model for skull base tumors is used to randomly select two types of MRI image data from at least two types of MRI image data, and to perform feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. Cue points are added to the tumor region of the at least two types of MRI image data, and after encoding the clue points, the fused features are updated based on the clue point encoding.

[0027] Automatic segmentation models for skull base tumors can be deep learning models, such as models using a three-dimensional VisionTransformer (ViT) architecture to process voxel data.

[0028] In this embodiment, the automatic segmentation model for skull base tumors first randomly selects two types of MRI image data from at least two types of MRI image data. It then performs feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. The first type of MRI image data can be any of three sequences: T1-weighted image (T1WI), T2-weighted image (T2WI), and contrast-enhanced T1-weighted image (CE-T1WI). The second type of MRI image data can be any MRI data other than the first type.

[0029] Alternatively, cue points can be annotated in at least two types of MRI image data. When feature fusion is performed on a second type of MRI image data using the first type of MRI image data (i.e., the second type of MRI image data is the primary data and the first type of MRI image data is the secondary data), cue points in the second type of MRI image data can be encoded. The fused features are then updated based on the cue point encoding, thereby guiding the automatic segmentation model for skull base tumors to segment tumors in a specified region. Here, the cue points represent data from the tumor region.

[0030] S103. Obtain at least two types of MRI image data from the patient to be tested, and input the at least two types of MRI image data into the trained automatic segmentation model for skull base tumors to segment the tumor region in the MRI images of the patient to be tested.

[0031] For at least two types of MRI image data obtained from the patient to be tested, at least two types of MRI image data can be input into a trained automatic segmentation model for skull base tumors. For example, two types of MRI image data can be selected from the at least two types of MRI image data. One type of MRI image data can be used as auxiliary data, and the second type of MRI image data can be used as the main data for feature fusion to obtain fused features. The skull base tumor region can then be segmented based on the fused features.

[0032] The technical solution of this invention involves acquiring rare skull base tumor data from multiple patients and constructing a dataset. This rare skull base tumor data includes at least two types of MRI image data. An automatically segmented skull base tumor model is trained based on the dataset, resulting in a trained automatic segmented skull base tumor model. This model randomly selects two types of MRI image data from the at least two types, and performs feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. This approach fully considers the information differences and complementarities of MRI with different parameters. By comprehensively utilizing MRI data with multiple parameters from the same patient, the versatility and robustness of the rare skull base tumor segmentation model are improved. Furthermore, for complex examples, this invention can input more parameter MRI to assist the model segmentation. Under conditions of limited resources or missing parameter MRI, it can reduce the types of input parameter MRI, greatly improving the versatility and automation level of the rare skull base tumor segmentation model.

[0033] In one embodiment, after step S101, the rare skull base tumor data is further preprocessed; the preprocessing includes at least normalization, alignment, annotation, and slicing.

[0034] For rare skull base tumor data, to ensure that the subsequent deep learning model can fully exploit the complementary information between different MRI parameters, the acquired MRI image data can undergo unified preprocessing. Specifically, firstly, non-uniform magnetic field correction and noise suppression can be performed on the T1WI, T2WI, and CE-T1WI images of each patient to reduce the impact of common MRI artifacts on segmentation accuracy; then, all MRI sequence images are uniformly resampled to 1×1×1 mm. 3 The equal voxel resolution ensures the consistency of spatial coordinates to the greatest extent.

[0035] In addition, intensity normalization can be performed on at least two types of MRI image data; affine transformations can be applied to the other data based on one of the at least two types of MRI image data to achieve data alignment of at least two types of MRI image data; tumor regions in at least two types of MRI image data can be labeled, and MRI image data can be sliced ​​according to a preset volume window to obtain slice data with a uniform volume window.

[0036] Specifically, standardization methods such as Z-score can be used to normalize the intensity of at least two types of MRI image data to eliminate signal intensity differences between different devices, individuals, and parameters. Using unenhanced T1WI as the spatial benchmark, affine transformations are first performed on T2WI and CE-T1WI for global alignment. Then, a multi-resolution non-rigid registration algorithm is used to further correct local inconsistencies caused by anatomical deformation between sequences, ensuring a high degree of spatial correspondence among the three sequences. Next, two or more experienced neuroradiologists or neurosurgeons collaboratively label the tumor with true labels based on the multi-sequence images, ensuring the reliability of the gold standard label. Finally, all MRI sequences and labels are sliced ​​into 128×128×128 volume windows to form uniformly sized three-dimensional data blocks that can be directly input into neural networks, serving as input for the automatic segmentation model of skull base tumors.

[0037] In one embodiment, the automatic segmentation model for skull base tumors includes a parameter selection module, an image encoder, a cue encoder, and a mask decoder. The parameter selection module is used to randomly select two types of MRI image data from at least two types of MRI image data. The image encoder is used to perform feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. The cue encoder is used to encode cue points marked by the user in the MRI image data. The mask decoder is used to take the fused features of the cue point encoding and the fused features as input, and to perform alternating self-attention and cross-attention processing on the cue point encoding and the fused features to achieve bidirectional updates from cue point encoding to fused features and from fused features to cue point encoding, and output the updated fused features, and generate a three-dimensional mask based on the updated fused features.

[0038] It should be noted that the parameter selection module can discard various MRI sequences with the same probability p using a Bernoulli distribution, thus retaining at least one MRI sequence. This simulates the segmentation requirements under conditions of limited MRI sequence acquisition, thereby enhancing the generalization ability of the automatic segmentation model for skull base tumors. For example... Figure 2 In one embodiment shown, the parameter selection module of the automatic segmentation model for skull base tumors can select two MRI sequences, T1WI and T2WI, from three images: T1WI, T2WI, and CE-T1WI, by discarding them with the same probability p according to a Bernoulli distribution.

[0039] In this embodiment, to fully explore the expressive ability of multimodal MRI images to represent the heterogeneity of rare skull base tumors, a multi-parameter MRI image encoder based on a three-dimensional Vision Transformer (ViT) architecture is adopted. The image encoder is used to perform feature fusion on the second MRI image data using the first MRI image data and the second MRI image data to obtain fused features.

[0040] In one specific embodiment, the image encoder includes a downsampling module, a feature extraction module, a master parameter selection module, a feature fusion module, a depth modeling module, and a serialization module. The downsampling module is used to downsample two selected MRI image data sets. The feature extraction module is used to extract features from the two MRI image data sets respectively. The master parameter selection module is used to select features from the features of one MRI image data set as the master parameter and the other as an auxiliary parameter. The feature fusion module is used to fuse the features of the auxiliary parameter into the master parameter through a multi-head self-attention mechanism to obtain fused features. The depth modeling module is used to perform depth modeling on the fused features. The serialization module is used to serialize the depth-modeled fused features after passing them through convolutional layers and normalization layers.

[0041] like Figure 2 As shown, for the preprocessed T1WI and T2WI MRI sequences, a downsampling module can be used to downsample the selected MRI image data of the two types. A feature extraction module then extracts features from each type of MRI image data. In this embodiment, a 3D convolutional layer can project the data blocks of the T1WI and T2WI MRI sequences to a unified dimension, such as an embedding space with a resolution of 1 / 16 of the original size. Subsequently, a principal parameter selection module selects features from the features of one type of MRI image data as the principal parameter and the other as an auxiliary parameter. Figure 2This method employs a primary parameter MRI random selection mechanism to choose features from T1WI and T2WI MRI image data as primary parameter MRI features, and the other as auxiliary parameter MRI features. A multi-head self-attention mechanism is then used to fuse the auxiliary parameter features into the primary parameter features, adaptively adjusting the contribution weights of each parameter. This achieves information supplementation and feature fusion from all auxiliary parameter MRI features, enhancing the embedded feature representation capability of the primary parameter MRI and yielding fused features. Furthermore, for the fused features, a multi-layer 3D Transformer Block is used to perform depth modeling on the modulated primary parameter MRI fused features. Finally, sequential processing is performed through 1×1 3D convolution, normalization layer, 3×3 3D convolution, and normalization layer to output a 3D high-level feature map integrating bone, soft tissue, and other structural information as image features, ensuring accurate characterization of tumor morphology and invasion range in complex anatomical regions. The 3D convolutional layer projection and 3D Transformer Block structures in this process are initialized using pre-trained parameters from SAM-Med3D, providing a high-quality feature foundation for the segmentation of rare skull base tumors.

[0042] Alternatively, a cue encoder can be used to encode the cue points marked by the user in the MRI image data to obtain cue point codes. In one embodiment, random Fourier coding can be used to encode the position of the cue points; alternatively, the cue point type can be encoded to obtain a cue point type code, and then the cue point position code and the cue point type code can be concatenated to obtain the cue point code.

[0043] For the fusion features of the backbone parameters of the fusion auxiliary parameters of the MRI features output by the image encoder, as well as the cue point encoding, the fusion features of the cue point encoding and the cue point encoding can be subjected to alternating self-attention and cross-attention processing in the mask decoder to achieve bidirectional updates from cue point encoding to fusion features and from fusion features to cue point encoding, and output the updated fusion features, and generate a three-dimensional mask based on the updated fusion features.

[0044] In one specific embodiment, the mask decoder includes an update module, an upsampling module, and a mask generation module. The update module includes multiple TwoWayTransformers, which take the fused features output from the cue point encoding and serialization processing modules as input. They perform alternating self-attention and cross-attention operations on the cue point encoding and the fused features, achieving bidirectional updates from cue point encoding to fused features and from fused features to cue point encoding, and outputting the updated fused features. The TwoWayTransformer achieves bidirectional information updates between cue point encoding and fused features by alternating self-attention and cross-attention operations, thereby enabling the automatic skull base tumor segmentation model to segment tumors within a specified region using cue point encoding.

[0045] The fused features output by the update module can be upsampled by the upsampling module. For example, the fused features can be upsampled through two levels of transposed convolution, i.e., sequentially through a 2×2 3D convolution, a normalization layer, another 2×2 3D convolution, and a normalization layer, to restore the spatial resolution to 1 / 4 the size of the original image data. The mask generation module is used to process the upsampled fused features using a supernetwork structure composed of a three-layer fully connected MLP network to generate a 3D mask.

[0046] In one embodiment, the automatic segmentation model for skull base tumors trained based on the dataset includes: dividing the dataset into a training set, a test set, and a validation set to train the automatic segmentation model for skull base tumors; and using Dice loss and cross-entropy loss as objective functions to optimize the model parameters by minimizing the loss functions.

[0047] Specifically, in each training session, one type of MRI image data can be randomly selected as the primary parameter input, while at least one other MRI image data can be used as an auxiliary parameter input, thereby improving the model's adaptability to various parameter sequences. This allows the trained automatic skull base tumor segmentation model to handle complex cases with more parameter MRI inputs for segmentation. Under resource constraints or when some parameter MRIs are missing, the number of input parameter MRI types can be reduced while still achieving relatively accurate segmentation results. This significantly improves the versatility and automation level of the rare skull base tumor segmentation model. In this embodiment, data augmentation strategies such as rotation, flipping, and elastic deformation can also be employed during training to further expand sample diversity and mitigate the impact of data scarcity on model performance.

[0048] This invention acquires rare skull base tumor data from multiple patients and constructs a dataset. The rare skull base tumor data includes at least two types of MRI image data. An automatically segmented skull base tumor model is trained based on this dataset. The automatically segmented skull base tumor model randomly selects two types of MRI image data from the at least two types, and performs feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. This fully considers the information differences and complementarities of MRI with different parameters. By comprehensively utilizing MRI data with multiple parameters from the same patient, the versatility and robustness of the rare skull base tumor segmentation model are improved. Furthermore, for complex examples, this invention can input more parameter MRI to assist the model segmentation. Under conditions of limited resources or missing parameter MRI, it can reduce the types of input parameter MRI, greatly improving the versatility and automation level of the rare skull base tumor segmentation model. In addition, by introducing cue point encoding and performing alternating self-attention and cross-attention processing on cue point encoding and fused features, bidirectional updates from cue point encoding to fused features and from fused features to cue point encoding are achieved, enabling the segmentation model to effectively identify tumor regions.

[0049] Example 2 Figure 3 This is a schematic diagram of an automatic skull base tumor segmentation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: The data acquisition unit 301 is used to acquire rare skull base tumor data from multiple patients and construct a dataset; the rare skull base tumor data includes at least two types of MRI image data; Training unit 302 is used to train a pre-constructed automatic segmentation model for skull base tumors based on a dataset, thereby obtaining a trained automatic segmentation model for skull base tumors. The automatic segmentation model for skull base tumors is used to randomly select two types of MRI image data from at least two types of MRI image data, and to perform feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. Cue points are added to the tumor region of the at least two types of MRI image data, and after encoding the clue points, the fused features are updated based on the clue point encoding. The detection unit 303 is used to acquire at least two types of MRI image data of the patient to be tested, and input the at least two types of MRI image data into a trained automatic segmentation model for skull base tumors to segment the tumor region in the MRI images of the patient to be tested.

[0050] The automatic skull base tumor segmentation device provided in the embodiments of the present invention can execute the automatic skull base tumor segmentation device provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0051] Example 3 Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0052] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0053] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0054] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as an automated segmentation method for skull base tumors.

[0055] In some embodiments, an automated skull base tumor segmentation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the automated skull base tumor segmentation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform an automated skull base tumor segmentation method by any other suitable means (e.g., by means of firmware).

[0056] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0057] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0058] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0059] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0060] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0061] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0062] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0063] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An automatic segmentation method for skull base tumors, characterized in that, include: Data on rare skull base tumors from multiple patients were acquired and a dataset was constructed; the rare skull base tumor data included at least two types of MRI image data. The automatic segmentation model for skull base tumors is trained and constructed based on the dataset to obtain the trained automatic segmentation model for skull base tumors. The automatic segmentation model for skull base tumors is used to randomly select two types of MRI image data from at least two types of MRI image data, and to perform feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. Add cue points to the tumor region of the at least two MRI image data, encode the cue points, and update the fusion features based on the cue point encoding; Acquire at least two types of MRI image data from the patient to be tested, and input the at least two types of MRI image data into a trained automatic segmentation model for skull base tumors to segment the tumor region in the MRI images of the patient to be tested.

2. The automatic segmentation method for skull base tumors according to claim 1, characterized in that, Also includes: The rare skull base tumor data are preprocessed; the preprocessing includes at least normalization, alignment, annotation and slicing.

3. The automatic segmentation method for skull base tumors according to claim 2, characterized in that, The preprocessing includes: Intensity normalization was performed on at least two types of MRI image data; Using one of at least two MRI image datasets as a reference, perform affine transformations on the other datasets to achieve data alignment between the at least two MRI image datasets; Tumor regions in at least two types of MRI image data are labeled, and the MRI image data are sliced ​​according to a preset volume window to obtain slice data with a uniform volume window.

4. The automatic segmentation method for skull base tumors according to claim 1, characterized in that, The automatic segmentation model for skull base tumors includes a parameter selection module, an image encoder, a cue encoder, and a mask decoder. The parameter selection module is used to randomly select two MRI image data from at least two types of MRI image data. The image encoder is used to perform feature fusion on the second MRI image data using the first MRI image data of two types of MRI image data to obtain fused features; The prompt encoder is used to encode the prompt points marked by the user in the MRI image data; The mask decoder is used to take the cue point encoding and the fused features as input, and performs alternating self-attention and cross-attention processing on the cue point encoding and the fused features to achieve bidirectional updates from cue point encoding to fused features and from fused features to cue point encoding, and outputs the updated fused features, and generates a three-dimensional mask based on the updated fused features.

5. The automatic segmentation method for skull base tumors according to claim 4, characterized in that, Also includes: The image encoder includes a downsampling module, a feature extraction module, a master parameter selection module, a feature fusion module, a depth modeling module, and a serialization processing module; The mask decoder includes an update module, an upsampling module, and a mask generation module; The downsampling module is used to downsample the two selected MRI image data. The feature extraction module is used to extract features from the two types of MRI image data respectively; The main parameter selection module is used to select one feature of MRI image data as the main parameter and the other as an auxiliary parameter from the features of two MRI image data. The feature fusion module is used to fuse the features of auxiliary parameters into the main parameters through a multi-head self-attention mechanism to obtain fused features; The deep modeling module is used for deep modeling of the fused features; The serialization processing module is used to serialize the fused features after deep modeling through convolutional layers and normalization layers; The update module is used to take the fused features output by the cue point encoding and serialization processing module as input, and perform alternating self-attention and cross-attention processing on the cue point encoding and the fused features to achieve bidirectional updates from cue point encoding to fused features and from fused features to cue point encoding, and output the updated fused features. The upsampling module is used to upsample the fusion features output by the update module; The mask generation module is used to process the upsampled fused features using a supernetwork structure composed of three fully connected networks to generate a three-dimensional mask.

6. The automatic segmentation method for skull base tumors according to claim 5, characterized in that, Also includes: The depth modeling module includes a multi-layer 3D Transformer Block; The update module includes TwoWayTransformer.

7. The automatic segmentation method for skull base tumors according to claim 1, characterized in that, The automatic segmentation model for skull base tumors, trained and constructed based on the dataset, includes: The dataset was divided into training, testing, and validation sets to train the automatic segmentation model for skull base tumors. Dice loss and cross-entropy loss were used as objective functions, and the model parameters were optimized by minimizing the loss functions.

8. An automatic segmentation device for skull base tumors, characterized in that, include: A data acquisition unit is used to acquire rare skull base tumor data from multiple patients and construct a dataset; the rare skull base tumor data includes at least two types of MRI image data; The training unit is used to train the pre-constructed automatic segmentation model for skull base tumors based on the dataset, and obtain the trained automatic segmentation model for skull base tumors. The automatic segmentation model for skull base tumors is used to randomly select two types of MRI image data from at least two types of MRI image data, and perform feature fusion on the second type of MRI image data using the first type of MRI image data to obtain fused features. Add cue points to the tumor region of the at least two MRI image data, encode the cue points, and update the fusion features based on the cue point encoding; The detection unit is used to acquire at least two types of MRI image data from the patient to be tested, and input the at least two types of MRI image data into a trained automatic segmentation model for skull base tumors to segment the tumor region in the MRI images of the patient to be tested.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the automatic segmentation method for skull base tumors according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the automatic segmentation method for skull base tumors according to any one of claims 1-7.

Images