Gastric tumor image segmentation method, system, terminal and storage medium based on mixed model

Abstract

The present application relates to the technical field of image processing, and discloses a gastric tumor image segmentation method, system, terminal and storage medium based on a hybrid model, the method comprising: resampling a gastric tumor image and normalizing voxels to obtain a compressed image set of each target object; using a generative adversarial network to iteratively optimize the compressed image set to obtain a target fake image set, input an encoder of a target segmentation network, and obtain dimension fusion information after fusing the target fake image set; using a gating mechanism to screen multiple expert data to generate corresponding fusion weights; and using the expert data to train a tumor segmentation model to predict an output image prediction result for the compressed image set. The present application iteratively optimizes images, finally inputs high-quality samples to the model for training, and finally improves the generalization ability and robustness of the model without increasing the labeling cost, thereby improving the accuracy of the prediction result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, system, terminal, and computer-readable storage medium for gastric tumor image segmentation based on a hybrid model. Background Technology

[0002] Image segmentation of gastric tumors is of great significance for early diagnosis, screening, accurate staging, and treatment planning of gastric cancer. However, the complex structure of the stomach, blurred tumor boundaries, high image noise, and individual variability, coupled with the fact that tumor tissue and normal gastric wall tissue often overlap in grayscale and have blurred boundaries in images, lead to low accuracy and poor robustness of some traditional segmentation methods that rely on manually defined features, making it difficult to meet the standardized and efficient requirements of precision diagnosis and treatment.

[0003] Existing deep learning-based network models also have certain limitations. Segmentation models based on CNNs (Convolutional Neural Networks) have limited receptive fields, which restrict their ability to model long-distance dependencies and cannot effectively capture the global feature information of complex tumors in images. Segmentation models based on Transformers (a deep learning architecture based on self-attention mechanisms) can effectively enhance the segmentation ability by introducing self-attention mechanisms to assign weights to important information, thereby focusing on "key information". However, this also increases the number of parameters and the complexity of the training process.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a gastric tumor image segmentation method, system, terminal, and computer-readable storage medium based on a hybrid model, aiming to solve the problem that existing gastric tumor image segmentation methods still cannot meet the high performance and high precision requirements in clinical practice.

[0006] To achieve the above objectives, the present invention provides a gastric tumor image segmentation method based on a hybrid model, the gastric tumor image segmentation method based on a hybrid model comprising the following steps:

[0007] Multiple gastric tumor images are resampled based on the obtained target voxel spacing, and all voxels in each gastric tumor image are normalized to obtain a compressed image set of all gastric tumor images.

[0008] The compressed image set is iteratively optimized using a generative adversarial network to output a target fake image set.

[0009] The target fake image set is input into the encoder of the target segmentation network. The encoder filters multiple expert data in the constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data.

[0010] The initial tumor segmentation model is trained using all the expert data to obtain the target tumor segmentation model. The compressed image set is then input into the target tumor segmentation model, and prediction is performed using all the fusion weights to output the image prediction result.

[0011] Optionally, the gastric tumor image segmentation method based on a hybrid model, wherein the resampling of multiple gastric tumor images according to the acquired target voxel spacing, and the normalization of all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images, further includes:

[0012] Multiple gastric tumor images of target objects are acquired, and the voxel spacing between voxels in each gastric tumor image is statistically analyzed to obtain the voxel size difference, voxel spacing distribution, label set, and distribution intensity of each modality for each gastric tumor image. The data is then normalized to obtain the pre-training data.

[0013] Multiple spline interpolation operations are performed on the pre-training data to obtain multiple isotropic voxel axes. If the voxel spacing between all voxels on a voxel axis is greater than the voxel spacing between all voxels on other voxel axes, then the voxel axis is defined as an anisotropic target voxel axis.

[0014] Linear low-order interpolation is performed on the target voxel axis, and a training plan configuration file is constructed based on the voxel spacing, voxel intensity distribution, and memory budget of all voxels on the target voxel axis.

[0015] Optionally, the gastric tumor image segmentation method based on a hybrid model, wherein the resampling of multiple gastric tumor images according to the acquired target voxel spacing, and the normalization of all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images, specifically includes:

[0016] The gastric tumor images of all the target objects are resampled to the position of the target voxel spacing to obtain multiple foreground voxels in each gastric tumor image;

[0017] Multiple spline interpolation processes are performed on all the gastric tumor images, and the labels of each gastric tumor image are unified into tensors of a preset format to obtain the segmented image corresponding to each gastric tumor image;

[0018] Each segmented image is subjected to ROI settling to crop the noisy background in each segmented image to obtain the corresponding ROI region, and a corresponding key value is added to each ROI region and the corresponding gastric tumor image;

[0019] The voxel intensities of all foreground voxels in the ROI region are normalized to obtain a compressed image set for each target object.

[0020] Optionally, the gastric tumor image segmentation method based on a hybrid model, wherein the step of inputting the compressed image set into a generative adversarial network for iterative optimization and outputting a target fake image set specifically includes:

[0021] The compressed image set is sampled to obtain a random noise vector and a conditional label. The random noise vector and the conditional label are concatenated to obtain a fused embedding vector.

[0022] The fused embedding vector is input into the generator in the generative adversarial network, and the generator amplifies and decodes the fused embedding vector to obtain a three-dimensional data volume.

[0023] By constraining the three-dimensional data volume using a preset activation function, the grayscale intensity of all gastric tumor images in the compressed image set is obtained;

[0024] The three-dimensional data volume and all the grayscale intensities are input into the discriminator in the generative adversarial network. When the discriminator determines that there are multiple gastric tumor images as fake images based on all the grayscale intensities, the splicing points in the three-dimensional data volume are detected, optimized details are generated, and gradient signals are generated.

[0025] The compressed image set is optimized according to the optimization details, the generator is optimized according to the gradient signal, and the optimized compressed image set is iteratively optimized using the optimized generator until a qualified target fake image set is generated and output.

[0026] Optionally, the gastric tumor image segmentation method based on a hybrid model, wherein the step of inputting the target fake image set into the encoder of the target segmentation network, wherein the encoder filters multiple expert data from a pre-constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data set, further includes:

[0027] The target fake image set is divided into a target training image set, a target verification image set, and a target test image set, and training loss function and verification loss function are constructed using the Sigmoid activation function and binary cross-entropy;

[0028] The constructed segmentation network is iteratively trained using the target training image set. After one training iteration, the segmentation network is validated using the validation image set.

[0029] If there is a consecutive preset number of decreases in the training loss function and an increase in the verification loss function, then training is stopped, and the target segmentation network is obtained.

[0030] The decrease in the training loss function indicates that the difference between the true and predicted labels of the target training image set is smaller, while the increase in the validation loss function indicates that the difference between the true and predicted labels of the target training image set is larger.

[0031] Optionally, the gastric tumor image segmentation method based on a hybrid model, wherein the step of inputting the target fake image set into the encoder of the target segmentation network, wherein the encoder filters multiple expert data from a pre-constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data set, specifically includes:

[0032] The target training image set is input into the encoder of the target segmentation network. The residual block of the encoder performs convolutional alignment processing on the features corresponding to each foreground voxel in the target training image set, and adds all the processed features to obtain dimensionality fusion information.

[0033] The dimensionality fusion information is input into the Mamba module of the encoder, and the Mamba module calculates the computational score for each image type in the training plan configuration file based on the weight matrix of the gated network.

[0034] Gaussian noise is added to each of the calculated scores and then normalized to obtain a probability score for each of the image types.

[0035] Based on the target probability score with the highest preset number of scores, select the corresponding expert data from the training plan configuration file, and normalize all the target probability scores to obtain the corresponding fusion weights.

[0036] Optionally, the gastric tumor image segmentation method based on a hybrid model, wherein training an initial tumor segmentation model using all the expert data to obtain a target tumor segmentation model, inputting the compressed image set into the target tumor segmentation model, performing prediction using all the fusion weights, and outputting the image prediction result specifically includes:

[0037] All the expert data were input into the initial tumor segmentation model for training to obtain the target tumor segmentation model;

[0038] A test image set is extracted from the compressed image set, and the test image set is input into the target tumor segmentation network;

[0039] The target tumor segmentation network predicts the test image set based on all the fusion weights and outputs the image prediction results for all the gastric tumor images.

[0040] Furthermore, to achieve the above objectives, the present invention also provides a gastric tumor image segmentation system based on a hybrid model, wherein the gastric tumor image segmentation system based on a hybrid model includes:

[0041] The image preprocessing module is used to resample multiple gastric tumor images according to the acquired target voxel spacing, and to normalize all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images.

[0042] The image optimization module is used to iteratively optimize the compressed image set using a generative adversarial network and output a target fake image set.

[0043] The encoding module is used to input the target fake image set into the encoder of the target segmentation network. The encoder filters multiple expert data in the pre-constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data.

[0044] The prediction module is used to train the initial tumor segmentation model using all the expert data to obtain the target tumor segmentation model, input the compressed image set into the target tumor segmentation model, perform prediction using all the fusion weights, and output the image prediction result.

[0045] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a gastric tumor image segmentation program based on a hybrid model stored in the memory and executable on the processor, wherein when the gastric tumor image segmentation program based on the hybrid model is executed by the processor, it implements the steps of the gastric tumor image segmentation method based on a hybrid model as described above.

[0046] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a gastric tumor image segmentation program based on a hybrid model, and the gastric tumor image segmentation program based on a hybrid model, when executed by a processor, implements the steps of the gastric tumor image segmentation method based on a hybrid model as described above.

[0047] In this invention, multiple gastric tumor images are resampled based on the obtained target voxel spacing, and all voxels in each gastric tumor image are normalized to obtain a compressed image set of all gastric tumor images. A generative adversarial network (GAN) is used to iteratively optimize the compressed image set, outputting a target fake image set. The target fake image set is input into the encoder of the target segmentation network. The encoder uses a gating mechanism to filter multiple expert data from a pre-constructed training plan configuration file and generates fusion weights for each expert data set. An initial tumor segmentation model is trained using all the expert data to obtain a target tumor segmentation model. The compressed image set is then input into the target tumor segmentation model, and prediction is performed using all the fusion weights to output the image prediction result. This invention iteratively optimizes the images, ultimately inputting higher-quality samples into the model for training. This improves generalization ability and robustness without increasing annotation costs, thereby enhancing the accuracy of prediction results. Attached Figure Description

[0048] Figure 1 This is a flowchart of a preferred embodiment of the gastric tumor image segmentation method based on a hybrid model of the present invention;

[0049] Figure 2 This is a schematic diagram of the structure of a generative adversarial network in a preferred embodiment of the gastric tumor image segmentation method based on a hybrid model of the present invention.

[0050] Figure 3 This is a schematic diagram of the structure of the hybrid model in a preferred embodiment of the gastric tumor image segmentation method based on the hybrid model of the present invention;

[0051] Figure 4 This is a schematic diagram of the structure of the hybrid expert model, which is a preferred embodiment of the gastric tumor image segmentation method based on a hybrid model of the present invention.

[0052] Figure 5 This is a schematic diagram of the expert network structure of a preferred embodiment of the gastric tumor image segmentation method based on a hybrid model of the present invention;

[0053] Figure 6 This is a structural diagram of a preferred embodiment of the gastric tumor image segmentation system based on a hybrid model of the present invention;

[0054] Figure 7 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0056] The preferred embodiment of the present invention describes a gastric tumor image segmentation method based on a hybrid model, such as... Figure 1 As shown, the gastric tumor image segmentation method based on a hybrid model includes the following steps:

[0057] Step S10: Resample multiple gastric tumor images according to the obtained target voxel spacing, and normalize all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images.

[0058] In one of the embodiments disclosed in this invention, the gastric tumor dataset from ISICDM2024 (a dataset provided by a certain image computing and digital medicine symposium) was used for the experiment. The classification task of the experiment was binary classification, in which computed tomography (CT) was channel 0 and gastric tumor images were channel 1; the raw data was stored in NIFTI format (Neuroimaging Informatics Technology Initiative, a standardized file format designed specifically for neuroimaging).

[0059] First, gastric tumor images of multiple target objects are acquired. The voxel spacing between voxels in each gastric tumor image is statistically analyzed to obtain the voxel size difference, voxel spacing distribution, label set, and distribution intensity of each modality for each gastric tumor image. These values ​​are then normalized to obtain pre-training data. Multiple spline interpolation operations are performed on the pre-training data to obtain multiple isotropic voxel axes. If the voxel spacing between all voxels on a given voxel axis is greater than the voxel spacing between all voxels on other voxel axes, then that voxel axis is defined as an anisotropic target voxel axis. Linear low-order interpolation is performed on the target voxel axis, and a training plan configuration file is constructed based on the voxel spacing, voxel intensity distribution, and memory budget on the target voxel axis.

[0060] Before conducting the entire experiment, a standard training plan configuration file needs to be constructed to provide a standard for subsequent image preprocessing and reduce background noise outside the task objective. By reading all target objects, the median and distribution of voxel spacing, the degree of anisotropy of voxel size, the intensity distribution of each modality (mean, variance, quantile of foreground or global), and the label set are statistically analyzed in the gastric tumor images of each target object. The content is saved in JSON (JavaScript Object Notation, a lightweight data exchange format) for subsequent cropping and normalization processing, improving data clarity and reducing redundant characters.

[0061] Furthermore, by reading the JSON file containing the feature statistics of the dataset (and the features in each gastric tumor image as described above), the voxel spacing in each gastric tumor image is calculated. Cubic spline interpolation is performed on isotropic voxels (i.e., each voxel has the same voxel resolution on the horizontal, vertical, and 3D axes). For each voxel, neighboring voxels are interpolated using cubic spline interpolation, thereby preserving a smooth gastric tumor image and retaining the detailed features in the gastric tumor image.

[0062] If the voxel spacing on a certain axis is significantly greater than that on other axes, it can be considered that the anisotropy on that axis is strong. The interpolation method for the voxels on that axis is then changed to linear low-order interpolation with a two-point linear transition. This avoids smooth transitions or artifacts in the washing direction. Then, based on the voxel spacing, voxel distribution, and memory budget in each gastric tumor image, a training plan configuration file is generated to guide subsequent data optimization.

[0063] Further, the gastric tumor images of all the target objects are resampled to the target voxel spacing position to obtain multiple foreground voxels in each gastric tumor image; multiple spline interpolation processes are performed on all the gastric tumor images, and the labels of each gastric tumor image are unified into tensors of a preset format to obtain segmented images corresponding to each gastric tumor image; ROI (Region of Interest) settling is performed on each segmented image to crop the noisy background in each segmented image to obtain the corresponding ROI region, and a corresponding key value is added to each ROI region and the corresponding gastric tumor image; the voxel intensity of all the foreground voxels in the ROI region is normalized to obtain a compressed image set for each target object.

[0064] In the process of resampling each gastric tumor image, the aforementioned training plan configuration file is used. Images are cropped according to the configuration file, target voxels are extracted, and cubic spline interpolation is performed on the processed images. The nearest neighbor algorithm is used for the image labels to ensure that the class boundaries between images are not disrupted, while also allowing storage in tensor form. Before cropping the gastric tumor images, the tumor region in each image is first determined according to the training plan configuration file, and foreground voxels (i.e., voxels associated with the task target) are identified. Cropping is then performed based on these foreground voxels to remove unnecessary noise and background, effectively reducing storage and training redundancy. Then, a key value is added to each sample (i.e., the type of gastric tumor image) within the cropped area, facilitating the reversible restoration of the original image coordinates during inference. Finally, the foreground voxels of each resampled image are normalized within the range of 0.5%-99.5%, facilitating subsequent image processing and model input, and improving data training efficiency.

[0065] Furthermore, the segmented images of all the target objects are processed to obtain compressed data for each target object. This effectively improves the efficiency of data loading during training, facilitates model reading, and significantly improves GPU memory efficiency.

[0066] Step S20: Iteratively optimize the compressed image set using a generative adversarial network to output a target fake image set.

[0067] In existing technologies, the insufficient scale of medical image data often leads to difficulties in model training convergence and insufficient segmentation capabilities. Therefore, Reinforcement Generative Adversarial Networks (RGANs) are frequently introduced to synthesize unlabeled samples and organically integrate them into the training process. However, traditional GANs suffer from poor interpretability and are difficult to train, primarily due to the lack of effective information. The generator input of a GAN is randomly generated noise z, and the update information for iterative sample generation comes from the data produced by the generator being input into the discriminator to obtain discrimination information. However, the information provided by the discriminator is often very limited and does not tell the generator which direction to improve in order to create higher-fidelity "fake data". Therefore, this invention changes the sample input of the generator to real samples and real labels mask. The generator G generates a single-channel CT with the conditional input [mask, noise] (noise represents noise); the discriminator D performs local fidelity discrimination on the input string. During training, real (ct, mask) and (fake, mask) pairs (fake represents generated fake images) are used in a least squares generative adversarial network (LSGAN). L1 (a loss function, i.e., mean absolute error) is used to reconstruct and ensure strong constraints, thereby improving the confidence in the discriminative information.

[0068] Specifically, the compressed image set is sampled to obtain a random noise vector and a conditional label. The random noise vector and the conditional label are concatenated to obtain a fused embedding vector. The fused embedding vector is input into the generator in the generative adversarial network (GAN). The generator amplifies and decodes the fused embedding vector to obtain a three-dimensional data volume. The three-dimensional data volume is constrained using a preset activation function to obtain the grayscale intensity of all gastric tumor images in the compressed image set. The three-dimensional data volume and all the grayscale intensities are input into the discriminator in the GAN. When the discriminator determines that multiple gastric tumor images are fake images based on all the grayscale intensities, the concatenation points in the three-dimensional data volume are detected, optimization details are generated, and gradient signals are generated. The compressed image set is optimized based on the optimization details, the generator is optimized based on the gradient signals, and the optimized compressed image set is iteratively optimized using the optimized generator until a qualified target fake image set is generated and output.

[0069] Among them, such as Figure 2As shown, the conditional branch of the RGAN disclosed in this invention includes a generator and a discriminator. The generator receives a random noise vector and a conditional label sampled from a normally distributed data set. First, the random noise vector and the conditional label are concatenated to obtain a fused embedding vector. Then, the vector passes through a series of layers in the generator (including transposed convolutional layers, upsampling layers, etc.). These layers decode and amplify the fused embedding vector to obtain a 3D data volume with the target spatial size. Finally, the generator's last layer uses an activation function (such as the Tanh activation function or the Sigmoid activation function) to constrain the output value to a reasonable range of pixel values ​​to simulate the grayscale of a CT image.

[0070] Furthermore, the discriminator returns discriminative information to the image input to the generator. The discriminator first constrains the generator's boundaries at the local texture and label alignment (i.e., where fusion occurs) to generate more realistic details, and then continuously optimizes the generator's output. The two modules undergo stable adversarial training to pursue "high-fidelity" fake images, thereby enhancing generalization ability and segmentation performance.

[0071] Further, the target fake image set is divided into a target training image set, a target verification image set, and a target test image set. A training loss function and a verification loss function are constructed using the Sigmoid activation function and binary cross-entropy. The constructed segmentation network is iteratively trained using the target training image set. After one training iteration, the segmentation network is verified using the verification image set. If the training loss function decreases for a consecutive preset number of times, and the verification loss function increases, training is stopped, and the target segmentation network is obtained. Here, a decrease in the training loss function indicates a smaller difference between the true and predicted labels in the target training image set, and an increase in the verification loss function indicates a larger difference between the true and predicted labels in the target training image set.

[0072] In the embodiments disclosed in this invention, before training, the dataset is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The Sigmoid activation function and the binary cross-entropy loss function are used as loss functions. The training cycle is set to a maximum of 350 epochs (one training cycle). When the training loss decreases for ten consecutive epochs while the validation loss increases, an early stopping strategy is used to stop model training. On two A800 graphics cards with 80GB of VRAM (graphics cards specifically designed for AI computing and high-performance optimization), the batch size is set to 4. Each time, only the weights of the graphics card under rank 0 (the primary card) are changed to prevent VRAM occupation and write locks. The optimizer uses an SGD (Stochastic Gradient Descent) optimizer with Nesterov momentum, combined with a poly (Polynomial Decay) learning slightly decay strategy to dynamically adjust the learning rate.

[0073] The iterative optimization process is as follows: First, a batch of real images and their labels are sampled from a real image set. A set of noise is randomly generated and input into the generator. After the generator outputs grayscale intensity, the discriminator performs detail optimization on the input image to generate a fake image. During the separate operation of the generator and discriminator, corresponding loss functions are constructed and trained based on these functions. The generator is trained to make its output infinitely close to the intensity distribution of the real image; the discriminator is trained to make it infinitely close to "real" (i.e., classify as a real image) for the input real image and infinitely close to "fake" (i.e., classify as a fake image) for the generated fake image. When the image generated by the generator is identified as "fake" by the discriminator, the discriminator backpropagates a gradient signal to the generator. This signal clearly indicates in which aspects (such as local texture, boundary sharpness, and overall structure) the generated image differs from the real data. The generator uses this gradient to update its parameters through the backpropagation algorithm, thereby improving these deficiencies in the next generation. This process is repeated continuously. Guided by the "criticism" provided by the discriminator, the generator continuously fine-tunes itself, gradually learning to synthesize high-quality images that contain rich details and can deceive the discriminator.

[0074] Furthermore, in addition to RGAN, the Stable Diffusion model can also be utilized. This model is highly scalable, not only limited to data sample generation but also exhibiting high performance in feature information capture and boundary processing. Moreover, research combining the diffusion model with medical image analysis has significant potential value. Different medical imaging modalities each have their own advantages, disadvantages, and applicable scenarios. It can not only learn the mapping relationships between images of different modalities to achieve cross-modal conversion but also reconstruct image resolution and denoise. Such a module with diverse features can greatly improve model performance and segmentation accuracy, helping users further enhance the models they need.

[0075] Step S30: Input the target fake image set into the encoder of the target segmentation network. The encoder filters multiple expert data in the constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data.

[0076] To address the shortcomings of existing segmentation models, such as low segmentation accuracy and insufficient key feature extraction capabilities when handling complex and ambiguous tumor information, this invention introduces MoE (Mixture of Experts) during the upsampling stage to further enhance feature extraction capabilities. The MoE model dynamically selects 1-2 of the most suitable expert networks to process a feature, avoiding the limitations of traditional "single architecture handling all features." This specialized, fine-grained division of labor significantly improves the specificity and completeness of feature extraction. Secondly, MoE's gating is a dynamic weighting, unlike traditional feature extraction-static fusion (fixed weights). This dynamic weighting adjusts the weights of expert outputs based on the importance of real-time input features, achieving "on-demand integration." This better adapts to the dynamic changes in gastric tumor images, avoids "invalid feature interference" (such as noise features in areas obscured by gastric mucus), and enables intelligent collaboration of multi-dimensional features.

[0077] Specifically, the target training image set is input into the encoder of the target segmentation network. The residual block of the encoder performs convolutional alignment processing on the features corresponding to each foreground voxel in the target training image set, and adds all the processed features to obtain dimensionality fusion information. The dimensionality fusion information is input into the Mamba module of the encoder. The Mamba module calculates the computational score for each image type in the training plan configuration file according to the weight matrix of the gated network. Gaussian noise is added to each computational score and normalized to obtain the probability score for each image type. Based on the target probability score with the highest preset number of scores, corresponding expert data is selected from the training plan configuration file. After normalizing all the target probability scores, the corresponding fusion weights are obtained.

[0078] Among them, such as Figure 3 As shown, the encoder in the hybrid network model disclosed in this invention is a five-layer cascaded downsampling layer. Each layer consists of two consecutive basic residual blocks, followed by a Mamba block (a block in SSM) based on SSM (Selective State Space Model), and then the input is fed into the MoE model. The residual block aligns the input through two 1×1 convolutions, and after non-linear activation, it sums the features at each corresponding position to integrate information from different dimensions. The Mamba block further enhances the long-distance dependency modeling capability on the semantic basis, allowing the model to better understand the relationships between different parts of the image.

[0079] In this process, for the target training image set input into the encoder, the gated network calculates a score for each target training image in the training plan configuration file based on its weight matrix to express the correlation between each expert and the input feature. Gaussian noise is introduced into this score. Then, the Softmax operation selects the two "experts" with the highest probability according to the probability distribution. The probability scores of these two "experts" are then normalized again and used as the fusion weight output of the expert network, thereby amplifying the boundary ambiguity information and feature extraction that were ignored by the original network.

[0080] Furthermore, the decoder in the hybrid network model has five cascaded upsampling layers to restore the original image size. The bottom layer consists of only a single basic decoder, while the other layers are preceded by Mamba blocks. The decoder consists of a set of upsampling modules and two 3x3 convolutional layers. Upsampling is performed through transposed convolution or bilinear interpolation, gradually doubling the feature map space size to restore the image resolution. Each convolution is followed by a batch normalization and activation function to refine and fuse information, reduce the number of channels, and ultimately restore the image to the size matching the segmentation labels.

[0081] Furthermore, in the hybrid expert model disclosed in this invention, each expert is a separate perceptron. Each expert network or auxiliary loss function can be improved according to the specific task in the experimental requirements to optimize the allocation of routes each time, encourage a more even distribution of expert networks, and avoid the situation where a few expert networks are "overworked" and most expert networks are "idle".

[0082] Step S40: Train the initial tumor segmentation model using all the expert data to obtain the target tumor segmentation model. Input the compressed image set into the target tumor segmentation model, use all the fusion weights to make predictions, and output the image prediction results.

[0083] Specifically, all the expert data is input into the initial tumor segmentation model for training to obtain the target tumor segmentation model; a test image set is extracted from the compressed image set and input into the target tumor segmentation network; the target tumor segmentation network predicts the test image set according to all the fusion weights and outputs the image prediction results of all the gastric tumor images.

[0084] Among them, such as Figure 4 As shown, the hybrid expert model selects suitable expert data based on the test image set. Using this selected expert data, the initial tumor segmentation model is trained to obtain the target tumor segmentation model. Further, as... Figure 5 The diagram illustrates the resulting structure of each expert network. This invention adjusts the training strategy to a hybrid supervised learning paradigm, with weak reinforcement of the teacher network's reception and strong reinforcement of the student network's reception. High-confidence pseudo-labels output by the teacher are used to constrain student consistency, and a separate consistency branch is added for "synthetic unlabeled samples." In addition to the original consistency loss for real unlabeled samples, an extra loss is calculated for the generated unlabeled samples to utilize more pseudo-data to constrain students and improve fitting diversity. Through these improvements, the model's generalization, robustness, and segmentation accuracy can be significantly increased.

[0085] Furthermore, the segmented test set is input into the pre-trained target tumor segmentation model, and a pre-written script is executed to call the model with the best performance on the validation set, which contains the trained model parameters. Executing this prediction script will allow the unlabeled test set to go through the model structure once, and finally output the true label mask to predict the test set, obtain the final prediction result, and save the image in NIFTI format.

[0086] In the embodiments disclosed in this invention, the gastric tumor dataset from ISICDM2024 (a dataset provided by a conference on image computing and digital medicine) was used for training, validation, and testing. Compared to other types of tumor segmentation, this invention addresses issues such as unclear foreground features, blurred boundaries, or overlapping areas on CT images by incorporating the MoE module. This module assigns different challenges to different experts for on-demand processing and output, and activates only the two most suitable experts each time based on routing scores. The module also exhibits sparsity, allowing computation to be performed on specific parts of the system without activating all parameters, enabling the model to scale without incurring excessive computational costs. The "expert consultation" mechanism of MoE outperformed other models in this experiment, fully demonstrating the effectiveness of the proposed MoE and significantly improving the model's segmentation performance.

[0087] Furthermore, to address the scarcity of CT image data for gastric tumors, this invention designs an enhanced generative adversarial network (RGAN) that generates fake data based on real images and labels as prior samples for training, thereby improving the model's generalization ability and robustness. Through this design, some diffuse gastric tumors can be identified and accurately segmented by the model, and experimental results fully demonstrate the effectiveness of this module.

[0088] This invention iteratively optimizes images and ultimately inputs high-quality samples into the model for training. This improves generalization ability and robustness without increasing annotation costs, thereby enhancing the accuracy of prediction results.

[0089] Furthermore, such as Figure 6 As shown, based on the above-mentioned gastric tumor image segmentation method based on a hybrid model, the present invention also provides a gastric tumor image segmentation system based on a hybrid model, wherein the gastric tumor image segmentation system based on a hybrid model includes:

[0090] Image preprocessing module 51 is used to resample multiple gastric tumor images according to the acquired target voxel spacing, and normalize all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images.

[0091] Image optimization module 52 is used to iteratively optimize the compressed image set using a generative adversarial network and output a target fake image set;

[0092] The encoding module 53 is used to input the target fake image set into the encoder of the target segmentation network. The encoder filters multiple expert data in the constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data.

[0093] The prediction module 54 is used to train the initial tumor segmentation model using all the expert data to obtain the target tumor segmentation model, input the compressed image set into the target tumor segmentation model, perform prediction using all the fusion weights, and output the image prediction result.

[0094] Furthermore, such as Figure 7 As shown, based on the above-mentioned gastric tumor image segmentation method and system based on a hybrid model, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 7 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0095] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a gastric tumor image segmentation program 40 based on a hybrid model, which can be executed by the processor 10 to implement the gastric tumor image segmentation method based on a hybrid model in this application.

[0096] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the gastric tumor image segmentation method based on the hybrid model.

[0097] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0098] In one embodiment, when the processor 10 executes the gastric tumor image segmentation program 40 based on the hybrid model in the memory 20, the following steps are performed:

[0099] Multiple gastric tumor images are resampled based on the obtained target voxel spacing, and all voxels in each gastric tumor image are normalized to obtain a compressed image set of all gastric tumor images.

[0100] The compressed image set is iteratively optimized using a generative adversarial network to output a target fake image set.

[0101] The target fake image set is input into the encoder of the target segmentation network. The encoder filters multiple expert data in the constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data.

[0102] The initial tumor segmentation model is trained using all the expert data to obtain the target tumor segmentation model. The compressed image set is then input into the target tumor segmentation model, and prediction is performed using all the fusion weights to output the image prediction result.

[0103] The step of resampling multiple gastric tumor images based on the acquired target voxel spacing, and normalizing all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images, further includes:

[0104] Multiple gastric tumor images of target objects are acquired, and the voxel spacing between voxels in each gastric tumor image is statistically analyzed to obtain the voxel size difference, voxel spacing distribution, label set, and distribution intensity of each modality for each gastric tumor image. The data is then normalized to obtain the pre-training data.

[0105] Multiple spline interpolation operations are performed on the pre-training data to obtain multiple isotropic voxel axes. If the voxel spacing between all voxels on a voxel axis is greater than the voxel spacing between all voxels on other voxel axes, then the voxel axis is defined as an anisotropic target voxel axis.

[0106] Linear low-order interpolation is performed on the target voxel axis, and a training plan configuration file is constructed based on the voxel spacing, voxel intensity distribution, and memory budget of all voxels on the target voxel axis.

[0107] Specifically, the step of resampling multiple gastric tumor images based on the acquired target voxel spacing, and normalizing all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images includes:

[0108] The gastric tumor images of all the target objects are resampled to the position of the target voxel spacing to obtain multiple foreground voxels in each gastric tumor image;

[0109] Multiple spline interpolation processes are performed on all the gastric tumor images, and the labels of each gastric tumor image are unified into tensors of a preset format to obtain the segmented image corresponding to each gastric tumor image;

[0110] Each segmented image is subjected to ROI settling to crop the noisy background in each segmented image to obtain the corresponding ROI region, and a corresponding key value is added to each ROI region and the corresponding gastric tumor image;

[0111] The voxel intensities of all foreground voxels in the ROI region are normalized to obtain a compressed image set for each target object.

[0112] Specifically, the step of inputting the compressed image set into a generative adversarial network for iterative optimization and outputting a target fake image set includes:

[0113] The compressed image set is sampled to obtain a random noise vector and a conditional label. The random noise vector and the conditional label are concatenated to obtain a fused embedding vector.

[0114] The fused embedding vector is input into the generator in the generative adversarial network, and the generator amplifies and decodes the fused embedding vector to obtain a three-dimensional data volume.

[0115] By constraining the three-dimensional data volume using a preset activation function, the grayscale intensity of all gastric tumor images in the compressed image set is obtained;

[0116] The three-dimensional data volume and all the grayscale intensities are input into the discriminator in the generative adversarial network. When the discriminator determines that there are multiple gastric tumor images as fake images based on all the grayscale intensities, the splicing points in the three-dimensional data volume are detected, optimized details are generated, and gradient signals are generated.

[0117] The compressed image set is optimized according to the optimization details, the generator is optimized according to the gradient signal, and the optimized compressed image set is iteratively optimized using the optimized generator until a qualified target fake image set is generated and output.

[0118] The process of inputting the target fake image set into the encoder of the target segmentation network, wherein the encoder filters multiple expert data from a pre-constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data set, further includes:

[0119] The target fake image set is divided into a target training image set, a target verification image set, and a target test image set, and training loss function and verification loss function are constructed using the Sigmoid activation function and binary cross-entropy;

[0120] The constructed segmentation network is iteratively trained using the target training image set. After one training iteration, the segmentation network is validated using the validation image set.

[0121] If there is a consecutive preset number of decreases in the training loss function and an increase in the verification loss function, then training is stopped, and the target segmentation network is obtained.

[0122] The decrease in the training loss function indicates that the difference between the true and predicted labels of the target training image set is smaller, while the increase in the validation loss function indicates that the difference between the true and predicted labels of the target training image set is larger.

[0123] Specifically, the step of inputting the target fake image set into the encoder of the target segmentation network, wherein the encoder filters multiple expert data from the constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data set, includes:

[0124] The target training image set is input into the encoder of the target segmentation network. The residual block of the encoder performs convolutional alignment processing on the features corresponding to each foreground voxel in the target training image set, and adds all the processed features to obtain dimensionality fusion information.

[0125] The dimensionality fusion information is input into the Mamba module of the encoder, and the Mamba module calculates the computational score for each image type in the training plan configuration file based on the weight matrix of the gated network.

[0126] Gaussian noise is added to each of the calculated scores and then normalized to obtain a probability score for each of the image types.

[0127] Based on the target probability score with the highest preset number of scores, select the corresponding expert data from the training plan configuration file, and normalize all the target probability scores to obtain the corresponding fusion weights.

[0128] Specifically, the process of training an initial tumor segmentation model using all the expert data to obtain a target tumor segmentation model, inputting the compressed image set into the target tumor segmentation model, performing prediction using all the fusion weights, and outputting the image prediction result includes:

[0129] All the expert data were input into the initial tumor segmentation model for training to obtain the target tumor segmentation model;

[0130] A test image set is extracted from the compressed image set, and the test image set is input into the target tumor segmentation network;

[0131] The target tumor segmentation network predicts the test image set based on all the fusion weights and outputs the image prediction results for all the gastric tumor images.

[0132] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a gastric tumor image segmentation program based on a hybrid model, the gastric tumor image segmentation program based on a hybrid model being executed by a processor to implement the steps of the gastric tumor image segmentation method based on a hybrid model as described above.

[0133] In summary, this invention provides a gastric tumor image segmentation method and related equipment based on a hybrid model. The method includes: resampling multiple gastric tumor images according to the obtained target voxel spacing, and normalizing all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images; iteratively optimizing the compressed image set using a generative adversarial network to output a target fake image set; inputting the target fake image set into the encoder of a target segmentation network, wherein the encoder filters multiple expert data in a pre-constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data; training an initial tumor segmentation model using all the expert data to obtain a target tumor segmentation model; inputting the compressed image set into the target tumor segmentation model; using all the fusion weights for prediction; and outputting the image prediction result. This invention iteratively optimizes the images, ultimately inputting higher-quality samples into the model for training, thereby improving generalization ability and robustness without increasing annotation costs, and improving the accuracy of prediction results.

[0134] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0135] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0136] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A gastric tumor image segmentation method based on a hybrid model, characterized in that, The gastric tumor image segmentation method based on a hybrid model includes: Multiple gastric tumor images are resampled based on the obtained target voxel spacing, and all voxels in each gastric tumor image are normalized to obtain a compressed image set of all gastric tumor images. The compressed image set is iteratively optimized using a generative adversarial network to output a target fake image set, specifically including: The compressed image set is sampled to obtain a random noise vector and a conditional label. The random noise vector and the conditional label are concatenated to obtain a fused embedding vector. The fused embedding vector is input into the generator in the generative adversarial network, and the generator amplifies and decodes the fused embedding vector to obtain a three-dimensional data volume. By constraining the three-dimensional data volume using a preset activation function, the grayscale intensity of all gastric tumor images in the compressed image set is obtained; The three-dimensional data volume and all the grayscale intensities are input into the discriminator in the generative adversarial network. When the discriminator determines that there are multiple gastric tumor images as fake images based on all the grayscale intensities, the splicing points in the three-dimensional data volume are detected, optimized details are generated, and gradient signals are generated. The compressed image set is optimized according to the optimization details, the generator is optimized according to the gradient signal, and the optimized compressed image set is iteratively optimized using the optimized generator until a qualified target fake image set is generated and output. The target fake image set is input into the encoder of the target segmentation network. The encoder filters multiple expert data in the constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data. The initial tumor segmentation model is trained using all the expert data to obtain the target tumor segmentation model. The compressed image set is then input into the target tumor segmentation model, and prediction is performed using all the fusion weights to output the image prediction result.

2. The gastric tumor image segmentation method based on a hybrid model according to claim 1, characterized in that, The step of resampling multiple gastric tumor images based on the acquired target voxel spacing, and normalizing all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images, further includes: Multiple gastric tumor images of target objects are acquired, and the voxel spacing between voxels in each gastric tumor image is statistically analyzed to obtain the voxel size difference, voxel spacing distribution, label set, and distribution intensity of each modality for each gastric tumor image. The data is then normalized to obtain the pre-training data. Multiple spline interpolation operations are performed on the pre-training data to obtain multiple isotropic voxel axes. If the voxel spacing between all voxels on a voxel axis is greater than the voxel spacing between all voxels on other voxel axes, then the voxel axis is defined as an anisotropic target voxel axis. Linear low-order interpolation is performed on the target voxel axis, and a training plan configuration file is constructed based on the voxel spacing, voxel intensity distribution, and memory budget of all voxels on the target voxel axis.

3. The gastric tumor image segmentation method based on a hybrid model according to claim 2, characterized in that, The step of resampling multiple gastric tumor images based on the acquired target voxel spacing, and normalizing all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images specifically includes: The gastric tumor images of all the target objects are resampled to the position of the target voxel spacing to obtain multiple foreground voxels in each gastric tumor image; Multiple spline interpolation processes are performed on all the gastric tumor images, and the labels of each gastric tumor image are unified into tensors of a preset format to obtain the segmented image corresponding to each gastric tumor image; Each segmented image is subjected to ROI settling to crop the noisy background in each segmented image to obtain the corresponding ROI region, and a corresponding key value is added to each ROI region and the corresponding gastric tumor image; The voxel intensities of all foreground voxels in the ROI region are normalized to obtain a compressed image set for each target object.

4. The gastric tumor image segmentation method based on a hybrid model according to claim 1, characterized in that, The process of inputting the target fake image set into the encoder of the target segmentation network, wherein the encoder filters multiple expert data from a pre-constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data set, further includes: The target fake image set is divided into a target training image set, a target verification image set, and a target test image set, and training loss function and verification loss function are constructed using the Sigmoid activation function and binary cross-entropy; The constructed segmentation network is iteratively trained using the target training image set. After one training iteration, the segmentation network is validated using the validation image set. If there is a consecutive preset number of decreases in the training loss function and an increase in the verification loss function, then training is stopped, and the target segmentation network is obtained. The decrease in the training loss function indicates that the difference between the true and predicted labels of the target training image set is smaller, while the increase in the validation loss function indicates that the difference between the true and predicted labels of the target training image set is larger.

5. The gastric tumor image segmentation method based on a hybrid model according to claim 4, characterized in that, The step of inputting the target fake image set into the encoder of the target segmentation network, wherein the encoder filters multiple expert data from the constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data set, specifically includes: The target training image set is input into the encoder of the target segmentation network. The residual block of the encoder performs convolutional alignment processing on the features corresponding to each foreground voxel in the target training image set, and adds all the processed features to obtain dimensionality fusion information. The dimensionality fusion information is input into the Mamba module of the encoder, and the Mamba module calculates the computational score for each image type in the training plan configuration file based on the weight matrix of the gated network. Gaussian noise is added to each of the calculated scores and then normalized to obtain a probability score for each of the image types. Based on the target probability score with the highest preset number of scores, select the corresponding expert data from the training plan configuration file, and normalize all the target probability scores to obtain the corresponding fusion weights.

6. The gastric tumor image segmentation method based on a hybrid model according to claim 1, characterized in that, The process of training an initial tumor segmentation model using all the expert data to obtain a target tumor segmentation model, inputting the compressed image set into the target tumor segmentation model, performing prediction using all the fusion weights, and outputting the image prediction result specifically includes: All the expert data were input into the initial tumor segmentation model for training to obtain the target tumor segmentation model; A test image set is extracted from the compressed image set, and the test image set is input into the target tumor segmentation model; The target tumor segmentation model predicts the test image set based on all the fusion weights and outputs the image prediction results for all the gastric tumor images.

7. A gastric tumor image segmentation system based on a hybrid model, characterized in that, The gastric tumor image segmentation system based on the hybrid model is applied to the gastric tumor image segmentation method based on the hybrid model as described in any one of claims 1-6, wherein the gastric tumor image segmentation system based on the hybrid model comprises: The image preprocessing module is used to resample multiple gastric tumor images according to the acquired target voxel spacing, and to normalize all voxels in each gastric tumor image to obtain a compressed image set of all gastric tumor images. The image optimization module is used to iteratively optimize the compressed image set using a generative adversarial network and output a target fake image set. The encoding module is used to input the target fake image set into the encoder of the target segmentation network. The encoder filters multiple expert data in the pre-constructed training plan configuration file through a gating mechanism and generates fusion weights for each expert data. The prediction module is used to train the initial tumor segmentation model using all the expert data to obtain the target tumor segmentation model, input the compressed image set into the target tumor segmentation model, perform prediction using all the fusion weights, and output the image prediction result.

8. A terminal, characterized in that, The terminal includes: a memory, a processor, and a gastric tumor image segmentation program based on a hybrid model stored in the memory and executable on the processor. When the gastric tumor image segmentation program based on the hybrid model is executed by the processor, it implements the steps of the gastric tumor image segmentation method based on a hybrid model as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a gastric tumor image segmentation program based on a hybrid model, which, when executed by a processor, implements the steps of the gastric tumor image segmentation method based on a hybrid model as described in any one of claims 1-6.

Images