Individualized abdominal organ segmentation and reconstruction method based on clinical big data and state space model
By combining clinical big data and state-space models to create an individualized abdominal organ segmentation method, and utilizing a joint architecture of CNN and Mamba models, the time-consuming and inaccurate problems of existing technologies are solved, enabling fast and accurate medical image processing and 3D reconstruction.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- QINGDAO UNIV
- Filing Date
- 2024-12-30
- Publication Date
- 2026-07-02
AI Technical Summary
Existing medical image processing methods are time-consuming and inaccurate when processing large-scale datasets, especially when segmenting complex lesions such as tumors, where it is difficult to accurately capture boundaries and internal structures. Furthermore, existing automated segmentation methods lack individualized processing, resulting in insufficient accuracy in diagnosis and treatment.
An individualized abdominal organ segmentation method based on clinical big data and state-space model is adopted. By combining CNN and Mamba model to capture local features in the low-order part of the encoder and extract global dependency features in the high-order part, combined with data augmentation and post-processing algorithms, the segmentation accuracy and 3D reconstruction effect are improved.
It enables rapid and accurate medical image processing, improves the effect and segmentation accuracy of 3D reconstruction, adapts to the complex structure of clinical big data, and enhances the model's adaptability to changes in lighting and tissue deformation.
Smart Images
Figure CN2024144002_02072026_PF_FP_ABST
Abstract
Description
A Personalized Method for Peritoneal Organ Segmentation and Reconstruction Based on Clinical Big Data and State-Space Model Technical Field
[0001] This application relates to the field of medical image processing technology, specifically a method for individualized abdominal organ segmentation and reconstruction based on clinical big data and state space models. Background Technology
[0002] Traditional medical image processing methods, especially manual segmentation, require doctors or technicians to examine each pixel one by one, which is extremely time-consuming when processing large-scale datasets. Due to human factors, the accuracy and repeatability of manual segmentation are difficult to guarantee, and there may be significant differences between different operators.
[0003] Existing automated segmentation methods often rely on predefined rules, which may not be adaptable to the diversity of different patients and pathological states. Furthermore, for complex lesions such as tumors, current technologies may fail to accurately capture their boundaries and internal structures during identification and segmentation. While big data holds immense potential in medical image analysis, it remains largely untapped, primarily due to high data heterogeneity and processing difficulties. Medical image data originates from diverse scanning devices and parameter settings, posing challenges to unified data processing and analysis. High-precision automated segmentation and 3D reconstruction often require substantial computational resources, which may be difficult to achieve in resource-constrained medical institutions. Moreover, existing image processing technologies often employ general algorithms, lacking personalized processing for individual differences, which may affect the accuracy of diagnosis and treatment.
[0004] Chinese patent application CN118691820A discloses a multimodal fusion segmentation method and apparatus based on prior information and a Mamba hybrid model. It inputs a set of remote sensing images to be processed into a pre-trained multimodal fusion segmentation network for remote sensing images; outputs semantic segmentation results of ground features in the processed remote sensing image set. The pre-constructed multimodal fusion segmentation network consists of a U-shaped symmetrical encoder-decoder structure; and the encoder part adopts a Mamba-CNN hybrid model. In this way, prior information can be combined with multimodal fusion to effectively fuse RGB and DSM remote sensing images, thereby improving semantic segmentation accuracy. However, this method cannot be directly applied to the 3D reconstruction of individualized abdominal organs, nor can it solve the problems of large processing volume and high data heterogeneity in medical imaging data.
[0005] Therefore, there is an urgent need to develop a new personalized method for segmenting and reconstructing abdominal organs to effectively improve the speed and accuracy of medical image processing and enhance the effect of three-dimensional reconstruction. Summary of the Invention
[0006] This application aims to at least partially address one of the technical problems in related technologies. To this end, this application provides a method for abdominal organ segmentation and reconstruction based on clinical big data and a state-space model, which can effectively improve the speed and accuracy of medical image processing and enhance the effect of three-dimensional reconstruction.
[0007] To achieve the above objectives, in a first aspect, this application provides a method for individualized abdominal organ segmentation and reconstruction, comprising the following steps:
[0008] S1. Acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data augmentation on the clinical medical imaging data;
[0009] S2. Construct a joint segmentation model and train the joint segmentation model using the clinical medical image data obtained in step S1; wherein, the joint segmentation model uses a CNN network to capture local and detailed features in the low-order part of the encoder and a Mamba model to extract global dependency features in the high-order part of the encoder.
[0010] S3. Acquire clinical medical imaging data of individual abdominal organs and preprocess the clinical medical imaging data;
[0011] S4. Input the clinical medical image data obtained from the preprocessing in step S3 into the joint segmentation model for segmentation processing, and output the segmentation result of a specific region.
[0012] S5. Based on the segmentation results of the specific region obtained in step S4, perform individualized three-dimensional reconstruction of abdominal organs.
[0013] Preferably, the preprocessing in step S1 includes:
[0014] The clinical medical image data is subjected to data standardization processing, and spatial resampling and cubic interpolation methods are used to adjust the clinical medical image data with different image intensities to a uniform resolution and size.
[0015] Gaussian filtering and median filtering are combined to reduce random noise in clinical medical images. When processing images involving complex tumor lesions, nonlocal mean denoising techniques are further combined to reduce noise in clinical medical images.
[0016] Preferably, the data augmentation process in step S1 includes:
[0017] Geometric transformations, brightness and contrast intensity transformations, and elastic deformations are used to simulate the variations that clinical medical images may exhibit in real medical environments, thereby enhancing the adaptability of the joint segmentation model to changes in illumination and tissue deformation.
[0018] The geometric transformations include rotation, scaling, flipping, and distortion transformations; for complex tumor lesion images, diffusion tensor technology is used to generate synthetic data to improve the performance and robustness of the joint segmentation model in processing diverse clinical medical image data.
[0019] Preferably, step S5 uses the Marching Cubes algorithm to convert the voxelized data in the segmentation results into a three-dimensional surface model for individualized abdominal organ segmentation and reconstruction.
[0020] Preferably, the step of training the joint segmentation model in step S2 includes:
[0021] A GPU-accelerated algorithm for the Mamba model is used to improve learning speed;
[0022] An optimization scheme for a joint segmentation model is constructed by combining the Adam optimizer with an adaptive learning rate adjustment strategy; wherein, the adaptive learning rate adjustment strategy includes dynamically adjusting the learning rate according to the performance of the joint segmentation model;
[0023] The Dice loss function is used to optimize the accuracy of the joint segmentation model in the abdominal organ segmentation task.
[0024] Preferably, the step of dynamically adjusting the learning rate according to the performance of the joint segmentation model includes: using a higher learning rate in the initial stage to achieve rapid convergence; if the rate of decrease of the loss on the validation set is less than a preset threshold in multiple consecutive training cycles, then gradually reducing the learning rate according to a preset decay rate.
[0025] Among them, the early stopping strategy is used to monitor the performance on the validation set as a criterion for reducing the learning rate.
[0026] Preferably, the preprocessing in step S3 includes: analyzing the CT value distribution using histogram equalization technology to enhance the local contrast between the liver and surrounding tissues.
[0027] Preferably, after the joint segmentation model performs segmentation processing, it further includes:
[0028] The abnormal regions, including micro-gaps, micro-fractures, or uneven and discontinuous areas, are removed or repaired using erosion and expansion morphological operations; the segmentation results for specific regions are output, including the segmentation results for the liver.
[0029] The connected component labeling algorithm is used to select the liver region and remove regions that are not continuous with the main liver volume and whose volume is smaller than a preset threshold.
[0030] The Gaussian blur algorithm was used to smooth the edges of the liver segmentation results.
[0031] Preferably, the staged encoder in the joint segmentation model is represented as follows: Σ e =[ξ 1 ,ξ 2 ,...,ξ s / / 2 ,ψ s / / 2+1 ,...,ψ s-1 ,ψ s ]
[0032] Where, ∑ e This represents the staged structure of the entire encoder, where ζ and ψ represent the CNN model and the Mamba model, respectively, / / represents integer division, s is the total number of stages in the encoder, and ψ s ζ represents the Mamba model of the encoder at the s-th stage. s This represents the CNN model at the s-th stage of the encoder.
[0033] Secondly, this application provides a personalized abdominal organ segmentation and reconstruction device, comprising:
[0034] The training data acquisition module is used to acquire diverse clinical medical image data of abdominal organs and to perform preprocessing and data augmentation on the clinical medical image data.
[0035] A model training module is used to construct a joint segmentation model and train the joint segmentation model using clinical medical image data obtained by the training data acquisition module. The joint segmentation model uses a CNN network to capture local and detailed features in the low-order part of the encoder and a Mamba model to extract global dependency features in the high-order part of the encoder.
[0036] The individual data acquisition module is used to acquire clinical medical imaging data of an individual's abdominal organs and to preprocess the clinical medical imaging data.
[0037] The 3D reconstruction module is used to input the clinical medical image data preprocessed by the individual data acquisition module into the joint segmentation model for segmentation processing, output the segmentation results of specific regions, and perform individualized 3D reconstruction of abdominal organs based on the segmentation results of specific regions.
[0038] Thirdly, this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the individualized abdominal organ segmentation and reconstruction method described in any of the preceding claims.
[0039] Fourthly, this application provides a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform the individualized abdominal organ segmentation and reconstruction method described in any of the preceding claims.
[0040] Fifthly, this application provides a personalized virtual surgical operation device, including the aforementioned electronic device.
[0041] Based on the above technical solutions, the individualized abdominal organ segmentation and reconstruction method based on clinical imaging and artificial intelligence proposed in this application has at least one of the following beneficial effects compared to the prior art:
[0042] 1. This invention combines the Mamba model with the traditional CNN model in the encoder stage, adopting an innovative phased design approach. In the low-order part of the encoder, the CNN is used to capture local and detailed features, while in the high-order part of the encoder, the Mamba model is used to extract global dependency features. This joint model architecture can effectively cope with the complex structure of clinical big data and effectively identify the biological structural contours of tissues and organs.
[0043] 2. In order to fully capture the important information carried in clinical big data, this invention adopts a large training batch and uses a GPU acceleration algorithm with Mamba structure to improve the model learning speed and improve the stability during training. It uses Adam optimizer combined with adaptive learning rate adjustment strategy to construct model optimization scheme and adopts Dice loss function to optimize the accuracy of model in tissue and organ segmentation task, thereby improving the processing speed and accuracy of medical images and improving the effect of 3D reconstruction.
[0044] 3. This invention employs geometric transformation, brightness and contrast intensity transformation, and elastic deformation to simulate the variations that may occur in clinical medical images in actual medical environments. This enhances the adaptability of the joint segmentation model to changes in illumination and tissue deformation. Through data augmentation strategies, it ensures that the joint segmentation model can learn liver or other tissue structures from multiple angles and of different sizes, thereby improving the model's generalization ability.
[0045] 4. This invention improves the liver segmentation results by using a specific post-processing algorithm to address issues such as reduced segmentation accuracy, discontinuous pixel blocks, noise, and artifacts. It avoids or reduces erroneous segmentation caused by noise and artifacts during the imaging process, thereby improving the segmentation accuracy of the joint segmentation model.
[0046] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description, or may be realized by practicing the application. The purpose and other advantages of this application can be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description
[0047] Figure 1 is a flowchart illustrating the individualized abdominal organ segmentation and reconstruction method of this application;
[0048] Figure 2 is a flowchart of the architecture of the joint segmentation model in this application;
[0049] Figure 3 is a diagram of the overall architecture of the individualized abdominal organ segmentation and reconstruction method of this application;
[0050] Figure 4 is a schematic diagram of the results of lung tumor segmentation and three-dimensional reconstruction in this application;
[0051] Figure 5 is a schematic diagram of the results of three-dimensional reconstruction of pulmonary vascular tissue in this application;
[0052] Figure 6 is a comparison of the segmentation and three-dimensional reconstruction results of liver tumors using the combined segmentation model of this application and the segmentation model of the prior art.
[0053] Figure 7 is a schematic diagram of the results of abdominal organ segmentation and three-dimensional reconstruction in this application. Detailed Implementation
[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.
[0055] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.
[0056] To address the shortcomings of existing technologies, the purpose of this invention is to provide an individualized method for abdominal organ segmentation and reconstruction based on clinical big data and state-space models, with the aim of improving the speed and accuracy of medical image processing through the combined use of segmentation models.
[0057] The basic idea of this invention is to utilize artificial intelligence (AI) and machine learning technologies to achieve automatic segmentation and 3D reconstruction of tissues and organs in medical images through big data analysis. It employs a segmentation model that combines a selective state space (Mamba model) and a convolutional neural network (CNN). In the low-order part of the encoder, the CNN captures local and detailed features, while in the high-order part, the Mamba model extracts globally dependent features. This joint model architecture effectively addresses the complex structure of clinical big data and effectively identifies the biological structural contours of tissues and organs. Furthermore, through specific data processing and AI model training methods, it fully utilizes the characteristics of big data to achieve fast and accurate image segmentation and 3D reconstruction.
[0058] Example 1
[0059] In order to develop a precise and individualized method for abdominal organ segmentation and reconstruction, the inventors conducted in-depth research on artificial intelligence and machine learning technologies and proposed an individualized method for abdominal organ segmentation and reconstruction based on clinical imaging and artificial intelligence.
[0060] Specifically, as shown in Figure 1, a personalized method for abdominal organ segmentation and reconstruction is provided, including the following steps:
[0061] S1. Acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data augmentation on the clinical medical imaging data;
[0062] S2. Construct a joint segmentation model and train the joint segmentation model using the clinical medical image data obtained in step S1; wherein, the joint segmentation model uses a CNN network to capture local and detailed features in the low-order part of the encoder and a Mamba model to extract global dependency features in the high-order part of the encoder.
[0063] S3. Acquire clinical medical imaging data of individual abdominal organs and preprocess the clinical medical imaging data;
[0064] S4. Input the clinical medical image data obtained from the preprocessing in step S3 into the joint segmentation model for segmentation processing, and output the segmentation result of a specific region.
[0065] S5. Based on the segmentation results of the specific region obtained in step S4, perform individualized three-dimensional reconstruction of abdominal organs.
[0066] To design a training framework tailored to the characteristics of clinical big data, this application employs a joint modeling method combining selective state space (Mamba model) and convolutional neural networks (CNN). In the low-order part of the encoder, a CNN network is used to capture local features of medical images, automatically extracting and learning local features (such as tissue edges and organ textures) from the image data. In the high-order part of the encoder, the Mamba model is used to summarize the CNN output, accurately capturing the global dependencies of temporal or spatial sequences in the image data. This joint model architecture can effectively handle the complex structure of clinical big data and effectively identify the biological structural contours of tissues and organs. The architecture of the Mamba model can be formally expressed as follows: m0=σ(BN(Conv(m in )))+m in m0=SSM(SiLU(dwConv(Linear(LN(m0))))) m0=SiLU(Linear(LN(m0)))
[0067] Where, m inThe input features of the Mamba model are represented by m0, and the intermediate features of the model are m0. out The final output features are defined by Conv, a basic convolutional structure; LN and BN represent layer normalization and batch normalization, respectively; σ is a non-linear activation function; dwConv is a separable convolutional structure; Linear is a linear transformation process; SiLU is the sigmoid activation function; and MLP is a multilayer perceptron structure.
[0068] Figure 2 shows the architecture flowchart of the joint segmentation model combining the Mamba and CNN models. The input image is typically a medical scan image such as a CT or MRI. The data deformation step involves resizing or normalizing the input image to fit the model's input requirements. Downsampling reduces the spatial dimensions (width, height, or depth) of the data, typically used to reduce computational complexity or extract more abstract features. Convolutional model (CNN): In the lower-order part, CNN captures local and detailed features. Convolutional layers automatically learn and extract local features in the image, such as edges and textures. State-space model (Mamba): In the higher-order part, the Mamba model extracts global dependency features. The Mamba model captures global dependencies in the temporal or spatial sequences of image data. Residual module: After the CNN and Mamba models, the residual module learns the residual mapping between the input and output. This design helps solve the vanishing gradient problem in deep networks and facilitates the training of deeper networks. Nonlinear module: After the residual module, a nonlinear module (such as the ReLU activation function) introduces nonlinear characteristics, enabling the model to learn more complex features. Upsampling: In contrast to downsampling, upsampling is used to increase the spatial dimension of data, typically to restore an image to its original size or higher resolution. Output: The final output of the model is the segmented image, where different tissues or structures are labeled with different colors or tags; Output Image: This is the final result after model processing, showing the segmented medical image where different tissues and organs are clearly distinguished.
[0069] Figure 3 shows the overall architecture of the personalized abdominal organ segmentation and reconstruction method based on clinical imaging and artificial intelligence proposed in this application.
[0070] In this invention, the clinical medical imaging big data primarily focuses on computed tomography (CT) scans. This data originates from multiple medical institutions, including hospitals and research centers. To ensure the diversity and representativeness of the big data, the collected data covers different age groups and various pathological conditions. Due to the diverse types of clinical big data samples, this invention designs a medical image preprocessing framework for diverse big data to ensure data consistency during preprocessing. This primarily involves using a combination of Gaussian filtering and median filtering to reduce random noise in the images. When processing images involving complex lesions such as tumors, non-local mean denoising techniques are used to effectively remove noise without losing important details. Since the big data samples come from different imaging scanning devices, the resulting image intensities and resolutions are inconsistent. Therefore, a data standardization strategy is implemented, utilizing spatial resampling and cubic interpolation methods to adjust data with different image intensities to a uniform resolution and size, ensuring consistency in the training of the big data model.
[0071] Data augmentation is a key technology for fully leveraging the advantages of medical big data and improving the generalization ability of machine learning models. In terms of data augmentation, this application employs geometric transformations (including rotation, scaling, flipping, and warping), intensity transformations (adjusting brightness and contrast), and elastic transformations to simulate the variations that CT images may exhibit in real medical environments, thereby enhancing the model's adaptability to changes in lighting and tissue deformation. For complex clinical scenarios (such as tumors or infected areas), diffusion tensor technology is used to generate synthetic data, improving the model's performance and robustness in handling diverse large-scale image data.
[0072] This application combines the aforementioned Mamba model with a traditional CNN model in the encoder stage, employing an innovative staged design approach. In the lower-order parts of the encoder, a CNN is used to capture local and detailed features, while in the higher-order parts, the Mamba model is used to extract global dependency features. This staged encoder is expressed by the following formula: Σ e =[ξ 1 ,ξ 2 ,...,ξ s / / 2 ,ψ s / / 2+1 ,...,ψ s-1 ,ψ s ]
[0073] Where, ∑ e This represents the staged structure of the entire encoder, where ζ and ψ represent the CNN model and the Mamba model, respectively, / / represents integer division, s is the total number of stages in the encoder, and ψ s ζ represents the Mamba model of the encoder at the s-th stage. s This represents the CNN model at the s-th stage of the encoder.
[0074] To fully capture the crucial information carried in clinical big data, this application employs large training batches and leverages a GPU-accelerated algorithm based on the Mamba architecture to improve model learning speed and enhance stability during training. An optimization scheme is constructed using the Adam optimizer combined with an adaptive learning rate adjustment strategy. The Dice loss function is employed to optimize the model's accuracy in organ segmentation tasks. A high learning rate is used initially to achieve rapid convergence. An early stopping strategy is used to monitor performance on the validation set as a criterion for reducing the learning rate. Combined with a dynamic learning rate adjustment strategy, if the validation loss does not decrease significantly, the initial learning rate is gradually reduced according to the decay rate. The learning rate is halved or reduced by a fixed percentage at specific epochs.
[0075] This paper utilizes a pre-trained joint model of selective state space and convolutional network to automatically segment abdominal organs and tissues from individualized cases. This model is the first to employ a staged framework to jointly model Mamba and CNN, extracting 3D medical image features. Inputting individualized case CT images (i.e., CT images from a single case) undergoes preprocessing steps such as filtering, resampling, and standardization before being fed into the joint segmentation model. The segmentation model labels each voxel as part of a specific organ or background, outputting the segmentation result for that specific region. To further improve the accuracy and practicality of segmentation, post-processing steps are performed on the segmented images, including morphological operations such as erosion and dilation to ensure tissue continuity and smooth boundaries. The erosion operation utilizes a dynamic sliding window technique to continuously scan the entire image, identifying and removing minute structures. This function can be used to remove abnormal results in organ and tissue segmentation, such as ruptured blood vessels. The dilation operation is the reaction to the erosion operation, used to fill in minute breaks in medical images. It is suitable for pixel offsets in the segmentation of large organs (such as the liver and spleen), which often result in small gaps in the overall liver structure. Assuming the image input is I, and f is a trained CNN and Mamba joint model, this model accepts the input image I and outputs a segmented image S, as follows: S(x,y)=f(I(x,y))
[0076] Here, S(x, y) represents the segmentation result at position (x, y) in the image, where (x, y) represents the coordinates of a single pixel in the medical image, with each position labeled as part of a specific tissue or organ or as background. Let G be a function that performs morphological operations (such as erosion and dilation), which takes the segmented image S and outputs an optimized segmented image S′. S′(x, y) = G(S(x, y))
[0077] Personalized 3D Reconstruction: This application employs the Marching Cubes (MC) algorithm to convert voxelized data into a 3D surface model. The Marching Cubes algorithm is a voxel-to-surface conversion algorithm used to extract isosurfaces from voxel data, which are sets of points with the same density value in 3D space. These isosurfaces can be considered the surface of a 3D object. The Marching Cubes algorithm traverses each cube cell of the voxel data and determines whether the cube contains part of an isosurface based on whether the voxel value exceeds a specific threshold. The algorithm calculates the intersections of the isosurfaces with each edge of the cube and constructs triangles that constitute the surface. Since clinical imaging data has a 3D data structure with a complex voxel distribution, this invention uses GPU parallel acceleration for the MC algorithm. The voxel mesh to be processed by the MC algorithm is divided into multiple small blocks, with each GPU thread responsible for processing one or more small blocks. This allows for simultaneous surface judgment and triangle generation for multiple cubes, shortening the overall computation time and meeting the time efficiency requirements of clinical applications. The process ultimately outputs a personalized 3D reconstructed model of human abdominal tissues and organs.
[0078] Individualized treatment involves optimizing the segmentation and reconstruction results for each case, including adjusting organ size, shape, and relative position to other tissues. The segmentation and reconstruction results can be fine-tuned based on the doctor's feedback to achieve the best clinical outcome.
[0079] Structural correction is a post-processing step performed on the segmentation results to correct potential errors.
[0080] Offset correction is used to correct offsets that may occur during 3D reconstruction.
[0081] Post-processing correction is used to further optimize the 3D model and improve its accuracy and usability.
[0082] Figure 4 shows the results of lung tumor segmentation and 3D reconstruction in this application. The left half of Figure 4 shows the lung tumor segmentation results, with the highlighted green area on the left representing the segmented lung tumor. The right half of Figure 4 shows the corresponding 3D reconstruction results. Figure 5 shows the results of 3D reconstruction of pulmonary vascular tissue in this application. The four figures in Figure 5 demonstrate the 3D reconstruction structure in different ways: displaying the 3D reconstruction segmentation results within a volumetric rendering model; displaying the 3D reconstruction segmentation results of pulmonary vascular tissue alone; displaying the 3D reconstruction segmentation results within a volumetric rendering model with a weakened background; and displaying the 3D reconstruction segmentation results of pulmonary vascular tissue within the original image.
[0083] Figure 6 shows a comparison of the segmentation and 3D reconstruction results of liver tumors using the combined segmentation model of this application and existing segmentation models. Segmentation model 1 displays the segmentation result of the combined segmentation model of this application, while segmentation models 2, 3, and 4 display the results of other existing segmentation models. It can be seen that the segmentation result of segmentation model 1 is superior to that of segmentation models 2, 3, and 4. Figure 7 shows a schematic diagram of the results of abdominal organ segmentation and 3D reconstruction using this application. The left half of Figure 7 shows the segmentation results of different abdominal tissues and organs, and the right half shows the corresponding 3D reconstruction results of different abdominal tissues and organs. It is evident that the individualized abdominal organ segmentation and reconstruction method provided in this application generates a 3D reconstruction model with high fidelity and high segmentation accuracy.
[0084] Example 2
[0085] This embodiment provides a personalized abdominal organ segmentation and reconstruction device, including:
[0086] The training data acquisition module is used to acquire diverse clinical medical image data of abdominal organs and to perform preprocessing and data augmentation on the clinical medical image data.
[0087] A model training module is used to construct a joint segmentation model and train the joint segmentation model using clinical medical image data obtained by the training data acquisition module. The joint segmentation model uses a CNN network to capture local and detailed features in the low-order part of the encoder and a Mamba model to extract global dependency features in the high-order part of the encoder.
[0088] The individual data acquisition module is used to acquire clinical medical imaging data of an individual's abdominal organs and to preprocess the clinical medical imaging data.
[0089] The 3D reconstruction module is used to input the clinical medical image data preprocessed by the individual data acquisition module into the joint segmentation model for segmentation processing, output the segmentation results of specific regions, and perform individualized 3D reconstruction of abdominal organs based on the segmentation results of specific regions.
[0090] Preferably, the preprocessing of clinical medical image data in the training data acquisition module specifically includes:
[0091] The clinical medical image data is subjected to data standardization processing, and spatial resampling and cubic interpolation methods are used to adjust the clinical medical image data with different image intensities to a uniform resolution and size.
[0092] Gaussian filtering and median filtering are combined to reduce random noise in clinical medical images. When processing images involving complex tumor lesions, nonlocal mean denoising techniques are further combined to reduce noise in clinical medical images.
[0093] Preferably, the data augmentation processing of clinical medical image data in the training data acquisition module specifically includes:
[0094] Geometric transformations, brightness and contrast intensity transformations, and elastic deformations are used to simulate the variations that clinical medical images may exhibit in real medical environments, thereby enhancing the adaptability of the joint segmentation model to changes in illumination and tissue deformation.
[0095] The geometric transformations include rotation, scaling, flipping, and distortion transformations; for complex tumor lesion images, diffusion tensor technology is used to generate synthetic data to improve the performance and robustness of the joint segmentation model in processing diverse clinical medical image data.
[0096] This invention employs geometric transformations, brightness and contrast intensity transformations, and elastic deformation to simulate the variations that may occur in clinical medical images in real medical environments. This enhances the adaptability of the joint segmentation model to changes in illumination and tissue deformation. Through data augmentation strategies, it ensures that the joint segmentation model can learn liver or other tissue structures from multiple angles and of different sizes, thereby improving the model's generalization ability.
[0097] Preferably, individualized three-dimensional reconstruction of abdominal organs is performed based on the segmentation results of a specific region. Specifically, the Marching Cubes algorithm is used to convert the voxelized data in the segmentation results into a three-dimensional surface model for individualized abdominal organ segmentation and reconstruction.
[0098] Preferably, training the joint segmentation model in the model training module specifically includes:
[0099] A GPU-accelerated algorithm for the Mamba model is used to improve learning speed;
[0100] A joint segmentation model optimization scheme is constructed by combining the Adam optimizer with an adaptive learning rate adjustment strategy.
[0101] The Dice loss function is used to optimize the accuracy of the joint segmentation model in the abdominal organ segmentation task.
[0102] To fully capture the important information carried in clinical big data, this invention employs large training batches and leverages a GPU acceleration algorithm based on the Mamba architecture to improve model learning speed and enhance stability during training. It utilizes the Adam optimizer combined with an adaptive learning rate adjustment strategy to construct a model optimization scheme and employs the Dice loss function to optimize the model's accuracy in tissue and organ segmentation tasks, thereby improving the processing speed and accuracy of medical images.
[0103] The adaptive learning rate adjustment strategy includes dynamically adjusting the learning rate based on the performance of the joint segmentation model.
[0104] Preferably, dynamically adjusting the learning rate based on the performance of the joint segmentation model specifically includes: using a higher learning rate in the initial stage to achieve rapid convergence; if the rate of decrease of the loss on the validation set is less than a preset threshold in multiple consecutive training cycles, then gradually reducing the learning rate according to a preset decay rate.
[0105] Among them, the early stopping strategy is used to monitor the performance on the validation set as a criterion for reducing the learning rate.
[0106] Preferably, the individual data acquisition module preprocesses the acquired clinical medical image data by: using histogram equalization technology to analyze the CT value distribution, in order to enhance the local contrast between the liver and surrounding tissues.
[0107] Preferably, after the joint segmentation model performs segmentation processing, it further includes:
[0108] The abnormal regions, including micro-gaps, micro-fractures, or uneven and discontinuous areas, are removed or repaired using erosion and expansion morphological operations; the segmentation results for specific regions are output, including the segmentation results for the liver.
[0109] The connected component labeling algorithm is used to select the liver region and remove regions that are not continuous with the main liver volume and whose volume is smaller than a preset threshold.
[0110] The Gaussian blur algorithm was used to smooth the edges of the liver segmentation results.
[0111] This invention improves the liver segmentation results by using a specific post-processing algorithm to address issues such as reduced segmentation accuracy, discontinuous pixel blocks, noise, and artifacts. It avoids or reduces erroneous segmentation caused by noise and artifacts during the imaging process, thereby improving the segmentation accuracy of the joint segmentation model.
[0112] Preferably, the staged encoder in the joint segmentation model is represented as follows: Σ e =[ξ 1 ,ξ 2 ,...,ξ s / / 2 ,ψ s / / 2+1 ,...,ψ s-1 ,ψ s ]
[0113] Where, ∑ e This represents the staged structure of the entire encoder, where ζ and ψ represent the CNN model and the Mamba model, respectively, / / represents integer division, s is the total number of stages in the encoder, and ψ s ζ represents the Mamba model of the encoder at the s-th stage. s This represents the CNN model at the s-th stage of the encoder.
[0114] This invention combines the Mamba model with the traditional CNN model in the encoder stage, adopting an innovative phased design approach. In the low-order part of the encoder, the CNN is used to capture local and detailed features, while in the high-order part of the encoder, the Mamba model is used to extract global dependency features. This joint model architecture can effectively cope with the complex structure of clinical big data and effectively identify the biological structural contours of tissues and organs.
[0115] Example 3
[0116] This application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the individualized abdominal organ segmentation and reconstruction method described in Embodiment 1 above, and to achieve the following functions: It combines a Mamba model with a traditional CNN model in the encoder stage, employing an innovative phased design approach. In the low-order part of the encoder, CNN is used to capture local and detailed features, while in the high-order part, the Mamba model is used to extract global dependency features. This joint model architecture can effectively handle the complex structure of clinical big data and effectively identify the biological structural contours of tissues and organs. To fully capture the important information carried in clinical big data, this invention uses a large training batch, leverages a GPU-accelerated algorithm based on the Mamba structure to improve model learning speed and stability during training, utilizes the Adam optimizer combined with an adaptive learning rate adjustment strategy to construct a model optimization scheme, and employs the Dice loss function to optimize the model's accuracy in tissue and organ segmentation tasks, thereby improving the processing speed and accuracy of medical images. Specifically, this electronic device can be a controller in a specific device.
[0117] Example 4
[0118] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing a computer program. When the computer program runs on a computer or processor, it causes the computer or processor to execute the steps of the aforementioned individualized abdominal organ segmentation and reconstruction method. It achieves the following functions: It combines the Mamba model with a traditional CNN model in the encoder stage, employing an innovative phased design approach. In the low-order part of the encoder, CNN captures local and detailed features, while in the high-order part, the Mamba model extracts global dependency features. This joint model architecture effectively addresses the complex structure of clinical big data and effectively identifies the biological structural contours of tissues and organs. To fully capture the important information carried in clinical big data, this invention uses large training batches, leverages a GPU-accelerated algorithm based on the Mamba structure to improve model learning speed and stability during training, utilizes the Adam optimizer combined with an adaptive learning rate adjustment strategy to construct a model optimization scheme, and employs the Dice loss function to optimize the model's accuracy in tissue and organ segmentation tasks, thereby improving the processing speed and accuracy of medical images.
[0119] Example 5
[0120] This application provides a personalized virtual surgical operation device, including the electronic device described in Embodiment 3 above. This electronic device is the controller within the personalized virtual surgical operation device. The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the personalized abdominal organ segmentation and reconstruction method described in Embodiment 1 above, and to achieve the following functions: It combines the Mamba model with a traditional CNN model in the encoder stage, employing an innovative phased design approach. In the low-order part of the encoder, CNN is used to capture local and detailed features, while in the high-order part, the Mamba model is used to extract global dependency features. This joint model architecture can effectively handle the complex structure of clinical big data and effectively identify the biological structural contours of tissues and organs. To fully capture the important information carried in clinical big data, this invention uses a large training batch, leverages a GPU acceleration algorithm based on the Mamba structure to improve model learning speed and enhance stability during training, utilizes the Adam optimizer combined with an adaptive learning rate adjustment strategy to construct a model optimization scheme, and employs the Dice loss function to optimize the model's accuracy in tissue and organ segmentation tasks, thereby improving the processing speed and accuracy of medical images. This personalized virtual surgical device allows doctors to perform individualized abdominal organ segmentation and reconstruction before actual surgery, helping them understand the real situation of abdominal surgery.
[0121] The foregoing has described specific embodiments of the present invention. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.
[0122] In the description of the embodiments of the present invention, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In the embodiments of the present invention, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in the embodiments of the present invention, as well as the features of the different embodiments or examples.
[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features, excluding any ordering. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature and is used to distinguish it from another. In the description of embodiments of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0124] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.
[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for individualized abdominal organ segmentation and reconstruction, characterized in that, Includes the following steps: S1. Acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data augmentation on the clinical medical imaging data; S2. Construct a joint segmentation model and train the joint segmentation model using the clinical medical image data obtained in step S1; wherein, the joint segmentation model uses a CNN network to capture local and detailed features in the low-order part of the encoder and a Mamba model to extract global dependency features in the high-order part of the encoder. S3. Acquire clinical medical imaging data of individual abdominal organs and preprocess the clinical medical imaging data; S4. Input the clinical medical image data obtained from the preprocessing in step S3 into the joint segmentation model for segmentation processing, and output the segmentation result of a specific region. S5. Based on the segmentation results of the specific region obtained in step S4, perform individualized three-dimensional reconstruction of abdominal organs.
2. The individualized abdominal organ segmentation and reconstruction method according to claim 1, characterized in that, The preprocessing in step S1 includes: The clinical medical image data is subjected to data standardization processing, and spatial resampling and cubic interpolation methods are used to adjust the clinical medical image data with different image intensities to a uniform resolution and size. Gaussian filtering and median filtering are combined to reduce random noise in clinical medical images. When processing images involving complex tumor lesions, nonlocal mean denoising techniques are further combined to reduce noise in clinical medical images.
3. The individualized abdominal organ segmentation and reconstruction method according to claim 1, characterized in that, The data augmentation process in step S1 includes: Geometric transformations, brightness and contrast intensity transformations, and elastic deformations are used to simulate the variations that clinical medical images may exhibit in real medical environments, thereby enhancing the adaptability of the joint segmentation model to changes in illumination and tissue deformation. The geometric transformations include rotation, scaling, flipping, and distortion transformations; for complex tumor lesion images, diffusion tensor technology is used to generate synthetic data to improve the performance and robustness of the joint segmentation model in processing diverse clinical medical image data.
4. The individualized abdominal organ segmentation and reconstruction method according to claim 1, characterized in that, Step S5 uses the Marching Cubes algorithm to convert the voxelized data in the segmentation results into a three-dimensional surface model for individualized abdominal organ segmentation and reconstruction.
5. The individualized abdominal organ segmentation and reconstruction method according to claim 1, characterized in that, The step of training the joint segmentation model in step S2 includes: A GPU-accelerated algorithm for the Mamba model is used to improve learning speed; An optimization scheme for a joint segmentation model is constructed by combining the Adam optimizer with an adaptive learning rate adjustment strategy; wherein, the adaptive learning rate adjustment strategy includes dynamically adjusting the learning rate according to the performance of the joint segmentation model; The Dice loss function is used to optimize the accuracy of the joint segmentation model in the abdominal organ segmentation task.
6. The individualized abdominal organ segmentation and reconstruction method according to claim 5, characterized in that, The steps for dynamically adjusting the learning rate based on the performance of the joint segmentation model include: using a higher learning rate in the initial stage to achieve rapid convergence; if the rate of decrease of the loss on the validation set is less than a preset threshold in multiple consecutive training cycles, then gradually reducing the learning rate according to the preset decay rate. Among them, the early stopping strategy is used to monitor the performance on the validation set as a criterion for reducing the learning rate.
7. The individualized abdominal organ segmentation and reconstruction method according to any one of claims 1-6, characterized in that, The preprocessing in step S3 includes: using histogram equalization technology to analyze the CT value distribution to enhance the local contrast between the liver and surrounding tissues.
8. The individualized abdominal organ segmentation and reconstruction method according to claim 7, characterized in that, After the joint segmentation model performs segmentation processing, it also includes: The abnormal regions, including micro-gaps, micro-fractures, or uneven and discontinuous areas, are removed or repaired using erosion and expansion morphological operations; the segmentation results for specific regions are output, including the segmentation results for the liver. The connected component labeling algorithm is used to select the liver region and remove regions that are not continuous with the main liver volume and whose volume is smaller than a preset threshold. The Gaussian blur algorithm was used to smooth the edges of the liver segmentation results.
9. The method for individualized abdominal organ segmentation and reconstruction according to claim 7, characterized in that, The staged encoder in the joint segmentation model is represented as follows: Σ e =[ξ 1 ,ξ 2 ,...,ξ s / / 2 ,ψ s / / 2+1 ,...,ψ s-1 ,ψ s ] Where, ∑ e This represents the staged structure of the entire encoder, where ζ and ψ represent the CNN model and the Mamba model, respectively, / / represents integer division, s is the total number of stages in the encoder, and ψ s ζ represents the Mamba model of the encoder at the s-th stage. s This represents the CNN model at the s-th stage of the encoder.
10. A personalized abdominal organ segmentation and reconstruction device, characterized in that, include: The training data acquisition module is used to acquire diverse clinical medical image data of abdominal organs and to perform preprocessing and data augmentation on the clinical medical image data. A model training module is used to construct a joint segmentation model and train the joint segmentation model using clinical medical image data obtained by the training data acquisition module. The joint segmentation model uses a CNN network to capture local and detailed features in the low-order part of the encoder and a Mamba model to extract global dependency features in the high-order part of the encoder. The individual data acquisition module is used to acquire clinical medical imaging data of an individual's abdominal organs and to preprocess the clinical medical imaging data. The 3D reconstruction module is used to input the clinical medical image data preprocessed by the individual data acquisition module into the joint segmentation model for segmentation processing, output the segmentation results of specific regions, and perform individualized 3D reconstruction of abdominal organs based on the segmentation results of specific regions.
11. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the individualized abdominal organ segmentation and reconstruction method as described in any one of claims 1-9.
12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program runs on a computer or processor, it causes the computer or processor to perform the individualized abdominal organ segmentation and reconstruction method as described in any one of claims 1-9.
13. A personalized virtual surgical operation device, characterized in that, Includes the electronic device as described in claim 11.