Individualized abdominal organ segmentation and reconstruction method based on clinical big data and state space model

Abstract

The invention discloses an individualized abdominal organ segmentation and reconstruction method based on clinical big data and a state space model, and belongs to the technical field of medical image processing. Comprising the following steps: S1, acquiring diversified clinical medical image data of abdominal organs, and performing preprocessing and data enhancement processing on the clinical medical image data; s2, constructing a joint segmentation model, and training the joint segmentation model by using the clinical medical image data processed in the step S1; s3, acquiring clinical medical image data of an individual abdominal cavity organ, and preprocessing the clinical medical image data; s4, inputting the clinical medical image data pre-processed in the step S3 into a joint segmentation model for segmentation processing, and outputting a segmentation result of a specific region; and S5, performing individual abdominal organ three-dimensional reconstruction based on the segmentation result of the specific region obtained in the step S4. The invention designs a three-dimensional reconstruction method capable of effectively improving the medical image processing speed and accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and specifically to an individualized abdominal organ segmentation and reconstruction method based on clinical big data and a state-space model. Background Art

[0002] Traditional medical image processing methods, especially manual segmentation, require doctors or technicians to check each pixel one by one, which is very time-consuming when processing large-scale data sets. Due to human factors, the accuracy and repeatability of manual segmentation are difficult to guarantee, and there may be large differences between different operators.

[0003] Existing automated segmentation methods often rely on predefined rules that may not adapt to the diversity of different patients and different pathological conditions, and for complex lesions such as tumors, existing technologies may not accurately capture their boundaries and internal structures when identifying and segmenting them. Although big data has great potential in medical image analysis, it has not been fully utilized, mainly because of the high heterogeneity and difficulty in processing data. Medical image data comes from different scanning devices and parameter settings. This heterogeneity poses challenges to the unified processing and analysis of data. High-precision automated segmentation and three-dimensional reconstruction often require a lot of computing resources, which may be difficult to achieve in medical institutions with limited resources. In addition, existing image processing technologies often use general algorithms and lack personalized processing for individual differences, which may affect the accuracy of diagnosis and treatment.

[0004] The Chinese patent application document with publication number CN118691820A discloses a multimodal fusion segmentation method and device based on prior information and Mamba hybrid model, which inputs the remote sensing image set to be processed into a pre-trained remote sensing image multimodal fusion segmentation network; outputs the semantic segmentation results of the ground object interpretation of the remote sensing image set to be processed, and the pre-constructed remote sensing image multimodal fusion segmentation network is composed of a U-shaped symmetrical encoder-decoder structure; and the encoder part thereof adopts a Mamba-CNN hybrid model. In this way, prior information can be combined with multimodal fusion, and RGB and DSM remote sensing images can be effectively fused, thereby improving the accuracy of semantic segmentation. However, this method cannot be directly applied to the three-dimensional reconstruction of individualized abdominal organs, nor can it solve the problems of large processing volume of medical image data and high data heterogeneity.

[0005] It can be seen that there is an urgent need to develop a new individualized abdominal organ segmentation and reconstruction method to effectively improve the speed and accuracy of medical image processing and improve the effect of three-dimensional reconstruction. Summary of the invention

[0006] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the present application provides a method for segmenting and reconstructing abdominal organs based on clinical big data and a state space model, which can effectively improve the speed and accuracy of medical image processing and improve the effect of three-dimensional reconstruction.

[0007] To achieve the above objectives, in a first aspect, the present application provides a method for individualized abdominal organ segmentation and reconstruction, comprising the following steps:

[0008] S1. Acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data enhancement processing on the clinical medical imaging data;

[0009] S2, constructing a joint segmentation model, and using the clinical medical image data obtained by processing in step S1 to train the joint segmentation model; wherein the joint segmentation model uses a CNN network in the low-order part of the encoder to capture local and detail features, and uses a Mamba model in the high-order part of the encoder to extract global dependency features;

[0010] S3, obtaining clinical medical imaging data of individual abdominal organs, and preprocessing the clinical medical imaging data;

[0011] S4, inputting the clinical medical image data obtained by the preprocessing in step S3 into the joint segmentation model for segmentation processing, and outputting the segmentation result of the specific area;

[0012] S5. Perform individualized three-dimensional reconstruction of abdominal organs based on the segmentation result of the specific area obtained in step S4.

[0013] Preferably, the preprocessing in step S1 includes:

[0014] Performing data standardization processing on the clinical medical imaging data, and adjusting the clinical medical imaging data of different imaging intensities to a uniform resolution and size by using spatial resampling and cubic interpolation methods;

[0015] Gaussian filtering and median filtering are combined to reduce random noise in clinical medical images. When processing images involving complex tumor lesions, non-local mean denoising technology is combined to reduce noise in clinical medical images.

[0016] Preferably, the data enhancement processing in step S1 includes:

[0017] Geometric transformation, brightness and contrast intensity transformation, and elastic deformation are used to simulate the possible variations of clinical medical images in actual medical environments, so as to enhance the adaptability of the joint segmentation model to illumination changes and tissue deformation.

[0018] Among them, the geometric transformation includes rotation, scaling, flipping and twisting transformation; for complex tumor lesion images, diffusion tensor technology is used to generate synthetic data to improve the performance and robustness of the joint segmentation model in processing diverse clinical medical imaging data.

[0019] Preferably, the step S5 converts the voxelized data in the segmentation result into a three-dimensional surface model by using the Marching Cubes algorithm to perform individualized abdominal organ segmentation and reconstruction.

[0020] Preferably, the step of training the joint segmentation model in step S2 includes:

[0021] Use the GPU acceleration algorithm of the Mamba model to improve learning speed;

[0022] A joint segmentation model optimization scheme is constructed by using the Adam optimizer combined with an adaptive learning rate adjustment strategy; wherein the adaptive learning rate adjustment strategy includes dynamically adjusting the learning rate according to the performance of the joint segmentation model;

[0023] The Dice loss function is used to optimize the accuracy of the joint segmentation model in the abdominal organ segmentation task.

[0024] Preferably, the step of dynamically adjusting the learning rate according to the performance of the joint segmentation model includes: using a higher learning rate in the initial stage to converge quickly, and if the decrease rate of the loss on the validation set in multiple consecutive training cycles is less than a preset threshold, then gradually reducing the learning rate according to a preset decay rate;

[0025] Among them, the early stopping strategy is used to monitor the performance on the validation set as a criterion for reducing the learning rate.

[0026] Preferably, the preprocessing in step S3 includes: analyzing the CT value distribution by using a histogram equalization technique to enhance the local contrast between the liver and surrounding tissues.

[0027] Preferably, after the joint segmentation model performs segmentation processing, the joint segmentation model further comprises:

[0028] Using corrosion and dilation morphological operations to remove or repair abnormal areas, wherein the abnormal areas include tiny gaps, tiny fractures, or non-smooth, discontinuous areas; wherein outputting the segmentation result of the specific area includes outputting the segmentation result of the liver;

[0029] The liver regions were selected using a connected component labeling algorithm, and regions that were discontinuous with the main liver volume and whose volume was smaller than a preset threshold were removed;

[0030] Gaussian blur algorithm is used to smooth the edges of the liver segmentation results.

[0031] Preferably, the staged encoder in the joint segmentation model is represented as follows:

[0032]

[0033] Among them, ∑ e represents the staged structure of the entire encoder, ζ and ψ represent the CNN model and Mamba model respectively, / / represents integer division, s is the number of overall encoder stages, ψ s represents the Mamba model of the encoder at the sth stage, ζ s Represents the CNN model of the encoder at the sth stage.

[0034] In a second aspect, the present application provides a personalized abdominal organ segmentation and reconstruction device, comprising:

[0035] A training data acquisition module, used to acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data enhancement processing on the clinical medical imaging data;

[0036] A model training module; used to construct a joint segmentation model, and train the joint segmentation model using the clinical medical image data processed by the training data acquisition module; wherein the joint segmentation model uses a CNN network in the low-order part of the encoder to capture local and detail features, and uses a Mamba model in the high-order part of the encoder to extract global dependency features;

[0037] An individual data acquisition module, used to acquire clinical medical imaging data of individual abdominal organs and pre-process the clinical medical imaging data;

[0038] The three-dimensional reconstruction module is used to input the clinical medical image data preprocessed by the individual data acquisition module into the joint segmentation model for segmentation processing, output the segmentation results of the specific area, and perform individualized three-dimensional reconstruction of abdominal organs based on the segmentation results of the specific area.

[0039] In a third aspect, the present application provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above-described individualized abdominal organ segmentation and reconstruction methods.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, comprising a computer program, which, when executed on an electronic device, enables the electronic device to execute any one of the above-described individualized abdominal organ segmentation and reconstruction methods.

[0041] In a fifth aspect, the present application provides a personalized virtual surgical operation device, including the above-mentioned electronic device.

[0042] Based on the above technical solutions, it can be seen that the individualized abdominal organ segmentation and reconstruction method based on clinical images and artificial intelligence in this application has at least one of the following beneficial effects compared with the prior art:

[0043] 1. The present invention combines the Mamba model with the traditional CNN model at the encoder stage, adopts an innovative phased design approach, uses CNN in the low-order part of the encoder to capture local and detail features, and uses the Mamba model in the high-order part of the encoder to extract global dependency features. This joint model architecture can effectively cope with the complex structure of clinical big data and effectively identify the biological structure contours of tissues and organs.

[0044] 2. In order to fully capture the important information carried in clinical big data, the present invention adopts a larger training batch and uses the GPU acceleration algorithm of the Mamba structure to improve the model learning speed and the stability during the training process. The Adam optimizer is combined with the adaptive learning rate adjustment strategy to construct a model optimization scheme. The Dice loss function is used to optimize the accuracy of the model in the task of tissue and organ segmentation, thereby improving the processing speed and accuracy of medical images and the effect of three-dimensional reconstruction.

[0045] 3. The present invention uses geometric transformation, brightness and contrast intensity transformation, and elastic deformation to simulate the possible variations of clinical medical images in actual medical environments, so as to enhance the adaptability of the joint segmentation model to illumination changes and tissue deformation. The data enhancement strategy ensures that the joint cutting model can learn the liver or other tissue structures from multiple angles and different sizes, thereby improving the generalization ability of the model.

[0046] 4. The present invention improves the liver segmentation results by a specific post-processing algorithm, such as reduced segmentation accuracy, discontinuous pixel blocks, noise and artifacts, avoids or reduces erroneous segmentation caused by noise and artifacts during the imaging process, and improves the segmentation accuracy of the joint segmentation model.

[0047] Other features and advantages of the present application will be described in the subsequent description, and in part will become apparent from the description, or it may be understood through the implementation of the present application that the objects and other advantages of the present application can be realized and obtained through the written description and the structures particularly pointed out in the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the process of the individualized abdominal organ segmentation and reconstruction method of the present application;

[0049] Figure 2 This is a flow chart of the architecture of the joint segmentation model in this application;

[0050] Figure 3The overall framework diagram of the individualized abdominal organ segmentation and reconstruction method of the present application;

[0051] Figure 4 A schematic diagram of the results of lung tumor segmentation and three-dimensional reconstruction performed for this application;

[0052] Figure 5 A schematic diagram of the results of three-dimensional reconstruction of pulmonary vascular tissue for this application;

[0053] Figure 6 A comparison diagram of the results of liver tumor segmentation and three-dimensional reconstruction performed by the joint segmentation model of the present application and the segmentation model of the prior art;

[0054] Figure 7 This is a schematic diagram of the results of abdominal organ segmentation and 3D reconstruction performed in this application. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0056] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings.

[0057] In view of the deficiencies in the prior art, the purpose of the present invention is to provide an individualized abdominal organ segmentation and reconstruction method based on clinical big data and a state-space model, in the hope of improving the speed and accuracy of medical image processing through the use of a joint segmentation model.

[0058] The basic idea of ​​the present invention is to use artificial intelligence (AI) and machine learning technology to realize the automatic segmentation and three-dimensional reconstruction of tissues and organs in medical images through big data analysis; a segmentation model combined with a selective state space (Mamba model) and a convolutional neural network (CNN) is adopted, CNN is used in the low-order part of the encoder to capture local and detail features, and the Mamba model is used in the high-order part of the encoder to extract global dependent features. Through this joint model architecture, the complex structure of clinical big data can be effectively dealt with, and the biological structure contours of tissues and organs can be effectively identified. And through specific data processing and AI model training methods, the characteristics of big data are fully utilized to achieve fast and accurate image segmentation and three-dimensional reconstruction.

[0059] Embodiment 1

[0060] In order to develop an accurate individualized abdominal organ segmentation and reconstruction method, the inventors conducted in-depth research on artificial intelligence and machine learning technologies, and proposed an individualized abdominal organ segmentation and reconstruction method based on clinical images and artificial intelligence.

[0061] Specifically, Figure 1 As shown, a method for individualized abdominal organ segmentation and reconstruction is provided, comprising the following steps:

[0062] S1. Acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data enhancement processing on the clinical medical imaging data;

[0063] S2, constructing a joint segmentation model, and using the clinical medical image data obtained by processing in step S1 to train the joint segmentation model; wherein the joint segmentation model uses a CNN network in the low-order part of the encoder to capture local and detail features, and uses a Mamba model in the high-order part of the encoder to extract global dependency features;

[0064] S3, obtaining clinical medical imaging data of individual abdominal organs, and preprocessing the clinical medical imaging data;

[0065] S4, inputting the clinical medical image data obtained by the preprocessing in step S3 into the joint segmentation model for segmentation processing, and outputting the segmentation result of the specific area;

[0066] S5. Perform individualized three-dimensional reconstruction of abdominal organs based on the segmentation result of the specific area obtained in step S4.

[0067] In order to design a training framework based on the characteristics of clinical big data, this application adopts a modeling method that combines a selective state space (Mamba model) and a convolutional neural network (CNN). In the low-order part of the model encoder, a CNN network is used to capture local features of medical images, and automatically extract and learn local features (such as tissue edges, organ textures, etc.) from the image data; in the high-order part of the encoder, the Mamba model is used to summarize the CNN output results and correctly capture the global dependencies of the temporal or spatial sequences in the image data. This joint model architecture can effectively cope with the complex structure of clinical big data and effectively identify the biological structural contours of tissues and organs. The architecture of the Mamba model can be formulated as follows:

[0068] m 0 =σ(BN(Conv(m in )))+m in

[0069] m 0 =SSM(SiLU(dwConv(Linear(LN(m 0 )))))

[0070] m 0 =SiLU(Linear(LN(m 0 )))

[0071]

[0072] Among them, m in Represents the input features of the Mamba model, m 0 is the middle layer feature of the model, m out is the final output feature. Conv is a basic convolution structure; LN and BN represent layer normalization and batch normalization processes respectively, σ is a nonlinear activation function; dwConv is a separable convolution structure; Linear is a linear conversion process; SiLU is a sigmoid activation function; MLP is a multi-layer perceptron layer structure.

[0073] like Figure 2 As shown in FIG. 1 , the architecture flow chart of the joint segmentation model combining the Mamba model and the CNN model is shown, wherein the input image is usually a medical scan image such as CT or MRI. The data deformation step involves resizing or normalizing the input image to adapt to the input requirements of the model. Downsampling can reduce the spatial dimension (width, height or depth) of the data, which is usually used to reduce computational complexity or extract more abstract features. Convolutional model (CNN): In the low-order part, CNN is used to capture local and detailed features. The convolutional layer can automatically learn and extract local features in the image, such as edges, textures, etc. State space model (Mamba): In the high-order part, the Mamba model is used to extract global dependency features. The Mamba model can capture the global dependency of the temporal or spatial sequence in the image data. Residual module: After the CNN and Mamba models, the residual module is used to learn the residual mapping between the input and output. This design helps to solve the gradient vanishing problem in deep networks and facilitates the training of deeper networks. Nonlinear module: After the residual module, the nonlinear module (such as the ReLU activation function) is used to introduce nonlinear characteristics, enabling the model to learn more complex features. Upsampling: In contrast to downsampling, upsampling is used to increase the spatial dimension of the data, usually used to restore the image to its original size or higher resolution. Output: The final output of the model is a segmented image in which different tissues or structures are marked with different colors or labels; Output image: This is the final result after model processing, showing the segmented medical image in which different tissues and organs are clearly distinguished.

[0074] like Figure 3 As shown, this is the overall architecture diagram of the individualized abdominal organ segmentation and reconstruction method based on clinical images and artificial intelligence of this application.

[0075] In the present invention, clinical medical imaging big data is mainly aimed at computed tomography (CT). These data come from multiple medical institutions, including hospitals and research centers. To ensure the diversity and representativeness of big data, the collected data covers people of different age groups and various pathological conditions. Due to the diversity of clinical big data sample types, in order to ensure the data consistency of the preprocessing process, the present invention designs a medical image preprocessing framework for diversified big data. It mainly includes the use of a combination of Gaussian filtering and median filtering to reduce random noise in the image. When processing complex lesion images such as tumors, the non-local mean denoising technology is used to effectively remove noise without losing important details. Since the big data samples come from different image scanning devices, the image intensity and resolution obtained are inconsistent. Therefore, a data standardization strategy will be implemented, and the data of different image intensities will be adjusted to a uniform resolution and size using spatial resampling and cubic interpolation methods to ensure the consistency of big data model training.

[0076] Data enhancement computing is a key technology to give full play to the advantages of medical big data and improve the generalization ability of machine learning models. In terms of data enhancement, this application uses geometric transformations (including rotation, scaling, flipping and twisting), intensity transformations (adjusting brightness and contrast), and elastic transformations to simulate possible variations in CT images in actual medical environments, and to enhance the model's ability to adapt to changes in illumination and tissue deformation. For complex clinical scenarios (such as tumors or infected areas), diffusion tensor technology is used to generate synthetic data to improve the performance and robustness of the model in processing diverse imaging big data.

[0077] This application combines the above-mentioned Mamba model with the traditional CNN model at the encoder stage, adopts an innovative phased design idea, uses CNN to capture local and detail features in the low-order part of the encoder, and uses the Mamba model to extract global dependency features in the high-order part of the encoder. This phased encoder is expressed by the following formula:

[0078] Σ e =[ξ 1 ,ξ 2 ,...,ξ s / / 2 ,ψ s / / 2+1 ,...,ψ s-1 ,ψ s ]

[0079] Among them, ∑ e represents the staged structure of the entire encoder, ζ and ψ represent the CNN model and Mamba model respectively, / / represents integer division, s is the number of overall encoder stages, ψ s represents the Mamba model of the encoder at the sth stage, ζ s Represents the CNN model of the encoder at the sth stage.

[0080] In order to fully capture the important information carried in clinical big data, this application adopts a larger training batch and uses the GPU acceleration algorithm of the Mamba structure to improve the model learning speed and improve the stability during the training process. The Adam optimizer is combined with an adaptive learning rate adjustment strategy to build a model optimization solution. The Dice loss function is used to optimize the accuracy of the model in tissue and organ segmentation tasks. A higher learning rate is used in the initial stage to converge quickly. The Early Stopping strategy is used to monitor the performance on the validation set as a criterion for reducing the learning rate. Combined with the dynamic learning rate adjustment strategy, if the validation loss does not decrease significantly, the initial learning rate is step-attenuated according to the decay rate. The learning rate is halved or reduced by a fixed ratio at a specific epoch.

[0081] The above-mentioned trained selective state space and convolutional network joint model is used to complete the automatic segmentation of abdominal tissues and organs of individual cases. This model uses a staged framework for the first time to jointly model Mamba and CNN to extract three-dimensional medical image features. For the input individual case CT image (i.e., CT image from a single case), after pre-processing steps such as filtering, resampling and standardization, it is input into the joint segmentation model. The segmentation model marks each voxel as a part of a specific tissue or organ or background, and outputs the segmentation result of a specific area. In order to further improve the accuracy and practicality of segmentation, the segmented image performs post-processing steps in this application. Including morphological operations such as corrosion and expansion to ensure the continuity of the tissue and the smoothness of the boundary. Among them, the corrosion operation uses dynamic sliding window technology to continuously scan all images, identify and eliminate tiny structures. This function can be used to remove abnormal results in tissue and organ segmentation, such as vascular rupture. The expansion operation is the reaction of the corrosion operation, which is used to fill the tiny fractures in medical images. It is applicable to the pixel offset in the segmentation of large organs (such as liver, spleen, etc.) in this application, which often leads to tiny gaps in the overall liver structure. Assume that the image input is I, and f is a trained CNN and Mamba joint model, which accepts the input image I and outputs the segmented image S, which is expressed as follows:

[0082] S(x,y)=f(I(x,y))

[0083] Where S(x, y) represents the segmentation result of the position (x, y) in the image, (x, y) represents the coordinates of a single pixel in the medical image, and each position is marked as part of a specific tissue or organ or background. Let G be a function that performs morphological operations (such as erosion and dilation), which accepts the segmented image S and outputs the optimized segmented image S'.

[0084] S′(x,y)=G(S(x,y))

[0085] Individualized three-dimensional reconstruction: This application uses the Marching Cubes (MC) algorithm to convert voxelized data into a three-dimensional surface model. The Marching Cubes algorithm is a voxel-to-surface conversion algorithm used to extract isosurfaces from voxel data, that is, a collection of points with the same density value in three-dimensional space. This isosurface can be regarded as the surface of a three-dimensional object. The Marching Cubes algorithm traverses each cube unit (cube) of the voxel data, and determines whether the cube contains a part of the isosurface according to whether the voxel value exceeds a specific threshold. The algorithm calculates the intersection of the isosurface and each side of the cube, and constructs the triangles that constitute the surface. Since clinical imaging data is a three-dimensional data structure with a complex voxel distribution, the present invention uses a GPU parallel acceleration method for the MC algorithm, and divides the voxel grid to be dealt with by the MC algorithm into multiple small blocks, and each GPU thread is responsible for the processing of one or more small blocks. In this way, surface judgment and triangle generation of multiple cubes can be performed simultaneously, shortening the overall calculation time and meeting the requirements for time efficiency in clinical applications. The process ultimately outputs an individualized three-dimensional reconstruction model of human abdominal tissues and organs.

[0086] Individualized treatment involves optimizing the segmentation and reconstruction results based on the specific circumstances of each case, including the adjustment of organ size, shape, and relative position with other tissues. The segmentation and reconstruction results can be fine-tuned based on the doctor's feedback to achieve the best clinical effect.

[0087] Structural correction is the post-processing of segmentation results to correct possible errors.

[0088] Offset correction is to correct the offset that may occur during 3D reconstruction.

[0089] Post-processing correction is to further optimize the 3D model to improve its accuracy and usability.

[0090] like Figure 4 , which is a schematic diagram of the results of lung tumor segmentation and three-dimensional reconstruction performed in this application, Figure 4 The left half is the lung tumor segmentation result. The highlighted green area on the left is the segmented lung tumor. Figure 4 The right half is the corresponding 3D reconstruction result. Figure 5 , which is a schematic diagram of the result of three-dimensional reconstruction of pulmonary vascular tissue performed in this application; Figure 5 The four figures in the figure show the three-dimensional reconstructed structure in different forms of expression, namely, displaying the three-dimensional reconstruction segmentation results in a volume rendering model, displaying the three-dimensional reconstruction segmentation results of the lung vascular tissue alone, displaying the three-dimensional reconstruction segmentation results in a volume rendering model and displaying them in a weakened background, and displaying the three-dimensional reconstruction segmentation results of the lung vascular tissue in the original image.

[0091] like Figure 6 As shown in FIG. 1 , it is a comparison diagram of the segmentation and 3D reconstruction results of liver tumors by the joint segmentation model of the present application and the segmentation model of the prior art. Among them, segmentation model 1 shows the segmentation result of the joint segmentation model of the present application, and segmentation models 2, 3, and 4 are the display results of other existing segmentation models. It can be seen that the segmentation result of segmentation model 1 is better than the segmentation results of segmentation models 2, 3, and 4. Figure 7 As shown, it is a schematic diagram of the results of abdominal organ segmentation and three-dimensional reconstruction performed in this application. Figure 7 The left half is the segmentation result of different abdominal tissues and organs, and the right half is the corresponding 3D reconstruction result of different abdominal tissues and organs. It can be seen that the 3D reconstruction model generated by the individualized abdominal organ segmentation and reconstruction method provided in the application has high restoration degree and high segmentation accuracy.

[0092] Embodiment 2

[0093] This embodiment provides an individualized abdominal organ segmentation and reconstruction device, comprising:

[0094] A training data acquisition module, used to acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data enhancement processing on the clinical medical imaging data;

[0095] A model training module; used to construct a joint segmentation model, and train the joint segmentation model using the clinical medical image data processed by the training data acquisition module; wherein the joint segmentation model uses a CNN network in the low-order part of the encoder to capture local and detail features, and uses a Mamba model in the high-order part of the encoder to extract global dependency features;

[0096] An individual data acquisition module, used to acquire clinical medical imaging data of individual abdominal organs and pre-process the clinical medical imaging data;

[0097] The three-dimensional reconstruction module is used to input the clinical medical image data preprocessed by the individual data acquisition module into the joint segmentation model for segmentation processing, output the segmentation results of the specific area, and perform individualized three-dimensional reconstruction of abdominal organs based on the segmentation results of the specific area.

[0098] Preferably, the preprocessing of clinical medical imaging data in the training data acquisition module specifically includes:

[0099] Performing data standardization processing on the clinical medical imaging data, and adjusting the clinical medical imaging data of different imaging intensities to a uniform resolution and size by using spatial resampling and cubic interpolation methods;

[0100] Gaussian filtering and median filtering are combined to reduce random noise in clinical medical images. When processing images involving complex tumor lesions, non-local mean denoising technology is combined to reduce noise in clinical medical images.

[0101] Preferably, the data enhancement processing of clinical medical imaging data in the training data acquisition module specifically includes:

[0102] Geometric transformation, brightness and contrast intensity transformation, and elastic deformation are used to simulate the possible variations of clinical medical images in actual medical environments, so as to enhance the adaptability of the joint segmentation model to illumination changes and tissue deformation.

[0103] Among them, the geometric transformation includes rotation, scaling, flipping and twisting transformation; for complex tumor lesion images, diffusion tensor technology is used to generate synthetic data to improve the performance and robustness of the joint segmentation model in processing diverse clinical medical imaging data.

[0104] The present invention uses geometric transformation, brightness and contrast intensity transformation, and elastic deformation to simulate the possible variations of clinical medical images in actual medical environments, so as to enhance the adaptability of the joint segmentation model to illumination changes and tissue deformation. The data enhancement strategy ensures that the joint cutting model can learn the liver or other tissue structures from multiple angles and different sizes, thereby improving the generalization ability of the model.

[0105] Preferably, the individualized abdominal organ three-dimensional reconstruction is performed based on the segmentation result of the specific area, specifically by using the Marching Cubes algorithm to convert the voxelized data in the segmentation result into a three-dimensional surface model to perform individualized abdominal organ segmentation and reconstruction.

[0106] Preferably, the training of the joint segmentation model in the model training module specifically includes:

[0107] Use the GPU acceleration algorithm of the Mamba model to improve learning speed;

[0108] The Adam optimizer is used in combination with the adaptive learning rate adjustment strategy to build a joint segmentation model optimization solution;

[0109] The Dice loss function is used to optimize the accuracy of the joint segmentation model in the abdominal organ segmentation task;

[0110] In order to fully capture the important information carried in clinical big data, the present invention adopts a larger training batch and uses the GPU acceleration algorithm of the Mamba structure to improve the model learning speed and the stability during the training process. The Adam optimizer is combined with the adaptive learning rate adjustment strategy to construct a model optimization scheme. The Dice loss function is used to optimize the accuracy of the model in the task of tissue and organ segmentation, thereby improving the processing speed and accuracy of medical images.

[0111] The adaptive learning rate adjustment strategy includes dynamically adjusting the learning rate according to the performance of the joint segmentation model.

[0112] Preferably, dynamically adjusting the learning rate according to the performance of the joint segmentation model specifically includes: using a higher learning rate in the initial stage to converge quickly, and if the decrease rate of the loss on the validation set in multiple consecutive training cycles is less than a preset threshold, then gradually reducing the learning rate according to a preset decay rate;

[0113] Among them, the early stopping strategy is used to monitor the performance on the validation set as a criterion for reducing the learning rate.

[0114] Preferably, the individual data acquisition module preprocesses the acquired clinical medical image data, specifically including: using a histogram equalization technique to analyze the CT value distribution, so as to enhance the local contrast between the liver and surrounding tissues.

[0115] Preferably, after the joint segmentation model performs segmentation processing, the following steps are specifically included:

[0116] Using corrosion and dilation morphological operations to remove or repair abnormal areas, wherein the abnormal areas include tiny gaps, tiny fractures, or non-smooth, discontinuous areas; wherein outputting the segmentation result of the specific area includes outputting the segmentation result of the liver;

[0117] The liver regions were selected using a connected component labeling algorithm, and regions that were discontinuous with the main liver volume and whose volume was smaller than a preset threshold were removed;

[0118] Gaussian blur algorithm is used to smooth the edges of the liver segmentation results.

[0119] The present invention improves the liver segmentation results by a specific post-processing algorithm, such as reduced segmentation accuracy, discontinuous pixel blocks, noise and artifacts, avoids or reduces erroneous segmentation caused by noise and artifacts in the imaging process, and improves the segmentation accuracy of the joint segmentation model.

[0120] Preferably, the staged encoder in the joint segmentation model is represented as follows:

[0121] Σ e =[ξ 1 ,ξ 2 ,...,ξ s / / 2,ψ s / / 2+1 ,...,ψ s-1 ,ψ s ]

[0122] Among them, ∑ e represents the staged structure of the entire encoder, ζ and ψ represent the CNN model and Mamba model respectively, / / represents integer division, s is the number of overall encoder stages, ψ s represents the Mamba model of the encoder at the sth stage, ζ s Represents the CNN model of the encoder at the sth stage.

[0123] The present invention combines the Mamba model with the traditional CNN model at the encoder stage, adopts an innovative phased design approach, uses CNN in the low-order part of the encoder to capture local and detail features, and uses the Mamba model in the high-order part of the encoder to extract global dependency features. This joint model architecture can effectively cope with the complex structure of clinical big data and effectively identify the biological structure contours of tissues and organs.

[0124] Embodiment 3

[0125] The present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the individualized abdominal organ segmentation and reconstruction method described in the first embodiment, and implements the following functions: the Mamba model is combined with the traditional CNN model at the encoder stage, an innovative phased design idea is adopted, CNN is used in the low-order part of the encoder to capture local and detail features, and the Mamba model is used in the high-order part of the encoder to extract global dependency features, and this joint model architecture can effectively cope with the complex structure of clinical big data and effectively identify the biological structure contours of tissues and organs. In order to fully capture the important information carried in clinical big data, the present invention adopts a larger training batch, with the help of the GPU acceleration algorithm of the Mamba structure, improves the model learning speed, improves the stability during the training process, uses the Adam optimizer combined with the adaptive learning rate adjustment strategy, constructs a model optimization scheme, adopts the Dice loss function, optimizes the accuracy of the model in the task of tissue and organ segmentation, and improves the processing speed and accuracy of medical images. Specifically, the electronic device can be a controller in a specific device.

[0126] Embodiment 4

[0127] Based on the same technical concept, the embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the steps of the above-mentioned individualized abdominal organ segmentation and reconstruction method. And realize the following functions: the Mamba model is combined with the traditional CNN model at the encoder stage, and an innovative phased design idea is adopted. CNN is used in the low-order part of the encoder to capture local and detail features, and the Mamba model is used in the high-order part of the encoder to extract global dependency features. Through this joint model architecture, the complex structure of clinical big data can be effectively dealt with, and the biological structure contours of tissues and organs can be effectively identified. In order to fully capture the important information carried in clinical big data, the present invention adopts a larger training batch, with the help of the GPU acceleration algorithm of the Mamba structure, improves the model learning speed, improves the stability during the training process, uses the Adam optimizer combined with the adaptive learning rate adjustment strategy, constructs a model optimization scheme, and adopts the Dice loss function to optimize the model's accuracy in the task of tissue and organ segmentation, thereby improving the processing speed and accuracy of medical images.

[0128] Embodiment 5

[0129] The present application provides an individualized virtual surgical operation device, including the electronic device described in the above embodiment three, which is a controller in the individualized virtual surgical operation device. The electronic device includes: a memory, a processor, and a computer program stored in the memory and can be run on the processor, and the processor executes the program to implement the individualized abdominal organ segmentation and reconstruction method described in the above embodiment one, and realizes the following functions: the Mamba model is combined with the traditional CNN model in the encoder stage, and an innovative phased design idea is adopted. CNN is used in the low-order part of the encoder to capture local and detail features, and the Mamba model is used in the high-order part of the encoder to extract global dependency features. Through this joint model architecture, the complex structure of clinical big data can be effectively dealt with, and the biological structure contour of tissues and organs can be effectively identified. In order to fully capture the important information carried in clinical big data, the present invention adopts a larger training batch, with the help of the GPU acceleration algorithm of the Mamba structure, improves the model learning speed, improves the stability during the training process, uses the Adam optimizer combined with the adaptive learning rate adjustment strategy, constructs a model optimization scheme, and adopts the Dice loss function to optimize the accuracy of the model in the task of tissue and organ segmentation, thereby improving the processing speed and accuracy of medical images. The personalized virtual surgical operation device can use the electronic device to perform personalized abdominal organ segmentation and reconstruction before the actual operation, helping doctors understand the real situation of abdominal surgery.

[0130] The above describes specific embodiments of the present invention. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In the embodiments of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in the embodiments of the present invention and the features of the different embodiments or examples, without contradiction.

[0132] In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features, and do not include any ordering. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features and are used to distinguish each other. In the description of the embodiments of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0133] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred implementation of the embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0134] The above description is only a preferred embodiment of the embodiment of the present invention and is not intended to limit the embodiment of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiment of the present invention should be included in the scope of protection of the embodiment of the present invention.

Claims

1. A method for individualized abdominal organ segmentation and reconstruction, characterized in that: The steps include: S1. Acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data enhancement processing on the clinical medical imaging data; S2, constructing a joint segmentation model, and using the clinical medical image data obtained by processing in step S1 to train the joint segmentation model; wherein the joint segmentation model uses a CNN network in the low-order part of the encoder to capture local and detail features, and uses a Mamba model in the high-order part of the encoder to extract global dependency features; S3, obtaining clinical medical imaging data of individual abdominal organs, and preprocessing the clinical medical imaging data; S4, inputting the clinical medical image data obtained by the preprocessing in step S3 into the joint segmentation model for segmentation processing, and outputting the segmentation result of the specific area; S5. Perform individualized three-dimensional reconstruction of abdominal organs based on the segmentation result of the specific area obtained in step S4.

2. The method for individualized abdominal organ segmentation and reconstruction according to claim 1, characterized in that: The preprocessing in step S1 includes: Performing data standardization processing on the clinical medical imaging data, and adjusting the clinical medical imaging data of different imaging intensities to a uniform resolution and size by using spatial resampling and cubic interpolation methods; Gaussian filtering and median filtering are combined to reduce random noise in clinical medical images. When processing images involving complex tumor lesions, non-local mean denoising technology is combined to reduce noise in clinical medical images.

3. The method for individualized abdominal organ segmentation and reconstruction according to claim 1, characterized in that: The data enhancement process in step S1 includes: Geometric transformation, brightness and contrast intensity transformation, and elastic deformation are used to simulate the possible variations of clinical medical images in actual medical environments, so as to enhance the adaptability of the joint segmentation model to illumination changes and tissue deformation. Among them, the geometric transformation includes rotation, scaling, flipping and twisting transformation; for complex tumor lesion images, diffusion tensor technology is used to generate synthetic data to improve the performance and robustness of the joint segmentation model in processing diverse clinical medical imaging data.

4. The method for individualized abdominal organ segmentation and reconstruction according to claim 1, characterized in that: The step S5 converts the voxelized data in the segmentation result into a three-dimensional surface model by using the Marching Cubes algorithm to perform individualized abdominal organ segmentation and reconstruction.

5. The method for individualized abdominal organ segmentation and reconstruction according to claim 1, characterized in that: The step of training the joint segmentation model in step S2 includes: Use the GPU acceleration algorithm of the Mamba model to improve learning speed; A joint segmentation model optimization scheme is constructed by using the Adam optimizer combined with an adaptive learning rate adjustment strategy; wherein the adaptive learning rate adjustment strategy includes dynamically adjusting the learning rate according to the performance of the joint segmentation model; The Dice loss function is used to optimize the accuracy of the joint segmentation model in the abdominal organ segmentation task.

6. The method for individualized abdominal organ segmentation and reconstruction according to claim 5, characterized in that: The steps of dynamically adjusting the learning rate according to the performance of the joint segmentation model include: using a higher learning rate in the initial stage to converge quickly, and if the decrease rate of the loss on the validation set is less than a preset threshold in multiple consecutive training cycles, then gradually reducing the learning rate according to a preset decay rate; Among them, the early stopping strategy is used to monitor the performance on the validation set as a criterion for reducing the learning rate.

7. The method for individualized abdominal organ segmentation and reconstruction according to any one of claims 1 to 6, characterized in that: The preprocessing in step S3 includes: using a histogram equalization technique to analyze the CT value distribution, so as to enhance the local contrast between the liver and surrounding tissues.

8. The method for individualized abdominal organ segmentation and reconstruction according to claim 7, characterized in that: After the joint segmentation model performs segmentation processing, the following steps are further included: Using corrosion and dilation morphological operations to remove or repair abnormal areas, wherein the abnormal areas include tiny gaps, tiny fractures, or non-smooth, discontinuous areas; wherein outputting the segmentation result of the specific area includes outputting the segmentation result of the liver; The liver regions were selected using a connected component labeling algorithm, and regions that were discontinuous with the main liver volume and whose volume was smaller than a preset threshold were removed; Gaussian blur algorithm is used to smooth the edges of the liver segmentation results.

9. The method for individualized abdominal organ segmentation and reconstruction according to claim 7, characterized in that: The staged encoder in the joint segmentation model is represented as follows: S e =[ξ 1 ,x 2 ,...,x s / / 2 ,ψ s / / 2+1 ,...,ψ s-1 ,ψ s ] Among them, ∑ e represents the staged structure of the entire encoder, ζ and ψ represent the CNN model and Mamba model respectively, / / represents integer division, s is the number of overall encoder stages, ψ s represents the Mamba model of the encoder at the sth stage, ζ s Represents the CNN model of the encoder at the sth stage.

10. An individualized abdominal organ segmentation and reconstruction device, characterized in that: include: A training data acquisition module, used to acquire diverse clinical medical imaging data of abdominal organs, and perform preprocessing and data enhancement processing on the clinical medical imaging data; A model training module; used to construct a joint segmentation model, and train the joint segmentation model using the clinical medical image data processed by the training data acquisition module; wherein the joint segmentation model uses a CNN network in the low-order part of the encoder to capture local and detail features, and uses a Mamba model in the high-order part of the encoder to extract global dependency features; An individual data acquisition module, used to acquire clinical medical imaging data of individual abdominal organs and pre-process the clinical medical imaging data; The three-dimensional reconstruction module is used to input the clinical medical image data preprocessed by the individual data acquisition module into the joint segmentation model for segmentation processing, output the segmentation results of the specific area, and perform individualized three-dimensional reconstruction of abdominal organs based on the segmentation results of the specific area.

11. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the individualized abdominal organ segmentation and reconstruction method according to any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a computer or a processor, the computer or the processor executes the individualized abdominal organ segmentation and reconstruction method according to any one of claims 1 to 9.

13. An individualized virtual surgical operation device, characterized in that: The electronic device comprising claim 11.

Images