Image three-dimensional post-processing method and system based on multiple modes

Through the three-dimensional post-processing method of multimodal images, combined with feature extraction and registration of CT, MRI and PET images, a three-dimensional comprehensive image is generated, which solves the problem of inaccurate information loss and alignment, and achieves a more comprehensive image information reflection and Improved diagnostic efficiency.

CN119048694APending Publication Date: 2024-11-29FUJIAN ZHIKANGYUN MEDICAL TECH CO LTD

Patent Information

Application Number
CN202411538123.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-31
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing medical imaging technologies can easily lead to information loss and inaccurate image alignment in the post-processing of multimodal images, especially in three-dimensional reconstruction and analysis, which cannot fully reflect information about bone, soft tissue and metabolic activities.

Method used

Using a multimodal-based three-dimensional post-processing method, the fusion of three-dimensional synthesis is generated by acquiring CT, MRI and PET image data, preprocessing, feature extraction, rigid and elastic registration, three-dimensional reconstruction, lesion analysis and dynamic weighted fusion. Image data.

Benefits of technology

Improve the spatial alignment accuracy of multimodal images, fully reflecting information about bone, soft tissue and metabolic activities. The automated lesion identification and labeling process greatly improves diagnostic efficiency. Dynamic weighted fusion and interactive visual display functions make image display more Flexibility enhances diagnostic value.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048694A_ABST
    Figure CN119048694A_ABST
Patent Text Reader

Abstract

The invention provides an image three-dimensional post-processing method and system based on multiple modalities, and relates to the technical field of image processing.The method comprises the steps that CT image data, MRI image data and PET image data are obtained and preprocessed, and preprocessed data are obtained; performing feature extraction on the preprocessed data to obtain skeleton structure features, soft tissue structure features and metabolic activity features; performing rigid registration on the skeleton structure features and the soft tissue structure features to obtain preliminary registration data; performing elastic registration on the preliminary registration data set and the metabolic activity characteristics to obtain multi-modal image data; generating a three-dimensional reconstruction model according to the multi-modal image data; segmenting the three-dimensional reconstruction model to obtain lesion analysis data; analyzing according to the lesion analysis data to obtain lesion marking data; and performing dynamic weight fusion processing on the lesion mark data and the tissue structure data to obtain three-dimensional comprehensive image data. The invention provides a more accurate influence processing method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a multi-modal three-dimensional image post-processing method and system. Background Art

[0002] Currently, medical imaging technology has become an indispensable tool in clinical diagnosis and is widely used in disease detection and treatment planning. Common types of medical imaging include CT (computed tomography), MRI (magnetic resonance imaging), and PET (positron emission tomography). These imaging technologies can provide structural and functional information about the patient's internal tissues, but each imaging technology has its limitations. For example, CT images can display bones and hard tissues well, but the resolution of soft tissues is limited; while MRI images can show soft tissue details more clearly, but the effect is poor in bone imaging. In order to obtain more comprehensive diagnostic information, it is usually necessary to comprehensively process and analyze multiple imaging data.

[0003] Among them, for the post-processing of multimodal images, the common practice is to analyze each modality independently in turn, and then simply superimpose the analysis results. This method is prone to information loss or inaccurate image alignment, especially in three-dimensional reconstruction and analysis. Summary of the invention

[0004] The purpose of the present invention is to provide a multimodal three-dimensional image post-processing method and system, aiming to solve the problems mentioned in the background technology.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] In a first aspect, a multimodal three-dimensional image post-processing method is provided, the method comprising:

[0007] Acquire CT image data, MRI image data and PET image data;

[0008] Preprocessing the CT image data, the MRI image data, and the PET image data to obtain first CT image data, first MRI image data, and first PET image data;

[0009] Extracting features from the first CT image data, the first MRI image data, and the first PET image data to obtain bone structure features, soft tissue structure features, and metabolic activity features;

[0010] Perform rigid spatial registration on the bone structure features and soft tissue structure features to obtain preliminary registration data;

[0011] Perform elastic spatial registration on the preliminary registration data set and metabolic activity features to obtain multimodal imaging data;

[0012] Perform three-dimensional reconstruction processing based on multimodal imaging data to generate a three-dimensional reconstruction model that integrates anatomical information, soft tissue information, and metabolic information;

[0013] Segment the three-dimensional reconstructed model to obtain lesion analysis data and tissue structure data;

[0014] According to the lesion analysis data, lesion feature data is extracted and separated to obtain lesion marking data;

[0015] Perform dynamic weighted fusion processing on the lesion marking data and the tissue structure data to obtain three-dimensional comprehensive image data;

[0016] The three-dimensional comprehensive image data is dynamically displayed through a visual interface.

[0017] Preferably, preprocessing the CT image data, the MRI image data and the PET image data to obtain the first CT image data, the first MRI image data and the first PET image data includes:

[0018] Performing denoising processing on the CT image data to obtain first CT image data;

[0019] Performing enhancement processing on the MRI image data to obtain first MRI image data;

[0020] Perform artifact removal processing on the PET image data to obtain first PET image data.

[0021] Preferably, the bone structure features The extraction formula is: ,

[0022] in, is the first CT image data, and are the gradients of the CT image in the x-axis and y-axis directions, is with adaptive Gaussian smoothing and is the gradient operator of , and σ is the standard deviation of the Gaussian filter.

[0023] Preferably, the soft tissue structural features The extraction formula is: ,

[0024] in, is the first MRI image data, S is the soft tissue segmentation function, and are the lower threshold and the upper threshold respectively. The pixel values ​​that meet the threshold range are soft tissue features. Represents the segmented soft tissue area.

[0025] Preferably, the metabolic activity characteristic The extraction formula is: , ,

[0026] in, is the first PET image data, M is the metabolic activity extraction function, The threshold extracted for metabolic activity, is the weight, Indicates that the intensity of all pixels in the metabolically active area is accumulated.

[0027] Preferably, rigid spatial registration is performed on the bone structure features and the soft tissue structure features to obtain preliminary registration data, including:

[0028] Obtaining the spatial coordinates of bone structure features and soft tissue structure features;

[0029] According to the spatial coordinates, through the rigid transformation matrix Spatial alignment of bone structure features and soft tissue structure features to obtain registered bone structure features and registration of soft tissue structural features ;

[0030] The expression of the rigid transformation matrix is: ,

[0031] Where θ is the rotation angle, is the translation vector;

[0032] Registering bone structure features and registration of soft tissue structural features The calculation formula is: , .

[0033] Preferably, elastic spatial registration is performed on the preliminary registration data set and the metabolic activity features to obtain multimodal imaging data, including:

[0034] According to the registered bone structure features and the registered soft tissue structure features, the similarity difference of the preset area is calculated to obtain the similarity difference data;

[0035] Determining deformation data of a preset area according to the similarity difference data;

[0036] Based on the deformation data, a deformation parameter field is generated to describe the displacement of each pixel. ;

[0037] The metabolic activity characteristics are adjusted according to the deformation parameter field to obtain the registered metabolic activity characteristics ;

[0038] The extraction formula for the registered metabolic activity features is: .

[0039] Multimodal imaging data are generated based on the registered bone structure features, the registered soft tissue structure features and the registered metabolic activity features.

[0040] Preferably, according to the lesion analysis data, lesion feature data is extracted and separated to obtain lesion marking data, including:

[0041] Based on the lesion analysis data, the lesion volume, surface area and compactness are calculated, and the shape features are extracted. ;

[0042] The calculation formula of the lesion volume is: ,in, is the total number of voxels within the lesion area, is the volume of each voxel, is the index;

[0043] The surface area is calculated as: ,in, is the total number of triangle elements in the boundary mesh based on the lesion area, represents the area of ​​each triangular unit, is the index;

[0044] The calculation formula of the compactness is: ;

[0045] According to the lesion analysis data, the gray level co-occurrence matrix is ​​constructed, and the entropy value and contrast are calculated through the gray level co-occurrence matrix to extract the texture features. ;

[0046] The calculation formula of the entropy value is: ,in, Represents the gray level in the gray level co-occurrence matrix and The co-occurrence probability between is the total number of gray levels;

[0047] The calculation formula of the contrast ratio is: ,in, Indicates grayscale level and The difference between

[0048] Extract metabolic activity features based on lesion analysis data ;

[0049] The metabolic activity characteristics Obtained by the following formula: ,in, is the weight;

[0050] The shape features, texture features and metabolic activity features are integrated to obtain the lesion feature data;

[0051] Label and separate the lesion feature data to obtain lesion label data .

[0052] Preferably, the lesion marking data and the tissue structure data are dynamically weighted and fused to obtain three-dimensional comprehensive image data, including:

[0053] Assign weights to the lesion marking data and tissue structure data, and perform dynamic fusion processing to obtain three-dimensional comprehensive image data. The weighted formula is: ,

[0054] in, Labeling data for lesions The weight coefficient of for and The weight coefficient of .

[0055] In a second aspect, a three-dimensional post-processing system based on multimodal images, the system comprising:

[0056] A data collection module, used for acquiring CT image data, MRI image data and PET image data;

[0057] A denoising processing module, used for performing denoising processing on the CT image data to obtain first CT image data;

[0058] An enhancement processing module, used for performing enhancement processing on the MRI image data to obtain first MRI image data;

[0059] An artifact removal module is used to perform artifact removal processing on the PET image data to obtain first PET image data;

[0060] A feature extraction module, used to extract features from the first CT image data, the first MRI image data, and the first PET image data to obtain bone structure features, soft tissue structure features, and metabolic activity features;

[0061] A rigid registration module is used to perform rigid spatial registration of bone structure features and soft tissue structure features to obtain preliminary registration data;

[0062] The elastic registration module is used to perform elastic spatial registration of the preliminary registration data set and metabolic activity features to obtain multimodal imaging data;

[0063] A three-dimensional reconstruction module is used to perform three-dimensional reconstruction processing based on multimodal imaging data to generate a three-dimensional reconstruction model that integrates anatomical information, soft tissue information, and metabolic information;

[0064] A segmentation processing module is used to perform segmentation processing on the three-dimensional reconstruction model to obtain lesion analysis data and tissue structure data;

[0065] A lesion extraction module is used to extract and separate lesion feature data based on lesion analysis data to obtain lesion marking data;

[0066] The weighted fusion module is used to dynamically perform weighted fusion processing on the lesion marking data and the tissue structure data to obtain three-dimensional comprehensive image data;

[0067] The visualization module is used to dynamically display the three-dimensional comprehensive image data through a visualization interface.

[0068] The above solution of the present invention includes at least the following beneficial effects:

[0069] First, the comprehensive processing of multimodal images can comprehensively reflect the information of bones, soft tissues and metabolic activities, solving the limitation that a single modality image cannot cover all information; second, the combination of rigid and elastic registration greatly improves the spatial alignment accuracy of images of different modalities, especially solves the inconsistency problem caused by local area deformation; third, the automated lesion recognition and marking process greatly improves the diagnostic efficiency and avoids the errors that may be caused by manual marking; finally, dynamic weighted fusion and interactive visualization display functions make image display more flexible, and doctors can adjust the image display method according to actual needs, further enhancing the diagnostic value of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a flowchart of a multi-modal three-dimensional image post-processing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0072] like Figure 1 As shown, an embodiment of the present invention provides a multi-modal three-dimensional image post-processing method, the method comprising the following steps:

[0073] Acquire CT image data, MRI image data and PET image data;

[0074] Performing denoising processing on the CT image data to obtain first CT image data;

[0075] Performing enhancement processing on the MRI image data to obtain first MRI image data;

[0076] Performing artifact removal processing on the PET image data to obtain first PET image data;

[0077] Extracting features from the first CT image data, the first MRI image data, and the first PET image data to obtain bone structure features, soft tissue structure features, and metabolic activity features;

[0078] Perform rigid spatial registration on the bone structure features and soft tissue structure features to obtain preliminary registration data;

[0079] Perform elastic spatial registration on the preliminary registration data set and metabolic activity features to obtain multimodal imaging data;

[0080] Perform three-dimensional reconstruction processing based on multimodal imaging data to generate a three-dimensional reconstruction model that integrates anatomical information, soft tissue information, and metabolic information;

[0081] Segment the three-dimensional reconstructed model to obtain lesion analysis data and tissue structure data;

[0082] According to the lesion analysis data, lesion feature data is extracted and separated to obtain lesion marking data;

[0083] Perform dynamic weighted fusion processing on the lesion marking data and the tissue structure data to obtain three-dimensional comprehensive image data;

[0084] The three-dimensional comprehensive image data is dynamically displayed through a visual interface.

[0085] In an embodiment of the present invention, first, data is acquired from different imaging modalities, specifically including CT images, MRI images, and PET images. CT images provide information about bone structure, MRI images enhance the details of soft tissues, and PET images provide data on metabolic activity. Each modality image has different application advantages. CT images have advantages in displaying bones and hard tissues, MRI images focus on displaying the details of soft tissues, and PET images are mainly used to display metabolic activity areas. The integration of these data makes subsequent analysis more comprehensive and accurate, and can provide multi-angle support for diagnosis.

[0086] After acquiring the image data, the data is preprocessed. The preprocessing of CT images includes denoising, which removes noise by applying a denoising filter to ensure the high quality of the bone structure data. The preprocessing of MRI images adjusts the contrast through an image enhancement algorithm to highlight the details of the soft tissue structure. For PET images, artifact removal is performed to eliminate artifact interference and ensure the accuracy of metabolic activity data. After these preprocessing, the image data obtained are respectively referred to as the first CT image data, the first MRI image data, and the first PET image data.

[0087] Next, key features are extracted from the three types of image data. CT images are mainly used to extract bone structure features, MRI images are used to extract soft tissue features, and PET images are used to extract metabolic activity features. Each feature data is used for subsequent registration and reconstruction operations to ensure the integrity and consistency of the data processed in subsequent steps.

[0088] Registration of multimodal images is an important step to achieve subsequent reconstruction and analysis. First, the extracted bone structure features and soft tissue structure features are rigidly registered in space. Rigid registration keeps the geometric shape of the image unchanged and only rotates and translates the image to ensure that the images of different modalities are aligned in space. After rigid registration, the metabolic activity features are further processed through elastic registration. Elastic registration allows the image to be slightly deformed and adjusted in local areas to ensure that the metabolic activity features are accurately aligned with the bone and soft tissue structures. In particular, for local differences caused by changes in patient position or physiological activities, elastic registration can significantly improve the accuracy of image alignment.

[0089] After the registration is completed, the multimodal imaging data is combined for 3D reconstruction. By comprehensively analyzing the characteristics of bone structure, soft tissue structure, and metabolic activity, a 3D reconstruction model that can integrate anatomical, soft tissue, and metabolic information is generated. This model provides more comprehensive imaging information and provides strong support for subsequent lesion identification and clinical diagnosis.

[0090] After the three-dimensional reconstruction is completed, the lesion area is automatically identified and marked. The identification of the lesion area is based on the comprehensive analysis of multiple features, especially through the comprehensive extraction of shape features, texture features and metabolic features, the location and range of the lesion area can be accurately identified. The shape feature analyzes the volume, surface area and compactness of the lesion area, the texture feature extracts the entropy value and contrast of the lesion area, and the metabolic feature analyzes the metabolically active area in the PET image. These multi-angle analyses greatly improve the accuracy of lesion identification and ensure the complete marking of the lesion area.

[0091] After the lesion is marked, the lesion marking data and tissue structure data are fused through a dynamic weighted fusion algorithm. The weighted fusion algorithm allows the user to adjust the display ratio of the lesion area to the normal tissue according to the weight set by the user. In this way, doctors can more flexibly display specific areas in the image according to actual needs, such as focusing on metabolically active areas or highlighting soft tissue structures. The image data after weighted fusion processing can achieve dynamic fusion display between the lesion area and normal tissue.

[0092] Finally, the fused multimodal image data is displayed through a visualization interface. Users can interact with the image data through the three-dimensional visualization interface and adjust the display mode and transparency of the image. It allows users to observe the lesion area from different angles to ensure that doctors can obtain more comprehensive diagnostic information. Users can also dynamically adjust the display weights of different modal images so that the fused image can meet specific diagnostic needs. Through this flexible visualization, doctors can analyze the lesion area from multiple dimensions and improve the accuracy and efficiency of diagnosis.

[0093] Compared with the prior art, the present invention has the following significant advantages: first, the comprehensive processing of multimodal images can comprehensively reflect the information of bones, soft tissues and metabolic activities, solving the limitation that a single modality image cannot cover all information; second, the combination of rigid and elastic registration greatly improves the spatial alignment accuracy of images of different modalities, especially solves the inconsistency problem caused by local area deformation; third, the automated lesion recognition and marking process greatly improves the diagnostic efficiency and avoids the errors that may be caused by manual marking; finally, the dynamic weighted fusion and interactive visualization display functions make the image display more flexible, and doctors can adjust the image display method according to actual needs, further enhancing the diagnostic value of the image.

[0094] The main purpose of 3D reconstruction is to fuse the features extracted from different modal images (CT, MRI, PET) to generate a comprehensive 3D model that can simultaneously reflect the anatomical structure, soft tissue characteristics and metabolic information. The image data of each modality provides different tissue type information: CT images provide the structure of hard tissues such as bones, MRI images provide details of soft tissues, and PET images show the distribution of metabolic activities. Through the fusion of multimodal images, a 3D model that fully reflects the internal structure of the patient's body can be generated.

[0095] Specifically, the 3D reconstruction process first spatially registers the bone structure features in the CT image, the soft tissue structure features in the MRI image, and the metabolic activity features in the PET image, so that they are accurately aligned in space. Then, these feature data are combined to construct a 3D model through a 3D reconstruction algorithm. This 3D model not only contains the anatomical structure information of the bones and soft tissues, but also reflects the metabolic activity of the patient, especially the metabolically active lesions (such as tumors).

[0096] In the prior art, usually only single-modality image data can be processed, and the characteristics of different tissue types cannot be reflected at the same time. However, the present invention realizes the comprehensive integration of anatomical information, soft tissue information and metabolic information through three-dimensional reconstruction of multi-modal images, and the generated three-dimensional reconstruction model can provide a more comprehensive and accurate basis for diagnosis. For example, in the treatment of cancer, PET images can show metabolically active areas, but cannot accurately reflect the anatomical structure. Through the three-dimensional reconstruction model of the present invention, doctors can not only see the metabolic status of the tumor, but also understand the specific location of the tumor in the anatomical structure.

[0097] Segmentation is a further analysis step of the image data after the 3D reconstruction model is generated. Its purpose is to automatically segment the lesion area and tissue structure area from the reconstructed 3D model. Segmentation is to analyze the volume data in the 3D model through an algorithm to identify the lesion area and normal tissue area.

[0098] Lesion analysis data is obtained by preliminarily identifying the volume, shape, and metabolic activity of the lesion area. Shape analysis can detect the geometric features of the lesion area (such as volume, surface area, and compactness), while texture analysis can extract the complexity of the lesion area (such as entropy and contrast), and metabolic analysis provides the metabolic activity of the lesion area (reflected by PET images). These data can help doctors better understand the characteristics of the lesion area, especially in cancer diagnosis. Segmentation processing can accurately identify the boundaries, volume, and metabolic level of the tumor area, and assist doctors in determining the type and development of the tumor.

[0099] The tissue structure data corresponds to the normal anatomical and soft tissue areas. The segmentation algorithm can distinguish the bones, soft tissues and lesion areas, and provide detailed information about the tissue structure. This has important application value for surgical planning and the selection of radiotherapy targets. For example, doctors can determine the important anatomical areas that need to be avoided during surgery by analyzing the tissue structure data, thereby ensuring the accuracy and safety of the surgery.

[0100] In a preferred embodiment of the present invention, feature extraction is performed on the first CT image data, the first MRI image data, and the first PET image data to obtain bone structure features, soft tissue structure features, and metabolic activity features, including:

[0101] Extracting features from the first CT image data to obtain bone structure features;

[0102] Extracting features from the first MRI image data to obtain soft tissue structure features;

[0103] Feature extraction is performed on the first PET image data to obtain metabolic activity features.

[0104] In the embodiment of the present invention, first, by processing the first CT image data, the bone structure features are extracted. CT images have advantages in representing hard tissues such as bones, and the extracted bone features are used for subsequent spatial registration. For the first MRI image data, soft tissue features are extracted by a segmentation algorithm. MRI images have advantages in showing soft tissue details. The extracted soft tissue features are used for subsequent processing. Finally, by processing the first PET image data, metabolic activity features are extracted. PET images can provide information on metabolic activities. The extracted metabolic features are used to detect metabolic activities in the lesion area.

[0105] Compared with the prior art, the present invention can achieve a more comprehensive analysis of the patient's internal tissues by extracting features from three different modal images, which is especially important in lesion identification and treatment planning. Single modality images are prone to miss some important information, and feature extraction from multimodal images can greatly improve the accuracy and comprehensiveness of diagnosis.

[0106] In a preferred embodiment of the present invention, the bone structure features The extraction formula is: ,

[0107] in, is the first CT image data, and are the gradients of the CT image in the x-axis and y-axis directions, is with adaptive Gaussian smoothing and is the gradient operator of , and σ is the standard deviation of the Gaussian filter.

[0108] In the embodiment of the present invention, the edge features of the bone structure in the first CT image can be accurately extracted through the formula. Compared with the existing edge detection technology, the present invention uses adaptive Gaussian smoothing, which can better process the noise in the first CT image, ensure that the edge detection of the bone structure is more accurate, and facilitate the subsequent registration process.

[0109] The formula extracts edge information from the image by performing gradient calculations on the x-axis and y-axis of the first CT image. By introducing a Gaussian smoothing filter , reducing the impact of noise on gradient calculation, thereby more accurately extracting bone structure features.

[0110] This method ensures the accurate extraction of bone edges, which is beneficial for subsequent registration and three-dimensional reconstruction.

[0111] In a preferred embodiment of the present invention, the soft tissue structure features The extraction formula is: ,

[0112] in, is the first MRI image data, S is the soft tissue segmentation function, and are the lower threshold and the upper threshold respectively, and the pixel values ​​that meet the threshold range are soft tissue features.

[0113] In the embodiment of the present invention, by setting a suitable threshold, the present invention can accurately extract the soft tissue area in the first MRI image, especially when the soft tissue display is more complex, and can effectively improve the accuracy of soft tissue extraction, thereby solving the problem of difficulty in distinguishing soft tissue from background in the prior art.

[0114] The formula extracts the soft tissue area in the first MRI image by threshold segmentation. and When the pixel is between , the pixel is classified as a soft tissue feature, otherwise it does not belong to soft tissue.

[0115] This method ensures accurate extraction of soft tissue, especially in the first MRI image, where the extraction of soft tissue features is crucial for the identification of lesion areas.

[0116] In a preferred embodiment of the present invention, the metabolic activity characteristics The extraction formula is: , ,

[0117] in, is the first PET image data, M is the metabolic activity extraction function, The threshold extracted for metabolic activity, is the weight, Indicates that the intensity of all pixels in the metabolically active area is accumulated.

[0118] In the embodiment of the present invention, the above formula can be used to accurately extract the metabolically active area in the first PET image, and the weights are Improve the visibility of metabolic activity. This extraction method is particularly important in cancer detection, because metabolically active areas are often an indicator of the presence of tumors. Compared with traditional metabolic extraction methods, the algorithm in this paper can better adapt to changes in different thresholds and enhancement factors, ensuring the flexibility and accuracy of the extraction results.

[0119] The formula extracts metabolic features by identifying metabolically active areas in the first PET image. First, the threshold Pixels with metabolic activity intensity higher than the threshold are screened out, and then The weights amplify the metabolic activity intensity of these areas.

[0120] The cumulative processing ensures that the metabolic activity of the entire lesion area can be comprehensively extracted for subsequent analysis of the metabolic level of the lesion area.

[0121] In a preferred embodiment of the present invention, rigid spatial registration is performed on the bone structure features and the soft tissue structure features to obtain preliminary registration data, including:

[0122] Obtaining the spatial coordinates of bone structure features and soft tissue structure features;

[0123] According to the spatial coordinates, through the rigid transformation matrix Spatial alignment of bone structure features and soft tissue structure features to obtain registered bone structure features and registration of soft tissue structural features ;

[0124] The expression of the rigid transformation matrix is: ,

[0125] Where θ is the rotation angle, is the translation vector;

[0126] Registering bone structure features and registration of soft tissue structural features The calculation formula is: , .

[0127] In the embodiment of the present invention, through the rigid transformation formula, the CT image and the MRI image can be spatially aligned while maintaining the shape unchanged. This process is particularly important for the registration of multimodal images, which can ensure the consistency of the anatomical structure of different modal images. Compared with the prior art, the rigid registration formula of the present invention can adapt to more complex rotation and translation situations, avoiding the problem of image information loss or inaccurate overlap caused by spatial misalignment.

[0128] In a preferred embodiment of the present invention, elastic spatial registration is performed on the preliminary registration data set and the metabolic activity features to obtain multimodal imaging data, including:

[0129] According to the registered bone structure features and the registered soft tissue structure features, the similarity difference of the preset area is calculated to obtain the similarity difference data;

[0130] Determining deformation data of a preset area according to the similarity difference data;

[0131] Based on the deformation data, a deformation parameter field is generated to describe the displacement of each pixel. ;

[0132] The metabolic activity characteristics are adjusted according to the deformation parameter field to obtain the registered metabolic activity characteristics ;

[0133] The extraction formula for the registered metabolic activity features is: .

[0134] Multimodal imaging data are generated based on the registered bone structure features, the registered soft tissue structure features and the registered metabolic activity features.

[0135] In the embodiment of the present invention, through this elastic registration algorithm, it is possible to correct the subtle differences of local anatomical structures in different modality images, so that metabolic activity is more closely integrated with bone and soft tissue information. Compared with the prior art, elastic registration has higher accuracy and is particularly suitable for problems of local distortion or deformation of images caused by patient movement, breathing, etc.

[0136] The core of elastic registration is the generation of deformation parameter fields, which describe the displacement of each pixel in space. Through this elastic registration algorithm, it is possible to correct the subtle differences in local anatomical structures in different modal images, so that metabolic activities are more closely integrated with bone and soft tissue information. Compared with existing technologies, elastic registration has higher accuracy and is particularly suitable for problems with local distortion or deformation of images caused by patient movement, breathing, etc.

[0137] Among them, according to the registered bone structure features and the registered soft tissue structure features, the similarity difference of the preset area is calculated to obtain the similarity difference data, which can be calculated by using the mean square error to calculate the sum of square errors between the corresponding pixels of the two feature images. The smaller the mean square error, the more similar the images are. Then, the difference data that meets the requirements can be obtained.

[0138] Among them, the deformation data of the preset area is determined based on the similarity difference data, and the deformation data can be calculated based on a deformation model, such as a B-spline model, a finite element model, etc. The deformation data describes the spatial displacement of each pixel or voxel in the preset area.

[0139] The B-spline model is used to describe the deformation of a preset area by means of control points and spline interpolation. The displacement of each control point is adjusted by an optimization algorithm, and the deformation field is generated by the movement of these control points. The advantage of B-spline is that it can generate a smooth deformation field, which is suitable for complex biological deformation scenarios.

[0140] The finite element method is used to divide the image into multiple small finite element regions and calculate the deformation field by minimizing the energy difference between these regions. The finite element method is suitable for processing regions with obvious structural differences in the image, especially for the deformation of soft tissue.

[0141] In a preferred embodiment of the present invention, based on the lesion analysis data, lesion feature data is extracted and separated to obtain lesion marking data, including:

[0142] Based on the lesion analysis data, the lesion volume, surface area and compactness are calculated, and the shape features are extracted. ;

[0143] The calculation formula of the lesion volume is: ,in, is the total number of voxels within the lesion area, is the volume of each voxel, is the index;

[0144] The surface area is calculated as: ,in, is the total number of triangle elements in the boundary mesh based on the lesion area, represents the area of ​​each triangular unit, is the index;

[0145] The calculation formula of the compactness is: ;

[0146] According to the lesion analysis data, the gray level co-occurrence matrix is ​​constructed, and the entropy value and contrast are calculated through the gray level co-occurrence matrix to extract the texture features. ;

[0147] The calculation formula of the entropy value is: ,in, Represents the gray level in the gray level co-occurrence matrix and The co-occurrence probability between is the total number of gray levels;

[0148] The calculation formula of the contrast ratio is: ,in, Indicates grayscale level and The difference between

[0149] Extract metabolic activity features based on lesion analysis data ;

[0150] The metabolic activity characteristics Obtained by the following formula: ,in, is the weight;

[0151] The shape features, texture features and metabolic activity features are integrated to obtain the lesion feature data;

[0152] Label and separate the lesion feature data to obtain lesion label data .

[0153] In an embodiment of the present invention, the lesion feature extraction in the present invention is based on a comprehensive analysis of shape features, texture features and metabolic activity features. Through this lesion recognition based on multiple features, the present invention can effectively improve the accuracy of lesion detection, especially in complex lesion areas in multimodal images, which can be comprehensively analyzed by integrating shape, texture and metabolic information. Compared with the prior art, the present invention can more accurately identify complex lesion areas, especially in the case of complex textures or abnormal metabolic activities.

[0154] Volume reflects the size of the lesion, surface area reflects the shape of the lesion, and compactness characterizes the regularity of the lesion. Through the comprehensive analysis of these small features, the system can obtain the overall shape characteristics of the lesion area, providing doctors with key information about the growth pattern and appearance characteristics of the lesion.

[0155] The entropy value reflects the texture complexity of the lesion area, while the contrast reflects the grayscale difference between the lesion area and the surrounding tissue. By combining these two small features, the overall texture characteristics of the lesion area can be extracted. Areas with high entropy values ​​and high contrast usually mean that the lesion area is complex and has obvious differences from the surrounding tissue, which is of great significance for evaluating the structural characteristics of complex lesions such as tumors.

[0156] By weight Can increase the clarity of metabolic activity characteristics.

[0157] The lesion volume It refers to the volume occupied by the lesion area in three-dimensional space. It is usually obtained by accumulating the voxels in the lesion area. The specific steps are as follows:

[0158] In the above, the segmentation algorithm extracts the lesion analysis area and divides the lesion analysis area into multiple small voxel units. A voxel is a unit volume with a fixed size in three-dimensional space (usually a cube). Calculate the volume of each voxel , and add up the volumes of all voxels in the lesion area to get the total volume of the lesion area.

[0159] The formula is: ,

[0160] in, is the total number of voxels within the lesion area, is the volume of each voxel (voxel volume is determined by the resolution of the CT or MRI image). This voxel accumulation method ensures the accuracy of volume calculation.

[0161] Lesion surface area It refers to the bounding surface area of ​​the lesion area. The steps to calculate the lesion surface area are as follows:

[0162] Extract the three-dimensional boundary of the lesion area and use the triangular mesh method or other geometric algorithms to calculate the total area of ​​the boundary. The triangular mesh method approximates the boundary of the lesion area as multiple triangular units. The area of ​​each triangle is calculated and accumulated to form the surface area of ​​the entire boundary. The formula is: ,

[0163] in, is the total number of triangle elements in the boundary mesh, represents the area of ​​each triangular unit. This formula can accurately calculate the surface area of ​​irregular lesion areas.

[0164] Firmness It is a geometric metric used to describe the regularity of the shape of the lesion area. It is usually calculated by the ratio of volume to surface area. It is used to measure whether the lesion area is close to a regular shape (such as a sphere).

[0165] The formula for firmness is: ,

[0166] in, and are the volume and surface area of ​​the lesion area, respectively. The more regular the shape of the lesion area, the higher its compactness value.

[0167] Among them, the entropy value It is a parameter used to measure the complexity of the grayscale distribution of the lesion area. The higher the entropy value, the more complex the grayscale distribution of the lesion area is and the greater the amount of information. The calculation method of the entropy value is usually based on the grayscale co-occurrence matrix. The steps to calculate the entropy value are as follows:

[0168] First, the gray level co-occurrence matrix of the lesion area is constructed to record the gray level relationship between adjacent pixels in the image. The size of the gray level co-occurrence matrix is ​​determined by the number of gray levels in the image. Determine. Use the following formula to calculate the entropy value: ,

[0169] in, Represents the gray level in the gray level co-occurrence matrix and The co-occurrence probability between is the total number of gray levels. This formula calculates the probability distribution of pixel gray co-occurrence to obtain the entropy value, which reflects the texture complexity of the lesion area.

[0170] Contrast It is used to measure the degree of gray value difference in the lesion area, reflecting the difference between the lesion area and the surrounding tissue. High contrast usually means that the gray value difference in the lesion area is obvious, which makes it easier for doctors to identify the lesion boundary. The calculation of contrast is also based on the gray level co-occurrence matrix. The specific steps are as follows:

[0171] Construct the gray level co-occurrence matrix of the lesion area and record the gray values ​​of adjacent pixels. Calculate the contrast using the following formula: ,

[0172] in, Indicates grayscale level and Through this formula, the grayscale contrast of the lesion area can be accurately calculated, and this parameter can help identify the clarity of the boundary of the lesion area.

[0173] In a preferred embodiment of the present invention, the lesion marking data and the tissue structure data are dynamically weighted and fused to obtain three-dimensional comprehensive image data, including:

[0174] Assign weights to the lesion marking data and tissue structure data, and perform dynamic fusion processing to obtain three-dimensional comprehensive image data. The weighted formula is: ,

[0175] in, Labeling data for lesions The weight coefficient of for and The weight coefficient of .

[0176] In an embodiment of the present invention, dynamic weighted fusion processing is an important step to achieve comprehensive display of lesion marking data and tissue structure data. By assigning weights to different types of data, the present invention can achieve fusion processing of multimodal data. Through this weighted fusion method, the present invention can dynamically adjust the display weight between the lesion area and the tissue structure according to user needs. Compared with the prior art, this method not only improves the flexibility of image display, but also can adjust the display effect of the image through user interaction, providing clearer lesion information.

[0177] An embodiment of the present invention further provides a three-dimensional post-processing system based on multimodal images, which is applied to the above method, and the system includes:

[0178] A data collection module, used for acquiring CT image data, MRI image data and PET image data;

[0179] A denoising processing module, used for performing denoising processing on the CT image data to obtain first CT image data;

[0180] An enhancement processing module, used for performing enhancement processing on the MRI image data to obtain first MRI image data;

[0181] An artifact removal module is used to perform artifact removal processing on the PET image data to obtain first PET image data;

[0182] A feature extraction module, used to extract features from the first CT image data, the first MRI image data, and the first PET image data to obtain bone structure features, soft tissue structure features, and metabolic activity features;

[0183] A rigid registration module is used to perform rigid spatial registration of bone structure features and soft tissue structure features to obtain preliminary registration data;

[0184] The elastic registration module is used to perform elastic spatial registration of the preliminary registration data set and metabolic activity features to obtain multimodal imaging data;

[0185] A three-dimensional reconstruction module is used to perform three-dimensional reconstruction processing based on multimodal imaging data to generate a three-dimensional reconstruction model that integrates anatomical information, soft tissue information, and metabolic information;

[0186] A segmentation processing module is used to perform segmentation processing on the three-dimensional reconstruction model to obtain lesion analysis data and tissue structure data;

[0187] A lesion extraction module is used to extract and separate lesion feature data based on lesion analysis data to obtain lesion marking data;

[0188] The weighted fusion module is used to dynamically perform weighted fusion processing on the lesion marking data and the tissue structure data to obtain three-dimensional comprehensive image data;

[0189] The visualization module is used to dynamically display the three-dimensional comprehensive image data through a visualization interface.

[0190] The denoising process of the image data is implemented by a Gaussian filtering algorithm, and the Gaussian filtering algorithm is used to:

[0191] Noise estimation of CT image data;

[0192] Adjusting filter parameters based on the noise estimation result;

[0193] The image data is smoothed to obtain denoised CT image data.

[0194] The image data enhancement processing is implemented by a contrast stretching algorithm, and the contrast stretching algorithm is used to:

[0195] Obtaining a grayscale histogram of MRI image data;

[0196] Adjust the contrast range of image data according to the grayscale histogram;

[0197] The image data is subjected to contrast stretching processing to obtain enhanced MRI image data.

[0198] It should be noted that the system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0199] The embodiment of the present invention further provides a computing device, including: a processor, a memory storing a computer program, and when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0200] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0201] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in the field of image processing technology, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A three-dimensional image post-processing method based on multimodality, characterized in that: The method comprises: Acquire CT image data, MRI image data and PET image data; Preprocessing the CT image data, the MRI image data, and the PET image data to obtain first CT image data, first MRI image data, and first PET image data; Extracting features from the first CT image data, the first MRI image data, and the first PET image data to obtain bone structure features, soft tissue structure features, and metabolic activity features; Perform rigid spatial registration on the bone structure features and soft tissue structure features to obtain preliminary registration data; Perform elastic spatial registration on the preliminary registration data set and metabolic activity features to obtain multimodal imaging data; Perform three-dimensional reconstruction processing based on multimodal imaging data to generate a three-dimensional reconstruction model that integrates anatomical information, soft tissue information, and metabolic information; Segment the three-dimensional reconstructed model to obtain lesion analysis data and tissue structure data; According to the lesion analysis data, lesion feature data is extracted and separated to obtain lesion marking data; Perform dynamic weighted fusion processing on the lesion marking data and the tissue structure data to obtain three-dimensional comprehensive image data; The three-dimensional comprehensive image data is dynamically displayed through a visual interface.

2. The method for three-dimensional post-processing of multimodal images according to claim 1, characterized in that: Preprocessing the CT image data, the MRI image data, and the PET image data to obtain first CT image data, first MRI image data, and first PET image data includes: Performing denoising processing on the CT image data to obtain first CT image data; Performing enhancement processing on the MRI image data to obtain first MRI image data; Perform artifact removal processing on the PET image data to obtain first PET image data.

3. The multimodal three-dimensional image post-processing method according to claim 2, characterized in that: Bone structure characteristics The extraction formula is: , in, is the first CT image data, and are the gradients of the CT image in the x-axis and y-axis directions, is with adaptive Gaussian smoothing and is the gradient operator of , and σ is the standard deviation of the Gaussian filter.

4. The method for three-dimensional post-processing of multimodal images according to claim 3, characterized in that: Soft tissue structural characteristics The extraction formula is: , in, is the first MRI image data, S is the soft tissue segmentation function, and are the lower threshold and the upper threshold respectively. The pixel values ​​that meet the threshold range are soft tissue features. Represents the segmented soft tissue area.

5. The multimodal three-dimensional image post-processing method according to claim 4, characterized in that: Metabolic activity characteristics The extraction formula is: , , in, is the first PET image data, M is the metabolic activity extraction function, The threshold extracted for metabolic activity, is the weight, Indicates that the intensity of all pixels in the metabolically active area is accumulated.

6. The multimodal three-dimensional image post-processing method according to claim 5, characterized in that: Perform rigid spatial registration on the bone structure features and soft tissue structure features to obtain preliminary registration data, including: Obtaining the spatial coordinates of bone structure features and soft tissue structure features; According to the spatial coordinates, through the rigid transformation matrix Spatial alignment of bone structure features and soft tissue structure features to obtain registered bone structure features and registration of soft tissue structural features ; The expression of the rigid transformation matrix is: , Where θ is the rotation angle, is the translation vector; Registering bone structure features and registration of soft tissue structural features The calculation formula is: , 。 7. The method for three-dimensional post-processing of multimodal images according to claim 6, characterized in that: Perform elastic spatial registration of the preliminary registration data set and metabolic activity features to obtain multimodal imaging data, including: According to the registered bone structure features and the registered soft tissue structure features, the similarity difference of the preset area is calculated to obtain the similarity difference data; Determining deformation data of a preset area according to the similarity difference data; Based on the deformation data, a deformation parameter field is generated to describe the displacement of each pixel. ; The metabolic activity characteristics are adjusted according to the deformation parameter field to obtain the registered metabolic activity characteristics ; The extraction formula for the registered metabolic activity features is: ; Multimodal imaging data are generated based on the registered bone structure features, the registered soft tissue structure features and the registered metabolic activity features.

8. The multimodal three-dimensional image post-processing method according to claim 7, characterized in that: According to the lesion analysis data, the lesion feature data is extracted and separated to obtain the lesion marking data, including: Based on the lesion analysis data, the lesion volume, surface area and compactness are calculated, and the shape features are extracted. ; The calculation formula of the lesion volume is: ,in, is the total number of voxels within the lesion area, is the volume of each voxel, is the index; The surface area is calculated as: ,in, is the total number of triangle elements in the boundary mesh based on the lesion area, represents the area of ​​each triangular unit, is the index; The calculation formula of the compactness is: ; According to the lesion analysis data, the gray level co-occurrence matrix is ​​constructed, and the entropy value and contrast are calculated through the gray level co-occurrence matrix to extract the texture features. ; The calculation formula of the entropy value is: ,in, Represents the gray level in the gray level co-occurrence matrix and The co-occurrence probability between is the total number of gray levels; The calculation formula of the contrast ratio is: ,in, Indicates grayscale level and The difference between Extract metabolic activity features based on lesion analysis data ; The metabolic activity characteristics Obtained by the following formula: ,in, is the weight; The shape features, texture features and metabolic activity features are integrated to obtain the lesion feature data; Label and separate the lesion feature data to obtain lesion label data .

9. The multimodal three-dimensional image post-processing method according to claim 8, characterized in that: The lesion marking data and tissue structure data are dynamically weighted and fused to obtain three-dimensional comprehensive image data, including: Assign weights to the lesion marking data and tissue structure data, and perform dynamic fusion processing to obtain three-dimensional comprehensive image data. The weighted formula is: , in, Labeling data for lesions The weight coefficient of for and The weight coefficient of .

10. A three-dimensional post-processing system based on multimodal images, characterized in that: Applied to the method according to any one of claims 1 to 9, the system comprises: A data collection module, used for acquiring CT image data, MRI image data and PET image data; A denoising processing module, used for performing denoising processing on the CT image data to obtain first CT image data; An enhancement processing module, used for performing enhancement processing on the MRI image data to obtain first MRI image data; An artifact removal module is used to perform artifact removal processing on the PET image data to obtain first PET image data; A feature extraction module, used to extract features from the first CT image data, the first MRI image data, and the first PET image data to obtain bone structure features, soft tissue structure features, and metabolic activity features; A rigid registration module is used to perform rigid spatial registration of bone structure features and soft tissue structure features to obtain preliminary registration data; The elastic registration module is used to perform elastic spatial registration of the preliminary registration data set and metabolic activity features to obtain multimodal imaging data; A three-dimensional reconstruction module is used to perform three-dimensional reconstruction processing based on multimodal imaging data to generate a three-dimensional reconstruction model that integrates anatomical information, soft tissue information, and metabolic information; A segmentation processing module is used to perform segmentation processing on the three-dimensional reconstruction model to obtain lesion analysis data and tissue structure data; A lesion extraction module is used to extract and separate lesion feature data based on lesion analysis data to obtain lesion marking data; The weighted fusion module is used to dynamically perform weighted fusion processing on the lesion marking data and the tissue structure data to obtain three-dimensional comprehensive image data; The visualization module is used to dynamically display the three-dimensional comprehensive image data through a visualization interface.

Citation Information

Patent Citations

  • Automated diagnosis and alignment supplemented with pet / mr flow estimation

    CN101626727A

  • Medical image diagnosis, comparison and reading method

    CN118262875A

  • Processing method and device for three-dimensional focus positioning based on multi-modal image

    CN118334006A

  • Multimodal image acquisition-, processing-, archiving- and visualization system for patient, has merging-, registering and visualization tool for linking, registering and archiving image data of two-dimensional cross sections

    DE102007018630A1

Cited By

  • Intelligent digital image processing system for urinary surgery

    CN119446437A

  • Multi-angle photography and three-dimensional reconstruction method and system for soft tissue

    CN119540468A

  • Tumor angiogenesis detection and analysis method based on multi-modal image fusion

    CN119850621A

  • Method for detecting and analyzing tumor angiogenesis based on multimodal image fusion

    CN119850621B

  • Medical image recognition processing system and method based on multi-modal image fusion

    CN119887773A