Multi-modal imaging tumor positioning method and system
Through multimodal imaging technology, combined with linear stretching, mutual information registration and feature pyramid network, high-precision and non-invasive positioning of tumors are achieved, solving the shortcomings of traditional tumor positioning methods in accurate positioning and functional evaluation, and improving detection accuracy and safety.
Patent Information
- Application Number
- CN202411118524.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional tumor localization methods have shortcomings in early tumor detection, precise localization and functional evaluation, especially B-ultrasound is difficult to determine the type of lesions, CT and MRI have radiation risks and limited resolution to soft tissues.
Multimodal imaging equipment is used to obtain original image data of multiple modalities, perform pre-processing, registration and fusion processing, and combine image recognition technology to achieve high-precision positioning of tumors. The specific steps include using CT, MRI, PET and ultrasound imaging equipment to enhance contrast through linear stretching method, image registration based on mutual information, image fusion through feature pyramid network, and finally using a classifier to perform tumors. position.
It achieves high-precision and non-invasive positioning of the tumor, solves the shortcomings of traditional methods in accurate positioning and functional evaluation, and improves the accuracy and safety of tumor detection.
Smart Images

Figure CN119027504A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of multimodal imaging, and in particular to a tumor localization method and system for multimodal imaging. Background Art
[0002] Tumor localization is an important step in tumor treatment. It helps doctors determine the location, size, shape, and relationship of the tumor to surrounding tissues, so as to develop an appropriate treatment plan. Although traditional tumor localization methods, such as B-ultrasound, CT, MRI, etc., each have their own advantages, they still have shortcomings in early detection, precise positioning, and functional evaluation of tumors. For example, although B-ultrasound is non-invasive and easy to operate, it is difficult to accurately determine the type of lesion; although CT and MRI have high resolution, there are radiation risks and limited resolution for soft tissues. Therefore, a multimodal imaging tumor localization method and system are urgently needed to solve this problem. Summary of the invention
[0003] In view of the above problems, the present invention is proposed to provide a multi-modal imaging tumor localization method and system that overcomes the above problems or at least partially solves the above problems.
[0004] In order to solve the above technical problems, the embodiments of the present application disclose the following technical solutions:
[0005] In a first aspect, an embodiment of the present invention discloses a tumor localization method using multimodal imaging, characterized in that it comprises:
[0006] S100. Using a multi-modal imaging device, obtaining raw image data of multiple modalities;
[0007] S200. Preprocessing the raw image data of the multiple modalities;
[0008] S300. Registering the preprocessed image;
[0009] S400. Fusing the registered multi-modality images to obtain a fused image;
[0010] S500. Perform image recognition on the fused image to locate the tumor.
[0011] Furthermore, in S100, the multimodal imaging device at least includes a CT imaging device, an MRI imaging device and a PET imaging device.
[0012] Furthermore, in S200, the multimodal imaging device at least includes a CT imaging device, an MRI imaging device, a PET imaging device and an ultrasound imaging device.
[0013] Furthermore, a multimodal ultrasound examination method is used to form a new modality image by integrating or fusing multiple ultrasound imaging methods, wherein the multiple ultrasound imaging methods at least include high-frequency ultrasound probe, elastic imaging, ultrasound angiography,
[0014] Furthermore, the raw image data of the multiple modalities are preprocessed, specifically including denoising and contrast enhancement of the raw image data, and using a linear stretching method to enhance the contrast of the raw image data. The specific method includes:
[0015] Calculate the minimum pixel value and the maximum pixel value in the original image;
[0016] Apply a linear transformation formula to each pixel value in the original image, the transformation formula being: new pixel value = slope * original pixel value + intercept;
[0017] Adjust the parameters of the linear conversion formula to achieve the desired enhancement effect.
[0018] Furthermore, in S300, the pre-processed images are registered using an image registration algorithm based on mutual information, and the specific method includes:
[0019] Extracting features from the preprocessed image, including at least key points, edges or regional features in the image;
[0020] The mutual information between two images is used as a similarity measure, and the transformation parameters that maximize the mutual information are obtained through an optimization algorithm to align the two images;
[0021] The registered images are optimized for registration and the registration results are evaluated.
[0022] Furthermore, the registration image is optimized for registration, and the specific method includes:
[0023] Initialize registration; provide an initial transformation parameter for the registration process to accelerate the optimization process and improve the registration accuracy;
[0024] Iterative optimization: by iteratively adjusting the transformation parameters, the mutual information or other similarity measures are maximized or minimized;
[0025] Convergence criteria: Set the convergence criteria, including at least the number of iterations and the change in the similarity measure, to determine when to stop iterating.
[0026] Furthermore, in S400, the registered multi-modality images are fused, and the specific method includes:
[0027] Starting from the original image, the features of the image are gradually extracted through a series of convolutional layers and pooling layers; the feature map output by each layer gradually reduces in size and increases in semantic information;
[0028] Upsample the highest-level feature map output by the backbone to match its size with the feature map of the previous layer, and then fuse the upsampled feature map with the feature map of the previous layer by element-by-element addition or concatenation;
[0029] Before fusion, a 1x1 convolution operation is performed on the feature map of the bottom-up path to adjust its number of channels to match the feature map of the top-down path, and then the two are fused by element-by-element addition;
[0030] The fused feature map is further processed to eliminate the aliasing effect that may occur during the upsampling process and generate the final feature map.
[0031] Furthermore, in S500, image recognition is performed on the fused image to achieve the purpose of locating the tumor, and the specific method includes:
[0032] Train the classifier. Use the labeled dataset to train an SVM or random forest classifier, where each candidate region is labeled as "tumor" or "non-tumor." During training, the classifier learns how to map feature vectors to class labels.
[0033] Tumor localization; extract features for each candidate region of the new image and classify them using the trained classifier; based on the classification results, the candidate region marked as "tumor" is considered to be the tumor location; use post-processing steps to optimize the boundaries and shape of the tumor region;
[0034] Result evaluation: Use an image dataset that did not participate in training to evaluate the accuracy of tumor localization. The evaluation indicators include at least accuracy, recall, F1 score and intersection-over-union ratio.
[0035] In a second aspect, an embodiment of the present invention discloses a multimodal imaging tumor localization system, characterized in that it includes: an original image data acquisition unit, an original image data preprocessing unit, an image registration unit, an image fusion unit and a tumor localization unit; wherein:
[0036] The original image data acquisition unit is used to acquire original image data of multiple modes using a multi-modal imaging device;
[0037] A raw image data preprocessing unit, used for preprocessing the raw image data of the multiple modalities;
[0038] An image registration unit, used for registering the preprocessed image;
[0039] An image fusion unit, used for fusing the registered multi-modal images to obtain a fused image;
[0040] The tumor localization unit is used to perform image recognition on the fused image to achieve the purpose of localizing the tumor.
[0041] In a third aspect, an embodiment of the present invention discloses an electronic device, including:
[0042] one or more processors;
[0043] A memory for storing one or more programs;
[0044] When the one or more programs are executed by the one or more processors, the one or more processors implement the tumor localization method.
[0045] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:
[0046] The embodiment of the present invention discloses a method and system for tumor localization using multimodal imaging, the method comprising: S100. Using a multimodal imaging device to obtain raw image data of multiple modalities; S200. Preprocessing the raw image data of the multiple modalities; S300. Registering the preprocessed images; S400. Fusing the registered multimodal images to obtain a fused image; S500. Performing image recognition on the fused image to achieve the purpose of tumor localization. The present invention achieves high-precision, non-invasive localization of tumors by integrating the advantages of multiple imaging modalities, solving the problem of insufficient precise localization and functional evaluation of traditional tumor localization methods.
[0047] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0049] Figure 1 This is a flow chart of a tumor localization method using multimodal imaging in Example 1 of the present invention;
[0050] Figure 2 This is a schematic diagram of the structure of an electronic device in Example 3 of the present invention. DETAILED DESCRIPTION
[0051] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0052] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a tumor localization method and system using multimodal imaging.
[0053] Example 1
[0054] The present invention discloses a tumor localization method using multimodal imaging, such as Figure 1 ,include:
[0055] S100. Using a multi-modal imaging device, obtain raw image data of multiple modalities; in S100 of this embodiment, the multi-modal imaging device at least includes a CT imaging device, an MRI imaging device, and a PET imaging device.
[0056] Among them, CT (computed tomography) equipment uses X-rays to perform tomographic scans on the human body and obtains clear images after computer processing. CT (computed tomography) equipment is widely used in the auxiliary diagnosis and positioning of various solid malignant tumors, and can provide key information such as the location, size, adjacent conditions, invasion conditions, and the presence or absence of distant metastasis of the tumor. MRI (magnetic resonance imaging) equipment uses magnetic fields and radio frequency waves to cause the precessing hydrogen nuclei (i.e., protons) in human tissue to nutate and generate signals, which are processed by computers to form images. It is widely used in the positioning of malignant tumors such as the brain, liver, and rectum. PET (positron annihilation imaging) imaging equipment uses the distribution of radionuclide-labeled tracers in the human body for imaging. It is mostly used for the auxiliary diagnosis of thyroid tumors and the auxiliary diagnosis of bone metastasis of malignant tumors. Ultrasound imaging equipment uses the reflection and scattering of ultrasound in the human body to form images. It is widely used for tumor positioning in superficial organs (such as thyroid, breast, etc.) and abdominal organs. Ultrasound imaging has the advantages of being non-invasive, real-time, and low-cost, and is particularly suitable for early screening and follow-up observation. In this embodiment, a multimodal ultrasound examination method is adopted to form a new modality image by integrating or fusing a plurality of ultrasound imaging methods. The plurality of ultrasound imaging methods include at least a high-frequency ultrasound probe, elastic imaging, and ultrasound angiography. The raw image data of the plurality of modalities are preprocessed, specifically including denoising and contrast enhancement of the raw image data.
[0057] In this embodiment, a linear stretching method is used to enhance the contrast of the original image data. The linear stretching method is an image enhancement technology, which is mainly used to adjust the brightness value range of the image to improve the contrast and visual effect of the image. The specific method includes:
[0058] Calculate the minimum pixel value and the maximum pixel value in the original image;
[0059] Apply a linear transformation formula to each pixel value in the original image, the transformation formula being: new pixel value = slope * original pixel value + intercept;
[0060] Adjust the parameters of the linear conversion formula to achieve the desired enhancement effect.
[0061] S300. Register the preprocessed image; image registration is an important step in medical image processing, especially in multimodal imaging (such as CT, MRI, PET, etc.), images of different modalities often have different resolutions, noise levels and contrasts, so they need to be aligned to the same coordinate system through registration for fusion analysis or comparison. Specifically, the image registration algorithm based on mutual information is an image registration method that uses mutual information as a similarity measure, which has high registration accuracy and wide applicability in the field of image registration. In S300 of this embodiment, the preprocessed image is registered using an image registration algorithm based on mutual information, and the specific method includes:
[0062] Extracting features from the preprocessed image, including at least key points, edges or regional features in the image;
[0063] The mutual information between two images is used as a similarity measure, and the transformation parameters that maximize the mutual information are obtained through an optimization algorithm to align the two images;
[0064] The registered images are optimized for registration and the registration results are evaluated.
[0065] In some preferred embodiments, the registration image is subjected to registration optimization, and the specific method includes:
[0066] Initialize registration; provide an initial transformation parameter for the registration process to accelerate the optimization process and improve the registration accuracy;
[0067] Iterative optimization: by iteratively adjusting the transformation parameters, the mutual information or other similarity measures are maximized or minimized;
[0068] Convergence criteria: Set the convergence criteria, including at least the number of iterations and the change in the similarity measure, to determine when to stop iterating.
[0069] S400. The registered multi-modal images are fused to obtain a fused image; specifically, this embodiment adopts a feature pyramid network feature fusion method to fuse the registered multi-modal images. In S400 of this embodiment, the registered multi-modal images are fused, and the specific method includes:
[0070] Starting from the original image, the features of the image are gradually extracted through a series of convolutional layers and pooling layers; the feature map output by each layer gradually reduces in size and increases in semantic information;
[0071] Upsample the highest-level feature map output by the backbone to match its size with the feature map of the previous layer, and then fuse the upsampled feature map with the feature map of the previous layer by element-by-element addition or concatenation;
[0072] Before fusion, a 1x1 convolution operation is performed on the feature map of the bottom-up path to adjust its number of channels to match the feature map of the top-down path, and then the two are fused by element-by-element addition;
[0073] The fused feature map is further processed to eliminate the aliasing effect that may occur during the upsampling process and generate the final feature map.
[0074] This embodiment constructs a multi-scale feature pyramid through bottom-up feature extraction, top-down feature propagation, and horizontally connected feature fusion. This structure enables the network to simultaneously utilize the strong semantic information of high-level features and the rich detail information of low-level features, thereby improving the accuracy and robustness of target detection. FPN has become a standard component in the field of target detection and is widely used in various detection models.
[0075] S500. Perform image recognition on the fused image to locate the tumor. In S500 of this embodiment, perform image recognition on the fused image to locate the tumor. The specific method includes:
[0076] Train the classifier. Use the labeled dataset to train an SVM or random forest classifier, where each candidate region is labeled as "tumor" or "non-tumor." During training, the classifier learns how to map feature vectors to class labels.
[0077] Tumor localization; extract features for each candidate region of the new image and classify them using the trained classifier; based on the classification results, the candidate region marked as "tumor" is considered to be the tumor location; use post-processing steps to optimize the boundaries and shape of the tumor region;
[0078] Result evaluation: Use an image dataset that did not participate in training to evaluate the accuracy of tumor localization. The evaluation indicators include at least accuracy, recall, F1 score and intersection-over-union ratio.
[0079] This embodiment discloses a method and system for tumor localization using multimodal imaging, the method comprising: S100. Using a multimodal imaging device to obtain raw image data of multiple modalities; S200. Preprocessing the raw image data of the multiple modalities; S300. Registering the preprocessed images; S400. Fusing the registered multimodal images to obtain a fused image; S500. Performing image recognition on the fused image to achieve the purpose of tumor localization. The present invention achieves high-precision, non-invasive localization of tumors by integrating the advantages of multiple imaging modalities, solving the problem of insufficient precise localization and functional evaluation of traditional tumor localization methods.
[0080] Example 2
[0081] Based on the multimodal imaging tumor localization method in Example 1, this embodiment discloses a multimodal imaging tumor localization system, including: an original image data acquisition unit, an original image data preprocessing unit, an image registration unit, an image fusion unit and a tumor localization unit; wherein:
[0082] The original image data acquisition unit is used to acquire original image data of multiple modes using a multi-modal imaging device;
[0083] A raw image data preprocessing unit, used for preprocessing the raw image data of the multiple modalities;
[0084] An image registration unit, used for registering the preprocessed image;
[0085] An image fusion unit, used for fusing the registered multi-modal images to obtain a fused image;
[0086] The tumor localization unit is used to perform image recognition on the fused image to achieve the purpose of localizing the tumor.
[0087] Among them, the specific working methods of the original image data acquisition unit, the original image data preprocessing unit, the image registration unit, the image fusion unit and the tumor localization unit have been described in detail in Example 1, and will not be repeated in this embodiment.
[0088] Example 3
[0089] Based on the same inventive concept, an embodiment of the present disclosure also provides an electronic device. Figure 2 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Figure 2As shown, an embodiment of the present disclosure provides an electronic device including: one or more processors 101, a memory 102, and one or more I / 0 interfaces 103. The memory 102 stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement any optimization method in the above embodiments; the one or more I / 0 interfaces 103 are connected between the processor and the memory, and are configured to implement information interaction between the processor and the memory.
[0090] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU), etc.; the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus), etc.
[0091] In some embodiments, the processor 101, the memory 102, and the I / O interface 103 are connected to each other via a bus 104, and further connected to other components of the computing device.
[0092] In some embodiments, the one or more processors 101 include a field programmable gate array.
[0093] According to an embodiment of the present disclosure, a computer-readable medium is further provided, wherein a computer program is stored on the computer-readable medium, wherein when the program is executed by a processor, the steps in any optimization method in the above-mentioned embodiment are implemented.
[0094] Example 4
[0095] Compared with the first embodiment, the only difference is that S200. preprocessing the raw image data of the multiple modalities specifically includes:
[0096] S200.1 Determine the reference image: Based on the pre-set boundary clarity principle, select the image of the modality with the clearest boundary as the reference image. This usually means selecting an image with the highest resolution and the best soft tissue contrast, such as a high-resolution MRI or CT image.
[0097] S200.2 Modality Correlation Analysis: Use the known correlations between modalities, such as the similarities in anatomical structures between CT and MRI, to predict images of other modalities. This can be achieved by analyzing the imaging principles of each modality and their performance in tumor detection.
[0098] S200.3 Predicted Image Generation: Generate predicted images based on the reference images and the results of the modality correlation analysis. This may involve using image processing techniques such as image interpolation, image enhancement, or machine learning models to predict features of other modality images.
[0099] S200.4 Compare predicted image with actual image: Compare the predicted image with the actual acquired image of the other modality. This can be achieved by calculating the difference between the predicted image and the actual image, such as using mean square error (MSE), structural similarity index (SSIM), or other image similarity measurement methods.
[0100] S200.5 Preliminary suspected area determination: Based on the comparison results, preliminary suspected tumor areas are quickly identified. These areas may be areas that show abnormalities in multiple modality images, such as contrast enhancement or morphological abnormalities.
[0101] S200.6 Denoising: Perform denoising on all modalities to reduce random noise in the image and improve image quality. This can be achieved by using filters such as Gaussian filtering, median filtering, or more advanced denoising algorithms such as non-local means denoising (NLM).
[0102] S200.7 Contrast Enhancement: The original image data is enhanced by linear stretching method. First, the minimum and maximum pixel values in the original image are calculated, and then the linear transformation formula is applied to each pixel value: new pixel value = slope * original pixel value + intercept; the parameters of slope and intercept are adjusted to achieve the goal of enhancing the contrast between the tumor area and the surrounding normal tissue.
[0103] S200.8 parameter optimization: Determine the best linear transformation parameters through experiments or algorithm optimization (obtained through historical training) to achieve the best boundary clarity and contrast enhancement effect (facilitating S300 registration and S400 fusion, making the boundary of the fused image clearer).
[0104] Through the above steps, the preprocessing of S200 not only improves the quality of image data, but also lays the foundation for the rapid identification of suspected tumor areas and provides high-quality input data for subsequent image registration, fusion and recognition steps.
[0105] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.
[0106] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
[0107] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein can all be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above around their functions. Whether such functions are implemented as hardware or software depends on specific applications and the design constraints imposed on the entire system. A skilled person can implement the described functions in an alternative manner for each specific application, but such implementation decisions should not be interpreted as departing from the scope of protection of the present disclosure.
[0108] The method described in conjunction with the embodiments of this paper or the step of the algorithm can be directly embodied as hardware, a software module or its combination executed by a processor. The software module can be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a mobile disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. The ASIC can be located in a user terminal. Of course, the processor and the storage medium can also be present in a user terminal as discrete components.
[0109] For software implementation, the techniques described in this application can be implemented with modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes can be stored in a memory unit and executed by a processor. The memory unit can be implemented within the processor or outside the processor. In the latter case, it is coupled to the processor in a communication manner via various means, which are well known in the art.
[0110] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".
Claims
1. A tumor localization method using multimodal imaging, characterized in that: include: S100. Using a multi-modal imaging device, obtaining raw image data of multiple modalities; S200. Preprocessing the raw image data of the multiple modalities; S300. Registering the preprocessed image; S400. Fusing the registered multi-modality images to obtain a fused image; S500. Perform image recognition on the fused image to locate the tumor.
2. The multimodal imaging tumor localization method according to claim 1, characterized in that: In S100, the multimodal imaging device at least includes a CT imaging device, an MRI imaging device, a PET imaging device, and an ultrasound imaging device.
3. The tumor localization method of multimodal imaging according to claim 1, characterized in that: In S100, a multimodal ultrasound examination method is used to form a new modality image by integrating or fusing multiple ultrasound imaging methods. The multiple ultrasound imaging methods at least include high-frequency ultrasound probe, elastic imaging, and ultrasound angiography.
4. The multimodal imaging tumor localization method according to claim 3, characterized in that: In S200, the raw image data of the plurality of modalities are preprocessed, specifically including denoising and contrast enhancement of the raw image data; and the raw image data is enhanced contrast by using a linear stretching method, specifically including: Calculate the minimum pixel value and the maximum pixel value in the original image; Apply a linear transformation formula to each pixel value in the original image, the transformation formula being: new pixel value = slope * original pixel value + intercept; Adjust the parameters of the linear conversion formula to achieve the desired enhancement effect.
5. The multimodal imaging tumor localization method according to claim 1, characterized in that: In S300, the pre-processed images are registered using an image registration algorithm based on mutual information, and the specific method includes: Extracting features from the preprocessed image, including at least key points, edges or regional features in the image; The mutual information between two images is used as a similarity measure, and the transformation parameters that maximize the mutual information are obtained through an optimization algorithm to align the two images; The registered images are optimized for registration and the registration results are evaluated.
6. The multimodal imaging tumor localization method according to claim 5, characterized in that: The registration image is optimized for registration, and the specific method includes: Initialize registration; provide an initial transformation parameter for the registration process to accelerate the optimization process and improve the registration accuracy; Iterative optimization: by iteratively adjusting the transformation parameters, the mutual information or other similarity measures are maximized or minimized; Convergence criteria: Set the convergence criteria, including at least the number of iterations and the change in the similarity measure, to determine when to stop iterating.
7. The multimodal imaging tumor localization method according to claim 1, characterized in that: In S400, the registered multiple modality images are fused, and the specific method includes: Starting from the original image, the features of the image are gradually extracted through a series of convolutional layers and pooling layers; the feature map output by each layer gradually reduces in size and increases in semantic information; Upsample the highest-level feature map output by the backbone to match its size with the feature map of the previous layer, and then fuse the upsampled feature map with the feature map of the previous layer by element-by-element addition or concatenation; Before fusion, a 1x1 convolution operation is performed on the feature map of the bottom-up path to adjust its number of channels to match the feature map of the top-down path, and then the two are fused by element-by-element addition; The fused feature map is further processed to eliminate the aliasing effect that may occur during the upsampling process and generate the final feature map.
8. The multimodal imaging tumor localization method according to claim 1, characterized in that: In S500, image recognition is performed on the fused image to locate the tumor. The specific method includes: Train the classifier. Use the labeled dataset to train an SVM or random forest classifier, where each candidate region is labeled as "tumor" or "non-tumor." During training, the classifier learns how to map feature vectors to class labels. Tumor localization; extract features for each candidate region of the new image and classify them using the trained classifier; based on the classification results, the candidate region marked as "tumor" is considered to be the tumor location; use post-processing steps to optimize the boundaries and shape of the tumor region; Result evaluation: Use an image dataset that did not participate in training to evaluate the accuracy of tumor localization. The evaluation indicators include at least accuracy, recall, F1 score and intersection-over-union ratio.
9. A multimodal imaging tumor localization system, characterized in that: include: Original image data acquisition unit, original image data preprocessing unit, image registration unit, image fusion unit and tumor localization unit; wherein: The original image data acquisition unit is used to acquire original image data of multiple modes using a multi-modal imaging device; A raw image data preprocessing unit, used for preprocessing the raw image data of the multiple modalities; An image registration unit, used for registering the preprocessed image; An image fusion unit, used for fusing the registered multi-modal images to obtain a fused image; The tumor localization unit is used to perform image recognition on the fused image to achieve the purpose of localizing the tumor.
10. An electronic device comprising: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the tumor localization method of claims 1-8.
Citation Information
Patent Citations
Multimodal medical image fusion method and system based on nonlinear least square
CN106683042A
Intelligent and automatic delineation method for gross tumor volume and organs at risk
CN107403201A
CNN and feature pyramid-based traffic prohibited product identification method
CN110097145A
Multi-image fusion tumor three-dimensional position accurate positioning system
CN110660063A
Kidney tumor ultrasonic video classification method and device, equipment and storage medium
CN116740609A
Cited By
Multi-modal segmentation method and system for brain tumor imaging
CN120070480A
A multi-modal segmentation method and system for brain tumor imaging
CN120070480B
Liver and gall tumor three-dimensional positioning method and system based on detection data and medium
CN120147407A
A hepatobiliary tumor three-dimensional positioning method and system based on detection data and a medium
CN120147407B
Near-infrared two-region fluorescence-magnetic particle-CT (Computed Tomography) three-mode fusion imaging method
CN120876566A
