Photodynamic therapy imaging positioning method and system

Abstract

The present invention discloses a method and system for photodynamic therapy imaging positioning, which belongs to the field of image processing technology. It includes an image acquisition module, an image preprocessing module, an image fusion module, and a lesion identification and positioning module that are interconnected. The present invention provides more comprehensive and intuitive physiological information by preprocessing, fusing, and registering different types of images, reducing the difficulty of doctors switching between different images and imagining spatial structural relationships during diagnosis. After image preprocessing, the image specifications are unified, eliminating the heterogeneity of multimodal images, effectively improving the accuracy and real-time performance of multimodal image registration, and improving the accuracy of photodynamic therapy imaging positioning.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a photodynamic therapy imaging positioning method and system. Background Art

[0002] Photodynamic therapy (PDT) is a targeted therapy that uses the combined effects of photosensitizers, light irradiation of specific wavelengths, and oxygen molecules to eliminate diseased cells (such as tumors) or microorganisms. During PDT, patients undergo multiple tests, which produce a variety of images, including fluorescence images, ultrasound images, CT (computed tomography) images, and MRI (magnetic resonance imaging). Physicians use the information provided by these different types of images to comprehensively determine characteristic features such as the location, size, and boundaries of the lesion. However, because different types of images often differ significantly, physicians must manually switch images and rely on experience to visualize the three-dimensional structure. Furthermore, fused images often lose key information (such as fluorescence signals being overwritten by grayscale), resulting in ineffective auxiliary analysis and difficulty in interpreting the provided visual information.

[0003] Imaging positioning technology achieves accurate calibration of the spatial position, boundaries and functional characteristics of diseased tissues through the acquisition, registration, fusion and visualization of multimodal medical images, providing medical staff with visual information to assist them in analyzing and judging the condition, and to a certain extent improving the accuracy of their analysis and judgment.

[0004] However, in the actual application of PDT, imaging positioning technology has large differences in resolution, imaging principles, and acquisition time among different imaging devices (such as CT, MRI, and fluorescence), resulting in errors in registration (such as ±1-2mm). In addition, the patient's breathing and body movement during surgery may cause the pre-fusion image to not match the actual anatomical position, resulting in insufficient accuracy and real-time performance of multimodal image registration, thereby affecting the accuracy of medical staff's analysis and judgment. Summary of the Invention

[0005] In order to solve the problems existing in the above background technology, the present invention adopts the following technical solutions:

[0006] The present application provides a photodynamic therapy imaging positioning system, comprising: an image acquisition module and an image preprocessing module that are communicatively connected to each other;

[0007] The image acquisition module is configured to collect fluorescence images, CT images, and MRI images of the same patient from different devices, detect whether the collected image format is DICOM format, and if not, perform format conversion and metadata integration on the image, unifying all images from different devices into DICOM format and generating standardized DICOM format images;

[0008] The image preprocessing module is configured to preprocess the standardized DICOM format images generated after the image acquisition module unifies the format, and generate a pre-preprocessing result image; the preprocessing includes pre-preprocessing, and the pre-preprocessing includes pseudo-color processing of the fluorescence image using a grayscale-color conversion method and noise reduction processing of the CT image and the MRI image using a median filter method to obtain the pre-preprocessing result image; including:

[0009] The grayscale-color transformation method is used to perform pseudo-color processing on fluorescence images. The method involves extracting the grayscale values ​​of each region of the fluorescence image and mapping the grayscale values ​​of the pixels of the fluorescence image to different colors in a hierarchical manner to generate a pseudo-color fluorescence image. The mapping relationship is: low grayscale is mapped to blue, medium grayscale is mapped to green, and high grayscale is mapped to red.

[0010] The CT and MRI images are subjected to noise reduction processing using a median filter method. The method includes dividing the CT and MRI images into several regions, counting the grayscale values ​​of the pixels in each region, sorting the pixel values ​​of each point from small to large grayscale values, selecting the middle value of the grayscale value sorting as the final calculation result of the filtering, and setting the final calculation result of the filtering as the pixel value at the center of the region to generate an enhanced CT / MRI image.

[0011] Normalized brightness and size adjustment are performed on pseudo-color fluorescence images and enhanced CT / MRI images to ensure consistency of different image scales and generate pre-processing result images.

[0012] Furthermore, the image acquisition module includes an image data receiving unit, a format detection and conversion unit, and a metadata integration unit, wherein:

[0013] The image data receiving unit is connected to the fluorescence imaging device, CT and MRI imaging devices and receives image data from these devices;

[0014] The format detection and conversion unit performs format detection on the image data received by the image data receiving unit and uniformly converts all detected image data into the DICOM format; specifically, it includes:

[0015] Parse the file header of the received image data, determine the format of the file header, and classify the image data into DICOM format and non-DICOM format according to the format of the file header;

[0016] If the detected format is DICOM, the metadata in the image data is extracted and entered into the metadata integration unit;

[0017] If the detected format is non-DICOM, the non-DICOM format is converted and metadata is injected into the metadata integration unit.

[0018] Furthermore, the metadata integration unit performs metadata integration, data verification and storage on each compliant data, verifies the Study UID / Patient ID of the image data after metadata integration, ensures that the required tags are not empty to complete the integrity check, and the CT / MRI / fluorescence images of the same patient need to share the same Study UID to complete the consistency verification, and the data that pass the integrity check and consistency verification are uniformly stored in the PACS system.

[0019] Furthermore, the image acquisition module further includes an exception handling unit, which is configured to perform exception measures when detecting an abnormality in the image data received by the image data receiving unit, wherein the abnormality includes the abnormality of missing non-DICOM metadata and inconsistent Study UIDs of multiple devices, including:

[0020] When it is detected that non-DICOM metadata is missing in the image data received by the image data receiving unit, the format detection and conversion unit enters the HIS system API to query and complete the missing metadata in real time;

[0021] When it is detected that the image data received by the meta-image data receiving unit has inconsistent Study UIDs of multiple devices, the metadata integration unit enters and forcibly unifies the Study UIDs of multiple devices into the Study UID of the first inspection.

[0022] Furthermore, the normalized brightness processing includes:

[0023] Step 1: Extract the target channel: For pseudo-color fluorescence images, extract the green channel; for enhanced CT / MRI images, directly use the original grayscale data;

[0024] Step 2: Modality-specific normalization: The green channel of the pseudo-color fluorescence image is linearly normalized to [0, 1]. For enhanced CT images, the images are first truncated to the typical window width and then normalized to [0, 1]. For enhanced MRI images, quantile-based normalization is performed to [0, 1].

[0025] Step 3: Verify the brightness distribution: Check whether the peak of each modal histogram is within the target range. If the brightness distribution of a modal deviates from the expected range, trigger recalibration and repeat steps 1-3.

[0026] Furthermore, the size adjustment process includes:

[0027] Step 1: Determine the baseline resolution, read the pixel spacing from the DICOM tag, and select the minimum spacing as the baseline;

[0028] Step 2: resampling calculation, upsampling from low-resolution image to high-resolution;

[0029] Step 3: Verify spatial alignment. Check the image position tag and image orientation tag in the DICOM tag to ensure that the physical coordinates are consistent. Calculate the feature point matching error between the resampled image and the original image.

[0030] Step 4: Edge filling processing: fill the modality-specific background value in the area beyond the boundary after upsampling.

[0031] Furthermore, the image preprocessing module further includes image segmentation, and the image segmentation configuration is configured to perform threshold segmentation on the pre-preprocessing result image, outline the boundary of the target area in the image, and remove image information outside the area, including:

[0032] Image type detection: detect whether the pre-processed image is a fluorescence image or a CT / MRI image and calculate the optimal segmentation threshold, specifically:

[0033] If the image detected is a fluorescent image, the green channel after pseudo-color processing is extracted, and the optimal segmentation threshold is calculated using the Otsu method automatic thresholding;

[0034] If the detected image is a CT / MRI image, the grayscale image after noise reduction is directly used, and the adaptive threshold is used to calculate the optimal segmentation threshold;

[0035] Generate a binary mask and divide the grayscale of the entire image into two parts through the optimal segmentation threshold, so that the grayscale value difference between the two parts is maximized and the difference within each part is minimized to achieve boundary delineation.

[0036] Furthermore, it also includes an image fusion module that is communicatively connected to the image acquisition module and the image preprocessing module. The image fusion module is configured to perform multimodal registration and optimize the registration results according to the areas segmented by the image preprocessing module, and output a fused multimodal comprehensive view.

[0037] Furthermore, the optimization of the registration result and output of the fused multimodal comprehensive view include:

[0038] Use mask to constrain the fusion area, and use the binary mask provided by the image preprocessing module to limit the fusion range;

[0039] The grayscale image of the fused registration result is combined with the pseudo-color fluorescence image to generate a color fused image. Specifically, red represents the high fluorescence area, green represents the transition area, and the gray background represents the anatomical structure of CT / MRI;

[0040] Output standardized image data, which is a fused multimodal comprehensive view including DICOM fused images and quantitative reports.

[0041] Furthermore, it also includes a lesion identification and positioning module that is communicatively connected to the image acquisition module and the image preprocessing module, and is configured to accurately mark the location, size and boundary of the lesion based on the fused multimodal comprehensive view of the image fusion module, and display it on the screen in an intuitive manner.

[0042] The present application also provides a photodynamic therapy imaging positioning method, which is applied to the aforementioned photodynamic therapy imaging positioning system, and the method comprises the following steps:

[0043] Collect fluorescence images, CT images, and MRI images of the same patient from different devices, detect whether the collected image format is DICOM format, and if not, perform format conversion and metadata integration on the image, unify all images from different devices into DICOM format, and generate standardized DICOM format images;

[0044] Preprocess the standardized DICOM format images generated after the unified format to generate pre-processing result images;

[0045] Perform multimodal registration and registration result optimization on the areas segmented by the image preprocessing module, and output a fused multimodal comprehensive view;

[0046] The location, size and boundary of the lesion are marked based on the fused multimodal comprehensive view and displayed on the screen in an intuitive manner.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention generates standardized DICOM format images through the image acquisition module, eliminating the heterogeneity of multi-device data, providing unified input for subsequent fusion / analysis, and achieving standardized output; automated processing reduces manual intervention and significantly improves processing efficiency; at the same time, it supports the access of new device types (such as PET) through plug-in interfaces, improving the scalability of the system, thereby further improving the accuracy of photodynamic therapy imaging positioning.

[0049] The image preprocessing module of the present invention reduces noise and redundant information through noise reduction and segmentation, improves the visual / mathematical separability of lesions and backgrounds in images of different modalities (fluorescence / CT / MRI), eliminates the interference of inherent noise of the equipment on the analysis, provides structured input data for subsequent intelligent analysis, and improves the accuracy of the entire photodynamic therapy imaging and positioning system.

[0050] The image fusion module of the present invention relies on the output of the image preprocessing module, which has undergone enhancement and noise reduction, brightness / size consistency processing, and segmentation processing or obtained. This high-quality image data input enables the image fusion module to achieve precise registration. Furthermore, the image fusion module uses DCT-CG and Toet as registration algorithms to further correct spatial deviations in the image data. This, in conjunction with the image preprocessing module, addresses core issues in medical imaging, such as noise, registration errors, and blurred boundaries, laying a solid foundation for lesion identification and localization.

[0051] The present invention provides more comprehensive and intuitive physiological information through preprocessing, fusion and registration of different types of images, reducing the difficulty for doctors to switch between different images and imagine spatial structural relationships during diagnosis. After image preprocessing, the image specifications are unified, eliminating the heterogeneity of multimodal images, effectively improving the accuracy and real-time performance of multimodal image registration, and improving the accuracy of photodynamic therapy imaging positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A schematic structural diagram of a photodynamic therapy imaging positioning system provided by one embodiment of the present invention;

[0054] Figure 2 A schematic structural diagram of an image acquisition module in a photodynamic therapy imaging and positioning system according to an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of the process flow of an image acquisition module in a photodynamic therapy imaging positioning system provided by one embodiment of the present invention;

[0056] Figure 4 A schematic diagram of the flow of threshold segmentation in a photodynamic therapy imaging positioning system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0058] Example 1

[0059] See also Figure 1 The present invention provides a photodynamic therapy imaging positioning system, which includes an image acquisition module, an image preprocessing module, an image fusion module and a lesion recognition and positioning module that are communicatively connected to each other.

[0060] Among them, the image acquisition module is configured to collect fluorescence images, CT images and MRI images of the same patient from different devices, detect whether the collected image format is DICOM format, and if not, convert the image format and integrate metadata, unify all images from different devices into DICOM format, and generate standardized DICOM format images.

[0061] In photodynamic therapy, medical staff use multimodal imaging sources for lesions, including fluorescence images, CT images, and MRI images. Fluorescence images can provide the distribution of photosensitizers, clearly identify the location of photosensitizer concentration in the lesion area, and determine the lesion boundaries. CT images clearly display high-density tissues such as bones and lungs. MRI images have strong resolution for soft tissues such as muscles, spinal cord, and nerves.

[0062] DICOM (Digital Imaging Format) is an international standard format in the field of medical imaging. It contains image data (image content) and metadata (patient information, examination parameters, etc.). It unifies the format and communication protocol of medical images, ensuring that images generated by equipment from different manufacturers (such as CT, ultrasound, etc.) can be seamlessly exchanged across systems.

[0063] Metadata refers to information that describes the attributes of image data. In the photodynamic therapy imaging positioning system, it mainly includes identity binding: associating patient ID, examination time, etc. with pixel data to avoid confusion (such as mixing CT images of different patients); process navigation: providing key parameters for subsequent fusion / registration (such as CT layer thickness, excitation wavelength of fluorescence images, etc.); compliance assurance: meeting the mandatory requirements of the DICOM standard for necessary tags (such as Modality and Study UID).

[0064] Standardized DICOM format images include fluorescence images, CT images, and MRI images that are unified in DICOM format.

[0065] like Figure 2 As shown, the image acquisition module includes an image data receiving unit, a format detection and conversion unit, and a metadata integration unit. Figure 3 As shown in the figure, the specific working process of the image acquisition module is as follows:

[0066] The image data receiving unit receives images: It connects to various medical imaging devices, such as fluorescence imaging devices, CT scans, and MRI scans, and receives image data from these devices. Fluorescence imaging devices (such as fluorescence endoscopes / microscopes) typically output pseudo-color or grayscale fluorescence images (common formats: JPEG and TIFF). CT / MRI devices typically output directly in DICOM format, but some older devices may use NIFTI or RAW formats.

[0067] The format detection and conversion unit performs format detection on the image data received by the image data receiving unit and converts all detected image data into DICOM format. Specifically, it includes:

[0068] Received image data is parsed for file headers to determine the format. Based on the format, the image data is classified as either DICOM or non-DICOM. Generally, if the file header contains the "DICM" identifier, it is considered DICOM; otherwise, it is considered non-DICOM. Non-DICOM files may be stored with extensions such as .jpg or .nii (NIFTI), but further content verification is required. If the format is unrecognizable, a warning is triggered, a log is recorded, and manual intervention is required.

[0069] If the detected format is DICOM, the metadata in the image data is extracted and entered into the metadata integration unit.

[0070] If the detection format is non-DICOM, it will be converted to the non-DICOM format and metadata will be injected into the metadata integration unit, including pixel data extraction. Taking JPEG fluorescence images as an example, a library (such as OpenCV) is used to read the JPEG pixel matrix, retaining the original grayscale or RGB values; metadata injection is to construct DICOM metadata, including required tags (such as PatientID, patient identifier) ​​and custom tags; generate a DICOM file, and use libraries such as pydicom to encapsulate the pixel data and metadata into a .dcm file.

[0071] The metadata integration unit integrates, verifies, and stores the aforementioned compliant data. It verifies the Study UID / Patient ID of the image data after metadata integration, ensuring that required tags are not empty to complete the integrity check. CT / MRI / fluorescence images of the same patient must share the same Study UID (Study Unique Identifier, e.g., 0020,000D) for consistency verification. Data that passes the integrity check and consistency verification is stored uniformly in the Picture Archiving and Communication System (PACS).

[0072] The present invention generates standardized DICOM format images through the above-mentioned image acquisition module, eliminates the heterogeneity of multi-device data, provides unified input for subsequent fusion / analysis, and realizes standardized output; automated processing reduces manual intervention and significantly improves processing efficiency; at the same time, it supports the access of new device types (such as PET) through plug-in interfaces, improves the scalability of the system, and thus further improves the accuracy of photodynamic therapy imaging positioning.

[0073] Preferably, in other embodiments of the present invention, the image acquisition module further includes an exception handling unit configured to perform exception handling upon detecting an abnormality in the image data received by the image data receiving unit. The abnormality includes, for example, missing non-DICOM metadata and inconsistent Study UIDs across multiple devices. For the aforementioned abnormalities, the present invention proposes, but is not limited to, the following solutions:

[0074] Non-DICOM metadata loss includes image files that do not conform to the DICOM standard (such as JPEG / PNG / AVI), which lack key descriptive information such as patient ID and examination time.

[0075] When the image data receiving unit detects missing non-DICOM metadata, the format detection and conversion unit accesses the HIS system API in real time to complete the missing metadata. The HIS system, a hospital information system, stores core data such as patient registration and examinations, including patient registration information (such as ID and name) and examination appointment records (time, department, and equipment). Real-time API (Application Programming Interface) queries automatically retrieve data from the HIS system through programming interfaces.

[0076] The present invention uses real-time query and completion through the HIS system API. Even images generated by non-DICOM devices can be seamlessly integrated into the medical information system, ensuring that "every image is traceable and every inspection is compliant."

[0077] When it is detected that the image data received by the meta-image data receiving unit has inconsistent Study UIDs of multiple devices, the metadata integration unit enters and forcibly unifies the Study UIDs of multiple devices into the Study UID of the first inspection.

[0078] The present invention can further improve the quality of data collected by the image acquisition module through the above-mentioned exception processing unit, provide an accurate data source for subsequent process processing, and improve the accuracy of the imaging positioning system.

[0079] The image preprocessing module is configured to preprocess the standardized DICOM format images generated after the image acquisition module unifies the format, and generate pre-processing result images. The preprocessing includes pre-processing and image segmentation.

[0080] The pre-processing includes performing pseudo-color processing on the fluorescence image using the grayscale-color conversion method and performing noise reduction processing on the CT image and the MRI image using the median filter method to obtain the pre-processing result image. Specifically:

[0081] First, pseudo-color processing of fluorescence images using grayscale-to-color conversion involves extracting the grayscale values ​​of each region of the fluorescence image and mapping the pixel grayscale values ​​to different colors in a hierarchical manner to generate a pseudo-color fluorescence image. The mapping relationship is as follows: low grayscale is mapped to blue, medium grayscale is mapped to green, and high grayscale is mapped to red. Typically, high grayscale corresponds to the core of the lesion (e.g., the tumor core, where photosensitizer is most enriched), with the strongest fluorescence signal (mapped to the red channel), but may result in blurred boundaries due to oversaturation. Medium grayscale corresponds to the edge of the lesion: where the photosensitizer concentration gradually decreases at the interface between the lesion and normal tissue, the fluorescence signal is moderate (mapped to the green channel), forming a clear transition zone. Low grayscale corresponds to the background area of ​​the lesion: normal tissue has almost no photosensitizer uptake (mapped to the blue channel).

[0082] The green channel (medium gray area) is usually associated with the edge of the lesion because the design of the green channel (medium gray area) corresponding to the edge of the lesion is essentially a combination of biological characteristics (gradient distribution of photosensitizers, for example, studies have shown that the fluorescence intensity at the front of tumor infiltration is usually 30%-70% of the maximum value, which corresponds exactly to the medium gray range (green channel) of pseudo-color processing), human perception (visual optimization, for example, a multi-center survey (n>=200) showed that the accuracy of physicians using green markers in judging tumor boundaries is 22% higher than that of using a single gray scale; the decision time is shortened by 35% (due to reduced visual fatigue)) and computational requirements. The goal is to achieve a triple coupling (feature extraction, such as the most significant gradient change (maximum first-order derivative) in the mid-grayscale region, less affected by uneven illumination than high / low grayscale regions, and maximization of inter-class variance, which typically occurs in the mid-grayscale region, the optimal range for Otsu's method, and high grayscale (red) is easily saturated, low grayscale (blue) is easily submerged by noise, and mid-grayscale (green) retains the most effective information). This mapping relationship can improve diagnostic accuracy by clearly defining the "resectable boundary"; optimize algorithm performance by providing ideal input for registration / segmentation; and conform to clinical practices by being highly consistent with physicians' cognitive patterns.

[0083] Fluorescence images are typically grayscale data, resulting in low contrast and unclear lesion boundaries when viewed directly. The preprocessing module uses a grayscale-to-color transformation method to map different grayscale intervals to different colors (specifically, low grayscale → blue, medium grayscale → green, and high grayscale → red). This grayscale-to-color transformation method performs pseudo-color processing to convert fluorescence intensity differences into color differences. This color difference highlights the lesion area, enhancing image contrast and making the size and location of the lesion easier to distinguish. This provides a high-contrast color channel for subsequent image segmentation. Simultaneously, the grayscale-to-color transformation method optimizes feature extraction, mapping the green channel (medium grayscale area) to the lesion edge. This pseudo-color processing facilitates lesion localization and subsequent image fusion.

[0084] Secondly, CT and MRI images may be subject to interference during the image acquisition process (such as random black and white spots caused by sensor failure or transmission interference, often referred to as salt and pepper noise). To address this issue, the present invention uses a median filter method to reduce noise on CT and MRI images. This method involves dividing the CT and MRI images into several regions, statistically analyzing the grayscale values ​​of the pixels in each region, sorting the pixel values ​​of each point from smallest to largest grayscale value, selecting the median value of the grayscale value sort as the final filter calculation result, and setting the final filter calculation result as the pixel value at the center of the region to generate an enhanced CT / MRI image. For example:

[0085] Original pixel values: [120, 125, 255 (noise), 130, 122],

[0086] After sorting: [120, 122, 125, 130, 255],

[0087] Median result: 125.

[0088] Compared with the mean filtering method, the present invention can effectively filter out interference (such as salt and pepper noise) through the above-mentioned median filtering method, and can better protect the boundaries of high-contrast structures such as bones and soft tissues. The grayscale distribution of the filtered image is more concentrated, making the subsequent adaptive threshold effect more stable, avoiding threshold offset caused by noise (such as noise may be misjudged as bones when not filtered).

[0089] Preferably, the pre-processing further includes normalizing the brightness and resizing the pseudo-color fluorescence image and the enhanced CT / MRI image to ensure that the scales of different images are consistent, thereby generating a pre-processing result image. Specifically:

[0090] Normalizing brightness refers to mapping pixel values ​​of different modalities to a unified range (such as [0, 1]), eliminating brightness differences between devices while preserving the relative contrast between the lesion and the background. This includes:

[0091] Step 1: Extract the target channel: For pseudo-color fluorescence images, extract the green channel (lesion edge area) because it has the best contrast; for enhanced CT / MRI images, directly use the original grayscale data.

[0092] Step 2: Modality-specific normalization:

[0093] The green channel of the pseudo-color fluorescence image was linearly normalized to [0, 1];

[0094] For enhanced CT images (Hounsfield (HU) units), they were first truncated to a typical window width (e.g., [-1000, 1000]) and then normalized to [0, 1];

[0095] The enhanced MRI image (signal intensity) is processed to [0, 1] based on quantile normalization (excluding extreme values). For example, the quantile normalization formula can be:

[0096] ,in I norm is the normalized value, P 1% and P 99% are the 1% and 99% quantiles of the image.

[0097] Step 3: Verify brightness distribution:

[0098] Check whether the peak of each modal histogram is within the target range (e.g., [0.3, 0.7]). If the brightness distribution of a modality deviates from the expected value (e.g., CT is all negative), trigger recalibration and repeat steps 1-3.

[0099] Resizing involves aligning the spatial resolution of all images to the highest resolution modality (usually fluorescence) to ensure pixel-level registration accuracy. This includes:

[0100] Step 1: Determine the baseline resolution. Read the pixel spacing from the DICOM tag and select the minimum spacing as the baseline (e.g., 0.1 mm / pixel for fluorescence images).

[0101] Step 2: Resampling calculation, upsampling from low-resolution images (CT / MRI) to high-resolution (fluorescence), for example, can be performed using cubic spline interpolation (scipy.ndimage.zoom), as expressed as:

[0102] , where represents the scaling factor, represents the spacing of CT images, Indicates the spacing of the fluorescence image.

[0103] Example: CT original resolution 0.5 mm / pixel → target 0.1 mm / pixel, scaling factor is 5.

[0104] Step 3: Verify spatial alignment. Check the Image Position Patient and Image Orientation Patient tags in the DICOM tag to ensure that the physical coordinates are consistent. Calculate the matching error between the feature points (e.g., SIFT) of the resampled image and the original image, with a threshold of <0.5 mm.

[0105] Step 4: Edge filling processing: For the areas beyond the boundaries after upsampling, the modality-specific background values ​​are filled, including: CT: -1000 (air HU value), MRI: 0 (background signal), fluorescence: 0 (no fluorescence).

[0106] The pre-processed image generated after the above-mentioned normalized brightness and size adjustment processing is of great clinical significance. It can effectively eliminate equipment differences and make images from different hospitals and different models of equipment comparable. After the brightness / size is consistent, the registration error can be reduced to less than 0.3 mm, effectively improving the fusion accuracy and standardizing the AI ​​input, so that the deep learning model does not need to be trained separately for different modalities.

[0107] Based on this, the present invention uses the above-mentioned normalized brightness and size adjustment process, and the system ultimately outputs a pre-processed image with uniform scale and controllable quality, providing a reliable basis for subsequent fusion and diagnostic analysis.

[0108] The image segmentation configuration is to perform threshold segmentation on the pre-processing result image, outline the boundary of the target area in the image, and remove the image information outside the area. Figure 4 As shown, specifically, the working process of the threshold segmentation includes:

[0109] Image type detection: detect whether the pre-processed image is a fluorescence image or a CT / MRI image and calculate the optimal segmentation threshold, specifically:

[0110] If the image is detected as a fluorescent image, the green channel (i.e., the mid-grayscale mapping area) after pseudocolor processing is extracted and the optimal segmentation threshold is calculated. During pseudocolor processing, the mid-grayscale (lesion transition area) is mapped to green, and this channel is most effective for distinguishing lesions from normal tissue. The optimal segmentation threshold is calculated using the Otsu method for automatic threshold calculation. After calculating the threshold, the inter-class variance of each threshold is calculated, and the threshold with the largest inter-class variance is determined as the optimal threshold T.

[0111] If the image is detected as a CT / MRI image, the optimal segmentation threshold is calculated directly using the grayscale image after noise reduction. Different regions of a CT / MRI image may have uneven brightness (for example, lungs vs. bones in a CT image), making global thresholding ineffective. Therefore, adaptive thresholding is used to calculate the optimal segmentation threshold for CT / MRI images.

[0112] Generate a binary mask and divide the grayscale of the entire image into two parts through the optimal segmentation threshold, so that the grayscale value difference between the two parts is maximized and the difference within each part is minimized, thereby achieving boundary delineation.

[0113] The above-mentioned image segmentation of the present invention uses threshold segmentation to separate the target area based on grayscale difference, and finally outputs accurate boundary information, laying the foundation for subsequent fusion and quantitative analysis.

[0114] The image preprocessing module of the present invention reduces noise and redundant information through noise reduction and segmentation, improves the visual / mathematical separability of lesions and backgrounds in images of different modalities (fluorescence / CT / MRI), eliminates the interference of inherent noise of the equipment on the analysis, provides structured input data for subsequent intelligent analysis, and improves the accuracy of the entire photodynamic therapy imaging and positioning system.

[0115] Among them, the image fusion module is configured to perform multimodal registration and optimize the registration results according to the areas segmented by the image preprocessing module, and output a fused multimodal comprehensive view.

[0116] First, the multimodal registration includes using the DCT-CG algorithm to fuse the CT image and the MRI image to obtain a fused image A, and using the Toet algorithm to fuse the fused image A and the fluorescence image B based on the fused image A to obtain the final fused image Z as the registration result.

[0117] The existing method for fusing CT and MRI images is DCT-AC, but this method has unavoidable block effect characteristics. To address this block effect problem, the present invention proposes a DCT-CG algorithm to fuse CT and MRI images. Although CT and MRI display different tissues (bone vs. soft tissue), their anatomical structures are consistent. The specific formula of the DCT-CG algorithm is based on conventional formulas in the field and will not be described in detail in this invention. The DCT-CG algorithm fusion module aligns and fuses the CT and MRI images through the following steps:

[0118] First, frequency domain feature extraction is performed: discrete cosine transform (DCT) is performed on the CT and MRI image blocks obtained by the image preprocessing module to extract frequency domain energy features.

[0119] Then, graph cut optimization is performed: an energy function is constructed, combining the data term (frequency domain difference) and the smoothing term (spatial continuity) to solve the optimal registration parameters.

[0120] Finally, pixel-level fusion is performed: based on the registration results, the bone information of CT and the soft tissue information of MRI are fused into a grayscale image, namely the fused image A.

[0121] The functional information (photosensitizer distribution) of fluorescence image B needs to be superimposed on the anatomical image (fused image A). Due to possible deformation (such as respiratory movement), the Toet algorithm is used to fuse fused image A and fluorescence image B to obtain the final fused image Z as the registration result. The Toet algorithm fusion includes:

[0122] First, feature decomposition is performed: the fused image A and the fluorescence image B are decomposed into a common background (A∩B) and their respective features (A*, B*).

[0123] Secondly, dynamic weight fusion is performed: weights are assigned according to fluorescence intensity to highlight the lesion area (e.g., high fluorescence = high weight), while retaining the anatomical structure to obtain the registration result.

[0124] The specific formula process of the Toet algorithm is performed using conventional means in the art, and the present invention will not elaborate on it here.

[0125] Secondly, the registration result is optimized and a fused multimodal comprehensive view is output, including:

[0126] First, a mask is used to constrain the fusion area. The binary mask provided by the image preprocessing module is used to limit the fusion range (e.g., only the tumor area is superimposed with fluorescence) to avoid background interference.

[0127] Then, the grayscale image of the fused registration result is combined with the pseudo-color fluorescence image to generate a color fused image. Specifically, red represents the high fluorescence area (such as the tumor core), green represents the transition area, and the gray background represents the anatomical structure of CT / MRI.

[0128] Finally, the standardized image data is output as a fused multimodal comprehensive view, including DICOM fused images (including registration parameter labels) and quantitative reports (such as lesion volume and fluorescence intensity distribution). Specifically, the fused multimodal comprehensive view includes the following key information: 1. Anatomy (CT / MRI fusion layer): high-resolution bone / soft tissue contours (CT shows bony landmarks, MRI shows neurovascular structures). 2. Functional information (fluorescence fusion layer): photosensitizer-enriched areas (typically displayed in pseudocolor as a red core and green edge). 3. Metadata: spatial coordinates (Image Position Patient) and pixel physical size (Pixel Spacing).

[0129] The image fusion module of the present invention relies on the output of the image preprocessing module, which has undergone enhancement and noise reduction, brightness / size consistency processing, and segmentation processing or obtained. The high-quality image data input enables the image fusion module to achieve precise registration. At the same time, the image fusion module further corrects the spatial deviation of the image data using DCT-CG and Toet as registration algorithms. Collaborating with the image preprocessing module, it solves core problems in medical imaging, such as noise, registration errors, and blurred boundaries. This lays a solid foundation for lesion identification and localization, thereby further improving the accuracy of photodynamic therapy imaging positioning.

[0130] The lesion identification and positioning module is configured to accurately mark the location, size and boundary of the lesion based on the fused multimodal comprehensive view of the image fusion module and display it on the screen in an intuitive manner. Specifically, it includes:

[0131] Detection of candidate lesion regions involves extracting areas in the fused multimodal integrated view where the fluorescence signal exceeds a threshold (e.g., red channel value > 0.7). Within the candidate regions, the corresponding CT / MRI pixel values ​​are checked using the CT / MRI fusion layer to exclude false positives (e.g., calcifications that do not take up photosensitizer, edema or necrosis that may be accompanied by photosensitizer retention, etc.).

[0132] Precise boundary delineation and multi-feature fusion segmentation are performed, including fluorescence gradient extraction. The maximum first-order derivative (i.e., gradient maximum) of the green channel (mid-grayscale region) corresponds to the pathological invasion front. Anatomical constraints are also implemented: CT / MRI organ segmentation masks (e.g., liver contours) are used to limit the spread of abnormal fluorescence and avoid boundary overestimation caused by fluorescence diffusion (reducing error by 2.3mm). Dynamic contour optimization is performed using an improved active contour model (Snake algorithm) to output closed boundaries with sub-pixel accuracy.

[0133] Perform three-dimensional spatial positioning, including coordinate system conversion and volume calculation. The coordinate system conversion includes reading the DICOM tag Image Position Patient (e.g., the physical coordinates of the upper left corner of the image) and converting the pixel coordinates to the patient coordinate system, thereby ensuring that the surgical navigation system is consistent with the image coordinates. The volume calculation can, for example, include calculating the lesion area layer by layer and integrating it to obtain the total lesion volume V:

[0134] , where V is the total volume of the lesion, which represents the cumulative sum of the volumes of all slices. is the current slice number, indicating the image layer index (such as layer 1 to layer N), N is the total number of slices covered by the lesion, indicating the number of slices in the DICOM tag, represents the cross-sectional area of ​​the k-th layer lesion, Indicates slice thickness (the distance between adjacent slices), which is represented by Slice Thickness in the DICOM tag (such as 1.0 mm).

[0135] Where, Compute by segmentation mask: = Number of pixels in the k-th layer segmentation mask × physical area of ​​a single pixel . Indicates the physical size of a single pixel (resolution in the X / Y direction, respectively), specifically the Pixel Spacing in the DICOM tag (e.g., [0.5, 0.5] means each pixel = 0.5 mm × 0.5 mm).

[0136] Visual output, including augmented reality (AR) overlays on the display screen, including marking lesions with a semi-transparent color overlay on the original image (e.g., red = core, yellow = edge); real-time display of key parameters: maximum diameter, volume, distance to key structures (e.g., "2.3 mm from the optic nerve", etc.).

[0137] Visual output also includes the generation of quantitative reports, including but not limited to DICOM SR files (machine-readable standardized reports containing all measurement data and confidence intervals), PDF graphic reports (containing a simplified version of key parameter tables, lesion diagrams and treatment recommendations) and real-time navigation interfaces (dynamically displaying lesion boundaries and risk areas on the surgical / radiotherapy equipment screen).

[0138] The lesion identification and positioning module of the present invention improves specificity through collaborative analysis of multimodal features (fluorescence + anatomy), submillimeter spatial mapping ensures positioning accuracy, and intuitive interactive design assists clinical decision-making, achieving a closed loop from "seeing the lesion" to "precise intervention", thereby further improving the accuracy of photodynamic therapy imaging positioning.

[0139] The present invention provides more comprehensive and intuitive physiological information through preprocessing, fusion and registration of different types of images, reducing the difficulty for doctors to switch between different images and imagine spatial structural relationships during diagnosis. After image preprocessing, the image specifications are unified, eliminating the heterogeneity of multimodal images, effectively improving the accuracy and real-time performance of multimodal image registration, and improving the accuracy of photodynamic therapy imaging positioning.

[0140] Example 2

[0141] Another embodiment of the present application provides a photodynamic therapy imaging positioning method, which is applied to the photodynamic therapy imaging positioning system described in the first embodiment, including the following steps:

[0142] The image acquisition module collects fluorescence images, CT images, and MRI images of the same patient from different devices, detects whether the collected images are in DICOM format, and if not, performs format conversion and metadata integration on the images, unifying all images from different devices into DICOM format and generating standardized DICOM format images.

[0143] The image preprocessing module preprocesses the standardized DICOM format images generated after the image acquisition module unifies the format, and generates a pre-processing result image;

[0144] The image fusion module performs multimodal registration and registration result optimization based on the regions segmented by the image preprocessing module, and outputs a fused multimodal comprehensive view;

[0145] The lesion recognition and positioning module accurately marks the location, size and boundary of the lesion based on the fused multimodal comprehensive view of the image fusion module, and displays it on the screen in an intuitive manner.

[0146] The present invention provides more comprehensive and intuitive physiological information through preprocessing, fusion and registration of different types of images, reducing the difficulty for doctors to switch between different images and imagine spatial structural relationships during diagnosis. After image preprocessing, the image specifications are unified, eliminating the heterogeneity of multimodal images, effectively improving the accuracy and real-time performance of multimodal image registration, and improving the accuracy of photodynamic therapy imaging positioning.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the modules can be electrical, mechanical or other forms.

[0148] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0149] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0150] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A photodynamic therapy imaging positioning system, characterized by: include: An image acquisition module, an image preprocessing module and an image fusion module that are communicatively connected to each other; The image acquisition module is configured to collect fluorescence images, CT images, and MRI images of the same patient from different devices, detect whether the collected image format is DICOM format, and if not, perform format conversion and metadata integration on the image, unifying all images from different devices into DICOM format and generating standardized DICOM format images; The image preprocessing module is configured to preprocess the standardized DICOM format images generated after the image acquisition module unifies the format, including: The grayscale-color transformation method is used to perform pseudo-color processing on fluorescence images. The method involves extracting the grayscale values ​​of each region of the fluorescence image and mapping the grayscale values ​​of the pixels of the fluorescence image to different colors in a hierarchical manner to generate a pseudo-color fluorescence image. The mapping relationship is: low grayscale is mapped to blue, medium grayscale is mapped to green, and high grayscale is mapped to red. The CT and MRI images are subjected to noise reduction processing using a median filter method. The method includes dividing the CT and MRI images into several regions, counting the grayscale values ​​of the pixels in each region, sorting the pixel values ​​of each point from small to large grayscale values, selecting the middle value of the grayscale value sorting as the final calculation result of the filtering, and setting the final calculation result of the filtering as the pixel value at the center of the region to generate an enhanced CT / MRI image. Normalize the brightness and size of pseudo-color fluorescence images and enhanced CT / MRI images to ensure the consistency of different image scales and generate pre-processing result images; Image type detection: detect whether the pre-processed image is a fluorescence image or a CT / MRI image and calculate the optimal segmentation threshold, specifically: If the image detected is a fluorescent image, the green channel after pseudo-color processing is extracted, and the optimal segmentation threshold is calculated using the Otsu method automatic thresholding; If the detected image is a CT / MRI image, the grayscale image after noise reduction is directly used, and the adaptive threshold is used to calculate the optimal segmentation threshold; Generate a binary mask and divide the grayscale of the entire image into two parts through the optimal segmentation threshold, so that the grayscale value difference between the two parts is maximized and the difference within each part is minimized to achieve boundary delineation; The image fusion module is configured to perform multimodal registration and registration result optimization according to the regions segmented by the image preprocessing module, and output a fused multimodal comprehensive view; The registration result optimization and output of the fused multimodal comprehensive view include: Use mask to constrain the fusion area, and use the binary mask provided by the image preprocessing module to limit the fusion range; The grayscale image of the fused registration result is combined with the pseudo-color fluorescence image to generate a color fused image. Specifically, red represents the high fluorescence area, green represents the transition area, and the gray background represents the anatomical structure of CT / MRI; Output standardized image data, which is a fused multimodal comprehensive view including DICOM fused images and quantitative reports.

2. The photodynamic therapy imaging positioning system according to claim 1, characterized in that: The image acquisition module includes an image data receiving unit, a format detection and conversion unit, and a metadata integration unit, wherein: The image data receiving unit is connected to the fluorescence imaging device, CT and MRI imaging devices and receives image data from these devices; The format detection and conversion unit performs format detection on the image data received by the image data receiving unit and uniformly converts all detected image data into the DICOM format; specifically, it includes: Parse the file header of the received image data, determine the format of the file header, and classify the image data into DICOM format and non-DICOM format according to the format of the file header; If the detected format is DICOM, the metadata in the image data is extracted and entered into the metadata integration unit; If the detected format is non-DICOM, the non-DICOM format is converted and metadata is injected into the metadata integration unit.

3. The photodynamic therapy imaging positioning system according to claim 2, characterized in that: The metadata integration unit integrates metadata, verifies data and stores it, verifies the Study UID / Patient ID of the image data after metadata integration, ensures that the required tags are not empty to complete the integrity check, and that the CT / MRI / fluorescence images of the same patient need to share the same Study UID to complete the consistency verification, and stores the data that pass the integrity check and consistency verification in the PACS system.

4. The photodynamic therapy imaging positioning system according to claim 3, characterized in that: The image acquisition module further includes an exception handling unit configured to perform exception measures when detecting an abnormality in the image data received by the image data receiving unit, wherein the abnormality includes the absence of non-DICOM metadata and the inconsistency of multiple device Study UIDs, including: When it is detected that non-DICOM metadata is missing in the image data received by the image data receiving unit, the format detection and conversion unit enters the HIS system API to query and complete the missing metadata in real time; When it is detected that the image data received by the meta-image data receiving unit has inconsistent Study UIDs of multiple devices, the metadata integration unit enters and forcibly unifies the Study UIDs of multiple devices into the Study UID of the first inspection.

5. The photodynamic therapy imaging positioning system according to claim 1, characterized in that: The normalized brightness processing includes: Step 1: Extract the target channel: For pseudo-color fluorescence images, extract the green channel; for enhanced CT / MRI images, directly use the original grayscale data; Step 2: Modality-specific normalization: The green channel of the pseudo-color fluorescence image is linearly normalized to [0, 1]. For enhanced CT images, the images are first truncated to the typical window width and then normalized to [0, 1]. For enhanced MRI images, quantile-based normalization is performed to [0, 1]. Step 3: Verify the brightness distribution: Check whether the peak of each modal histogram is within the target range. If the brightness distribution of a modal deviates from the expected range, trigger recalibration and repeat steps 1-3.

6. The photodynamic therapy imaging positioning system according to claim 5, characterized in that: The resizing process includes: Step 1: Determine the baseline resolution, read the pixel spacing from the DICOM tag, and select the minimum spacing as the baseline; Step 2: resampling calculation, upsampling from low-resolution image to high-resolution; Step 3: Verify spatial alignment. Check the image position tag and image orientation tag in the DICOM tag to ensure that the physical coordinates are consistent. Calculate the feature point matching error between the resampled image and the original image. Step 4: Edge filling processing: fill the modality-specific background value in the area beyond the boundary after upsampling.

7. The photodynamic therapy imaging positioning system according to claim 1, characterized in that: It also includes a lesion identification and positioning module that is communicatively connected to the image acquisition module and the image preprocessing module, and is configured to accurately mark the location, size and boundary of the lesion based on the fused multimodal comprehensive view of the image fusion module, and display it on the screen in an intuitive manner.

8. A method for imaging positioning of photodynamic therapy, characterized by: Applied to the photodynamic therapy imaging positioning system according to any one of claims 1 to 7, the method comprises the following steps: Collect fluorescence images, CT images, and MRI images of the same patient from different devices, detect whether the collected image format is DICOM format, and if not, perform format conversion and metadata integration on the image, unify all images from different devices into DICOM format, and generate standardized DICOM format images; Preprocess the standardized DICOM format images generated after the unified format to generate pre-processing result images; Perform multimodal registration and registration result optimization on the areas segmented by the image preprocessing module, and output a fused multimodal comprehensive view; The location, size and boundary of the lesion are marked based on the fused multimodal comprehensive view and displayed on the screen in an intuitive manner.

Images