Photosensitizer image processing identification method and system for endoscopic photodynamic therapy

Abstract

The application provides a photosensitizer image processing recognition method and system for endoscopic photodynamic power, relates to the technical field of image processing, determines a pre-imaging modal type and photosensitizer self characteristics, performs light excitation tracing analysis to determine target photosensitizer information, executes light excitation tracing imaging operation, collects and returns multimodal images, and performs spatial position mapping, positions image overlapping boundaries, and performs multimodal image feature extraction and same-modal conversion fitting, determines a fusion image, and performs local reinforcement of element image features based on visual elements, and performs hierarchical mapping correlation to determine a hierarchical display image. The technical problem that the image recognition is limited due to the lack of a systematic photosensitizer imaging processing method in the prior art, which cannot guarantee image quality, is solved. Through multimodal image fusion analysis, image errors are reduced, local enhancement and hierarchical visualization are performed, and the information intuitiveness of the display image is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method and system for image processing and recognition of photosensitizers used in endoscopic photodynamic therapy. Background Technology

[0002] Currently, photodynamic endoscopy photosensitizer imaging has been widely used in the medical field for the detection of lesion features. By utilizing the photochemical reactions generated by photosensitizers under specific light, these reactions can be observed and recorded through an endoscope, thereby achieving precise imaging and diagnosis of diseased tissues.

[0003] Currently, most methods for selecting and tracing photosensitizers are based on single imaging techniques, which inevitably introduce imaging errors, resulting in insufficient detection accuracy. Other methods combine multiple imaging techniques, but these technologies are not yet mature enough. Limitations in the effective selection of photosensitizers and multi-image analysis lead to discrepancies between the acquired image information and the actual situation. In summary, existing technologies lack a systematic imaging processing method for photosensitizers, failing to guarantee image quality and thus limiting image recognition capabilities. Summary of the Invention

[0004] This application provides a method and system for photosensitizer image processing and recognition for endoscopic photodynamic therapy, which addresses the technical problem that existing imaging processing methods for photosensitizers lack systematic approaches, cannot guarantee image quality, and thus limit image recognition.

[0005] In view of the above problems, this application provides a method and system for photosensitizer image processing and recognition for endoscopic photodynamic therapy.

[0006] This application provides a photosensitizer image processing and recognition method for endoscopic photodynamic therapy, the method comprising:

[0007] The pre-imaging modality type is determined in relation to the photosensitizer's intrinsic properties, wherein the pre-imaging modality type includes fluorescence imaging modality, chemiluminescence imaging modality, and photothermal imaging modality;

[0008] Based on the pre-imaging modality type and the photosensitizer's intrinsic properties, photoexcitation tracing analysis is performed using an adaptive matching model to determine the target photosensitizer information, which includes photosensitizer type, photosensitizer dosage, and laser parameters.

[0009] Perform photoexcitation tracing imaging based on the target photosensitizer information, and acquire and transmit multimodal images;

[0010] Spatial location mapping is performed on the multimodal images, and boundary localization is performed based on image overlap to determine the image overlap boundary;

[0011] A supervised training image fusion model is used to identify overlapping boundaries of the images, perform multimodal image feature extraction and same-modal transformation fitting, and determine the fused image.

[0012] Interactive visualization elements are used to enhance the local features of the element images in the fused image, and to perform hierarchical mapping and association to determine the hierarchical display image, wherein the mapping layer corresponds one-to-one with the visualization element;

[0013] Based on the hierarchical image display, lesion features are identified.

[0014] This application provides a photosensitizer image processing and recognition system for endoscopic photodynamic therapy, the system comprising:

[0015] An information acquisition unit is used to determine the pre-imaging modality type and the photosensitizer's intrinsic properties, wherein the pre-imaging modality type includes fluorescence imaging modality, chemiluminescence imaging modality, and photothermal imaging modality;

[0016] The photoexcitation analysis unit is used to perform photoexcitation tracing analysis based on the pre-imaging mode type and the photosensitizer's intrinsic properties, combined with an adaptive matching model, to determine the target photosensitizer information, which includes photosensitizer type, photosensitizer dosage, and laser parameters.

[0017] An image acquisition unit is used to perform a photoexcitation tracer imaging operation based on the target photosensitizer information and acquire and transmit multimodal images.

[0018] An overlapping boundary localization unit is used to perform spatial position mapping on the multimodal image, perform boundary localization based on image overlap, and determine the image overlapping boundary.

[0019] An image fusion unit is used to supervise the training of an image fusion model, identify overlapping boundaries of the images, perform multimodal image feature extraction and same-modal transformation fitting, and determine the fused image.

[0020] An image processing unit is used for interactive visualization elements, performing local enhancement of element image features on the fused image, and performing hierarchical mapping association to determine the hierarchical display image, wherein the mapping layer corresponds one-to-one with the visualization element;

[0021] An image recognition unit is used to identify lesion features based on the layered displayed image.

[0022] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0023] The photosensitizer image processing and recognition method for endoscopic photodynamic therapy provided in this application determines the pre-imaging modality type, including fluorescence imaging, chemiluminescence imaging, and photothermal imaging, along with the photosensitizer's intrinsic properties. It then combines this with an adaptive matching model for photoexcitation tracing analysis to determine target photosensitizer information, including photosensitizer type, dosage, and laser parameters. The method performs photoexcitation tracing imaging based on this target photosensitizer information, acquires and transmits multimodal images, performs spatial location mapping, determines image overlap boundaries based on image overlap, and uses an image fusion model to extract multimodal image features and perform same-modality transformation fitting. The method determines the fused image and performs local enhancement of image features based on visualization elements. Finally, it performs hierarchical mapping and association to determine the hierarchical display image. This method solves the technical problem in existing technologies where the lack of a systematic photosensitizer imaging processing method results in limited image quality and restricted image recognition. By performing multimodal image fusion analysis, it reduces image errors, enhances local details, and visualizes hierarchical levels, improving the intuitiveness of the displayed image. Attached Figure Description

[0024] Figure 1 This application provides a schematic flowchart of a photosensitizer image processing and recognition method for endoscopic photodynamic therapy;

[0025] Figure 2 This application provides a schematic diagram of the structure of a photosensitizer image processing and recognition system for endoscopic photodynamic therapy.

[0026] Explanation of reference numerals in the attached figures:

[0027] Information acquisition unit 01, light excitation analysis unit 02, image acquisition unit 03, overlapping boundary positioning unit 04, image fusion unit 05, image processing unit 06, and image recognition unit 07. Detailed Implementation

[0028] This application provides a method and system for photosensitizer image processing and recognition for endoscopic photodynamic therapy. The method, based on the photoexcitation tracing principle of photosensitizers, determines the optimal photosensitizer and dosage to meet the image acquisition modality, guiding endoscopic detection and implementing multimodal imaging to avoid errors associated with single-mode detection. Multimodal feature fitting is performed to determine a fused image, improving image accuracy. Furthermore, the fused image undergoes multi-feature level enhancement display for intuitive and globally refined display of image information. This effectively solves the technical problem in existing technologies where the lack of systematic photosensitizer imaging processing methods leads to unreliable image quality and limited image recognition.

[0029] Example 1

[0030] like Figure 1As shown, this application provides a photosensitizer image processing and recognition method for endoscopic photodynamic therapy, the method comprising:

[0031] S1: Determine the pre-imaging mode type and the photosensitizer's intrinsic properties, wherein the pre-imaging mode type includes fluorescence imaging mode, chemiluminescence imaging mode and photothermal imaging mode;

[0032] With the widespread application of photodynamic endoscopic photosensitizer imaging in the medical field, higher demands are placed on the quality of endoscopic images to adapt to the analysis of diverse patient information. Currently, there are still certain deficiencies in the technical control of imaging, making it impossible to minimize technical losses, resulting in limited image quality and impacting subsequent recognition. This application provides a photosensitizer image processing and recognition method for endoscopic photodynamic therapy. Based on the photoexcitation tracing principle of photosensitizers, it determines the optimal photosensitizer and dosage to meet the image acquisition modality, guides endoscopic detection, and implements multimodal imaging to avoid errors inherent in single detection. Multimodal feature fitting is performed to determine a fused image to improve image accuracy; further, the fused image undergoes multi-feature level enhancement display to provide a more intuitive and globally refined display of image information.

[0033] First, multiple imaging modalities for image acquisition are identified as pre-imaging modal types, including fluorescence imaging, chemiluminescence imaging, and photothermal imaging. Multi-modal imaging is combined to ensure the globality and completeness of information and improve accuracy. The principles of each modality differ: fluorescence in the visible or near-infrared region is emitted upon excitation by light of a specific wavelength; chemiluminescence is achieved based on a photodynamic reaction for chemiluminescence imaging; and photothermal imaging is achieved by converting light energy into heat energy based on strong light absorption and conversion capabilities. Simultaneously, the intrinsic characteristics of different photosensitizers are acquired, such as biocompatibility, stability, targeting, and distribution and metabolism in vivo, as different photosensitizers are suitable for different imaging types. Based on the pre-imaging modal types and the intrinsic characteristics of the photosensitizers, the photosensitizer with the highest compatibility is determined.

[0034] S2: Based on the pre-imaging mode type and the photosensitizer's intrinsic properties, photoexcitation tracing analysis is performed using an adaptive matching model to determine the target photosensitizer information, which includes photosensitizer type, photosensitizer dosage, and laser parameters.

[0035] The step S2, which combines the adaptive matching model for photoexcitation tracing analysis, further includes: S21: The adaptive matching model is a multi-layer fully connected neural network model, including a feature matching layer, a photoanalysis layer, and an excitation verification layer, which is trained to convergence using sample data; S22: A hierarchical matching analysis is performed using the compatibility between the intrinsic properties of each photosensitizer and the pre-imaging modality type as a first-layer matching standard, and the photoexcitation tracing effect of the parameter triplet as a second-layer matching standard, to obtain the photosensitizer type and laser parameters, wherein the parameter triplet includes laser wavelength, laser intensity, and beam diameter.

[0036] The adaptive matching model is an autonomous analysis model for matching photosensitizers. By constructing the model for analysis and processing, the subjectivity of the analysis results can be effectively guaranteed, and the analysis efficiency and accuracy can be improved.

[0037] The following describes the construction method of the adaptive matching model. Specifically, the model structure is a three-layer fully connected neural network model, including the feature matching layer, the optical analysis layer, and the excitation verification layer. The feature matching layer is used for extracting the intrinsic features of the photosensitizer and identifying the photosensitizer characteristics suitable for different imaging modalities. The optical analysis layer is used for optical imaging tracer analysis under different laser types and parameters to determine the most suitable photosensitizer information and laser parameters. The excitation verification layer is used to verify the analysis results and determine whether the requirements of the imaging modality are met.

[0038] The adaptive matching model is constructed by collecting sample data, dividing the training samples for supervised training, and further verifying the convergence of the trained model based on the divided verification samples. If the convergence condition is not met, that is, the model accuracy is insufficient and there is overfitting, the samples are re-divided and trained and verified until the convergence condition is met, and the adaptive matching model is obtained.

[0039] The pre-imaging mode type and the photosensitizer's intrinsic properties are then input into the adaptive matching model to determine the fit threshold that is satisfied by fluorescence imaging, chemiluminescence imaging, and photothermal imaging. This fit threshold serves as the first-layer matching standard, and is a critical value for fit, which can be customized by those skilled in the art based on imaging quality requirements. Furthermore, the laser's control parameters, including laser wavelength, laser intensity, and beam diameter, are determined and used as a parameter triplet as the second-layer matching standard. This parameter triplet represents the laser parameter type for which optimal parameter analysis is to be performed.

[0040] By combining the adaptive matching model, feature extraction and matching analysis are performed on the pre-imaging modality type and the intrinsic properties of the photosensitizer to determine the most suitable photosensitizer and dosage. This information is then transferred to the photoexcitation layer to determine the optimal control parameters of the laser, which are then verified. The acquired photosensitizer type, photosensitizer dosage, and laser parameters are used as the target photosensitizer information. Based on this target photosensitizer information, the tracking accuracy and imaging quality of the lesion area can be maximized.

[0041] S3: Perform photoexcitation tracer imaging based on the target photosensitizer information, and acquire and transmit multimodal images;

[0042] S4: Perform spatial location mapping on the multimodal image, perform boundary localization based on image overlap, and determine the image overlap boundary;

[0043] The target photosensitizer information is the optimal configuration information to meet the requirements of multimodal imaging. The user takes in the photosensitizer based on the type and dosage. Based on the laser parameters, the laser installed in the endoscope is configured for parameter control. The user is subjected to photoexcitation tracer imaging operation, and the acquired multimodal images are transmitted back to the display.

[0044] Furthermore, a point-to-point correspondence is performed on the multimodal images to determine the mapped image portions corresponding to the same spatial location. The overlapping portions of the fluorescence modal image, chemiluminescence modal image, and photothermal modal image are identified, and the overlapping boundaries are defined to determine the image overlap boundaries. This facilitates subsequent multimodal imaging fitting processing of the overlapping image portions by directly identifying the image overlap boundaries.

[0045] Specifically, step S4 further includes: performing spatial location mapping on the multimodal images, locating boundaries based on image overlap, and determining image overlap boundaries; reading the spatial distribution structure of the human detection location and establishing a spatial coordinate system; identifying the multimodal images, distributing the images in the spatial coordinate system, and performing point-to-point mapping to determine the mapped distribution images; dividing the mapped distribution images into overlapping regions and identifying the image overlap boundaries, wherein the overlapping regions include double-overlapping regions and triple-overlapping regions.

[0046] Specifically, the spatial distribution structure of the human body to be detected is determined, such as the stomach structure. This spatial distribution structure can be a standardized structure determined based on the human anatomy. A spatial coordinate system is then established based on this spatial distribution structure. Further, within this spatial coordinate system, the multimodal images are spatially distributed, determining three image coordinate systems mapped onto the multimodal images. Based on the mapping correspondence of the spatial structure, point-to-point mapping is performed on the three image coordinate systems to ensure that corresponding positions are at the same structural position, serving as the mapped distribution image. Further, overlapping regions are identified in the mapped distribution image. These overlapping regions include the overlapping areas of any two modal images and areas that overlap with all three modal images. The overlapping boundaries of the images are identified and marked. By performing point-to-point mapping of the images based on the spatial distribution structure, it is ensured that each corresponding position is at the same structural position, thus guaranteeing the accuracy of overlapping boundary positioning.

[0047] S5: Supervised training of the image fusion model, identifying the overlapping boundaries of the images, performing multimodal image feature extraction and same-modal transformation fitting, and determining the fused image;

[0048] The multimodal image feature extraction and same-modal transformation fitting process includes the following steps: S51: Identifying the overlapping regions and determining double-overlapping and triple-overlapping regions; S52: Performing multimodal image feature extraction and same-modal transformation fitting on the double-overlapping and triple-overlapping regions to determine effective modal features; S53: Spatially distributing and stitching the effective modal features based on the image mapping position to determine the fused image.

[0049] The process of performing feature extraction and modality transformation fitting on the multimodal images of the double-overlapping region and the triple-overlapping region, step S52, further includes: S521: Based on the feature extraction layer of the image fusion model, performing image convolution feature extraction by traversing the multimodal images, wherein the feature extraction layer includes three parallel image extraction branches; S522: Determining the image visualization modality, transferring the image convolution feature flow to the modality transformation layer, using the visualization modality as the transformation standard, performing feature modality transformation, and determining the modality feature mapping set; S523: Combining with the feature fitting layer, performing multi-angle feature fitting by traversing the modality feature mapping set, and determining the effective modality features.

[0050] The image fusion model is a functional model for fusing overlapping images. The model's construction method is as follows: Specifically, the model structure includes a pre- and post-associative feature extraction layer, a modality transformation layer, and a feature fitting layer. The feature extraction layer includes three parallel feature extraction branches, each used to extract features from different modalities, improving processing specificity. A large-scale data retrieval is performed to determine a sample image set, whose image type and format are consistent with the multimodal images. Based on the sample image set, supervised training based on the model structure is performed to obtain the image fusion model that meets an accuracy threshold. This accuracy threshold is the convergence criterion for model training and can be customized by those skilled in the art based on processing quality requirements.

[0051] Furthermore, the overlapping boundaries of the images are identified, and the overlapping image regions of the multimodal images, including the double-overlapping region and the triple-overlapping region, are input into the image fusion model. By matching the feature extraction branches of the feature extraction layer, the branch to be processed that fits the image modality is determined, and image convolution feature extraction is performed to obtain the image convolution features. Then, the image convolution features are transferred to the modality conversion layer, where the image visualization modality is the final image display modality and can be any one of the three image modalities. Using the image visualization modality as the conversion modality, modality conversion is performed on the image convolution features of the other two image modalities, converting them all to the image visualization modality. Based on the same modality, the imaging features of multiple imaging modalities are fitted, for example, by stitching feature angles and fusing feature ranges, to determine the fused image features as the effective modal features.

[0052] Furthermore, based on the image mapping position, the spatial distribution position of the effective modal features is determined, and the effective modal features are spatially stitched together. The stitched image is used as the fused image, wherein the fused image represents the visualization modality. Note that under different imaging methods, there may be differences in the feature angle, feature intensity, etc., for the same feature. Feature fusion is performed to ensure the completeness and accuracy of the features, thereby improving image quality.

[0053] S6: Interactive visualization elements, local enhancement of element image features is performed on the fused image, and hierarchical mapping association is performed to determine the hierarchical display image, wherein the mapping layer corresponds one-to-one with the visualization elements;

[0054] S7: Based on the layered display image, identify the lesion features.

[0055] The step S6 of determining the hierarchical display image further includes: S61: the visualization elements are determined based on the information requirements of the image display; S62: the visualization elements are identified, located in the fused image, and the element partition images are determined; S63: the element display standards are read, the element partition images are preprocessed, and batch enhancement images are determined, wherein the element display standards of each batch enhancement image are the same; S64: the batch enhancement images and the fused image are mapped between layers to generate the hierarchical display image.

[0056] The process of reading the feature display standard and preprocessing the feature partition image includes the following steps: S631: Based on the feature display standard, perform defect identification on the mapped feature partition image to determine the partition image defects; S632: For the partition image defects, match the image processing algorithm based on the defect type, wherein the defect determination criterion is whether the feature display standard can be met; S633: Based on the image processing algorithm, preprocess the feature partition image to determine the batch enhancement image.

[0057] The visualization elements are the image information that needs to be displayed intuitively, determined based on the information requirements that the image needs to show, such as local areas, multi-angle features, etc. The fused image is traversed, and image matching mapping is performed on the visualization elements to determine the image regions where the visualization elements are distributed, which serve as the element partitioning image.

[0058] Further determination is made when the element partition images meet the display requirements; otherwise, image preprocessing and enhancement are required. Specifically, the element display standard, i.e., the degree standard to which each visualized element needs to be displayed, is read and customized based on the display requirements. Different element partition images correspond to different element display standards. Based on the mapping relationship, it is determined whether each element partition image conforms to the mapped element display standard. If it does, no image processing is needed; if it does not, based on the differences in element partition images according to the element display standard, such as insufficient feature strength or the presence of noise, these are considered defects in the partition images.

[0059] Based on the defects in the partitioned image, an image processing algorithm matching the defect type is configured. For example, for image noise, a noise reduction algorithm, such as wavelet noise reduction, is configured. Based on the configured image processing algorithm, the defects in the partitioned image are preprocessed to meet the feature display standard. Specifically, based on the defect type of the partitioned image defects, defects of the same type are clustered and batch-processed to improve the processing effect, thereby obtaining the batch-enhanced images.

[0060] Furthermore, based on the aforementioned visualization elements, display layers are created, such as magnified detail display and full-angle 3D display. Using the fused image as a base, display layers are created and inter-layer mapping connections are established to determine the hierarchical display image. The hierarchical display image can intuitively display various detailed features, facilitating the intuitive identification and extraction of various image feature information.

[0061] The hierarchical images are then visualized using a display interface, allowing for direct identification of lesion features. Preferably, detailed feature identification can be achieved by combining the hierarchical mapping relationship.

[0062] This embodiment provides a photosensitizer image processing and recognition method for endoscopic photodynamic therapy, which has the following technical effects:

[0063] 1. By providing an image processing and recognition method for photosensitizers used in endoscopic photodynamics, based on the photoexcitation tracing principle of photosensitizers, the optimal photosensitizer and dosage that meet the image acquisition modality are determined, endoscopic detection is guided, multimodal imaging is implemented, errors existing in single detection are avoided, and image quality and information completeness are improved.

[0064] 2. Overlap analysis was performed on the acquired multimodal images, and multimodal feature fusion was performed on the overlapping parts to determine the final image, ensuring the completeness and accuracy of the image coverage information, and laying a solid foundation for subsequent lesion analysis.

[0065] 3. By introducing image recognition and image processing algorithms, targeted modeling is performed to complete diversified processing of the acquired images and determine the processed detection image. Based on a targeted, visualized, hierarchical analytical structure diagram, the fused image is enhanced with multi-feature levels to provide a more intuitive and refined global display of image information, facilitating the identification and analysis of effective image information.

[0066] Example 2

[0067] Based on the same inventive concept as the photosensitizer image processing and recognition method for endoscopic photodynamics in the foregoing embodiments, such as Figure 2 As shown, this application provides a photosensitizer image processing and recognition system for endoscopic photodynamic therapy, the system comprising:

[0068] Information acquisition unit 01, the information acquisition unit 01 is used to determine the pre-imaging mode type and the photosensitizer's own characteristics, wherein the pre-imaging mode type includes fluorescence imaging mode, chemiluminescence imaging mode and photothermal imaging mode;

[0069] The photoexcitation analysis unit 02 is used to perform photoexcitation tracing analysis based on the pre-imaging mode type and the photosensitizer's intrinsic properties, combined with an adaptive matching model, to determine the target photosensitizer information, which includes photosensitizer type, photosensitizer dosage, and laser parameters.

[0070] Image acquisition unit 03 is used to perform photoexcitation tracer imaging operation based on the target photosensitizer information and acquire and transmit multimodal images;

[0071] Overlapping boundary positioning unit 04 is used to perform spatial position mapping on the multimodal image, perform boundary positioning based on image overlap, and determine the image overlapping boundary.

[0072] Image fusion unit 05 is used to supervise the training of an image fusion model, identify the overlapping boundaries of the images, perform multimodal image feature extraction and same-modal transformation fitting, and determine the fused image.

[0073] Image processing unit 06 is used for interactive visualization elements, performing local enhancement of element image features on the fused image, and performing hierarchical mapping association to determine the hierarchical display image, wherein the mapping layer corresponds one-to-one with the visualization element;

[0074] Image recognition unit 07 is used to identify lesion features based on the layered display image.

[0075] Furthermore, the photoexcitation analysis unit 02 also includes the following steps: the adaptive matching model is a multi-layer fully connected neural network model, including a characteristic matching layer, a photoanalysis layer, and an excitation verification layer, which is trained to convergence through supervised training with sample data; using the compatibility between the intrinsic properties of each photosensitizer and the pre-imaging modality type as a first-layer matching standard, and the photoexcitation tracing effect of the parameter triplet as a second-layer matching standard, a hierarchical matching analysis is performed to obtain the photosensitizer type and laser parameters, wherein the parameter triplet includes laser wavelength, laser intensity, and beam diameter.

[0076] Furthermore, the overlapping boundary positioning unit 04 also includes the following steps: reading the spatial distribution structure of the human body detection position and establishing a spatial coordinate system; identifying the multimodal image, distributing the image in the spatial coordinate system respectively, and performing point-to-point mapping to determine the mapped distribution image; dividing the mapped distribution image into overlapping regions and identifying the image overlapping boundaries, wherein the overlapping regions include double overlapping regions and triple overlapping regions.

[0077] Furthermore, the image fusion unit 05 also includes the following steps: identifying the overlapping regions and determining double-overlapping regions and triple-overlapping regions; performing feature extraction and same-modality transformation fitting on the double-overlapping regions and triple-overlapping regions using multimodal images to determine effective modal features; and performing spatial distribution stitching on the effective modal features based on the image mapping position to determine the fused image.

[0078] Furthermore, the image fusion unit 05 also includes the following steps: based on the feature extraction layer of the image fusion model, traversing the multimodal images to perform image convolutional feature extraction, wherein the feature extraction layer includes three parallel image extraction branches; determining the image visualization modality, transferring the image convolutional feature flow to the modality transformation layer, using the visualization modality as the transformation standard to perform feature modality transformation, and determining the modality feature mapping set; combined with the feature fitting layer, traversing the modality feature mapping set to perform multi-angle feature fitting, and determining effective modality features.

[0079] Furthermore, the image processing unit 06 further includes the following steps: the visualization elements are determined based on the information requirements of the image display; the visualization elements are identified and located in the fused image to determine the element partition image; the element display standard is read, the element partition image is preprocessed, and batch enhancement images are determined, wherein the element display standard of each batch enhancement image is the same; the batch enhancement images and the fused image are mapped between layers to generate the hierarchical display image.

[0080] Furthermore, the image processing unit 06 further includes the following steps: based on the feature display standard, performing defect identification on the mapped feature partition image to determine the partition image defects; for the partition image defects, matching the image processing algorithm based on the defect type, wherein the defect judgment standard is whether the feature display standard can be met; based on the image processing algorithm, preprocessing the feature partition image to determine the batch enhancement image.

[0081] Through the foregoing detailed description of the photosensitizer image processing and recognition method for endoscopic photodynamic therapy, those skilled in the art can clearly understand the photosensitizer image processing and recognition method and system for endoscopic photodynamic therapy in this embodiment. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be referred to the method section description.

[0082] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for photosensitizer image processing and recognition for endoscopic photodynamic therapy, characterized in that, The method includes: The pre-imaging modality type is determined in relation to the photosensitizer's intrinsic properties, wherein the pre-imaging modality type includes fluorescence imaging modality, chemiluminescence imaging modality, and photothermal imaging modality; Based on the pre-imaging modality type and the photosensitizer's intrinsic properties, photoexcitation tracing analysis is performed using an adaptive matching model to determine the target photosensitizer information, which includes photosensitizer type, photosensitizer dosage, and laser parameters. Perform photoexcitation tracing imaging based on the target photosensitizer information, and acquire and transmit multimodal images; Spatial location mapping is performed on the multimodal images, and boundary localization is performed based on image overlap to determine the image overlap boundary; A supervised training image fusion model is used to identify overlapping boundaries of the images, perform multimodal image feature extraction and same-modal transformation fitting, and determine the fused image. Interactive visualization elements are used to enhance the local features of the fused image and perform hierarchical mapping and association to determine the display level of the image, wherein the mapping layer corresponds one-to-one with the visualization element; Based on the hierarchical image display, lesion features are identified.

2. The method as described in claim 1, characterized in that, The method for photoexcitation tracing analysis using an adaptive matching model further includes: The adaptive matching model is a multi-layer fully connected neural network model, including a feature matching layer, a light analysis layer, and an excitation verification layer, which is trained to convergence through supervised training with sample data. Using the compatibility between the intrinsic properties of each photosensitizer and the pre-imaging mode type as a first-level matching standard, and the photoexcitation tracing effect of the parameter triplets as a second-level matching standard, a hierarchical matching analysis is performed to obtain the photosensitizer type and laser parameters. The parameter triplets include laser wavelength, laser intensity, and beam diameter.

3. The method as described in claim 1, characterized in that, The method further includes: performing spatial location mapping on the multimodal images, locating boundaries based on image overlap, and determining image overlap boundaries; and performing spatial location mapping on the multimodal images. Read the spatial distribution structure of the human body detection location and establish a spatial coordinate system; The multimodal images are identified, their distribution is performed in the spatial coordinate system, and point-to-point mapping is conducted to determine the mapped distribution images. The mapped distribution image is divided into overlapping regions, and the overlapping boundaries of the images are identified. The overlapping regions include double-overlapping regions and triple-overlapping regions.

4. The method as described in claim 3, characterized in that, The method further includes performing multimodal image feature extraction and same-modal transformation fitting: Identify the overlapping regions to determine double-overlapping and triple-overlapping regions; Feature extraction and same-modal transformation fitting are performed on the multimodal images of the double-overlapping region and the triple-overlapping region to determine the effective modal features; Based on the image mapping location, the effective modal features are spatially distributed and stitched together to determine the fused image.

5. The method as described in claim 4, characterized in that, The method further includes performing feature extraction and isomodality transformation fitting on the multimodal images of the double-overlapping region and the triple-overlapping region. Based on the feature extraction layer of the image fusion model, image convolution feature extraction is performed by traversing the multimodal images. The feature extraction layer includes three parallel image extraction branches. The image visualization modality is determined, and the image convolutional feature stream is transferred to the modality transformation layer. The visualization modality is used as the transformation standard to perform feature modality transformation and determine the modality feature mapping set. By combining the feature fitting layer, the modal feature mapping set is traversed to perform multi-angle feature fitting and determine the effective modal features.

6. The method as described in claim 1, characterized in that, The method for determining the display hierarchy of images further includes: The visualization elements are determined based on the information requirements of the image display. Identify the visualized elements, locate them in the fused image, and determine the element partition image; Read the feature display standard, preprocess the feature partition image, and determine the batch enhancement images, wherein the feature display standard of each batch enhancement image is the same; The batch enhanced image and the fused image are mapped between layers to generate the layered display image.

7. The method as described in claim 6, characterized in that, The method further includes reading the feature display standard, preprocessing the feature partition image, and then: Based on the aforementioned feature display standard, defect identification is performed on the mapped feature partition image to determine the partition image defects; To address the defects in the partitioned image, an image processing algorithm based on defect type matching is used, wherein the defect determination criterion is whether the element display standard can be met; Based on the image processing algorithm, the feature partition image is preprocessed to determine the batch enhanced image.

8. A photosensitizer image processing and recognition system for endoscopic photodynamic therapy, characterized in that, The system is used to implement the photosensitizer image processing and recognition method for endoscopic photodynamic therapy according to any one of claims 1-7, the system comprising: An information acquisition unit is used to determine the pre-imaging modality type and the photosensitizer's intrinsic properties, wherein the pre-imaging modality type includes fluorescence imaging modality, chemiluminescence imaging modality, and photothermal imaging modality; The photoexcitation analysis unit is used to perform photoexcitation tracing analysis based on the pre-imaging mode type and the photosensitizer's intrinsic properties, combined with an adaptive matching model, to determine the target photosensitizer information, which includes photosensitizer type, photosensitizer dosage, and laser parameters. An image acquisition unit is used to perform a photoexcitation tracer imaging operation based on the target photosensitizer information and acquire and transmit multimodal images. An overlapping boundary localization unit is used to perform spatial position mapping on the multimodal image, perform boundary localization based on image overlap, and determine the image overlapping boundary. An image fusion unit is used to supervise the training of an image fusion model, identify overlapping boundaries of the images, perform multimodal image feature extraction and same-modal transformation fitting, and determine the fused image. An image processing unit is used for interactive visualization elements, performing local enhancement of element image features on the fused image, and performing hierarchical mapping association to determine the hierarchical display image, wherein the mapping layer corresponds one-to-one with the visualization element; An image recognition unit is used to identify lesion features based on the layered displayed image.

Images