In-vivo confocal cell quantitative analysis system for lung transplantation rejection diagnosis

Abstract

The invention provides an in-vivo confocal cell quantitative analysis system for lung transplantation rejection diagnosis, and the system comprises a database module which is used for obtaining a multi-class image library and an original pCLE video; the video processing module is used for processing an original pCLE video by using an Image J-AI analysis center; the cell segmentation module is used for obtaining label masks with the same size and the position corresponding to the size and the position of the original image; the feature extraction module is used for identifying single cells and carrying out cell morphology and space analysis; the rejection risk assessment module is used for constructing a score prediction model and calculating a risk score and an overall risk level of a patient; and the interface feedback module is used for visually displaying results and feeding back abnormity. According to the method, through pCLE + ImageJ + AI, cell-level cell segmentation, feature quantification and rejection scoring are achieved, and early precise diagnosis and treatment of lung transplantation rejection are powerfully assisted.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to an in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection. Background Art

[0002] Lung transplantation (LTx) is currently the ultimate treatment option for end-stage lung diseases (such as interstitial lung disease, pulmonary hypertension, bronchiectasis, cystic fibrosis, etc.).

[0003] To clarify the etiology, non-invasive tests, such as imaging and routine biomarker testing, are essential. However, CT images often have overlapping imaging features, such as consolidation and ground-glass opacities, which have limited accuracy in assessing acute rejection, and traditional biological cultures are time-consuming. Invasive tests, such as traditional transbronchial forceps biopsies (TFBx), although the gold standard, were found in a single study of ventilated lung transplant recipients to have low sensitivity and a tendency to underestimate the severity of severe rejection, potentially leading to higher rates of pneumothorax, bleeding, and mortality.

[0004] Confocal laser endoscopy has been widely used in a variety of respiratory diseases. Probe-based confocal laser endomicroscopy (pCLE) is an emerging technology that combines non-invasive and real-time cellular imaging. When the probe is inserted into the airway, it emits a low-power laser beam that interacts with cells, enabling real-time, high-resolution microscopic imaging of lung tissue and cellular structures. However, limited research on the application of pCLE in lung transplant recipient complications exists, and confocal images of lung transplant recipients have not been further analyzed. In vivo imaging feature reading and diagnostic technology for confocal laser endoscopy in patients with lung transplant infection and rejection has not yet been established. This results in limited imaging depth and resolution, making effective monitoring impossible. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide an in vivo confocal cell quantitative analysis system for the diagnosis of lung transplant rejection reaction. Through pCLE+ImageJ+AI, cell segmentation, feature quantification and rejection scoring at the cellular level are achieved, which effectively assists in the early and accurate diagnosis and treatment of lung transplant rejection reaction and improves the survival rate and quality of life of lung transplant patients.

[0006] To achieve the above objectives, the present invention provides the following solution: an in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection, comprising:

[0007] The database module is used to collect relevant diagnostic data of lung transplant patients to obtain a standard data set, perform multi-level annotation on the standard data set to obtain a multi-category image library, and then use a probe-type confocal laser endoscope to obtain high-resolution fluorescence images of the transplanted lung bronchi and alveolar mucosa in real time to obtain the original pCLE video;

[0008] A video processing module is used to import the original pCLE video into the ImageJ-AI analysis center to perform video stabilization, artifact processing, fixed-frequency frame extraction processing and normalization processing to obtain a normalized frame sequence and a frame sequence quality report;

[0009] A cell segmentation module is used to perform contrast enhancement, binary image segmentation and edge correction based on the normalized frame sequence to obtain a position label mask of the same size and position as the original image;

[0010] a feature extraction module for performing individualized cell identification based on the binary label mask of the same size to obtain multiple single cells, and then performing cell morphology analysis and spatial analysis based on the single cells to calculate and integrate the abnormal morphology quantification value, cell density, spatial distribution index and local heterogeneity index to obtain a quantitative feature table;

[0011] A rejection risk assessment module is used to construct a scoring prediction model, input the quantitative feature table into the scoring prediction model, perform parallel calculations of acute risk, chronic risk, and infection risk to obtain a risk score, and then calculate the patient's overall risk level based on the risk score;

[0012] An interface feedback module is used to visually display the original frame information of this examination, the same-size position label mask, the cell density and the risk score through the decision terminal, and preset the risk score threshold to provide abnormal feedback;

[0013] Wherein, the database module, the video processing module, the cell segmentation module, the feature extraction module, the rejection risk assessment module and the interface feedback module are interconnected.

[0014] Optionally, the ImageJ-AI analysis center includes a GPU workstation, ImageJ, an ImageJ self-developed plug-in, and a TensorFlow runtime.

[0015] Optionally, the database module includes:

[0016] A multi-category image library unit is used to collect case data, postoperative examination data, pCLE images and basic clinical data based on lung transplant patients to obtain an original data set, and unify the image format, resolution and naming standards in the original data set to obtain a standard data set; wherein the basic clinical data is bound to the pCLE frame of the pCLE image;

[0017] The labeling unit is used to perform macro-category labeling on the standard data set by experts, and then use ImageJ software to perform micro-labeling on the standard data set that has completed the macro-category labeling to complete the construction of a multi-category library; wherein, the macro-category labeling includes normal, mild acute rejection, moderate acute rejection, severe acute rejection, chronic rejection and infection, and the micro-category labeling includes immune cell areas, tissue and vascular lesion areas and special pathological features.

[0018] The real-time acquisition unit is used to acquire high-resolution fluorescence images of the transplanted lung bronchus and alveolar mucosa in real time using a probe-type confocal laser endoscope to obtain original pCLE videos.

[0019] Optionally, the video processing module includes:

[0020] A video import unit, configured to import the original pCLE video into the ImageJ-AI analysis center through a standardized upload interface, and to store and manage the original pCLE video according to the patient ID, acquisition time, location, and detection sequence number;

[0021] A video stabilization processing unit is configured to call FFmpeg based on the original PCLE video to detect and correct horizontal, vertical, and rotational offsets to complete primary image registration, and then use an image feature extraction algorithm to perform local offset correction on the original PCLE video after the primary image registration to make the video sequence consistent and complete image stabilization processing;

[0022] An artifact processing unit is used to automatically detect brightness and contrast fluctuations of the starting frame of the video using an ImageJ macro and a brightness histogram analysis method based on the original pCLE video that has completed the image stabilization processing, and automatically detect whether there are shading, out of focus, reflection artifacts and background signal clutter frames in the video using a frame-level content recognition algorithm to complete artifact processing;

[0023] A still frame collection unit is configured to uniformly extract frames at a fixed frequency based on the original pCLE video after artifact processing using a frame extraction command of ImageJ, and compare differences between frames to retain and mark frames with diverse content, thereby obtaining a still frame collection;

[0024] A normalization unit is used to convert the still frame set into 16-bit TIFF format and embed EXIF metadata to obtain a normalized frame sequence and a frame sequence quality report, and then send the frame sequence quality report to a quality inspector for review and verification; the frame sequence quality report includes the number of valid frames, average brightness, focus index and rejection rate.

[0025] Optionally, the cell segmentation module includes:

[0026] a contrast enhancement unit, configured to enhance the contrast between cells and background in each frame of the image by using an adaptive histogram equalization technique based on the normalized frame sequence;

[0027] A cell segmentation unit is used to integrate a U-Net++ model into the ImageJ self-developed plug-in to obtain an initial model, and train the initial model using the multi-category library to obtain a cell segmentation model. Using the cell segmentation model, combined with instance segmentation and semantic segmentation, the normalized sequence that has completed contrast enhancement is subjected to cell segmentation to obtain a binary segmentation mask of the normalized frame sequence;

[0028] A cell correction unit is used to fuse the segmented area and the background in the binary segmentation mask using a Poisson-blending boundary smoothing algorithm to correct jagged edges, breaks, over-smoothing, and discontinuities.

[0029] The log and output unit is used to output the binary segmentation mask after edge correction, and correspond the output binary segmentation mask to the original image size and position to obtain a binary label mask of the same size.

[0030] Optionally, the feature extraction module includes:

[0031] A parameter extraction unit is used to separate all individual cell regions based on the binary label mask of the same size using a connected domain analysis method to calculate the number of connected cells in each frame, and to correct the adhesion or split artifact regions using a dynamic threshold morphological segmentation method to identify single cells, complete individual cell identification, and obtain multiple single cells;

[0032] a morphological feature extraction unit, configured to extract the area, perimeter, roundness, and major-minor axis ratio of the single cell, respectively, and define a morphological threshold to determine the morphology of the single cell and obtain a determination result; and when the result is determined to be morphologically abnormal, calculate the degree of deviation of the morphological abnormality to obtain a quantitative value of the abnormal morphology;

[0033] a spatial analysis unit, configured to calculate cell density, spatial distribution index, and local heterogeneity index based on the binary label mask of the same size;

[0034] An AI auxiliary unit is used to automatically extract high-level textures, cell clusters, and edge complexity based on the binary label mask of the same size using AI transfer learning to obtain a supplementary feature vector;

[0035] The feature matrix unit is used to integrate the abnormal morphology quantification value, the cell density, the spatial distribution index, the local heterogeneity index and the supplementary feature vector to obtain a quantitative feature table.

[0036] Optionally, the calculation expression of the abnormal morphology quantization value is:

[0037] CMI i =1-α·Circularity i -β·(1-Eccentricity i )

[0038] Among them, CMI i Circularity is the quantitative value of abnormal cell morphology. i is the cell roundness, Eccentricity i is the ratio of the cell's major axis to minor axis, α and β are both adjustable weights;

[0039] The calculation expression of the cell density is:

[0040]

[0041] Where D is the cell density, N total is the total number of cells, A is the visual field area;

[0042] The calculation expression of the spatial distribution index is:

[0043]

[0044] Among them, K(r) is the aggregation index, I is the indicator function, and d ij is the Euclidean distance between the i-th cell and the j-th cell, r is the spatial scale radius, w ij is the boundary correction weight, DI std is the normalized vergence index, K(r) obs is the actual observed K function value, K(r) CSR is the K function value under theoretical completely random distribution, sd CSR is the standard deviation of the K function under completely random distribution;

[0045] The calculation expression of the local heterogeneity index is:

[0046] LHI=CV(S1,S2,...,S i )

[0047] Where LHI is the local heterogeneity index, CV is the coefficient of variation, S i is the area of the Voronoi polygon for each cell.

[0048] Optionally, the rejection risk assessment module includes:

[0049] A model framework construction unit is used to combine the time series window, organize the quantitative feature table into a standardized input vector, and use the multi-category image library to train a multi-output random forest regressor to obtain a score prediction model;

[0050] A model inference unit inputs the standardized input vector into the score prediction model, introduces a regression algorithm based on ensemble learning, performs parallel calculations of acute risk, chronic risk, and infection risk, obtains a risk score, and uses SHAP to output a ranking of the impact of each feature on the risk score;

[0051] The overall risk analysis unit is used to obtain the overall risk level of the patient based on multiple frames of data of the same patient using a weighted moving average-maximum update strategy.

[0052] Optionally, the calculation expression of the overall risk level is:

[0053] R Patient =γ×max(Score frames )+(1-γ)×mean(Score frames )

[0054] Among them, R Patient is the overall risk value, max(Score frames ) is the highest risk score among all frames this time, mean(Score frames ) is the average score of all frames this time, and γ is the weight coefficient.

[0055] Optionally, the interface feedback module includes:

[0056] A visualization unit is used to load the original frame information of the current examination, the same-size position label mask, the cell density, and the risk score using a data interface to obtain multimodal information, and then use WebGL or high-performance Canvas multi-layer rendering to process the multimodal information to ensure smooth interface response;

[0057] an information overlay unit, configured to set the original frame information as a background layer to visually display a pCLE color microscopic tissue image, set the same-size position mask as a semi-transparent overlay layer to visually display cell distribution, set the cell density as an upper layer to visually display a cell density heat map, and set the risk score as a sidebar or a lower bar to visually display a risk score bar chart;

[0058] The abnormal feedback unit is used to preset the acute risk score threshold and the infection risk score threshold. When the acute risk score and the infection risk score are greater than the corresponding threshold, the abnormal warning is automatically pushed to the decision terminal.

[0059] The present invention discloses the following technical effects by providing an in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection:

[0060] 1. High image quality: Through processes such as video stabilization, artifact processing, fixed-frequency frame extraction, and normalization, it can effectively filter out invalid information, improve video consistency, and provide high-quality and stable medical image data for subsequent analysis. It realizes automated control of image sequence quality and significantly improves data utilization and analysis accuracy.

[0061] 2. Accurate cell segmentation: Through adaptive contrast enhancement, deep learning segmentation model and edge smoothing fusion, the problem that traditional methods have difficulty in distinguishing subtle cell structures is effectively solved. Accurate cell-level segmentation and edge correction are achieved, providing a reliable foundation for subsequent feature extraction.

[0062] 3. Accurate feature extraction: Utilizing advanced morphological and spatial analysis methods and introducing AI-assisted deep feature extraction, the system overcomes the limitations of previous reliance on manual and subjective interpretation, achieves higher-dimensional feature integration, and obtains comprehensive, quantitative, and intelligent cell phenotypic characteristics and spatial distribution indicators, supporting more objective quantitative analysis of disease progression.

[0063] 4. Risk Assessment Science: Combining ensemble learning with explanatory algorithms (such as SHAP), the scoring is both highly accurate and interpretable. It can scientifically quantify various risks, achieve multi-dimensional risk parallel grading and overall patient risk comprehensive management, and provide precision medicine decision support.

[0064] 5. Intuitive result feedback: Through multi-layer overlay display of images, segmentation masks, heat maps and risk scores, combined with an automatic push warning mechanism, modal and layered information visualization is achieved, examination results are presented quickly and intuitively, and abnormal warnings are fed back immediately, greatly enhancing doctors' information perception and decision-making response capabilities, and improving clinical application efficiency.

[0065] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 A schematic diagram of the system architecture provided by an embodiment of the present invention;

[0068] Figure 2 A schematic diagram of a feature extraction process according to an embodiment of the present invention;

[0069] Figure 3 A schematic diagram of the risk assessment process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0071] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0072] like Figure 1 As shown, the present invention provides an in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection reaction, including a database module, a video processing module, a cell segmentation module, a feature extraction module, a rejection risk assessment module and an interface feedback module, which are interconnected.

[0073] 1. Database module

[0074] It is used to collect relevant diagnostic data of lung transplant patients to obtain a standard data set, and perform multi-level annotation on the standard data set to obtain a multi-category library. Then, using a probe-type confocal laser endoscope, high-resolution fluorescence images of the transplanted lung bronchi and alveolar mucosa are obtained in real time to obtain original pCLE videos. The database module includes:

[0075] 1.1 Multi-category gallery unit

[0076] It is used to collect case data (covering normal, mild, moderate and severe acute rejection, chronic rejection and various types of infection) based on lung transplant patients, postoperative examination data (each examination data of each lung transplant patient at different times after surgery), pCLE images and basic clinical data (such as inflammatory markers, immunosuppressant concentrations, pathological biopsy results, imaging), obtain the original data set, and unify the image format, resolution and naming standards in the original data set to obtain a standard data set; wherein, the basic clinical data is bound one-to-one with the pCLE frames of the pCLE image.

[0077] 1.2 Annotation Unit

[0078] It is used to perform macro-category annotation on the standard data set by experts, and then use ImageJ software to perform micro-annotation on the standard data set that has completed the macro-category annotation, so as to complete the construction of a multi-category library.

[0079] The macro-category labels include: normal, mild acute rejection, moderate acute rejection, severe acute rejection, chronic rejection and infection.

[0080] The microscopic category annotation is achieved by drawing ROIs on key frames in ImageJ. That is, in video processing or animation production, the key frames in the video are marked and drawn with regions of interest (ROIs). The microscopic category annotation includes immune cell regions, tissue and vascular lesion regions, and special pathological features.

[0081] 1.3 Real-time acquisition unit

[0082] It is used to obtain high-resolution fluorescence images of the transplanted lung bronchi and alveolar mucosa in real time using a probe-type confocal laser endoscope to obtain original pCLE videos.

[0083] Within two weeks of developing pulmonary function abnormalities after lung transplantation, and at each subsequent follow-up visit, the same technical team performed the procedure to ensure consistent acquisition. A pCLE probe (2.6mm fiberoptic probe) was introduced via oral / nasal bronchoscope and gradually inserted into the CLE probe via the operating aperture. Based on chest CT scan abnormalities, pCLE alveolar imaging was performed in the affected area, and confocal real-time scanning was performed in at least four other lung segments.

[0084] 2. Video processing module

[0085] Importing the original pCLE video into the ImageJ-AI analysis center to perform video stabilization, artifact processing, fixed-frequency frame extraction, and normalization processing to obtain a normalized frame sequence and a frame sequence quality report;

[0086] The ImageJ-AI Analysis Center includes a GPU workstation, ImageJ, ImageJ self-developed plug-ins, and the TensorFlow runtime. ImageJ self-developed plug-ins are plug-ins written by users or developers based on specific needs to extend the functionality of ImageJ. These plug-ins can implement special image processing, data analysis, automated operations, etc. By writing Macros, Scripts, and Plugins, users can add new functionality to ImageJ to meet specific research needs. The TensorFlow runtime is the core component of the TensorFlow framework responsible for managing computational graph execution and resource allocation.

[0087] The video processing module includes:

[0088] 2.1 Video Import Unit

[0089] It is used to import the original pCLE video into the ImageJ-AI analysis center through a standardized upload interface, and store and manage the original pCLE video according to patient ID, acquisition time, location and detection sequence number.

[0090] 2.2 Video Stabilization Processing Unit

[0091] Based on the original PCLE video, it calls FFmpeg (an open source computer program that can be used to record, convert digital audio and video, and convert them into streams) to detect and correct horizontal, vertical, and rotational offsets to complete the initial image registration. Then, it uses an image feature extraction algorithm (such as SURF / SIFT feature point detection method) to perform local offset correction on the original PCLE video after the initial image registration to make the video sequence consistent and complete the image stabilization processing.

[0092] 2.3 Artifact Processing Unit

[0093] Based on the original pCLE video that has undergone image stabilization processing, the system uses an ImageJ macro to automatically detect brightness and contrast fluctuations in the starting frame of the video through a brightness histogram analysis method. A frame-level content recognition algorithm is then used to automatically detect whether there are shading, unfocused, reflective artifacts, and background signal clutter frames in the video, and automatically remove them to complete artifact processing.

[0094] 2.4 Still Frame Collection Unit

[0095] Based on the original pCLE video after artifact processing, the frame extraction command of ImageJ is used to uniformly extract frames at a fixed frequency (such as 5 frames / second to ensure time domain coverage and reasonable redundancy), and the differences between frames are compared to retain and mark frames with diverse content to obtain a still frame set.

[0096] 2.5 Normalized Unit

[0097] It is used to convert the still frame set into 16-bit TIFF format and embed EXIF metadata (such as acquisition time, location, stability score, key frame markers, etc.) to obtain a normalized frame sequence and a frame sequence quality report (including parameters such as the number of valid frames, mean brightness, focus index, and rejection rate), and then send the frame sequence quality report to the quality inspector for review and verification.

[0098] 3. Cell segmentation module

[0099] It is used to perform contrast enhancement, binary image segmentation and edge correction based on the normalized frame sequence to obtain a position label mask of the same size and position corresponding to the original image size and position; the cell segmentation module includes:

[0100] 3.1 Contrast Enhancement Unit

[0101] Based on the normalized frame sequence, the system uses adaptive histogram equalization technology to automatically set the block size and clipping extreme value, automatically adjust the histogram distribution according to the local block, and enhance the contrast between cells and background in each frame image to solve the problem of blurred cell edges caused by low contrast and uneven background in the original pCLE image, thereby ensuring the robustness of subsequent segmentation.

[0102] 3.2 Cell segmentation unit

[0103] It is used to integrate the U-Net++ model in the ImageJ self-developed plug-in ("pCLE-CellSeg.jar") to obtain an initial model, and use the multi-category library to train the initial model to obtain a cell segmentation model. Using the cell segmentation model, combined with instance segmentation and semantic segmentation, the normalized sequence that has completed contrast enhancement is subjected to cell segmentation to obtain a binary segmentation mask of the normalized frame sequence.

[0104] 3.3 Cell Correction Unit

[0105] It is used to use the Poisson-blending boundary smoothing algorithm to fuse the segmented area and background in the binary segmentation mask to correct the jagged, broken, over-smoothed and discontinuous phenomena of the edge and maintain the true morphology of the cell.

[0106] 3.4 Log and output unit

[0107] This outputs the edge-corrected binary segmentation mask and aligns it with the original image size and position, generating a binary label mask of the same size as each frame. This provides accurate and structured cell segmentation for subsequent quantitative analysis. The output format is unified as TIFF / PNG, with a serial number and metadata (patient ID, time, segmentation algorithm version, model confidence score, etc.).

[0108] 4. Feature extraction module

[0109] like Figure 2 As shown, it is used to perform individual cell identification based on the same-size binary label mask to obtain multiple single cells, and then perform cell morphology analysis and spatial analysis based on the single cells to calculate and integrate the abnormal morphology quantification value, cell density, spatial distribution index and local heterogeneity index to obtain a quantitative feature table; the feature extraction module includes:

[0110] 4.1 Parameter Extraction Unit

[0111] It is used to separate all individual cell areas based on the same-size binary label mask and use the connected domain analysis method to calculate the number of connected cells in each frame, and use the dynamic threshold morphological segmentation method to correct the adhesion or split artifact areas to identify single cells, complete individual cell identification, and obtain multiple single cells.

[0112] 4.2 Morphological feature extraction unit

[0113] It is used to extract the area, perimeter, roundness and major-minor axis ratio based on the single cell, and define a morphological threshold to determine the morphology of the single cell and obtain a determination result. When the result is determined to be morphologically abnormal, the degree of deviation of the morphological abnormality is calculated to obtain a quantitative value of the abnormal morphology.

[0114] The calculation expression of the abnormal morphology quantization value is:

[0115] CMI i =1-α·Circularity i -β·(1-Eccentricity i )

[0116] Among them, CMI i CMI is the quantitative value of abnormal cell morphology i The bigger, the more it deviates from the norm, Circularity i is the cell roundness, Eccentricity i is the ratio of the cell's major axis to minor axis, and α and β are both adjustable weights.

[0117] 4.3 Spatial Analysis Unit

[0118] Used to calculate cell density, spatial distribution index and local heterogeneity index based on the binary label mask of the same size;

[0119] The calculation expression of the cell density is:

[0120]

[0121] Among them, D is the cell density (which is the basis for the intensity of inflammation and rejection), N total is the total number of cells, and A is the visual field area.

[0122] The calculation expression of the spatial distribution index is:

[0123]

[0124] Among them, K(r) is the aggregation index. The higher the value, the stronger the aggregation, and the lower the value, the more uniform or sparse the clustering. I is the indicator function (I(d ij <r), if d ij < r, it is 1, otherwise it is 0, indicating whether the two points are counted into the aggregation statistics within the radius r); d ij is the Euclidean distance between the i-th cell and the j-th cell; r is the spatial scale radius; w ij Correct weights for boundaries to prevent statistical shift of points close to the edge.

[0125] DI std DI is the standardized aggregation index. Positive values represent aggregation (such as inflammatory groups), and negative values represent dispersion. The larger the absolute value, the more significant the deviation. std Reflects changes in tissue spatial structure, and is often evident in the early stages of rejection / infection; K(r) obs is the actual observed K function value; K(r) CSR is the K function value under theoretical completely random distribution; sd CSR is the standard deviation of the K function under completely random distribution.

[0126] The calculation expression of the local heterogeneity index is:

[0127] LHI=CV(S1,S2,...,S i )

[0128] Among them, LHI is the local heterogeneity index. The larger the value, the more uneven the cell distribution is, and the higher the probability of "inflammatory plaque" or abnormal structure is. CV is the coefficient of variation; S i The area of the Voronoi polygon of each cell (i.e., the nearest neighbor segmentation area centered on the cell).

[0129] 4.4 AI Assisted Unit

[0130] It is used to automatically extract high-level textures, cell clusters and edge complexity based on the same-size binary label mask and obtain a supplementary feature vector by using AI transfer learning.

[0131] 4.5 Feature Matrix Unit

[0132] Used to integrate the abnormal morphology quantification value, the cell density, the spatial distribution index, the local heterogeneity index and the supplementary feature vector to obtain a quantitative feature table.

[0133] 5. Rejection risk assessment module

[0134] like Figure 3 As shown, it is used to construct a scoring prediction model, input the quantitative feature table into the scoring prediction model, perform parallel calculations of acute risk, chronic risk and infection risk to obtain a risk score, and then calculate the patient's overall risk level based on the risk score; the rejection risk assessment module includes:

[0135] 5.1 Model framework construction unit

[0136] It is used to combine the time series window, organize the quantitative feature table into a standardized input vector, and use the multi-category image library to train a multi-output random forest regressor to obtain a score prediction model.

[0137] 5.2 Model Inference Unit

[0138] The standardized input vector is input into the score prediction model, and a regression algorithm based on ensemble learning is introduced to perform parallel calculations of acute risk, chronic risk, and infection risk to obtain a risk score. SHAP is then used to output the ranking of the impact of each feature on the risk score.

[0139] For example:

[0140] Input Xi: structured feature vector of each frame (normalized input vector, which may contain derived features across frame windows);

[0141] Output Yi:

[0142] AcuteScore: A score reflecting the risk of acute rejection, with a value range of 0-100. A higher score indicates a greater risk.

[0143] ChronicScore: reflects the risk score of "chronic rejection", with a value range of 0-100, which is predicted by the model based on the characteristics related to chronic rejection;

[0144] InfectionLikelihood: The probability that this frame / collection is similar to "infection", usually expressed as a percentage (0%-100%).

[0145] 5.3 Overall Risk Analysis Unit

[0146] It is used to obtain the patient's overall risk level based on multi-frame data of the same patient using a weighted moving average-maximum update strategy.

[0147] The calculation expression of the overall risk level is:

[0148] R Patient =γ×max(Score frames )+(1-γ)×mean(Score frames )

[0149] Among them, R Patient is the overall risk value, max(Score frames ) is the highest risk score among all frames this time, mean(Score frames ) is the average score of all frames this time, γ is the weight coefficient (usually 0.8, reflecting the sensitivity to high-risk frames), Score frames All scores for a patient collected in this or multiple frames (can be any risk score such as AcuteScore, ChronicScore, etc.)

[0150] 6. Interface feedback module

[0151] The interface feedback module is used to visually display the original frame information of this examination, the same-size position label mask, the cell density and the risk score through the decision terminal, and preset the risk score threshold to provide abnormal feedback; the interface feedback module includes:

[0152] 6.1 Visualization Unit

[0153] It is used to use the data interface to load the original frame information of this examination, the same-size position label mask, the cell density and the risk score to obtain multimodal information, and then use WebGL or high-performance Canvas multi-layer rendering to process the multimodal information to make the interface respond smoothly.

[0154] 6.2 Information Overlay Unit

[0155] Original frame information (background layer): displays the pCLE color microscopic tissue image.

[0156] Same-size position mask (translucent overlay): The segmented area is superimposed with medically friendly colors (such as bright green / blue translucent), with smooth edges and dynamically adjustable transparency of the area boundaries, making it easier for doctors to see the cell distribution at a glance.

[0157] Cell density (upper density heat map): Displays the cell density per unit area using a thermal color band (low-high, yellow-red), with key areas automatically highlighted.

[0158] Risk score (sidebar / bottom bar chart): Visually display risk score bar charts by frame or sequence, such as AcuteScore, ChronicScore, InfectionLikelihood, and other risk scores, supporting one-click comparison of historical follow-up longitudinal trends.

[0159] 6.3 Abnormal feedback unit

[0160] It is used to preset the acute risk score threshold (AcuteScore ≥ 60) and the infection risk score threshold (InfectionLikelihood ≥ 40%). When the acute risk score and infection risk score are greater than the corresponding threshold, an abnormal warning is automatically pushed to the decision-making terminal.

[0161] Therefore, the present invention provides an in vivo confocal cell quantitative analysis system for the diagnosis of lung transplant rejection reaction. Through pCLE+ImageJ+AI, it realizes cell segmentation, feature quantification and rejection scoring at the cellular level, which effectively assists in the early and accurate diagnosis and treatment of lung transplant rejection reaction, and improves the survival rate and quality of life of lung transplant patients.

[0162] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0163] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. An in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection, characterized in that: include: The database module is used to collect relevant diagnostic data of lung transplant patients to obtain a standard data set, perform multi-level annotation on the standard data set to obtain a multi-category image library, and then use a probe-type confocal laser endoscope to obtain high-resolution fluorescence images of the transplanted lung bronchi and alveolar mucosa in real time to obtain the original pCLE video; A video processing module is used to import the original pCLE video into the ImageJ-AI analysis center to perform video stabilization, artifact processing, fixed-frequency frame extraction processing and normalization processing to obtain a normalized frame sequence and a frame sequence quality report; A cell segmentation module is used to perform contrast enhancement, binary image segmentation and edge correction based on the normalized frame sequence to obtain a position label mask of the same size and position as the original image; a feature extraction module for performing individualized cell identification based on the binary label mask of the same size to obtain multiple single cells, and then performing cell morphology analysis and spatial analysis based on the single cells to calculate and integrate the abnormal morphology quantification value, cell density, spatial distribution index and local heterogeneity index to obtain a quantitative feature table; A rejection risk assessment module is used to construct a scoring prediction model, input the quantitative feature table into the scoring prediction model, perform parallel calculations of acute risk, chronic risk, and infection risk to obtain a risk score, and then calculate the patient's overall risk level based on the risk score; An interface feedback module is used to visually display the original frame information of this examination, the same-size position label mask, the cell density and the risk score through the decision terminal, and preset the risk score threshold to provide abnormal feedback; Wherein, the database module, the video processing module, the cell segmentation module, the feature extraction module, the rejection risk assessment module and the interface feedback module are interconnected.

2. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 1, characterized in that: The ImageJ-AI analysis center includes a GPU workstation, ImageJ, ImageJ self-developed plug-ins and TensorFlow runtime.

3. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 2, characterized in that: The database module includes: A multi-category image library unit is used to collect case data, postoperative examination data, pCLE images and basic clinical data based on lung transplant patients to obtain an original data set, and unify the image format, resolution and naming standards in the original data set to obtain a standard data set; wherein the basic clinical data is bound to the pCLE frame of the pCLE image; The labeling unit is used to perform macro-category labeling on the standard data set by experts, and then use ImageJ software to perform micro-labeling on the standard data set that has completed the macro-category labeling to complete the construction of a multi-category library; wherein, the macro-category labeling includes normal, mild acute rejection, moderate acute rejection, severe acute rejection, chronic rejection and infection, and the micro-category labeling includes immune cell areas, tissue and vascular lesion areas and special pathological features. The real-time acquisition unit is used to acquire high-resolution fluorescence images of the transplanted lung bronchus and alveolar mucosa in real time using a probe-type confocal laser endoscope to obtain original pCLE videos.

4. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 3, characterized in that: The video processing module includes: A video import unit, configured to import the original pCLE video into the ImageJ-AI analysis center through a standardized upload interface, and to store and manage the original pCLE video according to the patient ID, acquisition time, location, and detection sequence number; A video stabilization processing unit is configured to call FFmpeg based on the original PCLE video to detect and correct horizontal, vertical, and rotational offsets to complete primary image registration, and then use an image feature extraction algorithm to perform local offset correction on the original PCLE video after the primary image registration to make the video sequence consistent and complete image stabilization processing; An artifact processing unit is used to automatically detect brightness and contrast fluctuations of the starting frame of the video using an ImageJ macro and a brightness histogram analysis method based on the original pCLE video that has completed the image stabilization processing, and automatically detect whether there are shading, out of focus, reflection artifacts and background signal clutter frames in the video using a frame-level content recognition algorithm to complete artifact processing; A still frame collection unit is configured to uniformly extract frames at a fixed frequency based on the original pCLE video after artifact processing using a frame extraction command of ImageJ, and compare differences between frames to retain and mark frames with diverse content, thereby obtaining a still frame collection; A normalization unit is used to convert the still frame set into 16-bit TIFF format and embed EXIF metadata to obtain a normalized frame sequence and a frame sequence quality report, and then send the frame sequence quality report to a quality inspector for review and verification; the frame sequence quality report includes the number of valid frames, average brightness, focus index and rejection rate.

5. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 4, characterized in that: The cell segmentation module includes: a contrast enhancement unit, configured to enhance the contrast between cells and background in each frame of the image by using an adaptive histogram equalization technique based on the normalized frame sequence; A cell segmentation unit is used to integrate a U-Net++ model into the ImageJ self-developed plug-in to obtain an initial model, and train the initial model using the multi-category library to obtain a cell segmentation model. Using the cell segmentation model, combined with instance segmentation and semantic segmentation, the normalized sequence that has completed contrast enhancement is subjected to cell segmentation to obtain a binary segmentation mask of the normalized frame sequence; A cell correction unit is used to fuse the segmented area and the background in the binary segmentation mask using a Poisson-blending boundary smoothing algorithm to correct jagged edges, breaks, over-smoothing, and discontinuities. The log and output unit is used to output the binary segmentation mask after edge correction, and correspond the output binary segmentation mask to the original image size and position to obtain a binary label mask of the same size.

6. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 5, characterized in that: The feature extraction module includes: A parameter extraction unit is used to separate all individual cell regions based on the binary label mask of the same size using a connected domain analysis method to calculate the number of connected cells in each frame, and to correct the adhesion or split artifact regions using a dynamic threshold morphological segmentation method to identify single cells, complete individual cell identification, and obtain multiple single cells; a morphological feature extraction unit, configured to extract the area, perimeter, roundness, and major-minor axis ratio of the single cell, respectively, and define a morphological threshold to determine the morphology of the single cell and obtain a determination result; and when the result is determined to be morphologically abnormal, calculate the degree of deviation of the morphological abnormality to obtain a quantitative value of the abnormal morphology; a spatial analysis unit, configured to calculate cell density, spatial distribution index, and local heterogeneity index based on the binary label mask of the same size; An AI auxiliary unit is configured to automatically extract high-level textures, cell clusters, and edge complexity based on the binary label mask of the same size using AI transfer learning to obtain a supplementary feature vector; The feature matrix unit is used to integrate the abnormal morphology quantification value, the cell density, the spatial distribution index, the local heterogeneity index and the supplementary feature vector to obtain a quantitative feature table.

7. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 6, characterized in that: The calculation expression of the abnormal morphology quantization value is: CMI i =1-α·Circularity i -β·(1-Eccentricity i ) Among them, CMI i Circularity is the quantitative value of abnormal cell morphology. i is the cell roundness, Eccentricity i is the ratio of the cell's major axis to minor axis, α and β are both adjustable weights; The calculation expression of the cell density is: Where D is the cell density, N total is the total number of cells, A is the visual field area; The calculation expression of the spatial distribution index is: Among them, K(r) is the aggregation index, I is the indicator function, and d ij is the Euclidean distance between the i-th cell and the j-th cell, r is the spatial scale radius, w ij is the boundary correction weight, DI std is the normalized vergence index, K(r) obs is the actual observed K function value, K(r) CSR is the K function value under theoretical completely random distribution, sd CSR is the standard deviation of the K function under completely random distribution; The calculation expression of the local heterogeneity index is: <h2 style=";text-align:left;direction:ltr">LHI=CV(S1,S2,...,S<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> ) Where LHI is the local heterogeneity index, CV is the coefficient of variation, S i is the area of the Voronoi polygon for each cell.

8. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 7, characterized in that: The rejection risk assessment module includes: A model framework construction unit is used to combine the time series window, organize the quantitative feature table into a standardized input vector, and use the multi-category image library to train a multi-output random forest regressor to obtain a score prediction model; A model inference unit inputs the standardized input vector into the score prediction model, introduces a regression algorithm based on ensemble learning, performs parallel calculations of acute risk, chronic risk, and infection risk, obtains a risk score, and uses SHAP to output a ranking of the impact of each feature on the risk score; The overall risk analysis unit is used to obtain the overall risk level of the patient based on multiple frames of data of the same patient using a weighted moving average-maximum update strategy.

9. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 8, characterized in that: The calculation expression of the overall risk level is: R Patient =γ×max(Score frames )+(1-γ)×mean(Score frames ) Among them, R Patient is the overall risk value, max(Score frames ) is the highest risk score among all frames this time, mean(Score frames ) is the average score of all frames this time, and γ is the weight coefficient.

10. The in vivo confocal cell quantitative analysis system for diagnosing lung transplant rejection according to claim 9, characterized in that: The interface feedback module includes: A visualization unit is used to load the original frame information of the current examination, the same-size position label mask, the cell density, and the risk score using a data interface to obtain multimodal information, and then use WebGL or high-performance Canvas multi-layer rendering to process the multimodal information to ensure smooth interface response; an information overlay unit, configured to set the original frame information as a background layer to visually display a pCLE color microscopic tissue image, set the same-size position mask as a semi-transparent overlay layer to visually display cell distribution, set the cell density as an upper layer to visually display a cell density heat map, and set the risk score as a sidebar or a lower bar to visually display a risk score bar chart; The abnormal feedback unit is used to preset the acute risk score threshold and the infection risk score threshold. When the acute risk score and the infection risk score are greater than the corresponding threshold, the abnormal warning is automatically pushed to the decision terminal.

Images