A cle combined with me-nbi multi-modal endoscopic image early esophageal cancer grading and treatment decision system

Abstract

The present application belongs to the technical field of tumor diagnosis system, and particularly relates to a CLE combined with ME-NBI multi-modal endoscopic image early esophageal cancer grading and treatment decision system. The system comprises an image access module, an ME-NBI lesion coordinate establishment module, a CLE sampling point mapping module, an image evidence stratification module, a cross-modal feature recording module, an image grade determination module, an auxiliary path prompting module and a result output module. Based on the lesion outer edge line, the longest diameter axis and the like, the CLE sampling points are mapped to the corresponding coordinate positions, and the image quality grades and the like are generated. According to this, the lesion image grade label, the evidence credibility mark, the review prompt mark and the auxiliary path prompt label are outputted. The macroscopic endoscopic observation and the microscopic CLE sampling form a spatial corresponding relationship, the traceability of the early esophageal lesion grading result is improved, and the auxiliary basis for review sampling, additional biopsy, endoscopic resection or comprehensive treatment evaluation is provided.

Description

Technical Field

[0001] This invention belongs to the technical field of tumor diagnostic systems, specifically relating to a CLE combined with ME-NBI multimodal endoscopic imaging system for early esophageal cancer grading and treatment decision-making. Background Technology

[0002] Esophageal cancer is one of the most common malignant tumors of the digestive tract. Early lesions often occur in the esophageal mucosa or superficial submucosa, with subtle lesions and atypical clinical presentations. With the development of endoscopic imaging technology, high-definition white light endoscopy, narrow-band imaging magnification endoscopy, and confocal laser microscopy are gradually being applied to the observation of esophageal mucosal lesions. These technologies can provide imaging evidence for the identification of early esophageal cancer and precancerous lesions from the perspectives of mucosal surface morphology, microvascular structure, glands, or cells.

[0003] In existing technologies, the endoscopic diagnosis of early esophageal cancer typically relies on white light endoscopy for initial screening, iodine staining or narrow-band imaging for auxiliary observation, and pathological biopsy for definitive diagnosis. Some medical institutions further employ ME-NBI to observe changes in microvascular morphology and mucosal structure on the lesion surface, and use information such as vascular morphology classification, lesion extent, and invasion depth to help determine the lesion grade. Other technologies attempt to introduce CLE for real-time cellular imaging of suspicious areas to improve the accuracy of identifying dysplasia, carcinoma in situ, or early invasive cancer. However, these approaches still largely depend on physician experience and subjective judgment of single or scattered images, lacking a unified framework for image registration, feature correspondence, and comprehensive evaluation between different imaging modalities. ME-NBI excels at displaying microvascular and mucosal surface structures, but has limited ability to assess local cell morphology and glandular details; CLE can provide subtle information at a near-historical level, but its observation range is small and easily affected by sampling location and image quality. Existing systems often only output lesion identification or grading results, making it difficult to link lesion extent, invasion risk, image reliability, and treatment indications. This results in a lack of continuous and objective decision support between early esophageal cancer grading and endoscopic treatment, surgical treatment, or follow-up strategies.

[0004] Therefore, it is necessary to establish a multimodal image comprehensive analysis scheme for the endoscopic diagnosis and treatment of early esophageal cancer, so that different endoscopic image information can form a complementary judgment relationship and provide a more stable and clear technical basis for lesion grading and treatment selection. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide: a CLE combined with ME-NBI multimodal endoscopic imaging system for early esophageal lesion auxiliary grading and pathway indication, comprising: The image access module receives ME-NBI image sequences and CLE image sequences of the same esophageal lesion from the same subject, and records the distance of the lesion from the incisors, circumferential orientation, magnification, acquisition time, image frame number, and CLE sampling point number. The ME-NBI lesion coordinate establishment module marks the outer edge of the lesion, the bifurcation point of the blood vessel, the mucosal texture change point, the color change point and the suspected infiltration key area in the ME-NBI image, and establishes the lesion coordinate area with the outer edge of the lesion, the longest diameter axis and the short diameter axis; The CLE sampling point mapping module uses two vascular bifurcation points and one mucosal texture mutation point or color mutation point to form a three-point reference area, maps the CLE sampling point to the lesion coordinate area, and records the distance deviation, orientation deviation and coverage status between the sampling point and the suspected infiltration key area. The image evidence stratification module generates ME-NBI quality level, CLE quality level, and sampling coverage level. The cross-modal feature recording module records ME-NBI microvascular features, ME-NBI surface structural features, CLE cell morphology features, and CLE structural boundary features; The image grading module generates lesion image grading labels, evidence credibility markers, and review prompt markers based on contiguous evidence, adjacent evidence, low-confidence evidence, and uncovered markers. The auxiliary path suggestion module generates auxiliary path suggestion labels; The results output module outputs lesion coordinate map, lesion image level label, evidence credibility mark, review prompt mark, and auxiliary path prompt label.

[0006] As a preferred technical solution, the ME-NBI lesion coordinate establishment module includes: The outer edge calibration unit generates the outer edge of the lesion by changing the boundary along the background coloring, the boundary of microvascular morphological mutation, and the boundary of mucosal texture interruption. Reference point calibration unit, calibrating blood vessel bifurcation points, mucosal texture change points, color change points, and boundary inflection points; The coordinate axis generation unit generates the longest radial axis in the direction of the maximum distance within the outer edge of the lesion, and generates the short radial axis in the direction that passes through the midpoint of the longest radial axis and is perpendicular to the longest radial axis. The lesion coordinate area is divided into central area, edge area, transition area and suspected infiltration key area by combining the outer edge line of the lesion, the longest diameter axis and the short diameter axis, and a coordinate number is set for each zone.

[0007] As a preferred technical solution, the CLE sampling point mapping module includes: The three-point reference region generation unit selects two adjacent blood vessel bifurcation points and one mucosal texture mutation point or color mutation point to form a three-point reference region. The sampling point writing unit writes the CLE sampling point number, CLE image frame number, probe contact time, and corresponding ME-NBI image frame number into the position of the corresponding three reference points in the lesion coordinate area. The deviation recording unit records the distance deviation of the CLE sampling point relative to the center of the three reference areas and the orientation deviation relative to the longest radial axis; The coverage marker unit generates a coverage marker for the key area when the CLE sampling point is located within the suspected infiltration key area, or when the distance between the sampling point and the boundary of the suspected infiltration key area is no more than 1 mm; otherwise, it generates an uncovered marker for the key area.

[0008] As a preferred technical solution, the image evidence layering module includes: The ME-NBI quality recording unit records the proportion of reflective occlusion area, the proportion of motion-blurred area, the proportion of microvascular identifiable area, and the continuity ratio of the outer edge of the lesion. Based on the recording results, the ME-NBI quality level is generated. The CLE quality recording unit records the proportion of areas with uneven fluorescence brightness, the proportion of areas with discernible cell outlines, the proportion of areas with probe compression artifacts, and the number of consecutive effective frames, and generates a CLE quality level based on the recording results. The sampling coverage level recording unit generates a level 1, 2, or 3 sampling coverage level based on the coverage of CLE sampling points in the central area, edge area, and suspected infiltration key area.

[0009] As a preferred technical solution, the cross-modal feature recording module includes: The ME-NBI vascular feature table records the coordinate numbers of dilated vessels, serpentine vessels, reticular vessels, annular vessels, and vascular rupture zones. ME-NBI surface structure feature table records the coordinate numbers of mucosal texture disappearance area, surface uneven area, background color deepening area and boundary blurring area; CLE cell characteristic table records the coordinate numbers of areas with uneven cell size, disordered cell arrangement, increased dark areas of cell nuclei, and altered intercellular spaces. The CLE structural boundary feature table records the coordinate numbers of the glandular structure boundary interruption region, the epithelial structure blur region, and the fluorescence leakage region.

[0010] As a preferred technical solution, the image level determination module includes: The co-existence evidence determination unit identifies records containing both ME-NBI microvascular abnormality features and CLE microstructural abnormality features within the same coordinate number as co-existence evidence. The proximity evidence determination unit identifies records with ME-NBI microvascular abnormality features and CLE microstructural abnormality features respectively within adjacent coordinate numbers as proximity evidence. The low-confidence evidence recording unit generates low-confidence evidence and review prompts when the ME-NBI quality level or CLE quality level is level three, the sampling coverage level is level three, or the CLE sampling point does not cover the suspected infiltration key area. The grade label generation unit generates low-level abnormality image labels, high-level abnormality image labels, mucosal cancer-prone image labels, or suspected submucosal infiltration image labels.

[0011] As a preferred technical solution, the auxiliary path prompting module includes: The lesion extent reading unit reads the lesion's long diameter, short diameter, area, and circumferential proportion. The key area reading unit reads the coordinate numbers of suspected infiltration key areas, as well as the corresponding ME-NBI microvascular features and CLE microstructure features; The coverage status reading unit reads whether the CLE sampling points cover the central area, edge area, and suspected infiltration key area; The path labeling unit generates labels for regular follow-up examinations, additional biopsy review, endoscopic mucosal resection, endoscopic submucosal dissection, or surgical and comprehensive treatment assessment.

[0012] As a preferred technical solution, the path labeling unit records a periodic review reminder label or an additional biopsy review reminder label when the lesion image level label is a low-level abnormal image label and the sampling coverage level is level one or level two.

[0013] As a preferred technical solution, when the lesion image grade label is a high-level abnormal image label or an intramucosal carcinoma tendency image label, and the outer edge of the lesion is continuous and the CLE sampling point covers the central and marginal areas, the candidate prompt label for endoscopic mucosal resection or endoscopic submucosal dissection is recorded.

[0014] As a preferred technical solution, when the lesion image grade label is a suspected submucosal infiltration image label, or when there is isotopic evidence in the suspected infiltration key area, a surgical and comprehensive treatment assessment prompt label is recorded.

[0015] As a preferred technical solution, when the CLE sampling point does not cover the suspected infiltration key area, a verification sampling prompt label is added in addition to the auxiliary path prompt label.

[0016] The method of using the CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway guidance system includes: S1. Acquire ME-NBI image sequences and CLE image sequences of the same esophageal lesion in the same subject, and record the distance of the lesion from the incisors, circumferential orientation, magnification, acquisition time, image frame number and CLE sampling point number; S2. In ME-NBI images, mark the outer edge of the lesion, the bifurcation point of the blood vessel, the mucosal texture change point, the color change point, and the suspected key area of ​​invasion. Establish the lesion coordinate area with the outer edge of the lesion, the longest diameter axis, and the short diameter axis, and divide the central area, the edge area, the transition area, and the suspected key area of ​​invasion. S3. A three-point reference area is formed by two adjacent blood vessel bifurcation points and one mucosal texture mutation point or color mutation point. The CLE sampling point number, CLE image frame number, probe contact time and corresponding ME-NBI image frame number are written into the lesion coordinate area, and the distance deviation, orientation deviation and coverage status are recorded. S4. Record ME-NBI microvascular characteristics, ME-NBI surface structural characteristics, CLE cell morphology characteristics, and CLE structural boundary characteristics, and generate ME-NBI quality grade, CLE quality grade, and sampling coverage grade; S5. Generate lesion image grade labels, evidence credibility labels, and review prompt labels based on co-located evidence, adjacent evidence, low-confidence evidence, and uncovered markers; S6. Generate auxiliary path prompt labels based on lesion image level labels, lesion range, suspected infiltration key areas and CLE sampling coverage level, and output lesion coordinate map, evidence credibility mark, review prompt mark and auxiliary path prompt labels.

[0017] As a preferred technical solution, in step S5: When both ME-NBI microvascular abnormalities and CLE microstructural abnormalities exist within the same coordinate number, isotopic evidence is formed. When ME-NBI microvascular abnormality features and CLE microstructural abnormality features exist in adjacent coordinate numbers respectively, they form proximity evidence; when the ME-NBI quality level or CLE quality level is three, the corresponding image feature record is low confidence evidence. When a CLE sampling point does not cover a suspected infiltration hotspot area, or when the distance between a CLE sampling point and the boundary of a suspected infiltration hotspot area exceeds 1 mm, an uncovered mark is formed. When the co-existing evidence is located in the suspected infiltration key area and there is no low-confidence evidence, a higher confidence marker is generated; When there is adjacent evidence, low-confidence evidence, or uncovered markers, the lesion image grade label and review prompt label are output together.

[0018] Beneficial effects: This invention establishes a lesion coordinate region by using the outer edge of the lesion, the longest diameter axis, the short diameter axis, as well as the bifurcation points of blood vessels, mucosal texture mutation points, and color mutation points in ME-NBI images. CLE sampling points are then mapped into this coordinate region, creating a traceable spatial correspondence between the macroscopic lesion range of ME-NBI and the microscopic sampling location of CLE. This reduces the positional deviation caused by relying solely on image frame order or textual descriptions of sampling locations, enabling subsequent lesion grading to be determined based on clear image coordinates and sampling coverage.

[0019] This invention establishes ME-NBI quality levels, CLE quality levels, and sampling coverage levels, and further distinguishes between co-located evidence, adjacent evidence, low-confidence evidence, and uncovered markers. This allows for confidence stratification of different modalities of image evidence before lesion grading. Consequently, the system not only records the abnormal image features themselves but also simultaneously records the corresponding image quality and sampling adequacy. This avoids directly assigning excessively high grades to low-quality images or uncovered key areas, making image grading results easier to verify and trace.

[0020] This invention uses lesion image level labels, suspected infiltrative key areas, lesion extent, and CLE sampling coverage level as auxiliary path prompts. While outputting lesion coordinate maps and evidence credibility markers, it generates prompts such as review sampling, additional biopsy, endoscopic resection candidates, or surgical and comprehensive treatment assessments. This approach ensures that path prompts are not based solely on the degree of a single image abnormality, but rather combine spatial coverage, key area evidence, and evidence credibility status, facilitating subsequent judgments by clinicians based on key image evidence. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the first-level structure of the system of the present invention; Figure 2 This is a schematic diagram of the secondary structure of the ME-NBI lesion coordinate establishment module of the present invention; Figure 3 This is a schematic diagram of the secondary structure of the CLE sampling point mapping module of the present invention; Figure 4 This is a schematic diagram of the two-level structure of the image evidence layering module of the present invention; Figure 5 This is a schematic diagram of the secondary structure of the cross-modal feature recording module of the present invention; Figure 6 This is a schematic diagram of the secondary structure of the image level determination module of the present invention. Detailed Implementation

[0022] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0023] Example 1 This embodiment provides a multimodal endoscopic imaging system for early esophageal lesion grading and pathway guidance using CLE combined with ME-NBI. This system receives ME-NBI and CLE image sequences of the same esophageal lesion from the same subject. It spatially correlates and stratifies the image information from the ME-NBI images (reflecting lesion extent, surface morphology, and microvascular changes) with the image information from the CLE images (reflecting cell morphology, microstructural boundaries, and local tissue structure changes), thereby outputting a lesion coordinate map, lesion image grade labels, evidence credibility markers, review prompt markers, and auxiliary pathway guidance labels. This system does not directly replace physicians in making pathological diagnoses or treatment decisions; rather, it organizes, labels, grades, and provides guidance on endoscopic image evidence, giving the image review process clear spatial basis and evidence sources.

[0024] like Figure 1 As shown, the system includes an image access module, a ME-NBI lesion coordinate establishment module, a CLE sampling point mapping module, an image evidence stratification module, a cross-modal feature recording module, an image grade determination module, an auxiliary path prompting module, and a result output module. Data is transferred between the modules in the following order: image access, lesion coordinate establishment, CLE sampling point mapping, image quality and coverage grade recording, cross-modal abnormal feature recording, lesion image grade determination, auxiliary path prompt generation, and result output.

[0025] Among them, such as Figure 2 As shown, the lesion coordinate region output by the ME-NBI lesion coordinate establishment module serves as a unified spatial benchmark for subsequent CLE sampling point mapping, abnormal feature recording, and grade determination; the ME-NBI quality level, CLE quality level, and sampling coverage level output by the image evidence stratification module serve as the basis for the image grade determination module to judge the credibility of the evidence; and the various abnormal image features output by the cross-modal feature recording module serve as the basis for generating co-located evidence, adjacent evidence, and suspected infiltration key area evidence.

[0026] The image acquisition module receives ME-NBI and CLE image sequences. ME-NBI image sequences can be continuous image frames obtained through narrowband imaging combined with magnifying endoscopy, while CLE image sequences can be continuous image frames obtained through confocal laser microscopy at multiple sampling points. When receiving images, the image acquisition module simultaneously records the distance of the lesion from the incisors, circumferential orientation, magnification, acquisition time, image frame number, and CLE sampling point number. The distance of the lesion from the incisors describes the longitudinal position of the lesion in the esophagus, the circumferential orientation describes the circumferential position of the lesion in the esophageal lumen, the image frame number binds different image frames to their corresponding acquisition time and location, and the CLE sampling point number distinguishes different local microscopic sampling locations. This unified access to the above information prevents ME-NBI and CLE images from deviating from the same lesion source during subsequent analysis.

[0027] The ME-NBI lesion coordinate establishment module is used to establish lesion coordinate regions in ME-NBI images. This module includes an outer edge calibration unit, a reference point calibration unit, a coordinate axis generation unit, and a partition numbering unit. The outer edge calibration unit generates the outer edge of the lesion along the background color change boundary, the microvessel morphology change boundary, and the mucosal texture interruption boundary. The background color change boundary is mainly used to describe the color difference between the lesion area and the surrounding normal mucosa; the microvessel morphology change boundary is mainly used to describe the location where the vessel morphology changes from regular to dilated, serpentine, broken, or circular arrangement; the mucosal texture interruption boundary is mainly used to describe the location where the continuity of the mucosal surface texture is interrupted or disappears. The outer edge calibration unit can superimpose the above boundary information to form a closed or nearly closed lesion outer edge, and use the lesion outer edge as the outer boundary of the lesion coordinate region.

[0028] The reference point calibration unit is used to calibrate vascular bifurcation points, mucosal texture abrupt changes, color abrupt changes, and boundary inflection points in ME-NBI images. Vascular bifurcation points provide a stable vascular morphological reference for subsequent CLE sampling point mapping; mucosal texture abrupt changes and color abrupt changes help determine the relative position of CLE sampling points within the lesion coordinate region; and boundary inflection points improve the positioning accuracy of the lesion's outer edge in irregular lesion morphologies. The coordinate axis generation unit generates the longest radial axis along the direction of maximum distance within the lesion's outer edge and generates a shorter radial axis in a direction passing through the midpoint of the longest radial axis and perpendicular to it.

[0029] The longest radial axis is used to characterize the main extension direction of the lesion, and the shortest radial axis is used to characterize the lateral width of the lesion. The partition numbering unit, combining the lesion's outer edge, longest radial axis, and shortest radial axis, divides the lesion coordinate area into a central region, a peripheral region, a transition region, and a suspected infiltration focus region, and assigns coordinate numbers to each region. The central region corresponds to a relatively stable area within the lesion, the peripheral region corresponds to the area near the lesion's outer edge, the transition region corresponds to the image variation area between the central and peripheral regions, and the suspected infiltration focus region corresponds to areas in the ME-NBI image with significant microvascular disturbances, abnormal surface irregularities, or blurred boundaries.

[0030] like Figure 3 As shown, the CLE sampling point mapping module is used to establish a correspondence between CLE sampling points and ME-NBI lesion coordinate regions. This module includes a three-point reference area generation unit, a sampling point writing unit, a deviation recording unit, and a coverage marker unit. The three-point reference area generation unit selects two adjacent vessel bifurcation points and one mucosal texture mutation point or color mutation point to form a three-point reference area.

[0031] This three-point reference area does not rely on a single image point for localization. Instead, it utilizes vascular bifurcation and mucosal texture or color changes to form a local spatial reference, enabling CLE sampling points to be more stably mapped to the ME-NBI lesion coordinate area. The sampling point writing unit writes the CLE sampling point number, CLE image frame number, probe contact time, and corresponding ME-NBI image frame number into the position of the corresponding three-point reference area in the lesion coordinate area, ensuring that each CLE sampling point has a corresponding ME-NBI spatial location and image frame source.

[0032] The deviation recording unit records the distance deviation of the CLE sampling point relative to the center of the three reference areas and the azimuth deviation relative to the longest radial axis. The distance deviation reflects the positional difference between the CLE sampling point and the center of the local reference area, while the azimuth deviation reflects the degree of offset of the CLE sampling point relative to the main extension direction of the lesion. The coverage marking unit determines whether the CLE sampling point covers the suspected infiltration focus area. When the CLE sampling point is located within the suspected infiltration focus area, or the distance to the boundary of the suspected infiltration focus area is no greater than 1 mm, the coverage marking unit generates a focus area coverage mark; when the CLE sampling point is not located within the suspected infiltration focus area, and the distance to the boundary of the suspected infiltration focus area is greater than 1 mm, the coverage marking unit generates a focus area uncovered mark. Through this setting, the system can distinguish whether the CLE image truly covers the key abnormal area indicated by ME-NBI, avoiding the assumption that the key area has obtained microscopic image evidence simply because a CLE image exists.

[0033] like Figure 4As shown, the image evidence stratification module is used to record ME-NBI image quality, CLE image quality, and CLE sampling coverage in a stratified manner. This module includes a ME-NBI quality recording unit, a CLE quality recording unit, and a sampling coverage level recording unit. The ME-NBI quality recording unit records the proportion of reflective occlusion area, the proportion of motion-blurred area, the proportion of identifiable microvessels, and the continuity ratio of the lesion's outer edge, and generates a ME-NBI quality level based on the recording results.

[0034] For example, a higher quality grade can be recorded when the proportion of reflective obstruction area is low, the proportion of motion-blurred area is low, the proportion of identifiable microvessels is high, and the proportion of continuous lesion margins is high; a lower quality grade can be recorded when reflective obstruction or motion blur affects microvessel observation, or when the lesion margins are discontinuous. This quality grade is used to subsequently determine whether the abnormal ME-NBI features constitute credible evidence.

[0035] The CLE quality recording unit records the percentage of areas with uneven fluorescence brightness, the percentage of areas with discernible cell outlines, the percentage of areas affected by probe compression artifacts, and the number of consecutive valid frames, and generates a CLE quality grade based on the recording results. The percentage of areas with uneven fluorescence brightness reflects whether the brightness distribution within the CLE image field of view is uniform; the percentage of areas with discernible cell outlines reflects whether the cell morphology is clear; the percentage of areas affected by probe compression artifacts reflects the degree of local deformation or artifacts caused by probe contact; and the number of consecutive valid frames reflects whether there are sufficiently continuous interpretable images at the same sampling point.

[0036] The sampling coverage level recording unit generates a Level 1, Level 2, or Level 3 sampling coverage level based on the coverage of CLE sampling points in the central area, edge area, and suspected infiltration key areas. A Level 1 sampling coverage level indicates that CLE sampling points cover the central area, edge area, and suspected infiltration key areas; a Level 2 sampling coverage level indicates that CLE sampling points cover the central area and edge area, but do not fully cover suspected infiltration key areas; a Level 3 sampling coverage level indicates insufficient CLE sampling point coverage, or that key areas have not been effectively sampled.

[0037] like Figure 5 As shown, the cross-modal feature recording module is used to record abnormal image features in both ME-NBI and CLE modalities. This module includes the ME-NBI vascular feature table, ME-NBI surface structure feature table, CLE cell feature table, and CLE structural boundary feature table. The ME-NBI vascular feature table records the coordinates of dilated vessels, serpentine vessels, reticular vessels, annular vessels, and vascular rupture zones.

[0038] The ME-NBI surface structure feature table records the coordinates of areas where mucosal texture disappears, surface unevenness, background darkening, and blurred boundaries. The CLE cell feature table records the coordinates of areas with uneven cell size, disordered cell arrangement, increased dark areas of cell nuclei, and altered intercellular spaces. The CLE structural boundary feature table records the coordinates of areas with interrupted glandular structure boundaries, blurred epithelial structures, and fluorescence leakage. These features are not recorded as isolated image text, but are all bound to coordinates within the lesion's coordinate region, enabling comparison of ME-NBI and CLE abnormal features within a unified coordinate system.

[0039] like Figure 6 As shown, the image grading module is used to generate lesion image grading labels, evidence credibility markers, and review prompt markers based on cross-modal image evidence. This module includes a territorial evidence grading unit, a proximity evidence grading unit, a low-credibility evidence recording unit, and a grading label generation unit. The territorial evidence grading unit identifies records containing both ME-NBI microvascular abnormalities and CLE microstructural abnormalities within the same coordinate number as territorial evidence.

[0040] For example, if a coordinate number simultaneously records a region of vascular rupture and a region of disordered cell arrangement, or simultaneously records a region of annular vessels and a region of blurred epithelial structure, the cross-modal abnormal record corresponding to that coordinate number can be identified as isotopic evidence. The proximity evidence determination unit identifies records with ME-NBI microvascular abnormalities and CLE microstructural abnormalities, respectively, within adjacent coordinate numbers as proximity evidence. For example, if an ME-NBI image records serpentine vessels within one coordinate number, and a CLE image records a region of uneven cell size within an adjacent coordinate number, this record can be identified as proximity evidence.

[0041] The low-confidence evidence recording unit is used to generate low-confidence evidence and review prompts when image quality or sampling coverage is insufficient. When the ME-NBI quality level or CLE quality level is level three, the sampling coverage level is level three, or the CLE sampling points do not cover the suspected infiltration key area, the low-confidence evidence recording unit records the corresponding image features as low-confidence evidence and generates a review prompt.

[0042] The review prompt marker can be bound to specific coordinate numbers, image frame numbers, or CLE sampling point numbers, enabling reviewers to quickly locate the image area requiring review. The grade label generation unit generates low-grade abnormality image labels, high-grade abnormality image labels, intramucosal carcinoma tendency image labels, or suspected submucosal invasion image labels based on synodic evidence, adjacent evidence, low-confidence evidence, and uncovered markers. When synodic evidence is located in a suspected invasion focus area and no low-confidence evidence exists, a higher-confidence marker can be generated; when adjacent evidence, low-confidence evidence, or uncovered markers exist, the lesion image grade label and the review prompt marker are output together to indicate that the grade judgment requires supplementary sampling or manual review.

[0043] The auxiliary path prompting module generates auxiliary path prompt labels based on lesion image level labels, lesion extent, suspected infiltrative key areas, and CLE sampling coverage level. This module includes a lesion extent reading unit, a key area reading unit, a coverage status reading unit, and a path label unit. The lesion extent reading unit reads the lesion's long diameter, short diameter, area, and circumferential percentage.

[0044] The key area reading unit reads the coordinates of suspected infiltration key areas, along with the corresponding ME-NBI microvascular features and CLE microstructural features. The coverage status reading unit reads whether the CLE sampling points cover the central area, the peripheral area, and suspected infiltration key areas. The path labeling unit generates, based on the above information, labels for regular follow-up, additional biopsy review, endoscopic mucosal resection, endoscopic submucosal dissection, or surgical and comprehensive treatment assessment.

[0045] In a specific operating state, when the lesion image level label is a low-level abnormal image label and the sampling coverage level is level one or two, the path label unit records a regular review prompt label or an additional biopsy review prompt label; when the lesion image level label is a high-level abnormal image label or an intramucosal carcinoma tendency image label, and the lesion outer edge is continuous and the CLE sampling point covers the central and marginal areas, it records an endoscopic mucosal resection candidate prompt label or an endoscopic submucosal dissection candidate prompt label; when the lesion image level label is a suspected submucosal invasion image label, or when there is isotopic evidence in the suspected invasion key area, it records a surgical and comprehensive treatment assessment prompt label; when the CLE sampling point does not cover the suspected invasion key area, a review sampling prompt label is added in addition to the auxiliary path prompt label.

[0046] The above path labels are only used as supplementary prompts based on image evidence. The final processing method is still determined by clinicians in combination with pathology, patient condition and other examination results.

[0047] The results output module outputs lesion coordinate maps, lesion image grade labels, evidence credibility markers, review prompt markers, and auxiliary path prompt labels. The lesion coordinate map displays the lesion's outer edge, longest axis, shortest axis, central area, marginal area, transition area, suspected infiltration focus area, CLE sampling point location, and the coordinate numbers corresponding to each abnormal feature. The lesion image grade labels display the image grading results generated by the system based on cross-modal image evidence; the evidence credibility markers indicate the credibility status of the evidence upon which the grading result is based; the review prompt markers indicate areas with low-credibility evidence, insufficient sampling, or uncovered focus areas; and the auxiliary path prompt labels provide auxiliary prompts related to subsequent follow-up examinations, additional biopsies, endoscopic resection candidates, or surgical and comprehensive treatment assessments.

[0048] Through the system architecture of this embodiment, the macroscopic lesion extent, surface structure, and microvascular abnormalities in ME-NBI images can be correlated with the local cell and structural boundary information in CLE images within the same lesion coordinate region. This allows image evidence originally scattered across different image modalities, sampling points, and frame numbers to be organized into a structured record with coordinate numbers, quality levels, and coverage status. Simultaneously, the system incorporates isotopic evidence, proximity evidence, low-confidence evidence, and uncovered markers into the grading determination, ensuring that the lesion image grade labels not only reflect the abnormal image features themselves but also the spatial correspondence of those features, image quality, and sampling adequacy. Therefore, the lesion coordinate map and related prompts output by the results output module can provide a clearer image evidence basis for image review, sampling supplementation, and path selection in early esophageal lesions.

[0049] Example 2 This embodiment provides a method for early esophageal lesion auxiliary grading and pathway indication using CLE combined with ME-NBI multimodal endoscopic images. This method is executed using the CLE combined with ME-NBI multimodal endoscopic image early esophageal lesion auxiliary grading and pathway indication system described in Embodiment 1.

[0050] This method uses ME-NBI and CLE image sequences of the same esophageal lesion from the same subject as processing objects. It correlates the image information from the ME-NBI images, which reflects changes in lesion extent, microvascular morphology, and mucosal surface structure, with the image information from the CLE images, which reflects changes in cell morphology, cell arrangement, and microstructural boundaries. Within a unified lesion coordinate area, it generates image evidence, credibility status, and auxiliary path prompts. The lesion image level labels and auxiliary path prompt labels output by the method described in this embodiment are only used as a basis for image evidence processing and verification, and do not directly replace pathological diagnosis or clinical treatment decisions.

[0051] S1. Image and Basic Information Acquisition: Acquire ME-NBI image sequences and CLE image sequences of the same esophageal lesion in the same subject, and record the distance of the lesion from the incisors, circumferential orientation, magnification, acquisition time, image frame number, and CLE sampling point number.

[0052] In practice, during the endoscopic examination, a magnified narrowband image sequence of the esophageal lesion area is first acquired using ME-NBI (Medium-to-Narrow Band Image Acquisition). During acquisition, the distance from the lesion to the incisors is recorded, for example, as 28cm, 30cm, or other actual measured values. Simultaneously, the position of the lesion in the circumferential direction of the esophagus is recorded, for example, according to clock positions, such as 3 o'clock to 5 o'clock, 6 o'clock to 8 o'clock, etc. For each ME-NBI image sequence, a ME-NBI image frame number is assigned according to the acquisition sequence, and the corresponding magnification and acquisition time are recorded.

[0053] Subsequently, CLE image sampling was performed within the same esophageal lesion area. Each CLE sampling point was assigned a CLE sampling point number according to the sampling order, and the CLE image frame number, probe contact time, and corresponding ME-NBI image frame number of each CLE sampling point were recorded and associated.

[0054] In this step, the image sequence and basic information are not saved independently, but rather linked based on the same subject, the same lesion location, and the same examination time period. For multiple ME-NBI image frames of the same lesion, an ME-NBI image sequence can be formed based on the image frame number and acquisition time; for multiple CLE sampling points of the same lesion, a CLE sampling point set can be formed based on the CLE sampling point number. This method ensures that each subsequent CLE sampling point can be traced back to the corresponding ME-NBI observation area, preventing CLE images from existing as isolated local images that cannot correspond to the overall lesion area.

[0055] S2. Establishment of lesion coordinate area: In ME-NBI images, mark the outer edge of the lesion, the bifurcation point of the blood vessel, the mucosal texture change point, the color change point, and the suspected infiltration key area. Establish the lesion coordinate area with the outer edge of the lesion, the longest diameter axis, and the short diameter axis, and divide the area into the central area, the edge area, the transition area, and the suspected infiltration key area.

[0056] In practice, ME-NBI image frames that clearly display the lesion's extent and surface structure are selected as coordinates to establish the image. First, the outer edge of the lesion is defined based on background color change boundaries, microvascular morphological mutation boundaries, and mucosal texture interruption boundaries. Background color change boundaries are used to identify areas of color difference between the lesion region and the surrounding mucosa; microvascular morphological mutation boundaries are used to identify boundaries where blood vessels change from a regular arrangement to dilation, serpentine, reticular, annular, or broken states; and mucosal texture interruption boundaries are used to identify locations where surface texture changes from a continuous state to a disappearance, disorder, or unclear state. The outer edge of the lesion is formed by combining the above boundary information. This outer edge can be a closed line or an approximately closed line formed by connecting boundary inflection points and adjacent boundary segments.

[0057] Then, within the outer edge of the lesion or its adjacent area, vascular bifurcation points, mucosal texture abrupt change points, color abrupt change points, and boundary inflection points are marked. Vascular bifurcation points are selected where the morphology of the bifurcation is relatively clear and the location is stable in the ME-NBI image; mucosal texture abrupt change points are selected where the texture changes from continuous to interrupted, sparse, disappears, or changes direction; color abrupt change points are selected where the background color changes from light to dark, from uniform to uneven, or where the color boundary is more obvious; boundary inflection points are selected where the direction of the outer edge of the lesion changes significantly. These reference points are used for subsequent lesion coordinate region division and CLE sampling point mapping.

[0058] After the outer edge of the lesion and the reference point are marked, the longest radial axis is generated along the direction of the maximum distance within the outer edge of the lesion, and the short radial axis is generated along the direction that passes through the midpoint of the longest radial axis and is perpendicular to the longest radial axis. The longest radial axis is used to represent the main extension direction of the lesion in the ME-NBI image, and the short radial axis is used to represent the lateral extent of the lesion relative to the longest radial axis.

[0059] A lesion coordinate region was established based on the outer edge of the lesion, its longest and shortest radial axes, and further divided into a central region, a peripheral region, a transitional region, and a suspected infiltration focus region according to the relative position of the lesion. Specifically, the central region is located within the outer edge of the lesion and is close to the intersection of the longest and shortest radial axes; the peripheral region is located near the outer edge of the lesion; the transitional region is the area between the central and peripheral regions; and the suspected infiltration focus region is the area in the ME-NBI image that simultaneously exhibits microvascular morphological abnormalities, surface irregularities, blurred boundaries, darkened background color, or significant loss of mucosal texture. Each region was assigned coordinate numbers, for example, by dividing it vertically and horizontally into multiple sub-regions, each assigned a coordinate number for subsequent recording of abnormal features and CLE sampling point locations.

[0060] S3, CLE sampling point mapping: A three-point reference area is formed by two adjacent blood vessel bifurcation points and one mucosal texture mutation point or color mutation point. The CLE sampling point number, CLE image frame number, probe contact time and corresponding ME-NBI image frame number are written into the lesion coordinate area, and the distance deviation, orientation deviation and coverage status are recorded.

[0061] In practice, two adjacent vessel bifurcation points near the CLE sampling point are selected within the lesion coordinate region, and combined with a mucosal texture mutation point or color mutation point to form a three-point reference region. This three-point reference region is used to express the local spatial reference relationship of the CLE sampling point in the ME-NBI image. Since the CLE image is a local microscopic image, relying solely on textual records of probe contact is insufficient to accurately reflect the location of the sampling point within the entire lesion. Therefore, this step uses two vessel bifurcation points and one mucosal texture mutation point or color mutation point to jointly constitute the reference region, mapping the CLE sampling point to the ME-NBI lesion coordinate region.

[0062] After generating the three-point reference area, the CLE sampling point number, CLE image frame number, probe contact time, and corresponding ME-NBI image frame number are written into the position of the corresponding three-point reference area in the lesion coordinate region. For cases where a lesion has multiple CLE sampling points, a corresponding three-point reference area is established for each CLE sampling point, and each sampling point is written into its corresponding coordinate number. When a CLE sampling point is located at the intersection of two coordinate numbers, it can be recorded in the primary coverage coordinate number based on its position relative to the center of the three-point reference area and its orientation relative to the longest radial axis, while simultaneously recording its relationship with adjacent coordinate numbers.

[0063] Simultaneously, the distance deviation of the CLE sampling point relative to the center of the three reference areas, and the azimuth deviation relative to the longest radial axis, are recorded. The distance deviation represents the difference in distance between the CLE sampling point and the center of the three reference areas, while the azimuth deviation represents the direction and degree of offset of the CLE sampling point relative to the lesion's main axis. Further determination is made as to whether the CLE sampling point covers the suspected infiltration focus area: when the CLE sampling point is located within the suspected infiltration focus area, or the distance to the boundary of the suspected infiltration focus area is no greater than 1 mm, a focus area coverage marker is formed; when the CLE sampling point is not located within the suspected infiltration focus area, and the distance to the boundary of the suspected infiltration focus area is greater than 1 mm, a focus area non-coverage marker is formed. Through this processing, it can be clearly determined whether the CLE image corresponds to the key abnormal area indicated by ME-NBI.

[0064] S4. Image Evidence Recording and Layering: Record ME-NBI microvascular features, ME-NBI surface structural features, CLE cell morphology features, and CLE structural boundary features, and form ME-NBI quality grade, CLE quality grade, and sampling coverage grade.

[0065] In practice, ME-NBI microvascular features and ME-NBI surface structure features are recorded one by one within the lesion coordinate area. ME-NBI microvascular features include dilated vessels, serpentine vessels, reticular vessels, ring vessels, and vascular rupture areas, and these features are assigned to their corresponding coordinate numbers. ME-NBI surface structure features include areas where mucosal texture disappears, surface unevenness, areas with deepened background color, and areas with blurred boundaries, and their corresponding coordinate numbers are recorded. After recording using coordinate numbers, the abnormal features in the ME-NBI image are no longer simply described in natural language, but rather correspond to specific regions within the lesion coordinate area.

[0066] For CLE image sequences, CLE cell morphology and structural boundary features are recorded. CLE cell morphology features include areas of uneven cell size, disordered cell arrangement, increased dark areas of the cell nucleus, and altered intercellular spaces. CLE structural boundary features include areas of interrupted glandular structure boundaries, blurred epithelial structures, and areas of fluorescence leakage. Each CLE image feature is associated with the CLE sampling point number, the CLE image frame number, and the corresponding lesion coordinate number. Therefore, macroscopic abnormal areas in ME-NBI images and microscopic abnormal areas in CLE images can be compared within the same coordinate system.

[0067] Simultaneously, the ME-NBI image quality is recorded, including the proportion of area obscured by reflected light, the proportion of motion-blurred areas, the proportion of areas where microvessels are discernible, and the continuity ratio of lesion outer edges. Based on these recorded results, a ME-NBI quality level is determined. For example, a low proportion of reflected light obscured areas and motion-blurred areas, a high proportion of areas where microvessels are discernible, and a high proportion of lesion outer edges continuity can be recorded as a Level 1 quality level; when some of the above indicators affect interpretation, it can be recorded as a Level 2 quality level; when reflection, blurring, or discontinuous outer edges significantly affect interpretation, it can be recorded as a Level 3 quality level.

[0068] CLE image quality is recorded, including the percentage of areas with uneven fluorescence intensity, the percentage of areas with discernible cell outlines, the percentage of areas affected by probe compression artifacts, and the number of consecutive effective frames. CLE quality is graded based on these records. A higher quality grade is achieved when the percentage of areas with discernible cell outlines is high, the percentage of areas affected by probe compression artifacts is low, and the number of consecutive effective frames is high. A lower quality grade is achieved if uneven fluorescence intensity, compression artifacts, or insufficient effective frames affect the observation of cell and structural boundaries.

[0069] Furthermore, sampling coverage levels are established based on the coverage of CLE sampling points in the central area, edge area, and suspected infiltration key areas. A Level 1 sampling coverage level is established when CLE sampling points cover the central area, edge area, and suspected infiltration key areas; a Level 2 sampling coverage level is established when CLE sampling points cover the central area and edge area but do not adequately cover the suspected infiltration key areas; and a Level 3 sampling coverage level is established when the CLE sampling point coverage area is insufficient, or when no effective CLE sampling is obtained in the suspected infiltration key areas. The sampling coverage level serves as an important basis for subsequent assessment of the credibility of the evidence.

[0070] S5. Lesion Image Grade Determination: Generate lesion image grade labels, evidence credibility markers, and review prompt markers based on co-located evidence, adjacent evidence, low-confidence evidence, and uncovered markers.

[0071] In practice, ME-NBI microvascular abnormalities and CLE microstructural abnormalities are correlated and judged within the lesion coordinate region. When both ME-NBI microvascular abnormalities and CLE microstructural abnormalities exist simultaneously within the same coordinate number, isotopic evidence is formed. For example, if a vascular rupture area is recorded along with an area of ​​disordered cell arrangement, blurred epithelial structure, or fluorescence leakage within the same coordinate number, this can be considered isotopic evidence. Isotopic evidence indicates that both image modalities record abnormal image features within the same spatial region, and their evidentiary correspondence is relatively clear.

[0072] Proximity evidence is formed when ME-NBI microvascular abnormalities and CLE microstructural abnormalities are present in adjacent coordinate numbers, respectively. For example, if serpentine or ring-shaped vessels are recorded in one coordinate number, and areas of uneven cell size or interrupted glandular structure boundaries are recorded in adjacent coordinate numbers, this can be identified as proximity evidence. Proximity evidence indicates that the abnormal image features of the two modalities are located in adjacent regions, but do not fall completely within the same coordinate number. Further judgment is needed by considering sampling bias, orientation bias, and image quality.

[0073] When the ME-NBI quality level or CLE quality level is level three, the corresponding image features are recorded as low-confidence evidence. When the sampling coverage level is level three, or when the CLE sampling points do not cover the suspected infiltration hotspot area, the corresponding results are also treated as low-confidence evidence or uncovered markings. For cases where the CLE sampling points do not cover the suspected infiltration hotspot area, or the distance between the CLE sampling points and the boundary of the suspected infiltration hotspot area exceeds 1 mm, an uncovered marking is formed. Low-confidence evidence and uncovered markings do not directly negate the existence of anomalous image features, but rather indicate insufficient spatial correspondence, image quality, or sampling sufficiency of the anomalous features, and need to be reflected simultaneously in the output results.

[0074] After generating the evidence type, lesion image grade labels are generated based on isotopic evidence, proximity evidence, low-confidence evidence, and uncovered markers. Lesion image grade labels include low-grade abnormality image labels, high-grade abnormality image labels, intramucosal carcinoma-prone image labels, and suspected submucosal invasion image labels. A low-grade abnormality image label can be generated when the abnormal features mainly manifest as mild local vascular or surface structural changes, and the CLE image does not show obvious cell arrangement disorder or structural boundary disruption. A high-grade abnormality image label can be generated when the ME-NBI image shows obvious microvascular abnormalities, and the CLE image shows features such as uneven cell size, disordered arrangement, or increased dark areas in the cell nuclei. An intramucosal carcinoma-prone image label or a suspected submucosal invasion image label can be generated when the isotopic evidence is located in the central area of ​​the lesion or the suspected key area of ​​invasion, and the CLE structural boundary features show blurred epithelial structure, interrupted glandular structure boundaries, or fluorescent leakage areas.

[0075] When the lesion evidence is located in a suspected infiltration hotspot and no low-confidence evidence exists, a high-confidence marker is generated. When there is adjacent evidence, low-confidence evidence, or an uncovered marker, the lesion image level label and a review prompt marker are output together. The review prompt marker can correspond to a specific coordinate number, image frame number, or CLE sampling point number, so that reviewers can clearly identify the specific area that needs to be re-examined or supplemented with sampling.

[0076] S6. Auxiliary Path Hints and Result Output: Generate auxiliary path hint labels based on lesion image level labels, lesion range, suspected infiltration key areas, and CLE sampling coverage level, and output lesion coordinate map, evidence credibility marker, review hint marker, and auxiliary path hint labels.

[0077] In practice, the lesion extent information is first read, including the lesion's long diameter, short diameter, area, and circumferential percentage. The long and short diameters are obtained from the longest and shortest axes generated in S2, the area is calculated or recorded based on the area enclosed by the lesion's outer edge, and the circumferential percentage is determined based on the lesion's distribution along the circumferential direction of the esophagus. Then, the coordinate numbers of suspected infiltration hotspots and the corresponding ME-NBI microvascular features and CLE microstructural features are read, and it is determined whether the CLE sampling points cover the central area, the peripheral area, and the suspected infiltration hotspots.

[0078] When generating auxiliary pathway prompt labels, if the lesion image level label is a low-level abnormal image label and the sampling coverage level is level one or two, a regular follow-up prompt label or an additional biopsy review prompt label is generated. If the lesion image level label is a high-level abnormal image label or an intramucosal carcinoma-prone image label, and the lesion's outer edge is continuous and the CLE sampling points cover the central and marginal areas, an endoscopic mucosal resection candidate prompt label or an endoscopic submucosal dissection candidate prompt label is generated. If the lesion image level label is a suspected submucosal invasion image label, or if isotopic evidence exists in the suspected key invasion area, a surgical and comprehensive treatment assessment prompt label is generated. If the CLE sampling points do not cover the suspected key invasion area, a review sampling prompt label is added in addition to the auxiliary pathway prompt label to indicate that the key area needs further judgment in conjunction with supplementary sampling or manual review.

[0079] Finally, the system outputs a lesion coordinate map, lesion image grade labels, evidence credibility markers, review prompt markers, and auxiliary path prompt labels. The lesion coordinate map displays the coordinate numbers corresponding to the lesion's outer edge, longest axis, shortest axis, central area, marginal area, transition area, suspected infiltration focus area, CLE sampling point location, and abnormal image features. The lesion image grade labels display the grading results based on cross-modal image evidence; the evidence credibility markers indicate the credibility status of the evidence upon which the grade determination is based; the review prompt markers indicate specific locations of low-credibility evidence, adjacent evidence, or key areas not covered; and the auxiliary path prompt labels indicate pathways for subsequent follow-up examinations, additional biopsies, endoscopic resection candidates, or surgical and comprehensive treatment assessments.

[0080] The method in this embodiment eliminates the need for separate image descriptions of ME-NBI and CLE images. Instead, spatial correspondence, feature recording, and evidence stratification are completed within the lesion coordinate region. This method allows for the clear recording of whether CLE sampling points cover suspected infiltration hotspots indicated by ME-NBI, while simultaneously incorporating image quality, sampling coverage, and cross-modal abnormality correspondence into the lesion image grade determination. Therefore, the output not only includes the lesion image grade itself but also simultaneously includes evidence credibility markers, review prompt markers, and auxiliary path prompt labels, providing a clearer spatial basis and chain of evidence for the endoscopic image review process of early esophageal lesions.

[0081] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A CLE combined with ME-NBI multimodal endoscopic imaging system for early esophageal lesion assisted grading and pathway indication, characterized in that, include: The image access module receives ME-NBI image sequences and CLE image sequences of the same esophageal lesion from the same subject, and records the distance of the lesion from the incisors, circumferential orientation, magnification, acquisition time, image frame number, and CLE sampling point number. The ME-NBI lesion coordinate establishment module marks the outer edge of the lesion, the bifurcation point of the blood vessel, the mucosal texture change point, the color change point and the suspected infiltration key area in the ME-NBI image, and establishes the lesion coordinate area with the outer edge of the lesion, the longest diameter axis and the short diameter axis; The CLE sampling point mapping module uses two vascular bifurcation points and one mucosal texture mutation point or color mutation point to form a three-point reference area, maps the CLE sampling point to the lesion coordinate area, and records the distance deviation, orientation deviation and coverage status between the sampling point and the suspected infiltration key area. Two adjacent vascular bifurcation points and one mucosal texture change point or color change point are selected to form a three-point reference area. This three-point reference area does not use a single image point as the localization basis, but uses vascular bifurcation and mucosal texture or color change to form a local spatial reference, so that CLE sampling points can be mapped to the ME-NBI lesion coordinate area more stably. The image evidence stratification module generates ME-NBI quality level, CLE quality level, and sampling coverage level. The cross-modal feature recording module records ME-NBI microvascular features, ME-NBI surface structural features, CLE cell morphology features, and CLE structural boundary features; The image grading module generates lesion image grading labels, evidence credibility markers, and review prompt markers based on contiguous evidence, adjacent evidence, low-confidence evidence, and uncovered markers. The image level determination module includes: The co-existence evidence determination unit identifies records containing both ME-NBI microvascular abnormality features and CLE microstructural abnormality features within the same coordinate number as co-existence evidence. The proximity evidence determination unit identifies records with ME-NBI microvascular abnormality features and CLE microstructural abnormality features respectively within adjacent coordinate numbers as proximity evidence. The low-confidence evidence recording unit generates low-confidence evidence and review prompts when the ME-NBI quality level or CLE quality level is level three, the sampling coverage level is level three, or the CLE sampling point does not cover the suspected infiltration key area. The grade label generation unit generates low-level abnormality image labels, high-level abnormality image labels, mucosal carcinoma tendency image labels, or suspected submucosal invasion image labels. The auxiliary path suggestion module generates auxiliary path suggestion labels; The results output module outputs lesion coordinate map, lesion image level label, evidence credibility mark, review prompt mark, and auxiliary path prompt label.

2. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway indication system according to claim 1, characterized in that, The ME-NBI lesion coordinate establishment module includes: The outer edge calibration unit generates the outer edge of the lesion by changing the boundary along the background coloring, the boundary of microvascular morphological mutation, and the boundary of mucosal texture interruption. Reference point calibration unit, calibrating blood vessel bifurcation points, mucosal texture change points, color change points, and boundary inflection points; The coordinate axis generation unit generates the longest radial axis in the direction of the maximum distance within the outer edge of the lesion, and generates the short radial axis in the direction that passes through the midpoint of the longest radial axis and is perpendicular to the longest radial axis. The lesion coordinate area is divided into central area, edge area, transition area and suspected infiltration key area by combining the outer edge line of the lesion, the longest diameter axis and the short diameter axis, and a coordinate number is set for each zone.

3. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway guidance system according to claim 1, characterized in that, The CLE sampling point mapping module includes: The three-point reference region generation unit selects two adjacent blood vessel bifurcation points and one mucosal texture mutation point or color mutation point to form a three-point reference region. The sampling point writing unit writes the CLE sampling point number, CLE image frame number, probe contact time, and corresponding ME-NBI image frame number into the position of the corresponding three reference points in the lesion coordinate area. The deviation recording unit records the distance deviation of the CLE sampling point relative to the center of the three reference areas and the orientation deviation relative to the longest radial axis; The coverage marker unit generates a coverage marker for the key area when the CLE sampling point is located within the suspected infiltration key area, or when the distance between the sampling point and the boundary of the suspected infiltration key area is no more than 1 mm; otherwise, it generates an uncovered marker for the key area.

4. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway indication system according to claim 1, characterized in that, The image evidence layering module includes: The ME-NBI quality recording unit records the proportion of reflective occlusion area, the proportion of motion-blurred area, the proportion of microvascular identifiable area, and the continuity ratio of the outer edge of the lesion, and generates the ME-NBI quality level based on the recording results. The CLE quality recording unit records the proportion of areas with uneven fluorescence brightness, the proportion of areas with discernible cell outlines, the proportion of areas with probe compression artifacts, and the number of consecutive effective frames, and generates a CLE quality level based on the recording results. The sampling coverage level recording unit generates a level 1, 2, or 3 sampling coverage level based on the coverage of CLE sampling points in the central area, edge area, and suspected infiltration key area.

5. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway indication system according to claim 1, characterized in that, The cross-modal feature recording module includes: The ME-NBI vascular feature table records the coordinate numbers of dilated vessels, serpentine vessels, reticular vessels, annular vessels, and vascular rupture zones. ME-NBI surface structure feature table records the coordinate numbers of mucosal texture disappearance area, surface uneven area, background color deepening area and boundary blurring area; CLE cell characteristic table records the coordinate numbers of areas with uneven cell size, disordered cell arrangement, increased dark areas of cell nuclei, and altered intercellular spaces. The CLE structural boundary feature table records the coordinate numbers of the glandular structure boundary interruption region, the epithelial structure blur region, and the fluorescence leakage region.

6. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway indication system according to claim 1, characterized in that, The auxiliary path suggestion module includes: The lesion extent reading unit reads the lesion's long diameter, short diameter, area, and circumferential proportion. The key area reading unit reads the coordinate numbers of suspected infiltration key areas, as well as the corresponding ME-NBI microvascular features and CLE microstructure features; The coverage status reading unit reads whether the CLE sampling points cover the central area, edge area, and suspected infiltration key area; The path labeling unit generates labels for regular follow-up examinations, additional biopsy review, endoscopic mucosal resection, endoscopic submucosal dissection, or surgical and comprehensive treatment assessment.

7. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway guidance system according to claim 6, characterized in that, When the path labeling unit records a periodic review prompt label or an additional biopsy verification prompt label when the lesion image level label is a low-level abnormal image label and the sampling coverage level is level one or level two.

8. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway guidance system according to claim 7, characterized in that, When the lesion image grade label is a high-grade abnormal image label or an intramucosal carcinoma tendency image label, and the outer edge of the lesion is continuous and the CLE sampling point covers the central and marginal areas, record the endoscopic mucosal resection candidate prompt label or the endoscopic submucosal dissection candidate prompt label.

9. The CLE combined with ME-NBI multimodal endoscopic imaging early esophageal lesion auxiliary grading and pathway guidance system according to claim 7, characterized in that, When the lesion image grade label is a suspected submucosal invasion image label, or when there is isotopic evidence in the suspected key area of ​​invasion, record the surgical and comprehensive treatment assessment prompt label; when the CLE sampling point does not cover the suspected key area of ​​invasion, add a review sampling prompt label in addition to the auxiliary path prompt label.

Images