A multi-modal based colon polyp visualization intelligent processing method and system

Abstract

This invention discloses a multimodal-based intelligent visualization processing method and system for colonic polyps, belonging to the field of colonic polyp processing technology. It constructs a multimodal colonoscopy image database and a multi-source image dataset; further constructs a polyp candidate target database, a multimodal feature verification database, a polyp envelope trajectory stability dataset, and a morphological stability dataset; calculates stability coefficients based on stability data, calculates verification scores based on multimodal features, and fuses these to obtain a comprehensive confidence score and the confidence difference between candidate targets; standardizes the confidence difference into a similarity score, presets a comprehensive confidence score threshold and a similarity threshold, determines stable polyps or interference signals, and constructs a collaborative monitoring pair for interference signals for coordinated visualization processing or individual removal. This invention, by integrating multimodal dynamic stability and cross-modal feature verification, significantly reduces the risk of false positives and missed diagnoses, improving the accuracy and clinical interpretability of intelligent colonic polyp detection.

Description

Technical Field

[0001] This invention relates to the field of colon polyp treatment technology, specifically to a multimodal, visualized, intelligent treatment method and system for colon polyps. Background Technology

[0002] With the rapid development of endoscopic technology, electronic colonoscopy has become the gold standard for screening and diagnosing colorectal polyps. Traditionally, white light imaging (WLI) has been the basic mode of colonoscopy, providing a direct view of the surface morphology of the colonic mucosa. To further enhance the identification of polyp histology and microvascular structures, multimodal imaging technologies such as narrow-band imaging (NBI) and automated fluorescence imaging (AFI) have emerged and gradually become widespread. NBI, by filtering out red light, enhances the contrast of the mucosal surface vascular structure, making the observation of the polyp glandular openings (Pit pattern) and capillary morphology (CP pattern) clearer. AFI, on the other hand, utilizes the difference in autofluorescence properties between normal and polyp tissues to capture fluorescence signals in abnormal metabolic areas, achieving "electronic staining" and indication of lesions. This series of multimodal imaging technologies provides clinicians with multidimensional information from macroscopic morphology to microscopic functional structure, significantly improving the detection potential of polyps.

[0003] Existing intelligent processing methods still face numerous technical bottlenecks and shortcomings. First, most solutions remain at the level of single-modal image analysis or simple stitching of two modalities, lacking a mechanism for organically fusing and synergistically verifying morphological stability under white light, microvascular features under narrow-band imaging, and metabolic abnormalities under fluorescence imaging. For example, an artifact caused by intestinal peristalsis or probe movement might be misjudged as a micropolyp in a single modality, leading to a false positive; while an atypical, flat polyp might be missed due to a lack of significant morphological features. Second, existing methods primarily focus on static feature analysis of single-frame images, neglecting the dynamic evolution of polyp candidates in continuous image sequences. The morphology, edge envelope trajectory, and temporal stability of colonic polyps are key features for distinguishing real polyps from interfering signals such as bubbles and residual feces. Existing technologies lack effective methods for quantifying and calculating this type of dynamic stability, resulting in insufficient robustness of the processing results. Furthermore, the current system's visualization of processing results is rather rudimentary, lacking differentiated displays of confidence levels and a collaborative monitoring mechanism between highly similar interference targets. Doctors find it difficult to intuitively determine which markers output by the system are highly credible polyps and which are potential interferences that require vigilance, increasing the workload of clinical review. Summary of the Invention

[0004] The purpose of this invention is to provide a multimodal-based intelligent visualization method and system for processing colon polyps, in order to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A multimodal intelligent processing method for visualizing colonic polyps is disclosed. The method includes the following steps: Step S1: Constructing a multimodal colonoscopy image database; constructing a multi-source image dataset; Step S2: Constructing a polyp candidate target database; constructing a multimodal feature verification database; based on the multi-source image dataset, constructing a polyp envelope trajectory stability dataset and a polyp morphology stability dataset; Step S3: Calculating the stability coefficient of the polyp candidate targets based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset; calculating the verification score of the multimodal features for the polyp candidate targets based on the multimodal feature verification database; calculating the comprehensive confidence difference between polyp candidate targets; Step S4: Based on the comprehensive confidence difference between polyp candidate targets, standardizing the comprehensive confidence difference to the similarity between polyp candidate targets; setting a preset threshold, analyzing and performing visualization processing.

[0007] As a preferred embodiment of the multimodal-based intelligent processing method for visualizing colonic polyps described in this invention, a multimodal colonoscopy image database is constructed. The multimodal colonoscopy image database is used to collect image data generated when realizing the multimodal colonic polyp visualization function. The multimodal colonic polyp visualization function includes white light imaging function, narrow band imaging function, and automatic fluorescence imaging function. The white light imaging function realizes the acquisition of the morphology of the colonic mucosa surface, the narrow band imaging function realizes the enhanced display of the vascular structure of the mucosa surface, and the automatic fluorescence imaging function realizes the capture of fluorescence signals of abnormal metabolism of polyp tissue.

[0008] Any image data in the multimodal colonoscopy image database that implements the multimodal colon polyp visualization function is uniformly numbered, and a multi-source image data set is constructed, denoted as . ,in, This represents the m-th image data that enables multimodal visualization of colon polyps. This represents the total number of image data used to achieve the multimodal colon polyp visualization function.

[0009] As a preferred embodiment of the multimodal-based intelligent visualization method for colonic polyps described in this invention, a candidate polyp database is constructed. ,in, Indicates the first One polyp candidate The total number of candidate polyps is represented (the candidate polyps are numbered according to their temporal or spatial order of appearance in the multimodal colonoscopy image sequence, with adjacent numbers indicating temporal or spatial proximity); a multimodal feature verification database is constructed, denoted as... ,in, Indicates the first The multimodal feature for the first Validation data for each polyp candidate (values ​​are continuous values ​​between 0 and 1, where 1 indicates that the feature is fully met and 0 indicates that the feature is not met at all). This represents the total number of multimodal feature verification data; the multimodal feature verification refers to the confirmation of the morphological and functional characteristics of polyps by different imaging modalities.

[0010] Based on the multi-source image dataset, envelope trajectory stability data and morphological stability data (both scalar data) of polyp candidate targets are obtained, and polyp envelope trajectory stability datasets are constructed respectively. Polyp morphological stability dataset ,in, Indicates the first One polyp candidate target in Data on the stability of envelope trajectories within a time window (value range [0,1], larger values ​​indicate more stable envelope trajectories). Indicates the first One polyp candidate target in Morphological stability data within a time window (value range [0,1], larger values ​​indicate more stable morphology). This indicates the total number of time windows.

[0011] As a preferred embodiment of the multimodal-based intelligent processing method for visualizing colonic polyps described in this invention, it is based on a polyp envelope trajectory stability dataset. Polyp morphological stability dataset Calculate candidate polyps The stability coefficient is calculated using the following formula:

[0012] ;

[0013] in, Indicates polyp candidate target stability coefficient, This indicates a weighting coefficient preset by experts based on historical experience;

[0014] Verification database based on multimodal features Calculate multimodal features for polyp candidate targets The verification score is calculated using the following formula:

[0015] ;

[0016] in, Indicating multimodal features for polyp candidate targets The verification score;

[0017] Based on polyp candidate targets stability coefficient and the multimodal features for polyp candidate targets Validation score Calculate candidate polyps The overall confidence level is calculated using the following formula:

[0018] ;

[0019] in, Indicates polyp candidate target The overall confidence level, This refers to the stability coefficient influence factor, pre-set by experts based on historical experience. This indicates the influence factor of the verification score, pre-set by experts based on historical experience, and ;

[0020] Calculate polyp candidate targets and polyp candidate targets The formula for calculating the overall confidence difference between them is: ,in, Indicates polyp candidate target and polyp candidate targets The difference in overall confidence levels between them Indicates polyp candidate target The overall confidence level.

[0021] As a preferred embodiment of the multimodal-based intelligent processing method for visualizing colonic polyps described in this invention, based on polyp candidate targets... and polyp candidate targets Difference in overall confidence levels between The overall confidence level difference Standardization as a candidate target for polyps and polyp candidate targets The similarity between them is calculated using the following formula: ,in, Indicates polyp candidate target and polyp candidate targets The similarity between them;

[0022] Preset comprehensive confidence threshold If polyp candidate target Overall confidence level Then determine the candidate target for polyps. To stabilize the polyp and output its location, contour, and modal information; if the polyp is a candidate target Overall confidence level Then determine the candidate target for polyps. This is an interference signal;

[0023] Preset similarity threshold If polyp candidate target If identified as an interference signal, it will be added to the polyp candidate target database. Searching for polyp candidates The similarity is greater than the similarity threshold For other potential polyp targets, if at least one potential polyp target exists, a joint regulatory pair of polyp targets is constructed and visual processing is performed (to indicate that this group of targets needs to be jointly reviewed or filtered in subsequent processing). If no potential polyp target exists, the potential polyp target is... Individually marked as isolated interference signals and removed.

[0024] A multimodal-based intelligent visualization system for colon polyps, comprising: a dataset construction module, a database construction module, a stability coefficient calculation and difference calculation module, and a similarity calculation and visualization processing module;

[0025] The dataset construction module includes: constructing a multimodal colonoscopy image database; and constructing a multi-source image data set.

[0026] The database construction module includes: constructing a polyp candidate target database; constructing a multimodal feature verification database; and constructing a polyp envelope trajectory stability dataset and a polyp morphology stability dataset based on the multi-source image data set.

[0027] The stability coefficient calculation and difference calculation module calculates the stability coefficient of polyp candidate targets based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset; calculates the verification score of multimodal features for polyp candidate targets based on the multimodal feature verification database; and calculates the comprehensive confidence difference between polyp candidate targets.

[0028] The similarity calculation and visualization module: based on the comprehensive confidence difference between polyp candidate targets, standardizes the comprehensive confidence difference into the similarity between polyp candidate targets; presets a threshold, analyzes and performs visualization processing.

[0029] Furthermore, the dataset construction module includes a dataset construction unit;

[0030] The dataset construction unit constructs a multimodal colonoscopy image database. This database collects image data generated during the implementation of the multimodal colonic polyp visualization function. The multimodal colonic polyp visualization function includes white light imaging, narrow band imaging, and autofluorescence imaging. White light imaging captures the morphology of the colonic mucosa, narrow band imaging enhances the display of the mucosal surface vascular structure, and autofluorescence imaging captures the fluorescence signal of abnormal polyp tissue metabolism. Any image data in the multimodal colonoscopy image database that implements the multimodal colonic polyp visualization function is uniformly numbered, and a multi-source image data set is constructed.

[0031] Furthermore, the database construction module includes a database construction unit;

[0032] The database construction unit: constructs a polyp candidate target database; constructs a multimodal feature verification database; and, based on the multi-source image data set, obtains the envelope trajectory stability data and morphological stability data of the polyp candidate targets, and constructs a polyp envelope trajectory stability dataset and a polyp morphological stability dataset, respectively.

[0033] Furthermore, the stability coefficient calculation and difference calculation module includes a stability coefficient calculation unit and a difference calculation unit;

[0034] The stability coefficient calculation unit calculates the stability coefficient of candidate polyps based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset.

[0035] The difference calculation unit: calculates the verification score of the multimodal features for the polyp candidate targets based on the multimodal feature verification database; calculates the comprehensive confidence of the polyp candidate targets based on the stability coefficient of the polyp candidate targets and the verification score of the multimodal features for the polyp candidate targets; and calculates the difference in comprehensive confidence between the polyp candidate targets.

[0036] Furthermore, the similarity calculation and visualization processing module includes a similarity calculation unit and a visualization processing unit;

[0037] The similarity calculation unit: based on the comprehensive confidence difference between polyp candidate targets, standardizes the comprehensive confidence difference into the similarity between polyp candidate targets;

[0038] The visualization processing unit has the following features: a preset comprehensive confidence threshold; if the comprehensive confidence of a polyp candidate is greater than or equal to the comprehensive confidence threshold, the polyp candidate is determined to be a stable polyp; if the comprehensive confidence of a polyp candidate is less than the comprehensive confidence threshold, the polyp candidate is determined to be an interference signal; and a preset similarity threshold; if a polyp candidate is determined to be an interference signal, other polyp candidate targets with a similarity greater than the similarity threshold are searched in the polyp candidate target database. If at least one polyp candidate target exists, a linked monitoring pair of polyp targets is constructed and visualization processing is performed; if no polyp candidate target exists, the polyp candidate target is marked as an isolated interference signal and removed.

[0039] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention provides a multimodal intelligent processing method and system for visualizing colonic polyps. By constructing a multimodal colonoscopy image database including white light imaging, narrow-band imaging, and automatic fluorescence imaging, and uniformly numbering it as a multi-source image data set, it achieves standardized aggregation and temporal-spatial alignment of different physical attribute information. This fundamentally solves the problem of isolated storage and difficulty in correlation and fusion of multimodal data, laying a reliable data foundation for subsequent high-precision analysis. Furthermore, by constructing a polyp candidate target database, a multimodal feature verification database, and envelope trajectory stability and morphological stability datasets, it transforms static detection results into dynamically trackable candidate target trajectories. It also uniquely introduces the concept of a time window to numerically represent edge contour jitter and the rate of change of area perimeter, enabling mathematical modeling of dynamic interference factors such as intestinal peristalsis and probe movement. This effectively distinguishes real polyps from artifacts such as bubbles and feces, significantly reducing... The system reduces the false positive rate; it then calculates the stability coefficient of candidate targets based on the stability dataset, calculates the verification score based on the multimodal feature verification database, and further integrates these to obtain the comprehensive confidence score and the confidence score difference between candidate targets. This solves the technical problems of insufficient multimodal information fusion depth and difficulty in balancing dynamic and static evidence in existing technologies. The system can flexibly adjust weights according to clinical scenarios and directly compare polyp confidence and non-polyp interference signals under the same dimension, greatly improving the scientific nature and accuracy of decision-making. Finally, the confidence score difference is standardized into similarity, and a comprehensive confidence score threshold and a similarity score threshold are preset: stable polyp information is output for high-confidence targets, and for low-confidence interference signals, similarity analysis is used to identify clustered interference and construct a linkage monitoring pair for collaborative visualization processing, while isolated interference is eliminated. This ensures the efficiency of positive detection and avoids the potential risk of missed diagnosis that may be caused by "one-size-fits-all" elimination, while also enhancing the interpretability of the system output and the user-friendliness of human-computer interaction. In summary, through the synergistic effect of the above steps, this invention achieves fully automated processing from multimodal data access, dynamic stability quantification, cross-modal evidence fusion to intelligent visual decision-making, resulting in comprehensive benefits such as improving the accuracy of colon polyp detection, reducing the risk of false positives and missed diagnoses, and enhancing clinical applicability. Attached Figure Description

[0040] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0041] Figure 1 This is a schematic diagram illustrating the steps of a multimodal-based intelligent visualization method for processing colon polyps according to the present invention.

[0042] Figure 2This is a schematic diagram of the structure of a multimodal-based intelligent visualization system for colon polyps according to the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Please see Figure 1 In this first embodiment: a multimodal-based intelligent visualization method for colon polyps is provided, which includes the following steps:

[0045] Step S1: Construct a multimodal colonoscopy image database; construct a multi-source image data set.

[0046] Specifically, a multimodal colonoscopy image database is constructed. This database is used to collect image data generated when implementing the multimodal colon polyp visualization function. The multimodal colon polyp visualization function includes white light imaging, narrow band imaging, and automatic fluorescence imaging. The white light imaging function acquires the morphology of the colonic mucosa surface, the narrow band imaging function enhances the display of the vascular structure of the mucosa surface, and the automatic fluorescence imaging function captures the fluorescence signal of abnormal metabolism in polyp tissue.

[0047] Any image data in the multimodal colonoscopy image database that implements the multimodal colon polyp visualization function is uniformly numbered, and a multi-source image data set is constructed, denoted as . ,in, This represents the m-th image data that enables multimodal visualization of colon polyps. This represents the total number of image data used to achieve the multimodal colon polyp visualization function.

[0048] In this invention, a multimodal colonoscopy image database containing multimodal functions such as white light imaging, narrow band imaging, and autofluorescence imaging is constructed, and each image data is uniformly numbered to form a structured multi-source image data set. This achieves unified characterization and traceable management of multiple heterogeneous modal data under the same colonoscopy examination scenario. It brings together information on three different physical properties—colonial mucosal surface morphology (white light), microvascular structure (narrow band imaging), and abnormal tissue metabolic fluorescence signals (autofluorescence)—in a standardized form within the same data framework, providing a structured data foundation for subsequent cross-modal feature verification. At the same time, the unified numbering ensures strict alignment of time series and spatial location, eliminating heterogeneity between multimodal data.

[0049] This step improves the systematic nature of multimodal data organization and the efficiency of collaborative retrieval, fundamentally solving the problem of isolated storage and difficulty in correlation and fusion of images of different modalities in existing technologies. It lays a reliable data foundation for subsequent high-precision polyp identification and stability analysis, and enhances the method's universal adaptability to multimodal endoscopic equipment.

[0050] Step S2: Construct a polyp candidate target database; construct a multimodal feature verification database; based on the multi-source image data set, construct a polyp envelope trajectory stability dataset and a polyp morphology stability dataset.

[0051] Specifically, construct a database of potential polyps. ,in, Indicates the first One polyp candidate The total number of candidate polyps is represented (the candidate polyps are numbered according to their temporal or spatial order of appearance in the multimodal colonoscopy image sequence, with adjacent numbers indicating temporal or spatial proximity); a multimodal feature verification database is constructed, denoted as... ,in, Indicates the first The multimodal feature for the first Validation data for each polyp candidate (values ​​are continuous values ​​between 0 and 1, where 1 indicates that the feature is fully met and 0 indicates that the feature is not met at all). This represents the total number of multimodal feature verification data; the multimodal feature verification refers to the confirmation of the morphological and functional characteristics of polyps by different imaging modalities.

[0052] Based on the multi-source image dataset, envelope trajectory stability data and morphological stability data (both scalar data) of polyp candidate targets are obtained, and polyp envelope trajectory stability datasets are constructed respectively. Polyp morphological stability dataset ,in, Indicates the first One polyp candidate target in Data on the stability of envelope trajectories within a time window (value range [0,1], larger values ​​indicate more stable envelope trajectories). Indicates the first One polyp candidate target in Morphological stability data within a time window (value range [0,1], larger values ​​indicate more stable morphology). This indicates the total number of time windows.

[0053] It should be noted that regarding the envelope trajectory stability data: For each frame of multimodal colonoscopy image (white light, narrow-band imaging, or autofluorescence imaging) within each time window, an image segmentation algorithm (such as a deep learning-based polyp segmentation model) is used to extract the precise edge contours of the polyp candidate targets, obtaining the set of boundary points of the polyp region in that frame. To eliminate the overall translational effects caused by colonoscopy probe movement or intestinal peristalsis, the geometric centroid of the polyp contour in each frame is calculated, and the contours of all frames are translated to the same centroid position, making the center points of the contours coincide. The aligned contours are then described by shape features, for example, using Fourier descriptors or shape context methods, converting the geometry of the contour into a set of numerical feature vectors. This feature vector can uniquely characterize the shape information of the contour and has a certain robustness to slight nonlinear deformations. Within the same time window, the degree of difference in the contour feature vectors between adjacent frames (e.g., Euclidean distance or cosine distance) is calculated sequentially. The smaller the difference, the better the shape of the polyp is maintained between adjacent frames, and the more stable the contour trajectory. The original instability index for a given time window is obtained by averaging the differences between all adjacent frames within that window. To ensure a positive correlation between the value and stability (i.e., a higher value indicates greater stability), this original instability is mapped to a stability score between 0 and 1 using an exponential transformation or by taking its reciprocal. The closer the score is to 1, the more stable the polyp's envelope trajectory is within that time window; conversely, the closer it is, the less stable it is. The stability scores calculated for the same polyp candidate target across different time windows are averaged again to obtain the overall envelope trajectory stability value for that polyp candidate target.

[0054] Regarding morphological stability data: For the polyp region segmented in each frame of the image, its pixel area is calculated (or a single morphological parameter such as perimeter or major axis can be used). To eliminate scale differences caused by different imaging distances, the area can be normalized by dividing it by the area of ​​a fixed reference structure in the image (such as the diameter of the intestinal lumen) to obtain the relative area. Within the same time window, the absolute change in polyp area between adjacent frames (absolute value of the difference) is calculated sequentially and divided by the area of ​​the previous frame (a very small constant is added to prevent the denominator from being zero) to obtain the relative rate of change between adjacent frames. This relative rate of change reflects the degree of drastic expansion or contraction of the polyp between each pair of frames. The average relative rate of change of all adjacent frames within a time window is calculated to obtain the average morphological change rate of that window. The larger this rate value, the more drastic the morphological change of the polyp within that window (fluctuating in size or rapid growth / shrinkage); the smaller the rate, the more stable the polyp morphology remains. To avoid directional conflicts when used in conjunction with envelope trajectory stability data (where a larger value indicates greater stability), the average rate of change is inversely mapped: for example, using an exponential decay function or reciprocal transformation, the rate of change is converted into a stability contribution value between 0 and 1. After conversion, the closer the value is to 1, the slower the morphological change (the more stable), and the closer it is to 0, the faster the morphological change (the less stable). The morphological stability contribution values ​​of the same polyp candidate target in different time windows are averaged to obtain the overall morphological change rate characteristic value of the polyp candidate target (at this point, the value has been converted into "morphological stability" semantics, i.e., the larger the value, the more stable the morphology).

[0055] The candidate polyps are derived from preliminary detection results in a multimodal colonoscopy image sequence. Specifically, the system uses target detection algorithms (such as a deep learning-based polyp detection model) to extract all image regions suspected of having polyps from each frame of the image (white light, narrow-band imaging, and autofluorescence imaging). These regions are temporally associated using matching algorithms (such as IoU tracking and feature point matching) to form candidate target trajectories. Each trajectory is considered an independent candidate polyp and assigned a unique number. This target has not yet undergone stability and multimodal feature verification and is considered an intermediate result awaiting confirmation.

[0056] Step S3: Calculate the stability coefficient of polyp candidate targets based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset; calculate the verification score of multimodal features for polyp candidate targets based on the multimodal feature verification database; calculate the comprehensive confidence difference between polyp candidate targets.

[0057] Specifically, based on the polyp envelope trajectory stability dataset Polyp morphological stability dataset Calculate candidate polyps The stability coefficient is calculated using the following formula:

[0058] ;

[0059] in, Indicates polyp candidate target stability coefficient, This indicates a weighting coefficient preset by experts based on historical experience;

[0060] It should be noted that in clinical diagnosis, real colonic polyps exhibit high temporal stability in their edge envelope and overall morphology within a continuous sequence of colonoscopy frames; while artifacts caused by air bubbles, fecal residue, intestinal peristalsis / probe jitter will show dramatic contour jitter and abrupt morphological changes in time sequence. This is the core characteristic that distinguishes real polyps from interference signals.

[0061] The stability coefficient calculation formula weights and fuses the envelope trajectory stability and morphological stability within a continuous time window, and then eliminates single-frame random errors by using multi-window averaging. Finally, it transforms the visually discernible temporal stability into a standardized stability coefficient that can be calculated and compared within the 0-1 interval, thereby realizing the mathematical identification of dynamic interference.

[0062] Verification database based on multimodal features Calculate multimodal features for polyp candidate targets The verification score is calculated using the following formula:

[0063] ;

[0064] in, Indicating multimodal features for polyp candidate targets The verification score;

[0065] Based on polyp candidate targets stability coefficient and the multimodal features for polyp candidate targets Validation score Calculate candidate polyps The overall confidence level is calculated using the following formula:

[0066] ;

[0067] in, Indicates polyp candidate target The overall confidence level, This refers to the stability coefficient influence factor, pre-set by experts based on historical experience. This indicates the influence factor of the verification score, pre-set by experts based on historical experience, and ;

[0068] Calculate polyp candidate targets and polyp candidate targets The formula for calculating the overall confidence difference between them is: ,in, Indicates polyp candidate target and polyp candidate targets The difference in overall confidence levels between them Indicates polyp candidate target The overall confidence level.

[0069] In this invention, two fundamentally different sources of confidence—dynamic stability (temporal dimension) and multimodal feature consistency (modal dimension)—are organically fused through adjustable weight coefficients to form a single, physically interpretable comprehensive confidence index. The calculation of confidence differences provides a direct basis for subsequent target similarity measurement, solving the technical challenges of insufficient multimodal information fusion depth and difficulty in balancing dynamic and static evidence in existing technologies. The separate calculation and re-fusion of stability coefficients and verification scores allows the system to flexibly adjust weights according to clinical scenarios (e.g., increasing the stability coefficient weight for areas with vigorous peristalsis), exhibiting adaptability. The comprehensive confidence score, as a unified judgment scale, allows for direct comparison of polyp credibility with non-polyp interference signals on the same scale, greatly improving the scientific rigor and accuracy of decision-making. Furthermore, the confidence difference provides a quantitative link for subsequent coordinated supervision of highly similar interference targets.

[0070] Step S4: Based on the comprehensive confidence difference among polyp candidates, standardize the comprehensive confidence difference into the similarity between polyp candidates; preset threshold, analyze and visualize.

[0071] Specifically, based on polyp candidate targets and polyp candidate targets Difference in overall confidence levels between The overall confidence level difference Standardization as a candidate target for polyps and polyp candidate targets The similarity between them is calculated using the following formula: ,in, Indicates polyp candidate target and polyp candidate targets The similarity between them;

[0072] Preset comprehensive confidence threshold If polyp candidate target Overall confidence level Then determine the candidate target for polyps. To stabilize the polyp and output its location, contour, and modal information; if the polyp is a candidate target Overall confidence level Then determine the candidate target for polyps. This is an interference signal;

[0073] Preset similarity threshold If polyp candidate target If identified as an interference signal, it will be added to the polyp candidate target database. Searching for polyp candidates The similarity is greater than the similarity threshold For other potential polyp targets, if at least one potential polyp target exists, a joint regulatory pair of polyp targets is constructed and visual processing is performed (to indicate that this group of targets needs to be jointly reviewed or filtered in subsequent processing). If no potential polyp target exists, the potential polyp target is... Individually marked as isolated interference signals and removed;

[0074] The visualization process includes: assigning a unique and consistent visual identifier (including color, icon, or outline shape) to the polyp candidate targets in the same joint monitoring pair, and overlaying it on the corresponding multimodal colonoscopy image; providing an information panel for the joint monitoring pair in the image display interface, showing the similarity value, average confidence, and time stability curve between each target in the group.

[0075] In this invention, this step achieves an intelligent transformation from numerical confidence levels to clinically understandable visualization results. Specifically, instead of simply discarding interference signals below the confidence threshold, similarity analysis identifies clusters of interference, such as multiple artifacts generated by the same physical disturbance, and constructs them into linked monitoring pairs to alert doctors or subsequent algorithms for joint review. Truly isolated noise is directly removed. On one hand, outputting highly informative polyps directly through the confidence threshold improves positive detection efficiency; on the other hand, a linked monitoring mechanism for interference signals is proposed, avoiding the potential risk of missed diagnoses that may result from the indiscriminate removal of interference in traditional methods (e.g., small, flat polyps being mistakenly removed due to slightly below the confidence threshold). The visualization of the linked monitoring pairs (consistent colors, icons, outlines, and information panels) significantly enhances the interpretability and user-friendliness of the system output, enabling clinicians to quickly understand the basis of the system's judgment, reducing the review burden, and thus improving the overall safety, practicality, and clinical acceptance of the intelligent colorectal polyp diagnosis system.

[0076] Please see Figure 2 In this second embodiment: a multimodal-based intelligent processing system for visualizing colon polyps is provided. The system includes: a dataset construction module, a database construction module, a stability coefficient calculation and difference calculation module, and a similarity calculation and visualization processing module.

[0077] The dataset construction module includes: constructing a multimodal colonoscopy image database; and constructing a multi-source image data set.

[0078] The database construction module includes: constructing a polyp candidate target database; constructing a multimodal feature verification database; and constructing a polyp envelope trajectory stability dataset and a polyp morphology stability dataset based on the multi-source image data set.

[0079] The stability coefficient calculation and difference calculation module calculates the stability coefficient of polyp candidate targets based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset; calculates the verification score of multimodal features for polyp candidate targets based on the multimodal feature verification database; and calculates the comprehensive confidence difference between polyp candidate targets.

[0080] The similarity calculation and visualization module: based on the comprehensive confidence difference between polyp candidate targets, standardizes the comprehensive confidence difference into the similarity between polyp candidate targets; presets a threshold, analyzes and performs visualization processing.

[0081] Furthermore, the dataset construction module includes a dataset construction unit;

[0082] The dataset construction unit constructs a multimodal colonoscopy image database. This database collects image data generated during the implementation of the multimodal colonic polyp visualization function. The multimodal colonic polyp visualization function includes white light imaging, narrow band imaging, and autofluorescence imaging. White light imaging captures the morphology of the colonic mucosa, narrow band imaging enhances the display of the mucosal surface vascular structure, and autofluorescence imaging captures the fluorescence signal of abnormal polyp tissue metabolism. Any image data in the multimodal colonoscopy image database that implements the multimodal colonic polyp visualization function is uniformly numbered, and a multi-source image data set is constructed.

[0083] Furthermore, the database construction module includes a database construction unit;

[0084] The database construction unit: constructs a polyp candidate target database; constructs a multimodal feature verification database; and, based on the multi-source image data set, obtains the envelope trajectory stability data and morphological stability data of the polyp candidate targets, and constructs a polyp envelope trajectory stability dataset and a polyp morphological stability dataset, respectively.

[0085] Furthermore, the stability coefficient calculation and difference calculation module includes a stability coefficient calculation unit and a difference calculation unit;

[0086] The stability coefficient calculation unit calculates the stability coefficient of candidate polyps based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset.

[0087] The difference calculation unit: calculates the verification score of the multimodal features for the polyp candidate targets based on the multimodal feature verification database; calculates the comprehensive confidence of the polyp candidate targets based on the stability coefficient of the polyp candidate targets and the verification score of the multimodal features for the polyp candidate targets; and calculates the difference in comprehensive confidence between the polyp candidate targets.

[0088] Furthermore, the similarity calculation and visualization processing module includes a similarity calculation unit and a visualization processing unit;

[0089] The similarity calculation unit: based on the comprehensive confidence difference between polyp candidate targets, standardizes the comprehensive confidence difference into the similarity between polyp candidate targets;

[0090] The visualization processing unit has the following features: a preset comprehensive confidence threshold; if the comprehensive confidence of a polyp candidate is greater than or equal to the comprehensive confidence threshold, the polyp candidate is determined to be a stable polyp; if the comprehensive confidence of a polyp candidate is less than the comprehensive confidence threshold, the polyp candidate is determined to be an interference signal; and a preset similarity threshold; if a polyp candidate is determined to be an interference signal, other polyp candidate targets with a similarity greater than the similarity threshold are searched in the polyp candidate target database. If at least one polyp candidate target exists, a linked monitoring pair of polyp targets is constructed and visualization processing is performed; if no polyp candidate target exists, the polyp candidate target is marked as an isolated interference signal and removed.

[0091] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0092] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal-based intelligent visualization method for colonic polyps, characterized in that, The method includes the following steps: Step S1: Construct a multimodal colonoscopy image database; construct a multi-source image data set; Step S2: Construct a candidate polyp database; construct a multimodal feature verification database; based on the multi-source image data set, construct a polyp envelope trajectory stability dataset and a polyp morphology stability dataset; Step S3: Based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset, calculate the stability coefficient of the polyp candidate targets; based on the multimodal feature verification database, calculate the verification score of the multimodal features for the polyp candidate targets; calculate the comprehensive confidence difference between the polyp candidate targets. Step S4: Based on the comprehensive confidence difference among polyp candidates, standardize the comprehensive confidence difference into the similarity between polyp candidates; preset threshold, analyze and visualize.

2. The intelligent processing method for visualizing colonic polyps based on multimodality according to claim 1, characterized in that, The specific implementation process of step S1 includes: A multimodal colonoscopy image database is constructed. The multimodal colonoscopy image database is used to collect image data generated when realizing the multimodal colon polyp visualization function. The multimodal colon polyp visualization function includes white light imaging function, narrow band imaging function and automatic fluorescence imaging function. The white light imaging function realizes the acquisition of the morphology of the colonic mucosa surface, the narrow band imaging function realizes the enhanced display of the vascular structure of the mucosa surface, and the automatic fluorescence imaging function realizes the capture of fluorescence signals of abnormal metabolism of polyp tissue. Any image data in the multimodal colonoscopy image database that implements the multimodal colon polyp visualization function is uniformly numbered, and a multi-source image data set is constructed, denoted as . ,in, This represents the m-th image data that enables multimodal visualization of colon polyps. This represents the total number of image data used to achieve the multimodal colon polyp visualization function.

3. The intelligent processing method for visualizing colonic polyps based on multimodality according to claim 2, characterized in that, The specific implementation process of step S2 includes: Construct a database of candidate polyps ,in, Indicates the first One polyp candidate This represents the total number of polyp candidates; a multimodal feature verification database is constructed, denoted as... ,in, Indicates the first The multimodal feature for the first Validation data for individual polyp candidates. This represents the total number of multimodal feature validation data. Based on the multi-source image data set, envelope trajectory stability data and morphological stability data of polyp candidate targets are obtained, and polyp envelope trajectory stability datasets are constructed respectively. Polyp morphological stability dataset ,in, Indicates the first One polyp candidate target in the Data on the stability of the envelope trajectory within a time window Indicates the first One polyp candidate target in the Morphological stability data within a time window This indicates the total number of time windows.

4. The intelligent processing method for visualizing colonic polyps based on multimodality according to claim 3, characterized in that, The specific implementation process of step S3 includes: Based on polyp envelope trajectory stability dataset Polyp morphological stability dataset Calculate candidate polyps The stability coefficient is calculated using the following formula: ； in, Indicates polyp candidate target stability coefficient, This indicates the preset weighting coefficients; Verification database based on multimodal features Calculate multimodal features for polyp candidate targets The verification score is calculated using the following formula: ； in, Indicating multimodal features for polyp candidate targets The verification score; Based on polyp candidate targets stability coefficient and the multimodal features for polyp candidate targets Validation score Calculate candidate polyps The overall confidence level is calculated using the following formula: ； in, Indicates polyp candidate target The overall confidence level, This represents the preset stability coefficient influence factor. This represents the preset validation score influence factor, and ; Calculate polyp candidate targets and polyp candidate targets The formula for calculating the overall confidence difference between them is: ,in, Indicates polyp candidate target and polyp candidate targets The difference in overall confidence levels between them Indicates polyp candidate target The overall confidence level.

5. The method for intelligent visualization processing of colonic polyps based on multimodality according to claim 4, characterized in that, The specific implementation process of step S4 includes: Based on polyp candidate targets and polyp candidate targets Difference in overall confidence levels between The overall confidence level difference Standardization as a candidate target for polyps and polyp candidate targets The similarity between them is calculated using the following formula: ,in, Indicates polyp candidate target and polyp candidate targets The similarity between them; Preset comprehensive confidence threshold If polyp candidate target Overall confidence level Then determine the candidate target for polyps. To stabilize polyps; if polyp candidates Overall confidence level Then determine the candidate target for polyps. This is an interference signal; Preset similarity threshold If polyp candidate target If identified as an interference signal, it will be added to the polyp candidate target database. Searching for polyp candidates The similarity is greater than the similarity threshold If at least one polyp candidate exists, a linked regulatory pair for the polyp candidate is constructed and visualization processing is performed; otherwise, the polyp candidate is... Individually marked as isolated interference signals and removed.

6. A multimodal-based intelligent processing system for visualizing colonic polyps, executing the multimodal-based intelligent processing method for visualizing colonic polyps as described in any one of claims 1-5, characterized in that, The system includes: a dataset construction module, a database construction module, a stability coefficient calculation and difference calculation module, and a similarity calculation and visualization processing module; The dataset construction module includes: constructing a multimodal colonoscopy image database; and constructing a multi-source image data set. The database construction module includes: constructing a polyp candidate target database; constructing a multimodal feature verification database; and constructing a polyp envelope trajectory stability dataset and a polyp morphology stability dataset based on the multi-source image data set. The stability coefficient calculation and difference calculation module calculates the stability coefficient of polyp candidate targets based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset; calculates the verification score of multimodal features for polyp candidate targets based on the multimodal feature verification database; and calculates the comprehensive confidence difference between polyp candidate targets. The similarity calculation and visualization module: based on the comprehensive confidence difference between polyp candidate targets, standardizes the comprehensive confidence difference into the similarity between polyp candidate targets; presets a threshold, analyzes and performs visualization processing.

7. The intelligent processing system for visualizing colonic polyps based on multimodality according to claim 6, characterized in that: The dataset construction module includes dataset construction units; The dataset construction unit constructs a multimodal colonoscopy image database. This database collects image data generated during the implementation of the multimodal colonic polyp visualization function. The multimodal colonic polyp visualization function includes white light imaging, narrow band imaging, and autofluorescence imaging. White light imaging captures the morphology of the colonic mucosa, narrow band imaging enhances the display of the mucosal surface vascular structure, and autofluorescence imaging captures the fluorescence signal of abnormal polyp tissue metabolism. Any image data in the multimodal colonoscopy image database that implements the multimodal colonic polyp visualization function is uniformly numbered, and a multi-source image data set is constructed.

8. The intelligent processing system for visualizing colonic polyps based on multimodality according to claim 7, characterized in that: The database construction module includes database construction units; The database construction unit includes: constructing a polyp candidate target database; and constructing a multimodal feature verification database. Based on the multi-source image data set, envelope trajectory stability data and morphological stability data of polyp candidate targets are obtained, and polyp envelope trajectory stability dataset and polyp morphological stability dataset are constructed respectively.

9. The intelligent visual processing system for colon polyps based on multimodality according to claim 8, characterized in that: The stability coefficient calculation and difference calculation module includes a stability coefficient calculation unit and a difference calculation unit; The stability coefficient calculation unit calculates the stability coefficient of candidate polyps based on the polyp envelope trajectory stability dataset and the polyp morphology stability dataset. The difference calculation unit calculates the verification score of multimodal features for polyp candidate targets based on the multimodal feature verification database. Based on the stability coefficient of the polyp candidate targets and the verification score of the polyp candidate targets by the multimodal features, the comprehensive confidence of the polyp candidate targets is calculated; the difference in comprehensive confidence among the polyp candidate targets is calculated.

10. The intelligent processing system for visualizing colonic polyps based on multimodality according to claim 9, characterized in that: The similarity calculation and visualization processing module includes a similarity calculation unit and a visualization processing unit; The similarity calculation unit: based on the comprehensive confidence difference between polyp candidate targets, standardizes the comprehensive confidence difference into the similarity between polyp candidate targets; The visualization processing unit: presets a comprehensive confidence threshold; if the comprehensive confidence of a polyp candidate is greater than or equal to the comprehensive confidence threshold, then the polyp candidate is determined to be a stable polyp. If the overall confidence level of the polyp candidate is less than the overall confidence level threshold, the polyp candidate is determined to be an interference signal; If a pre-set similarity threshold is set and a candidate polyp is identified as an interference signal, other candidate polyps with a similarity greater than the similarity threshold are searched in the candidate polyp database. If at least one candidate polyp exists, a linked monitoring pair of the polyp is constructed and visualization processing is performed. If no candidate polyp exists, the candidate polyp is marked as an isolated interference signal and removed.

Images