Gastrointestinal tract endoscope image segmentation method

Abstract

The invention relates to the technical field of image segmentation, and provides a gastrointestinal tract endoscope image segmentation method, which comprises the following steps: carrying out time sequence preprocessing on an obtained intestinal tract video, eliminating fuzzy and shielded invalid frames, constructing a rhythm signal based on a multi-scale time sequence visual feature, and estimating a creeping phase; outputting a phase time sequence and a phase label of the video; selecting a representative phase key frame, performing pixel-level focus segmentation on the key frame, and calculating a pixel-level uncertainty score to obtain a key frame segmentation result with high confidence; carrying out time sequence consistency recovery on the segmentation mask of the key frame based on phase consistency mapping and phase alignment, and recovering an uninterpretable section caused by shielding with the assistance of adaptive segmentation correction of motion perception; and carrying out time sequence aggregation on a cross-frame segmentation result, extracting fragment-level statistical representation and confidence, forming a fragment-level suspicious area report, and outputting structured data for clinical auditing and index retrieval.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, specifically a method for segmenting images from gastrointestinal endoscopy. Background Technology

[0002] Gastrointestinal endoscopes are devices that use a flexible tube equipped with a miniature camera to enter through the upper or lower digestive tract to examine and diagnose the internal condition of the esophagus, stomach, small intestine, or large intestine. They can diagnose a variety of diseases and, when necessary, perform biopsies and therapeutic procedures.

[0003] The Chinese patent application (CN202410856225.0) discloses a biological process control method based on a knowledge graph-directed graph. This method includes: acquiring multi-source gastrointestinal endoscopic images; expanding the data diversity of the multi-source gastrointestinal endoscopic images to obtain a multi-source image dataset; using a dual-feature extraction strategy to extract local classical features and global auxiliary features from the multi-source image dataset; fusing the local classical features and global auxiliary features to obtain multi-source feature fusion data; using an image hotspot algorithm to perform multi-source image fusion on the multi-source feature fusion data to obtain a multi-source fused feature image; constructing a pulse-coupled neural network to perform preliminary image segmentation on the multi-source fused feature image to obtain preliminary image segmentation data; and introducing a high-frequency enhancement algorithm to enhance the preliminary image segmentation data to complete the image segmentation. By acquiring multi-source images, extracting multiple features, and fusing them, the accuracy of gastrointestinal endoscopic image segmentation is improved.

[0004] In the field of image segmentation technology, although there are technical solutions that use two types of endoscopes to acquire multi-source endoscopic images of the same object and use image fusion algorithms to fuse the feature information in the multi-source images, in capsule endoscopy screening scenarios, there are problems such as high false alarm rate and unstable lesion boundaries when dealing with complex images. Especially under interference conditions such as bubbles and local bleeding, repeated confirmation is often required, resulting in long image reading time and high verification cost, making it difficult to form objective and comparable data results, thus increasing the dual risks of missed diagnosis and over-alarm.

[0005] To address the aforementioned issues, this invention proposes a method for segmenting gastrointestinal endoscopic images. This method involves initial screening of images that can be judged, estimating the peristaltic phase, selecting keyframes of the phase, segmenting lesions from these keyframes, mapping the keyframes to the corresponding time axis, compensating for unreadable segments, summarizing indicators of suspicious regions, and generating a segment-level report. Summary of the Invention

[0006] In view of the existing problems mentioned above, a method for segmenting gastrointestinal endoscopic images is proposed.

[0007] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a method for segmenting gastrointestinal endoscopic images, comprising:

[0008] The acquired intestinal video is preprocessed temporally to remove invalid frames with blurriness and occlusion. Based on the extracted and fused multi-scale temporal visual features, a rhythm signal is constructed and the peristaltic phase is estimated. The phase time series of the video and the phase label of each frame are output.

[0009] Based on the phase time series, representative phase keyframes are selected in each period, pixel-level lesion segmentation is performed on the keyframes, and pixel-level uncertainty scores are calculated to obtain high-confidence keyframe segmentation results.

[0010] The segmentation mask of the key frame is mapped back to the original video timeline based on phase consistency. Temporal consistency is restored based on phase alignment and supplemented by motion-aware adaptive segmentation correction to restore unreadable segments caused by occlusion. At the same time, the mapping result is weighted by confidence and adaptively smoothed.

[0011] The cross-frame segmentation results are time-series aggregated, and fragment-level statistical representations and confidence scores are extracted to form fragment-level suspicious region reports. Structured data for clinical review and index retrieval are output.

[0012] As a preferred embodiment, the specific steps for removing invalid frames that are blurred or occluded are as follows:

[0013] The video acquired by capsule endoscopy is processed frame by frame to remove distortion, normalize color and standardize brightness. Blur measurement and occlusion score are calculated for each frame and compared with preset thresholds. Invalid frames with blur and occlusion are removed, and confidence index is assigned to the retained low-quality frames.

[0014] Each frame of video is converted to a grayscale image. A Laplacian filter is applied to the grayscale image to obtain a Laplacian response image. The mean and variance of the response image are calculated, and the variance is used as the Laplacian sharpness measure. The formula is as follows:

[0015] ,

[0016] in Let N represent the mean and N represent the total number of pixels counted. Represents pixels Laplace response diagram, Indicates variance;

[0017] Convolve the grayscale image with horizontal and vertical gradient kernels respectively, calculate the gradient magnitude of each pixel and take the mean, then set the mean as the gradient sharpness metric; calculate the two-dimensional discrete Fourier transform of the grayscale image and obtain the energy spectrum, calculate the high-frequency power ratio, and set the high-frequency power ratio as the frequency domain sharpness metric, with the following formula:

[0018] ,

[0019] in Indicates the high-frequency power ratio. Represents discrete frequency index coordinates. Indicates a high-frequency mask. Indicated at frequency grid points Power spectral density at;

[0020] The Laplacian, gradient, and frequency domain sharpness metrics are normalized. A blur index is calculated by fusing these metrics according to pre-defined weights, with values ​​closer to 1 indicating greater blur and closer to 0 indicating greater sharpness. This index is then compared to a pre-defined blur threshold. For each frame, a pixel-level occlusion indicator is constructed using a combination of local brightness variance, local saturation statistics, local dark channel minimums, and high-brightness / low-saturation metrics in the color space. A pixel-level occlusion confidence map is generated using linear weighting, and a frame-level occlusion score is obtained through area-weighted summation. The blur index and occlusion score are then combined using a weighted linear combination to calculate a frame quality confidence score, categorized as valid, low-quality, or invalid.

[0021] As a preferred embodiment, the specific steps for constructing a rhythmic signal based on extracted and fused multi-scale temporal visual features and estimating the creep phase are as follows:

[0022] A spatial pyramid is constructed for valid, low-quality frames. At each scale, luminance histogram statistics, multi-scale local texture descriptors, and edge energy are calculated. Scale responses are normalized and weighted by scale confidence. Inter-frame motion fields and motion energy are calculated for adjacent frames. Motion amplitude and direction are statistically analyzed within a time window to obtain motion energy sequences and local dynamic characteristics. Time-frequency analysis is performed on the scaled temporal window within a sliding window to identify dominant frequency components and calculate their corresponding instantaneous phases. Noisy dominant frequencies are removed using spectral peak filtering and confidence thresholding. A weighted synthesis strategy is applied to the multi-scale results, where the weights are determined by scale confidence and spectral energy, to obtain frame-level instantaneous phase estimates. Invalid frames are recorded as verification items and include quality indicators in the output structured data.

[0023] As a preferred embodiment, the specific steps for selecting representative phase keyframes are as follows:

[0024] Based on the obtained phase time series, the video is divided into physiological rhythm cycles, and within each cycle, candidate frames with close phase and the highest frame quality score are selected as representative phase keyframes, using a predetermined reference phase as the benchmark.

[0025] As a preferred embodiment, the specific steps for performing pixel-level lesion segmentation on keyframes and calculating pixel-level uncertainty scores are as follows:

[0026] Phase keyframes are input into the pixel-level segmentation module to obtain the predicted probability distribution of each pixel. Inference is repeated using random perturbations during inference to obtain pixel-level prediction distribution statistics. Furthermore, the average predicted probability and uncertainty description of each pixel are derived. A joint judgment is made based on the average probability and uncertainty. When the average probability of a pixel is higher than a preset probability threshold and the uncertainty is lower than a preset uncertainty threshold, it is included in the high-confidence mask; otherwise, it is marked as low-confidence. Morphological refinement and boundary consistency correction are performed on the obtained high-confidence mask. At the same time, the average confidence and uncertainty of the region are calculated at the region level. The overall reliability of each candidate region is quantified. Candidate regions are screened based on the region confidence and uncertainty thresholds. When the average confidence of the region is higher than a preset probability threshold and the uncertainty is lower than a preset uncertainty threshold, it is confirmed as a high-confidence lesion region; otherwise, it is marked as a low-confidence candidate and included in the subsequent phase mapping compensation process.

[0027] Normalization and size adjustment are performed according to the preset resolution and channel order, and the images are fed into a pre-trained pixel-level segmentation model. The segmentation model extracts features layer by layer from the entire image and outputs the probability value of each pixel position belonging to the lesion category and not belonging to the lesion category, forming a probability map with the same size as the input image. The probability value at the same coordinate position is used to indicate the confidence level of the pixel being a lesion pixel.

[0028] As a preferred embodiment, the specific steps for timing consistency recovery based on phase alignment are as follows:

[0029] For ordinary frames with similar phases and adjacent times, the segmentation mask of the keyframe is forward-projected using pre-estimated inter-frame motion information. After spatial position correction, the keyframe mask is mapped onto the target frame, resulting in an initial segmentation result aligned with the target frame. The mapped segmentation result is then adaptively corrected by combining the target frame's own frame quality label, occlusion score, and phase position confidence. When the target frame quality is valid, its own segmentation is allowed to be retained. When the target frame is occluded or blurred and cannot be interpreted, the forward and backward mapping results from multiple adjacent keyframes are used, and weighted and fused according to phase distance, motion consistency, and keyframe confidence. The formula is as follows:

[0030] ,

[0031] Where t represents the index of the target frame, Represents the pixel coordinates on the target frame. Represents the pixels on target frame t The final fused mask value, where K represents the index set of the keyframe set, and k represents the keyframe index. This represents the weight of keyframe k;

[0032] As a preferred embodiment, the specific steps for performing confidence-weighted and adaptive smoothing on the mapping results are as follows:

[0033] For the segmentation results across the entire video, adaptive smoothing based on confidence is performed in the temporal dimension. The segmentation confidence from multiple adjacent frames at the same spatial location is weighted and averaged, and inconsistent isolated responses are suppressed to obtain a final temporal segmentation sequence that is continuous in phase.

[0034] As a preferred embodiment, the specific steps for temporally aggregating the cross-frame segmentation results and extracting segment-level statistical representations and confidence scores are as follows:

[0035] Lesion regions that are near the same spatial location, of the same category, and appear consecutively in adjacent frames are associated in chronological order. Temporally connected regions are merged into suspicious segments, and the start and end frame numbers, start and end timestamps, and phase range covered by each suspicious segment are recorded. Within each suspicious segment, lesion regions across frames are statistically summarized to obtain a segment-level statistical representation of the segment. A comprehensive confidence score and review mark are calculated for the segment based on pixel-level and region-level confidence. Furthermore, the suspicious segments are sorted and numbered according to chronological order and severity to generate a segment-level suspicious region report.

[0036] Beneficial effects

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. By scoring each frame for blur and occlusion, interference areas are identified. Then, pixel-level uncertainty and region-level confidence are combined to perform a second screening of the segmentation results, reducing the false alarm rate and the workload of repeated comparisons.

[0039] 2. By using multi-period, multi-view phase keyframes to mask and interpolate unreadable segments that are identified as occluded, blurry, or difficult to interpret, the segments are restored to clear phase, reducing the risk of missed diagnoses. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0041] Figure 1 This is a flowchart illustrating the present invention;

[0042] Figure 2 This is a comparison diagram of the effects of the present invention and the prior art, where gray bars represent the prior art and black bars represent the present invention. Detailed Implementation

[0043] To make the technical means, creative features, objectives, and effects of this invention easier to understand, the invention is further described below with reference to specific embodiments. However, the following embodiments are merely preferred embodiments of this invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments described herein without creative effort are all within the protection scope of this invention.

[0044] Example 1:

[0045] To achieve the above objectives, please refer to Figures 1 to 2 This invention provides a method for segmenting gastrointestinal endoscopic images, the method comprising the following steps:

[0046] Step S1: Initial screening can determine the image and estimate the creep phase;

[0047] Step S2: Select phase keyframes and segment the lesions;

[0048] Step S3: Map the phase to the corresponding time axis and compensate for unreadable segments;

[0049] Step S4: Summarize the indicators of suspicious areas to form a segment-level report.

[0050] This method is implemented in the order of S1–S4, and its overall process is as follows:

[0051] The acquired intestinal video is preprocessed temporally to remove invalid frames with blurriness and occlusion. Based on the extracted and fused multi-scale temporal visual features, a rhythm signal is constructed and the peristaltic phase is estimated. The phase time series of the video and the phase label of each frame are output.

[0052] Based on the phase time series, representative phase keyframes are selected in each period, pixel-level lesion segmentation is performed on the keyframes, and pixel-level uncertainty scores are calculated to obtain high-confidence keyframe segmentation results.

[0053] The segmentation mask of the key frame is mapped back to the original video timeline based on phase consistency. Temporal consistency is restored based on phase alignment and supplemented by motion-aware adaptive segmentation correction to restore unreadable segments caused by occlusion. At the same time, the mapping result is weighted by confidence and adaptively smoothed.

[0054] The cross-frame segmentation results are time-series aggregated, and fragment-level statistical representations and confidence scores are extracted to form fragment-level suspicious region reports. Structured data for clinical review and index retrieval are output.

[0055] Combination Figure 1The specific implementation methods and effects of steps S1–S4 are explained respectively. Among them, the initial screening can determine the image, and the specific steps of creep phase estimation are as follows:

[0056] The video acquired by capsule endoscopy is processed frame by frame to remove distortion, normalize color and standardize brightness. Blur measurement and occlusion score are calculated for each frame and compared with preset thresholds. Invalid frames with blur and occlusion are removed, and confidence index is assigned to the retained low-quality frames.

[0057] Specifically, the calculation of blur metric and occlusion score for each frame first involves converting each video frame into a grayscale image, applying a Laplacian filter to the grayscale image to obtain a Laplacian response image, and calculating the mean and variance of the response image. The variance is then used as the Laplacian sharpness metric, as shown in the formula:

[0058] ,

[0059] in Let N represent the mean and N represent the total number of pixels counted. Represents pixels Laplace response diagram, Indicates variance;

[0060] Convolve the grayscale image with horizontal and vertical gradient kernels respectively, calculate the gradient magnitude of each pixel and take the mean, then set the mean as the gradient sharpness metric; calculate the two-dimensional discrete Fourier transform of the grayscale image and obtain the energy spectrum, calculate the high-frequency power ratio, and set the high-frequency power ratio as the frequency domain sharpness metric, with the following formula:

[0061] ,

[0062] in Indicates the high-frequency power ratio. Represents discrete frequency index coordinates. Indicates a high-frequency mask. Indicated at frequency grid points Power spectral density at;

[0063] The three sharpness metrics are normalized and then fused according to pre-set weights to calculate a blur index. A value close to 1 indicates greater blurriness, and a value close to 0 indicates greater sharpness. This is compared with a set blur threshold; for example, a value greater than or equal to 0.6 is considered blurry. For each frame, a pixel-level occlusion indicator is constructed using a joint metric of local brightness variance, local saturation statistics, local dark channel minimum, and high brightness / low saturation in the color space. A pixel-level occlusion confidence map is generated by linear weighting, and a frame-level occlusion score is obtained by area-weighted summation. The blur index and occlusion score are then combined by weighted linear combination to calculate a frame quality confidence score, which is categorized as valid, low quality, or invalid. For example, a frame quality confidence score greater than or equal to 0.7 is considered valid, greater than 0.3 and less than 0.7 is considered low quality, and less than or equal to 0.3 is considered invalid.

[0064] A spatial pyramid is constructed for valid, low-quality frames. At each scale, luminance histogram statistics, multi-scale local texture descriptors, and edge energy are calculated. Scale responses are normalized and weighted by scale confidence. Inter-frame motion fields and motion energy are calculated for adjacent frames. Motion amplitude and direction are statistically analyzed within a time window to obtain motion energy sequences and local dynamic characteristics. Time-frequency analysis is performed on the scaled temporal window within a sliding window to identify dominant frequency components and calculate their corresponding instantaneous phases. Noisy dominant frequencies are removed using spectral peak filtering and confidence thresholding. A weighted synthesis strategy is applied to the multi-scale results, where the weights are determined by scale confidence and spectral energy, to obtain frame-level instantaneous phase estimates. Invalid frames are recorded as verification items and include quality indicators in the output structured data.

[0065] The specific steps for selecting phase keyframes and performing lesion segmentation are as follows:

[0066] Based on the obtained phase time series, the video is divided into physiological rhythm cycles, and within each cycle, candidate frames with close phase and the highest frame quality score are selected as representative phase keyframes, using a predetermined reference phase as the benchmark.

[0067] Phase keyframes are input into the pixel-level segmentation module to obtain the predicted probability distribution of each pixel. Inference is repeated using random perturbations during inference to obtain pixel-level prediction distribution statistics. Furthermore, the average predicted probability and uncertainty description of each pixel are derived. A joint judgment is made based on the average probability and uncertainty. When the average probability of a pixel is higher than a preset probability threshold and the uncertainty is lower than a preset uncertainty threshold, it is included in the high-confidence mask; otherwise, it is marked as low-confidence. Morphological refinement and boundary consistency correction are performed on the obtained high-confidence mask. At the same time, the average confidence and uncertainty of the region are calculated at the region level. The overall reliability of each candidate region is quantified. Based on the region confidence and uncertainty thresholds, the candidate regions are finally screened. When the average confidence of the region is higher than the preset probability threshold and the uncertainty is lower than the preset uncertainty threshold, it is confirmed as a high-confidence lesion region; otherwise, it is marked as a low-confidence candidate and included in the subsequent phase mapping compensation process.

[0068] Specifically, the phase keyframe is input into the pixel-level segmentation module to obtain the predicted probability distribution of each pixel, which is then normalized and resized according to a preset resolution and channel order, and fed into a pre-trained pixel-level segmentation model. This invention uses a U-Net neural network structure. The segmentation model extracts features layer by layer from the entire image and outputs the probability value of each pixel position belonging to the lesion category and not to the lesion category, forming a probability map with the same size as the input image. The probability value at the same coordinate position is used to indicate the confidence level of the pixel being a lesion pixel.

[0069] The specific steps for mapping the phase to the time axis and compensating for unreadable segments are as follows:

[0070] For ordinary frames with similar phases and adjacent times, the segmentation mask of the keyframe is forward-projected using pre-estimated inter-frame motion information. After spatial position correction, the keyframe mask is mapped onto the target frame, resulting in an initial segmentation result aligned with the target frame. The mapped segmentation result is then adaptively corrected by combining the target frame's own frame quality label, occlusion score, and phase position confidence. When the target frame quality is valid, its own segmentation is allowed to be retained. When the target frame is occluded or blurred and cannot be interpreted, the forward and backward mapping results from multiple adjacent keyframes are used, and weighted and fused according to phase distance, motion consistency, and keyframe confidence. The formula is as follows:

[0071] ,

[0072] Where t represents the index of the target frame, Represents the pixel coordinates on the target frame. Represents the pixels on target frame t The final fused mask value, where K represents the index set of the keyframe set, and k represents the keyframe index. This represents the weight of keyframe k;

[0073] For the segmentation results across the entire video, adaptive smoothing based on confidence is performed in the temporal dimension. Specifically, the segmentation confidence from multiple adjacent frames at the same spatial location is weighted and isolated inconsistent responses are suppressed to obtain a final temporal segmentation sequence that is continuous in phase.

[0074] The specific steps for summarizing indicators from suspicious areas and generating a segment-level report are as follows:

[0075] Lesion regions that are near the same spatial location, of the same category, and appear consecutively in adjacent frames are associated in chronological order. Temporally connected regions are merged into suspicious segments, and the start and end frame numbers, start and end timestamps, and phase range covered by each suspicious segment are recorded. Within each suspicious segment, lesion regions across frames are statistically summarized to obtain segment-level statistical characteristics such as the area variation range, average lesion area, duration of occurrence, frequency of occurrence across frames, center location trajectory, average prediction probability, uncertainty trend, and quality distribution of the involved frames. A comprehensive confidence score and review mark are calculated for the segment based on pixel-level and region-level confidence. Furthermore, suspicious segments are sorted and numbered according to chronological order and severity to generate segment-level suspicious area reports. The reports are output in structured data format, and suspicious segments can be quickly located on the timeline in the review interface to view the corresponding statistical information and key frame images.

[0076] like Figure 2 A comparison chart of the effects of a gastrointestinal endoscopic image segmentation method is presented. The horizontal axis lists key performance indicators, and the vertical axis represents the exemplified performance score, ranging from 0 to 100. Higher values ​​indicate better performance. The aim is to intuitively demonstrate the expected improvement of the present invention in key capabilities compared to typical prior art.

[0077] Example 2:

[0078] Based on the above embodiment 1, a gastrointestinal endoscopic image segmentation method for capsule endoscopy screening is specifically as follows:

[0079] Step 1: The video acquired by the capsule endoscopy is processed frame by frame for distortion correction and size normalization. Each frame is compressed into a sequence of small color patches with a fixed resolution and input into a frame-level feature extraction network to obtain multi-dimensional feature vectors representing brightness distribution, texture details, cavity morphology, and content distribution. The feature vectors of several adjacent frames are then fed into a temporal modeling network, such as a bidirectional recurrent network, in chronological order. The quality branch outputs the frame quality confidence and corresponding quality label for each frame, classifying them into three categories: valid, low-quality, and invalid. The rhythm branch performs periodic pattern recognition and phase regression on the feature sequences within the same time window, outputting the normalized phase value and phase position confidence of each frame within a standard peristaltic cycle. For frames deemed invalid by the quality branch, their timestamps and reasons for invalidity are recorded as a verification item. For valid and low-quality frames, their frame quality confidence, phase labels, and phase position confidence are retained as inputs for subsequent phase keyframe selection and phase alignment mapping.

[0080] Step 2: Based on the phase time series, each creeping cycle is divided into phase sub-intervals. Within each sub-interval, frames are jointly sorted according to frame quality score and viewpoint difference. Two frames are selected as complementary viewpoints to form the keyframe set for that cycle. After normalizing the phase keyframes according to a preset resolution, they are input into two pixel-level segmentation models with different structures: an encoding and decoding network and a semantic segmentation network with dilated convolutions, serving as the first and second segmentation models, respectively. For each pixel location, a predicted probability map of the lesion category is output. The outputs of the two models are then summarized and statistically analyzed at the pixel level to obtain the predicted probability map for each lesion category. The average predicted probability of each pixel and the difference between models are used to measure the difference. Pixels with high average probability and small difference between models are marked as high-confidence pixels, and the remaining pixels are marked as low-confidence pixels. Region growing and morphological operations are performed on the high-confidence pixel mask to obtain candidate lesion regions. The region-level confidence is calculated according to the proportion of high-confidence pixels within the region and the region's average probability. The candidate regions are screened according to the preset region-level probability threshold and difference threshold. Regions with high confidence and small difference are identified as high-confidence lesion regions, and the remaining regions are retained as low-confidence candidates for subsequent phase mapping and compensation.

[0081] Step 3: For each ordinary frame with similar phase and temporal proximity, the original image, phase label, frame quality label of that frame, along with the images and segmentation masks of multiple adjacent phase keyframes, are stacked in chronological order into a small temporal segment. This segment is then input into a pre-trained temporal segmentation network, such as a 3D convolutional network with temporal attention. The network first encodes the common spatial features of each frame, then adaptively assigns weights in the temporal and phase dimensions, weighted aggregates the features from different keyframes and ordinary frames, and outputs the fine segmentation result and pixel-level positioning of the current target frame. For target frames whose frame quality is deemed valid, the network automatically focuses more on their own texture and boundary features. For target frames that are marked as occluded or blurred and cannot be directly interpreted, the network learns more high-quality features and mask information from adjacent keyframes during training to complete the missing regions and correct boundaries. At the frame level, the segmentation confidence of each frame is obtained based on pixel confidence statistics, and a lightweight phase-sensitive smoothing filter is used on the time axis to smooth the segmentation results of adjacent frames, suppressing short-term isolated anomalous responses and obtaining a final temporal segmentation sequence that is continuous in phase.

[0082] Step 4: The video is segmented using a fixed-length sliding window. Within each window, lesion regions of frames are collected, and their timestamps, phase labels, centroids, areas, and frame-level confidence scores are recorded. These are encoded into temporal, spatial, and phase feature sequences and input into the segment clustering module. Based on time intervals and spatial distances, the regions are adaptively divided into candidate segments, and the start and end frame numbers, start and end timestamps, and phase intervals of each candidate segment are output. Each candidate segment is treated as an event, and its duration, average area, area fluctuation, center trajectory, and the mean and variance of frame-level confidence scores are statistically analyzed. These features are input into a pre-trained segment scoring network, which outputs the segment-level comprehensive confidence score and severity level, and provides a label indicating whether review is recommended. All candidate segments are sorted and numbered according to their start and end times and severity, generating a structured segment report. This report is then provided to the clinical review system via an interface for searching and displaying suspicious segments along the timeline.

[0083] The embodiments of the present invention described above are subject to modification and change of method by those skilled in the art without departing from the embodiments and broader aspects of the present invention. The appended claims are intended to include all such modifications and changes of method that do not depart from the present invention.

Claims

1. A method for segmenting images from gastrointestinal endoscopy, characterized in that, include: The acquired intestinal video is preprocessed temporally to remove invalid frames with blurriness and occlusion. Based on the extracted and fused multi-scale temporal visual features, a rhythm signal is constructed and the peristaltic phase is estimated. The phase time series of the video and the phase label of each frame are output. Based on the phase time series, representative phase keyframes are selected in each period, pixel-level lesion segmentation is performed on the keyframes, and pixel-level uncertainty scores are calculated to obtain high-confidence keyframe segmentation results. The segmentation mask of the key frame is mapped back to the original video timeline based on phase consistency. Temporal consistency is restored based on phase alignment and supplemented by motion-aware adaptive segmentation correction to restore unreadable segments caused by occlusion. At the same time, the mapping result is weighted by confidence and adaptively smoothed. The cross-frame segmentation results are time-series aggregated, and fragment-level statistical representations and confidence scores are extracted to form fragment-level suspicious region reports. Structured data for clinical review and index retrieval are output.

2. The method for segmenting gastrointestinal endoscopic images according to claim 1, characterized in that, The specific steps for removing invalid frames that are blurry or occluded are as follows: The video acquired by capsule endoscopy is processed frame by frame to remove distortion, normalize color and standardize brightness. Blur measurement and occlusion score are calculated for each frame and compared with preset thresholds. Invalid frames with blur and occlusion are removed, and confidence index is assigned to the retained low-quality frames. Each frame of video is converted to a grayscale image. A Laplacian filter is applied to the grayscale image to obtain a Laplacian response image. The mean and variance of the response image are calculated, and the variance is used as the Laplacian sharpness measure. The formula is as follows: ， in Let N represent the mean and N represent the total number of pixels counted. Represents pixels Laplace response diagram, Indicates variance.

3. The method for segmenting gastrointestinal endoscopic images according to claim 2, characterized in that, The specific steps for removing invalid frames that are blurry or occluded also include: Convolve the grayscale image with horizontal and vertical gradient kernels respectively, calculate the gradient magnitude of each pixel and take the mean, then set the mean as the gradient sharpness metric; calculate the two-dimensional discrete Fourier transform of the grayscale image and obtain the energy spectrum, calculate the high-frequency power ratio, and set the high-frequency power ratio as the frequency domain sharpness metric, with the following formula: ， in Indicates the high-frequency power ratio. Represents discrete frequency index coordinates. Indicates a high-frequency mask. Indicated at frequency grid points Power spectral density at; The Laplacian, gradient, and frequency domain sharpness metrics are normalized. A blur index is calculated by fusing these metrics according to pre-defined weights, with values ​​closer to 1 indicating greater blur and closer to 0 indicating greater sharpness. This index is then compared to a pre-defined blur threshold. For each frame, a pixel-level occlusion indicator is constructed using a combination of local brightness variance, local saturation statistics, local dark channel minimums, and high-brightness / low-saturation metrics in the color space. A pixel-level occlusion confidence map is generated using linear weighting, and a frame-level occlusion score is obtained through area-weighted summation. The blur index and occlusion score are then combined using a weighted linear combination to calculate a frame quality confidence score, categorized as valid, low-quality, or invalid.

4. The method for segmenting gastrointestinal endoscopic images according to claim 1, characterized in that, The specific steps for constructing rhythmic signals based on the extraction and fusion of multi-scale temporal visual features and estimating the creeping phase are as follows: A spatial pyramid is constructed for valid, low-quality frames. At each scale, luminance histogram statistics, multi-scale local texture descriptors, and edge energy are calculated. The scale response is normalized and weighted by scale confidence. The inter-frame motion field and motion energy are calculated for adjacent frames. The motion amplitude and direction are statistically analyzed within a time window to obtain the motion energy sequence and local dynamic characteristics. Time-frequency analysis is performed on the scaled temporal window within a sliding window to identify the dominant frequency components and calculate the corresponding instantaneous phase. Noisy dominant frequencies are removed by spectral peak screening and confidence thresholding. A weighted synthesis strategy is used for the multi-scale results, where the weights are determined by scale confidence and spectral energy, to obtain frame-level instantaneous phase estimation. For invalid frames, they are recorded as verification items and include a quality label in the output structured data.

5. The method for segmenting gastrointestinal endoscopic images according to claim 1, characterized in that, The specific steps for selecting representative phase keyframes are as follows: Based on the obtained phase time series, the video is divided into physiological rhythm cycles, and within each cycle, candidate frames with similar phases and the highest frame quality scores are selected as representative phase keyframes, using a predetermined reference phase as the benchmark.

6. The method for segmenting gastrointestinal endoscopic images according to claim 1, characterized in that, The specific steps for performing pixel-level lesion segmentation on keyframes and calculating pixel-level uncertainty scores are as follows: Phase keyframes are input into the pixel-level segmentation module to obtain the predicted probability distribution for each pixel. Random perturbations are used during inference to repeatedly perform inference, resulting in pixel-level prediction distribution statistics. Furthermore, the average predicted probability and uncertainty description for each pixel are derived. A joint judgment is made based on the average probability and uncertainty. When the average probability of a pixel is higher than a preset probability threshold and the uncertainty is lower than a preset uncertainty threshold, it is included in the high-confidence mask; otherwise, it is marked as low-confidence. Morphological refinement and boundary consistency correction are applied to the obtained high-confidence mask. Simultaneously, the average confidence and uncertainty of a region are calculated at the region level. The overall reliability of each candidate region is quantified. Candidate regions are screened based on the region confidence and uncertainty thresholds. When the average confidence of a region is higher than a preset probability threshold and the uncertainty is lower than a preset uncertainty threshold, it is confirmed as a high-confidence lesion region; otherwise, it is marked as a low-confidence candidate and included in the subsequent phase mapping compensation process.

7. The method for segmenting gastrointestinal endoscopic images according to claim 6, characterized in that, The specific steps for performing pixel-level lesion segmentation on keyframes and calculating pixel-level uncertainty scores also include: Normalization and size adjustment are performed according to the preset resolution and channel order, and the images are fed into a pre-trained pixel-level segmentation model. The segmentation model extracts features layer by layer from the entire image and outputs the probability value of each pixel position belonging to the lesion category and not belonging to the lesion category, forming a probability map with the same size as the input image. The probability value at the same coordinate position is used to indicate the confidence level of the pixel being a lesion pixel.

8. The method for segmenting gastrointestinal endoscopic images according to claim 1, characterized in that, The specific steps for timing consistency recovery based on phase alignment are as follows: For ordinary frames with similar phases and adjacent times, the segmentation mask of the keyframe is forward-projected using pre-estimated inter-frame motion information. After spatial position correction, the keyframe mask is mapped onto the target frame, resulting in an initial segmentation result aligned with the target frame. The mapped segmentation result is then adaptively corrected by combining the target frame's own frame quality label, occlusion score, and phase position confidence. When the target frame quality is valid, its own segmentation is allowed to be retained. When the target frame is occluded or blurred and cannot be interpreted, the forward and backward mapping results from multiple adjacent keyframes are used, and weighted and fused according to phase distance, motion consistency, and keyframe confidence. The formula is as follows: ， Where t represents the index of the target frame, Represents the pixel coordinates on the target frame. Represents the pixels on target frame t The final fused mask value, where K represents the index set of the keyframe set, and k represents the keyframe index. This represents the weight of keyframe k.

9. The method for segmenting gastrointestinal endoscopic images according to claim 1, characterized in that, The specific steps for performing confidence-weighted and adaptive smoothing on the mapping results are as follows: For the segmentation results across the entire video, adaptive smoothing based on confidence is performed in the temporal dimension. The segmentation confidence from multiple adjacent frames at the same spatial location is weighted and averaged, and inconsistent isolated responses are suppressed to obtain a final temporal segmentation sequence that is continuous in phase.

10. The method for segmenting gastrointestinal endoscopic images according to claim 1, characterized in that, The specific steps for temporal aggregation of cross-frame segmentation results and extraction of segment-level statistical representations and confidence scores are as follows: Lesion regions that are near the same spatial location, of the same category, and appear consecutively in adjacent frames are associated in chronological order. Temporally connected regions are merged into suspicious segments, and the start and end frame numbers, start and end timestamps, and phase range covered by each suspicious segment are recorded. Within each suspicious segment, lesion regions across frames are statistically summarized to obtain a segment-level statistical representation of the segment. A comprehensive confidence score and review mark are calculated for the segment based on pixel-level and region-level confidence. Furthermore, the suspicious segments are sorted and numbered according to chronological order and severity to generate a segment-level suspicious region report.

Images