A key frame extraction and report generation method for a light-reflecting endoscopic video

Abstract

The application provides a key frame extraction and report generation method for a light reflection-oriented endoscopic video, comprising: evaluating the aggregation degree of the visible arc length at the lesion edge, determining the attention priority order of the lesion edge area according to the aggregation degree; dynamically adjusting the key frame screening condition by combining the attention priority order with the contour concave-convex degree, preferentially retaining the image frames with obvious local morphology of the lesion, and obtaining a candidate key frame set; positioning the abnormal area in each frame according to the candidate key frame set, matching the abnormal area with a reference template of an anatomical position description, and evaluating the specific position of the abnormal area in the anatomical structure; combining the anatomical position description and the abnormal area positioning information according to the final output key frame combination, generating a structured endoscopic video analysis report, and completing the lesion identification and description task under the interference of light reflection.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for extracting keyframes and generating reports from reflective endoscopic videos. Background Technology

[0002] In the field of medical image processing, endoscopic video analysis is a crucial technology, playing an indispensable role, especially in disease diagnosis and surgical assistance. By extracting keyframes and generating reports from endoscopic videos, physicians can quickly locate abnormal areas and accurately describe the position of anatomical structures. However, current methods often struggle to balance the relationship between reflective interference and the visibility of anatomical structures when processing endoscopic videos. Many solutions tend to simply filter out reflective areas or focus only on the intensity and distribution of reflections, neglecting the impact of reflection morphology on the identification of anatomical landmarks. This leads to an inability to accurately determine which image frames provide valuable lesion information, thus affecting the reliability of diagnostic reports.

[0003] The shape and boundary features of the reflected light directly determine the degree to which it obscures anatomical structures and the visibility of lesion edges. If the reflected light contour is relatively uniform, such as a regular circle, it may completely cover certain key circular anatomical landmarks, such as the ureteral orifice, making it impossible for doctors to identify its complete boundary. Conversely, if the reflected light contour is irregular, such as an uneven polygon, although the obscured area may be larger, its irregular boundaries may sometimes expose parts of the curved edge of the lesion, forming fragmented visible areas. This morphological difference makes it difficult for the system to find a unified judgment standard when screening keyframes, thus resulting in the inability to accurately capture image frames useful for diagnosis.

[0004] In practical business scenarios, when analyzing endoscopic videos, doctors often encounter situations where, in one video, the reflective area, although clearly defined, completely obscures the crucial edges of the lesion, making it impossible to determine the lesion's complete morphology; while in another video, although the reflective pattern is complex and the interference range is wide, it unexpectedly reveals part of the lesion's boundary, providing clues for diagnosis. Therefore, accurately identifying image frames that preserve local lesion features under varying reflective patterns and generating reliable diagnostic reports based on these frames has become a key issue in endoscopic video analysis. Summary of the Invention

[0005] This invention provides a method for keyframe extraction and report generation of endoscopic videos for reflective surfaces, the method comprising: The original image data is obtained by the endoscopic image processing system. The reflective contour is initially segmented, the contour line of the reflective area is extracted, and the curvature radius of each arc of the reflective boundary is identified to obtain the geometric morphological description of the reflective contour. Based on the geometric description of the reflective contour, assess the degree of contour convexity, identify the location and area of ​​the anatomical structure obscured, extract the visible part of the edge of the obscured lesion, and determine the length of the visible arc segment. The degree of aggregation of the lesion edge is assessed based on the length of the visible arc segment, and the priority ranking of the lesion edge region is determined based on the degree of aggregation. By combining the attention priority sorting with the contour concavity and convexity, the key frame screening conditions are dynamically adjusted to prioritize the retention of image frames with obvious local lesion morphology, thus obtaining a candidate key frame set. Based on the candidate keyframe set, the abnormal region in each frame is located, and the abnormal region is matched with the reference template describing the anatomical location to evaluate the specific location of the abnormal region in the anatomical structure. By analyzing the specific location of the abnormal region within the anatomical structure, the visible portion of the lesion edge is fused with the attention weight value to generate a local morphology preservation result for each keyframe, thus determining the final output keyframe combination. Based on the combination of keyframes in the final output, the anatomical location description and abnormal area localization information are integrated to generate a structured endoscopic video analysis report.

[0006] Furthermore, the process of acquiring raw image data through an endoscopic image processing system, performing preliminary segmentation of the reflective contour morphology, extracting the contour lines of the reflective region, identifying the radius of curvature of each arc segment of the reflective boundary, and obtaining a geometric description of the reflective contour includes: The original image data is obtained through an endoscope image processing system. Color space conversion is performed on the reflective area from RGB space to HSV space. The initial reflective area is determined based on the pixels whose brightness channel V value exceeds the first preset threshold. Morphological closing operation is used to fill the holes inside the reflective area to obtain a connected reflective area mask. Edge extraction is performed based on the connected reflective area mask. The Canny edge detection algorithm is used to obtain the reflective contour pixel chain. For each target pixel on the contour pixel chain, a predetermined number of neighboring pixels before and after the target pixel are selected to form a local curve segment. The arc equation corresponding to the local curve segment is fitted by the least squares method. The curvature radius value at the target pixel is determined based on the fitted curvature radius value. By statistically analyzing the number of pixels with a radius of curvature exceeding a second preset threshold on the contour pixel chain and the total number of pixels on the contour, the ratio of the two is calculated as the contour regularity coefficient. The contour regularity coefficient is then compared with preset third and fourth thresholds to determine the geometric shape description of the reflective contour.

[0007] Furthermore, based on the candidate keyframe set, abnormal regions in each frame are located, and these abnormal regions are matched with reference templates describing anatomical locations to evaluate the specific location of the abnormal regions within the anatomical structure, including: Based on the candidate keyframe set, connected component analysis is used to identify continuous regions with abnormal pixel values ​​in each frame. The centroid coordinates and minimum bounding rectangle of the abnormal region are calculated. The area and perimeter of the abnormal region are extracted, and the ratio of the area to the square of the perimeter is calculated as the compactness to obtain the geometric feature set of the abnormal region. The geometric feature set is compared with the pre-stored anatomical location reference template, which records the standard position coordinates and feature value range of each anatomical landmark in the organ. The distance between the centroid coordinates of the abnormal region and the coordinates of each anatomical landmark in the template is calculated. The three closest landmarks are selected to construct a local coordinate system. An affine transformation is used to map the abnormal region to this local coordinate system. Based on the mapped coordinate values, a predefined anatomical region division table is queried to determine the specific location of the abnormal region within the anatomical structure.

[0008] Furthermore, based on the geometric description of the reflective contour, the degree of contour convexity is assessed, the location and area of ​​the anatomical structure obscured are identified, the visible portion of the obscured lesion edge is extracted, and the length of the visible arc segment is determined, including: Based on the geometric description of the reflective contour, the tangent direction of each pixel is calculated along the contour line. The curvature change rate of the contour is determined by the angle change of adjacent tangent directions. The proportion of high curvature region to the total length of the contour is calculated to obtain the quantitative value of the contour's concavity and convexity. Using the quantified value of the contour concavity and convexity as a weighting coefficient, the reflective area is superimposed and analyzed with the standard position of the anatomical structure in the endoscopic field of view to calculate the area of ​​the anatomical area covered by the reflective area, and the position coordinates and the area of ​​the obscured anatomical structure are recorded. Based on the location coordinates of the occluded area, search for the intersection of the lesion edge contour and the reflective boundary, obtain the unoccluded edge pixel sequence between the two intersections, calculate the arc length formed by the pixel sequence, and determine the length of the visible arc segment.

[0009] Furthermore, the assessment of the aggregation degree of visible arc segment lengths at the lesion edge, and the determination of the priority ranking of the lesion edge region based on the aggregation degree, includes: Based on the visible arc length data, calculate the geometric center coordinates of each visible arc, measure the Euclidean distance between the center points of adjacent arcs, group these arcs into the same cluster region, and count the number of arcs and the total length of arcs contained in each cluster region to obtain the arc clustering degree value of each cluster region. The spatial distribution density of each cluster region is calculated using the arc segment aggregation degree value. The cluster regions are then preliminarily sorted from high to low according to the spatial distribution density value to obtain the density distribution sequence of the lesion edge. Based on the sorting position of each clustered region in the density distribution sequence, the priority order of attention for the lesion edge region is determined.

[0010] Furthermore, by combining the attention priority ranking with the contour convexity, the keyframe selection criteria are dynamically adjusted, prioritizing the retention of image frames with obvious local lesion morphology, thus obtaining a candidate keyframe set, including: Priority values ​​for each lesion edge region are obtained by prioritizing the focus. Combined with the contour convexity quantification value, the priority values ​​are normalized and multiplied with the convexity quantification value to obtain the morphological saliency score of each image frame. The mean of the morphological saliency scores of all image frames is calculated as the dynamic screening threshold. The dynamic screening threshold is used to evaluate the continuous image frame sequence. The Sobel operator is used to calculate the average gradient magnitude of the lesion edge pixels in each frame as the edge sharpness index. The edge sharpness index is normalized and then marked as the frame to be retained. Based on the labeling results of the frames to be retained, a set of candidate keyframes is determined.

[0011] Furthermore, by analyzing the specific location of the abnormal region within the anatomical structure, the visible portion of the lesion edge is fused with the attention weight value to generate a local morphology preservation result for each keyframe, thus determining the final output keyframe combination, including: By querying a predefined location importance mapping table based on the specific location coordinates of the abnormal region in the anatomical structure, the importance score corresponding to the location is obtained. The pixel gray value sequence of the visible part of the lesion edge is multiplied point by point with the attention weight value of the corresponding location to obtain the weighted edge feature vector. Based on the weighted edge feature vector, the ratio of the number of non-zero elements in the vector to the total length of the vector is calculated as the morphological integrity score. The weighted edge feature vector is then concatenated with the anatomical location coordinates of the abnormal region to form a local morphological preservation result that includes both location and morphological information. Using the morphological integrity score from the local morphological preservation results, all candidate keyframes are sorted in descending order to determine the final output keyframe combination.

[0012] Furthermore, by fusing the visible portion of the lesion edge with the attention weight value based on the specific location of the abnormal region within the anatomical structure, a local morphology preservation result is generated for each keyframe, and the final output keyframe combination is determined, which also includes: The geometric center is calculated from the coordinates of the abnormal area and used as the anatomical location point. The contour line is traced pixel by pixel from the visible part of the lesion edge to obtain the sequence of contour exposure segments. Based on the sequence of outline-revealed segments, the Euclidean distance from the centroid of each segment to the anatomical positioning point is calculated, and the reciprocal of the distance is used as the position weight value. At the same time, the proportion of the number of segments in pixels to the total number of visible outline pixels is calculated as the length weight. The position weight value and the length weight are multiplied to obtain the segment importance score. By filtering based on the importance scores of the paragraphs, the key edge paragraphs are connected in order of their position on the original outline to determine the local shape to be preserved.

[0013] Furthermore, based on the final output keyframe combination, the anatomical location description and abnormal area localization information are integrated to generate a structured endoscopic video analysis report, including: Based on the final output keyframe combination, the center coordinates and boundary coordinates of the abnormal area in each frame are extracted, and the coordinate values ​​are converted into positional description text relative to the anatomical landmarks to form a sequence of positional information. Using the aforementioned location information sequence, the data is filled according to a report template that includes fields for lesion location, size, and morphology. The data on lesion edge morphology features and reflective occlusion obtained from the aforementioned processing are integrated to output a structured endoscopic video analysis report.

[0014] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention discloses a method for keyframe extraction and report generation in endoscopic videos oriented towards reflective lesions. It aims to address the challenges of lesion obscuring, difficult edge identification, and loss of crucial information caused by reflective lesions during endoscopic examinations. The invention segments the original image based on reflective contours, extracts the radius of curvature of reflective boundaries, assesses the degree of contour convexity, identifies the visible portion of obscured lesion edges, and dynamically adjusts keyframe selection criteria based on the length and clustering of visible arc segments, prioritizing the retention of image frames with clearly defined lesion morphology. Simultaneously, the invention combines anatomical location and attention weights to analyze edge contours and texture details, generating local morphology preservation results. Finally, it integrates abnormal region localization and structured description to output an endoscopic video analysis report. This invention effectively overcomes reflective interference, improves the accuracy of lesion edge identification and keyframe selection efficiency, and achieves precise description of lesion distribution and morphological characteristics, providing reliable support for clinical diagnosis. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method for keyframe extraction and report generation of reflective endoscopic video according to the present invention.

[0016] Figure 2This is a schematic diagram of a method for keyframe extraction and report generation of reflective endoscope video according to the present invention.

[0017] Figure 3 This is another schematic diagram of a method for keyframe extraction and report generation of reflective endoscope video according to the present invention. Detailed Implementation

[0018] To further understand the content of this invention, a detailed description of the invention is provided in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0019] like Figures 1-3 This embodiment of a method for keyframe extraction and report generation of reflective endoscopic videos may specifically include: S101. Obtain raw image data through the endoscopic image processing system, perform preliminary segmentation of the reflective contour morphology, extract the contour line of the reflective area, identify the curvature radius of each arc segment of the reflective boundary, and obtain a geometric morphological description of the reflective contour.

[0020] Raw image data is acquired using an endoscopic image processing system. Color space conversion is performed on the reflective areas from RGB to HSV. Initial reflective areas are determined based on pixels whose luminance channel V value exceeds a first preset threshold. Morphological closing operations are used to fill the holes within the reflective areas, resulting in a connected reflective area mask. Edge extraction is performed based on this connected reflective area mask. The Canny edge detection algorithm is used to obtain the reflective contour pixel chain. For each target pixel in the contour pixel chain, a predetermined number of neighboring pixels before and after the target pixel are selected to form a local curve segment. The least squares method is used to fit the arc equation corresponding to the local curve segment, and the radius of curvature at the target pixel is determined based on the fitted radius of curvature value. By statistically analyzing the number of pixels with a radius of curvature exceeding a second preset threshold on the contour pixel chain and the total number of pixels on the contour, the ratio of the two is calculated as the contour regularity coefficient. The contour regularity coefficient is then compared with preset third and fourth thresholds. When the contour regularity coefficient is greater than the third threshold, the reflective shape is determined to be a regular circle. When the contour regularity coefficient is between the fourth and third thresholds, the reflective shape is determined to be an ellipse. When the contour regularity coefficient is less than the fourth threshold, the reflective shape is determined to be an irregular polygon, thus determining the geometric shape description of the reflective contour.

[0021] In one implementation, the endoscopic image processing system first performs color space conversion on the acquired raw image.

[0022] Specifically, the main purpose of converting the RGB color space to the HSV color space is that the V channel in the HSV space directly reflects the brightness information of pixels, effectively separating reflective areas from normal tissue areas. When an endoscopic light source illuminates the moist inner wall of an organ, the bright reflective areas generated by specular reflection appear as a continuous cluster of high brightness values ​​in the V channel. By setting a first preset threshold, the system can quickly identify these high-brightness pixels, forming initial reflective area markers.

[0023] It is important to note that morphological closing operations play a crucial role in processing reflective area masks. Due to the scattering and refraction of light during endoscopic imaging, a small number of low-brightness holes may exist within the reflective area. These holes can affect the integrity of subsequent contour extraction. The closing operation, through a process of dilation followed by erosion, can fill these internal holes while maintaining the outer boundary of the reflective area without significant changes, thus obtaining a connected and complete reflective area mask.

[0024] In one possible implementation, the system uses a local curve fitting method to calculate the radius of curvature of the reflective profile.

[0025] For example, when processing reflections near the ureteral orifice during cystoscopy, the system analyzes the contour pixel chain point by point. For each target pixel, a predetermined number of neighboring pixels are selected, forming a local curve segment on a two-dimensional plane. Using the least squares method, the system fits these discrete pixels to the circular arc equation (xa)² + (yb)² = r², where (a, b) are the coordinates of the center and r is the radius of curvature. During the fitting process, the system solves for the optimal center coordinates (a, b) and radius of curvature r by minimizing the sum of the squared distances from all neighboring pixels to the fitted arc. The magnitude of the radius of curvature r directly reflects the degree of curvature of the contour at that point; a larger r value indicates a straighter contour, while a smaller r value indicates a more severe curvature. This point-by-point analysis method can accurately capture the local geometric features of the reflective contour, providing a quantitative basis for subsequent morphological classification.

[0026] Preferably, the contour regularity coefficient is calculated based on the statistical distribution of the radius of curvature. When the reflection is a regular circle, the radius of curvature values ​​of most pixels on the contour are similar and relatively large. Therefore, the number of pixels exceeding the second preset threshold is large, and the contour regularity coefficient is close to 1. Conversely, the contour of an irregular polygonal reflection has multiple turning points and concave-convex changes. The radius of curvature values ​​at these locations are small, resulting in a significant decrease in the contour regularity coefficient.

[0027] For example, in actual endoscopic examinations, when the reflective morphology is determined to be a regular circle, the system pays special attention to whether the reflective morphology completely obscures the circular anatomical landmark. When the reflective morphology is an irregular polygon, although its coverage area may be larger, the irregular boundaries often expose partial contour information of the lesion edge. These fragmented visible areas are of great value for lesion identification.

[0028] S102. Based on the geometric description of the reflective contour, assess the degree of contour convexity, identify the location and area of ​​the anatomical structure obscured, extract the visible part of the edge of the obscured lesion, and determine the length of the visible arc segment.

[0029] Based on the geometric description of the reflective contour, the tangent direction of each pixel is calculated along the contour line. The rate of change of contour curvature is determined by the angular change of adjacent tangent directions. When the rate of change of angle exceeds a first preset threshold, it is marked as a high curvature region. The proportion of the high curvature region to the total length of the contour is calculated to obtain a quantitative value of the contour's concavity and convexity. Using the quantitative value of the contour's concavity and convexity as a weighting coefficient, the reflective region is superimposed and analyzed with the standard position of the anatomical structure in the endoscopic field of view to calculate the area of ​​the anatomical region covered by the reflective material. When the coverage area exceeds a second preset threshold, it is determined to be an occluded region. The position coordinates and occluded area of ​​the occluded anatomical structure are recorded. Based on the position coordinates of the occluded region, the intersection point of the lesion edge contour and the reflective boundary is searched. Starting from the intersection point, the lesion edge is traced bidirectionally. When the next intersection point with the reflective boundary is encountered, the tracing stops, and the unoccluded edge pixel sequence between the two intersection points is obtained. The arc length formed by this pixel sequence is calculated. If the arc length is greater than a third preset threshold, it is marked as a valid visible arc segment. The length values ​​of all valid visible arc segments are counted to determine the length of the visible arc segment.

[0030] In one implementation, the quantification of the contour's convexity is based on the continuous variation characteristic of the tangential direction.

[0031] Specifically, the system samples along the reflective contour at fixed pixel intervals and calculates the local tangent direction for each sampling point. The tangent direction is calculated using the difference method, passing through the position coordinates of several pixels before and after the point. The tangent directions of two adjacent sampling points form an angle, which reflects the curvature of the contour at that point. When the tangent angle of multiple consecutive sampling points exceeds a first preset threshold, it indicates that there is a significant depression or convexity in the contour in that area.

[0032] It should be noted that the quantified value of the contour's unevenness plays a crucial role in the subsequent overlay analysis. During endoscopic examination, the occlusion effect of reflective surfaces with different degrees of unevenness varies significantly on anatomical structures. Highly uneven reflective surfaces often exhibit jagged or wavy boundaries. While these irregular boundaries increase the complexity of the occluded area, they may also expose some lesion edge information at the depressions. By using the quantified unevenness value as a weighting coefficient and weighting it over the standard anatomical location, the system can more accurately assess the actual occlusion effect.

[0033] For example, during cystoscopy, when a reflective area covers a suspected tumor lesion, the system determines the precise extent of the obscured area through overlay analysis.

[0034] Preferably, the system first establishes a two-dimensional coordinate system for the endoscopic field of view, mapping both reflective areas and anatomical structures into this coordinate system. The severity of occlusion is determined by calculating the area of ​​the overlapping region and combining it with a weighted average of the degree of concavity and convexity. When the weighted occlusion area exceeds a second preset threshold, the area is marked as an effective occlusion area, and its location coordinates are recorded for subsequent edge tracking.

[0035] In one possible implementation, the visible arc segment tracking of the lesion edge employs a bidirectional search strategy. Starting from the intersection of the reflective boundary and the lesion edge, the system simultaneously tracks edge pixels in both clockwise and counterclockwise directions. During tracking, the system continuously checks whether the current pixel is still within an area not obscured by reflective light. When the tracking path encounters the reflective boundary again, it indicates that a complete visible arc segment has been found. The actual length of the arc segment is obtained by accumulating the Euclidean distances between all pixels on that arc segment.

[0036] For example, when the lesion edge is irregular in shape and divided by multiple reflective areas, the system will identify multiple independent visible arc segments. Only arc segments with a length exceeding a third preset threshold are considered to have diagnostic value. This screening mechanism ensures that subsequent analysis focuses on edge segments containing sufficient morphological information.

[0037] S103. Assess the degree of aggregation of visible arc lengths at the lesion edge, and determine the priority of attention for the lesion edge area based on the degree of aggregation.

[0038] Based on the visible arc length data, the geometric center coordinates of each visible arc are calculated, and the Euclidean distance between the center points of adjacent arcs is measured. When the distance is less than a preset distance threshold, these arcs are grouped into the same cluster region. The number of arcs contained in each cluster region and the sum of the arc lengths are counted to obtain the arc clustering degree value of each cluster region. Using the arc clustering degree value, the spatial distribution density of each cluster region is calculated. The spatial distribution density is defined as the sum of the lengths of all arcs in the cluster region divided by the area of ​​the smallest bounding rectangle containing all arcs. The cluster regions are initially sorted from high to low according to the spatial distribution density value to obtain the density distribution sequence of the lesion edge. Using the sorting position of each cluster region in the density distribution sequence, the continuity characteristics of the arcs in the region are evaluated. The continuity coefficient is obtained by calculating the ratio of the sum of the interval distances between adjacent arcs to the sum of the arc lengths in the region. When the continuity coefficient is less than a preset threshold, it indicates that the arc continuity is high, and the region is given priority in the continuity weight. The spatial distribution density value is normalized and mapped to the zero-to-one interval, and the continuity coefficient is also normalized and mapped to the zero-to-one interval. The two normalized values ​​are then weighted and summed, with the weight ratio determined according to preset parameters, to obtain a comprehensive score. Based on the comprehensive score, the priority of attention for the lesion edge area is determined from high to low.

[0039] In one implementation, the clustering degree of visible arc segments is assessed based on the principle of spatial proximity.

[0040] Specifically, the system first calculates the geometric center coordinates of each visible arc segment, which represents the spatial location of the arc segment on the edge of the lesion. By measuring the Euclidean distance between the center points of any two arc segments and comparing it with a preset distance threshold, the system can identify spatially adjacent groups of arc segments. When the distance between the center points of multiple arc segments is less than the threshold, they are grouped into the same cluster, forming a local arc segment set.

[0041] It should be noted that the arc clustering value reflects not only the number of arcs within the region, but also the total length of the arcs. In endoscopic practice, some regions may contain multiple short arc segments, while other regions may contain only a few longer continuous arc segments, thus allowing for a more accurate assessment of the amount of lesion margin information contained in each clustered region.

[0042] For example, the calculation of spatial distribution density involves the construction of a minimum bounding rectangle. The system iterates through the pixel coordinates of all arc segments within the clustered region, finds the maximum and minimum values ​​of the horizontal and vertical coordinates, and thus determines the boundary of the minimum rectangle that can completely contain these arc segments. The area of ​​the minimum bounding rectangle reflects the spatial dispersion of the arc segments; when the arc segments are closely arranged, the area of ​​the bounding rectangle is smaller, and the density value is correspondingly higher. This density measurement method can effectively distinguish between tightly clustered groups of arc segments and loosely distributed sets of arc segments, providing a quantitative basis for subsequent priority determination.

[0043] Preferably, the evaluation of the continuity coefficient takes into account the interval characteristics between adjacent arc segments. To ensure dimensional consistency, the system normalizes both the spatial distribution density value and the continuity coefficient. The density value is normalized by dividing by the maximum density value in all clustered regions, and the continuity coefficient, which is already a dimensionless ratio, is normalized by dividing by a preset maximum continuity threshold. Only the two normalized dimensionless values ​​can be used for weighted summation.

[0044] In one possible implementation, the system calculates the distance between the endpoints of each pair of adjacent arc segments within the same aggregation region, sums these distances, and divides the sum of these distances by the total length of all arc segments within the region to obtain a continuity coefficient. A smaller continuity coefficient indicates smaller intervals between arc segments and better continuity at the lesion edges; such regions typically provide more complete morphological information.

[0045] For example, when a lesion suspected of being early-stage cancer is found during a gastroscopy, the spatial distribution density normalized value of one cluster is 0.8, and the continuity coefficient normalized value is 0.8, while the density normalized value of another cluster is 0.6, and the continuity normalized value is 0.9. The system performs a weighted summation using preset weight ratios, which are set based on clinical experience. The comprehensive score for the first cluster is 0.8 × 0.6 + 0.8 × 0.4 = 0.64, and for the second cluster it is 0.6 × 0.6 + 0.9 × 0.4 = 0.72. Therefore, the second cluster receives a higher priority.

[0046] S104. By focusing on priority sorting and combining the degree of contour concavity and convexity, the key frame screening conditions are dynamically adjusted to prioritize the retention of image frames with obvious local lesion morphology, thus obtaining a set of candidate key frames.

[0047] Priority values ​​for each lesion edge region are obtained by prioritizing the focus area. These values ​​are then combined with the quantized contour convexity value. After normalization, the priority values ​​are multiplied by the quantized contour convexity value to obtain the morphological saliency score for each image frame. The mean of the morphological saliency scores for all image frames is calculated as a dynamic screening threshold. This dynamic screening threshold is used to evaluate consecutive image frame sequences. The Sobel operator is used to calculate the horizontal and vertical gradients of the lesion edge pixels in each frame. The average gradient magnitude is taken as the edge sharpness index. After normalization, if the edge sharpness index exceeds the dynamic screening threshold, the frame is marked as a frame to be retained. Based on the marking results of the frames to be retained, a structural similarity index is calculated between adjacent frames. This index is obtained by comparing the brightness, contrast, and structural information of corresponding pixel blocks in two frames. When the structural similarity index exceeds a preset similarity threshold, the morphological saliency scores of the two frames are compared. A candidate keyframe set is determined based on the morphological saliency scores, thus completing the construction of the candidate keyframe set.

[0048] In one implementation, the morphological saliency score is calculated using normalization to eliminate the influence of different dimensions.

[0049] Specifically, priority values ​​typically range from 0 to 100, while contour convexity quantification values ​​may be between 0 and 1. Direct multiplication can lead to numerical bias. By normalizing the priority value by dividing it by its maximum value to map it to the 0-1 range, and then multiplying it with the contour convexity quantification value, a uniform morphological saliency score is obtained.

[0050] It should be noted that the dynamic filtering threshold is set based on statistical principles. The system calculates the mean of the morphological saliency scores for all image frames, which reflects the average morphological saliency level of the entire video sequence. Using the mean as the threshold allows for adaptive adjustment of the filtering criteria; when the overall video quality is high, the threshold is increased accordingly, and vice versa.

[0051] For example, the application of the Sobel operator in edge sharpness assessment involves convolution operations in two directions. The horizontal Sobel operator detects vertical edges, and the vertical Sobel operator detects horizontal edges. For each pixel at the edge of a lesion, the system calculates its horizontal and vertical gradients, and then calculates the gradient magnitude based on the gradient values ​​in both directions.

[0052] Specifically, the horizontal gradient is denoted as Gx, and the vertical gradient as Gy. The gradient magnitude is calculated using the formula sqrt(Gx² + Gy²). The gradient magnitude reflects the degree of change in image grayscale at that point; a larger magnitude indicates a sharper edge. The overall edge sharpness index is obtained by summing the gradient magnitudes of all pixels within the lesion edge region and dividing by the total number of pixels. This method can quantitatively assess the sharpness of lesion edges in different image frames, providing an objective basis for subsequent frame selection.

[0053] Preferably, the calculation of the structural similarity index comprehensively considers information from three dimensions. Brightness comparison is achieved by calculating the difference in average gray values ​​of corresponding regions in two frames, contrast comparison is measured by the ratio of standard deviations, and structural information is evaluated by normalized cross-correlation coefficients.

[0054] In one possible implementation, when the structural similarity index of adjacent frames to be retained exceeds a preset similarity threshold, it indicates that the two frames are highly similar. At this point, the system compares the morphological saliency scores of the two frames and retains the frame with the higher score, thereby avoiding redundant information in the candidate keyframe set and ensuring that the image frame with the most obvious morphological features is retained.

[0055] The distribution range of lesions on the organ wall is determined based on the anatomical location of the abnormal area. The edge contour morphology and edge texture state are collected from the visible part of the lesion edge. The correspondence between the edge contour and attention weight is analyzed, the supporting role of edge texture in local morphology recognition is evaluated, and local morphological results containing lesion edge contour and texture details are generated.

[0056] Based on the anatomical coordinates of the abnormal region, three-dimensional mesh data of the organ's inner wall is obtained. The shortest path length from the center point of the abnormal region along the inner wall surface to each mesh vertex is calculated. When the path length is less than a preset length threshold, the corresponding mesh region is marked as the lesion-affected region. The surface area of ​​the affected region is calculated to determine the distribution range of the lesion on the organ's inner wall. From the visible portion of the lesion edge within the distribution range, edge pixel sequences are extracted, and the directional angle change between adjacent pixels is calculated as a contour curvature feature. A spatial distribution matrix of pixel grayscale is constructed in the edge region. The grayscale difference, repetition pattern, directional correlation, and energy distribution of adjacent pixel pairs in the matrix are calculated to obtain the edge contour morphology sequence and texture state parameter set. Through the local extreme points in the contour curvature feature, the rate of curvature change of the neighborhood around each extreme point is calculated as the significance value of that point. Attention weights are assigned according to the significance value; the higher the significance value, the greater the weight. A correspondence table between contour feature points and weight values ​​is established. Based on the correspondence table, high-weight regions are identified, and the numerical range of grayscale difference and orientation correlation parameters in the texture state parameter set within the region is evaluated. If the value exceeds the preset recognition threshold, the contour morphology sequence and the texture state parameter set are concatenated and combined to generate a local morphological result containing lesion edge contours and texture details.

[0057] In one implementation, the acquisition of three-dimensional mesh data of the organ's inner wall is based on the depth sensing function of the endoscopy system.

[0058] Specifically, during the scanning process, the endoscope acquires depth information of the organ's inner wall using structured light or stereo vision technology, reconstructing a two-dimensional image sequence into a three-dimensional point cloud. The system employs Poisson reconstruction or moving cubes algorithm to convert the point cloud into a triangular mesh model. The conversion process includes estimating surface normals and generating continuous surfaces, or extracting isosurfaces after voxelizing the point cloud. The three-dimensional mesh consists of vertices, edges, and faces, with each vertex recording its coordinate position in three-dimensional space. The shortest path from the center point of the abnormal region to a mesh vertex is calculated on the mesh surface using Dijkstra's algorithm. This path propagates along the common edge of adjacent faces, avoiding passage through the organ's internal cavities, thus accurately reflecting the actual influence range of the lesion on the inner wall surface.

[0059] It should be noted that the calculation of the shortest path length takes into account the curved surface characteristics of the organ's inner wall. During cystoscopy, the bladder inner wall presents an irregular spherical structure, and straight-line distance cannot accurately reflect the actual extent of the lesion's influence. By calculating the geodesic distance along the inner wall surface, the system can identify regions that are geographically distant but adjacent on the inner wall surface. When the path length is less than a preset length threshold, it indicates that the region is within the potential spread range of the lesion and requires close monitoring.

[0060] For example, the extraction process of edge contour curvature features involves the analysis of continuous changes in orientation angles. The system tracks the visible portion of the lesion edge pixel by pixel, and for each pixel on the edge, calculates the orientation angle of the line connecting it to the previous pixel. The difference in orientation angles between adjacent pixels reflects the degree of curvature of the contour at that point; a larger difference indicates higher curvature. By recording the changes in orientation angles throughout the entire edge sequence, a contour curvature feature vector is formed, which preserves the geometric morphological information of the lesion edge.

[0061] In one possible implementation, the construction of the pixel gray-level spatial distribution matrix is ​​based on the co-occurrence matrix principle but with adaptive improvements. Traditional gray-level co-occurrence matrices statistically analyze pixel pair relationships in fixed directions and distances, while the spatial distribution matrix in this scheme considers the irregular shapes of edge regions. The system defines a local window around the lesion edge and counts the frequency of gray-level value combinations of adjacent pixel pairs in all directions within the window. For pixel pairs with gray-level values ​​i and j, if they are spatially adjacent, the count is accumulated at the i-th row and j-th column of the matrix. Through normalization, a probability distribution matrix reflecting local texture features is obtained. The gray-level difference parameter extracted from this matrix reflects the texture contrast, the repetition pattern parameter reflects the texture regularity, the direction correlation parameter reflects the directional characteristics of the texture, and the energy distribution parameter reflects the uniformity of the texture. These four parameters together constitute the texture state parameter set, providing a quantitative basis for subsequent lesion feature analysis.

[0062] Preferably, the significance value is calculated using a multi-scale analysis method. For each curvature extremum point on the contour, the system calculates the rate of curvature change within neighborhood windows of different sizes. Small windows capture local detail changes, while large windows reflect the overall morphological trend. The weighted average of the rate of curvature change at different scales is used to obtain the comprehensive significance value of that point. Points with high significance values ​​usually correspond to key morphological features of the lesion edge, such as protrusions, depressions, or sharp turns.

[0063] For example, when early gastric cancer lesions are detected during gastroscopy, the lesion edges often exhibit an irregular, serrated shape. The system identifies the tips of these serrations as having high significance values ​​and assigns them a larger attention weight. Meanwhile, the smooth transition regions between the serrations have lower significance values ​​and are assigned a correspondingly smaller weight.

[0064] Understandably, the allocation of attention weights follows a normalization principle. The system maps the saliency values ​​of all feature points to a range of 0 to 1, with the point having the highest saliency value having a weight of 1 and the lowest weight close to 0. By establishing a correspondence table between contour feature points and weight values, the system can prioritize regions with significant morphological meaning in subsequent processing. Furthermore, the texture parameter evaluation of high-weight regions involves a threshold judgment mechanism. When a region is identified as a high-weight region, the system checks whether the grayscale difference parameter within that region exceeds a preset contrast threshold and whether the orientation correlation parameter exceeds a preset texture regularity threshold. Only when both conditions are met is the texture information of that region considered to have significant value for lesion morphology recognition.

[0065] In one embodiment, the concatenation of the contour morphology sequence and the texture state parameter set employs a feature vector concatenation method. The system concatenates the contour curvature feature vector and the texture parameter vector in a predetermined order to form a comprehensive feature vector. The first half of this vector describes the geometric morphology of the lesion edge, while the second half describes the texture characteristics of the edge region. The combination of the two provides a complete local morphological description of the lesion. This concatenation method preserves the independence of each feature while simultaneously achieving the fusion of multimodal information.

[0066] S105. Locate the abnormal region in each frame based on the candidate keyframe set, match the abnormal region with the reference template describing the anatomical location, and evaluate the specific location of the abnormal region in the anatomical structure.

[0067] Based on the candidate keyframe set, connected component analysis is used in each frame to identify continuous regions with abnormal pixel values. The centroid coordinates and minimum bounding rectangle of the abnormal region are calculated. The area and perimeter of the abnormal region are extracted, and the ratio of area to the square of perimeter is calculated as compactness, resulting in a geometric feature set of the abnormal region. This geometric feature set is compared with a pre-stored anatomical location reference template, which records the standard position coordinates and feature value ranges of each anatomical landmark within the organ. The distance between the centroid coordinates of the abnormal region and the coordinates of each anatomical landmark point in the template is calculated, and the three closest landmark points are selected to construct a local coordinate system. Affine transformation is used to map the abnormal region into this local coordinate system. Based on the mapped coordinate values, a predefined anatomical region division table is consulted to determine which division of the bladder trigone, bladder dome, lateral wall, or neck the coordinates fall into. Combining the relative positional relationship between this division and the ureteral orifice or internal urethral orifice, the specific location of the abnormal region in the anatomical structure is determined.

[0068] In one implementation, connected component analysis identifies anomalous regions through pixel adjacency relationships.

[0069] Specifically, the system scans each pixel in the image. When a pixel's grayscale value exceeds an anomaly threshold, it uses that pixel as a seed point to search for neighboring anomaly pixels in eight directions, marking all connected anomaly pixels as belonging to the same region. The centroid coordinates are obtained by calculating the average of the coordinates of all pixels within the region, and the minimum bounding rectangle is determined by finding the smallest rectangular boundary that completely contains the region.

[0070] It should be noted that compactness, as a shape description parameter, reflects the regularity of the region's shape. By calculating the ratio of area to the square of perimeter, the system can distinguish between regular circular lesions and irregularly shaped lesions. Regular circular lesions have higher compactness values, while irregularly shaped lesions have significantly lower compactness values. This parameter provides important morphological basis for subsequent template matching.

[0071] For example, the anatomical location reference template is constructed based on statistical data from a large number of endoscopic examinations. During cystoscopy, the template records the standard locations of key anatomical landmarks such as the ureteral orifice, the bladder trigone, and the internal urethral orifice. Each landmark not only contains coordinate information but also a range of typical characteristic values ​​for that location, such as color and texture. When the geometric feature set of an abnormal region is compared with the template, the system first calculates the Euclidean distance from the centroid of the abnormal region to each landmark, selecting the three closest landmarks as reference points. The triangle formed by these three points defines a local coordinate system, with its center point as the origin and the direction of its longest side as the principal axis. Through an affine transformation matrix, the abnormal region is mapped from the image coordinate system to this local coordinate system, achieving a standardized representation of its location.

[0072] Preferably, the anatomical region division table is stored in a hierarchical structure. The first layer divides the bladder into four main regions: the trigone, the top, the lateral walls, and the neck, each with a clearly defined coordinate range. The second layer further subdivides these regions; for example, the lateral walls are divided into the left and right lateral walls, and the trigone is marked with its relative position to the ureteral orifice.

[0073] In one possible implementation, once an abnormal region is located to a specific anatomical region, the system also calculates the relative positional relationship between that region and key anatomical structures.

[0074] For example, if the abnormal area is located in the trigone of the bladder, the system will measure the distance from it to the openings of the left and right ureters to determine whether it affects the urine drainage pathway. This precise anatomical localization provides important reference information for clinical diagnosis.

[0075] S106. By fusing the visible part of the lesion edge with the attention weight value through the specific location of the abnormal region in the anatomical structure, the local morphology preservation result for each keyframe is generated, and the final output keyframe combination is determined.

[0076] By querying a predefined location importance mapping table using the specific coordinates of the abnormal region within the anatomical structure, the importance score corresponding to that location is obtained. The pixel grayscale value sequence of the visible portion of the lesion edge is then multiplied point-by-point with the attention weight value of the corresponding location to obtain a weighted edge feature vector. Based on this weighted edge feature vector, the ratio of the number of non-zero elements in the vector to the total length of the vector is calculated as the morphological integrity score. If the score exceeds a preset integrity threshold, the weighted edge feature vector is concatenated with the anatomical location coordinates of the abnormal region to form a local morphological preservation result containing both location and morphological information. Using the morphological integrity score from this local morphological preservation result, all candidate keyframes are sorted in descending order. The top few frames are selected until all anatomical regions are cumulatively covered or a preset frame limit is reached, determining the final output keyframe combination.

[0077] In one implementation, the location importance mapping table is constructed based on clinical statistics and medical expert knowledge.

[0078] Specifically, the system employs a standardized three-dimensional coordinate system, dividing the organ's inner wall into grid cells with a resolution of 1 cm x 1 cm. Each cell is assigned a different importance score based on its clinical diagnostic value. The bladder trigone, being a high-incidence area for tumors, has an importance score ranging from 0.9 to 1.0; the lateral wall region has a score of 0.6 to 0.8; and the top region has a score of 0.4 to 0.6. Once the coordinates of an abnormal area are determined, the system retrieves the corresponding importance score by querying a mapping table. This score reflects the degree of impact of the lesion at that location on clinical diagnosis.

[0079] It should be noted that the weighted edge feature vector is calculated using point-by-point multiplication. Each pixel in the visible part of the lesion edge has a corresponding grayscale value and attention weight value, which are multiplied together to obtain the weighted feature value of that pixel. By traversing all edge pixels, a complete weighted edge feature vector is formed. This weighting process gives more weight to edge regions with significant morphology in the vector, while reducing the weight of blurred or unimportant edge regions.

[0080] For example, the morphological integrity score reflects the visibility of the lesion edge. The system counts the number of non-zero elements in the weighted edge feature vector, which correspond to visible edge pixels. The morphological integrity score is obtained by dividing the number of non-zero elements by the total length of the vector. A score close to 1 indicates that the lesion edge is almost completely visible, while a score close to 0 indicates that the edge is severely occluded. When the score exceeds a preset integrity threshold of 0.6, the frame is considered to contain sufficient morphological information and is worth retaining.

[0081] Preferably, the local morphology preservation result achieves multi-dimensional information fusion through feature concatenation. The weighted edge feature vector preserves morphological information, while the anatomical coordinates of the abnormal region provide spatial positioning information. The vector formed by concatenating the two contains both the morphological features of "what it is" and the location information of "where it is".

[0082] In one possible implementation, a greedy strategy is used to determine the final keyframe composition. The system sorts all candidate frames in descending order of morphological integrity score, prioritizing the frame with the highest score. During the selection process, the system continuously tracks the covered anatomical regions, and prioritizes including frames that can cover previously unincluded anatomical areas. This strategy ensures that the final keyframe composition contains both morphologically clear images and comprehensive coverage of all anatomical regions, providing doctors with complete diagnostic information.

[0083] The anatomical location of the lesion is determined from the location of the abnormal area. The exposed segments of the contour are extracted from the visible part of the lesion edge. The distribution of the exposed segments of the contour corresponding to the anatomical location is analyzed. The importance of each segment in the morphological description is evaluated. Key edge segments are identified. Edge segments are merged to determine the local morphology to be preserved.

[0084] The geometric center of the abnormal area is calculated from its location coordinates as the anatomical positioning point. The lesion is traced pixel-by-pixel along the outline from the visible edge. When the gray value of consecutive pixels is below the visibility threshold, it is determined to be an occluded area, and the current segment is marked as ending. The next pixel with a gray value exceeding the visibility threshold is designated as the starting point of a new segment, resulting in a sequence of outline-revealed segments. Based on this sequence, the Euclidean distance from the centroid of each segment to the anatomical positioning point is calculated, and the reciprocal of this distance is used as the positional weight. Simultaneously, the proportion of the number of pixels in the segment to the total number of visible outline pixels is calculated as the length weight. The positional weight and length weight are multiplied to obtain the segment importance score. Segments with scores exceeding a preset importance threshold are selected as key edge segments based on their importance scores. These key edge segments are then concatenated according to their positional order on the original outline. Linear interpolation is used to connect the breaks between segments to determine the locally preserved morphology.

[0085] In one implementation, the anatomical location points are determined using a geometric center algorithm.

[0086] Specifically, the system iterates through all pixels within the abnormal region, sums their x and y coordinates, and then divides the sum by the total number of pixels. The resulting coordinates are the geometric center. This center point represents the spatial centroid of the lesion and serves as a reference for subsequent distance calculations.

[0087] It should be noted that the identification of contour segments is based on the continuity of grayscale values. In endoscopic images, the visible edges of lesions typically exhibit higher grayscale values, while the grayscale values ​​of areas obscured by reflected light or other tissues are significantly lower. The system sets a visibility threshold of 60% of the grayscale range; when the grayscale values ​​of three or more consecutive pixels fall below this threshold, it is determined as the beginning of an obscured area. This segmentation method based on grayscale thresholds can accurately identify the discontinuous features of the contour.

[0088] For example, the calculation of position weight reflects the principle of "the closer, the more important," using the inverse of distance as the weight allocation method, so that edge segments closer to the center of the lesion receive higher attention. At the same time, length weight reflects the proportion of the segment in the overall outline, and is calculated by dividing the number of segment pixels by the total number of visible outline pixels.

[0089] Preferably, the paragraph importance score is obtained by multiplying two weights.

[0090] For example, if a paragraph has a position weight of 0.2 and a length weight of 0.25, its importance score is 0.05. The system sets an importance threshold of 0.03; paragraphs exceeding this threshold are considered critical edge paragraphs.

[0091] In one possible implementation, linear interpolation is used to connect discontinuous key edge segments. When a break exists between two adjacent key segments, the system extracts the coordinates of the endpoints of the two segments and calculates the slope and intercept of the connecting line. Along this line, new pixels are inserted at pixel intervals from the original image, with their grayscale values ​​determined by a linear transition of the grayscale values ​​at the endpoints. While simple, this interpolation method effectively restores the continuity of the contour, forming a complete, locally preserved morphology. By concatenating all key edge segments and filling in the breaks, the system ultimately reconstructs a lesion contour description that retains important morphological features while maintaining continuity.

[0092] S107. Based on the final output keyframe combination, integrate the anatomical location description and abnormal area location information to generate a structured endoscopic video analysis report, completing the task of lesion identification and description under reflective interference.

[0093] Based on the final output keyframe combination, the center coordinates and boundary coordinates of the abnormal region in each frame are extracted. The coordinate values ​​are converted into location description text relative to anatomical landmarks to form a location information sequence. Using the location information sequence, a report template containing lesion location, size, and morphology fields is filled in. The lesion edge morphology features and reflective occlusion data obtained from the previous processing are integrated to output a structured endoscopic video analysis report.

[0094] In one implementation, the conversion of location description text uses a relative positioning method.

[0095] Specifically, the system compares the center coordinates of the abnormal area with the coordinates of pre-stored anatomical landmarks to calculate the relative distance and orientation.

[0096] For example, when the lesion center is located 15 mm above the ureteral orifice and 8 mm to the left, the system converts it to the text description "the lesion is located proximal to the upper part of the left ureteral orifice".

[0097] It should be noted that the report template adopts a layered structure design. The first layer contains basic examination information, such as examination time and equipment model; the second layer is lesion localization information, recording the anatomical location and spatial coordinates; the third layer is morphological description, including lesion size, edge features, and surface texture; the fourth layer is reflection interference assessment, recording the degree of reflection obstruction and visibility score.

[0098] For example, in the cystoscopy report, the system automatically fills in the contents of each field. The lesion location field is filled in as "bladder trigone, 1.5 cm from the left ureteral orifice"; the morphological characteristics field is filled in as "irregular bulge, rough surface, with clearly visible edges covering 70%"; and the reflection interference field is filled in as "mild reflection obstruction, mainly affecting the lower right margin of the lesion". By integrating this structured information, a complete endoscopic video analysis report is finally output, providing standardized imaging evidence for clinical diagnosis.

[0099] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A method for keyframe extraction and report generation of endoscopic videos for reflective surfaces, characterized in that, The method includes: The original image data is obtained by the endoscopic image processing system. The reflective contour is initially segmented, the contour line of the reflective area is extracted, and the curvature radius of each arc of the reflective boundary is identified to obtain the geometric morphological description of the reflective contour. Based on the geometric description of the reflective contour, assess the degree of contour convexity, identify the location and area of ​​the anatomical structure obscured, extract the visible part of the edge of the obscured lesion, and determine the length of the visible arc segment. The degree of aggregation of the lesion edge is assessed based on the length of the visible arc segment, and the priority ranking of the lesion edge region is determined based on the degree of aggregation. By combining the attention priority sorting with the contour concavity and convexity, the key frame screening conditions are dynamically adjusted to prioritize the retention of image frames with obvious local lesion morphology, thus obtaining a candidate key frame set. Based on the candidate keyframe set, the abnormal region in each frame is located, and the abnormal region is matched with the reference template describing the anatomical location to evaluate the specific location of the abnormal region in the anatomical structure. By analyzing the specific location of the abnormal region within the anatomical structure, the visible portion of the lesion edge is fused with the attention weight value to generate a local morphology preservation result for each keyframe, thus determining the final output keyframe combination. Based on the combination of keyframes in the final output, the anatomical location description and abnormal area localization information are integrated to generate a structured endoscopic video analysis report.

2. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 1, characterized in that, The process involves acquiring raw image data through an endoscopic image processing system, performing preliminary segmentation of the reflective contour, extracting the contour lines of the reflective region, identifying the radius of curvature of each arc segment of the reflective boundary, and obtaining a geometric description of the reflective contour, including: The original image data is obtained through an endoscope image processing system. Color space conversion is performed on the reflective area from RGB space to HSV space. The initial reflective area is determined based on the pixels whose brightness channel V value exceeds the first preset threshold. Morphological closing operation is used to fill the holes inside the reflective area to obtain a connected reflective area mask. Edge extraction is performed based on the connected reflective area mask. The Canny edge detection algorithm is used to obtain the reflective contour pixel chain. For each target pixel on the contour pixel chain, a predetermined number of neighboring pixels before and after the target pixel are selected to form a local curve segment. The arc equation corresponding to the local curve segment is fitted by the least squares method. The curvature radius value at the target pixel is determined based on the fitted curvature radius value. By statistically analyzing the number of pixels with a radius of curvature exceeding a second preset threshold on the contour pixel chain and the total number of pixels on the contour, the ratio of the two is calculated as the contour regularity coefficient. The contour regularity coefficient is then compared with preset third and fourth thresholds to determine the geometric shape description of the reflective contour.

3. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 1, characterized in that, Based on the candidate keyframe set, the abnormal region in each frame is located, and the abnormal region is matched with a reference template describing the anatomical location to evaluate the specific location of the abnormal region in the anatomical structure, including: Based on the candidate keyframe set, connected component analysis is used to identify continuous regions with abnormal pixel values ​​in each frame. The centroid coordinates and minimum bounding rectangle of the abnormal region are calculated. The area and perimeter of the abnormal region are extracted, and the ratio of the area to the square of the perimeter is calculated as the compactness to obtain the geometric feature set of the abnormal region. The geometric feature set is compared with the pre-stored anatomical location reference template, which records the standard position coordinates and feature value range of each anatomical landmark in the organ. The distance between the centroid coordinates of the abnormal region and the coordinates of each anatomical landmark in the template is calculated. The three closest landmarks are selected to construct a local coordinate system. An affine transformation is used to map the abnormal region to this local coordinate system. Based on the mapped coordinate values, a predefined anatomical region division table is queried to determine the specific location of the abnormal region within the anatomical structure.

4. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 1, characterized in that, Based on the geometric description of the reflective contour, assess the degree of contour convexity, identify the location and area of ​​the anatomical structure obscured, extract the visible portion of the edge of the obscured lesion, and determine the length of the visible arc segment, including: Based on the geometric description of the reflective contour, the tangent direction of each pixel is calculated along the contour line. The curvature change rate of the contour is determined by the angle change of adjacent tangent directions. The proportion of high curvature region to the total length of the contour is calculated to obtain the quantitative value of the contour's concavity and convexity. Using the quantified value of the contour concavity and convexity as a weighting coefficient, the reflective area is superimposed and analyzed with the standard position of the anatomical structure in the endoscopic field of view to calculate the area of ​​the anatomical area covered by the reflective area, and the position coordinates and the area of ​​the obscured anatomical structure are recorded. Based on the location coordinates of the occluded area, search for the intersection of the lesion edge contour and the reflective boundary, obtain the unoccluded edge pixel sequence between the two intersections, calculate the arc length formed by the pixel sequence, and determine the length of the visible arc segment.

5. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 1, characterized in that, The assessment of the concentration of visible arc segments at the lesion edge, and the determination of the priority ranking of the lesion edge region based on the concentration, includes: Based on the visible arc length data, calculate the geometric center coordinates of each visible arc, measure the Euclidean distance between the center points of adjacent arcs, group these arcs into the same cluster region, and count the number of arcs and the total length of arcs contained in each cluster region to obtain the arc clustering degree value of each cluster region. The spatial distribution density of each cluster region is calculated using the arc segment aggregation degree value. The cluster regions are then preliminarily sorted from high to low according to the spatial distribution density value to obtain the density distribution sequence of the lesion edge. Based on the sorting position of each clustered region in the density distribution sequence, the priority order of attention for the lesion edge region is determined.

6. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 1, characterized in that, By combining the attention priority ranking with the contour convexity, the keyframe selection criteria are dynamically adjusted, prioritizing the retention of image frames with obvious local lesion morphology, resulting in a candidate keyframe set, including: Priority values ​​for each lesion edge region are obtained by prioritizing the focus. Combined with the contour convexity quantification value, the priority values ​​are normalized and multiplied with the convexity quantification value to obtain the morphological saliency score of each image frame. The mean of the morphological saliency scores of all image frames is calculated as the dynamic screening threshold. The dynamic screening threshold is used to evaluate the continuous image frame sequence. The Sobel operator is used to calculate the average gradient magnitude of the lesion edge pixels in each frame as the edge sharpness index. The edge sharpness index is normalized and then marked as the frame to be retained. Based on the tagging results of the frames to be retained, a set of candidate keyframes is determined.

7. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 1, characterized in that, By analyzing the specific location of the abnormal region within the anatomical structure, and fusing the visible portion of the lesion edge with attention weight values, a local morphology preservation result is generated for each keyframe. This determines the final output keyframe combination, including: By querying a predefined location importance mapping table based on the specific location coordinates of the abnormal region in the anatomical structure, the importance score corresponding to the location is obtained. The pixel gray value sequence of the visible part of the lesion edge is multiplied point by point with the attention weight value of the corresponding location to obtain the weighted edge feature vector. Based on the weighted edge feature vector, the ratio of the number of non-zero elements in the vector to the total length of the vector is calculated as the morphological integrity score. The weighted edge feature vector is then concatenated with the anatomical location coordinates of the abnormal region to form a local morphological preservation result that includes both location and morphological information. Using the morphological integrity score from the local morphological preservation results, all candidate keyframes are sorted in descending order to determine the final output keyframe combination.

8. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 7, characterized in that, By analyzing the specific location of the abnormal region within the anatomical structure, fusing the visible portion of the lesion edge with attention weight values, a local morphology preservation result is generated for each keyframe, determining the final output keyframe combination, and further including: The geometric center is calculated from the coordinates of the abnormal area and used as the anatomical location point. The contour line is traced pixel by pixel from the visible part of the lesion edge to obtain the sequence of contour exposure segments. Based on the sequence of outline-revealed segments, the Euclidean distance from the centroid of each segment to the anatomical positioning point is calculated, and the reciprocal of the distance is used as the position weight value. At the same time, the proportion of the number of segments in pixels to the total number of visible outline pixels is calculated as the length weight. The position weight value and the length weight are multiplied to obtain the segment importance score. By filtering based on the importance scores of the paragraphs, the key edge paragraphs are connected in order of their position on the original outline to determine the local shape to be preserved.

9. The method for keyframe extraction and report generation of reflective endoscopic video according to claim 1, characterized in that, The process involves integrating anatomical location descriptions and abnormal region localization information based on the final output keyframe combination to generate a structured endoscopic video analysis report, including: Based on the final output keyframe combination, the center coordinates and boundary coordinates of the abnormal area in each frame are extracted, and the coordinate values ​​are converted into positional description text relative to the anatomical landmarks to form a sequence of positional information. Using the aforementioned location information sequence, the data is filled according to a report template that includes fields for lesion location, size, and morphology. The data on lesion edge morphology features and reflective occlusion obtained from the aforementioned processing are integrated to output a structured endoscopic video analysis report.

Images