Artificial intelligence-based lumen focus feature identification method and system

Abstract

The invention provides an artificial intelligence-based lumen lesion feature recognition method and system, and the method comprises the steps: obtaining an original image frame sequence collected by an electronic endoscope and a collection timestamp of each frame, and forming a pseudo-time image sequence set; inputting the pseudo time image sequence set into a pre-constructed candidate region extraction model, and outputting a focus candidate region set; performing vascular structure enhancement on the lesion candidate region set, constructing a graph neural network, extracting structural features through the graph neural network, outputting a lesion confidence score in combination with the region features, and finally generating an enhanced structured candidate region set in combination with the lesion candidate region set, the structural features and the lesion confidence score; performing trajectory aggregation on the structured candidate region set based on structural feature similarity and time continuity to form a cross-frame focus trajectory set; and calculating a space center position and a time range of a focus track according to the cross-frame focus track set, and generating a structured report.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and in particular relates to an artificial intelligence-based method and system for recognizing the characteristics of intracavitary lesions. Background Art

[0002] With the development of electronic endoscopy technology, the acquisition of images of internal body structures such as the gastrointestinal tract and bronchi has become increasingly efficient and clear, and endoscopic examination has become one of the main means of clinical diagnosis of digestive tract tumors, inflammation and other lesions. In recent years, with the rapid development of artificial intelligence image recognition technology, the application of deep learning models to the automated analysis of endoscopic images has become an important direction for auxiliary diagnosis of endoscopic lesions. In existing research and products, common methods usually use convolutional neural networks (CNN) or improved feature extraction networks to classify or segment endoscopic images to achieve automatic identification of lesions such as polyps, ulcers, and early tumors. However, there are still many key difficulties in existing technologies that have not been effectively overcome, which limits the practical application of artificial intelligence in actual endoscopic scenarios.

[0003] First, most existing recognition methods model static images, ignoring the sequential nature of image data during endoscopic examinations. Because endoscopic examinations are essentially a dynamic process in which doctors operate equipment to advance frame by frame, lesions may appear at different angles and exposure conditions in multiple image frames. Single-frame recognition often results in missed or false detections due to poor viewing angles, blurred images, or partial occlusions. This is especially true for early-stage lesions, flat lesions, or lesions with blurred boundaries. Second, most current models rely primarily on visual representations such as color, shape, and texture for classification, lacking the ability to model medically interpretable internal structural features (such as vascular shape and mucosal arrangement). These structural features are often an important basis for clinicians to judge the severity of lesions. For example, early-stage tumor lesions often manifest as local vascular disorder, convergence, disappearance, or abnormal shape, which are difficult to accurately capture using traditional image recognition features alone. In addition, due to the lack of spatial position information in endoscopic images, existing methods often have difficulty in outputting positioning data that can be used for surgical navigation or subsequent follow-up. Even if the lesion is successfully identified, its specific location in the cavity structure cannot be accurately described, which greatly reduces the clinical reference value of the identification result. In the presence of multiple lesions or complex anatomical structures, spatiotemporal tracking and visual annotation of lesions cannot be achieved, which also limits the value of AI-assisted systems in the actual diagnosis and treatment process. Therefore, current AI recognition methods are still at the "image-level judgment" stage and lack the ability to comprehensively model temporal information, structural features, and spatial positioning. Summary of the Invention

[0004] The purpose of this invention is to propose an artificial intelligence-based method and system for identifying intracavitary lesion features, which forms a systematic solution path in dynamic scene understanding, medical feature modeling, and result space annotation, and significantly improves the clinical applicability and implementation capabilities of artificial intelligence in intracavitary lesion identification scenarios.

[0005] In order to achieve the above-mentioned object, a first aspect of the present invention provides an artificial intelligence-based method for recognizing features of endoluminal lesions, the method comprising:

[0006] S1. Obtaining an original image frame sequence acquired by an electronic endoscope and an acquisition timestamp of each frame, generating a time position coding vector by normalizing the timestamp and embedding a position code, and forming a pseudo-time image sequence set based on the original image frame sequence, the normalized relative time position, and the time position coding vector;

[0007] S2, inputting the pseudo-time image sequence set into a pre-built candidate region extraction model, and outputting a set of lesion candidate regions, including: region position, normalized relative time position, time position encoding vector, and region features;

[0008] S3, performing vascular structure enhancement on the set of lesion candidate regions, constructing a graph neural network and extracting structural features through the graph neural network, outputting a lesion confidence score in combination with the regional features, and finally generating an enhanced structured candidate region set by combining the lesion candidate region set, the structural features, and the lesion confidence score;

[0009] S4. performing trajectory aggregation on the set of structured candidate regions based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set;

[0010] S5. Calculating the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set, and generating a structured report;

[0011] Among them, the candidate region extraction model includes an image feature extraction module, a spatiotemporal attention fusion module and a region proposal module; the image feature extraction module is used to extract the image feature tensor of each frame image; the spatiotemporal attention fusion module is used to construct a joint attention mechanism across frames; the region proposal module is used to extract a set of lesion candidate regions.

[0012] Furthermore, the normalizing of the timestamp and embedding the position code to generate the time position code vector specifically includes:

[0013] Generate a time-dependent temporal position encoding vector through a sine function.

[0014] Furthermore, the step of inputting the pseudo-time image sequence set into a pre-built candidate region extraction model and outputting a set of lesion candidate regions specifically includes:

[0015] A convolutional structure with a channel attention mechanism is used to extract the basic visual features of each frame image and output an image feature tensor; the image feature tensor is input into the spatiotemporal attention fusion module to construct a cross-frame joint attention mechanism, and the image features of adjacent frames and their position codes are fed into the multi-head attention structure to generate a cross-frame spatiotemporal response tensor;

[0016] The spatiotemporal response tensor is input into the region proposal module, and the region proposal module inputs the fused spatiotemporal response tensor of each frame into a lightweight convolution-upsampling structure, predicts the candidate lesion region heat map on each frame image, and then extracts the candidate region set by setting a confidence threshold to obtain the lesion candidate region set.

[0017] Furthermore, the candidate region is represented as a quad in is the position bounding box of the i-th candidate region in the t-th frame, is the activation vector of the region on the feature map, τ t is the normalized relative time position, e t is the time position encoding vector;

[0018] The convolution-upsampling structure is a 3×3 convolution, ReLU and upsampling structure, which outputs a two-dimensional probability map.

[0019] Furthermore, the step of performing vascular structure enhancement on the set of candidate lesion regions specifically includes:

[0020] An improved Retinex spectral reconstruction algorithm was used to convert the image of the lesion candidate region set from RGB space to Lab color space, and adaptive contrast enhancement was performed on the L channel. Combined with detail enhancement of the green channel, a set of lesion candidate regions with enhanced vascular structure was obtained.

[0021] Furthermore, the graph neural network is constructed and structural features are extracted through the graph neural network, lesion confidence scores are output in combination with regional features, and finally an enhanced structured candidate region set is generated by combining the lesion candidate region set, structural features and lesion confidence scores, specifically including:

[0022] Obtain a set of candidate lesion regions with enhanced vascular structure, extract vascular centerline pixels using the Steger line detection algorithm, and use them as a node set of the graph; dynamically construct adjacent edges based on the Euclidean distance in pixel space and the continuity of the direction vector; and construct a graph neural network based on the node set and adjacent edges.

[0023] A two-layer graph attention network is used to extract embedding vectors for each graph neural network to generate structural features. A regularization term is combined with the graph neural network to enhance structural anomaly perception and perform graph convolution.

[0024] Splicing the structural features and the regional features to form a fusion vector;

[0025] The fusion vector is connected to a two-layer fully connected network to output a lesion confidence score; wherein the first layer of the two-layer fully connected network uses a ReLU activation function and the second layer uses a Sigmoid to output a probability value;

[0026] The set of lesion candidate regions, structural features and lesion confidence scores are combined to generate an enhanced set of structured candidate regions.

[0027] Furthermore, the trajectory aggregation of the structured candidate region set based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set specifically includes:

[0028] Obtaining the structured candidate region set, and selecting candidate regions on two different frames as a first candidate region and a second candidate region;

[0029] Based on the similarity scores of the first candidate region and the second candidate region, it is determined whether the two regions may belong to the same lesion, that is, the same trajectory node; if the similarity score is greater than a preset threshold, the two regions are considered to be connected to the same trajectory node;

[0030] Calculating similarity scores for all candidate regions in the structured candidate region set and combining all candidate regions into trajectories; wherein the trajectory construction adopts a sliding time window to control the matching search range;

[0031] Designing a structural consistency measurement function to perform structural consistency verification on the trajectory, so as to eliminate the region with the largest structural deviation and the lowest confidence;

[0032] Output a set of cross-frame lesion tracks; each track is the continuous expression of a lesion in multiple image frames.

[0033] Furthermore, the calculating of the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set specifically includes:

[0034] Obtaining the cross-frame lesion trajectory set;

[0035] Traverse each trajectory, extract the center coordinates of the region box of each frame in the trajectory, average the center points of all frames in the trajectory, and obtain the image space position of the lesion center;

[0036] In the trajectory, the frame with the largest lesion confidence score is selected for annotation, and the lesion number, time range and lesion confidence score are annotated with text next to it.

[0037] Furthermore, the structured report includes the image spatial location of the lesion center, time range, structural features, display frame annotations, and average lesion confidence score.

[0038] In another aspect of the present invention, an artificial intelligence-based intracavitary lesion feature recognition system is provided, the system comprising:

[0039] Sequence construction module: used to obtain the original image frame sequence acquired by the electronic endoscope and the acquisition timestamp of each frame, generate a time position coding vector by normalizing the timestamp and embedding the position coding, and construct a pseudo-time image sequence set based on the original image frame sequence, normalized relative time position and time position coding vector;

[0040] Candidate extraction module: used to input the pseudo-time image sequence set into a pre-built candidate region extraction model, and output a set of lesion candidate regions, including: region position, normalized relative time position, time position encoding vector and region features;

[0041] Structural modeling module: used to enhance the vascular structure of the lesion candidate region set, construct a graph neural network and extract structural features through the graph neural network, output a lesion confidence score based on the regional features, and finally generate an enhanced structured candidate region set by combining the lesion candidate region set, structural features and lesion confidence score;

[0042] Trajectory aggregation module: used to perform trajectory aggregation on the set of structured candidate regions based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set;

[0043] Positioning output module: used to calculate the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set, and generate a structured report;

[0044] Among them, the candidate region extraction model includes an image feature extraction module, a spatiotemporal attention fusion module and a region proposal module; the image feature extraction module is used to extract the image feature tensor of each frame image; the spatiotemporal attention fusion module is used to construct a joint attention mechanism across frames; the region proposal module is used to extract a set of lesion candidate regions.

[0045] The beneficial technical effects of the present invention are at least as follows:

[0046] In response to the above problems, the present invention proposes an intelligent analysis method and system for intracavitary lesion recognition scenarios, aiming to achieve an accurate, interpretable and locatable recognition process of lesions in endoscopic images.

[0047] First, to solve the problem that existing methods cannot utilize the temporal characteristics of images, the present invention designs a dynamic sequence perception mechanism that can reconstruct the image sequence continuously acquired during the advancement of the endoscope into a visual trajectory with a temporal order. By analyzing the evolution and morphological consistency of the lesions in multiple frames, the recognition stability of blurred, occluded or tiny lesions is improved.

[0048] Secondly, to overcome the problem that traditional AI methods lack the ability to model structural medical features, the present invention introduces a structural sensitivity analysis mechanism that can extract medically relevant features such as abnormal vascular arrangement and texture mutations from candidate regions, thereby achieving effective detection and judgment of early or morphologically atypical lesions, thereby improving the interpretability and medical credibility of the model.

[0049] Finally, to address the difficulty of outputting spatial positioning information in existing technologies, the present invention designs a lesion localization strategy based on inter-image frame information backtracking and regional trajectory tracking. This allows the identified lesions to not only have semantic labels but also be accompanied by their relative spatial position in the cavity structure, facilitating doctors' structured recording, lesion comparison, and intraoperative navigation. Through the collaborative improvements in the above three aspects, the present invention has formed a systematic solution path in dynamic scene understanding, medical feature modeling, and result spatial annotation, significantly improving the clinical applicability and implementation capabilities of artificial intelligence in the context of intracavitary lesion identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0051] Figure 1 This is a flow chart of an artificial intelligence-based method for recognizing intracavitary lesion features according to an embodiment of the present invention.

[0052] Figure 2 This is a framework diagram of an artificial intelligence-based intracavitary lesion feature recognition system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0054] like Figure 1 As shown, an embodiment of the present invention provides an artificial intelligence-based method for recognizing intracavitary lesion features, the method comprising:

[0055] S1. Obtain the original image frame sequence captured by the electronic endoscope and the acquisition timestamp of each frame, generate a time position coding vector by normalizing the timestamp and embedding the position coding, and form a pseudo-time image sequence set based on the original image frame sequence, the normalized relative time position and the time position coding vector.

[0056] Specifically, the core purpose of this step is to convert the continuous image frames collected by the electronic endoscope into a sequence form with an "approximate time structure" according to their actual acquisition time and advancement order, so as to provide input support for the subsequent steps of building dynamic modeling and cross-frame lesion evolution perception capabilities. Traditional image classification models usually treat each frame as an independent sample, ignoring the temporal relationship between images, which is inappropriate in the endocrine environment. Because the lesion may gradually appear at different angles, exposures, and distances during the advancement process, it is necessary to use timing information to restore its "dynamic trajectory." This step innovatively introduces the "pseudo-time normalization + encoding embedding" method, which can construct an input sequence structure with time perception capabilities without the need for hardware positioning, taking into account both ease of implementation and modeling effectiveness.

[0057] Furthermore, the input is a sequence of original image frames And the acquisition timestamp of each frame in:

[0058] Image frame I t The image output module from the electronic endoscope uses a standard HDMI or USB stream interface, which is converted by the image acquisition card into a tensor (of the form [H, W, C], where H and W are the image height and width, respectively, and C = 3 is the number of channels);

[0059] Timestamp T t It is an automatic tag of the system mainboard or control chip during acquisition, usually in milliseconds, and is extracted through the image synchronization time stream;

[0060] The order of images is the natural order when the doctor advances the lens, so there is no absolute spatial displacement between images, but there is semantic continuity.

[0061] For example, if the doctor pushes the endoscope from the fundus to the body of the stomach, the acquisition times are:

[0062] T1=100ms, T2=120ms, T3=150ms,..., T T =1080ms (1)

[0063] Corresponding image frames {I1,I2,I3,...,I T}.

[0064] First, the original timestamp {Tt} is normalized to embed the time information into the model in floating-point form between [0,1], eliminating frame rate fluctuations or uneven frame intervals. The calculation method is as follows:

[0065]

[0066] In this formula:

[0067] T t The original timestamp of the t-th frame image comes from the endoscope control module;

[0068] T1 and T T The timestamps of the first and last frames respectively;

[0069] τ t It is the pseudo-time position of the corresponding frame, which is used to represent the relative sequential position of the image frame in the entire advancement process and is a dimensionless value;

[0070] Normalization is used to deal with the phenomenon of uneven acquisition between frame intervals.

[0071] For example, a total of 100 frames of images are collected, and a certain frame T 30 =400ms, first frame T1=100ms, last frame T 100 =1000ms, then:

[0072]

[0073] This means that frame 30 is about one-third of the way through the advancement process.

[0074] Then, in order to allow the model to further understand the temporal structure, the present invention constructs a temporal position encoding vector e for each frame. t , specifically constructed through the sine function as follows:

[0075] e t = sin(ω·τ t +φ) (4)

[0076] Where ω is the frequency control factor, which is used to adjust the distribution of time positions in the coding space. It is usually set to 2π or adaptively adjusted according to the length of the image sequence. φ is the offset term, which is used to increase coding diversity. It can be 0 or randomly set during model initialization. The coding result is the time position coding vector e t It is a one-dimensional vector that can be copied or expanded to make its dimension consistent with the image tensor channel and then concatenated with the image;

[0077] In engineering implementation, e tIt is usually broadcast to the channel dimension of the image tensor, such as expanding [H, W, C] to [H, W, C+1], where the C+1th channel is the position encoding value, which facilitates unified processing of the model.

[0078] For example: 30th frame image I 30 For an RGB image of size [720,1280,3], the position vector e is constructed 30 = sin(2π·0.333), we get e 30 ≈0.866. After concatenation, the image tensor dimension is [720, 1280, 4], which is one of the input tensors of the subsequent model.

[0079] It is important to note that all variables in this step are tensors, floating-point numbers, or vectors. No specific physical quantities or dimensional inconsistencies are involved. All processing has been done through normalization and embedding to eliminate dimensionality. Dimensional checks show that the two formulas are self-consistent, and there are no dimensional conflicts among the variables.

[0080] The output of this step is a pseudo time series image set Each element is a frame image I t , its normalized relative time position τ t And the corresponding position encoding vector e t This set will serve as the input to the spatiotemporal attention mechanism model in the next step, driving the system to identify “in which section of advancement and in which frames the sign areas of the lesion may appear.”

[0081] S2. Input the pseudo-time image sequence set into a pre-built candidate region extraction model, and output a set of lesion candidate regions, including: region position, normalized relative time position, time position encoding vector and region features.

[0082] Specifically, the goal of this step is to extract candidate regions that may contain lesions from an endoscopic image sequence with a pseudo-temporal sequential structure, combining the visual content of the image itself with its relative position during advancement. This process must not only consider the spatial information of the current frame in the image, but also fully model the temporal dependencies between frames, so as to capture suspicious areas that show gradual or fuzzy morphological changes in consecutive frames. Since intracavitary lesions may only show weak texture perturbations in some frames, or appear intermittently across frames due to changes in shooting angle and illumination, traditional single-frame processing methods are prone to miss detection. To address this problem, this step innovatively introduces a candidate region extraction network based on a "joint spatiotemporal attention mechanism" to improve the stability and accuracy of region proposals with the help of cross-frame information integration.

[0083] The input of this step is the pseudo-time image sequence set output by the previous step in:

[0084] I t is the image tensor of the t-th frame, the data format is [H, W, C], directly from the endoscope acquisition module;

[0085] τ t is the normalized relative time position of the t-th frame image, calculated by the normalization formula in the previous step;

[0086] e t is the position encoding vector of the t-th frame image, which is obtained by using τ t The calculated sinusoidal position encoding result is concatenated with the image tensor to form the input tensor [H, W, C+1].

[0087] Furthermore, this step first constructs a candidate region extraction model through a three-stage modular structure, including:

[0088] (1) Image feature extraction module, which uses a convolutional structure with a channel attention mechanism (such as ResNet18 or MobileNetV2 variants) to extract the basic visual features of each frame image and output the feature tensor F t , whose dimension is [H′,W′,D], where D is the number of channels;

[0089] (2) The spatiotemporal attention fusion module builds a cross-frame joint attention mechanism, which feeds the image features of adjacent frames and their position codes into the multi-head attention structure. Specifically, the spatiotemporal response tensor A across frames is constructed using the following formula: t :

[0090]

[0091] In this formula:

[0092] F t Represents the image feature tensor of the t-th frame, extracted by the first stage convolution module;

[0093] e t is the pseudo-temporal position encoding vector concatenated to the image in the previous step;

[0094] Softmax represents the standard attention normalization operation;

[0095] D is the number of channels, used to scale the dot product results to avoid gradient explosion;

[0096] Attention is performed on the interaction of features between frame t and its previous frame (t-1), avoiding the computational burden of long sequences while ensuring the closest semantic focus;

[0097] The innovation of this module lies in that it does not directly model the stacking of image frames, but instead uses positional encoding to guide attention alignment, allowing the model to focus on the evolving features between adjacent frames in the progression path rather than the salient regions of a single frame. This mechanism is particularly suitable for intraluminal scenes where lesions are faint and easily missed.

[0098] (3) Region proposal module, which transforms the spatiotemporal response tensor A after fusion of each frame t Input to the lightweight convolution-upsampling structure to predict the candidate lesion area heat map R on each frame image t , and then extract the candidate region set by setting the confidence threshold σ:

[0099]

[0100] Among them, R t (x, y) is the pixel-level candidate heat map of the t-th frame after fusion; Region(x, y) represents the candidate region window centered on the coordinate (x, y) and aggregated by the non-maximum suppression (NMS) method; σ is the set threshold, empirically set to 0.5, used to filter regions with strong responses; all selected regions will retain their position in the original image, normalized time τ t and position vector e t , for use in subsequent steps.

[0101] Furthermore, in the implementation, the region proposal part adopts a 3×3 convolution + ReLU + upsampling structure, and finally outputs a two-dimensional probability map with the same size as the original image, which has low computational complexity and is easy to deploy in medical devices. The final representation of each candidate region is a four-tuple in is the position bounding box of the i-th candidate region in the t-th frame, is the activation vector of the region on the feature map.

[0102] Two formulas form the core computational path of this step within the entire module. The first constructs the spatiotemporal attention aggregation process across frames, while the second uses this aggregated feature to generate a set of candidate regions. Both ensure dimensional consistency through normalization and broadcasting, eliminating dimensionality conflicts.

[0103] The output of this step is a set of candidate lesion regions. Each of these elements contains:

[0104] Regional Location 2D bounding box;

[0105] Time information τ t : Normalized relative time position;

[0106] Time position encoding vector e t : Used for subsequent structural modeling alignment;

[0107] Regional characteristics Used for structural feature modeling and trajectory matching.

[0108] Through this step, the system can accurately extract suspected lesion areas from time-series images based on the dynamic relationship and spatial distribution characteristics between frames, avoiding misjudgment and missed detection problems caused by changes in endoscopic viewing angles. The spatiotemporal attention mechanism combined with sinusoidal position encoding not only enhances the model's ability to identify "small but persistent abnormalities" in the propulsion path, but also enables the candidate region itself to have clear time-space context information, providing a key prerequisite for subsequent structural analysis and trajectory aggregation. Compared with traditional image processing methods that rely solely on image intensity or texture, this solution incorporates propulsion time information into the region proposal stage. It is a dedicated optimization for the evolution characteristics of lesions in endoscopic scenes, and is highly innovative and reproducible.

[0109] S3. Perform vascular structure enhancement on the set of lesion candidate regions, construct a graph neural network and extract structural features through the graph neural network, output a lesion confidence score in combination with the regional features, and finally generate an enhanced structured candidate region set by combining the lesion candidate region set, structural features and lesion confidence score.

[0110] Specifically, the goal of this step is to use the fine-grained structural information within the region to further discriminate based on the candidate region screened in the previous step, and identify the lesion area that truly has structural abnormalities. This module is specially designed for situations where early endoluminal lesions are not visually significant but have undergone pathological changes in microstructure (such as vascular distribution). In clinical practice, such as Barrett's esophagus, mild mucosal erosion, flat gastric cancer, etc. in gastroscopy, their color and shape may not be significantly different from the surrounding tissue, but their disordered vascular arrangement or branching angle variation is often the only clue of the early lesion, so it cannot be accurately identified by visual convolution features alone. To this end, this step innovatively introduces a regional structure graph modeling mechanism, extracts the vascular centerline inside the candidate region as a graph structure, fuses time information and depth features, and realizes structural embedding and recognition through graph neural networks. This mechanism not only enhances recognition robustness, but also provides visual and interpretable structural evidence, truly serving the clinical landing goal of the AI ​​recognition system for endoluminal lesions.

[0111] The input of this step is the candidate region set output from step 2. First, for each candidate region The present invention starts from the original image I tThe local image block is cropped according to its boundary coordinates, and the resolution is adjusted to 64×64 to keep the basic structural information undistorted. On this image block, the present invention performs the first step of atlas extraction - vascular structure enhancement processing. Here, the present invention adopts an improved Retinex spectral reconstruction algorithm to convert the image from RGB space to Lab color space, and performs adaptive contrast enhancement on the L channel, combined with detail enhancement of the green channel to maximize the visibility of microvessels. The enhanced image is binarized and edge detected, and the Steger line detection algorithm is used to extract the pixel points of the vascular centerline. These pixel points constitute the node set of the graph. Adjacent edges The regional graph structure is formed based on the Euclidean distance of the pixel space and the continuity of the direction vector.

[0112] Furthermore, the feature vector of the node contains four dimensions: normalized position (relative coordinates), main direction angle, response intensity value, and local texture gradient. For example, a node may represent a blood vessel bending point in the lower left corner of the region, and its characteristics are (0.75, 0.9, 45°, 0.82), which means that it is located relatively below the region, with a direction angle of 45 degrees, a strong image response, and obvious texture changes. After the construction is completed, a two-layer graph attention network (GAT) is used to extract the embedding vector for each structural graph to generate structural features.

[0113] Furthermore, in graph convolution, the present invention introduces a regularization term that accounts for the structural complexity of the lesion region, with the intention of enhancing the perception of irregular structures. The structural embedding calculation formula is as follows:

[0114]

[0115] where x j is the original feature of the jth node, α ij is the attention allocation weight (calculated based on node similarity and graph distance), W is the linear transformation matrix in graph convolution, and λ is the coefficient that controls the influence of the regularization term, and its empirical value is set to 0.2. It is an innovative structural variation-aware regularization term that penalizes abnormal patterns in the graph structure and is defined as:

[0116]

[0117] In this formula, deg(v) represents the node degree, and the standard deviation measures the imbalance of the number of connections. This regularization term represents the angular rate of change of the node orientation field, used to capture bending anomalies. This regularization term is differentiable and can be used to update weights during backpropagation, greatly enhancing the model's ability to identify chaotic vascular lesions.

[0118] Furthermore, we obtain the structural embedding After that, compared with the original regional characteristics Splice to form a fusion vector As the final feature of the region, it is input into a two-layer fully connected network and outputs the lesion confidence score. The first layer uses the ReLU activation function, and the second layer uses Sigmoid to output probability values. The system will filter and retain high-confidence areas based on an empirical threshold (such as 0.5).

[0119] The innovation of the entire process lies not only in the introduction of medically significant vascular maps into structural modeling, but also in the embedding of structural complexity as explicit regularization information in the graph network design, enabling the model to learn lesion characteristics from the graph topology, breaking through the limitations of visual models.

[0120] The output is an enhanced set of structured candidate regions in Structural features extracted by graph neural networks, Scoring the confidence of the lesion. This output will serve as the basic data for trajectory clustering and structured report generation.

[0121] This step combines visual convolutional features with structural medical knowledge, using a trainable graph neural module to perform structural modeling of candidate lesion regions and, based on this, determine their structural abnormalities. This method not only improves the model's sensitivity to ambiguous lesions but also formally implements a "structure-driven recognition mechanism," meaning that recognition results no longer rely solely on saliency or shape features, but are judged based on structural rationality. This is of great significance for identifying tiny lesions that are easily overlooked in real intraluminal environments, and also provides more stable and reliable input for subsequent trajectory aggregation and spatial positioning.

[0122] S4. Perform trajectory aggregation on the set of structured candidate regions based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set.

[0123] Specifically, the core goal of this step is to merge the candidate lesion regions with structural features identified in the previous step into several continuous lesion trajectory units across frames based on their coherence and structural consistency in the time series. These trajectories not only help to avoid repeated identification (for example, the same lesion is repeatedly detected in multiple frames), but also provide a continuous data basis for subsequent spatial positioning and structured output. Unlike target tracking in traditional videos, the performance of intracavitary lesions in image frames has the problem of "non-rigid drift" - for example, the position and shape of the lesion in the image may change significantly due to changes in the propulsion angle, and may even appear discontinuously in several frames. Therefore, the present invention designs a trajectory aggregation mechanism that combines structural features, temporal position and spatial proximity, which can not only tolerate slight occlusions and inter-frame jumps, but also identify continuous lesion regions with a high degree of consistency in structure.

[0124] The input used in this step is the set of structured candidate regions output in step 3. Each element represents a candidate region numbered i in the t-th frame.

[0125] The overall strategy for trajectory construction is to use a high-confidence candidate region as the "trajectory starting point" and then search for other regions with similar structure and spatial proximity within several frames before and after it, incorporating them into the same trajectory. The entire process generally consists of three sub-stages: "candidate matching," "trajectory expansion," and "consistency verification."

[0126] First, to establish the pairing logic between regions, the present invention defines a fusion similarity score function For evaluating candidate regions on two different frames and Whether they may belong to the same lesion. The specific calculation method is:

[0127]

[0128] in, is the intersection-of-bounding-boxes ratio, which measures the degree of spatial overlap between two regions; is the cosine similarity of regional structural features; |τ t -τ t′ | is the pseudo time interval between the frames of the two regions; the coefficients α, β, and γ control the weights of the three effects, which are set to 0.4, 0.5, and 0.1 in the experiment; if (The threshold can be set to 0.6), then the two regions are considered to be connected as the same trajectory node.

[0129] Let's take a practical example: suppose there is a region in the t=12th frame Structural characteristics The blood vessels are radially distributed. The area in the next frame t=13 The cosine similarity of its structural vector is 0.89, the frame overlap ratio is 0.62, and the time difference between frames is 0.01. Above the threshold δ=0.6, they are therefore merged into the same trajectory.

[0130] Furthermore, as multiple matches are performed, the present invention merges candidate regions into trajectories Trajectory construction uses a sliding time window (e.g., window size K = 3) to control the matching search range and reduce false merging. Each trajectory initially consists of at least two regions with high confidence (e.g., s > 0.6), and the remaining regions are allowed to have slightly lower confidence but high structural similarity.

[0131] Furthermore, after the trajectory is initially constructed, a structural consistency check is required to prevent the introduction of "frame skipping" or "mismatching" areas. For this purpose, the present invention designs a structural consistency measurement function:

[0132]

[0133] This function measures the average angular difference between all structural features within the trajectory. If the structural similarity of a frame in a trajectory is significantly different from that of other nodes (e.g., less than 0.7), and the confidence of that frame is only 0.35, then that region will be removed.

[0134] Finally, the present invention retains all trajectory segments with a trajectory length greater than 3 frames and qualified structural consistency, and records the region boundaries corresponding to each frame. Time position τ t , structural characteristics Confidence This information will be directly fed into the next stage for spatial positioning and structured output. The output is a set of lesion trajectories across frames:

[0135]

[0136] Each trajectory D k Represents the continuous expression of a lesion in multiple image frames. Each trajectory element in the output comes from The specific candidates are obtained during the trajectory matching calculation and consistency elimination process. The internal information of the trajectory will be used in the next step to estimate the spatial position and generate the final result.

[0137] This step builds a trajectory aggregation mechanism that is highly customized for "continuous perception of lesions in intracavitary scenarios." It combines the semantic stability of structural features with the weak continuity of time series, solving the problems of mis-aggregation and missed aggregation in the traditional spatial matching method in the identification of flat and small lesions. The design gives the model the ability to automatically remove abnormal areas that deviate from the trajectory. It is an important link in the patented overall structural perception technology chain, ensuring that the trajectory output has both continuity and structural purity.

[0138] S5. Calculate the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set, and generate a structured report.

[0139] Specifically, this step is the final link in the patented technology chain. Its goal is to convert the lesion trajectory sequence data aggregated in the previous step into structured diagnostic information and annotate it on specific image frames, making the results visible, recordable, and callable. In the actual application of endoscopy, doctors need to see not only the "yes / no" judgment given by the system, but also "which frame, location, time period, and structural characteristics of the lesion are located."

[0140] The input is the cross-frame lesion trajectory set output in step 4:

[0141]

[0142] Each trajectory D k It is a continuous expression process of a lesion identified on the time axis, in which:

[0143] represents the two-dimensional bounding box of the candidate region; τ t is the normalized relative time position of the frame; It is a vector generated by the structural graph convolutional network, which represents the overall characteristics of microstructures such as blood vessels and textures; is the lesion confidence score output by the structure discrimination network.

[0144] All the above data are provided by the previous steps, and the meanings of all variables have been defined and used previously, so no new models or intermediate variables are introduced.

[0145] Furthermore, the first step is to estimate the spatial center. The system traverses each trajectory D k , for each frame t in the trajectory Extract the coordinates of its center point:

[0146]

[0147] This calculation can be performed at the pixel level to ensure positioning accuracy. The center point is (120, 180). The system then averages the center points of all frames in the trajectory to obtain the image space position of the lesion center:

[0148]

[0149] This position indicates "where the lesion is located in the image on average during the entire trajectory" and is the basis for the system to perform frame-level annotation.

[0150] The second step is to label the visual frame. The present invention selects the lesion confidence score in the trajectory The largest frame is used as the "display frame" * , on this frame image, The mark is a red dot, and the lesion number (such as lesion_01), time range (see below), and confidence value (rounded to two digits) are annotated next to it. For example, if the frame with the maximum confidence is t = 13, the center is (240, 320), and the time interval is [0.31, 0.47], the annotation content is as follows:

[0151] [lesion_01] Position: (240,320) Time: 0.31–0.47 Confidence: 0.88

[0152] The annotation operation can be completed by calling a standard drawing interface (such as OpenCV or image control component) at the system UI layer, or the data structure can be returned in JSON format for display on the web interface.

[0153] The third step is to generate structured results. The present invention performs statistical analysis on each frame in the trajectory and generates four structured fields:

[0154] Center of space: From the above get;

[0155] Time range: determined by the minimum and maximum timestamps in the trajectory, i.e. τ min =minτ t , τ max =maxτ t ;

[0156] Average confidence score:

[0157]

[0158] Used to evaluate the overall credibility of the lesion (usually >0.5 is considered a reliable trajectory);

[0159] Center of structure vector:

[0160]

[0161] The vector dimension is fixed and can be directly used in subsequent clustering, structural analysis, type classification and other modules.

[0162] For example, trajectory D k If it contains 5 frames, average coordinates (235, 280), average confidence 0.81, and time period [0.28, 0.39], then its structured output is:

[0163] "Number": lesion_02

[0164] Center Position: (235,280)

[0165] "timeframe": [0.28,0.39]

[0166] Average Confidence: 0.81

[0167] "Structural features": (128-dimensional floating point vector)

[0168] The fourth step is output and call. All the above structured information O k will write to a uniform collection of structures:

[0169]

[0170] And provide one of the following three output methods (configurable):

[0171] Image frame output: save the annotated frame as JPEG / PNG image;

[0172] Structured reports: saved as CSV / JSON for use by PACS systems or electronic medical records;

[0173] Diagnostic system interface call: The results are returned through the REST API or local interface for display on the auxiliary diagnosis platform.

[0174] Furthermore, the final output result set Each O k Contains the following fields:

[0175] Coordinates of the spatial center of the lesion;

[0176] [τ min ,τ max ]: time range of lesion appearance;

[0177] overall confidence of the lesion;

[0178] Structural embedding vector (for explanation and classification);

[0179] Display frame label: frame number t * Annotate the image coordinates.

[0180] like Figure 2 As shown, another embodiment of the present invention provides an artificial intelligence-based intracavitary lesion feature recognition system, the system comprising:

[0181] Sequence construction module 1011: used to obtain the original image frame sequence acquired by the electronic endoscope and the acquisition timestamp of each frame, generate a time position code vector by normalizing the timestamp and embedding the position code, and construct a pseudo-time image sequence set based on the original image frame sequence, the normalized relative time position and the time position code vector;

[0182] Candidate extraction module 1012: used to input the pseudo-time image sequence set into a pre-built candidate region extraction model, and output a set of lesion candidate regions, including: region position, normalized relative time position, time position encoding vector and region features;

[0183] Structural modeling module 1013: configured to perform vascular structure enhancement on the lesion candidate region set, construct a graph neural network and extract structural features through the graph neural network, output a lesion confidence score based on the regional features, and finally generate an enhanced structured candidate region set by combining the lesion candidate region set, the structural features, and the lesion confidence score;

[0184] Trajectory aggregation module 1014: configured to perform trajectory aggregation on the set of structured candidate regions based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set;

[0185] Positioning output module 1015: used to calculate the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set, and generate a structured report;

[0186] Among them, the candidate region extraction model includes an image feature extraction module, a spatiotemporal attention fusion module and a region proposal module; the image feature extraction module is used to extract the image feature tensor of each frame image; the spatiotemporal attention fusion module is used to construct a joint attention mechanism across frames; the region proposal module is used to extract a set of lesion candidate regions.

[0187] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the embodiment scheme, and no limitation is made here.

[0188] In addition, for technical details not fully described in this embodiment, please refer to the parameter operation method provided in any embodiment of the present invention, and will not be repeated here.

[0189] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0190] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0191] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0192] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An artificial intelligence-based method for recognizing intracavitary lesion features, characterized in that: The method comprises: S1. Obtaining an original image frame sequence acquired by an electronic endoscope and an acquisition timestamp of each frame, generating a time position coding vector by normalizing the timestamp and embedding a position code, and forming a pseudo-time image sequence set based on the original image frame sequence, the normalized relative time position, and the time position coding vector; S2, inputting the pseudo-time image sequence set into a pre-built candidate region extraction model, and outputting a set of lesion candidate regions, including: region position, normalized relative time position, time position encoding vector, and region features; S3, performing vascular structure enhancement on the set of lesion candidate regions, constructing a graph neural network and extracting structural features through the graph neural network, outputting a lesion confidence score in combination with the regional features, and finally generating an enhanced structured candidate region set by combining the lesion candidate region set, the structural features, and the lesion confidence score; S4. performing trajectory aggregation on the set of structured candidate regions based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set; S5. Calculating the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set, and generating a structured report; Among them, the candidate region extraction model includes an image feature extraction module, a spatiotemporal attention fusion module and a region proposal module; the image feature extraction module is used to extract the image feature tensor of each frame image; the spatiotemporal attention fusion module is used to construct a joint attention mechanism across frames; the region proposal module is used to extract a set of lesion candidate regions.

2. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 1, characterized in that: The normalizing of the timestamp and embedding the position code to generate a time position code vector specifically includes: Generate a time-dependent temporal position encoding vector through a sine function.

3. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 1, characterized in that: The step of inputting the pseudo-time image sequence set into a pre-built candidate region extraction model and outputting a set of lesion candidate regions specifically includes: A convolutional structure with a channel attention mechanism is used to extract the basic visual features of each frame image and output an image feature tensor; the image feature tensor is input into the spatiotemporal attention fusion module to construct a cross-frame joint attention mechanism, and the image features of adjacent frames and their position codes are fed into the multi-head attention structure to generate a cross-frame spatiotemporal response tensor; The spatiotemporal response tensor is input into the region proposal module, and the region proposal module inputs the fused spatiotemporal response tensor of each frame into a lightweight convolution-upsampling structure, predicts the candidate lesion region heat map on each frame image, and then extracts the candidate region set by setting a confidence threshold to obtain the lesion candidate region set.

4. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 3, characterized in that: The candidate region is represented as a quad in is the position bounding box of the i-th candidate region in the t-th frame, is the activation vector of the region on the feature map, τ t is the normalized relative time position, e t is the time position encoding vector; The convolution-upsampling structure is a 3×3 convolution, ReLU and upsampling structure, which outputs a two-dimensional probability map.

5. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 1, characterized in that: The step of performing vascular structure enhancement on the set of candidate lesion regions specifically includes: An improved Retinex spectral reconstruction algorithm was used to convert the image of the lesion candidate region set from RGB space to Lab color space, and adaptive contrast enhancement was performed on the L channel. Combined with detail enhancement of the green channel, a set of lesion candidate regions with enhanced vascular structure was obtained.

6. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 5, characterized in that: The method of constructing a graph neural network and extracting structural features through the graph neural network, outputting a lesion confidence score in combination with regional features, and finally generating an enhanced structured candidate region set by combining the lesion candidate region set, the structural features, and the lesion confidence score, specifically includes: Obtain a set of candidate lesion regions with enhanced vascular structure, extract vascular centerline pixels using the Steger line detection algorithm, and use them as a node set of the graph; dynamically construct adjacent edges based on the Euclidean distance in pixel space and the continuity of the direction vector; and construct a graph neural network based on the node set and adjacent edges. A two-layer graph attention network is used to extract embedding vectors for each graph neural network to generate structural features. A regularization term is combined with the graph neural network to enhance structural anomaly perception and perform graph convolution. Splicing the structural features and the regional features to form a fusion vector; The fusion vector is connected to a two-layer fully connected network to output a lesion confidence score; wherein the first layer of the two-layer fully connected network uses a ReLU activation function and the second layer uses a Sigmoid to output a probability value; The set of lesion candidate regions, structural features and lesion confidence scores are combined to generate an enhanced set of structured candidate regions.

7. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 1, characterized in that: The performing trajectory aggregation on the set of structured candidate regions based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set specifically includes: Obtaining the structured candidate region set, and selecting candidate regions on two different frames as a first candidate region and a second candidate region; Based on the similarity scores of the first candidate region and the second candidate region, it is determined whether the two regions may belong to the same lesion, that is, the same trajectory node; if the similarity score is greater than a preset threshold, the two regions are considered to be connected to the same trajectory node; Calculating similarity scores for all candidate regions in the structured candidate region set and combining all candidate regions into trajectories; wherein the trajectory construction adopts a sliding time window to control the matching search range; Designing a structural consistency measurement function to perform structural consistency verification on the trajectory, so as to eliminate the region with the largest structural deviation and the lowest confidence; Output a set of cross-frame lesion tracks; each track is the continuous expression of a lesion in multiple image frames.

8. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 1, characterized in that: The calculating the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set specifically includes: Obtaining the cross-frame lesion trajectory set; Traverse each trajectory, extract the center coordinates of the region box of each frame in the trajectory, average the center points of all frames in the trajectory, and obtain the image space position of the lesion center; In the trajectory, the frame with the largest lesion confidence score is selected for annotation, and the lesion number, time range and lesion confidence score are annotated with text next to it.

9. The method for recognizing intracavitary lesion features based on artificial intelligence according to claim 8, characterized in that: The structured report includes the image spatial location of the lesion center, time range, structural features, display frame annotations, and average lesion confidence score.

10. An artificial intelligence-based intracavitary lesion feature recognition system, characterized in that: The system comprises: Sequence construction module: used to obtain the original image frame sequence acquired by the electronic endoscope and the acquisition timestamp of each frame, generate a time position coding vector by normalizing the timestamp and embedding the position coding, and construct a pseudo-time image sequence set based on the original image frame sequence, normalized relative time position and time position coding vector; Candidate extraction module: used to input the pseudo-time image sequence set into a pre-built candidate region extraction model, and output a set of lesion candidate regions, including: region position, normalized relative time position, time position encoding vector and region features; Structural modeling module: used to enhance the vascular structure of the lesion candidate region set, construct a graph neural network and extract structural features through the graph neural network, output a lesion confidence score based on the regional features, and finally generate an enhanced structured candidate region set by combining the lesion candidate region set, structural features and lesion confidence score; Trajectory aggregation module: used to perform trajectory aggregation on the set of structured candidate regions based on structural feature similarity and temporal continuity to form a cross-frame lesion trajectory set; Positioning output module: used to calculate the spatial center position and time range of the lesion trajectory according to the cross-frame lesion trajectory set, and generate a structured report; Among them, the candidate region extraction model includes an image feature extraction module, a spatiotemporal attention fusion module and a region proposal module; the image feature extraction module is used to extract the image feature tensor of each frame image; the spatiotemporal attention fusion module is used to construct a joint attention mechanism across frames; the region proposal module is used to extract a set of lesion candidate regions.

Images