Capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring

CN122990304APending Publication Date: 2026-10-27THE SECOND AFFILIATED HOSPITAL ARMY MEDICAL UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611126985.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-28
Publication Date
2026-10-27

AI Technical Summary

Technical Problem

上述现有技术均针对静态单帧医学图像设计,未适配结肠胶囊内镜连续视频帧的时序特性,无法建模病灶时序依赖关系,存在时序信息利用缺失、帧间运动模糊与肠道蠕动冗余未处理的问题,仅采用通道-空间双注意力机制而未设计边缘高频特征增强模块,导致病灶细微特征表征能力不足,仅使用常规交叉熵损失或基础对比损失而未引入类别中心锚定约束,使得相似病灶特征区分度低、微小病灶特征易被肠道背景淹没,同时未针对胶囊内镜光照不均、黏液/泡沫干扰、成像质量不稳定的复杂肠道成像环境设计自适应预处理机制,模型泛化性与鲁棒性较差,无法满足结肠息肉、炎性病变、血管畸形等病灶的细粒度精准分类临床需求

Benefits of technology

[0029]本发明的有益效果在于:本发明与现有技术相比,通过时序三重注意力与边缘高频增强机制显著强化病灶边缘及细微纹理特征,有效提升病灶细粒度特征表征能力,创新结合胶囊网络与时序LSTM适配结肠胶囊内镜连续视频帧特性,可精准建模病灶时序依赖关系、消除帧间运动模糊与肠道蠕动冗余干扰,提升分类结果的稳定性与一致性,采用对比锚定特征校准策略同步优化类内特征紧凑性与类间特征区分度,大幅提高相似病灶与微小病灶的判别精度、降低误分率,依托肠道环境自适应预处理模块显著增强模型对光照不均、黏液/泡沫干扰等复杂肠道成像场景的鲁棒性与泛化性,整体采用端到端轻量化架构,在保障细粒度分类精度的同时兼顾高效推理速度,可满足结肠胶囊内镜临床实时筛查需求,为病灶精准分型提供可靠AI辅助支撑,有效降低人工漏诊、误诊风险。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122990304A_ABST
    Figure CN122990304A_ABST
Patent Text Reader

Abstract

The present application relates to a kind of timing triple attention and contrast anchoring capsule endoscopy lesion fine granularity classification method, belong to medical instrument image processing field, comprising the following steps: S1: to colon capsule endoscopy continuous video frame is preprocessed, extraction lesion area and standardization;S2: construct 3 branch adaptive receptive field convolution network, according to the size of lesion dynamic adjustment receptive field, synchronous capture global contour, local detail, small lesion characteristics;S3: on the basis of channel-space dual attention, newly add edge high-frequency attention, strengthen lesion edge details, suppress background noise;S4: adapt to capsule endoscopy continuous video frame, combine capsule network and LSTM, model interframe temporal dependence relationship of lesion;S5: adaptive weighted fusion multi-scale spatial features and temporal features, optimize feature distribution by contrast loss and class center anchoring loss;S6: the fusion feature is input into classification head, and the output lesion class probability is completed fine granularity classification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of medical device image processing technology, and relates to a fine-grained classification method for capsule endoscopy lesions based on temporal triple attention and contrast anchoring. Background Technology

[0002] Non-invasive screening using colon capsule endoscopy is a core method for the early diagnosis of colorectal diseases. Automatic fine-grained lesion classification is a key step in AI-assisted diagnosis, directly determining the accuracy of clinical subtyping. CN120976604A discloses an image classification method for ulcerative colitis based on a multi-scale attention fusion network, which uses the EfficientNet backbone network and a multi-scale attention fusion module to enhance and classify lesion features. CN119444690A discloses an image classification method for breast cancer tissue based on multi-scale feature fusion and attention mechanisms, which improves the classification accuracy of static medical images through multi-scale feature extraction and channel-space integrated attention mechanisms. The aforementioned existing technologies are all designed for static single-frame medical images and are not adapted to the temporal characteristics of continuous video frames in colon capsule endoscopy. They cannot model the temporal dependencies of lesions and suffer from problems such as missing temporal information utilization, unprocessed inter-frame motion blur, and unhandled intestinal peristalsis redundancy. They only use a channel-space dual attention mechanism without designing an edge high-frequency feature enhancement module, resulting in insufficient ability to represent subtle lesion features. They only use conventional cross-entropy loss or basic contrast loss without introducing class center anchoring constraints, resulting in low feature differentiation of similar lesions and small lesion features being easily submerged by the intestinal background. At the same time, they do not design an adaptive preprocessing mechanism for the complex intestinal imaging environment of capsule endoscopy, such as uneven illumination, mucus / foam interference, and unstable imaging quality. The model has poor generalization and robustness and cannot meet the clinical needs for fine-grained and accurate classification of lesions such as colonic polyps, inflammatory lesions, and vascular malformations. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a fine-grained classification method for lesions in colonic capsule endoscopy based on temporal triple attention, edge high-frequency enhancement, and contrast anchoring calibration. By constructing a temporal feature modeling, edge high-frequency enhancement, contrast anchoring calibration, and intestinal environment adaptive preprocessing mechanism, the method strengthens the temporal correlation and subtle feature representation capabilities of lesions, improves the classification accuracy of similar and small lesions, enhances the robustness of the model to complex imaging scenarios, and meets the clinical needs for fine-grained and accurate classification of lesions in colonic capsule endoscopy.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A fine-grained classification method for capsule endoscopy lesions based on temporal triple attention and contrast anchoring includes the following steps: S1: Preprocess the continuous video frames of the colon capsule endoscopy, extract the lesion area and standardize it; S2: Construct a 3-branch adaptive receptive field convolutional network to dynamically adjust the receptive field according to the lesion size and simultaneously capture global contours, local details, and small lesion features; S3: Based on channel-space dual attention, add edge high-frequency attention to enhance lesion edge details and suppress background noise; S4: Adapt to continuous video frames of capsule endoscopy, combine capsule network and LSTM to model the temporal dependency relationship between lesion frames; S5: Adaptive weighted fusion of multi-scale spatial and temporal features, optimizing feature distribution through contrast loss and class center anchoring loss; S6: Input the fused features into the classification head, output the lesion category probability, and complete the fine-grained classification.

[0005] Furthermore, step S1 specifically includes the following steps: S11: Locate the lesion in continuous video frames of a colonic capsule endoscopy, crop the region of interest (ROI) of the lesion, remove redundant background interference from the intestine, and simultaneously perform inter-frame spatial alignment to eliminate positional shifts caused by intestinal peristalsis, obtaining a standardized lesion region image.

[0006] In the formula This represents the tensor of the original image after lesion cropping; , , These represent the height, width, and number of channels of the lesion image, respectively. S12: Adaptive Local Normalization of the Intestinal Environment Local window adaptive normalization is used instead of global normalization to adapt to local light differences in the intestine and suppress mucus and foam noise.

[0007] In the formula This represents a standard lesion image after adaptive normalization. Represented by coordinates The average pixel value of the local window centered on the target; Represented by coordinates The standard deviation of pixels in the local window centered on the target; To represent a very small constant, avoid having a denominator of zero.

[0008] Furthermore, in step S2, a 3-branch adaptive receptive field convolutional network is constructed to extract features at large, medium, and small scales, forming a multi-scale feature set:

[0009] in ~ These are feature maps for scales 1 to 3; Single-scale feature calculation formula:

[0010] In the formula: Indicates the first The original multi-scale features directly output by each convolutional branch. ; Indicates the first Each convolutional branch, adaptive receptive field Convolution operation; Indicates the first Each convolutional branch convolutional layer bias term; This represents the ReLU activation function, used to introduce nonlinear features.

[0011] Furthermore, step S3 specifically includes the following steps: S31: Channel attention weighting: Focusing on key pathways characteristic of lesions and suppressing interference from redundant pathways:

[0012]

[0013] In the formula This represents the channel attention weight matrix; This indicates global average pooling, which extracts global features from each channel. Global max pooling is used to extract significant channel features. This represents a multilayer perceptron, used for channel feature dimension transformation and weight learning; This represents the feature map after channel attention weighting; This represents the original multi-scale feature directly output by the k-th convolutional branch; S32: Enhanced high-frequency edge attention: Extract gradient features at lesion edges to enhance the detailed representation of small lesions and blurred edges:

[0014]

[0015] In the formula This represents the image gradient operator, used to extract high-frequency features of lesion edges; express Convolution operations are used for edge feature filtering and enhancement. Indicates the high-frequency attention weights at the edge; This represents the feature map after high-frequency enhancement at the edges; S33: Spatial Attention Weighting: Focusing on the spatial area of ​​the lesion and suppressing interference from the intestinal background:

[0016]

[0017] In the formula This represents the concatenation operation of feature map channel dimensions; This represents a convolution operation, used for spatial weight learning; Represents the spatial attention weight matrix; This represents the final enhancement feature of temporal triple attention.

[0018] Furthermore, step S4 specifically includes the following steps: S41: Capsule Feature Encoding: The attention-enhanced features are converted into capsule vectors, preserving the spatial pose and structural information of the lesions.

[0019] In the formula Represents the capsule feature vector; Represents the weight matrix of the capsule network; Indicates the capsule network bias term; This represents the hyperbolic tangent activation function and the normalized capsule vector values. S42: Temporal LSTM Feature Modeling Temporal encoding of capsule features in consecutive video frames captures the temporal variation patterns of lesions:

[0020] In the formula express LSTM hidden layer state at time step; express Time capsule feature vector; express LSTM hidden layer state at time step; S43: Temporal Feature Fusion: By concatenating all hidden states within the timing window, the complete timing fusion feature is obtained:

[0021] In the formula Indicates the characteristics of time-series capsule fusion; Indicates the length of the timing window.

[0022] Furthermore, step S5 specifically includes the following steps: S51: Adaptive weighted feature fusion Dynamically allocate weights between multi-scale and temporal features to maximize the utilization of effective features:

[0023] In the formula Indicates the final fusion characteristics; Indicates the first Adaptive fusion weights based on scale features; Indicates the time series characteristic balance coefficient; S52: Comparison of anchoring loss calculation: Total loss By contrast loss Category center anchoring loss Weighted composition achieves both intra-class compactness and inter-class separation:

[0024] Comparative loss Used to bring similar lesion features closer together and to separate different lesion features;

[0025] Category center anchoring loss This is used to constrain similar features to aggregate towards the category center, thereby improving feature stability;

[0026] In the formula This represents the total loss of the model. Indicates comparative loss, This indicates the category center anchoring loss; This represents the anchoring loss weighting coefficient; This represents the cosine similarity function, which measures the similarity between features; Indicates the first Individual sample fusion features; Indicates the first Each sample possesses characteristics of positive samples of the same type; Indicates the first Features of out-of-class negative samples; Represents the smooth similarity distribution coefficient; Indicates the number of positive samples; Indicates the number of negative samples; Indicates the first Central features of lesion categories corresponding to each sample; express Norm, which calculates the distance between a feature and the class center.

[0027] Furthermore, in step S6, the fused features are input into the classification head, and the lesion category probability is output to complete fine-grained classification:

[0028] In the formula Represents the probability vector of lesion categories; This represents the weight matrix of the fully connected layer; Indicates the bias term of the fully connected layer; This represents the normalization exponential function, which converts features into probabilities for each category. The final classification result is the lesion category corresponding to the maximum value in the probability vector P.

[0029] The beneficial effects of this invention are as follows: Compared with the prior art, this invention significantly enhances the edge and fine texture features of lesions through temporal triple attention and edge high-frequency enhancement mechanisms, effectively improving the ability to represent fine-grained features of lesions. It innovatively combines capsule networks and temporal LSTM to adapt to the continuous video frame characteristics of colon capsule endoscopy, which can accurately model the temporal dependency of lesions, eliminate inter-frame motion blur and intestinal peristalsis redundancy interference, and improve the stability and consistency of classification results. It adopts a contrast anchoring feature calibration strategy to simultaneously optimize the compactness of intra-class features and the distinguishability of inter-class features, which greatly improves the discrimination accuracy of similar lesions and small lesions and reduces the misclassification rate. Relying on the intestinal environment adaptive preprocessing module, it significantly enhances the robustness and generalization of the model to complex intestinal imaging scenarios such as uneven illumination and mucus / foam interference. The overall end-to-end lightweight architecture ensures both fine-grained classification accuracy and efficient inference speed, which can meet the clinical real-time screening needs of colon capsule endoscopy, provide reliable AI-assisted support for accurate lesion classification, and effectively reduce the risk of missed diagnosis and misdiagnosis by humans.

[0030] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 Flowchart of a capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring. Detailed Implementation

[0032] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0033] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0034] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0035] Example 1: like Figure 1 As shown, this invention provides a fine-grained classification method for capsule endoscopy lesions based on temporal triple attention and contrast anchoring. It employs a complete technical path including adaptive intestinal environment preprocessing; multi-scale adaptive receptive field feature extraction; temporal triple attention feature enhancement; temporal capsule feature fusion; adaptive feature fusion and contrast anchoring calibration; and fine-grained classification output. Through the collaborative efforts of six core modules, it achieves accurate fine-grained classification of colonic capsule endoscopy lesions. This invention uses continuous video frames from colonic capsule endoscopy as input. First, it completes lesion region localization and adaptive intestinal environment preprocessing. Then, it extracts multi-scale adaptive receptive field features through a multi-branch network. High-frequency temporal triple attention (channel-space-edge) enhances subtle lesion features. Combining capsule networks and LSTM to model the temporal dependencies of lesions, it optimizes feature distribution through contrast anchoring loss, and finally outputs fine-grained lesion classification results. The entire process is lightweight and end-to-end, adaptable to real-time clinical screening needs. This method includes the following steps: Step 1: Adaptive pretreatment of the intestinal environment It solves the problems of uneven illumination, mucus / foam interference, and motion blur in capsule endoscopy, and accurately extracts and standardizes lesion areas.

[0036] (1) Precise cropping and frame alignment of lesion ROI Lesion localization was performed on continuous video frames of colon capsule endoscopy. Regions of interest (ROIs) were cropped, redundant background interference from the intestine was removed, and inter-frame spatial alignment was performed to eliminate positional shifts caused by intestinal peristalsis, resulting in standardized lesion region images.

[0037] In the formula: This represents the tensor of the original image after lesion cropping; , , These represent the height, width, and number of channels (3 for RGB) of the lesion image, respectively. (2) Adaptive local normalization of the intestinal environment Local window adaptive normalization is used instead of global normalization to adapt to local light differences in the intestine and suppress mucus / foam noise.

[0038] In the formula: This represents a standard lesion image after adaptive normalization. Indicates: with The average pixel value of the local window centered on the target; Indicated by The standard deviation of pixels in the local window centered on the target; Represents a minimal constant (with values) (This is to avoid the denominator being zero.)

[0039] Step 2: Multi-scale adaptive receptive field feature extraction The receptive field is dynamically adjusted based on lesion size to simultaneously capture global contours, local details, and features of minute lesions, addressing the issue of detail loss in single-scale features. A three-branch adaptive receptive field convolutional network is constructed to extract features at large, medium, and small scales, forming a multi-scale feature set.

[0040] Single-scale feature calculation formula:

[0041] In the formula: Indicates the first The original multi-scale features directly output by each convolutional branch ( ); Indicates the first Each convolutional branch, adaptive receptive field Convolution operation (microlesions) Shrinkage, large lesions expand); Indicates the first Each convolutional branch convolutional layer bias term; This represents the ReLU activation function, used to introduce nonlinear features and improve the model's representational ability.

[0042] Step 3: Temporal Triple Attention Feature Enhancement Based on channel-space dual attention, edge high-frequency attention is added to enhance the details of lesion edges and suppress background noise, specifically addressing the problems of blurred lesion edges and weak texture of small lesions in capsule endoscopy.

[0043] (1) Channel attention weighting Focusing on key pathways characteristic of lesions and suppressing interference from redundant pathways:

[0044]

[0045] In the formula: This represents the channel attention weight matrix; This indicates global average pooling, which extracts global features from each channel. Global max pooling is used to extract significant channel features. This represents a multilayer perceptron, used for channel feature dimension transformation and weight learning; This means that the original multi-scale features directly output by the k-th convolutional branch are used as the input to the subsequent triple attention module. It contains the basic multi-scale features of the lesion, and also carries redundant channels, background noise and other invalid information. The feature map after channel attention weighting is Channel attention weight Features obtained after element-wise weighting. Key channels contributing to lesion classification are enhanced, while redundant and noisy channels are suppressed. The result after the first round of feature enhancement.

[0046] (2) Enhanced high-frequency attention at the edge Extract gradient features at lesion edges to enhance the detailed representation of small lesions and blurred edges:

[0047]

[0048] In the formula: This represents the image gradient operator, used to extract high-frequency features of lesion edges; express Convolution operations (most target lesions in colon capsule endoscopy are tiny lesions <3mm, and the size of the convolution receptive field is highly matched with that of tiny lesions) are used for edge feature filtering and enhancement; Indicates the high-frequency attention weights at the edge; This represents the feature map after high-frequency enhancement at the edges.

[0049] (3) Spatial attention weighting Focusing on the lesion area, suppressing background interference from normal intestinal mucosa, mucus, etc.

[0050]

[0051] In the formula: This represents the concatenation operation of feature map channel dimensions; This represents a convolution operation, used for spatial weight learning; Represents the spatial attention weight matrix; This represents the final enhancement feature of temporal triple attention.

[0052] Step 4: Time Capsule Feature Fusion Adapting to continuous video frames of capsule endoscopy, and combining capsule network and LSTM, the temporal dependency relationship between lesion frames is modeled to solve motion blur and temporal redundancy problems.

[0053] (1) Capsule feature coding The attention-enhanced features are converted into capsule vectors, preserving the spatial pose and structural information of the lesions.

[0054] In the formula: Represents the capsule feature vector; Represents the weight matrix of the capsule network; Indicates the capsule network bias term; This represents the hyperbolic tangent activation function and the normalized capsule vector values.

[0055] (2) Temporal LSTM feature modeling Temporal encoding of capsule features in consecutive video frames captures the temporal variation patterns of lesions:

[0056] In the formula: express LSTM hidden layer state at time step; express Time capsule feature vector; express The state of the LSTM hidden layer at each time step.

[0057] (3) Fusion of temporal features By concatenating all hidden states within the timing window, the complete timing fusion feature is obtained:

[0058] In the formula: Indicates the characteristics of time-series capsule fusion; Indicates the length of the timing window (set according to the video frame rate, with a value of 8~16).

[0059] Step 5: Adaptive Feature Fusion and Contrast Anchoring Calibration Adaptive weighted fusion of multi-scale spatial and temporal features is used to optimize feature distribution through contrast loss and class center anchoring loss, thereby improving the distinguishability of similar lesions.

[0060] (1) Adaptive weighted feature fusion Dynamically allocate weights between multi-scale and temporal features to maximize the utilization of effective features:

[0061] In the formula: Indicates the final fusion characteristics; Indicates the first Adaptive fusion weights based on scale features; Represents the time series characteristic balance coefficient (a learnable parameter).

[0062] (2) Comparison of anchorage loss calculation The total loss is a weighted average of the contrast loss and the class center anchoring loss, achieving both intra-class compactness and inter-class separation.

[0063] Contrast loss: bringing similar lesion features closer together and distancing different lesion features away from them.

[0064]

[0065] Category center anchoring loss: Constrains similar features to aggregate towards the category center, improving feature stability.

[0066]

[0067] In the formula: This represents the total loss of the model; Indicates comparative loss; This indicates the category center anchoring loss; This represents the anchoring loss weighting coefficient (values ​​range from 0.2 to 0.5). This represents the cosine similarity function, which measures the similarity between features; Indicates the first Individual sample fusion features; Indicates the first Each sample possesses characteristics of positive samples of the same type; Indicates the first Features of out-of-class negative samples; This represents the smooth similarity distribution coefficient (values ​​range from 0.1 to 0.2). Indicates the number of positive samples; Indicates the number of negative samples; Indicates the first Central features of lesion categories corresponding to each sample; express Norm, which calculates the distance between a feature and the class center.

[0068] Step 6: Output of fine-grained lesion classification The fused features are input into the classification head, which outputs the probability of lesion category to complete fine-grained classification.

[0069]

[0070] In the formula: Represents the probability vector of lesion categories; This represents the weight matrix of the fully connected layer; Indicates the bias term of the fully connected layer; This represents the normalization exponential function, which converts features into probabilities for each category.

[0071] Final classification result: The lesion category corresponding to the maximum value in the probability vector P (colonial polyps, inflammatory lesions, ulcers, vascular malformations, etc.) is selected. Example 2: An electronic device, comprising a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method described in Embodiment 1 when executing the computer program.

[0072] Example 3: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0073] Example 4: A computer program product includes a computer program that, when executed by a processor, implements the method described in Example 1.

[0074] In the above embodiments, the reference to "this embodiment" in the specification indicates that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple appearances of "this embodiment" do not necessarily refer to the same embodiment.

[0075] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0076] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0077] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0078] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0079] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0080] This invention can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0081] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A fine-grained classification method for capsule endoscopy lesions based on temporal triple attention and contrast anchoring, characterized in that: Includes the following steps: S1: Preprocess the continuous video frames of the colon capsule endoscopy, extract the lesion area and standardize it; S2: Construct a 3-branch adaptive receptive field convolutional network to dynamically adjust the receptive field according to the lesion size and simultaneously capture global contours, local details, and small lesion features; S3: Based on channel-space dual attention, add edge high-frequency attention to enhance lesion edge details and suppress background noise; S4: Adapt to continuous video frames of capsule endoscopy, combine capsule network and LSTM to model the temporal dependency relationship between lesion frames; S5: Adaptive weighted fusion of multi-scale spatial and temporal features, optimizing feature distribution through contrast loss and class center anchoring loss; S6: Input the fused features into the classification head, output the lesion category probability, and complete the fine-grained classification.

2. The capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: Locate the lesion in continuous video frames of a colonic capsule endoscopy, crop the region of interest (ROI) of the lesion, remove redundant background interference from the intestine, and simultaneously perform inter-frame spatial alignment to eliminate positional shifts caused by intestinal peristalsis, obtaining a standardized lesion region image. In the formula This represents the tensor of the original image after lesion cropping; , , These represent the height, width, and number of channels of the lesion image, respectively. S12: Adaptive Local Normalization of the Intestinal Environment Local window adaptive normalization is used instead of global normalization to adapt to local light differences in the intestine and suppress mucus and foam noise. In the formula This represents a standard lesion image after adaptive normalization. Represented by coordinates The average pixel value of the local window centered on the target; Represented by coordinates The standard deviation of pixels in the local window centered on the target; To represent a very small constant, avoid having a denominator of zero.

3. The capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring according to claim 1, characterized in that: In step S2, a 3-branch adaptive receptive field convolutional network is constructed to extract features at large, medium, and small scales, forming a multi-scale feature set: in ~ These are feature maps for scales 1 to 3; Single-scale feature calculation formula: In the formula: Indicates the first The original multi-scale features directly output by each convolutional branch. ; Indicates the first Each convolutional branch, adaptive receptive field Convolution operation; Indicates the first Each convolutional branch convolutional layer bias term; This represents the ReLU activation function, used to introduce nonlinear features.

4. The capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31: Channel attention weighting: Focusing on key pathways characteristic of lesions and suppressing interference from redundant pathways: In the formula This represents the channel attention weight matrix; This indicates global average pooling, which extracts global features from each channel. Global max pooling is used to extract significant channel features. This represents a multilayer perceptron, used for channel feature dimension transformation and weight learning; This represents the feature map after channel attention weighting; This represents the original multi-scale feature directly output by the k-th convolutional branch; S32: Enhanced high-frequency edge attention: Extract gradient features at lesion edges to enhance the detailed representation of small lesions and blurred edges: In the formula This represents the image gradient operator, used to extract high-frequency features of lesion edges; express Convolution operations are used for edge feature filtering and enhancement. Indicates the high-frequency attention weights at the edge; This represents the feature map after high-frequency enhancement at the edges; S33: Spatial Attention Weighting: Focusing on the spatial area of ​​the lesion and suppressing interference from the intestinal background: In the formula This represents the concatenation operation of feature map channel dimensions; This represents a convolution operation, used for spatial weight learning; Represents the spatial attention weight matrix; This represents the final enhancement feature of temporal triple attention.

5. The capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring according to claim 1, characterized in that: Step S4 Specifically, the following steps are included: S41: Capsule Feature Encoding: The attention-enhanced features are converted into capsule vectors, preserving the spatial pose and structural information of the lesions. In the formula Represents the capsule feature vector; Represents the weight matrix of the capsule network; Indicates the capsule network bias term; This represents the hyperbolic tangent activation function and the normalized capsule vector values. S42: Temporal LSTM Feature Modeling Temporal encoding of capsule features in consecutive video frames captures the temporal variation patterns of lesions: In the formula express LSTM hidden layer state at time step; express Time capsule feature vector; express LSTM hidden layer state at time step; S43: Temporal Feature Fusion: By concatenating all hidden states within the timing window, the complete timing fusion feature is obtained: In the formula Indicates the characteristics of time-series capsule fusion; Indicates the length of the timing window.

6. The capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring according to claim 1, characterized in that: Step S5 specifically includes the following steps: S51: Adaptive weighted feature fusion Dynamically allocate weights between multi-scale and temporal features to maximize the utilization of effective features: In the formula Indicates the final fusion characteristics; Indicates the first Adaptive fusion weights based on scale features; Indicates the time series characteristic balance coefficient; S52: Comparison of anchoring loss calculation: Total loss By contrast loss Category center anchoring loss Weighted composition achieves both intra-class compactness and inter-class separation: Comparative loss Used to bring similar lesion features closer together and to separate different lesion features; Category center anchoring loss This is used to constrain similar features to aggregate towards the category center, thereby improving feature stability; In the formula This represents the total loss of the model. Indicates comparative loss, This indicates the category center anchoring loss; This represents the anchoring loss weighting coefficient; This represents the cosine similarity function, which measures the similarity between features; Indicates the first Individual sample fusion features; Indicates the first Each sample possesses characteristics of positive samples of the same type; Indicates the first Features of out-of-class negative samples; Represents the smooth similarity distribution coefficient; Indicates the number of positive samples; Indicates the number of negative samples; Indicates the first Central features of lesion categories corresponding to each sample; express Norm, which calculates the distance between a feature and the class center.

7. The capsule endoscopy lesion fine-grained classification method based on temporal triple attention and contrast anchoring according to claim 1, characterized in that: In step S6, the fused features are input into the classification head, and the lesion category probability is output to complete the fine-grained classification. In the formula Represents the probability vector of lesion categories; This represents the weight matrix of the fully connected layer; Indicates the bias term of the fully connected layer; This represents the normalization exponential function, which converts features into probabilities for each category. The final classification result is the lesion category corresponding to the maximum value in the probability vector P.

Citation Information

Patent Citations

  • Breast cancer tissue image classification method based on multi-scale feature fusion and attention mechanism

    CN119444690A