Medical image lesion recognition method and system based on double-path attention mechanism

Abstract

The application belongs to the technical field of medical image segmentation and recognition, and particularly relates to a medical image lesion recognition method and system based on a double-path attention mechanism, which comprises the following steps: obtaining a CT image to be recognized; extracting global features of the obtained CT image based on a global average attention mechanism; performing spatial perception of the obtained CT image according to the extracted global features and a learnable position-weighted global attention mechanism to obtain a global attention path of the CT image; introducing a soft Top-K attention mechanism on the obtained global attention path to obtain a sparse attention result of the CT image; and performing lesion classification and detection of the CT image according to the obtained sparse attention result and a dynamic fusion control mechanism based on variance driving to realize recognition of medical image lesions based on the double-path attention mechanism.

Description

Technical Field

[0001] This invention belongs to the field of medical image segmentation and recognition technology, specifically relating to a method and system for medical image lesion recognition based on a dual-path attention mechanism. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of medical artificial intelligence, constructing deep learning models that combine computational efficiency, discrimination accuracy, and interpretability has become a core issue in the field of medical image analysis. In the context of precision medicine, predicting key molecular biomarkers based on medical images is gradually becoming an important technological pathway for achieving non-invasive diagnosis and personalized treatment. Taking KRAS gene mutations in non-small cell lung cancer (NSCLC) as an example, these mutations directly affect the formulation of targeted therapy strategies and have significant clinical guiding significance. However, currently used tissue or liquid biopsy methods suffer from drawbacks such as high invasiveness, long cycles, and high costs, making it difficult to meet the clinical requirements for real-time performance and broad applicability. Therefore, there is an urgent need for non-invasive lesion identification and prediction methods based on conventional medical images.

[0004] In recent years, the Transformer architecture has been widely used in medical imaging (covering disease classification, tumor detection, organ segmentation, etc.) due to its excellent global modeling capabilities. However, medical images are typically characterized by high resolution, sparse lesions, and heterogeneous structural distribution, which poses two challenges to the dense attention mechanism of the standard Transformer in practical applications: first, the O(N²) computational complexity limits its deployment efficiency in high-resolution images; second, attention is easily scattered, making it difficult to accurately focus on key lesion regions, thus affecting the model's discriminative ability and clinical interpretability, and failing to meet the dual clinical requirements of efficiency and robustness.

[0005] Based on this, recognition models can be optimized through sparse attention, lightweight network structures, or dynamic mechanisms. However, model optimization is generally built on natural images or public medical datasets, lacking validation with real clinical data. Most of the sparse strategies of the models are statically designed, and local and global branches are often treated separately, making it difficult to achieve effective semantic collaboration. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a medical image lesion recognition method and system based on a dual-path attention mechanism. Employing a variance-driven dual-path attention mechanism, it introduces sparse Top-K local paths and learnable global paths within the standard Transformer framework. A dynamic fusion control mechanism based on global feature variance enables adaptive modeling of images with different structures, achieving adaptive integration of local and global features. This significantly improves the model's discriminative and spatial focusing capabilities without increasing computational load.

[0007] According to some embodiments, the first solution of the present invention provides a medical image lesion recognition method based on a dual-path attention mechanism, employing the following technical solution: A medical image lesion recognition method based on a dual-path attention mechanism includes: Acquire the CT image to be identified; Global features of the acquired CT images are extracted based on a global average attention mechanism; Spatial perception of the acquired CT images is performed based on the extracted global features and a learnable location-weighted global attention mechanism to obtain the global attention path of the CT images. A soft Top-K attention mechanism is introduced into the obtained global attention path to obtain sparse attention results for CT images; Based on the obtained sparse attention results and the variance-driven dynamic fusion control mechanism, lesion classification and detection of CT images are performed, realizing the identification of medical image lesions based on the dual-path attention mechanism.

[0008] As a further technical limitation, the variance-driven dynamic fusion control mechanism uses the variance of the global features of the extracted CT image as the control signal to measure the structural complexity and spatial dispersion of the CT image; when the variance is larger, the measured CT image has several active regions, and the Top-K branch is strengthened to focus the CT image; when the variance is smaller, the structure of the measured CT image is concentrated.

[0009] As a further technical limitation, the variance-driven dynamic fusion control mechanism dynamically calculates the fusion weights of the CT image to obtain the weighted output of the local-global path of the CT image. ;Right now ;in, This indicates sparse fusion, i.e. , This represents the global feature variance of the current image's channels. It is a learnable linear transformation function. For the Sigmoid activation function, ensure ; This represents the output of the sparse attention mechanism in the two-branch configuration. This represents the output of introducing a sparse Top-K local path in a two-branch configuration.

[0010] As a further technical limitation, in the process of extracting global features of the acquired CT image based on the global average attention mechanism, the global average attention mechanism is used as the starting point for lightweight modeling. Average pooling operation is performed on the CT image to ignore the spatial differences between various locations in the CT image and obtain unified global features.

[0011] As a further technical limitation, the learnable position-weighted global attention mechanism uses a trainable position weight vector to model position importance, adaptively learns the attention level of different positions during training, enhances the spatial selection of global attention, and realizes global perception convergence with learnable spatial weights.

[0012] As a further technical limitation, before introducing the soft Top-K attention mechanism, a hard Top-K attention mechanism is introduced on the global attention path of the obtained CT image to construct a sparse matrix of the attention score matrix. The obtained sparse matrix is ​​then subjected to Softmax normalization to obtain sparse attention weights. The attention output is obtained by combining the obtained sparse attention weights.

[0013] According to some embodiments, a second aspect of the present invention provides a medical image lesion recognition system based on a dual-path attention mechanism, employing the following technical solution: A medical image lesion recognition system based on a dual-path attention mechanism includes: The acquisition module is configured to acquire the CT image to be identified; The extraction module is configured to extract global features from the acquired CT images based on a global average attention mechanism. The perception module is configured to perform spatial perception of the acquired CT image based on the extracted global features and a learnable location-weighted global attention mechanism, thereby obtaining the global attention path of the CT image. The sparse module is configured to introduce a soft Top-K attention mechanism on the obtained global attention path to obtain sparse attention results for CT images; The recognition module is configured to classify and detect lesions in CT images based on the obtained sparse attention results and a variance-driven dynamic fusion control mechanism, thereby realizing the recognition of medical image lesions based on a dual-path attention mechanism.

[0014] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium, employing the following technical solution: A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the medical image lesion recognition method based on a dual-path attention mechanism as described in the first aspect of the present invention.

[0015] According to some embodiments, the fourth aspect of the present invention provides an electronic device, which adopts the following technical solution: An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps in the medical image lesion recognition method based on dual-path attention mechanism as described in the first aspect of the present invention.

[0016] According to some embodiments, the fifth aspect of the present invention provides a computer program product, which adopts the following technical solution: A computer program product includes software code, wherein the program in the software code performs the steps of the medical image lesion recognition method based on dual-path attention mechanism as described in the first aspect of the present invention.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention employs a variance-driven dual-path attention mechanism. Based on the standard Transformer framework, it introduces sparse Top-K local paths and learnable global paths. Based on a dynamic fusion control mechanism of global feature variance, it achieves adaptive modeling of images with different structures and adaptive integration of local and global features. This significantly improves the model's discriminative and spatial focusing capabilities without increasing computational load. Attached Figure Description

[0018] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0019] Figure 1 This is a flowchart of the medical image lesion recognition method based on dual-path attention mechanism in Embodiment 1 of the present invention; Figure 2 This is an architectural diagram of the dual-path attention mechanism in Embodiment 1 of the present invention; Figure 3 This is an architecture diagram of the global average attention mechanism in Embodiment 1 of the present invention; Figure 4 This is an architecture diagram of the Hard Top-K filtering in the sparse attention mechanism of Embodiment 1 of the present invention; Figure 5 This is a computational architecture diagram of the soft Top-K attention mechanism in Embodiment 1 of the present invention; Figure 6 This is an architecture diagram of the variance-driven dynamic fusion control mechanism in Embodiment 1 of the present invention; Figure 7 This is an architecture diagram of adaptive Class Token generation based on cosine similarity in Embodiment 1 of the present invention; Figure 8 This is a schematic diagram illustrating the relationship between AUC and computational complexity for each model in the KRAS mutation prediction task in Embodiment 1 of the present invention. Figure 9 This is a layered visualization diagram of the attention heatmaps generated by the different stages of the model in Embodiment 1 of the present invention. Figure 10 This is a visual comparison diagram of the confusion matrices of different representative models in Embodiment 1 of the present invention. Figure 11 This is a structural block diagram of the medical image lesion recognition system based on dual-path attention mechanism in Embodiment 2 of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0023] In this invention, terms such as "upper," "lower," "left," "right," "front," "back," "vertical," "horizontal," "side," and "bottom" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only to facilitate the description of the structural relationships of the various components or elements of this invention and do not specifically refer to any component or element in this invention. They should not be construed as limiting the invention.

[0024] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0025] Example 1 Embodiment 1 of this invention introduces a medical image lesion recognition method based on a dual-path attention mechanism.

[0026] like Figure 1 The method for identifying lesions in medical images based on a dual-path attention mechanism, as shown, includes: Acquire the CT image to be identified; Global features of the acquired CT images are extracted based on a global average attention mechanism; Spatial perception of the acquired CT images is performed based on the extracted global features and a learnable location-weighted global attention mechanism to obtain the global attention path of the CT images. A soft Top-K attention mechanism is introduced into the obtained global attention path to obtain sparse attention results for CT images; Based on the obtained sparse attention results and the variance-driven dynamic fusion control mechanism, lesion classification and detection of CT images are performed, realizing the identification of medical image lesions based on the dual-path attention mechanism.

[0027] To achieve lesion recognition in medical images, this embodiment adopts a variance-driven dual-path attention (VDPA) mechanism. With structural interpretability and dynamic modeling capabilities as the core, it jointly constructs sparse local paths and weighted global paths, and introduces a dynamic fusion control mechanism based on global feature variance to achieve adaptive modeling of images with different structures.

[0028] It should be noted that the VDPA module possesses excellent modularity and portability, making it suitable for seamless integration with various Transformer backbone architectures; specifically: The VDPA module includes: (1) Learnable Weighted Global Pooling (LWGP), which realizes global perception and convergence with learnable spatial weights; (2) Soft Top-K Attention, which achieves a balance between sparsity and differentiability; (3) Variance-Driven Fusion Controller (VDFC), which uses global feature variance to realize path dynamic fusion; and (4) Adaptive Local Pooling (ALP), which constructs context-aware dynamic class tokens to enhance global semantic expression.

[0029] The Transformer architecture has been widely used in natural language processing and computer vision due to its superior global modeling capabilities. Vision Transformer (ViT), a representative of visual Transformers, has been applied to various medical imaging tasks, such as tumor segmentation, disease classification, and organ recognition. Compared to traditional convolutional neural networks (CNNs), Transformers are better at capturing long-range feature dependencies and exhibit significant advantages in modeling complex spatial structures.

[0030] Because medical images are generally characterized by high resolution, sparse lesions, and uneven spatial distribution, the dense attention mechanism of the standard Transformer faces two challenges in practical applications: its O(N²) computational complexity limits its scalability and deployment efficiency under high-resolution inputs, and the difficulty in effectively focusing on key lesion regions due to easily scattered attention affects the model's diagnostic accuracy and clinical interpretability. Therefore, to meet the requirements of efficient structural adaptation modeling for medical image tasks, it is necessary to strengthen research on improved Transformer architectures.

[0031] To alleviate the computational burden of dense attention mechanisms in high-resolution image processing, the density of the attention matrix can be effectively reduced by using local window constraints, sliding expansion, or random connections of multiple sparse attention models, thereby reducing the original O(N²) complexity to a linear level.

[0032] In medical image analysis tasks, although existing studies have attempted to optimize computational efficiency using sparse attention mechanisms, the static regularization design employed is difficult to adapt to the characteristics of sparse lesion regions and irregular location distribution. In addition, the structural assumptions of natural language processing (such as sliding windows and global token injection) have limited adaptability in two-dimensional images and cannot be directly transferred. At the same time, the separation of global and local modeling, the lack of a collaborative mechanism, the complex structure, and the poor pluggability of modules make it difficult to integrate into mainstream visual Transformer frameworks (such as ViT and Swin) or actual medical AI systems.

[0033] To address the need for modeling sparse and heterogeneous structures in medical images, there is an urgent need for a sparse attention mechanism that is structurally simple, dynamically adjustable, and capable of global and local collaborative modeling. Therefore, this embodiment proposes a medical image lesion recognition method based on a dual-path attention mechanism.

[0034] Dynamic attention mechanism intelligently allocates attention resources according to the complexity of input features, thereby improving the model’s focusing ability and discrimination efficiency. However, medical image recognition based on dynamic attention spine has the following shortcomings: (1) It lacks a dynamic sparsity mechanism that adapts to the sparse characteristics of the structure in medical images, making it difficult to achieve efficient and accurate modeling of key regions; (2) It depends on specific architecture customization, lacks universality and transferability, and is difficult to deploy across models; (3) Although some lightweight attention strategies can improve inference efficiency, they are prone to sacrificing expressive ability and training stability.

[0035] Therefore, this embodiment integrates sparse modeling, adaptive control, and structural universality in its design. It can efficiently focus on key lesion areas and has good interpretability and easy integration, providing a universal, efficient, and practical attention mechanism solution for medical image analysis.

[0036] like Figure 2 The overall architecture diagram shown below, in which, Figure 2 The left side of the diagram represents the main workflow. After the input image undergoes PatchEmbedding, local features are extracted by the Adaptive Weighted Neighborhood Aggregation module. These features then enter a multi-layer Dynamic Dual-Path Attention Block to fuse local and global information. Finally, the KRAS mutation prediction result is output through the MLP Head. Figure 2 The right side of the image shows the details of the VDPA single-layer structure, namely Multi-Head QKV Projection, Dynamic Class Token building module (Dynamic Context-AwareTokenization), Top-K sparse attention branch, Learnable Weighted Pooling global branch, and fusion mechanism controlled by Adaptive Fusion Controller, which together achieve dynamic synergy between spatial focus and global modeling capabilities.

[0037] The Transformer self-attention mechanism, with its excellent global modeling capabilities, has become a core component in deep learning. However, its O(N²) computational complexity (where N is the number of tokens) leads to significant resource consumption and deployment bottlenecks in high-resolution input scenarios such as medical images. Therefore, this embodiment proposes a Global Average Attention (GAA) mechanism as a starting point for lightweight modeling, such as... Figure 3As shown, by performing average pooling on all Key(K) and Value(V), a unified global representation is generated, thereby avoiding the overhead of calculating the similarity between pairs of tokens in traditional self-attention; its mathematical expression is as follows: ; ; ; Where Q, K, and V represent the query vector, key vector, and value vector, respectively; Softmax represents the activation function; and T represents the transpose of the vector matrix.

[0038] The global average attention mechanism proposed in this embodiment greatly reduces computational complexity, making it particularly suitable for edge deployments or latency-sensitive medical scenarios. However, because it uses uniform average pooling to construct the global representation, it ignores the spatial differences between locations, resulting in a uniform attention response that is difficult to effectively focus on key lesion areas, thus limiting the model's expressive power in high-precision diagnostic tasks.

[0039] To enhance the spatial selectivity of global attention, this embodiment borrows the learnable positional encoding concept from Vision Transformer (ViT), introducing trainable positional weight vectors to dynamically model the importance of each position. For example... Figure 3 As shown, this embodiment uses learnable position-weighted global attention to introduce spatial awareness, replacing the average pooling operation in the original GlobalAttention branch with Learnable Weighted Pooling (LWGP) to achieve weighted aggregation of Key and Value; the calculation process is as follows: ; ; ; ; in, represents the learnable unknown parameters; softmax represents the activation function, such that w is distributed between 0 and 1; This means that w becomes a value between 0 and 1 after being normalized by the Sigmoid function; This represents the globally weighted Key; K i V represents the key vector at the i-th position; i Represents the Value vector at the i-th position; Value represents the globally weighted representation; d represents the scaling factor of the feature dimension, i.e., the dimension factor used for normalization in attention.

[0040] The learnable location-weighted global attention adopted in this embodiment breaks through the assumption of spatial homogeneity of average pooling, enabling the network to adaptively learn the attention level of different locations during training, thereby enhancing the spatial perception ability of global modeling. In actual training, the attention map shows stronger response contrast, indicating that its overall expressive ability is enhanced.

[0041] LWGP still has certain limitations in actual focusing performance. Due to the tendency of position weight distribution to be uniform in the early stage of training, although the model is enhanced overall, it is difficult to highlight key areas. This can lead to the phenomenon of "background enhancement masking lesions", which limits its ability to make fine discrimination in medical image tasks with sparse lesions and significant local differences. Although Learnable Weighted Pooling effectively improves the global perception intensity, its spatial selection ability is still insufficient, suggesting that introducing sparse attention with explicit selection mechanism will be the key direction for the next stage of optimization.

[0042] To address this, this embodiment employs a sparse strategy of forced focus using hard Top-K attention. Lesions in medical images typically exhibit spatially sparse and semantically key distribution characteristics. Traditional global attention mechanisms inevitably introduce a large amount of invalid background information when calculating the correlation between all tokens, which both distracts the model and results in a high computational cost of O(N²).

[0043] like Figure 4 As shown, this embodiment introduces a Hard Top-K Sparsity (HTOP-KA) mechanism on the global attention path, forcing each query to interact only with the top K most relevant keys, thereby improving the model's ability to focus on key regions and significantly reducing redundant computation, thus sparsifying the attention score matrix. The calculation is defined as follows: ; in, i and j These represent the x and y coordinates of the matrix, respectively.

[0044] The sparse matrix is ​​normalized using Softmax to obtain the sparse attention weights. ,Right now: ; The final attention output is ; This embodiment reduces the attention complexity from O(N²) to O(N·K) using a hard Top-K sparsity mechanism, significantly reducing invalid computations and enhancing the model's spatial focus, making it particularly suitable for medical imaging tasks dominated by "critical few" features. However, Top-K operations are inherently non-differentiable discrete choices and cannot participate in gradient backpropagation, often leading to training instability and convergence difficulties, becoming a major bottleneck for performance optimization. Therefore, a differentiable sparsity mechanism can be introduced to achieve a balance between training friendliness and focusing performance. To address the non-differentiability issue of the hard Top-K attention mechanism during training, existing research has introduced temperature factors and smoothing functions to construct continuous sparse masks, thereby achieving differentiable sparsity processing of the attention matrix; however, these methods are mostly derived from natural language or general image tasks and are difficult to directly adapt to the structurally sparse and highly focused characteristics of medical images. Therefore, this embodiment proposes a simpler, training-friendly, and more adaptable soft Top-K attention mechanism to balance sparsity and optimization stability; such as... Figure 5 As shown, the calculation process includes: (1) Introduce a temperature factor γ to scale the original attention score matrix A to generate a smooth sparse mask: ; A smaller γ value can amplify the weight differences of highly relevant tokens, making the attention distribution sparser. Meanwhile, Softmax maintains the function's continuity and differentiability, avoiding the gradient interruption problem caused by hard truncation.

[0045] (2) Combine the original attention matrix A with the mask Element-wise multiplication (Hadamard product) to achieve flexible Top-K weighting: ; (3) Use the sparsified attention weights to participate in the standard weighted operation and output the final attention result: ; The soft Top-K attention mechanism in this embodiment effectively improves the model's response to key regions without increasing structural complexity, alleviates training instability, and can be used to meet the needs of sparse attention for high-resolution input of medical images.

[0046] Key lesion regions in medical images typically exhibit spatial sparseness, small area, but semantic significance; while background regions possess extensive contextual information but suffer from information redundancy. The lesion identification model in this embodiment needs to possess both local focusing capabilities and global modeling capabilities to improve the completeness of lesion diagnosis and discrimination.

[0047] To achieve collaborative modeling of local and global attention paths, this embodiment introduces a static fusion strategy based on fixed weights, which is widely used in multi-path attention mechanisms and has demonstrated effectiveness in multimodal fusion tasks. It is simple to implement, has controllable computational overhead, and can effectively integrate local details and global semantic information as a baseline. It is a commonly used method in current multi-path attention fusion.

[0048] In this embodiment, Top-K attention and global attention outputs are integrated using a linear weighting method during the fusion process: ; This embodiment lacks adaptability across different image structures; however, the fusion weight α is a static hyperparameter, which can easily lead to problems such as attention shift, information redundancy, or insufficient key responses when faced with samples with strong structural heterogeneity and significant differences in lesions. Therefore, this embodiment introduces a feature variance-driven dynamic fusion mechanism (VDFC) to improve the flexibility and discrimination stability of attention fusion.

[0049] While static fusion strategies can initially coordinate local and global attention, their fixed weighting method struggles to adapt to medical image samples with vastly different structures. For example, in CT images of different patients, lesion areas may be densely or sparsely distributed, and background texture complexity varies significantly. In such cases, a uniform fusion ratio can easily lead to attention shift, information redundancy, or modeling imbalance. To enhance the model's adaptability to structural diversity, this embodiment proposes a method such as... Figure 6 The Variance-Driven Fusion Controller (VDFC) mechanism is shown. It introduces global feature variance as a control signal to measure the structural complexity and spatial dispersion of the input image. The larger the variance, the more active regions there are in the image, and the model should tend to strengthen the Top-K branches to achieve fine focusing. When the variance is small, the structure is more concentrated, and it is more suitable to rely on global path modeling to achieve semantic consistency.

[0050] Based on VDFC, the fusion weights are dynamically calculated, and the weighted output of the local-global path is implemented: ; Wherein, the fusion coefficient α(x) is ;in, This represents the global feature variance of the current image's channels. It is a learnable linear transformation function. For the Sigmoid activation function, ensure .

[0051] The variance-driven dynamic fusion control mechanism in this embodiment does not require manual setting of the fusion ratio. It can automatically learn the optimal weight configuration under each structural condition during training, realizing true input-aware path control. When processing medical images with complex structures and strong lesion heterogeneity, VDFC can significantly improve the model's discrimination stability and region focusing ability, and has good versatility and clinical adaptability.

[0052] To address the issue that fixed Class Tokens cannot adapt to structurally heterogeneous medical images, this embodiment introduces an Adaptive Local Pooling (ALP) mechanism to construct context-aware classification labels. Referring to strategies such as TokenLearner, and considering the characteristics of medical images—high sparsity and semantic concentration—a similarity-weighted dynamic aggregation process is designed. Specifically: Given a sequence of Patch Tokens for an input image ALP first constructs a local neighborhood of K nearest neighbors for each token. The normalized weights are calculated based on cosine similarity to measure spatial or semantic relevance within the neighborhood; the calculation is as follows: ; Based on the aforementioned weighted coefficients, local information is aggregated from the neighborhood, and a dynamic ClassToken is constructed through global average pooling to form the final context-aware representation. Its calculation is expressed as follows: ; like Figure 7 As shown, the adaptive local pooling mechanism in this embodiment has the advantages of dynamically adjusting the token expression range, enhancing structural sensitivity, and improving semantic compression quality. The final generated Class Token is sent to the classification head to participate in the discrimination task, forming a three-branch structure with the Top-K sparse path and the global weighted path in VDPA, thereby achieving comprehensive modeling of the feature space.

[0053] The data used in this embodiment came from CT image samples of 166 patients with non-small cell lung cancer (NSCLC), of whom 84 were KRAS mutation positive and 82 were KRAS mutation negative; the patients' ages ranged from 30 to 88 years, with a mean age of 63 years (±10); in terms of population composition, the proportion of women in the KRAS mutation group was significantly higher (77.4%), and the proportions of smokers and drinkers were also relatively higher; complete demographic characteristics are shown in Table 1.

[0054] In terms of pathological and clinical characteristics, the KRAS mutation group was mainly adenocarcinoma (95.2%), and there were also some differences in hypertension, diabetes and tumor stage; among them, the differences in tumors are shown in Table 2.

[0055] Table 1. Demographic Characteristics

[0056] Table 2 Tumor Characteristics

[0057] In this embodiment, all experiments were run on a single NVIDIA RTX 4090 GPU. The optimizer used momentum-driven SGD (momentum coefficient 0.9 in this embodiment, weight decay 5e-5), with an initial learning rate of 0.001, dynamically adjusted using cosine annealing and LambdaLR strategies. The total number of training epochs was set to 300, and the batch size was 128. To mitigate overfitting, Dropout (ratio 0.2) and data augmentation strategies including random horizontal flips and ±10° random rotations were introduced during training. An Early Stopping strategy was implemented; training was terminated early if the validation set AUC did not show significant improvement within 60 consecutive epochs. Model parameters were initialized using the Kaiming Uniform method, and some pre-trained layers were frozen in the early stages of training to accelerate convergence and improve stability.

[0058] CT image data were annotated by professional radiologists. For each patient, three consecutive single-channel slices were selected from the vicinity of the tumor area and stitched together into a three-channel image. All images were uniformly scaled to 512×512 resolution and normalized to adapt to the input format and numerical stability requirements of the deep learning model.

[0059] In this embodiment, model evaluation uses two core metrics: AUC (Area Under the Curve) and ACC (Accuracy). F1-Score and Precision are also introduced to provide a more comprehensive quantitative analysis of the model's classification performance. AUC, as an important indicator of the model's overall discriminative ability, is particularly representative in scenarios of class imbalance in medical images.

[0060] To understand the performance of different attention mechanisms in spatial focusing and semantic recognition, this embodiment introduces a multi-level attention heatmap visualization method to show the response characteristics and spatial coverage of lesion areas at different stages.

[0061] To comprehensively evaluate the performance-computational efficiency trade-off of the proposed attention mechanism, this embodiment constructs eight phased models based on standard Multi-Head Self-Attention (MHSA), as shown in Table 3. Lightweight strategies (GAA, HTK), global enhancement mechanisms (LGA, STK), static fusion mechanisms (STK-LGA), and dynamic fusion and semantic awareness modules (VDPA with / without ALP) are gradually introduced, ultimately forming a complete VDPA model. Except for comparing models with and without ALP, all other comparative models are built on a foundation including ALP to ensure consistent evaluation. In the table, F1 is the harmonic mean of precision and recall.

[0062] Table 3. Eight Stage Models

[0063] The relationship between AUC and computational complexity (GFLOPs) for each model in the KRAS mutation prediction task is as follows: Figure 8 As shown, we get: Group A (GAA → LGA): Under the premise of basically the same computational cost, the AUC increased from 0.765 to 0.810, which verified the enhancement effect of the global weighting mechanism LGA in information integration; Group B (HTK → STK): The soft Top-K attention mechanism improves training stability and sparse representation quality compared to hard screening, with AUC increasing to 0.834; Group C (STK-LGA → VDPA w / o ALP → VDPA with ALP): While keeping the number of FLOPs essentially unchanged, dynamic fusion and adaptive pooling mechanisms are gradually introduced, resulting in continuous improvement in model performance. VDPA improves AUC by +0.044 compared to STK-LGA and by +0.021 compared to the version without ALP, indicating that ALP has a significant positive effect on semantic compression and context fusion.

[0064] It should be noted that the VDPA (with ALP) model significantly outperforms MHSA (AUC 0.792) in AUC (0.932) and F1 (0.893), while the FLOPs are basically the same, fully demonstrating the overall advantages of this embodiment in terms of discrimination performance, computational efficiency and structural versatility.

[0065] To gain a deeper understanding of the differences between various attention mechanisms in terms of spatial focusing and semantic representation, this embodiment analyzes the attention heatmaps generated by the model at each stage as follows: Figure 9The layered visualization shown is as follows: HSA (Stage 1): Attention weights are evenly distributed, lacking obvious spatial focus; GAA and LGA (Stage 2–3): After introducing global weighting, the model initially exhibits spatial selectivity, but the response is still relatively smooth. HTK and STK (Stage 4–5): Sparse mechanisms enhance the concentration of response in critical regions; STK-LGA (Stage 6): Effective fusion of local sparsity and global position awareness, resulting in more stable structural focusing; VDPA (with vs. without ALP): The changes in the mid-layer heatmaps of both are not significant, but ALP brings an improvement of AUC +0.021 and F1 +0.019, indicating that its compression and expression enhancement effects at the semantic level are more significant.

[0066] In the NSCLC KRAS mutation prediction task, this embodiment compares and evaluates VDPA with a variety of mainstream CNN and Transformer models. The evaluation results are shown in Table 4. All models use the same data processing and training strategies to ensure the fairness of the comparison.

[0067] Table 4. Performance comparison of different models in the KRAS mutation prediction task

[0068] Experimental results show that VDPA outperforms representative architectures such as ResNet50 (AUC=0.719), EfficientNet-B0 (AUC=0.830), and Swin-Tiny (AUC=0.875) in all four core metrics, with a relative performance improvement of 4.8%–11.2%.

[0069] It's important to note that VDPA's performance improvement doesn't rely on simply increasing model size, but rather stems from the effective synergy of structural innovation and attention path control strategies. These include a Top-K sparsity mechanism, a variance-driven dynamic fusion controller, and contextual class tokens constructed using Adaptive Local Pooling. These modules collectively enhance the model's discriminative representation, spatial focusing ability, and training robustness, resulting in excellent adaptability to medical image tasks.

[0070] To verify the structural adaptability and cross-modal generalization ability of VDPA in different tasks, this embodiment conducts transfer testing on the publicly available Chest X-ray pneumonia classification dataset (using 5,856 chest X-ray images collected by a medical center). The dataset covers both normal and pneumonia (PNEUMONIA) classes, with images acquired uniformly at the PA position, and patients aged 1–5 years. All experiments strictly adhered to patient-based partitioning for both training and testing sets to ensure no data leakage.

[0071] Under the same data preprocessing and training configuration, VDPA was compared and evaluated with various mainstream CNN models (such as ResNet, MobileNet, and EfficientNet) and Transformer models (such as ViT and Swin). The comparison metrics included AUC, ACC, F1, Precision, and NORMAL class recognition rate, which comprehensively measured the model's discriminative performance and class balance ability. The results are shown in Table 5.

[0072] Table 5. Performance comparison of each model in the pneumonia binary classification task and its ability to identify the NORMAL class.

[0073] As shown in Table 5, the VDPA model achieved the best overall performance: AUC reached 0.9570, F1 score was 0.9060, Precision was 0.8530, and the NORMAL class recognition rate was 0.72. Figure 10 As shown, although ViT-Base has a high recall rate for the PNEUMONIA class, its recognition rate for the NORMAL class is only 24%, with a significantly high false positive rate. This indicates that the Transformer has a tendency to "over-globalize" in fine-grained structural modeling, making it difficult to distinguish normal tissues. CNN models such as ResNet50 have a more balanced recognition performance in the NORMAL class (71% and 77%), indicating that convolutional structures have certain advantages in local modeling and boundary preservation.

[0074] The VDPA model in this embodiment effectively achieves key structure perception and region focusing in medical images by combining a parallel-designed Top-K sparse path and a global attention path with a variance-driven fusion strategy, while maintaining controllable computational complexity. Confusion matrix visualization shows that VDPA achieves a more balanced classification accuracy across the two classes of samples (NORMAL: 0.72, PNEUMONIA: 0.97) while maintaining a low false positive rate, demonstrating excellent task robustness and clinical applicability.

[0075] This embodiment is based on real-collected NSCLC CT image data (a total of 166 cases, including 84 KRAS positive cases) to conduct evaluation experiments. The results show that the VDPA model achieved leading performance in the KRAS gene mutation prediction task with AUC 0.923, ACC 0.866, F1 0.870 and Precision 0.933, which comprehensively surpasses mainstream models such as ResNet, ViT, and Swin. In the ChestX-ray public pneumonia classification task, VDPA also maintained excellent generalization ability and NORMAL class recognition rate, demonstrating good cross-task transferability and structural generality, verifying its potential for wide application in intelligent diagnosis of medical images. In the real clinical non-small cell lung cancer (NSCLC) KRAS mutation prediction task, VDPA achieved AUC 0.923, ACC 0.866, F1 0.870 and Precision 0.933 on the CT image dataset (166 patients), which is better than mainstream models such as ResNet, ViT and Swin. Further generalization experiments on the ChestX-ray public pneumonia classification dataset showed that VDPA also maintained a high recognition rate and good class balance for the NORMAL class, verifying its cross-task transferability and structural universality. While improving the clinical adaptability of the Transformer architecture, VDPA achieved triple optimization of performance, efficiency and interpretability, providing a feasible new paradigm for building a reliable and deployable intelligent medical image analysis system.

[0076] This embodiment employs a variance-driven dual-path attention mechanism. Based on the standard Transformer framework, it introduces sparse Top-K local paths and learnable global paths. Based on a dynamic fusion control mechanism using global feature variance, it achieves adaptive modeling of images with different structures and adaptive integration of local and global features. This significantly improves the model's discriminative and spatial focusing capabilities without increasing computational load.

[0077] Example 2 Embodiment 2 of the present invention introduces a medical image lesion recognition system based on a dual-path attention mechanism.

[0078] like Figure 11 The illustrated medical image lesion recognition system based on a dual-path attention mechanism includes: The acquisition module is configured to acquire the CT image to be identified; The extraction module is configured to extract global features from the acquired CT images based on a global average attention mechanism. The perception module is configured to perform spatial perception of the acquired CT image based on the extracted global features and a learnable location-weighted global attention mechanism, thereby obtaining the global attention path of the CT image. The sparse module is configured to introduce a soft Top-K attention mechanism on the obtained global attention path to obtain sparse attention results for CT images; The recognition module is configured to classify and detect lesions in CT images based on the obtained sparse attention results and a variance-driven dynamic fusion control mechanism, thereby realizing the recognition of medical image lesions based on a dual-path attention mechanism.

[0079] The detailed steps are the same as those of the medical image lesion recognition method based on dual-path attention mechanism provided in Example 1, and will not be repeated here.

[0080] Example 3 Embodiment 3 of the present invention provides a computer-readable storage medium.

[0081] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the medical image lesion recognition method based on a dual-path attention mechanism as described in Embodiment 1 of the present invention.

[0082] The detailed steps are the same as those of the medical image lesion recognition method based on dual-path attention mechanism provided in Example 1, and will not be repeated here.

[0083] Example 4 Embodiment 4 of the present invention provides an electronic device.

[0084] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the medical image lesion recognition method based on dual-path attention mechanism as described in Embodiment 1 of the present invention.

[0085] The detailed steps are the same as those of the medical image lesion recognition method based on dual-path attention mechanism provided in Example 1, and will not be repeated here.

[0086] Example 5 Embodiment 5 of the present invention provides a computer program product.

[0087] A computer program product includes software code, wherein the program in the software code performs the steps of the medical image lesion recognition method based on dual-path attention mechanism as described in Embodiment 1 of the present invention.

[0088] The detailed steps are the same as those of the medical image lesion recognition method based on dual-path attention mechanism provided in Example 1, and will not be repeated here.

[0089] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0090] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0091] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0094] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0095] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for identifying lesions in medical images based on a dual-path attention mechanism, characterized in that, include: Acquire the CT image to be identified; Global features of the acquired CT images are extracted based on a global average attention mechanism; Spatial perception of the acquired CT images is performed based on the extracted global features and a learnable location-weighted global attention mechanism to obtain the global attention path of the CT images. The learnable position-weighted global attention mechanism is used to introduce spatial awareness, replacing the average pooling operation in the original Global Attention branch with LearnableWeighted Pooling to achieve weighted aggregation of Key and Value; the calculation process is as follows: ； ； ； ； in, represents the learnable unknown parameters; softmax represents the activation function, such that w is distributed between 0 and 1; This means that w becomes a value between 0 and 1 after being normalized by the Sigmoid function; This represents the globally weighted key; K i V represents the key vector at the i-th position; i The Value vector representing the i-th position; d represents the globally weighted Value; d represents the scaling factor of the feature dimension, i.e., the dimension factor used for normalization in attention. A soft Top-K attention mechanism is introduced onto the obtained global attention path to obtain sparse attention results for the CT image; the calculation process of the soft Top-K attention mechanism includes: Introducing a temperature factor γ to scale the original attention score matrix A generates a smooth sparse mask: ； Among them, a smaller γ value amplifies the weight differences of highly relevant tokens, making the attention distribution sparser; at the same time, Softmax keeps the function continuous and differentiable, avoiding the gradient interruption problem caused by hard truncation. Compare the original attention matrix A with the mask Element-wise multiplication to achieve flexible Top-K weighting: ； The sparsified attention weights are used in the standard weighted operation to output the final attention result: ； in, V Represents a value vector; Based on the obtained sparse attention results and a variance-driven dynamic fusion control mechanism, lesion classification and detection in CT images are performed, realizing the identification of medical image lesions based on a dual-path attention mechanism. Specifically, global feature variance is introduced as a modulation signal to measure the structural complexity and spatial dispersion of the input image; the variance-driven dynamic fusion control mechanism dynamically calculates the fusion weights and realizes the weighted output of local-global paths. ； Wherein, the fusion coefficient α(x) is ;in, This represents the global feature variance of the current image's channels. It is a learnable linear transformation function. For the Sigmoid activation function, ensure ; This represents the output of introducing sparse Top-K local paths in a two-branch configuration. This represents the output of the sparse attention mechanism in the two-branch approach.

2. The medical image lesion recognition method based on a dual-path attention mechanism as described in claim 1, characterized in that, The variance-driven dynamic fusion control mechanism uses the variance of the global features of the extracted CT image as the control signal to measure the structural complexity and spatial dispersion of the CT image. When the variance is larger, the measured CT image has several active regions, and the Top-K branch is strengthened to focus the CT image. When the variance is smaller, the measured CT image has a concentrated structure.

3. The medical image lesion recognition method based on a dual-path attention mechanism as described in claim 1, characterized in that, In the process of extracting global features of the acquired CT image based on the global average attention mechanism, the global average attention mechanism is used as the starting point for lightweight modeling. Average pooling operation is performed on the CT image to ignore the spatial differences between various locations in the CT image and obtain unified global features.

4. The medical image lesion recognition method based on a dual-path attention mechanism as described in claim 1, characterized in that, The learnable position-weighted global attention mechanism uses a trainable position weight vector to model position importance. During training, it adaptively learns the attention level of different positions, enhances the spatial selection of global attention, and realizes global perception and convergence of learnable spatial weights.

5. The medical image lesion recognition method based on a dual-path attention mechanism as described in claim 1, characterized in that, Before introducing the soft Top-K attention mechanism, a hard Top-K attention mechanism is introduced on the global attention path of the obtained CT image. A sparse matrix of attention score is constructed, and the resulting sparse matrix is ​​Softmax normalized to obtain sparse attention weights. The attention output is obtained by combining these sparse attention weights. Specifically, the hard Top-K attention mechanism forces each query to interact only with the top K most relevant keys, improving the model's ability to focus on key regions and significantly reducing redundant computation. The sparsity of the attention score matrix is ​​also addressed. The calculation is defined as follows: ； in, i and j These represent the x and y coordinates of the matrix, respectively. The sparse matrix of the attention score matrix is ​​subjected to Softmax normalization to obtain the sparse attention weights. ,Right now: ； The final attention output is .

6. A medical image lesion recognition system based on a dual-path attention mechanism, characterized in that, include: The acquisition module is configured to acquire the CT image to be identified; The extraction module is configured to extract global features from the acquired CT images based on a global average attention mechanism. The perception module is configured to perform spatial perception of the acquired CT image based on the extracted global features and a learnable location-weighted global attention mechanism, thereby obtaining the global attention path of the CT image. The learnable position-weighted global attention mechanism is used to introduce spatial awareness, replacing the average pooling operation in the original Global Attention branch with Learnable Weighted Pooling to achieve weighted aggregation of Key and Value; the calculation process is as follows: ； ； ； ； in, represents the learnable unknown parameters; softmax represents the activation function, such that w is distributed between 0 and 1; This means that w becomes a value between 0 and 1 after being normalized by the Sigmoid function; This represents the globally weighted key; K i V represents the key vector at the i-th position; i The Value vector representing the i-th position; d represents the globally weighted Value; d represents the scaling factor of the feature dimension, i.e., the dimension factor used for normalization in attention. A sparse module is configured to introduce a soft Top-K attention mechanism onto the obtained global attention path to obtain sparse attention results for the CT image; the calculation process of the soft Top-K attention mechanism includes: Introducing a temperature factor γ to scale the original attention score matrix A generates a smooth sparse mask: ； Among them, a smaller γ value amplifies the weight differences of highly relevant tokens, making the attention distribution sparser; at the same time, Softmax keeps the function continuous and differentiable, avoiding the gradient interruption problem caused by hard truncation. Compare the original attention matrix A with the mask Element-wise multiplication to achieve flexible Top-K weighting: ； The sparsified attention weights are used in the standard weighted operation to output the final attention result: ； in, V Represents a value vector; The recognition module is configured to classify and detect lesions in CT images based on the obtained sparse attention results and a variance-driven dynamic fusion control mechanism. This achieves lesion recognition in medical images based on a dual-path attention mechanism. Specifically, it introduces global feature variance as a control signal to measure the structural complexity and spatial dispersion of the input image; the variance-driven dynamic fusion control mechanism dynamically calculates the fusion weights and implements a weighted output of the local-global paths. ； Wherein, the fusion coefficient α(x) is ;in, This represents the global feature variance of the current image's channels. It is a learnable linear transformation function. For the Sigmoid activation function, ensure ; This represents the output of introducing sparse Top-K local paths in a two-branch configuration. This represents the output of the sparse attention mechanism in the two-branch approach.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the medical image lesion recognition method based on the dual-path attention mechanism as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the medical image lesion recognition method based on dual-path attention mechanism as described in any one of claims 1-5.

9. A computer program product, comprising software code, characterized in that, The program in the software code performs the steps of the medical image lesion recognition method based on the dual-path attention mechanism as described in any one of claims 1-5.

Images