Medical image focus identification method and system based on double-path attention mechanism

Abstract

The invention belongs to the technical field of medical image segmentation and recognition, and particularly relates to a medical image lesion recognition method and system based on a double-path attention mechanism, and the method comprises the steps: obtaining a to-be-recognized CT image; extracting global features of the obtained CT image based on a global average attention mechanism; spatial perception of the obtained CT image is carried out according to the extracted global features and a global attention mechanism capable of learning position weighting, and a global attention path of the CT image is obtained; introducing a soft Top-K attention mechanism on the obtained global attention path to obtain a sparse attention result of the CT image; according to the obtained sparse attention result and a dynamic fusion control mechanism based on variance driving, lesion classification detection of the CT image is carried out, and medical image lesion recognition based on a double-path attention mechanism is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation and recognition, and specifically relates to a medical image lesion recognition method and system based on a dual-path attention mechanism. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of medical artificial intelligence, building deep learning models that combine computational efficiency, discriminative accuracy, and interpretability has become a core issue in the field of medical image analysis. In the context of precision medicine, predicting key molecular markers based on medical images is gradually becoming an important technical path for achieving non-invasive diagnosis and personalized treatment. Taking the KRAS gene mutation in non-small cell lung cancer (NSCLC) as an example, this mutation directly affects the formulation of targeted treatment strategies and has significant clinical guidance significance. However, currently commonly used tissue or liquid biopsy methods have drawbacks such as high invasiveness, long cycle times, and high costs. They cannot meet the clinical requirements for real-time performance and wide adaptability. Non-invasive lesion recognition and prediction methods based on conventional medical images are urgently needed.

[0004] In recent years, the Transformer architecture has been widely used in medical imaging (covering disease classification, tumor detection, organ segmentation, and other areas) due to its excellent global modeling capabilities. However, medical images are typically characterized by high resolution, sparse lesions, and heterogeneous structural distribution. This poses two challenges to the standard Transformer's dense attention mechanism in practical applications: First, its O(N²) computational complexity limits its deployment efficiency in high-resolution images; second, attention is easily distracted, making it difficult to accurately focus on key lesion areas, which in turn affects the model's discriminative ability and clinical interpretability, making it difficult to meet the dual clinical requirements of efficiency and robustness.

[0005] Based on this, the recognition model can be optimized through sparse attention, lightweight network structure or dynamic mechanism. However, model optimization is generally built on natural images or public medical datasets, lacking real clinical data verification. The sparse strategy of the model is mostly static design, and local and global branches are often separated, making it difficult to achieve effective semantic collaboration. Summary of the Invention

[0006] To solve the above problems, the present invention proposes a medical image lesion recognition method and system based on a dual-path attention mechanism. It adopts a variance-driven dual-path attention mechanism. On the basis of the standard Transformer framework, it introduces a sparse Top-K local path and a learnable global path. Based on the dynamic fusion control mechanism of global feature variance, it realizes adaptive modeling of images with different structures and adaptive integration of local and global features, thereby significantly improving the model's discrimination ability and spatial focusing ability without increasing the amount of computation.

[0007] According to some embodiments, a first solution of the present invention provides a medical image lesion recognition method based on a dual-path attention mechanism, which adopts the following technical solutions: A medical image lesion recognition method based on a dual-path attention mechanism, comprising: Acquire a CT image to be identified; Extract global features of the acquired CT images based on the global average attention mechanism; Perform spatial perception of the acquired CT image based on the extracted global features and the learnable position-weighted global attention mechanism to obtain the global attention path of the CT image; A soft Top-K attention mechanism is introduced on the obtained global attention path to obtain sparse attention results of CT images; According to the obtained sparse attention results and the variance-driven dynamic fusion control mechanism, CT image lesion classification detection is performed to realize the recognition of medical image lesions based on the dual-path attention mechanism.

[0008] As a further technical limitation, the variance-driven dynamic fusion control mechanism uses the variance of the extracted global features of the CT image as a control signal to measure the structural complexity and spatial discreteness of the CT image; when the variance is larger, there are several active areas in the measured CT image, and the CT image is focused by strengthening the Top-K branch; when the variance is smaller, the structure of the measured CT image is concentrated.

[0009] As a further technical limitation, the variance-driven dynamic fusion control mechanism dynamically calculates the fusion weights of the CT image to obtain the weighted output of the local-global path of the CT image ;Right now ;in, Indicates sparse fusion, that is , Represents the channel global feature variance of the current image, is a learnable linear transformation function, For the Sigmoid activation function, ensure ; Represents the output of the sparse attention mechanism in the dual branch, Represents the output result of introducing sparse Top-K local paths in the dual branch.

[0010] As a further technical limitation, in the process of extracting the global features of the acquired CT image based on the global average attention mechanism, the global average attention mechanism is used as the starting point of lightweight modeling, and an average pooling operation is performed on the CT image, ignoring the spatial differences between various positions of the CT image to obtain a unified global feature.

[0011] As a further technical limitation, the learnable position-weighted global attention mechanism uses a trainable position weight vector to model position importance, adaptively learns the degree of attention of different positions during training, enhances the spatial selection of global attention, and realizes global perception convergence with learnable spatial weights.

[0012] As a further technical limitation, before introducing the soft Top-K attention mechanism, a hard Top-K attention mechanism is introduced on the global attention path of the obtained CT image, a sparse matrix of the attention score matrix is ​​constructed, and the obtained sparse matrix is ​​subjected to Softmax normalization to obtain sparse attention weights, and the obtained sparse attention weights are combined to obtain the attention output.

[0013] According to some embodiments, a second solution of the present invention provides a medical image lesion recognition system based on a dual-path attention mechanism, which adopts the following technical solutions: A medical image lesion recognition system based on a dual-path attention mechanism, comprising: an acquisition module configured to acquire a CT image to be identified; an extraction module configured to extract global features of the acquired CT image based on a global average attention mechanism; a perception module configured to perform spatial perception of the acquired CT image based on the extracted global features and a learnable position-weighted global attention mechanism to obtain a global attention path of the CT image; A sparse module, which is configured to introduce a soft Top-K attention mechanism on the obtained global attention path to obtain a sparse attention result for the CT image; The recognition module is configured to perform lesion classification detection of CT images based on the obtained sparse attention results and a variance-driven dynamic fusion control mechanism, thereby realizing the recognition of medical image lesions based on a dual-path attention mechanism.

[0014] According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of the medical image lesion recognition method based on a dual-path attention mechanism as described in the first embodiment of the present invention.

[0015] According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution: An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the medical image lesion recognition method based on the dual-path attention mechanism as described in the first embodiment of the present invention.

[0016] According to some embodiments, a fifth solution of the present invention provides a computer program product, which adopts the following technical solution: A computer program product includes software code, wherein the program in the software code executes the steps in the medical image lesion recognition method based on the dual-path attention mechanism as described in the first embodiment of the present invention.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This paper adopts a variance-driven dual-path attention mechanism. On the basis of the standard Transformer framework, it introduces a sparse Top-K local path and a learnable global path. Based on the dynamic fusion control mechanism of global feature variance, it realizes adaptive modeling of images with different structures and adaptive integration of local and global features, thereby significantly improving the model's discrimination ability and spatial focusing ability without increasing the amount of computation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings constituting a part of the specification of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments and descriptions of this embodiment are used to explain this embodiment and do not constitute an improper limitation on this embodiment.

[0019] Figure 1 Flowchart of the medical image lesion recognition method based on the dual-path attention mechanism in Example 1 of the present invention; Figure 2 This is an architecture diagram of the dual-path attention mechanism in Example 1 of the present invention; Figure 3 This is an architecture diagram of the global average attention mechanism in Example 1 of the present invention; Figure 4 This is an architecture diagram of Hard Top-K screening in the sparse attention mechanism in Example 1 of the present invention; Figure 5 This is a diagram of the computational architecture of the soft Top-K attention mechanism in Example 1 of the present invention; Figure 6 This is an architectural diagram of the variance-driven dynamic fusion control mechanism in the first embodiment of the present invention; Figure 7 This is an architecture diagram of adaptive Class Token generation based on cosine similarity in Example 1 of the present invention; Figure 8 Schematic diagram of the relationship between AUC and computational complexity of each model in the KRAS mutation prediction task in Example 1 of the present invention; Figure 9 A schematic diagram of the hierarchical visualization of the attention heat map generated by the model at different stages in the first embodiment of the present invention; Figure 10 Schematic diagram showing a visual comparison of confusion matrices of different representative models in Example 1 of the present invention; Figure 11 This is a structural block diagram of the medical image lesion recognition system based on the dual-path attention mechanism in Example 2 of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0022] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0023] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention, and should not be understood as limiting the present invention.

[0024] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0025] Example 1 Embodiment 1 of the present invention introduces a medical image lesion recognition method based on a dual-path attention mechanism.

[0026] like Figure 1 A medical image lesion recognition method based on a dual-path attention mechanism is shown, comprising: Acquire a CT image to be identified; Extract global features of the acquired CT images based on the global average attention mechanism; Perform spatial perception of the acquired CT image based on the extracted global features and the learnable position-weighted global attention mechanism to obtain the global attention path of the CT image; A soft Top-K attention mechanism is introduced on the obtained global attention path to obtain sparse attention results of CT images; According to the obtained sparse attention results and the variance-driven dynamic fusion control mechanism, CT image lesion classification detection is performed to realize the recognition of medical image lesions based on the dual-path attention mechanism.

[0027] To achieve lesion recognition in medical images, this embodiment adopts the variance-driven dual-path attention mechanism (VDPA). With structural interpretability and dynamic modeling capabilities as the core, it jointly constructs a sparse local path and a weighted global path, and introduces a dynamic fusion control mechanism based on global feature variance to achieve adaptive modeling of images with different structures.

[0028] It should be noted that the VDPA module has good modularity and portability, and is suitable for seamless integration with various Transformer backbone structures. Specifically: The VDPA module includes: (1) Learnable Weighted Global Pooling (LWGP), which realizes global perception aggregation with learnable spatial weights; (2) Soft Top-K Attention, which strikes a balance between sparsity and directivity; (3) Variance-Driven Fusion Controller (VDFC), which uses global feature variance to achieve dynamic path fusion; (4) Adaptive Local Pooling (ALP), which constructs context-aware dynamic class tokens to enhance global semantic expression.

[0029] The Transformer architecture has been widely used in natural language processing and computer vision due to its exceptional global modeling capabilities. The Vision Transformer (ViT), a representative model of the visual Transformer, has been applied to multiple medical imaging tasks, such as tumor segmentation, disease classification, and organ recognition. Compared to traditional convolutional neural networks (CNNs), the Transformer is better at capturing long-range feature dependencies and demonstrates a clear advantage in modeling complex spatial structures.

[0030] Because medical images are generally high-resolution, with sparse lesions and uneven spatial distribution, the standard Transformer's dense attention mechanism faces two challenges in practical applications: its O(N²) computational complexity limits its scalability and deployment efficiency for high-resolution inputs; and its easily distracted attention makes it difficult to effectively focus on key lesion areas, affecting the model's diagnostic accuracy and clinical interpretability. Therefore, to meet the requirements of efficient and structurally adaptable modeling for medical imaging tasks, research on improved Transformer architectures is needed.

[0031] To alleviate the computational complexity of dense attention mechanisms in high-resolution image processing, the density of the attention matrix can be effectively reduced by local window restrictions, sliding expansion, or random connections of multiple sparse attention models, reducing the original O(N²) complexity to a linear level.

[0032] In medical image analysis tasks, although some studies have attempted to optimize computational efficiency with the help of sparse attention mechanisms, the static regularization design adopted is difficult to adapt to the sparse and irregular distribution characteristics of lesion areas. In addition, the structural assumptions of natural language processing (such as sliding windows, global token injection, etc.) adopted have limited adaptability in two-dimensional images and cannot be directly migrated. At the same time, the global and local modeling are separated and processed, lacking a collaborative mechanism, resulting in a complex structure and poor module pluggability, making it difficult to integrate into mainstream visual Transformer frameworks (such as ViT, Swin) or actual medical AI systems.

[0033] In response to the need to model sparse heterogeneous structures in medical images, there is an urgent need for a sparse attention mechanism with a simple structure, dynamic adjustability, and global and local collaborative modeling; therefore, this embodiment proposes a medical image lesion recognition method based on a dual-path attention mechanism.

[0034] The dynamic attention mechanism intelligently allocates attention resources according to the complexity of input features to improve the model's focusing ability and discrimination efficiency. However, medical image recognition based on the dynamic attention spine has the following shortcomings: (1) There is a lack of a dynamic sparse mechanism that adapts to the structural sparsity characteristics of medical images, making it difficult to achieve efficient and accurate key area modeling; (2) It relies on specific architecture customization, lacks versatility and portability, and is difficult to deploy across models; (3) Although some lightweight attention strategies can improve reasoning efficiency, they are prone to sacrificing expressiveness and training stability.

[0035] Therefore, this embodiment integrates sparse modeling, adaptive control and structural versatility in design. It can not only efficiently focus on key lesion areas, but also has good interpretability and easy integration, providing a universal, efficient and feasible attention mechanism solution for medical image analysis.

[0036] like Figure 2 The overall architecture diagram shown in the figure, Figure 2 The left side of the figure is the main process, that is, after the input image passes through PatchEmbedding, the local features are extracted by the Adaptive Weighted Neighborhood Aggregation module, and then enters the multi-layer Dynamic Dual-Path Attention Block to achieve the fusion of local and global information. Finally, the KRAS mutation prediction result class is output through the MLP Head. Figure 2 The right side of the figure shows the details of the VDPA single-layer structure, namely Multi-Head QKV Projection, Dynamic Class Token building module (Dynamic Context-Aware Tokenization), Top-K sparse attention branch, Learnable Weighted Pooling global branch, and the fusion mechanism controlled by the Adaptive Fusion Controller, which jointly achieve dynamic coordination between spatial focus and global modeling capabilities.

[0037] The Transformer self-attention mechanism has excellent global modeling capabilities and has become a core component in deep learning. However, its O(N²) computational complexity (N is the number of tokens) leads to significant resource consumption and deployment bottlenecks in high-resolution input scenarios such as medical imaging. Therefore, this embodiment proposes a global average attention mechanism (GAA) as a starting point for lightweight modeling, such as Figure 3As shown in the figure, by performing an average pooling operation on all Keys (K) and Values ​​(V), a unified global representation is generated, thereby avoiding the overhead of calculating the similarity between two tokens in traditional self-attention. Its mathematical expression is as follows: ; ; ; Among them, Q, K, and V represent the query vector, key vector, and value vector respectively; Softmax represents the activation function; and T represents the transpose of the vector matrix.

[0038] The global average attention mechanism proposed in this example significantly reduces computational complexity and is particularly suitable for edge deployments or latency-sensitive medical scenarios. However, because it uses a unified average pooling algorithm to construct a global representation, it ignores the spatial differences between locations, resulting in a uniform attention response. This makes it difficult to effectively focus on key lesion areas, limiting the model's expressiveness in high-precision diagnostic tasks.

[0039] To enhance the spatial selectivity of global attention, this embodiment draws on the idea of ​​learnable position encoding in Vision Transformer (ViT) and introduces a trainable position weight vector to dynamically model the importance of each position. Figure 3 As shown in Figure 1, this embodiment uses learnable position-weighted global attention to introduce spatial perception and replaces the average pooling operation in the original GlobalAttention branch with Learnable Weighted Pooling (LWGP) to achieve weighted aggregation of Key and Value. The calculation process is as follows: ; ; ; ; in, Represents the unknown parameters that can be learned; softmax represents the activation function, which makes w distributed between 0 and 1; Indicates that w becomes a value between 0 and 1 after the normalization function Sigmod; Represents the globally weighted Key representation; K i Represents the Key vector of the i-th position; V i The Value vector representing the i-th position; represents the globally weighted Value representation; d represents the scaling factor of the feature dimension, that is, the dimension factor used for normalization in attention.

[0040] The learnable position-weighted global attention adopted in this embodiment breaks through the assumption of spatial homogeneity of average pooling, allowing the network to adaptively learn the degree of attention of different positions during training, thereby enhancing the spatial perception ability of global modeling; in actual training, the attention map shows a stronger response contrast, indicating that its overall expression ability is enhanced.

[0041] LWGP still has certain limitations in its actual focusing performance. Since the position weight distribution tends to be uniform in the early stages of training, although the model is enhanced overall, it is difficult to highlight key areas, and the phenomenon of "background enhancement obscuring lesions" is prone to occur. This in turn limits its fine discrimination ability in medical image tasks where lesions are sparse and local differences are significant. Although Learnable Weighted Pooling effectively improves global perception strength, its spatial selection ability is still insufficient, suggesting that the introduction of sparse attention with an explicit selection mechanism will become a key direction for optimization in the next stage.

[0042] To this end, this embodiment adopts a hard Top-K attention strategy to enforce focused sparseness. The lesion areas in medical images typically exhibit spatially sparse and semantically critical distribution characteristics. The traditional global attention mechanism inevitably introduces a large amount of invalid background information when calculating the correlation between all tokens, which not only distracts the model's attention but also brings high computational overhead of O(N²).

[0043] like Figure 4 As shown in the figure, this embodiment introduces a hard Top-K sparse mechanism (Hard Top-KAttention) on the global attention path, forcing each query to interact only with the top K most relevant keys, thereby improving the model's ability to focus on key areas and significantly reducing redundant calculations. The sparsification of the attention score matrix The calculation is defined as follows: ; in, i and j They represent the horizontal and vertical coordinates of the matrix respectively.

[0044] Perform Softmax normalization on the sparse matrix to obtain sparse attention weights ,Right now: ; The final attention output is ; This embodiment reduces the attention complexity from O(N²) to O(N·K) through a hard Top-K sparsity mechanism, significantly reducing invalid computations and strengthening the spatial focus of the model, which is particularly suitable for medical imaging tasks dominated by "critical few" features. However, the Top-K operation is essentially a non-differentiable discrete choice and cannot participate in gradient backpropagation, which often leads to training instability and convergence difficulties, becoming a major bottleneck in performance optimization. To this end, a differentiable sparsity mechanism can be introduced to achieve a balance between training friendliness and focus performance. In order to solve the problem of the non-differentiability of the hard Top-K attention mechanism during training, existing studies have introduced temperature factors and smoothing functions to construct continuous sparse masks, thereby achieving differentiable sparsification of the attention matrix; however, these methods are mostly derived from natural language or general image tasks, and it is difficult to directly adapt to the structural sparseness and strong focus characteristics of medical images. To this end, this embodiment proposes a soft Top-K attention mechanism (Soft Top-K Attention) with a simple structure, training-friendly, and more suitable for medical images, to balance sparsity and optimization stability; such as Figure 5 As shown, the calculation process includes: (1) Introducing the temperature factor γ, the original attention score matrix A is scaled to generate a smooth sparse mask: ; A smaller γ value can amplify the weight differences of highly correlated tokens, making the attention distribution more sparse. At the same time, Softmax keeps the function continuous and differentiable, avoiding the gradient interruption problem caused by hard truncation.

[0045] (2) Combine the original attention matrix A with the mask Element-wise multiplication (Hadamard product ⊙) to achieve flexible Top-K weighting: ; (3) Use the sparse attention weights to participate in the standard weighted operation and output the final attention result: ; The soft Top-K attention mechanism in this embodiment effectively improves the model's responsiveness to key areas without increasing structural complexity, alleviates the problem of training instability, and can be used to meet the needs of sparse attention for high-resolution input of medical images.

[0046] Key lesion regions in medical images are typically spatially sparse, small in size, but semantically significant; whereas background regions contain extensive contextual information but are also redundant. The lesion recognition model in this embodiment must possess both local focusing and global modeling capabilities to improve the integrity of lesion diagnosis.

[0047] To achieve collaborative modeling of local and global attention paths, this embodiment introduces a static fusion strategy based on fixed weights. This strategy is widely used in multi-path attention mechanisms and has demonstrated effectiveness in multimodal fusion tasks. It is simple to implement and has controllable computational overhead. As a baseline, it can effectively integrate local details and global semantic information and is a common method in current multi-path attention fusion.

[0048] This embodiment integrates the Top-K attention and global attention outputs in a linearly weighted manner during the fusion process: ; This embodiment lacks adaptability between different image structures. However, the fusion weight α is a static hyperparameter. When faced with samples with strong structural heterogeneity and significant lesion differences, it is easy to cause problems such as attention drift, information redundancy, or insufficient key response. To this end, this embodiment introduces a feature variance-driven dynamic fusion mechanism (VDFC) to improve the flexibility of attention fusion and the stability of discrimination.

[0049] Although the static fusion strategy can initially coordinate local and global attention, its fixed weighting method is difficult to adapt to medical image samples with great structural differences; for example, in CT images of different patients, the lesion area may be densely distributed or extremely sparse, and the background texture complexity also varies significantly. At this time, a unified fusion ratio is likely to lead to attention shift, information redundancy or modeling imbalance. In order to enhance the model's adaptability to structural diversity, this embodiment proposes the following Figure 6 The variance-driven dynamic fusion control mechanism (VDFC) shown in the figure introduces global feature variance as a control signal to measure the structural complexity and spatial discreteness of the input image. The larger the variance, the more active areas there are in the image, and the model should tend to strengthen the Top-K branch to achieve fine focus. When the variance is smaller, the structure is more concentrated, which is more suitable for relying on global path modeling for semantic consistency.

[0050] Dynamically calculate fusion weights based on VDFC and implement weighted output of local-global paths: ; Among them, the fusion coefficient α(x) is ;in, Represents the channel global feature variance of the current image, is a learnable linear transformation function, For the Sigmoid activation function, ensure .

[0051] The variance-driven dynamic fusion control mechanism in this embodiment does not require manual setting of the fusion ratio. It can automatically learn the optimal weight configuration under various structural conditions during training, realizing truly input-aware path regulation. When processing medical images with complex structures and strong lesion heterogeneity, VDFC can significantly improve the model's discrimination stability and regional focusing ability, and has good versatility and clinical adaptability.

[0052] To address the problem that fixed class tokens are unable to adapt to structurally heterogeneous medical images, this embodiment introduces an adaptive local pooling mechanism (ALP) to construct context-aware classification tags. Referring to strategies such as TokenLearner, and taking into account the "sparseness and semantic concentration" characteristics of medical images, a dynamic aggregation process based on similarity weighting is designed. Specifically: Patch Token sequence for a given input image , ALP first constructs a local neighborhood of K nearest neighbors for each token , a normalized weight is calculated based on cosine similarity to measure the spatial or semantic relevance within the neighborhood; it is calculated as follows: ; Based on the above weighted coefficients, local information of the neighborhood is aggregated, and dynamic ClassToken is constructed through global average pooling to form the final context-aware representation. , its calculation expression is as follows: ; like Figure 7 As shown in the figure, the adaptive local pooling mechanism in this embodiment has the advantages of dynamically adjusting the token expression range, enhancing structural sensitivity, and improving the quality of semantic compression; the finally generated Class Token is sent to the classification head to participate in the discrimination task, forming a three-branch structure with the Top-K sparse path and the global weighted path in VDPA, realizing comprehensive modeling of the feature space.

[0053] The data used in this example were collected from 166 CT image samples of non-small cell lung cancer (NSCLC) patients. Among them, 84 were KRAS mutation-positive and 82 were KRAS mutation-negative. The patient age range was 30–88 years old, with a mean age of 63 years (±10). In terms of population composition, the proportion of females in the KRAS mutation group was significantly higher (77.4%), and the proportion of smokers and drinkers was also relatively high. The complete demographic characteristics are shown in Table 1.

[0054] In terms of pathological and clinical characteristics, the KRAS mutation group was mainly characterized by adenocarcinoma (95.2%), with some differences in hypertension, diabetes, and tumor stage. The differences in tumors are shown in Table 2.

[0055] Table 1 Demographic characteristics

[0056] Table 2 Tumor characteristics

[0057] In this example, all experiments were run on a single NVIDIA RTX 4090 GPU. The optimizer used was SGD with momentum (in this example, the momentum coefficient was 0.9 and the weight decay was 5e-5). The initial learning rate was set to 0.001, dynamically adjusted using a combination of cosine annealing and LambdaLR strategies. The total number of training epochs was set to 300, with a batch size of 128. To mitigate overfitting, dropout (ratio 0.2) and data augmentation strategies including random horizontal flips and random rotations of ±10° were introduced during training. An early stopping strategy was introduced, terminating training early if the validation set AUC metric did not significantly improve within 60 consecutive epochs. Model parameter initialization used the Kaiming Uniform method, and some pre-trained layers were frozen at the beginning of training to accelerate convergence and improve stability.

[0058] CT image data was annotated by professional radiologists. For each patient, three consecutive single-channel slices were selected near the tumor area and concatenated into a three-channel image. All images were scaled to a 512×512 resolution and normalized to accommodate the input format and numerical stability requirements of the deep learning model.

[0059] The model evaluation in this embodiment uses two core metrics: AUC (area under the curve) and ACC (accuracy). F1-Score and Precision are also introduced to provide a more comprehensive quantitative analysis of model classification performance. AUC, as an important indicator of the model's overall discriminative ability, is particularly representative in scenarios with inter-class imbalance in medical images.

[0060] To understand the performance of different attention mechanisms in spatial focusing and semantic recognition, this embodiment introduces a multi-level attention heat map visualization method to display the response characteristics and spatial coverage capabilities of the lesion area at different stages.

[0061] To comprehensively evaluate the proposed attention mechanism's trade-off between performance and computational efficiency, this example constructed eight phased models (as shown in Table 3) based on standard multi-head self-attention (MHSA). These models gradually introduced lightweight strategies (GAA and HTK), global enhancement mechanisms (LGA and STK), static fusion mechanisms (STK-LGA), and dynamic fusion and semantic perception modules (VDPA with and without ALP), ultimately forming a complete VDPA model. Except for the comparison of models with and without ALP, all other compared models were built on the same basic architecture that included ALP to ensure consistency in evaluation. In the table, F1 is the harmonic mean of precision and recall.

[0062] Table 3 Eight stage models

[0063] The relationship between AUC and computational complexity (GFLOPs) of each model in the KRAS mutation prediction task is as follows: Figure 8 As shown, we get: Group A (GAA → LGA): With the same computational effort, the AUC increased from 0.765 to 0.810, verifying the enhanced effect of the global weighting mechanism LGA in information integration. Group B (HTK → STK): Soft Top-K attention improves training stability and sparse representation quality compared to hard filtering, with AUC increased to 0.834. Group C (STK-LGA → VDPA w / o ALP → VDPA with ALP): While maintaining essentially unchanged FLOPs, the model performance continued to improve by gradually introducing dynamic fusion and adaptive pooling mechanisms. VDPA improved AUC by +0.044 compared to STK-LGA and +0.021 compared to the version without ALP, demonstrating that ALP has a significant positive effect on semantic compression and context fusion.

[0064] It should be noted that the model VDPA (with ALP) significantly outperforms MHSA (AUC 0.792) in terms of AUC (0.932) and F1 (0.893), while the FLOPs are basically the same, which fully demonstrates the overall advantages of this embodiment in terms of discriminative performance, computational efficiency, and structural versatility.

[0065] In order to deeply understand the differences between different attention mechanisms in terms of spatial focus and semantic expression, this embodiment analyzes the attention heat maps generated by the models at each stage as follows: Figure 9The layered visualization shown, specifically: HSA (Stage 1): Attention weights are evenly distributed and lack obvious spatial focus; GAA and LGA (Stage 2–3): After the introduction of global weighting, the model initially becomes spatially selective, but the response is still relatively smooth; HTK and STK (Stage 4–5): The sparse mechanism enhances the response concentration in the key area; STK-LGA (Stage 6): effectively integrates local sparsity with global position awareness, resulting in more stable structural focus. VDPA (with vs. without ALP): The mid-layer heatmaps of the two methods do not change significantly, but ALP brings improvements of AUC +0.021 and F1 +0.019, indicating that its compression and expression enhancement effects at the semantic level are more significant.

[0066] In the NSCLC KRAS mutation prediction task, this example compared and evaluated VDPA with various mainstream CNN and Transformer models. The evaluation results are shown in Table 4. All models used the same data processing and training strategies to ensure fair comparison.

[0067] Table 4 Performance comparison of different models in the KRAS mutation prediction task

[0068] Experimental results show that VDPA leads in all four core indicators: AUC (0.932), ACC (0.880), F1 (0.893) and Precision (0.872), outperforming representative architectures such as ResNet50 (AUC=0.719), EfficientNet-B0 (AUC=0.830) and Swin-Tiny (AUC=0.875), with a relative performance improvement of 4.8%–11.2%.

[0069] It's important to note that VDPA's performance improvement isn't due to scaling models, but rather stems from the effective synergy of structural innovations and attentional pathway control strategies, including a Top-K sparsity mechanism, a variance-driven dynamic fusion controller, and contextual class tokens constructed using Adaptive Local Pooling. These modules collectively enhance the model's discriminative expression capabilities, spatial focus, and training robustness, making it highly adaptable to medical imaging tasks.

[0070] To validate VDPA's structural adaptability and cross-modal generalization across different tasks, this example conducted transfer testing on a publicly available Chest X-ray pneumonia classification dataset (5,856 chest X-ray images collected by a medical center). The dataset covers both normal and pneumonia categories, with images uniformly acquired using the PA position. The patient age ranged from 1 to 5 years old. All experiments were conducted using a strict patient-based training and test set to ensure no data leakage.

[0071] Using the same data preprocessing and training configuration, VDPA was compared and evaluated against various mainstream CNN models (such as ResNet, MobileNet, and EfficientNet) and Transformer models (such as ViT and Swin). Comparison metrics included AUC, ACC, F1, Precision, and NORMAL class recognition rate, which comprehensively measure the model's discriminative performance and class balance. The results are shown in Table 5.

[0072] Table 5 Comparison of the performance of each model in the pneumonia binary classification task and the recognition ability of the NORMAL class

[0073] As shown in Table 5, the VDPA model achieved the best overall performance: AUC reached 0.9570, F1 value was 0.9060, Precision was 0.8530, and the NORMAL class recognition rate was 0.72. Figure 10 As shown in the figure, although ViT-Base has a high recall rate on the PNEUMONIA class, its recognition rate for the NORMAL class is only 24%, and the misjudgment rate is significantly high, indicating that Transformer has an "over-globalization" bias in fine-grained structure modeling, making it difficult to distinguish normal tissues; CNN models such as ResNet50 have relatively more balanced recognition performance on the NORMAL class (71% and 77%), indicating that convolutional structures have certain advantages in local modeling and boundary preservation.

[0074] The VDPA model in this example utilizes a parallel Top-K sparse path and a global attention path, combined with a variance-driven fusion strategy, to effectively perceive key structures and focus on regions in medical images while maintaining manageable computational complexity. Confusion matrix visualization demonstrates that VDPA achieves a more balanced classification accuracy for both sample types (NORMAL: 0.72, PNEUMONIA: 0.97), while maintaining a low false positive rate, demonstrating excellent task robustness and clinical practicality.

[0075] This example conducts evaluation experiments based on real NSCLC CT image data (166 cases in total, including 84 KRAS-positive cases). The results show that the VDPA model achieves leading performance in the KRAS gene mutation prediction task with AUC 0.923, ACC 0.866, F1 0.870, and Precision 0.933, comprehensively surpassing mainstream models such as ResNet, ViT, and Swin. In the ChestX-ray public pneumonia classification task, VDPA also maintained excellent generalization ability and NORMAL class recognition rate, demonstrating good cross-task transferability and structural versatility, verifying its potential for wide application in intelligent diagnosis of medical images; in the real clinical non-small cell lung cancer (NSCLC) KRAS mutation prediction task, VDPA achieved AUC 0.923, ACC 0.866, F1 0.870 and Precision 0.933 on a CT image dataset (166 patients), overall outperforming mainstream models such as ResNet, ViT and Swin. Further generalization experiments conducted on the ChestX-ray public pneumonia classification dataset showed that VDPA also maintained a high recognition rate and good category balance for the NORMAL class, verifying its cross-task transfer capability and structural versatility. While improving the clinical adaptability of the Transformer architecture, VDPA achieved triple optimization of performance, efficiency, and interpretability, providing a feasible new paradigm for building a trustworthy and deployable medical image intelligent analysis system.

[0076] This embodiment adopts a variance-driven dual-path attention mechanism. Based on the standard Transformer framework, it introduces a sparse Top-K local path and a learnable global path. Based on the dynamic fusion control mechanism of global feature variance, it realizes adaptive modeling of images with different structures and adaptive integration of local and global features, thereby significantly improving the model's discrimination ability and spatial focusing ability without increasing the amount of computation.

[0077] Example 2 Embodiment 2 of the present invention introduces a medical image lesion recognition system based on a dual-path attention mechanism.

[0078] like Figure 11 A medical image lesion recognition system based on a dual-path attention mechanism is shown, comprising: an acquisition module configured to acquire a CT image to be identified; an extraction module configured to extract global features of the acquired CT image based on a global average attention mechanism; a perception module configured to perform spatial perception of the acquired CT image based on the extracted global features and a learnable position-weighted global attention mechanism to obtain a global attention path of the CT image; A sparse module, which is configured to introduce a soft Top-K attention mechanism on the obtained global attention path to obtain a sparse attention result for the CT image; The recognition module is configured to perform lesion classification detection of CT images based on the obtained sparse attention results and a variance-driven dynamic fusion control mechanism, thereby realizing the recognition of medical image lesions based on a dual-path attention mechanism.

[0079] The detailed steps are the same as those of the medical image lesion recognition method based on the dual-path attention mechanism provided in Example 1 and will not be repeated here.

[0080] Example 3 A third embodiment of the present invention provides a computer-readable storage medium.

[0081] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of the medical image lesion recognition method based on a dual-path attention mechanism as described in Example 1 of the present invention.

[0082] The detailed steps are the same as those of the medical image lesion recognition method based on the dual-path attention mechanism provided in Example 1 and will not be repeated here.

[0083] Example 4 A fourth embodiment of the present invention provides an electronic device.

[0084] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the medical image lesion recognition method based on the dual-path attention mechanism as described in Example 1 of the present invention.

[0085] The detailed steps are the same as those of the medical image lesion recognition method based on the dual-path attention mechanism provided in Example 1 and will not be repeated here.

[0086] Example 5 A fifth embodiment of the present invention provides a computer program product.

[0087] A computer program product includes software code, wherein the program in the software code executes the steps in the medical image lesion recognition method based on the dual-path attention mechanism as described in Example 1 of the present invention.

[0088] The detailed steps are the same as those of the medical image lesion recognition method based on the dual-path attention mechanism provided in Example 1 and will not be repeated here.

[0089] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk drives, CD-ROMs, optical storage devices, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention may be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0090] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0093] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0094] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

[0095] The above description is merely a preferred embodiment of this embodiment and is not intended to limit this embodiment. Those skilled in the art will readily appreciate that this embodiment may be modified and varied in various ways. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this embodiment shall be within the scope of protection of this embodiment.

Claims

1. A medical image lesion recognition method based on a dual-path attention mechanism, characterized in that: include: Acquire a CT image to be identified; Extract global features of the acquired CT images based on the global average attention mechanism; Perform spatial perception of the acquired CT image based on the extracted global features and the learnable position-weighted global attention mechanism to obtain the global attention path of the CT image; A soft Top-K attention mechanism is introduced on the obtained global attention path to obtain sparse attention results of CT images; According to the obtained sparse attention results and the variance-driven dynamic fusion control mechanism, CT image lesion classification detection is performed to realize the recognition of medical image lesions based on the dual-path attention mechanism.

2. A medical image lesion recognition method based on a dual-path attention mechanism as claimed in claim 1, characterized in that: The variance-driven dynamic fusion control mechanism uses the variance of the extracted global features of the CT image as a control signal to measure the structural complexity and spatial discreteness of the CT image; when the variance is larger, the measured CT image has several active areas, and the CT image is focused by strengthening the Top-K branch; when the variance is smaller, the structure of the measured CT image is concentrated.

3. A medical image lesion recognition method based on a dual-path attention mechanism as claimed in claim 1, characterized in that: The variance-driven dynamic fusion control mechanism dynamically calculates the fusion weights of the CT image to obtain the weighted output of the local-global path of the CT image. ;Right now ;in, Indicates sparse fusion, that is , Represents the channel global feature variance of the current image, is a learnable linear transformation function, For the Sigmoid activation function, ensure ; Represents the output of the sparse attention mechanism in the dual branch, Represents the output result of introducing sparse Top-K local paths in the dual branch.

4. A medical image lesion recognition method based on a dual-path attention mechanism as claimed in claim 1, characterized in that: In the process of extracting the global features of the acquired CT image based on the global average attention mechanism, the global average attention mechanism is used as the starting point of lightweight modeling, and an average pooling operation is performed on the CT image, ignoring the spatial differences between various positions of the CT image to obtain a unified global feature.

5. A medical image lesion recognition method based on a dual-path attention mechanism as claimed in claim 1, characterized in that: The learnable position-weighted global attention mechanism adopts a trainable position weight vector to model position importance, adaptively learns the degree of attention of different positions during training, enhances the spatial selection of global attention, and realizes global perception convergence with learnable spatial weights.

6. A medical image lesion recognition method based on a dual-path attention mechanism as claimed in claim 1, characterized in that: Before introducing the soft Top-K attention mechanism, a hard Top-K attention mechanism is introduced on the global attention path of the obtained CT image to construct a sparse matrix of the attention score matrix. The obtained sparse matrix is ​​subjected to Softmax normalization to obtain sparse attention weights, and the obtained sparse attention weights are combined to obtain the attention output.

7. A medical image lesion recognition system based on a dual-path attention mechanism, characterized in that: include: an acquisition module configured to acquire a CT image to be identified; an extraction module configured to extract global features of the acquired CT image based on a global average attention mechanism; a perception module configured to perform spatial perception of the acquired CT image based on the extracted global features and a learnable position-weighted global attention mechanism to obtain a global attention path of the CT image; A sparse module, which is configured to introduce a soft Top-K attention mechanism on the obtained global attention path to obtain a sparse attention result for the CT image; The recognition module is configured to perform lesion classification detection of CT images based on the obtained sparse attention results and a variance-driven dynamic fusion control mechanism, thereby realizing the recognition of medical image lesions based on a dual-path attention mechanism.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the medical image lesion recognition method based on the dual-path attention mechanism as described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the medical image lesion recognition method based on the dual-path attention mechanism are implemented as described in any one of claims 1-6.

10. A computer program product comprising software code, characterized in that The program in the software code executes the steps of the medical image lesion recognition method based on the dual-path attention mechanism as described in any one of claims 1-6.

Images