Medical image recognition method and device based on prototype guide context fusion

Abstract

The invention provides a medical image recognition method and device based on prototype guide context fusion, and relates to the technical field of medical health, and the method comprises the steps: inputting a medical image into a medical image recognition model, and obtaining a recognition result, for a target disease, outputted by the medical image recognition model; the medical image recognition model adopts an encoder-decoder architecture; the medical image recognition model is used for performing multi-stage feature extraction on a medical image and introducing a feature bank layer in a down-sampling process of each stage; the feature bank layer is configured to maintain a dynamically updated clustering center set as a feature prototype of various diseases. According to the medical image recognition method and device based on prototype guided context fusion provided by the invention, feature extraction is performed based on the improved model, the performance and robustness of the model in a complex task are remarkably improved, and organic combination of depth and breadth is realized while the calculation efficiency is maintained.

Description

Technical Field

[0001] This application relates to the field of medical and health technology, and in particular to a medical image recognition method and device based on prototype-guided context fusion. Background Technology

[0002] With the development of artificial intelligence (AI) technology, it is increasingly being applied to the medical field to help reduce costs and increase efficiency, improve diagnosis and treatment levels, enhance patient experience, and reduce the risk of disease. For example, using deep learning models to analyze medical images provides a reference for disease diagnosis, greatly improving the accuracy of disease diagnosis.

[0003] In related technologies, medical image analysis models face multi-dimensional technical challenges when dealing with complex anatomical structures, and dynamic perception mechanisms have significant limitations in capturing subtle tissue contrasts. Taking computed tomography (CT) images as an example, the density difference between the target area and surrounding tissues is often less than 50 Hounsfield units, compounded by the influence of partial volume effects, which places higher demands on the model's multi-scale feature extraction capabilities. Summary of the Invention

[0004] The purpose of this application is to provide a medical image recognition method and device based on prototype-guided context fusion. It extracts features based on an improved Swing Transformer model, which significantly improves the model's performance and robustness in complex tasks, achieving an organic combination of depth and breadth while maintaining computational efficiency.

[0005] This application provides a medical image recognition method based on prototype-guided context fusion, including: The process involves: acquiring a medical image to be identified; inputting the medical image into a medical image recognition model to obtain the model's output of the identification result for the target disease; wherein the medical image recognition model adopts an encoder-decoder architecture based on the Swin Transformer model; the medical image recognition model is used to: extract features from the medical image through multiple Transformer layers, and introduce a feature bank layer in each downsampling process; the feature bank layer is configured to: maintain a dynamically updated set of cluster centers as feature prototypes for various diseases; the feature bank layer generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map; the decoder is used to employ a soft-mask-guided progressive upsampling mechanism in the upsampling path, through multi-scale attention fusion and multi-path fusion with learnable weights, and attention-guided progressive upsampling.

[0006] Optionally, the step of inputting the medical image into a medical image recognition model to obtain the recognition result for the target disease output by the medical image recognition model includes: extracting features from the medical image to obtain first feature information, and performing multi-level feature extraction and fusion on the first feature information through the multi-level Transformer layer to obtain second feature information; each level of the multi-level Transformer layer contains multiple feature transformation stages, and adjacent Transformer layers reduce spatial resolution and increase the number of channels through a downsampling path to generate a hierarchical feature representation; upsampling the second feature information using a soft mask-guided upsampling mechanism to obtain third feature information that fuses different levels and channels, and generating the recognition result based on the third feature information.

[0007] Optionally, the step of performing multi-level feature extraction and fusion on the first feature information through the multi-level Transformer layers to obtain the second feature information includes: inputting the first feature information into the multi-level Transformer layers for multi-level feature extraction and fusion to obtain the second feature information; wherein each Transformer layer contains at least one VGG block and a parallel multi-branch module; the VGG block includes two consecutive convolution operations; the parallel multi-branch module includes multiple parallel branches with similar structures and different parameters to achieve differentiated feature extraction; each branch includes: initial feature processing through basic convolutional units, expanding the receptive field through convolutions with different dilation rates, enhancing the inter-channel dependencies through a channel attention mechanism, enriching the feature representation using convolutional kernels of different sizes, improving edge perception through a Laplacian edge enhancement layer, and performing final feature extraction through basic convolutional units.

[0008] Optionally, the deep feature extraction based on the feature bank layer in the downsampling path includes: dividing the feature map to be downsampled into multiple regions and calculating the importance index of each region; dividing the multiple regions into key regions and non-key regions based on a preset importance threshold and the importance index of each region; expanding the region-level mask corresponding to the key region into a pixel-level mask, and using the pixel-level mask of the key region to perform selective feature enhancement processing on the key regions and non-key regions in the feature map; wherein, the selective feature enhancement processing includes: applying complete feature bank processing to the key regions, and maintaining the original features or performing dimensionality reduction processing on the non-key regions.

[0009] Optionally, the step of using a soft mask-guided upsampling mechanism to upsample the second feature information to obtain a third feature information fused from different levels and channels includes: performing progressive upsampling on the second feature information, and fusing the result of the progressive upsampling with the result of the direct upsampling through learnable weights via multi-path fusion to obtain the third feature information; wherein the progressive upsampling process includes at least two consecutive 2x upsampling stages, and a feature refinement operation is performed after each upsampling stage; during the progressive upsampling process, a dynamic adaptive threshold filtering strategy is used to generate a soft mask to guide feature fusion and suppress background noise; the feature refinement operation includes: refining and enhancing the features so that the features have higher resolution while restoring and strengthening semantic information and structural details.

[0010] Optionally, generating the recognition result based on the third feature information includes: determining, based on the third feature information, an image region in the medical image that is similar to the medical image features of the target disease.

[0011] This application also provides a medical image recognition device based on prototype-guided context fusion, comprising: The system comprises an acquisition module for acquiring medical images to be identified, and an identification module for inputting the medical images into a medical image recognition model to obtain the identification results for the target disease output by the medical image recognition model. The medical image recognition model employs an encoder-decoder architecture based on the Swin Transformer model. The model extracts features from the medical images through multiple Transformer layers, introducing a feature bank layer at each downsampling stage. The feature bank layer is configured to maintain a dynamically updated set of cluster centers as feature prototypes for various diseases. It generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map. The decoder employs a soft-mask-guided progressive upsampling mechanism in the upsampling path, using multi-scale attention fusion and multi-path fusion with learnable weights, along with attention-guided progressive upsampling.

[0012] Optionally, the recognition module is specifically used to extract features from the medical image to obtain first feature information, and to perform multi-level feature extraction and fusion on the first feature information through the multi-level Transformer layer to obtain second feature information; each level of the multi-level Transformer layer contains multiple feature transformation stages, and adjacent Transformer layers reduce spatial resolution and increase the number of channels through a downsampling path to generate a hierarchical feature representation; the recognition module is further used to upsample the second feature information using a soft mask-guided upsampling mechanism to obtain third feature information that fuses different levels and channels, and to generate the recognition result based on the third feature information.

[0013] Optionally, the recognition module is specifically used to input the first feature information into the multi-level feature extraction and fusion in the multi-level Transformer layer to obtain the second feature information; wherein, each level of Transformer layer contains at least one VGG block and a parallel multi-branch module; the VGG block includes two consecutive convolution operations; the parallel multi-branch module includes multiple parallel branches with similar structures and different parameters to achieve differentiated feature extraction; each branch includes: initial feature processing through basic convolutional units, expanding the receptive field through convolutions with different dilation rates, enhancing the inter-channel dependency through a channel attention mechanism, enriching the feature representation using convolutional kernels of different sizes, improving edge perception through a Laplacian edge enhancement layer, and performing final feature extraction through basic convolutional units.

[0014] Optionally, the recognition module is specifically used to divide the feature map to be downsampled into multiple regions and calculate the importance index of each region; the recognition module is further used to divide the multiple regions into key regions and non-key regions based on a preset importance threshold and the importance index of each region; the recognition module is further used to expand the region-level mask corresponding to the key region into a pixel-level mask, and use the pixel-level mask of the key region to perform selective feature enhancement processing on the key regions and non-key regions in the feature map; wherein, the selective feature enhancement processing includes: applying complete feature bank processing to the key regions, and maintaining the original features or performing dimensionality reduction processing on the non-key regions.

[0015] Optionally, the recognition module is specifically used to perform progressive upsampling processing on the second feature information, and to fuse the result of the progressive upsampling with the result of the direct upsampling through a multi-path fusion using learnable weights to obtain the third feature information; wherein, the progressive upsampling processing includes at least two consecutive 2x upsampling stages, and a feature refinement operation is performed after each upsampling stage; during the progressive upsampling process, a dynamic adaptive threshold filtering strategy is used to generate a soft mask to guide feature fusion and suppress background noise; the feature refinement operation includes: refining and enhancing the features so that the features have higher resolution while restoring and strengthening semantic information and structural details.

[0016] Optionally, the recognition module is specifically used to determine, based on the third feature information, an image region in the medical image that is similar to the medical image features of the target disease.

[0017] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the medical image recognition method based on prototype-guided context fusion as described above.

[0018] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the medical image recognition method based on prototype-guided context fusion as described above.

[0019] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the medical image recognition method based on prototype-guided context fusion as described above.

[0020] The medical image recognition method and apparatus based on prototype-guided context fusion provided in this application first acquire a medical image to be recognized; then, the medical image is input into a medical image recognition model to obtain the recognition result for the target disease output by the medical image recognition model; wherein, the medical image recognition model adopts an encoder-decoder architecture based on the SwinTransformer model; the medical image recognition model is used to: extract features from the medical image through multi-level Transformer layers, and introduce a feature bank layer in each downsampling process; the feature bank layer is configured to: maintain a dynamically updated set of cluster centers as feature prototypes for various diseases; the feature bank layer generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map; the decoder is used to employ a soft-mask-guided progressive upsampling mechanism in the upsampling path, through multi-scale attention fusion and multi-path fusion with learnable weights and attention-guided progressive upsampling. Thus, feature extraction based on the improved Swing Transformer model significantly enhances the model's performance and robustness in complex tasks, achieving an organic combination of depth and breadth while maintaining computational efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the overall architecture of the medical image recognition model provided in this application; Figure 2 This is a flowchart illustrating the medical image recognition method based on prototype-guided context fusion provided in this application; Figure 3 This is a schematic diagram of the feature enhancement process that integrates multi-scale saliency contextual information provided in this application; Figure 4 This is a schematic diagram of the self-supervised feature alignment process based on prototype clustering supervision provided in this application; Figure 5 This is a schematic diagram of the incremental dynamic weight optimization process based on mask-guided constraints provided in this application; Figure 6 This is a schematic diagram of the medical image recognition device based on prototype-guided context fusion provided in this application; Figure 7This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] The technical solutions in the related technologies have exposure adaptability defects when dealing with dynamic imaging conditions. When faced with tissue deformation rates exceeding 15% or motion artifacts larger than 2mm, the stability of the model segmentation results is significantly reduced due to the lack of a dynamic feature fusion mechanism, with the standard deviation of the Dice coefficient reaching ±4.7%. This volatility may pose clinical decision risks in intraoperative navigation scenarios that require real-time feedback.

[0026] At the geometric constraint modeling level, traditional attention mechanisms struggle to balance detail response and computational efficiency. Under low signal-to-noise ratio conditions, conventional attention modules show a 22% decrease in detection rate for lesions smaller than 3mm compared to optimized convolution operators, with boundary structure similarity remaining at only 0.82. Although recent research has improved the IoU index to 0.743 using a boundary-aware module, the resulting 150 GFLOPs computational complexity leads to a single-frame processing latency exceeding 800ms, significantly deviating from the 200ms response threshold for real-time surgical navigation. This efficiency-accuracy contradiction is even more pronounced in mobile imaging devices, while the sub-millimeter-level structural deformation errors caused by conventional interpolation algorithms also exceed clinically acceptable standards.

[0027] In terms of feature representation learning, it exhibits three limitations in dynamic imaging sequences: static prototype clustering is difficult to adapt to signal intensity change rates exceeding 15% / s, causing the intra-class similarity to plummet from 0.82 to 0.67; the offline update strategy and the dynamic acquisition process with frame intervals within 2 seconds result in a temporal mismatch, causing a 23% increase in semantic confusion; the lack of anatomical topological continuity modeling results in an Euclidean distance of 0.45 between adjacent structures in the feature space, which is far higher than the ideal threshold of 0.2. In the test set containing 17 anatomical variants, the ±7.9% fluctuation in the accuracy of key structure identification reflects the weakening of spatial semantic association.

[0028] To address the aforementioned technical problems in related technologies, embodiments of this application provide a medical image recognition method based on prototype-guided context fusion: such as Figure 1 As shown, this method first processes the input data using a feature extraction module. Then, it extracts features using multiple sets of parallel Transform blocks and combines this with a self-supervised module based on prototype clustering for deep feature learning. Next, it enhances the model's expressive power through a multi-level feature transfer and interaction mechanism, and introduces a dedicated Feature Optimization Module (PMB) in the intermediate layer to strengthen key information. Finally, it effectively integrates feature representations from different levels and channels through a progressive multi-fusion module, fusing comprehensive features to significantly improve the model's performance and robustness in complex tasks. This architecture achieves an organic combination of depth and breadth while maintaining computational efficiency.

[0029] The medical image recognition method based on prototype-guided context fusion provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0030] like Figure 2 As shown in the embodiment of this application, a medical image recognition method based on prototype-guided context fusion is provided. This method may include the following steps 201 to 203: Step 201: Obtain the medical image to be identified.

[0031] For example, the above-mentioned medical images are any of the medical images, including: X-ray photography, ultrasound imaging, radionuclide imaging, computed tomography, computed tomography, magnetic resonance imaging, etc.

[0032] For example, after acquiring a patient's medical images, they can be input into a medical image recognition model for identification. The identification results of the medical image recognition model can be used for the auxiliary diagnosis of a target disease. The target disease can be skin diseases, arterial embolism, pulmonary embolism, heart disease, etc. In this embodiment, vascular embolism is used as an example for description.

[0033] Step 202: Input the medical image into the medical image recognition model to obtain the recognition result of the target disease output by the medical image recognition model.

[0034] The medical image recognition model employs an encoder-decoder architecture based on the Swin Transformer model. This model extracts features from the medical image through multiple Transformer layers, introducing a feature bank layer at each downsampling stage. The feature bank layer maintains a dynamically updated set of cluster centers as feature prototypes for various diseases. It generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map. The decoder uses a soft-mask-guided progressive upsampling mechanism in the upsampling path, employing multi-scale attention fusion and learnable weighted multi-path fusion with attention-guided progressive upsampling.

[0035] For example, a multi-scale gradient parallel feature enhancement module is provided after at least one Transformer stage in the encoder; the multi-scale gradient parallel feature enhancement module includes three parallel branches with similar structures but independent parameters, each branch adopts dilated convolution with different dilation rates and kernel sizes to capture multi-scale contextual information; each branch sequentially performs: basic convolution transformation, channel attention processing, spatial feature enrichment operation, Laplacian edge enhancement and final feature refinement; after all branch outputs are element-wise added and concatenated with channels, fused features are generated through dimensionality reduction convolution and residual connections.

[0036] For example, the selective feature enhancement based on the importance map mentioned above includes: dividing the current feature map into multiple spatial regions; calculating the cosine similarity between each region and a preset disease feature prototype as an importance index; dynamically adjusting the importance threshold based on the energy distribution of the current feature map to divide the regions into key regions and non-key regions; applying the complete feature bank processing flow to the key regions, and maintaining the original features or performing dimensionality reduction and compression on the non-key regions.

[0037] For example, the feature bank is further configured to: maintain a feature queue in the form of a circular buffer for storing sampled features from historical batches; calculate cluster centers using a momentum-updated K-means algorithm based on the feature queue, and reset cluster centers that have not been activated for a long time to ensure the effectiveness and diversity of feature prototypes.

[0038] For example, the soft mask-guided progressive upsampling mechanism includes: generating a multi-scale feature mask based on multi-level features of the decoder, and generating a soft mask by combining dynamic adaptive threshold filtering with a multi-attention mechanism (including channel, spatial and edge attention); adopting a dual-path upsampling strategy: one path achieves progressive recovery through two 2x upsamplings, and the other path preserves the global structure through direct 4x upsampling; the upsampling results from the two paths are weighted and fused through two learnable hybrid weights, which are jointly optimized during network training to output fused features.

[0039] Specifically, step 202 above may also include the following steps 202a and 202b: Step 202a: Extract features from the medical image to obtain first feature information, and then perform multi-level feature extraction and fusion on the first feature information through the multi-level Transformer layer to obtain second feature information.

[0040] Each of the multi-level Transformer layers contains multiple feature transformation stages, and adjacent Transformer layers reduce spatial resolution and increase the number of channels through a downsampling path to generate hierarchical feature representations.

[0041] Specifically, in step 202a above, the step of performing multi-level feature extraction and fusion on the first feature information through the multi-level Transformer layer to obtain the second feature information may further include the following step 202a1: Step 202a1: Input the first feature information into the multi-level feature extraction and fusion of the multi-level Transformer layer to obtain the second feature information.

[0042] Each Transformer layer contains at least one VGG block and one parallel multi-branch module. The VGG block includes two consecutive convolutional operations. The parallel multi-branch module includes multiple parallel branches with similar structures and different parameters to achieve differentiated feature extraction. Each branch includes: initial feature processing through basic convolutional units, expanding the receptive field through convolutions with different dilation rates, enhancing the inter-channel dependencies through a channel attention mechanism, enriching the feature representation using convolutional kernels of different sizes, improving edge perception through a Laplacian edge enhancement layer, and finally extracting features through basic convolutional units.

[0043] Step 202b: Upsample the second feature information using a soft mask-guided upsampling mechanism to obtain a third feature information that integrates different levels and channels, and generate the recognition result based on the third feature information.

[0044] For example, addressing the challenges commonly encountered in current medical image analysis, such as insufficient identification of small-scale lesions, easy neglect of minute targets, and difficulty in accurately capturing local fine structures, this application proposes a feature enhancement strategy that integrates multi-scale saliency contextual information. Specifically, this application innovatively designs a multi-scale gradient parallel feature enhancement method, PMBE. The core of this method lies in constructing three heterogeneous feature extraction branches and introducing a dilated convolution mechanism. By processing feature information at different scales and within the receptive field in parallel, PMBE effectively integrates contextual cues with significant discriminative power, significantly enhancing the network's ability to express features of fine structures and weak-contrast targets, thereby significantly improving the recognition accuracy and robustness of small-scale lesions and local details.

[0045] For example, the overall architecture of the aforementioned medical image recognition model adopts an encoder-decoder framework, achieving efficient visual representation learning through multi-level feature extraction and fusion. In the encoding stage, the input image is first converted into a feature vector by a feature embedding module, and then processed through multiple levels of Swing Transformer layers. Each level contains multiple feature transformation stages, achieving progressive abstraction and compression of features. Between each level, a downsampling layer reduces spatial resolution while increasing the number of channels, forming a hierarchical feature representation. In the decoding stage, a hierarchical upsampling strategy is used to restore the spatial resolution of the features, combined with skip connection features from the encoder, to effectively preserve detailed information. Finally, the features are upsampled to the original input resolution through a final feature expansion module, completing the dense prediction task.

[0046] For example, such as Figure 3 As shown in (a), multi-fusion skip connections are a key design feature for solving the information loss problem in deep networks. Unlike the simple skip connections in traditional U-shaped networks, this architecture achieves multi-level, multi-path feature transfer and fusion. In the encoding stage, the network extracts and retains feature maps from each level; in the subsequent decoding stage, these feature maps are selectively fused across multiple levels. In particular, the network uses multiple VGG blocks (such as...) Figure 3As shown in (b), this multi-level fusion mechanism handles feature fusion at different levels of abstraction, thereby constructing rich skip connection paths. This mechanism significantly improves the network's ability to reconstruct details and effectively alleviates the feature-semantic inconsistency problem that may be caused by directly passing low-level features to high-level semantic layers in traditional skip connections. Each VGG contains two consecutive convolutional operations, and batch normalization and ReLU nonlinear activation functions are executed sequentially after each convolutional layer, forming a standardized feature extraction unit. By adapting to the number of input channels of different VGGs, the network can flexibly process feature information from different network depths. This cascaded design ensures that high-resolution spatial detail information from shallow networks and strong semantic information from deep networks are effectively fused, ultimately generating a more discriminative comprehensive feature representation.

[0047] For example, such as Figure 3 As shown in (C), the Parallel Multi-Branch Module (PMB) comprises three parallel branches, each employing different convolutional parameter configurations to focus on feature extraction at different scales. Although the three branches share a similar sequence of components, their differentiated dilation rates and convolutional kernel sizes enable the capture of multi-scale feature representations. This parallel design allows the network to simultaneously integrate information from different receptive field ranges, significantly improving the model's ability to interpret complex scenes, which can be specifically expressed by the following Equation 1: (Formula 1) in, They are different branches. , It is a branching feature and.

[0048] Specifically, each PMB block contains three structurally similar but parameter-independent parallel branches to achieve differentiated feature extraction. The processing flow of each branch is as follows: First, through the basic convolutional unit (e.g., ... Figure 3 As shown in (d), an initial feature transformation is performed; subsequently, a convolution operation with a specific dilation rate is applied to expand the receptive field; then, a channel attention mechanism is introduced to model the dependencies between channels; next, convolution kernels of different sizes are used to further enrich the feature representation; finally, the edge perception capability is improved through a Laplacian edge enhancement layer, and feature refinement is completed through another basic convolution unit. The Laplacian convolution kernel can effectively extract the second derivative information of the image, thereby highlighting edge and texture details. By integrating Laplacian operator convolution operations to enhance edge features, this design significantly enhances the expression of edge information while preserving the original features, improving the model's perception accuracy of target boundaries. This is crucial for accurate segmentation tasks, and can be specifically represented by the following Equations 2 and 3: (Formula 2) (Formula 3) in, For Laplace convolution kernel, These are input features. It is the output feature.

[0049] Furthermore, the key advantage of the multi-branch parallel architecture lies in its efficient information flow mechanism. Since each branch is directly connected to the input and output, gradients can propagate back along multiple paths, effectively mitigating the gradient vanishing problem in deep networks. Simultaneously, complementary features captured by different branches are effectively integrated through a fusion strategy, generating richer and more robust feature representations. The fusion strategy for the PMB block is as follows: first, element-wise addition is performed on the outputs of all branches; then, all branch outputs are concatenated along the channel dimension and processed by dimensionality-reduced convolution; finally, residual connections are combined to construct multi-level fused features.

[0050] The basic convolutional block contains convolutional layers, batch normalization, and a ReLU activation function. The efficient channel attention mechanism (ECA), as a component of the PMB block, efficiently models inter-channel dependencies through one-dimensional convolution. Compared to traditional channel attention, ECA avoids the dimensionality reduction and restoration process, effectively maintaining the performance of the attention mechanism while significantly reducing the number of parameters. This mechanism first compresses the spatial dimension through global average pooling, then applies one-dimensional convolution to capture local dependencies between channels, and finally generates channel weights through an activation function. This lightweight design makes it suitable for resource-constrained environments. The global pooling formula is shown in Equation 4 below: (Formula 4) in, This represents the output scalar value of channel n. (x, y) is the feature value of the x-th channel of the input feature map at the spatial location (x, y), where x and y represent the height and width dimensions of the feature map, respectively, x is the channel index, and x and y represent the spatial coordinate indices in the vertical and horizontal directions, respectively. This formula compresses the two-dimensional feature map into a channel description vector by traversing all x×y spatial locations and calculating the average value.

[0051] like Figure 3 The spatial attention shown in (e) focuses on enhancing the information selection ability in the spatial dimension. First, two consecutive convolutional layers are used to extract spatial context information. Then, a spatial attention map is generated. Finally, this map is multiplied element-wise with the original feature map to achieve adaptive enhancement of the spatial dimension. The introduction of spatial attention enables the network to dynamically focus on important regions in the input, suppress irrelevant background, and improve the specificity and effectiveness of features.

[0052] The final output of the PMB block needs to undergo spatial attention processing to achieve comprehensive feature enhancement. This combination of channel and spatial dual attention mechanisms ensures that the network can collaboratively optimize feature representations from different dimensions, improving the accuracy and efficiency of visual task processing.

[0053] For example, the overall network design also incorporates various feature fusion and optimization techniques to further improve model performance. Architecturally, the network adds numerous residual connections between consecutive feature processing stages, including short-circuit connections within modules and long-range connections between different modules. These residual paths not only accelerate the network's convergence process but also improve the efficiency and effectiveness of feature propagation. The basic convolutional units implement explicit residual connections, while the parallel multi-branch modules also retain the overall residual paths, ensuring lossless information propagation. Another key optimization design is the feature upsampling strategy. The network not only supports simple bilinear interpolation upsampling but also implements more complex feature extension modules, achieving high-quality feature upsampling through parameter learning. This method retains more semantic information than traditional interpolation methods, improving the quality of upsampled features. The upsampling process also integrates skip connection features from the encoder, ensuring effective recovery of detailed information. Finally, the overall architecture design also emphasizes computational efficiency and memory optimization. Many modules in the model employ lightweight designs. For example, the channel attention mechanism uses one-dimensional convolutions instead of fully connected layers, and the basic convolutional units use batch normalization and ReLU activation instead of more complex activation functions. These designs significantly reduce computational complexity and memory requirements while maintaining the model's expressive power.

[0054] Specifically, step 202a above, the step of performing deep feature extraction based on the feature bank layer in the downsampling path, may further include steps 202a2 to 202a4: Step 202a2: Divide the feature map to be downsampled into multiple regions and calculate the importance index of each region.

[0055] Step 202a3: Based on the preset importance threshold and the importance index of each region, divide the multiple regions into key regions and non-key regions.

[0056] Step 202a4: Expand the region-level mask corresponding to the key region into a pixel-level mask, and use the pixel-level mask of the key region to perform selective feature enhancement processing on the key region and non-key region in the feature map.

[0057] The selective feature enhancement process includes: applying full feature bank processing to key regions, and maintaining the original features or performing dimensionality reduction processing on non-key regions.

[0058] For example, in the HUNet architecture, the Dynamic Feature Bank Clustering Supervised (DFBCS) mechanism provides a novel feature enhancement paradigm for medical image segmentation through the synergistic optimization of deep representation learning and manifold clustering. The core of this mechanism lies in constructing a dynamically evolving feature prototype dictionary. By continuously tracking the essential distribution patterns of target anatomical structures in the feature space, it effectively alleviates the intra-class discreteness and inter-class confusion problems caused by morphological variations in traditional methods. The set of cluster prototypes maintained in the system can be viewed as anchor points on high-dimensional feature vectors, with each prototype representing a typical feature pattern of a specific category (such as normal regions or thrombotic regions). These prototypes are updated with momentum using an exponential moving average strategy: for feature vectors extracted by the network, their similarity to existing prototypes is calculated, and the spatial position of matching prototypes is adjusted through momentum updates. This mechanism enables the prototype dictionary to adapt to the complex variations in anatomical structures in medical images. For example, in the case of mesenteric artery embolism, the system automatically evolves multiple subclass prototypes, corresponding to lesion regions with different densities and morphological features, significantly improving the model's ability to represent heterogeneous embolisms.

[0059] For example, the Feature Bank, as a key innovative component in the improved Swing Transformer network architecture, significantly enhances the model's representational power and generalization performance by establishing efficient feature memory, clustering, and fusion mechanisms. Figure 4 As shown in (a), the feature bank is located in the "Build & Update" module on the left, responsible for feature vectorization, clustering, and feature enhancement. It establishes a short-term memory to long-term memory conversion mechanism, enabling the network to retain and utilize historical feature information, thereby improving its ability to recognize complex patterns. The core value of the feature bank lies in three aspects: first, constructing a feature memory mechanism by storing historical feature information through feature queues and cluster centers; second, achieving adaptive feature enhancement by generating more discriminative feature representations through retrieval and fusion; and third, optimizing small target processing by concentrating resources on key regions through region partitioning and importance assessment. From a system architecture perspective, the feature bank is closely integrated with the model's downsampling path, capturing and optimizing feature representations at key nodes in feature-level transformation, providing the entire network with a richer and more discriminative feature foundation.

[0060] For example, the feature bank's workflow begins with the management of the feature queue, a process involving the collection, storage, and dynamic updating of features. In its implementation, the feature queue employs a circular buffer design to achieve an efficient first-in, first-out (FIFO) update strategy. Whenever a new batch of features arrives, the feature bank performs a queue update operation. First, the input features are randomly sampled to control computational burden, and then the sampled features are written to the appropriate position in the queue. To ensure numerical stability, all feature vectors are uniformly converted to the same precision, while asynchronous operations are used to improve update efficiency. As the underlying data structure of the feature bank, the feature queue not only provides a source of samples for cluster center updates but also implicitly builds a memory mechanism for historical features in the model, enabling the network to accumulate knowledge over time. The update frequency of the feature queue is controlled by specific parameters, typically performing a full update only every few iterations to balance computational efficiency and feature freshness.

[0061] For example, based on the accumulated feature queue, the feature bank extracts representative cluster centers using an adaptive K-means algorithm. The cluster centers serve as feature prototypes, adaptively set according to the feature dimensions. The clustering process employs a momentum update mechanism to smooth changes in cluster centers while dynamically adjusting the number of effective cluster centers to adapt to the sample distribution. To capture feature patterns at different scales, the clustering algorithm also combines batch processing and feature selection techniques, selectively processing important features with larger feature norms to improve clustering quality. The quality of cluster centers directly affects the effect of feature fusion; therefore, the feature bank also introduces an activity tracking mechanism to monitor the usage of each cluster center, periodically resetting inactive cluster centers to ensure all cluster centers remain effectively utilized. The cluster center momentum update can be represented by the following formula: (Formula 5) in, ∈[0,1) is the momentum coefficient. It is the feature set of the current batch.

[0062] For example, one of the core functions of the Feature Bank is feature retrieval, which involves finding the most similar cluster centers for input features. The retrieval process first calculates the cosine similarity through feature normalization, and then selects the k cluster centers with the highest similarity. To improve retrieval efficiency, the Feature Bank implements several optimization techniques: first, a similarity caching mechanism that records calculated results using hash values ​​to avoid redundant calculations; second, a sparse computation strategy that only processes important features with large feature norms; and third, active center filtering that considers only highly active cluster centers. When processing large amounts of data, the Feature Bank also uses random sampling techniques to reduce the computational burden. To further accelerate large-scale feature retrieval, the Feature Bank also integrates professional indexing technology, automatically selecting the most suitable index type based on the feature set size. These optimization measures collectively ensure the high efficiency and accuracy of feature retrieval, maintaining good performance even when processing a large number of features. The cosine similarity calculation process can be represented by the following formula: (Formula 6) in It is a query feature. It is the first The formula for calculating the attention weight for each cluster center can be expressed by the following formula: (Formula 7) Next, the similarity is normalized using the softmax function, and the residual fusion formula can be expressed by the following formula: (Formula 8) in, ∈[0,1] are learnable fusion weight parameters, and the feature fusion formula can be expressed by the following formula nine: (Formula 9) in, It is the first The reliability update formula for the feature vectors corresponding to each cluster center can be expressed by the following formula: (Formula 10) in, It is the momentum coefficient. It is the number of samples assigned to that cluster. It is the normalization coefficient.

[0063] For example, after feature retrieval, the feature bank uses a feature fusion mechanism to weightedly combine the original features with the retrieved cluster centers to generate enhanced features. The fusion process first calculates the weights of the cluster centers based on multiple factors, including similarity, reliability, and activity. Similarity reflects the degree of matching between the feature and the cluster center, reliability measures the sample support of the cluster center, and activity represents the frequency of use of the cluster center. The weights are normalized using softmax, where a temperature parameter controls the smoothness of the distribution. Then, batch matrix multiplication is used to efficiently calculate the weighted sum. To balance the contributions of the original and enhanced features, the feature bank uses adaptive fusion coefficients, with smaller fusion weights initially, gradually increasing as the feature bank trains. Furthermore, the feature bank also supports adaptive fusion based on feature importance, giving higher fusion weights to important features. Finally, the original and enhanced features are weighted to generate the final output. This adaptive fusion mechanism ensures the stability and effectiveness of feature enhancement, avoiding excessive modification of the original features during the fusion process. The reliability-weighted similarity formula can be expressed by the following formula eleven: (Formula Eleven) in, It is an indicator of the reliability of cluster centers. This is the reliability attenuation coefficient, and the dynamic temperature regulation formula can be expressed by the following formula twelve: (Formula 12) in, It is the base temperature. It is the adjustment coefficient. It is the entropy of the similarity distribution.

[0064] For example, such as Figure 4 As shown, to better handle features at different scales, especially improving the ability to recognize small targets, the feature bank implements a region optimization processing mechanism. This mechanism first divides the feature map into several regions, then calculates the importance index for each region, and highlights the differences between regions through normalization. Based on an importance threshold, the feature bank identifies and prioritizes key regions containing potential small targets, where the threshold parameter is optimized to balance detection rate and accuracy. The region mask is expanded to generate a pixel-level mask for selectively applying feature bank processing. This region optimization strategy allows the feature bank to concentrate resources on processing the most informative regions while maintaining computational efficiency, significantly improving sensitivity to small targets and detailed features. Experiments show that this mechanism is particularly effective in complex scenes containing multi-scale targets, helping the model better capture and retain small but important feature information, providing richer feature representations for subsequent classification or segmentation tasks.

[0065] For example, such as Figure 4 The diagram shows the feature storage and retrieval fusion process of a feature bank, which is divided into two core modules: (a) feature construction and updating and (b) feature retrieval and fusion, along with explanations of node status and data structure.

[0066] like Figure 4 As shown, module (a) Build & Update: This module is responsible for feature extraction, storage, and updating. Its core function is to build and dynamically maintain a "Feature Bank." 1. Feature Vectorization: Input data (such as images and text) is processed and transformed into feature vectors (blue bars in the diagram represent feature vectors). 2. L2 Score Selected: L2 distance calculation is performed on the feature vectors to select those that meet the criteria, which then proceed to the next step. 3. Cluster: The selected feature vectors are clustered (green grids in the diagram represent clustered feature clusters). During clustering, node states are divided into three categories: ActiveNode: Red nodes, representing core nodes currently participating in the calculation; Inactive Node: Yellow nodes, representing nodes not currently participating in the calculation; Normal Node: Green nodes, representing nodes in a normal state. 4. **Activity Decay:** After clustering, the "activity" of nodes decays over time (the dashed arrows in the diagram represent the decay process), simulating the dynamic changes in node state. 5. **Feature Bank:** Clustered feature clusters are stored in the "Feature Bank," forming long-term feature storage. 6. **Update:** When new features or data enter the system, the Feature Bank is dynamically updated to maintain the timeliness of features. 7. **Reset:** To reinitialize the Feature Bank, a "Reset" operation can be performed to clear or reset the stored content.

[0067] like Figure 4As shown, module (b) Retrieve & Fuse: This module is responsible for retrieving similar features from the feature repository and fusing them with the input features to generate the final output. 1. Input: New input data (such as images or text to be processed) enters the system. 2. TopK Similar Match: The input features are matched with features in the feature repository to select the "TopK most similar features" (K is the preset number of similar features). 3. L2 Score Selected: The matched TopK features are further filtered using L2 distance to ensure that the feature similarity meets the requirements. 4. Weighted Fusion: The filtered features are weighted and fused with the input features (the "⊕" symbol in the diagram represents the fusion operation) to generate the fused features. 5. Output: The fused features are used as the final output, completing the "retrieval-fusion" process.

[0068] like Figure 4 As shown, Eigenvector represents the input / processed feature vector; FeatureMap represents the feature representation of the input data; Feature Queue represents the clustered feature clusters (stored in queue form); Weighted Fusion represents the fusion operation; Active Node is the core computing node; Inactive Node is a node that is not currently participating in computation; Normal Node is a node in a normal state.

[0069] For example, the feature bank is deeply integrated with the SwinTransformer architecture through a dedicated downsampling layer, enhancing the model's feature representation capabilities while retaining the advantages of the original architecture. From an implementation perspective, the feature bank is integrated into the model's downsampling path, responsible for capturing and optimizing features at key nodes in the feature level transformation. Each feature bank layer contains standard downsampling operations and feature enhancement functions. First, it performs feature transformations that halve the spatial dimension and double the channel dimension, then adjusts the feature dimensions through normalization and linear projection. After feature computation, the feature bank layer extracts feature vectors, performs feature bank update, retrieval, and fusion operations, and generates enhanced features. Finally, the original and enhanced features are fused using learnable fusion weights. This design ensures that the feature bank can be seamlessly integrated into the SwinTransformer, providing additional feature enhancement capabilities without disrupting the original gradient flow and information transmission, thereby achieving better performance in various vision tasks.

[0070] For example, to achieve efficient computation while maintaining functionality, the feature bank implements a series of performance optimization and acceleration techniques. First, it uses a dimensionality reduction projector to reduce computational complexity, projecting high-dimensional features into a lower-dimensional space for processing, significantly reducing computational and storage overhead. Second, it employs a batch processing strategy to process a large number of features in segments, combined with multi-threaded parallel acceleration. Simultaneously, it implements a caching mechanism to avoid repeatedly calculating the same or similar features. Regarding hardware acceleration, the feature bank prioritizes GPU-accelerated feature computation and automatically selects the most suitable index structure based on the number of features. Furthermore, it implements an adaptive computation strategy, dynamically adjusting the computation frequency based on the steps and batch size, as well as an early stopping mechanism that quickly returns the original features when a potential failure is detected. These optimizations collectively ensure that the feature bank can operate efficiently under various hardware conditions, providing powerful functionality while maintaining low computational overhead.

[0071] Specifically, in step 202b above, the step of upsampling the second feature information using a soft mask-guided upsampling mechanism may further include the following step 202b1: Step 202b1: Perform progressive upsampling on the second feature information, and fuse the result of progressive upsampling with the result of direct upsampling through learnable weights to obtain the third feature information.

[0072] The progressive upsampling process includes at least two consecutive 2x upsampling stages, with a feature refinement operation performed after each upsampling stage. During the progressive upsampling process, a dynamic adaptive threshold filtering strategy is used to generate a soft mask to guide feature fusion and suppress background noise. The feature refinement operation includes: refining and enhancing the features so that the features have higher resolution while restoring and strengthening semantic information and structural details.

[0073] For example, this application also designs a soft mask-guided upsampling mechanism. This module enhances the expressive power of upsampled features through a progressive fusion strategy, restores the low-resolution, high-semantic feature maps extracted from deep layers of the network to the original input resolution, and performs fine-tuning to generate the final segmentation result, effectively solving the problems of detail loss and boundary blurring in the traditional upsampling process.

[0074] For example, the upsampling method first performs multi-scale feature acquisition, collecting feature representations from various levels of the decoder. These decoder feature maps of different scales are then uniformly adjusted to the target output resolution, and their key information is initially extracted to generate a set of scale-uniform feature masks. These masks form the basis for subsequent attention mechanisms and feature fusion. Feature maps at different levels carry information at different levels of abstraction: deep features contain rich semantic information and category discrimination capabilities, while shallow features retain more spatial details and texture information. By integrating multi-scale features, the model can achieve a balance between semantic understanding and spatial accuracy, which is crucial for high-precision segmentation tasks. Simultaneously, the main input feature tensor is reshaped and its dimensions are replaced, initializing it as a guiding feature map, which serves as the main body for subsequent multi-scale information fusion. This can be specifically represented by the following formulas thirteen and fourteen. (Formula Thirteen) in, Indicates the first Layer-uniform resolution feature mask, Decoder Layer features.

[0075] For example, the multi-scale adaptive attention fusion mechanism iteratively processes previously prepared multi-scale feature masks, constructing a complete attention computation framework that considers both channel and spatial dimensions. In each iteration, for a mask at a specific scale, an average pooling operation with a dynamic kernel size is first applied for smoothing to suppress noise and enhance local consistency. Subsequently, three types of attention maps are generated in parallel: channel attention, spatial attention, and edge attention. Channel attention generates a weight vector that emphasizes important feature channels by calculating the global channel mean of the smoothed mask and combining it with learnable weight parameters, used to identify and enhance channels containing discriminative information. Spatial attention generates an attention map that highlights spatially significant regions by calculating the ratio of the smoothed mask to its global mean and combining it with another set of learnable weights. Edge attention uses a predefined and cached Sobel gradient operator to calculate the gradient magnitude of the smoothed mask, and then combines it with a third set of learnable weights and a Sigmoid activation function to generate an attention map focused on object boundary regions. This design is particularly suitable for applications such as medical images where contour accuracy is extremely important. This parallel generation of multi-attention mechanisms enables the method to capture key information from feature maps across different dimensions. These multiple attention mechanisms create a complementary effect, comprehensively enhancing feature representation capabilities. (Edge Attention) The formula is shown in Formula Fifteen below: (Formula Fifteen) in, For predefined Sobel operators, Smoothed feature mask, Learnable weight parameters It's the Sigmoid activation function, ⊗ convolution operation. Convolution calculation.

[0076] For example, to further refine effective information and suppress irrelevant background, the method employs a dynamic adaptive threshold filtering strategy. Unlike traditional fixed threshold methods, a dynamic confidence threshold is calculated based on the statistical characteristics of the current processed mask and the baseline and scaling factor, which change iteratively and are controlled by learnable parameters. Specifically, the method extracts the mean and standard deviation of the feature map and combines them with learnable parameters to calculate an adaptive threshold for filtering low-confidence regions. Only when the value of the smoothed mask is higher than this threshold is the feature at that location considered reliable, thus generating a filtered mask. This step effectively focuses on high-confidence foreground regions. All generated attention components, original masks, filtered masks, and related statistics are integrated and stored for use in subsequent fusion stages. This adaptive mechanism enables the model to automatically adjust the threshold according to the distribution characteristics of the input data, significantly improving the model's adaptability to different scenarios. The dynamic threshold filtering formula is shown in Formula Sixteen below: (Formula Sixteen) in, A base threshold is used to prevent noise interference; the higher the value, the stricter the filtering. The hierarchical scaling factor controls the threshold decay rate at different network depths.

[0077] For example, the feature fusion and optimization process also adopts an iterative approach, progressively refining the guiding feature map. The method initializes a cumulative attention map and updates it in each iteration. At the start of an iteration, a hierarchical importance weight is calculated based on the current scale and learnable parameters. This strategy controls the proportion of historical information retained through a learnable decay factor, achieving a smooth transition of attention information. This design allows the model to adapt to new feature representations while retaining key historical information, enhancing consistency. Simultaneously, to ensure the coherence of spatial predictions, the method introduces spatial consistency constraints, making the prediction results of adjacent regions more consistent through averaging operations in local regions. The introduced feature selection mechanism applies different degrees of enhancement to different regions based on attention weights. Specifically, high-attention regions receive larger enhancement coefficients, while low-attention regions remain relatively unchanged. This selective enhancement strategy allows the model to concentrate computational resources on key regions, improving the relevance and effectiveness of feature representations. The integration of global contextual information significantly improves the model's semantic understanding ability. The previously generated channel, spatial, and edge attention maps are multiplied by their respective hierarchical importance weights or their dedicated learnable weights, and then combined to form a comprehensive attention weight map. To ensure the effectiveness of the weights, normalization is performed. A global representation of the feature map is extracted using global average pooling, then expanded to the original size and fused with local features. This design enables connections between distant pixels, mitigating the lack of contextual information caused by local receptive fields.

[0078] For example, the method employs a progressive approach rather than direct upsampling in its feature upsampling strategy. Progressive upsampling is performed in two stages: first, a 2x upsampling and feature refinement are performed, followed by a second 2x upsampling. Each stage includes feature expansion, shape rearrangement, and feature refinement steps, interspersed with complex feature processing and information fusion. This progressive strategy mitigates information distortion caused by a single large upsampling operation, improving upsampling quality, particularly showing significant advantages when handling complex textures and fine structures. Skip connection mechanisms are extensively applied in this method. Low-level features from the encoder path are introduced into the decoding process. These low-level features are first reduced in resolution through pooling, then the number of channels is adjusted through channel adaptation, and finally restored to the target size through bilinear interpolation. Low-level features are fused with the current features through a gating mechanism, with gating coefficients automatically learned by the network. This design embodies the idea of ​​complementary use of shallow and deep features, effectively mitigating the information decay problem in deep networks. The progressive upsampling path of this method fully integrates multi-scale features, effectively improving the model's generalization ability and robustness.

[0079] like Figure 5As shown, this is a dual-branch feature extraction architecture. The core is to achieve efficient extraction and fusion of multimodal features through the collaboration of the SPRU (Spatial Rearrangement Unit) branch and the attention fusion branch.

[0080] The SPRU branch: Starting from the input image (top left corner), the process proceeds through LinearExpand → LN (Layer Normalization) → SpatialRearrange → FeatureRefine → ConvRefine to finally output the features. This branch focuses on the extraction and optimization of spatial dimension features, enhancing the spatial correlation of features through spatial rearrangement, and further improving the feature representation ability through convolutional refinement.

[0081] Attention Fusion Branch: The other input image (bottom left corner) undergoes the following process: AttenRefine → Linear Transformation → Gate → BasicConv → ECA → 3×3Conv → Batch Normalization → ReLU, ultimately outputting features. This branch strengthens the channel weights of features through attention mechanisms (such as the ECA module), filters effective information using a gate mechanism, and then optimizes the feature distribution through convolution and normalization.

[0082] The model in this application embodiment achieved 81.63% accuracy on BUSI, which surpasses all competing models, including the most powerful alternatives. For example, SMM-UNet

[38] , as a novel hybrid architecture model, achieved an accuracy of 81.12%, significantly outperforming other comparative models. The EMCAD model, based on the spatial attention mechanism, achieved an accuracy of 80.25%, while MLFEU-NET, which uses multi-scale feature fusion, demonstrated an advantage with an accuracy of 80.36%.

[0083] In this study, embodiments of this application utilize more than 10 state-of-the-art architectures renowned for their effectiveness in medical image analysis tasks. The selected models include Unet, Att-UNet, UNet++, HSH_UNet, SegNet, TransUNet, FCBFormer, H_vmUNet, VmUNet, U-KAN, EMCAD, MSAByNet, MLFEU-NET, and SMM-UNet. These architectures were carefully chosen based on their proven performance in complex image recognition tasks and their suitability for medical applications. Their diverse architectural designs provide a comprehensive basis for evaluating and comparing their capabilities within the context of this model.

[0084] As shown in Table 1 below, the models in this application embodiment achieved 81.63% accuracy on BUSI, surpassing all competing models, including the most powerful alternatives. For example, SMM-UNet, as a novel hybrid architecture model (integrating Transformer modules and multi-scale convolutional operations), achieved an accuracy of 81.12%, significantly outperforming other comparative models. The EMCAD model, improved based on spatial attention mechanisms, achieved an accuracy of 80.25%, while MLFEU-NET, employing multi-scale feature fusion, demonstrated an advantage with an accuracy of 80.36%.

[0085] Model DSC U-Net 74.90 Att-UNet 78.14 HSHUNet 77.24 UNet++ 75.74 SegNet 75.07 TransUNet 79.35 FCBFormer 80.66 H_vmUNet 75.20 VmUNet 78.32 U-KAN 76.95 EMCAD 80.25 MSAByNet 79.42 MLFEU-NET 80.36 SMM-UNet 81.12 Ours 81.63 Table 1 As shown in Table 1, the proposed method achieves the best performance (DSC 81.63%) among various mainstream segmentation networks. This advantage is mainly due to the systematic improvements in detail modeling, boundary restoration, and cross-domain alignment. Specifically, the PMBE module enhances small-scale feature representation, the SHD-UP mechanism improves boundary continuity, and the PCSA framework enhances cross-domain generalization ability.

[0086] It should be noted that the improvements to the model are described in detail in this embodiment, while the functions and roles of other parts of the model can be referred to the relevant descriptions of the existing Swin Transformer model.

[0087] The medical image recognition method based on prototype-guided context fusion provided in this application first acquires a medical image to be recognized; then, the medical image is input into a medical image recognition model to obtain the recognition result for the target disease output by the medical image recognition model; wherein, the medical image recognition model adopts an encoder-decoder architecture based on the SwinTransformer model; the medical image recognition model is used to: extract features from the medical image through multi-level Transformer layers, and introduce a feature bank layer in each downsampling process; the feature bank layer is configured to: maintain a dynamically updated set of cluster centers as feature prototypes for various diseases; the feature bank layer generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map; the decoder is used to employ a soft-mask-guided progressive upsampling mechanism in the upsampling path, through multi-scale attention fusion and multi-path fusion with learnable weights and attention-guided progressive upsampling. Thus, feature extraction based on the improved Swing Transformer model significantly enhances the model's performance and robustness in complex tasks, achieving an organic combination of depth and breadth while maintaining computational efficiency.

[0088] It should be noted that the medical image recognition method based on prototype-guided context fusion provided in this application can be executed by a medical image recognition device based on prototype-guided context fusion, or by a control module within that device for executing the method. This application uses the execution of the medical image recognition method based on prototype-guided context fusion by a medical image recognition device as an example to illustrate the medical image recognition device based on prototype-guided context fusion provided in this application.

[0089] It should be noted that, in the embodiments of this application, the medical image recognition methods based on prototype-guided context fusion shown in the accompanying drawings are all illustrated using one accompanying drawing from one of the embodiments of this application as an example. In specific implementation, the medical image recognition methods based on prototype-guided context fusion shown in the accompanying drawings of the above methods can also be implemented in conjunction with any other accompanying drawings that can be combined as illustrated in the above embodiments, which will not be elaborated here.

[0090] The medical image recognition device based on prototype-guided context fusion provided in this application is described below. The medical image recognition method based on prototype-guided context fusion described below can be referred to in correspondence with the medical image recognition method based on prototype-guided context fusion described above.

[0091] Figure 6 A schematic diagram of the structure of the medical image recognition device based on prototype-guided context fusion provided in this application embodiment is shown below. Figure 6 As shown, it specifically includes: The system comprises an acquisition module for acquiring medical images to be identified, and an identification module for inputting the medical images into a medical image recognition model to obtain the identification results for the target disease output by the medical image recognition model. The medical image recognition model employs an encoder-decoder architecture based on the Swin Transformer model. The model extracts features from the medical images through multiple Transformer layers, introducing a feature bank layer at each downsampling stage. The feature bank layer is configured to maintain a dynamically updated set of cluster centers as feature prototypes for various diseases. It generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map. The decoder employs a soft-mask-guided progressive upsampling mechanism in the upsampling path, using multi-scale attention fusion and multi-path fusion with learnable weights, along with attention-guided progressive upsampling.

[0092] Optionally, the recognition module is specifically used to extract features from the medical image to obtain first feature information, and to perform multi-level feature extraction and fusion on the first feature information through the multi-level Transformer layer to obtain second feature information; each level of the multi-level Transformer layer contains multiple feature transformation stages, and adjacent Transformer layers reduce spatial resolution and increase the number of channels through a downsampling path to generate a hierarchical feature representation; the recognition module is further used to upsample the second feature information using a soft mask-guided upsampling mechanism to obtain third feature information that fuses different levels and channels, and to generate the recognition result based on the third feature information.

[0093] Optionally, the recognition module is specifically used to input the first feature information into the multi-level feature extraction and fusion in the multi-level Transformer layer to obtain the second feature information; wherein, each level of Transformer layer contains at least one VGG block and a parallel multi-branch module; the VGG block includes two consecutive convolution operations; the parallel multi-branch module includes multiple parallel branches with similar structures and different parameters to achieve differentiated feature extraction; each branch includes: initial feature processing through basic convolutional units, expanding the receptive field through convolutions with different dilation rates, enhancing the inter-channel dependency through a channel attention mechanism, enriching the feature representation using convolutional kernels of different sizes, improving edge perception through a Laplacian edge enhancement layer, and performing final feature extraction through basic convolutional units.

[0094] Optionally, the recognition module is specifically used to divide the feature map to be downsampled into multiple regions and calculate the importance index of each region; the recognition module is further used to divide the multiple regions into key regions and non-key regions based on a preset importance threshold and the importance index of each region; the recognition module is further used to expand the region-level mask corresponding to the key region into a pixel-level mask, and use the pixel-level mask of the key region to perform selective feature enhancement processing on the key regions and non-key regions in the feature map; wherein, the selective feature enhancement processing includes: applying complete feature bank processing to the key regions, and maintaining the original features or performing dimensionality reduction processing on the non-key regions.

[0095] Optionally, the recognition module is specifically used to perform progressive upsampling processing on the second feature information, and to fuse the result of the progressive upsampling with the result of the direct upsampling through a multi-path fusion using learnable weights to obtain the third feature information; wherein, the progressive upsampling processing includes at least two consecutive 2x upsampling stages, and a feature refinement operation is performed after each upsampling stage; during the progressive upsampling process, a dynamic adaptive threshold filtering strategy is used to generate a soft mask to guide feature fusion and suppress background noise; the feature refinement operation includes: refining and enhancing the features so that the features have higher resolution while restoring and strengthening semantic information and structural details.

[0096] Optionally, the recognition module is specifically used to determine, based on the third feature information, an image region in the medical image that is similar to the medical image features of the target disease.

[0097] The medical image recognition device based on prototype-guided context fusion provided in this application first acquires a medical image to be recognized; then, the medical image is input into a medical image recognition model to obtain the recognition result for the target disease output by the medical image recognition model; wherein, the medical image recognition model adopts an encoder-decoder architecture based on the SwinTransformer model; the medical image recognition model is used to: extract features from the medical image through multi-level Transformer layers, and introduce a feature bank layer in each downsampling process; the feature bank layer is configured to: maintain a dynamically updated set of cluster centers as feature prototypes for various diseases; the feature bank layer generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map; the decoder is used to employ a soft-mask-guided progressive upsampling mechanism in the upsampling path, through multi-scale attention fusion and multi-path fusion with learnable weights and attention-guided progressive upsampling. Thus, feature extraction based on the improved Swing Transformer model significantly enhances the model's performance and robustness in complex tasks, achieving an organic combination of depth and breadth while maintaining computational efficiency.

[0098] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logical instructions in the memory 530 to execute a medical image recognition method based on prototype-guided context fusion. This method includes: first, acquiring a medical image to be recognized; then, inputting the medical image into a medical image recognition model to obtain the recognition result for the target disease output by the medical image recognition model; wherein the medical image recognition model adopts an encoder-decoder architecture based on the Swing Transformer model; the medical image recognition model is used to: extract features from the medical image through multi-level Transformer layers, and introduce a feature bank layer during each downsampling process; the feature bank layer is configured to: maintain a dynamically updated set of cluster centers as feature prototypes for various diseases; the feature bank layer generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map; the decoder is used to employ a soft-mask-guided progressive upsampling mechanism in the upsampling path, through multi-scale attention fusion and multi-path fusion with learnable weights, and attention-guided progressive upsampling. Thus, feature extraction based on the improved Swing Transformer model significantly enhances the model's performance and robustness in complex tasks, achieving an organic combination of depth and breadth while maintaining computational efficiency.

[0099] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] On the other hand, this application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the prototype-guided context fusion-based medical image recognition method provided by the above methods. This method includes: first, acquiring a medical image to be recognized; then, inputting the medical image into a medical image recognition model to obtain a recognition result for the target disease output by the medical image recognition model; wherein the medical image recognition model adopts a Swing-based approach. The encoder-decoder architecture is built using a Transformer model. The medical image recognition model is used to: extract features from the medical image through multiple Transformer layers, introducing a feature bank layer during each downsampling stage; the feature bank layer is configured to maintain a dynamically updated set of cluster centers as feature prototypes for various diseases; the feature bank layer generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map; the decoder employs a soft-mask-guided progressive upsampling mechanism in the upsampling path, using multi-scale attention fusion and multi-path fusion with learnable weights, along with attention-guided progressive upsampling. Thus, feature extraction based on the improved Swin Transformer model significantly improves the model's performance and robustness in complex tasks, achieving an organic combination of depth and breadth while maintaining computational efficiency.

[0101] In another aspect, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the aforementioned prototype-guided context fusion-based medical image recognition methods. The method includes: first, acquiring a medical image to be recognized; then, inputting the medical image into a medical image recognition model to obtain a recognition result for a target disease output by the medical image recognition model; wherein the medical image recognition model adopts an encoder-decoder architecture based on a Swing Transformer model; the medical image recognition model is used to: extract features from the medical image through multi-level Transformer layers, and introduce a feature bank layer during each downsampling process; the feature bank layer is configured to: maintain a dynamically updated set of cluster centers as feature prototypes for various diseases; the feature bank layer generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map; the decoder is used to employ a soft-mask-guided progressive upsampling mechanism in the upsampling path, through multi-scale attention fusion and multi-path fusion with learnable weights, and attention-guided progressive upsampling. Thus, feature extraction based on the improved Swing Transformer model significantly enhances the model's performance and robustness in complex tasks, achieving an organic combination of depth and breadth while maintaining computational efficiency.

[0102] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0103] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A medical image recognition method based on prototype-guided context fusion, characterized in that, include: Acquire the medical image to be identified; The medical image is input into the medical image recognition model, and the medical image recognition model outputs the recognition result for the target disease. The medical image recognition model employs an encoder-decoder architecture based on the Swin Transformer model. This model extracts features from the medical image through multiple Transformer layers, introducing a feature bank layer at each downsampling stage. The feature bank layer maintains a dynamically updated set of cluster centers as feature prototypes for various diseases. It generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map. The decoder uses a soft-mask-guided progressive upsampling mechanism in the upsampling path, gradually recovering high-resolution semantic features through multi-scale attention fusion and a multi-path fusion strategy with learnable weights. The soft-mask-guided upsampling mechanism includes: using a dynamic adaptive threshold to filter noise and employing multi-path fusion and attention-guided progressive upsampling.

2. The method according to claim 1, characterized in that, The step of inputting the medical image into a medical image recognition model to obtain the recognition result of the target disease output by the medical image recognition model includes: The medical image is subjected to feature extraction to obtain first feature information, and the first feature information is subjected to multi-level feature extraction and fusion through the multi-level Transformer layer to obtain second feature information; each level of the multi-level Transformer layer contains multiple feature transformation stages, and adjacent two levels of Transformer layers reduce spatial resolution and increase the number of channels through downsampling path to generate hierarchical feature representation; The second feature information is upsampled using a soft mask-guided upsampling mechanism to obtain a third feature information that integrates different levels and channels, and the recognition result is generated based on the third feature information.

3. The method according to claim 2, characterized in that, The process of extracting and fusing the first feature information through the multi-level Transformer layers to obtain the second feature information includes: The first feature information is input into the multi-level Transformer layer for multi-level feature extraction and fusion to obtain the second feature information; Each Transformer layer contains at least one VGG block and one parallel multi-branch module. The VGG block includes two consecutive convolutional operations. The parallel multi-branch module includes multiple parallel branches with similar structures and different parameters to achieve differentiated feature extraction. Each branch includes: initial feature processing through basic convolutional units, expanding the receptive field through convolutions with different dilation rates, enhancing the inter-channel dependencies through a channel attention mechanism, enriching the feature representation using convolutional kernels of different sizes, improving edge perception through a Laplacian edge enhancement layer, and finally extracting features through basic convolutional units.

4. The method according to claim 2 or 3, characterized in that, The deep feature extraction based on the feature bank layer in the downsampling path includes: The feature map to be downsampled is divided into multiple regions, and the importance index of each region is calculated. Based on a preset importance threshold and an importance index for each region, the multiple regions are divided into critical regions and non-critical regions. The region-level mask corresponding to the key region is expanded into a pixel-level mask, and the pixel-level mask of the key region is used to perform selective feature enhancement processing on the key and non-key regions in the feature map. The selective feature enhancement process includes: applying full feature bank processing to key regions, and maintaining the original features or performing dimensionality reduction processing on non-key regions.

5. The method according to claim 2, characterized in that, The method of upsampling the second feature information using a soft mask-guided upsampling mechanism to obtain a third feature information that fuses different levels and channels includes: The second feature information is subjected to progressive upsampling, and the result of progressive upsampling is fused with the result of direct upsampling through learnable weights to obtain the third feature information. The progressive upsampling process includes at least two consecutive 2x upsampling stages, with a feature refinement operation performed after each upsampling stage. During the progressive upsampling process, a dynamic adaptive threshold filtering strategy is used to generate a soft mask to guide feature fusion and suppress background noise. The feature refinement operation includes: refining and enhancing the features so that the features have higher resolution while restoring and strengthening semantic information and structural details.

6. The method according to claim 2, characterized in that, The generation of the recognition result based on the third feature information includes: Based on the third feature information, image regions in the medical image that are similar to the medical image features of the target disease are determined.

7. A medical image recognition device based on prototype-guided context fusion, characterized in that, The device includes: The acquisition module is used to acquire the medical image to be identified; The recognition module is used to input the medical image into the medical image recognition model and obtain the recognition result of the medical image recognition model for the target disease. The medical image recognition model employs an encoder-decoder architecture based on the Swin Transformer model. This model extracts features from the medical image through multiple Transformer layers, introducing a feature bank layer at each downsampling stage. The feature bank layer maintains a dynamically updated set of cluster centers as feature prototypes for various diseases. It generates a region importance map based on the similarity between the current feature map and each feature prototype, and performs selective feature enhancement on key regions based on the importance map. The decoder uses a soft-mask-guided progressive upsampling mechanism in the upsampling path, employing multi-scale attention fusion and learnable weighted multi-path fusion with attention-guided progressive upsampling.

8. The apparatus according to claim 7, characterized in that, The recognition module is specifically used to extract features from the medical image to obtain first feature information, and to perform multi-level feature extraction and fusion on the first feature information through the multi-level Transformer layer to obtain second feature information; each level of the multi-level Transformer layer contains multiple feature transformation stages, and adjacent Transformer layers reduce spatial resolution and increase the number of channels through a downsampling path to generate a hierarchical feature representation; The recognition module is further configured to use a soft mask-guided upsampling mechanism to upsample the second feature information to obtain a third feature information that integrates different levels and channels, and to generate the recognition result based on the third feature information.

9. The apparatus according to claim 8, characterized in that, The recognition module is specifically used to input the first feature information into the multi-level feature extraction and fusion in the multi-level Transformer layer to obtain the second feature information; Each Transformer layer contains at least one VGG block and one parallel multi-branch module. The VGG block includes two consecutive convolutional operations. The parallel multi-branch module includes multiple parallel branches with similar structures and different parameters to achieve differentiated feature extraction. Each branch includes: initial feature processing through basic convolutional units, expanding the receptive field through convolutions with different dilation rates, enhancing the inter-channel dependencies through a channel attention mechanism, enriching the feature representation using convolutional kernels of different sizes, improving edge perception through a Laplacian edge enhancement layer, and finally extracting features through basic convolutional units.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the medical image recognition method based on prototype-guided context fusion as described in any one of claims 1 to 6.

Images