Lung cancer gene mutation classification method based on frequency domain multi-scale fusion guidance

Abstract

The invention discloses a lung cancer gene mutation classification method based on frequency domain multi-scale fusion guidance, and relates to the technical field of medical image processing and gene detection. According to the MFHA mechanism provided by the invention, the pathological image is decoupled into low-frequency global and high-frequency detail sub-bands through wavelet transform, and extraction of key high-frequency features such as cell nucleus morphology and local texture is enhanced by combining multi-scale convolution and up-sampling guided by high-frequency information; the problems of insufficient feature detail mining and low feature fusion efficiency in a traditional pathological image analysis method are solved; key features are screened and focused through a channel, frequency domain-space feature deep fusion is realized through up-sampling, robust representation is constructed by combining space attention with cosine similarity and multi-dimensional statistical features, a frequency domain analysis-space focusing collaborative optimization mechanism is formed, information redundancy caused by simple feature splicing is avoided, and the robustness of the system is improved. And the classification stability of the model in a complex pathological scene is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing and gene detection, in particular to a lung cancer gene mutation classification method based on frequency domain multi-scale fusion guidance. BACKGROUND

[0002] Lung cancer is one of the malignant tumors with the highest incidence and mortality in the world, and non-small cell lung cancer (NSCLC) accounts for 80%-90%. 70% of patients are in the advanced stage when diagnosed, and the 5-year survival rate is only 15%-20%. In the diagnosis of lung cancer, pathologists observe the preliminary diagnosis through hematoxylin-eosin (H&E) staining sections, but the pathological images have significant heterogeneity and massive detailed information, and manual analysis is time-consuming and subjective. Traditional gene detection methods such as polymerase chain reaction (PCR) and next-generation sequencing (NGS) can accurately identify gene mutations, but they are complex to operate, require high-quality samples, and rely on expensive equipment, resulting in high detection costs.

[0003] In recent years, deep learning technology has provided a new path for gene mutation prediction based on WSI, and multi-instance learning (MIL) and Transformer-based models have become mainstream solutions. However, the existing technology has two major limitations:

[0004] 1. Insufficient feature extraction: Most methods focus on global feature interaction, but ignore the mining of high-frequency details (such as cell nucleus morphology and local texture) in pathological images. Although this type of high-frequency information has low visual saliency, it is a key clue to distinguish gene mutation-related phenotypes, and its absence makes it difficult for the model to capture subtle pathological morphological differences.

[0005] 2. Low feature fusion efficiency: Traditional wavelet transform-related methods have a shallow use of decomposed frequency band subbands (especially high-frequency subbands), and mostly use simple concatenation to fuse features, which cannot realize deep collaborative optimization of frequency domain and spatial domain features, and global averaging of feature interaction easily smooths local subtle features, making it difficult to balance the needs of global semantic understanding and local detail enhancement.

[0006] Therefore, how to construct a method that can efficiently fuse frequency domain multi-scale information and strengthen high-frequency detail representation has become a key to improving the classification accuracy of lung cancer gene mutations, and therefore a new solution is needed to address the above problems. SUMMARY

[0007] In view of the problems of insufficient high-frequency detail mining of pathological images and low-efficiency of frequency domain-space feature fusion in the prior art, the purpose of the present application is to provide a lung cancer gene mutation classification method based on frequency domain multi-scale fusion guidance, which is suitable for realizing accurate classification of lung cancer related gene mutations through digital whole slide images, providing auxiliary support for clinical lung cancer diagnosis and treatment plan making, realizing deep decoupling and collaborative optimization of low-frequency global information and high-frequency detail information of pathological images through construction of a multi-scale frequency domain hybrid attention (MFHA) mechanism, and improving the accuracy and robustness of lung cancer gene mutation classification.

[0008] To achieve the above purpose, the present application provides the following technical scheme: a lung cancer gene mutation classification method based on frequency domain multi-scale fusion guidance, at least comprising the following steps:

[0009] S1: Collecting a lung cancer gene mutation related data set, using a weight resampling technique to divide the data set into a training set and a test set according to a 7:3 ratio;

[0010] S2: Removing the background area without histological information in the digital whole slide image (WSI) through an edge detection algorithm, and retaining the effective pathological area image containing tumor tissue and normal gland;

[0011] S3: Dividing the WSI subgraph filtered by the background into a plurality of local image blocks, i.e. Patch blocks, according to a fixed size;

[0012] S4: Using a multi-scale frequency domain hybrid attention mechanism, i.e. MFHA mechanism, to extract pathological features with global structure and high-frequency details for each Patch;

[0013] S5: Correlating the features of different regions in the Patch by means of the sliding window attention mechanism of Swin Transformer, and modeling the relevance of local pathological patterns;

[0014] S6: Integrating the local features of all Patch blocks by using LongNet to capture long-distance dependencies of the whole slide;

[0015] S7: Classifying lung cancer gene mutations based on the global embedding vector output by the LongNet.

[0016] Further, the lung cancer gene mutation related data set contains H&E stained WSI of lung cancer patients and corresponding gene mutation annotation information, and the gene mutation annotation information includes whether there is a gene mutation and a class label of a specific type of gene mutation;

[0017] After obtaining the lung cancer gene mutation related data set, a weight resampling is used to balance the class distribution, and the training set and the test set are divided according to a 7:3 ratio . ;

[0018] The sampling weight formula is as follows:

[0019]

[0020] wherein, is the number of samples of the category c; is the total number of samples;

[0021] The method dynamically calculates the weight coefficient according to the distribution frequency of samples of each category, gives higher sampling weight to the scarce category, and in the training stage, through the weighted random sampling mechanism, forces the model to balance the contact with different category features in the iteration process.

[0022] Further, the background area without histological information includes a blank area of a slide, and the image of the effective pathological area is compared with the original WSI through an edge detection algorithm to obtain after removing the area with a background pixel ratio exceeding a preset threshold.

[0023] Further, the Patch block size segmented in S3 is unified to 224x224 pixels, and each Patch block contains a tissue area that can reflect microscopic pathological morphology.

[0024] Further, the background filtering and image segmentation technology is used in S2 and S3.

[0025] The background filtering and image segmentation at least include the following steps:

[0026] The Canny edge detection algorithm is used for each WSI pathological image to peel off the non-tissue area and retain the effective pathological area, represented as:

[0027]

[0028] wherein, is the pixel value of the original WSI; is the standard deviation of Gaussian filtering; low and high are edge detection threshold values; represents the effective area pixel;

[0029] Then, the effective area is segmented into image blocks with a size of 224x224 pixels, and the effective tissue pixel ratio is , represented as:

[0030]

[0031] wherein, is the i-th image block.

[0032] ​Furthermore, the MFHA mechanism includes a channel filtering module, a wavelet decomposition module, an upsampling module, and a spatial attention module. It decouples the low-frequency global information and high-frequency detail information of the pathological image through discrete wavelet transform, and performs multi-scale enhancement and fusion on the high-frequency detail information.

[0033] The channel filtering module compresses channel spatial information through global average pooling, learns channel dependencies by combining channel transformation functions, and filters high-weight channels to generate sub-feature maps. The wavelet decomposition module uses Haar discrete wavelet transform to decompose the sub-feature maps into low-frequency approximate components and multiple high-frequency components. After fusing the low-frequency components and high-frequency components, attention maps are generated through multi-scale convolution, and weighted to obtain enhanced features.

[0034] In the channel filtering module, the input feature map Perform global average pooling, where H and W are the spatial resolutions and C is the number of channels, compressing the spatial information of each channel into... Channel descriptor Then, channel dependencies are learned through a channel transformation function with a dimensionality reduction ratio r, and channel weight vectors are generated. Finally, based on Filter the top N high-weight channels (N < C) to generate sub-feature maps. ;

[0035] In the wavelet decomposition module, the Haar Discrete Wavelet Transform (DWT) is used to decompose the wavelet decomposition. The process involves decomposition, constructing four convolutional kernels with a stride of 2 using low-pass and high-pass filters to generate four wavelet subbands: a low-frequency approximate component LL (preserving global structure), a high-frequency horizontal component HL (capturing vertical edges), a high-frequency vertical component LH (capturing horizontal edges), and a high-frequency diagonal component HH (capturing diagonal details). Subsequently, LL is fused with HL and LH respectively to generate... , This enhances the contextual coherence of high-frequency subbands; then, it undergoes multi-scale hybrid attention (MHA) to... , Multi-scale convolutions with kernel sizes of 3, 5, and 7 are performed respectively, and attention maps are generated by pointwise convolution and the Sigmoid function. At the same time, the method returns high-frequency component combinations hi-subs for upsampling guidance.

[0036] In the upsampling module, the output Y of the wavelet decomposition module is split along the channel dimension into... , ,right Perform a 1×1 convolution to expand the number of channels to 4N, and input the PixelShuffle layer to improve spatial resolution; After fusing with hi-subs, the image is input into an inverse discrete wavelet transform (IDWT) layer to restore it to its original size. Then, the PixelShuffle output is concatenated with the IDWT output, and feature fusion is achieved through contrastive perceptual attention (CCA) and an improved Fused-MBConv layer, outputting a feature map with restored size. ;

[0037] In the spatial attention module, the feature map Perform global average pooling to obtain the global feature Z, and calculate Z with... Cosine similarity of local features To measure the matching degree between global and local features; and at the same time... Max pooling, average pooling, standard pooling, and L2 norm are performed separately to extract multi-dimensional statistical features; then, the multi-dimensional statistical features are combined with cosine similarity and further processed by multi-scale convolution to generate an attention map. Element-wise multiplication with the original feature map is performed to further enhance the perception of image details and filter image noise.

[0038] Furthermore, the sliding window attention mechanism of the Swing Transformer realizes the association of features in different regions within the Patch block by moving between different windows and calculating attention;

[0039] The window size is adaptively adjusted or set to a preset value based on the size of the patch block and the distribution of pathological features.

[0040] The application of the Swin Transformer includes at least the following steps:

[0041] After MFHA processing, the feature image enters the 4-stage Swing Transformer layer, which divides the feature image into 8×8 windows, and performs self-attention calculation within each window.

[0042] The window slides in 56-pixel increments, and attention across the window is calculated to capture feature associations between different regions within the patch (such as tumor cell clusters and surrounding stroma).

[0043] Finally, the Patch-level feature embedding vector is output.

[0044] Furthermore, the LongNet captures long-distance dependencies such as the heterogeneity of tumor cells in different regions and global tissue structure patterns within the entire slice by performing long-sequence modeling on the local features of each patch block.

[0045] The method of integrating local features of all patch blocks using LongNet includes at least the following steps:

[0046] All Patch-level feature embedding vectors are arranged into a long sequence according to their original WSI spatial positions and then input into LongNet to model the long-distance dependency of the whole slice.

[0047] Since each WSI generates a large number of patches (1000-10000), the patch sequence is divided into several segments (each segment contains 64 patches);

[0048] LongNet then uses a combination of linear attention and local attention to perform attention calculations on the patch features within and between each segment, capturing long-distance dependencies in different regions.

[0049] Finally, the features of all segments are integrated to output the global embedding vector of the entire slice.

[0050] Furthermore, the classification head in S7 uses a fully connected layer combined with an activation function, takes the global embedding vector as input, and outputs the classification result of lung cancer gene mutations;

[0051] The activation function used in binary classification is the Sigmoid function, and the activation function used in multi-class classification is the Softmax function.

[0052] In this invention, the cross-entropy loss function is used during the training process, and the Adam optimizer is used to optimize the model parameters. After training on the training set, the classification performance is evaluated using the test set by accuracy (ACC), balanced accuracy (BACC), area under the ROC curve (AUROC), and area under the PR curve (AUPRC).

[0053] Compared with the prior art, the beneficial effects of the present invention are:

[0054] The MFHA mechanism proposed in this invention decouples pathological images into low-frequency global and high-frequency detail subbands through wavelet transform. Combined with multi-scale convolution and high-frequency information-guided upsampling, it enhances the extraction of key high-frequency features such as cell nuclear morphology and local texture, solving the problems of insufficient feature detail mining and inefficient feature fusion in traditional pathological image analysis methods, and further strengthening the high-frequency detail representation capability. By focusing on key features through channel screening, achieving deep fusion of frequency domain and spatial features through upsampling, and constructing robust representations by combining spatial attention with cosine similarity and multi-dimensional statistical features, a collaborative optimization mechanism of frequency domain analysis and spatial focusing is formed, avoiding information redundancy caused by simple feature splicing, and improving the classification stability of the model in complex pathological scenarios. At the same time, it also utilizes the sliding window attention of Swin Transformer to accurately capture the pathological pattern associations in local regions within a patch. LongNet further integrates all patch features to effectively capture long-distance dependencies at the whole slice scale (such as the heterogeneous association of distant tumor cells), achieving deep fusion of local and global features.

[0055] Furthermore, this method exhibits excellent generalization performance. MFHA is "plug-and-play" and can be seamlessly integrated into mainstream architectures such as ResNet and Transformer. It outperforms mainstream attention mechanisms such as CBAM and SE on the Sipakmed pathology dataset and Cifar 10 and Mini ImageNet natural image datasets, verifying its cross-domain adaptability. Attached Figure Description

[0056] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart of the present invention;

[0058] Figure 2 This is an overall flowchart of the model constituted by the present invention;

[0059] Figure 3 This is a diagram illustrating the architecture of the multi-scale frequency domain hybrid attention (MFHA) mechanism of this invention.

[0060] Figure 4 This is a schematic diagram of the Channel Select module of the present invention;

[0061] Figure 5 This is a schematic diagram of the wave up sampling module structure of the present invention;

[0062] Figure 6 This is a schematic diagram of the spatial attention module structure of the present invention;

[0063] Figure 7 This is a comparison diagram of the Grad-CAM thermal map of the present invention. Detailed Implementation

[0064] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0065] This invention addresses the problems of insufficient high-frequency detail mining and inefficient feature fusion in gene mutation classification of pathological images. It preprocesses lung cancer H&E stained whole-slide images (WSI) through background filtering and cropping, constructing a multi-scale frequency-domain hybrid attention (MFHA) mechanism. This mechanism focuses on key features through channel filtering and uses wavelet decomposition to decouple low-frequency global information from high-frequency details, such as cell nuclear morphology and texture. Upsampling achieves lossless feature dimensionality enhancement, and spatial attention combined with cosine similarity and statistical features strengthens the representation of key regions. Simultaneously, MFHA is integrated into the Swin Transformer to form a new feature extraction network, MFTrans, which is combined with a LongNet encoder to capture cross-image patch dependencies. This results in improved classification performance on a lung cancer gene mutation dataset (AUROC improvement of 1.19% compared to the baseline). These techniques, through deep mining and optimization of multi-scale features, aim to solve the challenges of gene mutation classification, providing efficient assistance for clinical diagnosis of lung cancer while effectively reducing the cost of detection technology, demonstrating good practicality and promotional value.

[0066] Based on the above, the following specific implementation examples are proposed:

[0067] This embodiment details a two-stage WSI lung cancer gene mutation classification method based on a multi-module collaborative approach of MFHA + Swin Transformer + LongNet, consisting of a feature extraction stage (MFHA + Swin Transformer) and a global aggregation stage (LongNet). First, using WSI as the initial input, a background processing operation is performed to filter out meaningless background regions, resulting in an image that retains valid tissue. Next, the processed image is segmented into several 224×224 patches. These patches incorporate multi-scale receptive fields through the MFHA mechanism to enhance local image structure and texture information. Then, a four-stage Swin Transformer layer is used, utilizing its sliding window mechanism to associate image features from different regions within the patch, ultimately generating a patch-level embedding representation.

[0068] Subsequently, the patch-level embeddings are passed to the global aggregation stage (LongNet), where the DilatedAttention mechanism captures the long-distance dependencies between different patches. In detail, the masked input is first encoded by a LongNet-based encoder, then passed to a LongNet-based decoder for decoding, and finally optimized using Reconstruction Loss to generate target tile embeddings, completing the entire processing chain from WSI image to high-level semantic embeddings.

[0069] See Figure 1 and Figure 2 Lung cancer gene mutation-based methods guided by frequency domain multi-scale fusion include:

[0070] S1: Collect lung cancer gene mutation related datasets; use weighted sampling technology to divide the dataset into training set and test set in a 7:3 ratio.

[0071] In this embodiment, H&E staining WSI (0.25 μm / pixel resolution) and corresponding gene mutation annotations were collected from 2012 lung cancer patients at a tertiary hospital between 2018 and 2023, covering 12 types of mutations including EGFR and KRAS, as well as wild-type WT. To address the data imbalance issue, class sampling weights were calculated for the training set, with the weights inversely proportional to the number of samples in each class. Random oversampling of the minority class and undersampling of the majority class were used to balance the number of samples in each class in the training set. Finally, after a 7:3 split, the training set contained 1408 cases, and the test set contained 604 cases.

[0072] S2: Use edge detection algorithms to remove background areas (such as blank areas on a slide) in WSI that lack histological information, retaining only images containing effective pathological areas such as tumor tissue and normal glands.

[0073] In this stage, the Canny edge detection algorithm is used to remove background regions lacking organizational information. First, the original WSI is subjected to Gaussian filtering (…). =1.0) noise reduction, then apply the Canny operator (low threshold 50, high threshold 150) to detect edges; then calculate the proportion of blank pixels and the contour area of ​​each connected region. If the proportion of blank pixels in the region is >90%, it is determined to be a background region and stripped. Finally, the effective region containing tumor tissue and normal glands is retained to obtain the background filtered WSI sub-image.

[0074] S3: The WSI sub-image after background filtering is evenly divided into multiple local image patches of size 224×224 pixels to ensure that the patch scale is uniform and contains sufficient microscopic pathological information.

[0075] In this embodiment, the background-filtered WSI sub-image is uniformly segmented using a sliding window at a size of 224×224 pixels, with a step size of 112 pixels to ensure that adjacent patches have 50% overlap and to preserve the continuity of edge region features. Each patch must meet the following requirement: the proportion of effective pathological tissue pixels > 70%; otherwise, it is re-segmented or discarded. Ultimately, an average of 1,000-10,000 effective patches are generated per WSI case.

[0076] S4: Each patch is extracted using MFHA to extract pathological features that combine global structure and high-frequency details, solving the problem of insufficient utilization of high-frequency details in traditional models.

[0077] Multi-scale frequency domain hybrid attention (MFHA) comprises four sub-modules: channel selection, wavelet decomposition, upsampling, and spatial attention. Figure 3 As shown, the specific implementation is as follows:

[0078] Given feature map Where H×W represents the spatial resolution and C is the number of channels. Unlike traditional attention mechanisms, MFHA does not perform calculations based on all channels. Instead, it uses a channel selection module to select the top N high-weight channels from the feature map F to form a sub-feature map. Then, after wavelet decomposition, The data is decoupled into LL, LH, HL, and HH subbands, and then processed by multi-scale hybrid attention (MHA) to enhance the information of each subband and eliminate noise. After upsampling (Wave Up) module, the enhanced features are obtained. .at last The multi-scale attention feature map is obtained after being processed by the unique Spatial Attention, which is multiplied by the original feature map F and undergoes a residual connection to obtain the enhanced feature map. .

[0079] In the Channel Select module, Channel Select is used to adaptively filter the key channels from the input feature map, aiming to focus on the task-related effective channels and improve the resource utilization efficiency while reducing the dimension. As Figure 4 shown, given the input feature map F:

[0080]

[0081] where represents global average pooling, is a non-linear activation function, is the Sigmoid function, represents the index corresponding to selecting the top N maximum values.

[0082] Specifically, for the input feature map , first, through the global pooling operation compress the spatial information of each channel into a channel-level descriptor to summarize the global features of each channel. The process is expressed as:

[0083]

[0084] where is the pooled channel descriptor, represents the value of the c-th channel of the feature map at position . Then, use the channel transformation function to learn the channel interdependence:

[0085]

[0086] where , and are learnable weight matrices, r is the reduction ratio (Reduction Ratio), is a non-linear activation function, is the Sigmoid function. The final output dimension remains transformed vector. Subsequently, through the regularization mechanism, N (N < C) key channels are selected from the C channels to generate a 1×1×N selection indicator. Finally, the corresponding channels of the original feature map are extracted according to this indicator to obtain the output :

[0087]

[0088]

[0089] in This represents the set of channel indices that select the indices corresponding to the N largest values. .

[0090] In the wavelet decomposition module, this invention introduces a wavelet subband-guided downsampling and upsampling module, which utilizes the inherent characteristics of wavelet transform to achieve lossless and reversible downsampling. Figure 3 The detailed downsampling process is shown.

[0091] Specifically, for a given feature map The Haar DWT operation decomposes it into four wavelet subbands. Specifically, the DWT uses a low-pass filter. and high-pass filter Four convolutional kernels with a stride of 2 were constructed (i.e. Subsequently, these convolutional kernels convolve the rows and columns of the feature map sequentially and downsample with a stride of 2 to generate four wavelet subbands, which are: .in As a low-frequency approximation component, it preserves the main structural information of the feature map at a coarse-grained level; , and For high-frequency components, high-frequency information of vertical edges, horizontal edges, and diagonal details is captured respectively, providing detailed information of the feature map at a fine-grained level.

[0092] To adapt to complex pathological features, this invention enhances sub-band features by fusing high- and low-frequency information. Specifically, firstly, in the high-frequency sub-band... and Incorporating low-frequency information generate The aim is to enable the network to utilize both low-frequency global information and high-frequency details in subsequent processing, avoiding feature fragmentation. Then, it undergoes MHA to learn and integrate multi-layer semantic information, enhancing the module's ability to represent complex scenes at multiple scales, which can be represented as:

[0093]

[0094]

[0095]

[0096] in , and These represent convolution operations with kernels of 3, 5, and 7, respectively. For pointwise convolution, It's the Sigmoid function. Then... and Fusion to generate attention map Furthermore, a channel attention mechanism is used to highlight channels that distinguish key features, thereby strengthening channels containing structural information in subsequent image reconstruction. This allows the model to utilize features more efficiently and optimizes overall performance. This process can be represented as:

[0097]

[0098]

[0099] in This represents pointwise convolution. Note that, to achieve a balance between computational efficiency and representational effectiveness, the... Expand the feature channels to twice their original size (i.e., High-frequency components Then, a 3x3 convolution is used to extract the detailed information, represented as... Because it also contains a significant amount of noise information, it was not utilized when generating the attention map. Finally, the original four wavelet subbands were concatenated along the channel dimension and then compressed into 2N channels through pointwise convolution, thus forming compressed information (e.g., high and low frequency information and noise) containing the four original wavelet subbands. Then will With the generated attention map Multiplying and performing residual connections to enhance the features of each subband can be represented as follows:

[0100]

[0101] in It also returns enhanced high-frequency components. Used to guide the upsampling module, providing it with frequency cues.

[0102] In the upsampling module, this invention introduces a dual upsampling module based on a fusion mechanism, WaveUp (Hwang S, Han D, Jung C, et al. WaveDH: Wavelet Sub-bands Guided ConvNet for Efficient Image Dehazing[J]. arXiv preprint arXiv:2404.01604, 2024.), whose architecture is as follows: Figure 5As shown. Specifically, Wave UP comprises two sub-modules: Inverse Discrete Wavelet Transform (IDWT) and PixelShuffle, as well as a fusion module. IDWT, leveraging the invertibility of wavelet transform, can accurately restore the high-frequency and low-frequency subbands decomposed in the downsampling stage to their original scale. Simultaneously, PixelShuffle enhances spatial resolution; however, PixelShuffle's representational capabilities are limited, potentially leading to irreversible loss of high-frequency details during image reconstruction. Therefore, the module uses the same-level high-frequency components (hi-subs) output from the wavelet decomposition stage as additional input to supplement and enhance the fineness of local features.

[0103] Given feature map Split it along the channel dimension into and Then, a 1x1 convolution operation is used to expand the... The feature channel count is then passed to the PixelShuffle layer. Meanwhile... The high-frequency components (hi-subs) returned from the wavelet decomposition module are merged and then fed into the IDWT layer:

[0104]

[0105]

[0106] Then and After concatenation, efficient feature fusion is achieved using collaborative attention. The Fusion module includes contrastive perceptual attention (CCA) and a slightly modified Fused-MBConv (FMBConv) layer. The interaction of the two upsampled features generates channel-level collaborative attention scores, which are used to recalibrate the concatenated features. This process can be represented as:

[0107]

[0108] Then, the feature map is mapped along the channel dimension. After splitting and performing element-wise addition, input the result into the FMBConv layer:

[0109]

[0110]

[0111] In the formula Output for the Wave Up module.

[0112] In the spatial attention module, due to the multi-scale decomposition and reconstruction of features after wavelet transform, features at different stages exhibit semantic and structural correlations and differences. To address this issue while further enhancing feature details, reducing noise interference, and highlighting key regions, this invention designs a unique spatial attention module, with the structure as follows: Figure 6 As shown.

[0113] In detail, for feature maps Global average pooling is used to obtain channel descriptors. Subsequently, the global feature Z and local features are calculated along the channel dimension. cosine similarity To measure the correlation and consistency between features, a "matching degree" basis is provided for subsequent feature fusion, reducing interference from invalid or conflicting features. This process is represented as follows:

[0114]

[0115]

[0116] In the formula This represents the eigenvector at position k. This represents the Sigmoid function. However, this operation may weaken the feature map. This invention addresses the risk of intensifying response to certain specific features (e.g., corners, edges) in a feature vector. To mitigate this issue, multi-dimensional statistical features are introduced: max pooling captures significant values ​​in the channel dimension; average pooling preserves the overall trend; standard pooling measures local feature fluctuations; and L2 norm measures the magnitude of the feature vector. A larger L2 norm indicates that the feature vector has greater "energy" and may contain more important information. These are respectively represented as... , and then with By combining these methods and employing multi-scale convolutions, the model can simultaneously acquire feature correlation and multi-dimensional statistical features. This not only reveals which features semantically match but also supplements global, local, and fluctuation information, constructing a more robust feature representation and improving the module's adaptability to complex scenarios and its feature parsing capabilities. The process can be represented as follows:

[0117]

[0118]

[0119] in , , This is a pointwise convolution. Then, for... After performing standardized operations to stabilize the distribution, a spatial attention map is generated after passing through the Sigmoid function. .

[0120] S5: Utilize the sliding window attention mechanism of the Swing Transformer to associate features of different regions within a Patch and model the correlation of local pathological patterns.

[0121] The feature map processed by MFHA is divided into 8×8 windows, and self-attention computation is performed within each window. The windows slide with a stride of 56 pixels, and cross-window attention is calculated to capture feature associations between different regions within the patch. After four Swing Transformer stages (each stage contains multiple Transformer blocks), the patch-level feature embedding is output. (L is the length of the effective feature sequence within the Patch, and D is the feature dimension).

[0122] S6: LongNet is used to integrate the local features of all patches to capture the long-distance dependencies of the whole slice (such as the heterogeneity of tumor cells in different regions and the global tissue structure pattern).

[0123] All Patch Arranged into a long sequence based on the original spatial location of WSI, it is input into LongNet to model the long-range dependencies of the entire slice. Since each WSI generates a large number of patches (1000-10000), the patch sequence is divided into several segments, each containing 64 patches. Subsequently, LongNet uses a combination of linear attention and local attention to perform attention calculations on the patch features within and between each segment, capturing the long-range dependencies of different regions. Finally, the features of all segments are integrated to output the global embedding vector of the entire slice. (D=768).

[0124] S7: Gene mutation classification is performed using the global embedding vector output by LongNet through a classification head.

[0125] enter Classification is achieved by mapping the dimension from 768 to the number of categories through a fully connected layer, with activation functions being Sigmoid (binary classification) or Softmax (multi-class classification).

[0126] Based on the above embodiments, the following evaluation and analysis are proposed:

[0127] This invention uses 2012 WSI (H&E staining) pathological images of lung cancer gene lesions provided by a medical center in Chongqing to evaluate the performance of MFHA. As shown in Table 1, this dataset is divided into 12 classes, containing various types of gene mutations. However, this dataset is highly imbalanced (for example, the EGFR genotype accounts for almost 50% of the entire dataset). Therefore, for the rationality and fairness of the experiment, this invention processes the dataset into two different binary classification datasets: cases with gene mutations and cases without gene mutations, and cases with EGFR gene mutations and cases without EGFR gene mutations, as shown in Tables 2 and 3. This processing aims to balance the number of classes as much as possible and to fully test the model performance.

[0128] Table 1 Initial WSI Lung Cancer Gene Mutation Dataset

[0129] Table 2. Division of Gene Mutation Datasets

[0130] Table 3 EGFR mutation dataset partitioning

[0131] Implementation details

[0132] The two-stage WSI diagnostic workflow consists of a patch-level encoder and a slide-level encoder. First, the input WSI pathological image is stripped of pathologically insignificant background regions using edge detection, retaining only the effective tissue blocks containing histological information. Then, the effective regions are segmented into several 224×224 patches with an MPP of 1.0. These patches are sequentially fed into the patch-level encoder (MFHA + Swin Transformer, named MFTrans) for feature extraction, outputting the patch-level embedding representation of the WSI. Subsequently, the patch-level embedding flows into the slide-level encoder (LongNet) to capture long-distance correlations across patch regions to generate a global embedding of the entire WSI. Finally, the global embedding is processed by a classification head to obtain the result. The classification head uses the Adam optimizer with an initial learning rate of 0.002, a batch size of 1, and employs a cross-entropy loss function for 40 epochs of two-fold cross-validation on a single 4×DCU 16G node.

[0133] Experimental results

[0134] To fully validate the model's performance in pathological images, this invention selected accuracy (ACC), balanced accuracy (BACC), area under the ROC curve (AUROC), and area under the PR curve (AUPRC) to evaluate the feature extraction performance of MFTrans and the baseline model CTranspath (Wang X, Yang S, Zhang J, et al. Transformer-based unsupervised contrastive learning for histopathological image classification[J]. Medical image analysis, 2022, 81: 102559.) on two datasets. Specifically, MFTrans and the baseline model extracted the same WSI pathological features using a patch-level encoder, and then performed classification using the same slide-level encoder. The results are shown in Tables 4 and 5, where AUROC_0 / 1 indicates the absence / presence of gene mutations, and AUPRC_0 / 1 also represents this. It is evident that MFTrans's two-fold cross-validation results exhibit less fluctuation, indicating more stable model performance. This may be related to the MFHA mechanism, which reduces the impact of single-scale feature fluctuations on overall performance through the complementary fusion of multi-band information.

[0135] In terms of overall performance, MFTrans outperforms the baseline model CTranspath across all evaluation metrics, with a more significant improvement on the gene mutation dataset. In this task, MFTrans's BACC is 0.6689, an improvement of approximately 1.59% compared to CTranspath (0.6531); its average AUROC reaches 0.7321, exceeding the baseline by approximately 1.2%; and its average AUPRC is improved by 2.69%. Particularly for positive samples with gene mutations (AUPRC_1), MFTrans achieves 0.8460, higher than CTranspath's 0.8212. This indicates that MFTrans not only exhibits better overall classification stability when capturing relevant pathological features, but also demonstrates a stronger precision-recall balance for positive samples. This is closely related to its designed MFHA mechanism, which decomposes low-frequency global semantics and high-frequency detail information through wavelet transform, combining multi-scale convolution and double upsampling fusion to enhance the perception of high-frequency components. This effectively enhances subtle morphological features related to mutations in pathological images (such as cell texture and edge structures) while filtering noise interference, thereby improving the model's accuracy in recognizing positive samples.

[0136] In the EGFR mutation dataset, MFTrans's advantage is relatively moderate but still clear, with an average improvement of 0.48% in BACC, 1.17% in AUROC, and 0.56% in AUPRC. It is worth noting that in the EGFR mutation dataset, although the sample ratio is more balanced compared to the gene mutation dataset (approximately 1:1.12), the magnitude and absolute value of the model performance improvement are lower than on the gene mutation dataset. Further analysis suggests this phenomenon may be related to the complexity of pathological features and the inherent differences in the task.

[0137] Specifically, in the gene mutation classification task, the positive class contains multiple gene mutation types. These mutations may be manifested through significant common pathological features (such as abnormal cell proliferation, disordered tissue structure, and nuclear morphological aberrations), and differ more significantly from the negative class (WT) samples without gene mutations in both global semantics and local details. MFTrans's MFHA mechanism effectively captures these common representations by decomposing and fusing low-frequency global information with high-frequency details using wavelet transform and learning multi-layer semantic features. Simultaneously, double upsampling fusion enhances the perception of high-frequency components, further amplifying mutation-related morphological differences, thus performing better in this task. EGFR mutations, on the other hand, are a single type of gene mutation, and their pathological features may be more subtle and have a higher degree of overlap with the features of negative class samples. For example, the pathological morphology of other mutations such as KRAS and ALK may have some similarity to EGFR mutations, increasing the difficulty of feature differentiation. Secondly, the negative class samples in the dataset are more complex in composition. This category includes wild-type (WT) samples and other gene mutation samples. The pathological features of these samples are not a single pattern, but rather exhibit diverse morphological features. Their feature distribution is more dispersed, which means that the model needs to identify a single target feature from diverse non-target features. The model needs to deal with more complex feature distribution overlap problems, resulting in the absolute value of the overall index being lower than that of the previous task.

[0138] Table 4 Results of the binary classification experiment for the presence or absence of gene mutations

[0139] (a) MFTrans (Ours) feature extraction and classification results

[0140] (b) CTranspath feature extraction and classification results

[0141] Table 5 Results of the binary classification experiment for the presence or absence of EGFR mutations

[0142] (a) MFTrans (Ours) feature extraction and classification results

[0143] (b) CTranspath feature extraction and classification results

[0144] Method generalization

[0145] As the core module of MFTrans, MFHA is not only designed for lung cancer gene mutation classification tasks, but also possesses plug-and-play generalization characteristics, allowing seamless integration into different network architectures (such as ResNet and Transformer series), and adaptability to multiple domain tasks including pathological images and natural images. To verify its cross-scenario generalization performance, generalization experiments were conducted on the Sipakmed pathology dataset (cervical cancer tissue pathology classification) and two natural image datasets (Cifar 10 and Mini ImageNet) using four evaluation metrics: Accuracy, F1 Score, Precision, and Recall. The performance differences between MFHA and mainstream attention mechanisms and related methods in recent years were compared.

[0146] Sipakmed is a classic publicly available dataset of cervical histopathology images, containing five classes of samples: Normal, Inflammation, Light Dysplasia, Moderate Dysplasia, and Severe Dysplasia / Carcinoma insitu, totaling approximately 4000 high-resolution pathological slide images. It is primarily used to evaluate the model's ability to distinguish subtle morphological differences in cervical lesions of varying degrees. The core challenges of this dataset lie in the blurred boundaries between lesion and normal tissue, the high morphological heterogeneity within the same type of lesion, and the susceptibility to noise interference introduced by uneven staining.

[0147] This experiment integrates MFHA into the backbone network based on ResNet and Vision Transformer, and compares it with other attention mechanisms and similar methods (wavelet transform) to verify its performance advantages. The results are shown in Table 6. ResNet-MFHA achieves an accuracy of 95.39% and an F1 score of 0.9541, which is better than mainstream attention mechanisms and traditional networks.

[0148] Specifically, the pathological features of cervical tissue (such as enlarged cell nuclei and disordered cell arrangement) are mainly reflected in high-frequency components, while the overall tissue structure belongs to low-frequency information. MFHA decouples features into low-frequency global (LL) and high-frequency edge detail (LH / HL) subbands through wavelet decomposition, then fuses different subbands and perceives multi-layer semantics. The dual upsampling fusion mechanism directionally enhances high-frequency details while filtering staining noise. The spatial attention module combines cosine similarity and multi-dimensional statistical features to capture cell density fluctuations while measuring the degree of matching between global and local features, thereby enhancing the features of the lesion area. In contrast, mechanisms such as CBAM rely only on spatial domain weighting and have difficulty distinguishing effective high-frequency details from noise. Although Transformer variants such as ViT can model global dependencies, their ability to resolve high-frequency microscopic features is insufficient, while MFHA precisely makes up for this deficiency.

[0149] Furthermore, the number of parameters and computational cost of MFHA increases very little compared to other methods, and can be almost ignored. This indicates that its performance improvement does not depend on additional computational costs, but rather on the efficient feature extraction capability of joint frequency-spatial domain modeling.

[0150] Table 6. Performance of MFHA on the Spiakmed dataset

[0151] Cifar 10 and Mini ImageNet are classic benchmark datasets in the field of natural image processing, used to validate a model's ability to learn common visual features. Cifar 10 contains 60,000 32×32 images across 10 common object classes, focusing on evaluating the model's basic recognition ability across different object categories. Mini ImageNet, on the other hand, contains 60,000 84×84 images across 100 fine-grained object classes, testing the model's ability to distinguish subtle differences within a class.

[0152] This experiment used ResNet as the backbone network and also embedded the attention mechanism. The results are shown in Tables 7 and 8. In Cifar 10, the ResNet18 architecture with MFHA integrated achieved a precision improvement of 1.07% compared to SGE (0.8533), and a 0.68% improvement compared to CBAM in the ResNet50 architecture. Similarly, in Mini ImageNet, MFHA-ResNet18 achieved a 0.29% improvement compared to other mechanisms, and a 0.83% improvement in the ResNet50 architecture.

[0153] As can be seen, MFHA still exhibits relatively good generalization performance in natural images. This is due to MFHA's unique hierarchical feature fusion enhancement mechanism. For example, when recognizing a "cat," the low-frequency component can capture the overall shape, while the high-frequency component captures the fur texture. Wavelet decomposition and multi-layer semantic learning enhance the synergy between the two, thereby improving model performance.

[0154] Table 7 shows the performance of MFHA on the Cifar 10 dataset.

[0155] Table 8. Performance of MFHA on the Mini ImageNet dataset

[0156] In addition, to verify the effectiveness of each core module of MFHA, this invention uses ResNet as the backbone network and conducts ablation experiments on the Sipakmed pathology dataset. The performance changes of the model are compared by gradually adding wavelet-guided upsampling and downsampling modules and spatial attention modules.

[0157] The experimental results are shown in Table 9. It can be seen that adding only the Wavelet module improves the model accuracy by 2.22% and the F1 score by 2.43%, indicating that wavelet decomposition's enhancement of high-frequency details and multi-scale fusion can effectively capture pathological features. Adding only the Spatial module improves the accuracy by 3.21% and the F1 score by 3.33%, verifying the effectiveness of spatial attention in constructing robust representations through cosine similarity and multi-dimensional statistical features. Combining Wavelet and Spatial further improves the accuracy to 95.56% and the F1 score to 0.9573, significantly higher than the individual modules. This demonstrates that the frequency domain features provided by wavelet decomposition provide a more accurate focusing basis for spatial attention, while spatial attention strengthens the weighting of key frequency domain regions, forming a synergistic optimization of frequency domain analysis and spatial focusing.

[0158] Table 9 Results of MFHA ablation experiments

[0159] Meanwhile, to intuitively demonstrate the feature learning mechanism of MFHA, this invention generates a heatmap using Gradient Weighted Class Activation Mapping (Grad-CAM) to compare the differences in the focus on key features between ordinary ResNet and ResNet-MFHA. The results are as follows: Figure 7 As shown.

[0160] Taking the "severe dysplasia" sample from the Sipakmed dataset as an example in pathological image analysis ( Figure 7(a, b) ResNet heatmaps are indistinct in distinguishing between lesion areas and normal glands, while MFHA heatmaps can locate lesion areas with enlarged and disordered cell nuclei. Taking the classification of "cat" in natural images as an example ( Figure 7 c) While ResNet's heatmap can cover the cat's head region, it relies solely on spatial domain convolution to extract features, making it difficult to distinguish effective information from noise in high-frequency details, thus leading to distraction. In contrast, MFHA separates the cat's facial contours (low frequency) and nose texture (high frequency) through wavelet decomposition, and then enhances the representation ability of high-frequency details through multi-scale fusion, enabling spatial attention to more clearly identify highly discriminative high-frequency features such as nose shape and eye edges.

[0161] The above are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and possess the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention, and these should also be considered within the scope of protection of the present invention. These will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

[0162] In summary:

[0163] This invention aims to address the core problems in existing lung cancer gene mutation classification technologies, such as insufficient mining of high-frequency details (e.g., cell nuclear morphology, local texture) in pathological images, low efficiency of frequency domain and spatial feature fusion, and difficulty in balancing global semantic understanding and local detail enhancement. First, it collects H&E-stained digital whole-slice images of lung cancer patients and corresponding gene mutation annotation information. Preprocessing is performed on WSI (Weighted Injection Sequence) images, including background filtering to remove non-organic regions and standardized segmentation to generate 224×224 pixel patches. The training and test sets are divided in a 7:3 ratio, and the class distribution is balanced through weighted resampling. Then, a multi-scale frequency domain hybrid attention mechanism is constructed. Key features are focused through channel selection, and low-frequency global structure and high-frequency detail information are decoupled using Haar discrete wavelet transform. Double upsampling is combined to achieve lossless feature dimensionality enhancement, and spatial attention is used to strengthen the representation of key regions, forming a feature optimization link of "frequency domain decoupling - detail enhancement - spatial focusing." Finally, MFHA is integrated into Swin. The Transformer constructs the MFTrans feature extraction network, which, combined with the LongNet encoder, models local pathological pattern associations within a patch and long-distance dependencies across the entire slice (such as tumor cell heterogeneity). This invention also verifies the effectiveness of each module of MFHA through ablation experiments and demonstrates the method's cross-scene adaptability in pathological and natural image tasks through generalization experiments. It provides an efficient technical path for the accurate classification of lung cancer gene mutations, helps standardize and reduce the cost of clinical lung cancer diagnosis, and enhances the application value of medical image processing in the field of gene detection.

[0164] It will be apparent to those skilled in the art that the present invention is not limited to the exemplary details described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as exemplary and not restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A lung cancer gene mutation classification method based on frequency domain multi-scale fusion, characterized in that: At least the following steps are included: S1: Collect lung cancer gene mutation-related datasets and use weighted resampling technology to divide the datasets into training and testing sets in a 7:3 ratio; S2: The background region of the histological information in the digitized whole slice image is stripped away by the edge detection algorithm, retaining the effective pathological region image containing tumor tissue and normal glands; S3: Divide the background-filtered WSI sub-image into multiple local image blocks of a fixed size, i.e., Patch blocks; S4: Using the multi-scale frequency domain hybrid attention mechanism, i.e., the MFHA mechanism, pathological features with both global structure and high-frequency details are extracted from each patch. S5: By using the sliding window attention mechanism of the Swing Transformer to associate features of different regions within the patch, the correlation of local pathological patterns is modeled. S6: LongNet is used to integrate the local features of all patch blocks to capture the long-distance dependencies of the entire slice; S7: Based on the global embedding vector output by the LongNet, lung cancer gene mutations are classified using a classification head.

2. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided by claim 1, characterized in that: The lung cancer gene mutation related dataset contains H&E staining WSI of lung cancer patients and corresponding gene mutation annotation information. The gene mutation annotation information includes whether a gene mutation exists and the category label of a specific type of gene mutation. After obtaining the lung cancer gene mutation-related dataset, weighted resampling was used to balance the class distribution, and the dataset was divided into training sets in a 7:3 ratio. With test set ; The sampling weight formula is as follows: in, The number of samples in category c; This represents the total number of samples.

3. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided according to claim 2, characterized in that: The background area without histological information includes the blank area of ​​the slide. The image of the effective pathological area is obtained by comparing it with the original WSI through an edge detection algorithm and removing the area where the background pixel ratio exceeds a preset threshold.

4. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided according to claim 3, characterized in that: The patch blocks obtained in S3 are all 224×224 pixels in size, and each patch block contains a tissue region that reflects the microscopic pathological morphology.

5. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided according to claim 4, characterized in that: Both S2 and S3 employ background filtering and image segmentation techniques. The background filtering and image segmentation process includes at least the following steps: For each WSI pathology image, the Canny edge detection algorithm is used to remove tissue-free areas and retain the effective pathological areas, as shown below: in, These are the original WSI pixel values; The standard deviation is the Gaussian filter value; low and high are the edge detection thresholds. Indicates the effective area pixels; The effective region was then divided into image blocks of 224×224 pixels to meet the requirements of effective organization pixel ratio. , represented as: in, For the first Image blocks.

6. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided according to claim 5, characterized in that: The MFHA mechanism includes a channel filtering module, a wavelet decomposition module, an upsampling module, and a spatial attention module. It decouples the low-frequency global information and high-frequency detail information of pathological images through discrete wavelet transform, and performs multi-scale enhancement and fusion on the high-frequency detail information. In the channel filtering module, the input feature map Perform global average pooling, where H and W are the spatial resolutions and C is the number of channels, compressing the spatial information of each channel into... Channel descriptor Then, channel dependencies are learned through a channel transformation function with a dimensionality reduction ratio r, and channel weight vectors are generated. Finally, based on Filter the top N high-weight channels to generate sub-feature maps. ; In the wavelet decomposition module, Haar discrete wavelet transform is used to... The decomposition process involves constructing four convolutional kernels with a stride of 2 using low-pass and high-pass filters, generating four wavelet subbands: a low-frequency approximate component LL, a high-frequency horizontal component HL, a high-frequency vertical component LH, and a high-frequency diagonal component HH. Subsequently, LL is fused with HL and LH respectively to generate… , This enhances the contextual coherence of high-frequency subbands; then, it undergoes multi-scale hybrid attention... , Multi-scale convolutions with kernel sizes of 3, 5, and 7 are performed respectively, and attention maps are generated by pointwise convolution and the Sigmoid function. At the same time, the method returns high-frequency component combinations hi-subs for upsampling guidance. In the upsampling module, the output Y of the wavelet decomposition module is split along the channel dimension into... , ,right Perform a 1×1 convolution to expand the number of channels to 4N, and input the PixelShuffle layer to improve spatial resolution; After fusing with hi-subs, the input is fed into an inverse discrete wavelet transform layer to restore the original image size. Then, the PixelShuffle output is concatenated with the IDWT output, and feature fusion is achieved through contrastive perceptual attention and an improved Fused-MBConv layer, outputting a feature map with restored size. ; In the spatial attention module, the feature map Perform global average pooling to obtain the global feature Z, and calculate Z with... Cosine similarity of local features To measure the matching degree between global and local features; and at the same time... Max pooling, average pooling, standard pooling, and L2 norm are performed respectively to extract multi-dimensional statistical features; Subsequently, multi-dimensional statistical features are combined with cosine similarity and then multi-scale convolution is performed to generate an attention map. Element-wise multiplication with the original feature map is performed to further enhance the perception of image details and filter image noise.

7. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided by claim 6, characterized in that: The sliding window attention mechanism of the Swing Transformer achieves the association of features in different regions within the Patch block by moving between different windows and calculating attention. The window size is adaptively adjusted or set to a preset value based on the size of the patch block and the distribution of pathological features. The application of the Swin Transformer includes at least the following steps: After MFHA processing, the feature image enters the 4-stage Swing Transformer layer, which divides the feature image into 8×8 windows, and performs self-attention calculation within each window. The window slides in 56-pixel increments, and cross-window attention is calculated to capture feature associations between different regions within the patch. Finally, the Patch-level feature embedding vector is output.

8. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided according to claim 7, characterized in that: The LongNet captures long-distance dependencies such as the heterogeneity of tumor cells in different regions and global tissue structure patterns across the entire slice by performing long-sequence modeling on the local features of each patch block. The method of integrating local features of all patch blocks using LongNet includes at least the following steps: All Patch-level feature embedding vectors are arranged into a long sequence according to their original WSI spatial positions and then input into LongNet to model the long-distance dependency of the whole slice. Since each WSI generates a large number of patches, the patch sequence is divided into several segments; LongNet then uses a combination of linear attention and local attention to perform attention calculations on the patch features within and between each segment, capturing long-distance dependencies in different regions. Finally, the features of all segments are integrated to output the global embedding vector of the entire slice.

9. The lung cancer gene mutation classification method based on frequency domain multi-scale fusion guided by claim 8, characterized in that: The classification head in S7 uses a fully connected layer combined with an activation function, takes the global embedding vector as input, and outputs the classification result of lung cancer gene mutations. The activation function used in binary classification is the Sigmoid function, and the activation function used in multi-class classification is the Softmax function.

Images