Linear transformer general lesion recognition method based on multiple perception and context guidance

Abstract

The application belongs to the technical field of medical image processing, and particularly relates to a linear Transformer general lesion recognition method based on multiple perception and context guidance. The method extracts multi-scale features of a medical CT image through a backbone network and obtains edge gradient features in parallel, compensates for the loss of lesion boundary information through an edge perception feature enhancement module, realizes global feature modeling under linear complexity through a polarity perception feature interaction module, generates high-discrimination multi-scale feature maps by using a context guidance feature pyramid network, and finally optimizes the model by combining the Hungarian algorithm and a joint loss function based on an end-to-end detection architecture of set prediction. The application effectively solves problems such as fuzzy lesion boundary, trade-off between feature interaction and calculation efficiency, and weak separation degree of lesions and backgrounds, realizes double improvement of calculation efficiency and detection accuracy on massive medical image data, and provides reliable support for clinical precise auxiliary diagnosis.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a general lesion recognition method based on multiple perception and context guidance using the linear Transformer. Background Technology

[0002] Automated lesion detection in medical imaging is of paramount value in clinical practice, serving as a crucial step in early diagnosis, accurate staging, treatment planning, and prognostic assessment. In traditional clinical procedures, radiologists must meticulously analyze numerous medical images to identify potential lesions. This process is not only time-consuming and labor-intensive but also highly dependent on the physician's professional experience, inherently subjective, and prone to missed or misdiagnosed lesions due to fatigue or human error. Therefore, developing an efficient and accurate automated lesion detection framework to alleviate the burden on physicians and improve diagnostic accuracy is of utmost urgency.

[0003] With the rapid development of deep learning technology, convolutional neural network-based methods have achieved great success in object detection in natural images. However, medical images have unique characteristics, such as low contrast, diverse lesion morphologies, and blurred boundaries, which often makes it difficult to obtain satisfactory results by directly applying natural image detection algorithms. Although some studies have proposed improved convolutional neural network methods for medical images, these methods still have shortcomings in capturing long-range dependencies and modeling global context due to the limitations of the local receptive field of convolution operations.

[0004] In recent years, the Transformer architecture has demonstrated great potential in the field of computer vision due to its powerful global modeling capabilities. DETR (Detection Transformer) methods, by introducing an end-to-end ensemble prediction paradigm, eliminate the complexity of anchor box design and non-maximum suppression in traditional detectors. However, directly applying existing Transformer detection methods to medical lesion detection still faces three main technical bottlenecks and challenges: inaccurate localization due to blurred lesion boundaries, the trade-off between feature interaction and computational efficiency, and weak separation between lesions and background features.

[0005] Achieving efficient and accurate detection of lesions in different organs is of great significance for clinical disease analysis. By combining edge perception enhancement and polarity perception interaction technologies, efficient, rapid, and accurate lesion detection of CT images can further enhance precise auxiliary diagnosis, provide reliable auxiliary evidence for less experienced doctors, reduce the workload of older doctors, assist doctors in quickly and efficiently conducting accurate disease assessments, and minimize missed diagnoses caused by human factors.

[0006] Therefore, this invention aims to provide a general lesion detection method based on multi-sensor and context-guided linear Transformer, so as to achieve a dual improvement in computational efficiency and detection accuracy on massive medical image data. Summary of the Invention

[0007] The purpose of this invention is to provide a general lesion identification method based on multiple perception and context guidance using the linear Transformer, in order to solve the technical problems existing in the prior art and achieve a dual improvement in computational efficiency and detection accuracy on massive medical image data.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0009] A general lesion identification method based on multiple sensing and context guidance using the linear Transformer includes the following steps:

[0010] S1: Configure the backbone network and edge extraction operator, extract multi-scale features of medical CT images through the backbone network, extract shallow high-resolution edge gradient features in parallel, and construct basic feature representation;

[0011] S2: Set up an edge-aware feature enhancement module to perform spatial alignment and nonlinear transformation on shallow edge features and inject them into deep semantic features to compensate for the lack of lesion boundary information.

[0012] S3: Use the polarity-aware feature interaction module to build an encoder, and perform global modeling of the enhanced features in linear complexity through polarity decomposition and dual-mode interaction;

[0013] S4: Design a context-guided feature pyramid network to generate highly discriminative multi-scale feature maps through relational context extraction, dynamic interpolation fusion, and semantic guidance;

[0014] S5: Establish an end-to-end detection architecture based on ensemble prediction, use the enhanced feature map to guide the iterative update of object query, perform bipartite graph matching through the Hungarian algorithm, and optimize model parameters by combining the joint loss function of classification and regression to achieve accurate lesion localization.

[0015] Preferably, the method is implemented through a system consisting of a backbone network, an edge-aware feature enhancement module, a polarity-aware feature interaction module, a context-guided feature pyramid network, and a detection head; the backbone network is used to extract basic features, the edge-aware feature enhancement module is used to enhance boundary information, the polarity-aware feature interaction module is used for complete feature interaction modeling under linear complexity, and the context-guided feature pyramid network is used for multi-scale fusion.

[0016] Preferably, the specific process of step S1 is as follows:

[0017] S11: For medical image I, a pre-trained ResNet-r18 network is used as the backbone to extract feature maps at four different stages, denoted as... ,include , , , ,in This indicates that the downsampling factor relative to the original image is [value missing]. Feature map;

[0018] S12: Select shallow features As an edge source, its horizontal gradient is calculated using the Sobel operator. and vertical gradient Thus, the edge feature map is obtained. ;

[0019] ;

[0020] S13: Extract the edge feature map with deep features , , Together they serve as input for subsequent modules.

[0021] Preferably, the specific process of step S2 is as follows:

[0022] S21: Based on edge feature map Construct a multi-scale edge generator; for deep features , , Max pooling operation is used to... Downsampling to With the same spatial resolution, adaptive edge features are obtained. ;

[0023] ;

[0024] in This represents a multi-scale edge information generator;

[0025] S22: Perform cross-level feature injection operation, injecting deep features Adaptive edge features The features are concatenated along the channel dimension and then subjected to multiple convolutions to obtain features with rich edge information.

[0026] ;

[0027] in, This indicates a concatenation operation. Indicates the core size is Convolution operation, Representing scale Enhanced features.

[0028] Preferably, the specific process of step S3 is as follows:

[0029] S31: The enhanced features of the obtained edge perception enhancement module Generate the query matrix using linear projection. Key matrix Sum matrix ;

[0030] , ;

[0031] , ;

[0032] in, The dimension is The tensor, in which Indicates batch size, Represented as sequence length, Represented as feature dimension, , , , These four are different learnable weight matrices. For the gated vector, This indicates a spatial downsampling operation. downsampling rate ;

[0033] S32: Reshape Q, K, and V into a form with... h The multi-head format enables parallel processing across multiple representation subspaces, with each head having a dimension of [missing information]. ,get , ;

[0034] in , , These are the query matrix, key matrix, and value matrix of a single reshaped header, respectively. The dimension is tensor, To reshape the number of attention heads, The dimension of a single head;

[0035] S33: Yes and Polarity decomposition is performed using the ReLU activation function to explicitly separate the positive and negative components:

[0036] , ;

[0037] , ;

[0038] in, , These represent the positive and negative components of a single query, respectively. , These represent the positive and negative components of a single bond, respectively. This is the activation function. It is a learnable scaling factor used to control the input range. For learnable parameters, This is the activation function. It is a learnable index used to adjust the nonlinear strength. This is the scaling factor. For the Sigmoid function, These are learnable weights;

[0039] S34: Connect the positive and negative components to construct a dual-mode interactive feature descriptor. By preserving polarity information, two complementary matching modes are achieved.

[0040] ;

[0041] in Same-polarity queries representing the same-polarity matching pattern. A heteropolar query representing a heteropolar matching pattern. This represents the concatenated key matrix;

[0042] S35: Calculate the average bond representation And the normalization factor is derived. :

[0043] ;

[0044] ;

[0045] in This is the transpose of the average bond. It is a small constant used for numerical stability;

[0046] S36: Construct two different feature interaction modes: homopolar interactions that capture feature consistency, and heteropolar interactions that capture feature contrast, value matrix Divided along the last dimension , The two interaction modes are calculated separately using a linear attention mechanism:

[0047] ;

[0048] in Indicates attention output, This indicates two modes: same polarity and opposite polarity. express The transpose of . Indicates the result of being segmented or . Normalization factor or . It is a normalization factor used for heteropolar interactions;

[0049] S37: Will come from all heads and To enhance the local spatial context, a deep convolution is introduced on the value features for concatenation. The aggregated features are then modulated by a gated signal and projected onto the output space.

[0050] ;

[0051] in, , It is the output projection matrix. This indicates a splicing operation, starting from the first... Height to the Size. Represents depthwise convolution. It's a gating signal from S31. This indicates element-wise multiplication;

[0052] S38: Obtained The output of the frequency domain sensing feedforward neural network For use in the subsequent feature pyramid network:

[0053] ;

[0054] in, This is the final output. Representation layer normalization, Through Fast Fourier Transform and Learnable Weight Matrix Inverse Fast Fourier Transform and convolution are applied to frequency domain modulation to capture low-frequency structural information and high-frequency detail information.

[0055] Preferably, the specific process of step S4 is as follows:

[0056] S41: Obtain the feature map , , The components are unified to the same spatial dimension, then concatenated, and global context information is extracted using the relational context module before being reallocated back to the original dimension. The formula is as follows:

[0057] ;

[0058] in, To extract global context information through the relational context module Layer features. Indicates adaptive average pooling. Indicates channel splicing. Refers to three stacked relational context modules. Indicates a channel separation operation;

[0059] S42: The spatial correlation between lesions and the background is captured through horizontal and vertical pooling operations, while attention mechanisms are combined to enhance lesion features and suppress background noise. The formula is as follows:

[0060] ;

[0061] in, For rectangular context attention, The features to be input, i.e. , , . and These represent horizontal and vertical pooling operations, respectively. This indicates element-wise summation. This indicates that asymmetric grouped convolution is followed by batch normalization and ReLU activation. This represents the Sigmoid activation function;

[0062] S43: During the top-down propagation of the feature pyramid, a dynamic interpolation and addition strategy is used to achieve multi-scale fusion, providing dynamic adjustment of high-level features. The resolution is adjusted to match the target scale, and the aligned features are compared with the low-level features. The formula for element-wise addition is as follows:

[0063] ;

[0064] in, The first element after element-wise addition Layer features. This indicates a dynamic channel adjustment operation. This indicates a dynamic interpolation operation;

[0065] S44: It is the feature obtained by extracting global context information from the previously stacked RCM, and the final Depend on and The formula is obtained by multiplying each element together, as follows:

[0066] ;

[0067] S45: In bottom-up path aggregation, through high-level features A position-sensitive attention map is generated, selectively enhancing low-level features while suppressing redundant information. The specific formula is as follows:

[0068] ;

[0069] in, Represents the final enhanced version of the first Layer features. and This represents 1×1 convolutional feature maps used for different purposes. This indicates an upsampling operation.

[0070] Preferably, the specific process of step S5 is as follows:

[0071] S51: Initialize the object query using an uncertainty minimization query selection strategy, that is, use the auxiliary prediction head to simultaneously evaluate the class confidence of the output features and the quality of the bounding box localization, and select the one with the highest comprehensive score. N Using these features as initial object queries and reference points, the queries are input into the Transformer decoder and interact with the feature maps. After iterative updates through multiple decoding layers, the multilayer perceptron predicts the lesion category score and bounding box coordinates corresponding to each query, as shown in the following formula:

[0072] ;

[0073] in, To predict the output set, This is the preset number of queries. Indicates the first The probability of a query belonging to a category. This represents the normalized bounding box parameters. For predicting the head, For decoder, This is the enhanced feature map that is the final output of step S4;

[0074] S52: The Hungarian algorithm is used for globally optimal bipartite graph matching. By minimizing the matching cost matrix, the optimal permutation between the predicted set and the true set is found. The formula is as follows:

[0075] ;

[0076] in, This represents the parameter that minimizes the objective. For arrangement space, Taking into account both the differences in classification probabilities and the positional deviation of the bounding boxes, this ensures that each real lesion matches only one optimal predicted box. For the first A real goal In order to be with the first A prediction that matches a real target. Permutation function Functions and Index The result;

[0077] S53: Based on the optimal matching result, construct a joint loss function to perform end-to-end optimization of the model. The specific formula is as follows:

[0078] ;

[0079] in, This is the total loss function. , , These represent classification loss, L1 loss, and Giou loss, respectively. Indicates the first The category labels of each predicted bounding box. The indicator function indicates that the regression loss is calculated only for queries that match the true target. This is the balance coefficient for each component loss. Indicates that the first one is assigned The predicted result of the actual target. This represents the probability predicted by the model. For the first The true bounding box of a real target. The bounding box predicted by the model.

[0080] The beneficial effects of this invention include:

[0081] 1. Accurately capture lesions with blurred boundaries: Through the edge perception feature enhancement module, the Sobel operator is used to extract shallow high-resolution edge gradient features. Combined with multi-scale edge generation and convolution fusion strategies, shallow edge details are efficiently injected into deep semantic features. This makes up for the lack of spatial details in deep features from the source, effectively solving the problem of inaccurate localization caused by blurred lesion boundaries in medical CT images, and greatly improving the perception and localization accuracy of small and blurred lesions.

[0082] 2. Complete preservation of key feature information: By overcoming the limitations of traditional linear attention mechanisms that lose key information due to ignoring negative features, polarity-aware feature interaction is used to achieve explicit separation of positive and negative components of features. A dual-mode interaction path of same polarity (capturing similarity within lesions) and opposite polarity (capturing differences at lesion boundaries) is constructed. While maintaining linear complexity, feature polarity information is completely preserved, enabling the model to keenly capture subtle contrasts between lesions and the background, and significantly enhancing feature discrimination ability.

[0083] 3. Enhanced separation between lesions and background: The context-guided feature pyramid network extracts global anatomical context information using the relational context module. Through top-down dynamic interpolation fusion and bottom-up semantic guidance mechanism, it achieves accurate fusion and dynamic calibration of multi-scale features, effectively suppresses background noise, enhances the feature separation between lesions and complex backgrounds, avoids feature confusion, and provides high-discriminative feature support for accurate detection.

[0084] 4. An end-to-end detection architecture based on set prediction is adopted. Object queries are initialized through an uncertainty-minimizing query selection strategy, and global optimal bipartite graph matching is achieved by combining the Hungarian algorithm. This eliminates the need for anchor box design and non-maximum suppression steps in traditional detection methods, simplifying the detection process and further improving the overall detection efficiency. Attached Figure Description

[0085] Figure 1 This is a flowchart illustrating the general lesion identification method of the linear Transformer based on multiple perception and context guidance of the present invention.

[0086] Figure 2 This image shows a comparison of the detection results of the linear Transformer general lesion identification method based on multiple perception and context guidance of the present invention with other detection methods on the DeepLesion dataset. GT represents the ground truth detection annotation.

[0087] Figure 3 This is a comparison chart of the detection results of the linear Transformer general lesion identification method based on multiple perception and context guidance of the present invention on the DeepLesion dataset with the remaining detection methods.

[0088] Figure 4 This is a comparison of the detection results of the linear Transformer general lesion identification method based on multiple perception and context guidance of the present invention on the LUNA16 dataset with other detection methods.

[0089] Figure 5This is a comparison chart of the detection results of the linear Transformer general lesion identification method based on multiple perception and context guidance of the present invention on the LUNA16 dataset with the remaining detection methods. Detailed Implementation

[0090] The following is in conjunction with the appendix Figures 1-5 The present invention will be further described in detail below:

[0091] Example 1

[0092] See appendix Figure 1 As shown, the general lesion identification method based on multiple perception and context guidance of the linear Transformer includes the following steps:

[0093] S1: Configure the backbone network and edge extraction operator, extract multi-scale features of medical CT images through the backbone network, extract shallow high-resolution edge gradient features in parallel, and construct basic feature representation;

[0094] S2: Set up an edge-aware feature enhancement module to perform spatial alignment and nonlinear transformation on shallow edge features and inject them into deep semantic features to compensate for the lack of lesion boundary information.

[0095] S3: Use the polarity-aware feature interaction module to build an encoder, and perform global modeling of the enhanced features in linear complexity through polarity decomposition and dual-mode interaction;

[0096] S4: Design a context-guided feature pyramid network to generate highly discriminative multi-scale feature maps through relational context extraction, dynamic interpolation fusion, and semantic guidance;

[0097] S5: Establish an end-to-end detection architecture based on ensemble prediction, use the enhanced feature map to guide the iterative update of object query, perform bipartite graph matching through the Hungarian algorithm, and optimize model parameters by combining the joint loss function of classification and regression to achieve accurate lesion localization.

[0098] In this embodiment, the method is implemented through a system consisting of a backbone network, an edge-aware feature enhancement module, a polarity-aware feature interaction module, a context-guided feature pyramid network, and a detection head. The backbone network is used to extract basic features, the edge-aware feature enhancement module is used to enhance boundary information, the polarity-aware feature interaction module is used for complete feature interaction modeling under linear complexity, and the context-guided feature pyramid network is used for multi-scale fusion.

[0099] This invention organically embeds the edge-aware feature enhancement module, polarity-aware interaction mechanism, and context-guided pyramid architecture into the core feature extraction and fusion path of the general lesion detection model, realizing a complete technical closed loop from bottom-level edge detail injection to high-level all-polarity feature interaction and then to global anatomical context guidance.

[0100] In the specific implementation process, addressing the pain point of blurred lesion boundaries in medical CT images, an information "highway" from shallow to deep layers was first constructed through an edge perception feature enhancement module. Unlike traditional networks that directly discard high-frequency information during downsampling, this approach uses the Sobel operator and a multi-scale generator to explicitly extract and inject gradient edge features, thereby compensating for the lack of spatial details in deep semantic features from the source and ensuring the model's ability to perceive small and blurred lesions.

[0101] Furthermore, addressing the theoretical shortcomings of existing linear attention mechanisms that lose key information due to forced nonnegative approximation, the proposed polarity-aware feature interaction module establishes a rigorous polarity decomposition and dual-mode interaction system. By introducing learnable scaling factors and exponents to explicitly separate the positive and negative components of features, and constructing dual-channel interaction paths of the same polarity (consistency) and opposite polarity (contrast), it successfully preserves the symbolic information of features while maintaining linear computational complexity. This breaks through the expression bottleneck of traditional linearization methods, enabling the model to capture subtle contrasts between lesions and background with the same sensitivity as a standard force, while enjoying the inference speed advantage of linear models.

[0102] Finally, to address the feature confusion issue in complex anatomical contexts, a context-guided feature pyramid network was used to dynamically calibrate local features based on global anatomical priors. Long-distance dependencies extracted using the relational context module were redistributed to multi-scale feature layers, and a top-down dynamic interpolation and bottom-up semantic guidance mechanism were employed to "remove falsehoods from true features" in the feature maps. This mechanism ensures that the final feature representation used for prediction contains both rich local texture and global anatomical location awareness.

[0103] The specific process of step S1 is as follows:

[0104] S11: For a given medical image I, using a pre-trained ResNet-r18 network as the backbone, feature maps at four different stages are extracted, denoted as... ,include , , , ,in This indicates that the downsampling factor relative to the original image is [value missing]. The feature map.

[0105] S12: Select shallow features As an edge source, its horizontal gradient is calculated using the Sobel operator. and vertical gradient Thus, the edge feature map is obtained. The specific formula is as follows:

[0106] ;

[0107] S13: Extract the edge feature map with deep features , , Together they serve as input for subsequent modules.

[0108] In this embodiment, the specific process of step S2 is as follows:

[0109] S21: Edge feature map obtained in step S1 Construct a multi-scale edge generator; for deep features , , Max pooling operation is used to... Downsampling to With the same spatial resolution, adaptive edge features are obtained. ;

[0110] ;

[0111] in This represents a multi-scale edge information generator;

[0112] S22: Perform cross-level feature injection operation; inject deep features Adaptive edge features The features are concatenated along the channel dimension and then subjected to multiple convolutions to obtain features with rich edge information.

[0113] ;

[0114] Wherein represents splicing operation, Indicates the core size is The convolution operation, and For scale Enhanced features.

[0115] Example 2

[0116] Based on Example 1, the specific process of step S3 is as follows:

[0117] S31: Obtain the enhanced features of the edge perception enhancement module. First, the query matrix, key matrix, and value matrix are generated through linear projection, as shown in the following formula;

[0118] , ;

[0119] , ;

[0120] in, The dimension is The tensor, in which Indicates batch size, Represented as sequence length, Represented as feature dimension, , , , The four are different learnable weight matrices. This is the gated vector. This indicates a spatial downsampling operation. downsampling rate ;

[0121] S32: In order to enable parallel processing across multiple representation subspaces, Q, K, and V are reshaped into representations with... h The format is a multi-head format, with each head having dimensions of [dimensions missing]. Finally obtained , ;

[0122] in , , These are the query matrix, key matrix, and value matrix of a single reshaped header, respectively. The dimension is tensor, To reshape the number of attention heads, The dimension of a single head;

[0123] S33: In order to fully utilize the polarity information in the lesion characteristics, and The ReLU activation function is applied to perform polarity decomposition, thereby explicitly separating the positive and negative components. The specific formula is as follows:

[0124] , ;

[0125] , ;

[0126] in, , These represent the positive and negative components of a single query, respectively. , These represent the positive and negative components of a single bond, respectively. This is the activation function. It is a learnable scaling factor used to control the input range. For learnable parameters, This is the activation function. It is a learnable index used to adjust the nonlinear strength. This is the scaling factor. For the Sigmoid function, These are learnable weights;

[0127] S34: Connect the positive and negative components to construct a dual-mode interactive feature descriptor. By preserving polarity information, two complementary matching modes are achieved, as shown in the following formula:

[0128] ;

[0129] in Same-polarity queries representing the same-polarity matching pattern. A heteropolar query representing a heteropolar matching pattern. This represents the concatenated key matrix;

[0130] S35: Calculate the average bond representation And the normalization factor is derived. The specific formula is as follows:

[0131] ;

[0132] ;

[0133] in This is the transpose of the average bond. It is a small constant used for numerical stability.

[0134] S36: Construct two different feature interaction modes: a homopolar interaction that captures feature consistency (similarity within lesions), and a heteropolar interaction that captures feature contrast (differences at lesion boundaries). Value matrix Divided along the last dimension , These two interaction modes are calculated separately using a linear attention mechanism, with the specific formulas as follows:

[0135] ;

[0136] in, Indicates attention output, This indicates two modes: same polarity and opposite polarity. express The transpose of . Indicates the result of being segmented or . Normalization factor or . It is a normalization factor used for heteropolar interactions;

[0137] S37: After bipolar interaction, from all heads and The features are concatenated to integrate homopolar and heteropolar information. To enhance the local spatial context, a deep convolution is introduced on the value features, which captures fine-grained patterns complementary to global attention. The aggregated features are then modulated by a gated signal and projected onto the output space, as shown in the following formula:

[0138] ;

[0139] in, , It is the output projection matrix. This indicates a splicing operation, starting from the first... Height to the Size. Represents depthwise convolution. It's a gating signal from S31. This indicates element-wise multiplication.

[0140] S38: Obtained The output of the designed frequency domain sensing feedforward neural network The specific formula for the subsequent feature pyramid network is as follows:

[0141] ;

[0142] This is the final output. , Representation layer normalization, Through Fast Fourier Transform and Learnable Weight Matrix Inverse Fast Fourier Transform and convolution are applied to frequency domain modulation to capture low-frequency structural information and high-frequency detail information.

[0143] Example 3

[0144] Based on Example 1 or Example 2, the specific process of step S4 is as follows:

[0145] S41: Obtain the feature map , , They are unified to the same spatial dimension, then concatenated, and then global context information is extracted through the relational context module before being redistributed back to the original dimension. The specific formula is as follows:

[0146] ;

[0147] in, To extract global context information through the relational context module Layer features. Indicates adaptive average pooling. Indicates channel splicing. Refers to three stacked relational context modules. This indicates a channel separation operation.

[0148] S42: The key component of the relational context module designed to extract global contextual information is rectangular contextual attention (RCA). It captures the spatial correlation between lesions and the background through horizontal and vertical pooling operations, while combining an attention mechanism to enhance lesion features and suppress background noise. The specific formula is as follows:

[0149] ;

[0150] in, For rectangular context attention, The features to be input, i.e. , , . and These represent horizontal and vertical pooling operations, respectively. This indicates element-wise summation. This indicates that asymmetric grouped convolution is followed by batch normalization and ReLU activation. This represents the Sigmoid activation function.

[0151] S43: During the top-down propagation of the feature pyramid, a dynamic interpolation and addition strategy is employed to achieve multi-scale fusion. This method dynamically adjusts high-level features. The resolution is adjusted to match the target scale, and then the aligned features are compared with the low-level features. The specific formula for element-by-element addition is as follows:

[0152] ;

[0153] in, The first element after element-wise addition Layer features. This indicates a dynamic channel adjustment operation. This indicates a dynamic interpolation operation.

[0154] S44: These features are obtained by extracting global context information from the previously stacked RCM. The final... Depend on and The formula is obtained by multiplying each element together, as follows:

[0155] ;

[0156] S45: In bottom-up path aggregation, a semantically guided fusion mechanism is designed, which utilizes high-level features... A position-sensitive attention map is generated, which selectively enhances low-level features to preserve key structural information while suppressing redundant information. The specific formula is as follows:

[0157] ;

[0158] in, Represents the final enhanced version of the first Layer features. and This represents 1×1 convolutional feature maps used for different purposes. This indicates an upsampling operation.

[0159] In this embodiment, the specific process of step S5 is as follows:

[0160] S51: Initialize the object query using an uncertainty minimization query selection strategy. This involves simultaneously evaluating the class confidence and bounding box localization quality of the features output from step S4 using an auxiliary prediction head, and selecting the option with the highest overall score. N These features serve as initial object queries and reference points. Subsequently, these queries are input into the Transformer decoder and interact with the feature map. After iterative updates across multiple decoding layers, a Multilayer Perceptron (MLP) predicts the lesion category score and bounding box coordinates corresponding to each query. The process is described below:

[0161] ;

[0162] in, To predict the output set, This is the preset number of queries. Indicates the first The probability of a query belonging to a category. This represents the normalized bounding box parameters. For predicting the head, For decoder, This is the enhanced feature map that is the final output of step S4;

[0163] S52: To address the label assignment problem between predicted bounding boxes and actual lesions, the Hungarian algorithm is employed for globally optimal bipartite graph matching. This algorithm finds the optimal permutation between the predicted set and the actual set by minimizing the matching cost matrix. :

[0164] ;

[0165] in, This represents the parameter that minimizes the objective. For arrangement space, Taking into account both the differences in classification probabilities and the positional deviation of the bounding boxes, this ensures that each real lesion matches only one optimal predicted box. For the first A real goal In order to be with the first A prediction that matches a real target. Permutation function Functions and Index The result;

[0166] S53: Based on the optimal matching results, a joint loss function is constructed to perform end-to-end optimization of the model. This loss function consists of a weighted average of classification loss and regression loss, aiming to simultaneously improve the accuracy of lesion identification and the precision of lesion localization. The specific formula is as follows:

[0167] ;

[0168] in, This is the total loss function. , , These represent classification loss, L1 loss, and Giou loss, respectively. Indicates the first The category labels of each predicted bounding box. The indicator function indicates that the regression loss is calculated only for queries that match the true target. This is the balance coefficient for each component loss. Indicates that the first one is assigned The predicted result of the actual target. This represents the probability predicted by the model. For the first The true bounding box of a real target. The bounding box predicted by the model.

[0169] Through the collaborative work of the three core modules mentioned above, this invention successfully constructs an end-to-end universal lesion detection framework that balances computational efficiency and detection accuracy, solving the problems of blurred lesion boundaries, difficulty in balancing feature interaction integrity and computational efficiency, and weak separation between lesions and background in existing technologies.

[0170] The trained model will ultimately predict the detection results as follows: Figure 1 As shown, different types of lesions are marked with different colored boxes.

[0171] See Figure 2 , Figure 3 , Figure 4 , Figure 5 These four figures represent model prediction examples. They respectively demonstrate the experimental results of this invention on the Deeppleison and LUNA16 datasets, with two examples from each dataset shown.

[0172] Figure 2 and Figure 3 The graph shows the experimental results on the Deepplesion dataset compared to other methods; the first row shows examples. Figure 1 Comparison of mediastinum test results; the second row shows case studies. Figure 2 Comparison of pelvic bone examination results. Among them... Figure 2 The paper presents ground truth (GT) images of Case 1 (first row) and Case 2 (second row), along with the detection results of our method, D-Fine, and RT-DETR. Figure 3 The detection results of Case 1 and Case 2 on YOLO12-M, Anchor-DETR, Focus-DETR, and SQR-DETR are presented.

[0173] Figure 4 and Figure 5 The figures show the experimental results compared to other methods on the LUNA16 dataset. The first and second rows of figures show two different cases of lung nodule detection. Figure 4 Case studies were presented. Figure 1 (First row) and case studies Figure 2 (Second row) True ground truth (GT) images, and our method, D-Fine, and RT-DETR lung nodule detection results. Figure 5 The results of Case 1 and Case 2 on YOLO12-M, Anchor-DETR, Focus-DETR, and SQR-DETR are presented.

[0174] The Deeplesion dataset includes lesions in eight different organs: lung, abdomen, mediastinum, liver, pelvis, soft tissue, kidney, and bone; LUNA16 is a dataset for detecting lung nodules. Figure 2 , Figure 4 The ground truth (GT) comparison images in the dataset are generated from the labeled data, while other comparison images are generated by training the model using the corresponding methods.

[0175] This paper compares Chen Fangyi et al.'s SQR-DETR (Enhanced Training of Query-Based Object Detection via Selective Query Recollection), published at the International Conference on Computer Vision and Pattern Recognition (CVPR) in 2023; Zheng Dehua et al.'s Focus-DETR (Less is More: Focus Attention for Efficient DETR), published at the International Conference on Computer Vision (ICCV) in 2023; Wang Yingming et al.'s Anhor-DETR (AnchorDETR: Query Design for Transformer-Based Object Detection), published at the AAAI Conference on Artificial Intelligence in 2022; and Lü Wenxu et al.'s RT-DETR (RT-DETR: DETRs Beat YOLOs on Real-time Object Detection), published at CVPR in 2024. Detection, RT-DETR: DETRs outperformed YOLO in real-time object detection. This included seven methods: YOLOv12 (Attention-Centric Real-Time Object Detectors) presented by Tian Yunjie et al. at the 2025 NIPS conference on Neural Information Processing Systems; and D-Fine (D-FINE: Redefine RegressionTask of DETRs as Fine-grained Distribution Refinement) presented by Peng Yansong et al. at the 2025 International Conference on Learning Representations (ICLR). The results of this method are closer to the actual results and exhibit good detection performance.

[0176] In summary, the linear Transformer-based general lesion recognition method based on multi-sensoring and context-guided learning provided by this invention rapidly and accurately detects medical lesions by combining edge perception, polarity perception feature interaction, and context-guided learning mechanisms. This offers a novel design approach for general medical lesion detection and solves the problems of blurred lesion boundaries, difficulty in balancing feature interaction and computational efficiency, and weak separation between lesions and background features in previous methods. First, an edge perception feature enhancement module employs multi-scale edge generation and convolutional fusion strategies to efficiently inject shallow edge details into deep features, accurately perceiving lesions with blurred boundaries. Then, a polarity perception feature interaction module is used to simultaneously retain positive and negative value information, achieving comprehensive feature interaction modeling while maintaining linear complexity. Next, a context-guided feature pyramid network is designed to guide multi-scale feature fusion using global context information, thereby strengthening the separation between lesions and background features and significantly improving feature discrimination capabilities. Considering general clinical detection scenarios, this method effectively balances the number of model parameters and detection accuracy, achieving efficient detection of various lesions in clinical CT images.

Claims

1. A general lesion identification method based on multiple perception and context guidance using Linear Transformer, characterized in that, Includes the following steps: S1: Configure the backbone network and edge extraction operator, extract multi-scale features of medical CT images through the backbone network, extract shallow high-resolution edge gradient features in parallel, and construct basic feature representation; S2: Set up an edge-aware feature enhancement module to perform spatial alignment and nonlinear transformation on shallow edge features and inject them into deep semantic features to compensate for the lack of lesion boundary information. S3: Use the polarity-aware feature interaction module to build an encoder, and perform global modeling of the enhanced features in linear complexity through polarity decomposition and dual-mode interaction; S4: Design a context-guided feature pyramid network to generate highly discriminative multi-scale feature maps through relational context extraction, dynamic interpolation fusion, and semantic guidance; S5: Establish an end-to-end detection architecture based on ensemble prediction, use the enhanced feature map to guide the iterative update of object query, perform bipartite graph matching through the Hungarian algorithm, and optimize model parameters by combining the joint loss function of classification and regression to achieve accurate lesion localization. The specific process of step S1 is as follows: S11: For medical image I, a pre-trained ResNet-r18 network is used as the backbone to extract feature maps at four different stages, denoted as... ,in This indicates that the downsampling factor relative to the original image is [value missing]. Feature map; S12: Select shallow features As an edge source, its horizontal gradient is calculated using the Sobel operator. and vertical gradient Thus, the edge feature map is obtained. ; ； S13: Extract the edge feature map with deep features Together they serve as input for subsequent modules; The specific process of step S2 is as follows: S21: Based on edge feature maps Construct a multi-scale edge generator; for deep features Max pooling operation is used to... downsampling to With the same spatial resolution, adaptive edge features are obtained. ; ； in This represents a multi-scale edge information generator; S22: Perform cross-level feature injection operation, injecting deep features Adapting edge features The features are concatenated along the channel dimension and then subjected to multiple convolutions to obtain features with rich edge information. ； in, This indicates a concatenation operation. Represented as core size Convolution operation, Representing scale Enhanced features.

2. The linear Transformer general lesion identification method based on multiple perception and context guidance according to claim 1, characterized in that, This method is implemented through a system consisting of a backbone network, an edge-aware feature enhancement module, a polarity-aware feature interaction module, a context-guided feature pyramid network, and a detection head. The backbone network is used to extract basic features, the edge-aware feature enhancement module is used to enhance boundary information, the polarity-aware feature interaction module is used for complete feature interaction modeling under linear complexity, and the context-guided feature pyramid network is used for multi-scale fusion.

3. The linear Transformer general lesion identification method based on multiple perception and context guidance according to claim 1, characterized in that, The specific process of step S3 is as follows: S31: The enhanced features of the obtained edge perception enhancement module Generate the query matrix using linear projection. Key matrix Sum matrix ; ， ； ， ； in, The dimension is The tensor, in which Indicates batch size, Represented as sequence length, Represented as feature dimension, , , , The four are different learnable weight matrices; This is the gate vector; This indicates a spatial downsampling operation. downsampling rate ; S32: Reshape Q, K, and V into a form with h The multi-head format enables parallel processing across multiple representation subspaces, with each head having a dimension of [missing information]. ,get , ; in , , These are the query matrix, key matrix, and value matrix of a single reshaped header, respectively. The dimension is tensor, To reshape the number of attention heads, The dimension of a single head; S33: Yes and Polarity decomposition is performed using the ReLU activation function to explicitly separate the positive and negative components: ， ； ， ； in, , These represent the positive and negative components of a single query, respectively. , These represent the positive and negative components of a single bond, respectively. For activation functions; It is a learnable scaling factor used to control the input range. For learnable parameters, For activation functions; It is a learnable index used to adjust the nonlinear strength. This is the scaling factor. For the Sigmoid function, These are learnable weights; S34: Connect the positive and negative components to construct a dual-mode interactive feature descriptor. By preserving polarity information, two complementary matching modes are achieved. ； in Same-polarity queries representing the same-polarity matching pattern. A heteropolar query representing a heteropolar matching pattern; This represents the concatenated key matrix; S35: Calculate the average bond representation And the normalization factor is derived. : ； ； in This is the transpose of the average bond. It is a small constant used for numerical stability; S36: Construct two different feature interaction modes: homopolar interactions that capture feature consistency, and heteropolar interactions that capture feature contrast, value matrix Divided along the last dimension , The two interaction modes are calculated separately using a linear attention mechanism: ； in Indicates attention output, This indicates two modes: same polarity and opposite polarity. express transpose; Indicates the result of being segmented or ; Normalization factor or ; It is a normalization factor used for heteropolar interactions; S37: Will come from all heads and To enhance the local spatial context, a deep convolution is introduced on the value features for concatenation. The aggregated features are then modulated by a gated signal and projected onto the output space. ； in, , It is the output projection matrix. This indicates a splicing operation, starting from the first... Height to the Size; Represents depthwise convolution. It's a gating signal from S31. This indicates element-wise multiplication; S38: Obtained Output after frequency domain sensing feedforward neural network For use in the subsequent feature pyramid network: ； in, This is the final output. Representation layer normalization, Through Fast Fourier Transform and Learnable Weight Matrix Inverse Fast Fourier Transform and convolution are applied to frequency domain modulation to capture low-frequency structural information and high-frequency detail information.

4. The linear Transformer general lesion identification method based on multiple perception and context guidance according to claim 3, characterized in that, The specific process of step S4 is as follows: S41: Obtain the feature map , , The components are unified to the same spatial dimension, then concatenated, and global context information is extracted using the relational context module before being reallocated back to the original dimension. The formula is as follows: ； in, To extract global context information through the relational context module Layer features; Indicates adaptive average pooling. Indicates channel splicing. Refers to three stacked relational context modules. Indicates a channel separation operation; S42: The spatial correlation between lesions and the background is captured through horizontal and vertical pooling operations, while attention mechanisms are combined to enhance lesion features and suppress background noise. The formula is as follows: ； in, For rectangular context attention, The features to be input, i.e. , , ; and These represent horizontal and vertical pooling operations, respectively. This indicates element-wise summation. This indicates that asymmetric grouped convolution is followed by batch normalization and ReLU activation. This represents the Sigmoid activation function; S43: During the top-down propagation of the feature pyramid, a dynamic interpolation and addition strategy is used to achieve multi-scale fusion, providing dynamic adjustment of high-level features. The resolution is adjusted to match the target scale, and the aligned features are compared with the low-level features. The formula for element-wise addition is as follows: ； in, The first element after element-wise addition Layer features; This indicates a dynamic channel adjustment operation. This indicates a dynamic interpolation operation; S44: It is the feature obtained by extracting global context information from the previously stacked RCM, the final Depend on and The formula is obtained by multiplying each element together, as follows: ； S45: In bottom-up path aggregation, through high-level features A position-sensitive attention map is generated, selectively enhancing low-level features while suppressing redundant information. The specific formula is as follows: ； in, Represents the final enhanced version of the first Layer features; and This represents 1×1 convolutional feature maps used for different purposes. This indicates an upsampling operation.

5. The linear Transformer general lesion identification method based on multiple perception and context guidance according to claim 3, characterized in that, The specific process of step S5 is as follows: S51: Initialize the object query using an uncertainty minimization query selection strategy, that is, use the auxiliary prediction head to simultaneously evaluate the class confidence and bounding box localization quality of the features output from step S4, and select the one with the highest comprehensive score. N Using these features as initial object queries and reference points, the queries are input into the Transformer decoder and interact with the feature maps. After iterative updates through multiple decoding layers, the multilayer perceptron predicts the lesion category score and bounding box coordinates corresponding to each query, as shown in the following formula: ； in, To predict the output set, This is the preset number of queries. Indicates the first The probability of a query belonging to a category This represents the normalized bounding box parameters. For predicting the head, For decoder, This is the enhanced feature map that is the final output of step S4; S52: The Hungarian algorithm is used for globally optimal bipartite graph matching. By minimizing the matching cost matrix, the optimal permutation between the predicted set and the true set is found. The formula is as follows: ； in, This represents the parameter that minimizes the objective. For arrangement space, Taking into account both the differences in classification probabilities and the positional deviation of the bounding boxes, this ensures that each true lesion matches only one optimal predicted box. For the first A real goal In order to be with the first A prediction that matches a real target. Permutation function Functions and Index The result; S53: Based on the optimal matching result, construct a joint loss function to perform end-to-end optimization of the model. The specific formula is as follows: ； in, This is the total loss function; , , These represent classification loss, L1 loss, and Giou loss, respectively. c i Indicates the first i The category labels of each prediction box; The indicator function indicates that the regression loss is calculated only for queries that match the true target. This is the balance coefficient for each component loss. Indicates that the first one is assigned The predicted result for the actual target; Indicates the model's predicted probability; For the first The true bounding box of a real target; The bounding box predicted by the model.

Images