Hybrid transformer-MLP architecture for ultrasound-based prediction of axillary lymph node metastasis in breast cancer
The hybrid DeiT-MLP-Mixer model effectively predicts axillary lymph node metastasis in breast cancer using grayscale ultrasound, addressing data and computational inefficiencies to achieve high accuracy and clinical applicability.
Patent Information
- Application Number
- PCT/IB2026/050926
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-01-31
- Publication Date
- 2026-11-12
AI Technical Summary
Existing methods for preoperative prediction of axillary lymph node metastasis in breast cancer using ultrasound imaging suffer from limitations such as over-reliance on local features, large data dependency, limited generalization, computational inefficiency, and lack of integrated global-local feature fusion, leading to suboptimal accuracy and clinical applicability.
A hybrid deep learning model combining Data-efficient Image Transformer (DeiT) for global feature extraction with an MLP-Mixer classifier for attention-free classification, utilizing grayscale ultrasound images to predict ALN metastasis, even with limited labeled datasets.
The model achieves high accuracy (95.70%) and computational efficiency, enabling robust preoperative decision-making with reduced computational overhead and generalization across clinical settings, minimizing unnecessary invasive procedures.
Smart Images

Figure PCTXMLIB-APPB-I000001 
Figure PCTXMLIB-APPB-I000002 
Figure PCTXMLIB-APPB-I000003
Description
Hybrid Transformer-MLP Architecture for Ultrasound-Based Prediction of Axillary Lymph Node Metastasis in Breast Cancer
[0001] Breast cancer remains the most frequently diagnosed cancer in women globally and a leading cause of cancer death. Axillary lymph node (ALN) metastasis is a key determinant in staging, prognosis, and treatment planning. Accurate preoperative prediction of ALN involvement is critical to avoid unnecessary invasive procedures such as sentinel lymph node biopsy (SLNB) or axillary lymph node dissection (ALND), which carry risks of lymphedema, infection, and restricted mobility. While ultrasound imaging is a widely used noninvasive modality for preoperative staging, conventional interpretation lacks sufficient accuracy, and existing computer-aided prediction systems often rely on local feature extraction or require large labeled datasets. There exists a clear need for a reliable, efficient, and data-adaptive deep learning system capable of accurately classifying ALN status from routine grayscale ultrasound images to support clinical decision-making.
[0002] In recent years, artificial intelligence, especially deep learning, has made significant advances in medical imaging. CNNs, including well-known models like ResNet and Mask R-CNN, have been used for detecting ALNs from ultrasound and MRI images by learning detailed local features. Radiomics-based methods combined with machine learning models, such as support vector machines and XGBoost, have also been applied to extract handcrafted features from ultrasound, MRI, or PET / CT images to predict ALN status before surgery.
[0003] A61B 8 / 08 - G06T 7 / 00 - G06N 3 / 08 - G16H 50 / 20
[0004] This review focuses on prior studies relevant to breast cancer metastasis detection, with particular attention to preoperative ALN involvement assessed via ultrasound imaging, which is the primary clinical task addressed by this invention.
[0005] Relevant Prior Patents:
[0006] US20190015059A1 -- Semantic Segmentation for Cancer Detection in Digital Breast Tomosynthesis
[0007] Uses CNN-based segmentation for breast lesion detection in mammography. Does not employ transformers or address ALN metastasis prediction via ultrasound.
[0008] CN117152054A -- Breast Cancer Patient Lymph Node Metastasis Prediction Model Without Incorporating Clinical Pathological Features
[0009] Relies on radiomic features from CT images and logistic regression. Lacks deep learning, transformers, or hybrid architectures.
[0010] DE202022100092U1 -- Artificial Intelligence-Based Breast Cancer Detection System
[0011] Employs AI for tumor detection using radar and mammography. Not specific to ALN or ultrasound, and does not integrate transformer-mixer hybrids.
[0012] US11205265B2 -- System and Method for Assessing Breast Cancer Risk Using Imagery
[0013] Uses deep learning on mammography for general cancer risk, not ALN-specific prediction. No transformer or MLP-Mixer components.
[0014] CN113208640B -- Method for Predicting Axillary Lymph Node Metastasis Based on Breast-Dedicated PET Imaging Histology
[0015] Combines PET radiomics with clinical data using logistic regression. Does not use ultrasound or deep feature learning.
[0016] US11835524B2 -- Machine Learning for Digital Pathology
[0017] Applies CNNs to histopathology slides for biomarker prediction. Unrelated to ultrasound imaging or ALN classification.
[0018] WO2006058306A2 -- Medical Hyperspectral Imaging for Evaluation of Tissue and Tumor
[0019] Uses hyperspectral imaging without AI. Focuses on tissue oxygenation, not ultrasound-based ALN prediction.
[0020] WO2023049207A1 -- Quantitative Magnetic Resonance Imaging and Tumor Forecasting
[0021] Employs MRI-based biophysical modeling, not deep learning. Addresses therapy response, not preoperative ALN status.
[0022] Differentiation and Inventive Step:
[0023] None of the cited prior arts disclose a hybrid architecture combining DeiT for global feature extraction with an MLP-Mixer classifier for ALN metastasis prediction via grayscale ultrasound. Existing systems either:
[0024] Use non-deep learning methods (radiomics, regression),
[0025] Rely on non-ultrasound modalities (CT, PET, MRI),
[0026] Employ CNNs without global attention,
[0027] Or lack the data-efficient hybrid design enabling performance on limited clinical datasets.
[0028] The present invention thus represents a non-obvious integration of two modern architectures (DeiT + MLP-Mixer) tailored to the specific challenges of ultrasound-based ALN classification, offering superior accuracy, efficiency, and clinical applicability.
[0029] The invention relates to a hybrid deep learning model for the preoperative prediction of axillary lymph node (ALN) metastasis in breast cancer patients using grayscale ultrasound images. The model combines a Data-efficient Image Transformer (DeiT) for global feature extraction with an MLP-Mixer-inspired classifier for final prediction. The DeiT module divides each ultrasound image into non-overlapping patches, converts them into token sequences, and processes them through transformer layers to capture long-range spatial relationships across the image. The extracted patch embeddings are then fed into an MLP-Mixer classifier that performs spatial mixing and channel mixing without attention mechanisms, efficiently integrating local and global features. This architecture enables high-accuracy classification even with limited labeled datasets. In validation on a real-world dataset of 502 patients (1,506 augmented images), the model achieved 95.70% accuracy, 0.9760 precision, 0.9242 recall, 0.9867 specificity, and 0.9494 F1 score, outperforming standalone ViT, DeiT, and pretrained CNNs. The invention provides a noninvasive, computationally efficient, and clinically deployable tool for supporting preoperative decision-making in breast cancer care.
[0030] Existing methods for preoperative ALN metastasis prediction suffer from several limitations:
[0031] Over-reliance on local features: Conventional convolutional neural networks (CNNs) excel at extracting local patterns but fail to capture long-range spatial dependencies essential for holistic image interpretation in medical contexts.
[0032] Large data dependency: Vision Transformers (ViTs) require extensive labeled datasets to perform effectively, which are often unavailable in clinical settings due to privacy, cost, and annotation burdens.
[0033] Limited generalization: Models trained on single-institution data or specific imaging protocols may not generalize across different ultrasound machines, operators, or patient populations.
[0034] Computational inefficiency: Many deep learning models, especially transformer-based architectures, are computationally heavy, limiting real-time deployment in resource-constrained clinical environments.
[0035] Lack of integrated global-local feature fusion: Prior hybrid approaches often combine CNNs with transformers but do not efficiently integrate both spatial and channel-wise information without attention overhead.
[0036] We propose a new hybrid deep learning model for predicting ALN metastasis from grayscale ultrasound images before surgery. Unlike existing hybrid models that combine CNNs and transformers or rely heavily on attention mechanisms, our approach uses DeiT for global feature extraction and an MLP-Mixer-style classifier for attention-free classification.
[0037] We demonstrate that replacing standard CNN or transformer classification heads with an MLP-Mixer-based classifier improves model stability and generalization, especially when training on small, single-center clinical datasets, which are common in breast ultrasound studies.
[0038] We evaluate the proposed model on a real-world ultrasound dataset of 502 breast cancer patients and show that it consistently performs better than standard CNN and transformer models, highlighting the benefit of separating global feature extraction from task-specific classification.Solution of Problem
[0039] 1.1 What the Invention Is
[0040] The invention is a hybrid deep learning model for the preoperative binary classification of axillary lymph node (ALN) status (involved vs. noninvolved) in breast cancer patients using grayscale ultrasound images.
[0041] Complete System Architecture
[0042] The complete processing pipeline is illustrated in, showing the sequential flow from image input to classification output.
[0043] 2.1 Input Data:
[0044] Source: 502 breast cancer patients from a specialist breast clinic in Iran
[0045] Imaging Device: Mindray Z5 Diagnostic Ultrasound System with 75L38EA linear-array probe (5.0-7.5 MHz)
[0046] Image Type: One grayscale ultrasound image per patient
[0047] Ground Truth: ALN status confirmed via postoperative pathology reports
[0048] Distribution: 283 noninvolved nodes, 219 involved nodes
[0049] All images were obtained using a high-frequency linear probe with a vertical probe orientation. Image acquisition was performed on a single ultrasound system from the same manufacturer and software version, which minimized variability related to device-specific image quality and acquisition settings. One grayscale ultrasound image was obtained from each patient before surgery, and the lymph node status was determined from postoperative pathology reports. All cases without lymph node involvement were confirmed by sentinel lymph node biopsy. Cases classified as lymph node-positive included only those with micro-metastasis, which have a clear impact on treatment planning. All lymph node-positive cases were confirmed by histopathological examination. All cases included in this study involved single tumors, multicentric and multifocal tumors were excluded from the analysis.
[0050] 2.2 Data Augmentation & Preprocessing:
[0051] Augmentation Techniques: Rotations, flips, intensity adjustments
[0052] Augmented Dataset Size: 1,506 images (from original 502)
[0053] Dataset Split [Table 1]:
[0054] Training: 70% (1,053 images)
[0055] Validation: 10% (151 images)
[0056] Testing: 20% (302 images)
[0057] Class Distribution in Augmented Set:
[0058] Involved: 657 total (459 train, 66 validation, 132 test)
[0059] Non-Involved: 849 total (594 train, 85 validation, 170 test)
[0060] contains schematics based on SEM images in [Pic. 2] showing representative examples of both involved and noninvolved lymph node ultrasound images from our clinical dataset.
[0061] The original dataset consisted of 502 ultrasound images, with one image per patient. These images were first split on a patient basis to prevent any data leakage between training, validation, and testing. Given the moderate class imbalance in the dataset (283 noninvolved vs. 219 involved cases), we used stratified sampling when splitting the data into training, validation, and test sets to maintain consistent class distribution across all subsets, rather than artificially rebalancing the data. No additional class-balancing methods, such as class weighting or resampling, were applied during training. To ensure a careful and unbiased evaluation, we followed a strict split → augment → evaluate workflow.
[0062] During model training and evaluation, different image processing pipelines were used through the data loaders. For all images, the first step was standardization, which involved converting the single-channel grayscale images to three-channel RGB and resizing them to a fixed size of 224 × 224 pixels. This provided a consistent input format and ensured compatibility with the DeiT-base architecture.
[0063] While this ensured consistency across the dataset, it may limit generalizability to images acquired using different ultrasound systems, which is acknowledged as a limitation.
[0064] 2.3 Image Transformation Pipeline:
[0065] For Training:
[0066] Convert grayscale to 3-channel RGB via channel duplication
[0067] Random resize and crop to 224×224 pixels
[0068] Random horizontal flip (50% probability)
[0069] Random rotation (±15 degrees)
[0070] Brightness and contrast adjustments (±0.2)
[0071] Random affine transformations (10° rotation, 10% translation)
[0072] Normalize with mean=0.5, std=0.5 per channel
[0073] For training, a stochastic pipeline was applied after standardization. In this step, images were augmented on the fly to represent common variations observed in real clinical ultrasound images. This included random resized cropping to 224 × 224 pixels, horizontal flips (probability = 0.5), random rotations (±15°), brightness and contrast adjustments (±0.2), random affine transformations (±10°), and color jitter to randomly perturb the brightness and contrast. Because the original ultrasound images were grayscale, RGB conversion was performed by duplicating the single grayscale channel across three channels to match the input requirements of the DeiT-based architecture.
[0074] For Validation / Testing:
[0075] Convert grayscale to RGB
[0076] Resize to 224×224 (deterministic)
[0077] Normalize with mean=0.5, std=0.5 per channel
[0078] In contrast, the validation and test sets were processed using a fixed and deterministic pipeline, consisting only of RGB conversion and resizing to 224 × 224 pixels, without any random transformations. All images were normalized using a mean and standard deviation of 0.5 per channel to ensure compatibility with the DeiT-base architecture.
[0079] All scans were collected by experienced breast specialists following a standardized protocol. This ensured consistent probe pressure, angle, and gain settings, focusing specifically on the relevant axillary region. In addition, geometric augmentation during training was used to simulate slight changes in probe angle and patient positioning, helping the model handle the normal variations in anatomy and positioning seen in clinical practice.
[0080] 2.4 DeiT Feature Extractor:
[0081] Model: DeiT-Base-Distilled-patch16-224
[0082] Input: 224×224×3 RGB image
[0083] Patch Division: 196 non-overlapping 16×16 patches
[0084] As shown in, the DeiT architecture processes images through patch embedding and transformer layers with distillation tokens.
[0085] Patch Processing:
[0086] Flatten patches to vectors
[0087] Linear projection to 768-dimensional embeddings
[0088] Append CLS token and DIST token → 198 total tokens
[0089] Add learnable positional embeddings
[0090] Transformer Processing: 12 transformer layers
[0091] Each layer: Multi-head self-attention + feedforward sublayers
[0092] Residual connections and layer normalization
[0093] Output Processing: Discard CLS and DIST tokens
[0094] Final Feature Tensor: Shape = (batch_size, 196, 768)
[0095] Standard DeiT architectures use the CLS and DIST tokens to compress global information into a single vector for classification. While useful for summarizing the image, these tokens lose the detailed spatial layout of the feature map. The MLP-Mixer, on the other hand, works with a fixed sequence of image patches to learn relationships between specific anatomical regions, such as the boundary of a lymph node compared to the surrounding tissue. Including the CLS or DIST tokens would disrupt this spatial structure. That's why we discard these summary tokens and let the MLP-Mixer focus directly on the 196 patch embeddings, using only the raw spatial features from each patch.
[0096] 2.5 MLP-Mixer Classifier (Novel Integration):
[0097] Input: (batch_size, 196, 768) from DeiT
[0098] The classification stage replaces traditional attention-based or linear heads with an MLP-Mixer-style classifier with a full two-layer MLP-Mixer stack (L=2). Each of the two layers sequentially performs spatial and channel mixing within a post-normalization residual structure.
[0099] Spatial Mixing (Novel in this medical context):
[0100] Transpose to (batch_size, 768, 196)
[0101] Linear layer: 196 → 512 (hidden dimension)
[0102] GELU activation
[0103] Dropout (rate=0.3)
[0104] Linear layer: 512 → 196
[0105] Dropout (rate=0.3)
[0106] Transpose back to (batch_size, 196, 768)
[0107] Residual connection with original input
[0108] Channel Mixing (Novel in this medical context):
[0109] Operates on (batch_size, 196, 768)
[0110] Linear layer: 768 → 512 (per patch)
[0111] GELU activation
[0112] Dropout (rate=0.3)
[0113] Linear layer: 512 → 768
[0114] Dropout (rate=0.3)
[0115] Residual connection with spatial-mixed output
[0116] illustrates the MLP-Mixer architecture with its distinct spatial and channel mixing layers that operate without attention mechanisms.
[0117] Post-Mixing Processing:
[0118] Layer normalization
[0119] Transpose to (batch_size, 768, 196)
[0120] Adaptive average pooling over spatial dimension → (batch_size, 768)
[0121] Final linear layer: 768 → 2 (binary logits)
[0122] Softmax for probability conversion (inference only)
[0123] 2.6 Training Configuration (Documented Parameters):
[0124] Hardware Environment:
[0125] GPU: NVIDIA Tesla T4 (16 GB VRAM)
[0126] CPU: Intel Xeon @ 2.00 GHz
[0127] RAM: 32 GB
[0128] Software Environment:
[0129] Python with Kaggle notebooks
[0130] PyTorch 2.5.1
[0131] Hugging Face Transformers 4.47.0
[0132] NumPy 1.26.4, scikit-learn 1.2.2, Matplotlib 3.7.5, torchvision 0.20.1
[0133] Training Parameters:
[0134] Optimizer: AdamW
[0135] Learning rate: 2e-5
[0136] Weight decay: 0.01
[0137] Batch size: 32
[0138] Maximum epochs: 50
[0139] Early stopping: Patience=5 epochs (based on validation accuracy)
[0140] Loss function: Binary cross-entropy
[0141] Learning rate scheduler: ReduceLROnPlateau
[0142] Factor: 0.1
[0143] Patience: 5 epochs (on validation loss plateau)
[0144] Training images per epoch: 1,053 (33 batches of size 32)
[0145] 2.7 Performance Evaluation Metrics:
[0146] The invention uses standard classification metrics:
[0147] Accuracy: (TP+TN) / (TP+TN+FP+FN)
[0148] Precision: TP / (TP+FP)
[0149] Recall / Sensitivity: TP / (TP+FN)
[0150] Specificity: TN / (TN+FP)
[0151] F1 Score: 2×(Precision×Recall) / (Precision+Recall)
[0152] Loss: Binary cross-entropy value
[0153] 2.8 Mathematical Formulations
[0154] The invention's performance is evaluated using standard classification metrics defined as follows:
[0155] Accuracy [Formula 1]: Overall correct prediction rate
[0156] Precision [Formula 2]: Positive predictive value
[0157] Recall [Formula 3]: Sensitivity or true positive rate
[0158] Specificity [Formula 4]: True negative rate
[0159] F1 Score [Formula 5]: Balanced measure of precision and recall
[0160] Cross-Entropy Loss [Formula 6]: Training optimization objective
[0161] These metrics provide comprehensive evaluation of the model's diagnostic performance, with particular clinical importance on specificity [Formula 4] to minimize unnecessary surgical procedures.
[0162] 2.9 Model Output:
[0163] Raw Output: Two logits [L0, L1]
[0164] Probability Conversion: Softmax → [P(noninvolved), P(involved)]
[0165] Decision Threshold: 0.5 (configurable as per standard practice)
[0166] Prediction: Involved if P(involved) ≥ 0.5, else Noninvolved
[0167] Novel Technical Aspects (From Document)
[0168] 3.1 First Hybrid of DeiT + MLP-Mixer for Medical Imaging:
[0169] Novel Combination: No prior art combines DeiT's data-efficient global feature extraction with MLP-Mixer's attention-free mixing for medical image classification
[0170] Medical Specificity: First application specifically for ALN metastasis prediction from grayscale ultrasound
[0171] specifically illustrates our novel hybrid architecture combining DeiT feature extraction with MLP-Mixer classification.
[0172] 3.2 Token Discarding Strategy:
[0173] Novel Approach: Removal of CLS and DIST tokens after DeiT processing
[0174] Rationale: Focuses exclusively on medically relevant patch embeddings (196×768) rather than aggregated token representations
[0175] Benefit: Preserves spatial information crucial for lymph node localization and characterization
[0176] 3.3 Dual-Mixing Mechanism for Medical Features:
[0177] Spatial Mixing (Documented as novel for this application):
[0178] Captures global relationships between lymph node regions and surrounding tissues
[0179] Enables understanding of contextual anatomical patterns
[0180] Channel Mixing (Documented as novel for this application):
[0181] Enhances discriminative feature representations within each patch
[0182] Amplifies subtle texture and echogenicity differences between involved / noninvolved nodes
[0183] 3.4 Data-Efficient Training on Limited Clinical Dataset:
[0184] Dataset Size: Only 502 original patients (1,506 augmented images)
[0185] Performance Achievement: 95.70% accuracy despite limited data
[0186] Key Enabler: DeiT's distillation mechanism combined with MLP-Mixer's efficiency
[0187] Implementation (Referring to Standard Practices)
[0188] 4.1 Standard Software Implementation:
[0189] The model is implemented using standard deep learning practices in PyTorch, comprising:
[0190] Custom Dataset class for ultrasound image loading
[0191] Standard training loop with forward / backward propagation
[0192] Model checkpointing and early stopping per common practice
[0193] Standard inference pipeline with batch processing
[0194] 4.2 Standard Clinical Integration Pathway:
[0195] Integration with hospital systems follows standard medical imaging pipelines:
[0196] DICOM image retrieval from PACS (standard protocol)
[0197] Image preprocessing via standard medical image processing libraries
[0198] Results storage in EMR using HL7 / FHIR standards (common practice)
[0199] Physician review interface following standard clinical software design
[0200] 4.3 Standard Deployment Architecture:
[0201] Deployment follows conventional AI model serving practices:
[0202] Model serialization using standard formats (PyTorch .pt)
[0203] REST API serving following standard microservices architecture
[0204] GPU acceleration using standard CUDA optimizations
[0205] Scalability via standard containerization (Docker) and orchestration (Kubernetes)
[0206] Documented Performance Results
[0207] 5.1 Test Set Performance:
[0208] Accuracy: 95.70%
[0209] Precision: 0.9760
[0210] Recall: 0.9242
[0211] Specificity: 0.9867
[0212] F1 Score: 0.9494
[0213] Loss: 0.1345
[0214] [Pic. 1] shows the training convergence (loss and accuracy curves) and final confusion matrix demonstrating the model's performance characteristics.
[0215] 5.2 Comparative Superiority [Table 3]:
[0216] 5.3 Training Convergence (Documented):
[0217] Training epochs: 44 (early stopping triggered)
[0218] Accuracy progression: 54.70% (epoch 1) → 93.26% (epoch 44)
[0219] Validation accuracy: Reached 94.70% at epoch 40
[0220] Loss reduction: Training loss 0.6907 → 0.1580; Validation loss to 0.1814
[0221] [Table 4] Different performance measures for the experimental models on the validation data.
[0222] The model demonstrates practical computational efficiency for clinical deployment. On standard hardware (NVIDIA Tesla T4 GPU), the system processes ultrasound images rapidly, enabling near-instant diagnostic feedback in real-time clinical settings without requiring expensive server infrastructure.
[0223] Enabling Disclosure
[0224] A person skilled in the art of deep learning and medical imaging can reproduce this invention by:
[0225] Implementing the documented DeiT-MLP-Mixer architecture as described
[0226] Applying the specified preprocessing and augmentation techniques
[0227] Training with the provided hyperparameters [Table 2] on similar ultrasound data
[0228] Following standard practices for model validation and deployment
[0229] generated schematics based on SEM images of [Pic. 3] provides example predictions on test images, demonstrating the model's application to individual cases.
[0230] Clinical Implications
[0231] From a clinical perspective, this model provides a noninvasive alternative to procedures such as SLNB and ALND, which can cause complications in patients. Accurate predictions could help reduce unnecessary surgeries, lower healthcare costs, and improve patient care. The primary objective is to support treatment decision-making by providing early information on ALN status, which may influence the selection of systemic therapies.Advantage Effects of the Invention
[0232] • High Accuracy: Achieves 95.70% accuracy on real-world ultrasound data.
[0233] • Data Efficiency: DeiT enables effective learning from limited labeled datasets.
[0234] • Global-Local Fusion: Combines DeiT’s global attention with MLP-Mixer’s spatial / channel mixing.
[0235] • Noninvasive: Relies solely on preoperative grayscale ultrasound.
[0236] • Computationally Light: MLP-Mixer replaces attention in classifier, reducing inference time.
[0237] • Clinically Deployable: Compatible with standard GPU hardware (e.g., NVIDIA T4).
[0238] • Generalizable: Robust to variability in ultrasound image quality and acquisition.
[0239] • Reduces Unnecessary Surgery: Accurate prediction can minimize SLNB / ALND in node-negative patients.
[0240] Architecture of the DeiT model.
[0241] Architecture of the MLP-Mixer.
[0242] schematic remake of (a) Involved and (b) noninvolved axillary lymph node ultrasound images from the dataset.
[0243] Flowchart of the proposed DeiT-MLP-Mixer model.
[0244] Architecture of the proposed DeiT-MLP-Mixer model.
[0245] Schematic remake of prediction of the model from ultrasound images.
[0246] illustrates the standard Data-efficient Image Transformer (DeiT) architecture, showing how input images are divided into patches, converted to tokens, processed through transformer layers with CLS and DIST tokens, and how knowledge distillation from a teacher CNN is implemented. The visual includes patch embedding, positional encoding, transformer blocks, and the classification head.
[0247] depicts the MLP-Mixer architecture showing the per-patch fully-connected layer, N Mixer Layers (each containing spatial and channel mixing components), global average pooling, and final fully-connected classification layer. The visual clearly separates spatial mixing (across patches) from channel mixing (within patches).
[0248] ultrasound-based schematics showing both involved (metastatic) and noninvolved (normal) axillary lymph nodes from the clinical dataset. The images demonstrate the visual characteristics the model learns to distinguish, including differences in echogenicity, morphology, and boundary definition. The original SEM frames are included in the Examples section of the application.
[0249] represents the complete processing pipeline from input ultrasound images through preprocessing, DataLoader batching, DeiT feature extraction (with token removal), MLP-Mixer processing (spatial and channel mixing), to final binary classification output.
[0250] shows the specific hybrid architecture combining the DeiT feature extractor with the MLP-Mixer classifier. It visually integrates the two main components, highlighting how patch embeddings from DeiT feed into the mixing layers, and demonstrates the residual connections and normalization steps.
[0251] shows generated schematics of five randomly selected ultrasound test images with both the model's prediction and actual label displayed. It provides visual validation of correct classifications, with examples of both involved and noninvolved cases that the model accurately identified. The original SEM frames are included in the Examples section.Examples
[0252] Clinical Application Scenario
[0253] The DeiT-MLP-Mixer model is implemented in a breast cancer clinic for preoperative ALN assessment following standard clinical AI integration practices:
[0254] Step 1: Image Acquisition
[0255] Ultrasound machine: Mindray Z5 with 75L38EA probe (5.0-7.5 MHz)
[0256] Image type: Single grayscale ultrasound image per patient
[0257] Patient population: Breast cancer patients scheduled for surgery
[0258] Step 2: Preprocessing
[0259] Convert grayscale to 3-channel RGB via duplication
[0260] Resize to 224×224 pixels
[0261] Apply normalization (mean=0.5, std=0.5 per channel)
[0262] Note: Clinical implementation would follow standard DICOM preprocessing pipelines
[0263] Step 3: Model Inference
[0264] Load trained DeiT-MLP-Mixer model (standard PyTorch model loading)
[0265] Process image through documented architecture
[0266] Obtain probability output: [P(noninvolved), P(involved)]
[0267] Apply standard clinical decision threshold (0.5)
[0268] Step 4: Clinical Decision Support
[0269] Prediction: "Involved" if P(involved) ≥ 0.5, else "Noninvolved"
[0270] Confidence: Based on probability value
[0271] Integration: Results presented to clinician via standard EMR interface
[0272] Reference Performance: Clinicians informed of model's documented 95.70% accuracy, 98.67% specificity
[0273] Step 5: Validation Against Pathology
[0274] Surgical procedure proceeds per standard protocols (SLNB or ALND)
[0275] Postoperative pathology provides ground truth
[0276] Model performance tracked per standard quality assurance practices
[0277] Documented Performance Expectation: Based on test results, ~95.7% of predictions should match pathology
[0278] System Output Example
[0279] For a patient with ALN involvement probability of 0.894:
[0280] ALN Status Prediction:
[0281] - Prediction: INVOLVED (High Probability)
[0282] - Confidence Score: 89.4%
[0283] - Model Performance Reference: Accuracy 95.7%, Specificity 98.7%
[0284] - Clinical Consideration: High specificity suggests low false positive rate
[0285] Scalability Considerations
[0286] Hardware: Standard GPU requirements (NVIDIA T4 or equivalent per document)
[0287] Throughput: Batch processing of multiple patients following standard hospital workflow
[0288] Integration: Standard PACS / EMR interfaces as commonly implemented in clinical AI systems
[0289] Maintenance: Standard model monitoring and retraining protocols
[0290] Novel Clinical Impact
[0291] The invention enables, for the first time:
[0292] High-accuracy ALN prediction (95.70%) from single grayscale ultrasound images
[0293] Noninvasive alternative to SLNB / ALND for negative predictions
[0294] Data-efficient learning suitable for single-institution deployment
[0295] Attention-free classification for faster inference in clinical settings
[0296] This implementation represents the first clinical application of a DeiT-MLP-Mixer hybrid for ultrasound-based ALN metastasis prediction, with all architectural and performance specifications directly derived from documented research.
[0297] [Table 1] Involved and noninvolved axillary lymph node ultrasound images from the dataset.
[0298]
[0299] [Table 2] Training parameters for the proposed model.
[0300]
[0301] [Table 3] Different performance measures for the experimental models on the validation data.
[0302]
[0303] [Table 4] Comparison of Different performance measures for the experimental models on the validation data.
[0304]
[0305] [Table 5] Comprehensive Comparison of Performance and Computational Complexity.
[0306]
[0307] [Table 6] Comparison of the proposed model with other models in related works.
[0308]
[0309] [Formula 1] Measures the proportion of correctly classified instances out of all cases in the dataset.
[0310] [Formula 1]
[0311] [Formula 2] Measures the accuracy of positive predictions made by the classifier (true positives divided by all predicted positives).
[0312] [Formula 2]
[0313] [Formula 3] Reflects the classifier's ability to identify all relevant positive instances (true positive rate).
[0314] [Formula 3]
[0315] [Formula 4] Measures the classifier's ability to correctly identify negative cases (true negative rate).
[0316] [Formula 4]
[0317] [Formula 5] Harmonic mean of precision and recall that balances both metrics.
[0318] [Formula 5]
[0319] [Formula 6] Measures the difference between predicted probability distribution and true class labels during training.
[0320] [Formula 6]
[0321] [Pic. 1] (a) Loss per epoch graph, (b) accuracy per epoch graph, (c) confusion matrix of the proposed model.
[0322] [Pic. 1]
[0323]
[0324] This composite figure shows training and validation metrics: (a) Loss curves demonstrating convergence over 44 epochs, (b) Accuracy progression showing learning improvement, and (c) Confusion matrix displaying true / false positive / negative counts on test data, visually confirming the model's high specificity and sensitivity.
[0325] [Pic. 2] example ultrasound images containing (a) Involved and (b) noninvolved axillary lymph node ultrasound images from the dataset.
[0326] [Pic. 2]
[0327]
[0328] [Pic. 3] five randomly selected ultrasound test images.
[0329] [Pic. 3]
[0330]
[0331] The ultrasound images above have been generated as schematics inandfor more clarity and visibility under patent processing.
[0332] [Pic. 4] ROC and PR curves for the DeiT-MLP-Mixer
[0333] [Pic. 4]
[0334]
[0335] This invention will be very widely used in various industrial sectors:
[0336] Hospitals & Breast Clinics: For preoperative staging without invasive biopsies.
[0337] Telemedicine Platforms: Integrated into cloud-based diagnostic support systems.
[0338] Medical Imaging Software: As a plug-in for ultrasound machines (e.g., GE, Siemens, Philips).
[0339] AI Healthcare Startups: As a deployable model for breast cancer diagnostics.
[0340] Research Institutions: For further development in hybrid deep learning for medical imaging.
[0341] Health Insurance: As a tool to reduce costly unnecessary surgeries.
Claims
A computer-implemented method for predicting axillary lymph node (ALN) metastasis in breast cancer patients, comprising: receiving a grayscale ultrasound image of an axillary region; preprocessing the image to generate a three-channel input of fixed resolution; extracting global spatial features using a Data-efficient Image Transformer (DeiT) module; classifying the ALN status via an MLP-Mixer classifier that performs spatial mixing and channel mixing on the extracted features; and outputting a binary prediction indicating involved or noninvolved ALN status.The method of claim 1, wherein the DeiT module is a distilled DeiT-Base model with patch size 16×16 and input resolution 224×224.The method of claim 1, wherein the MLP-Mixer classifier operates without attention mechanisms and includes at least one spatial mixing layer and one channel mixing layer.The method of claim 1, wherein preprocessing includes data augmentation techniques selected from rotation, flipping, brightness adjustment, and affine transformation.The method of claim 1, wherein the DeiT module discards classification (CLS) and distillation (DIST) tokens before passing patch embeddings to the MLP-Mixer classifier.The method of claim 1, wherein the MLP-Mixer classifier comprises a two-layer stack with each layer performing spatial mixing followed by channel mixing with residual connections.The method of claim 1, wherein the model is trained using binary cross-entropy loss and AdamW optimization with a learning rate of 2e-5.The method of claim 1, wherein the output prediction is used to guide preoperative surgical planning.A system comprising a processor and memory storing instructions that, when executed, perform the method of any of claims 1-8.A computer-implemented system for predicting axillary lymph node metastasis in breast cancer patients, comprising:a processor configured to execute instructions; memory storing instructions for implementing a hybrid deep learning model comprising: a) a Data-efficient Image Transformer (DeiT) feature extractor that divides input ultrasound images into non-overlapping patches and extracts global spatial features; b) an MLP-Mixer classifier connected serially to the DeiT feature extractor, configured to perform spatial mixing and channel mixing without attention mechanisms.
