A 3D breast ABUS image classification method based on a tokenized bi-branch selective state-space model, electronic devices, and computer-readable storage media.

Abstract

This invention discloses a three-dimensional breast ABUS image classification method, electronic device, and computer-readable storage medium based on a tokenized bi-branch selective state-space model. The method preprocesses the three-dimensional ABUS volume data and inputs it into a hierarchical pyramidal classification network. The network constructs local convolutional branches and a global state-space branch in its basic modules: the local branch uses lightweight three-dimensional grouped convolution to extract texture and boundary morphology; the global branch aggregates features into a token map through voxel token generation, then performs multi-axis bi-directional selective state-space scanning and adaptive fusion using routing weights. After detoxing, the core features are injected through a gating mechanism. Finally, the classification result is output through three-dimensional global average pooling and a fully connected layer, enhancing the ability to model long-range dependencies across slices with near-linear complexity.

Description

Technical Field

[0001] This invention relates to the fields of intelligent medical image analysis and computer vision technology, and in particular to a three-dimensional breast ABUS image classification method, electronic device and computer-readable storage medium based on a tokenized bi-branch selective state space model. Background Technology

[0002] Automated breast volumetric ultrasound (ABUS) can automatically scan the entire breast and obtain three-dimensional volumetric data containing information in the coronal, transverse, and sagittal planes. It has advantages such as good repeatability, low operator dependence, and suitability for screening dense breasts. However, three-dimensional ABUS volumetric data often suffers from significant speckle noise, weak tissue boundaries, diverse lesion morphologies, inconsistencies in resolution between thickness and in-plane directions (anisotropy), and cross-device / cross-center differences, making accurate benign / malignant differentiation of breast lesions quite challenging. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a three-dimensional breast ABUS image classification method, electronic device, and computer-readable storage medium based on a tokenized bi-branch selective state space model. This addresses the challenge mentioned in the background art of how to maintain three-dimensional spatial structural information while simultaneously considering sensitivity to local texture / boundary details of lesions and the ability to model long-range dependencies across slices in three-dimensional automated breast volume ultrasound (ABUS) data.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model, comprising the following steps: S1: Obtain the 3D ABUS volume data to be classified, preprocess the 3D ABUS volume data to obtain normalized volume data; S2: Input the normalized volume data into the 3D ABUS image classification network, output the category probability vector, and determine the classification result based on the category probability vector; The 3D ABUS image classification network includes a 3D feature embedding module, a hierarchical pyramid encoder, and a classification head; The 3D feature embedding module uses 3D convolution with a preset stride to downsample and map the normalized volume data to obtain a feature volume that preserves the 3D spatial structure. The hierarchical pyramid encoder consists of multiple cascaded stages, and the stages are transformed in resolution through a 3D downsampling fusion module to form a multi-scale feature pyramid from high resolution to low resolution. Each stage contains several tokenized dual-branch basic blocks, including a first branch and a second branch; The tokenized dual-branch basic block divides the input feature body along the channel dimension to obtain the first branch feature and the second branch feature. The first branch performs a lightweight 3D convolution operation on the first branch features to extract local texture and boundary morphology features, thus obtaining local features; The second branch performs the following processing on the second branch features to obtain the global features: The second branch performs feature normalization to obtain normalized features, generates gate vectors from normalized features, and generates a token graph from voxel tokens of normalized features, where each token corresponds to a three-dimensional local block in the normalized features; Perform a 3D multi-axis selective state-space scan on the token graph to obtain a multi-route scan output; Adaptive weighted fusion is performed on each scan output to obtain a fused token graph; Perform detokenization on the fused token graph to restore spatial resolution consistent with the normalized features; The global features are obtained by element-wise multiplying the features obtained from detokenization with the gated vector through gated modulation. After concatenating the local and global features along the channel dimension, channel shuffling is performed, and residual fusion is performed with the input feature volume to obtain the output feature volume; Voxel token generation aggregates normalized features into a token graph, with each token corresponding to a 3D local block; The classification head performs three-dimensional global average pooling on the feature volume output from the last stage to obtain a global vector, which is then passed through a fully connected layer or a multi-layer perceptron to output a class probability vector.

[0005] As a further improvement of the present invention, the preprocessing of the three-dimensional ABUS volume data includes at least one of the following methods: voxel spacing resampling, intensity normalization, noise reduction, contrast enhancement, and region of interest clipping.

[0006] As a further improvement of the present invention, the three-dimensional downsampling fusion module is implemented using three-dimensional PatchMerging with a step size of 2.

[0007] As a further improvement of the present invention, the three-dimensional multi-axis selective state space scanning includes multiple scanning directions, including at least one of thickness direction scanning and in-plane direction scanning, and each scanning direction performs forward scanning and reverse scanning.

[0008] As a further improvement of the present invention, adaptive weighted fusion is performed on each scan output, including predicting the weight vector of each scan direction through a route weight predictor and performing weighted summation on the scan output of each scan direction.

[0009] As a further improvement of the present invention, selective state-space scanning employs an input-related selective parameter modulation mechanism to capture long-range dependencies with near-linear computational complexity.

[0010] As a further improvement of the present invention, the gate vector is obtained by performing global pooling or pointwise convolution on the second branch features or normalized features and then passing it through the Sigmoid function, and is used to control the intensity of global information injection.

[0011] As a further improvement of the present invention, detokenization restores the fused token graph to a spatial resolution consistent with the normalized features by upsampling and / or deconvolution, and introduces skip connections during the restoration process to preserve low-level detail information.

[0012] An electronic device, comprising a processor and a memory; It stores computer program instructions, which, when executed by a processor, cause the processor to perform the three-dimensional breast ABUS image classification method based on the tokenized bi-branch selective state space model described above.

[0013] A computer-readable storage medium is characterized in that it stores a computer program, which, when executed by a processor, causes the processor to perform the three-dimensional breast ABUS image classification method based on the tokenized bi-branch selective state space model described above.

[0014] The beneficial effects of this invention include: (1) Unlike existing technologies that rely solely on three-dimensional convolution for local feature extraction and are insufficient for representing long-range dependencies across layers, this invention sets up a dual-branch complementary feature extraction mechanism by setting up a local three-dimensional convolution branch and a global selective state space scanning branch. Compared with the mainstream scheme that only uses three-dimensional convolution, this invention can enhance the representation of cross-slice structural continuity and long-range dependencies while preserving local texture features and boundary morphology information, which is beneficial to improving the accuracy and stability of breast ABUS image classification results.

[0015] (2) Unlike the high complexity and high storage overhead caused by directly performing global modeling on high-resolution 3D features in the prior art, the present invention first compresses and represents the 3D features through a voxel marker generation module, and then performs global dependency modeling on the compressed token graph. Compared with the existing mainstream Transformer-type schemes, the present invention can significantly shorten the sequence length, reduce computational complexity and memory usage while maintaining the 3D spatial structure information, and improve the engineering feasibility and clinical deployment applicability of the method to a certain extent.

[0016] (3) Unlike the existing technology that uses a single scanning path for sequence modeling and is prone to directional bias, this invention constructs a multi-axis bidirectional scanning mechanism and combines it with route adaptive fusion to jointly represent the contextual features of different directions. Compared with the single-path scanning scheme, this invention can reduce directional sensitivity, improve adaptability to different lesion morphological features and different scanning protocol data, and help enhance the generalization ability and robustness of the model. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall structure of the 3D ABUS image classification network of the present invention.

[0018] Figure 2 This is a schematic diagram of the tokenized dual-branch basic block structure of the present invention.

[0019] Figure 3 This is a schematic diagram of the voxel token generation module of the present invention.

[0020] Figure 4 This is a schematic diagram of the three-dimensional multi-axis bidirectional selective state space scanning and adaptive routing fusion of the present invention.

[0021] Figure 5 This is a flowchart illustrating the three-dimensional ABUS image classification method of the present invention. Detailed Implementation

[0022] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0023] Example 1: 3D ABUS Image Classification Method In this embodiment, the three-dimensional ABUS image classification method is applied to the benign and malignant classification task of breast automated volumetric ultrasound imaging data. The data used is three-dimensional ABUS volume data, which comes from breast examination data collected clinically in hospitals. The three-dimensional ABUS volume data is sub-volume data of the lesion region extracted from complete breast volume data.

[0024] The three-dimensional ABUS volume data is acquired by an automated breast volume ultrasound device, forming three-dimensional voxel data arranged continuously along the thickness direction. To facilitate network processing, in this embodiment, the original three-dimensional ABUS volume data is first resampled to unify the voxel spacing to a preset resolution; then, the image intensity is normalized, and preprocessing operations such as speckle noise removal, contrast enhancement, and background cropping are performed.

[0025] In a preferred embodiment, the preprocessed 3D ABUS volume data is scaled to H×W×D = 256×256×128 and input into the network. For scenarios with small lesion areas, a 3D ROI cropping method centered on the lesion center is used to generate the input sub-volume. For whole breast input scenarios, candidate region extraction is performed, and then the data is input into the classification network for discrimination.

[0026] The specific steps are as follows: Step S1: Acquire 3D ABUS volume data S1.1: The patient lies supine with both hands raised overhead to fully expose the breasts and armpits, maintaining steady breathing. The operating physician selects the most suitable scanning parameters based on the thickness and density of the patient's breast tissue and applies coupling gel to the breast surface. The physician smoothly places the ultrasound probe against the breast surface and activates the ABUS fully automated scanning system. After locking, the system performs an automated scan of the entire breast, ultimately obtaining three-dimensional volumetric data (X-rays) including transverse, sagittal, and coronal plane information.

[0027] Step S2: Image Preprocessing The three-dimensional volume data X is preprocessed to obtain normalized volume data X′, the preprocessing including: (a) Voxel spacing resampling: X is resampled to a preset voxel spacing to reduce scale differences caused by different devices and protocols; (b) Intensity normalization: linear normalization or z-score normalization of the volume data; (c) Denoising and Enhancement: Median filtering, anisotropic diffusion or other speckle noise suppression methods are used, and contrast enhancement can be performed; (d) ROI cropping: A fixed-size 3D ROI is cropped with the suspected mass location as the center to reduce irrelevant background.

[0028] Step S3: 3D Patch Embedding The normalized volume data X′ is input into the 3D feature embedding module, and downsampling and channel mapping are performed using a 3D convolution with a stride of s (s = 2) to obtain the feature volume. Unlike methods that completely flatten volume data into a one-dimensional sequence, this invention maintains a three-dimensional mesh structure at this stage to facilitate subsequent voxel-level / block-level modeling.

[0029] Step S4: Layered Pyramid Encoder Will A hierarchical pyramid encoder is input to extract multi-scale semantic features. The encoder contains four stages, within which (2,2,4,2) tokenized bi-branch basic blocks are stacked sequentially. Between adjacent stages, resolution downsampling is achieved through 3D PatchMerging, thereby expanding the receptive field while preserving details of small lesions.

[0030] Step S5: Tokenize the dual-branch basic block (TS-Conv-3DSSM) Let the input feature body be ,in The batch size, For the number of channels, The spatial dimensions are specified. The basic block includes the following sub-steps: S5.1 Channel Division: Along the channel dimension Divided into and Both have the same number of channels. ; S5.2 Local Branch (Conv-Branch): For Lightweight 3D convolutional operations are performed to extract local texture and boundary morphology features. Two layers are used. Three-dimensional grouped convolutions are used, with batch normalization and non-linear activation following each convolution layer. Channel fusion is then performed through 1×1×1 pointwise convolutions to obtain local features. ; S5.3 Global Branch (SSM-Branch): For Execution layer normalization yields features , and by Generate gate vector ; then The token graph is obtained by inputting a voxel token generation module. ; For the token graph Perform a three-dimensional multi-axis bidirectional selective state-space scan and adaptive route fusion to obtain a fused token graph. ;Will Detokenization recovery to Consistent spatial resolution yields features and through Obtain global features ,in This represents element-wise multiplication; S5.4 Channel Interaction and Residual Fusion: and After concatenating the channel dimensions, channel shuffling is performed to facilitate information exchange between the two branches; subsequently, the shuffled features are compared with the input. The residuals are summed to obtain the output feature volume. .

[0031] Step S6: Voxel Token Generation Module (VTG) The voxel token generation module is used to compress sequence length while preserving 3D structure. (This is related to) normalized features. Token graph is generated through 3D convolution aggregation. Each token corresponds to a local 3D block in the original feature body, and the block size is... Because ABUS often has anisotropic resolution, setting... To better match the acquisition geometry in the thickness direction and the in-plane direction.

[0032] Step S7: Three-dimensional multi-axis bidirectional selective state-space scan (3DSS) token graph Multiple scan routes are constructed along the three-dimensional spatial axes, and forward and reverse scans are performed on each route to obtain bidirectional context. The scan routes include thickness-first routes. Plane-first routing For each route, the token graph is flattened into a sequence according to the corresponding order and scanned into a selective state space cell to obtain the route output sequence; then the route output sequence is rearranged back into the token grid.

[0033] Step S8: Route-Gating Since different cases and different mass morphologies contribute different amounts of effective information under different scanning routes, this invention introduces a route weight predictor to estimate each route weight vector. The outputs of each route are weighted and merged to obtain a merged token graph. The route weight predictor consists of global pooling, a linear layer, and a Softmax layer to output normalized weights.

[0034] Step S9: Sorting Head The feature volume output from the final stage of the encoder is input into the classification head, where 3D global average pooling is performed to obtain a global vector, which is then passed through a fully connected layer to output the class probability vector. .according to The highest probability category determines the classification result. It can also output a malignancy risk score.

[0035] Step S10: Model Training and Validation Setup S10.1 Training / Validation / Testing Division: In this embodiment, labeled 3D ABUS samples are used for supervised training of the 3D ABUS image classification network. To avoid data leakage, the dataset is divided according to case level, meaning that different body data corresponding to the same subject only appear in one of the training, validation, and test sets. Preferably, the training, validation, and test sets are divided in a 7:1:2 ratio, where the training set is used for model parameter learning, the validation set is used for training process monitoring and hyperparameter selection, and the test set is used for model generalization performance evaluation.

[0036] S10.2 Loss Function: During the training phase, a binary cross-entropy loss function is used. In cases of imbalanced class distribution, a weighted cross-entropy loss function can be further employed to improve the identification ability of minority class samples, especially malicious samples.

[0037] S10.3 Optimizer: The network parameters are iteratively updated by an optimizer, preferably the Adam optimizer, AdamW optimizer, or stochastic gradient descent optimizer. In a preferred embodiment, the AdamW optimizer is used.

[0038] S10.4 Learning rate: The initial learning rate is set to 1e-4 to 1e-3, the weight decay coefficient is set to 1e-5 to 1e-2, and the learning rate is adjusted in combination with a cosine annealing decay strategy or a step decay strategy.

[0039] S10.5 batch size: The training batch size is set to 2 to 16, preferably 4 or 8.

[0040] S10.6 Training rounds: The number of training rounds is set to 50 to 300, preferably 100 to 200.

[0041] S10.7 Stopping Criteria: During training, the model performance is monitored using a validation set, and the optimal model parameters are selected based on at least one of the following metrics: validation set loss, accuracy, AUC, and F1 score. In a preferred embodiment, an early stopping mechanism is used as the stopping criterion: when the validation set loss does not decrease within 10 to 30 consecutive training epochs, or the validation set AUC does not improve within a preset number of consecutive epochs, training is stopped, and the model parameters corresponding to the optimal validation set performance are retained.

[0042] S10.8 Model Selection Metrics: After training, the test set is input into the trained 3D ABUS image classification network to evaluate the model's performance. Evaluation metrics include at least one of accuracy, sensitivity, specificity, precision, F1 score, and AUC, among which AUC, sensitivity, and specificity are preferred as the main evaluation metrics for the benign and malignant breast lesion classification task.

[0043] In summary, this invention explicitly models the long-range dependence of 3D ABUS across slices while maintaining near-linear complexity through voxel tokenization and multi-axis bidirectional selective state-space scanning, and complements and integrates the local representation of lightweight 3D convolution, thereby achieving efficient and robust classification of benign and malignant breast ABUS masses.

[0044] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model, characterized in that, Includes the following steps: S1: Obtain the 3D ABUS volume data to be classified, preprocess the 3D ABUS volume data to obtain normalized volume data; S2: Input the normalized volume data into the 3D ABUS image classification network, output the category probability vector, and determine the classification result based on the category probability vector; The 3D ABUS image classification network includes a 3D feature embedding module, a hierarchical pyramid encoder, and a classification head; The 3D feature embedding module uses 3D convolution with a preset stride to downsample and map the normalized volume data to obtain a feature volume that preserves the 3D spatial structure. The hierarchical pyramid encoder consists of multiple cascaded stages, and the stages are transformed in resolution through a 3D downsampling fusion module to form a multi-scale feature pyramid from high resolution to low resolution. Each stage contains several tokenized dual-branch basic blocks, including a first branch and a second branch; The tokenized dual-branch basic block divides the input feature body along the channel dimension to obtain the first branch feature and the second branch feature. The first branch performs a lightweight 3D convolution operation on the first branch features to extract local texture and boundary morphology features, thus obtaining local features; The second branch performs the following processing on the second branch features to obtain the global features: The second branch performs feature normalization to obtain normalized features, generates gate vectors from normalized features, and generates a token graph from voxel tokens of normalized features, where each token corresponds to a three-dimensional local block in the normalized features; Perform a 3D multi-axis selective state-space scan on the token graph to obtain a multi-route scan output; Adaptive weighted fusion is performed on each scan output to obtain a fused token graph; Perform detokenization on the fused token graph to restore spatial resolution consistent with the normalized features; The global features are obtained by element-wise multiplying the features obtained from detokenization with the gated vector through gated modulation. After concatenating the local and global features along the channel dimension, channel shuffling is performed, and residual fusion is performed with the input feature volume to obtain the output feature volume; Voxel token generation aggregates normalized features into a token graph, with each token corresponding to a 3D local block; The classification head performs three-dimensional global average pooling on the feature volume output by the last stage to obtain a global vector, which is then passed through a fully connected layer or a multi-layer perceptron to output a class probability vector.

2. The three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model according to claim 1, characterized in that, Preprocessing of 3D ABUS volume data should include at least one of the following methods: voxel spacing resampling, intensity normalization, denoising, contrast enhancement, and region of interest clipping.

3. The three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model according to claim 1, characterized in that, The 3D downsampling fusion module is implemented using 3D Patch Merging with a step size of 2.

4. The three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model according to claim 1, characterized in that, Three-dimensional multi-axis selective state space scanning includes multiple scanning directions, including at least one of thickness direction scanning and in-plane direction scanning. Each scanning direction performs forward scanning and reverse scanning.

5. The three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model according to claim 4, characterized in that, Adaptive weighted fusion is performed on each scan output, including predicting the weight vector for each scan direction through a route weight predictor and performing a weighted summation on the scan outputs for each scan direction.

6. The three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model according to claim 1, characterized in that, Selective state-space scanning employs an input-dependent selective parameter modulation mechanism to capture long-range dependencies with near-linear computational complexity.

7. The three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model according to claim 1, characterized in that, The gating vector is obtained by performing global pooling or pointwise convolution on the second branch features or normalized features and then applying the Sigmoid function. It is used to control the intensity of global information injection.

8. The three-dimensional breast ABUS image classification method based on a tokenized bi-branch selective state-space model according to claim 1, characterized in that, Detokenization restores the fused token map to a spatial resolution consistent with the normalized features through upsampling and / or deconvolution, and introduces skip connections during the restoration process to preserve low-level detail information.

9. An electronic device, characterized in that, Including processor and memory; The system stores computer program instructions that, when executed by a processor, cause the processor to perform the three-dimensional breast ABUS image classification method based on the tokenized bi-branch selective state space model as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The system stores a computer program that, when executed by a processor, causes the processor to perform the three-dimensional breast ABUS image classification method based on the tokenized bi-branch selective state space model as described in any one of claims 1-8.

Images