Intelligent screening and benign and malignant auxiliary identification system for tiny tumors in ultrasonic image

Abstract

The invention relates to the field of medical image processing, and provides an intelligent screening and benign and malignant auxiliary identification system for a tiny tumor in an ultrasonic image, which comprises a dynamic ultrasonic sequence preprocessing module, a space-time double-flow feature extraction module, a multi-scale focus detection module, a space-time feature fusion module and an interpretable classification decision module, the system extracts the elasticity and mobility characteristics of a focus in an ultrasonic dynamic image through an optical flow method, performs multi-scale focus detection and benign and malignant classification in combination with static morphological characteristics, outputs an interpretable thermodynamic diagram to provide decision support for a doctor, realizes parameter adaptive optimization through a closed-loop feedback mechanism, and improves the detection accuracy. The method is particularly suitable for screening small tumors with the diameter smaller than 1 cm, and the classification accuracy and the detection sensitivity are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence, specifically to an intelligent screening and benign / malignant tumor identification system in ultrasound imaging. Background Technology

[0002] Early detection and accurate identification of small tumors are crucial for improving the survival rate of cancer patients. Ultrasound examination, due to its non-invasive, radiation-free, and low-cost advantages, has become the preferred imaging method for screening tumors of superficial organs such as breast nodules and thyroid nodules. However, for small tumors less than 1 cm in diameter, due to their small size, indistinct borders, and atypical echogenicity, traditional ultrasound diagnosis relies heavily on the doctor's clinical experience and subjective judgment, which can easily lead to missed or misdiagnosed cases.

[0003] Chinese patent CN119515809A discloses a multimodal breast ultrasound diagnostic system based on a dynamic feature fusion network. The system includes a multimodal data preprocessing module, a multimodal convolutional neural network module, a model training module, and a model testing module. It achieves benign / malignant classification of breast tumors by extracting and fusing features from three static multimodal images: B-mode ultrasound, color Doppler flow imaging, and ultrasound elastography. The system employs a deformable spatial attention module to enhance feature extraction capabilities and quantifies the differences in the contribution of each modality's features to the diagnostic results using a Dirichlet distribution, thus achieving dynamic fusion of multimodal features.

[0004] However, the aforementioned existing technologies still have the following technical problems: First, the system only processes static multimodal images and cannot utilize the elastic deformation and mobility information of lesions generated by changes in probe pressure during dynamic ultrasound scanning. These dynamic features are of great reference value for differentiating between benign and malignant small tumors. Benign tumors usually exhibit good elasticity and mobility, while malignant tumors are relatively rigid and adhere to surrounding tissues. Second, the system lacks a dedicated detection mechanism for small tumors (less than 1 cm in diameter). Existing feature extraction networks are not sensitive enough to detect small targets and are prone to missing early small lesions. Third, although the system can output classification results, it lacks intuitive and interpretable output. Doctors find it difficult to understand the basis of the model's judgment, which is not conducive to clinical decision-making and diagnostic trust. Finally, the system lacks a closed-loop feedback mechanism between modules. The classification results cannot back-optimize the processing parameters of the preceding modules, limiting the system's adaptive adjustment capability. Summary of the Invention

[0005] To address the aforementioned technical problems in existing technologies, this invention provides an intelligent screening and benign / malignant tumor identification system for small tumors in ultrasound imaging. This system can simultaneously extract static morphological features and optical flow dynamic features from ultrasound dynamic video streams, enabling accurate detection and benign / malignant identification of small tumors, and providing doctors with intuitive decision support through interpretable heat maps.

[0006] The intelligent screening and benign / malignant tumor identification system for small tumors in ultrasound imaging provided by this invention includes a dynamic ultrasound sequence preprocessing module, a spatiotemporal dual-stream feature extraction module, a multi-scale lesion detection module, a spatiotemporal feature fusion module, and an interpretable classification decision module. The dynamic ultrasound sequence preprocessing module receives the ultrasound video stream acquired by the ultrasound equipment, extracts frames according to a preset temporal frame interval, performs grayscale normalization and spatial alignment on each extracted frame image, and outputs a standardized frame sequence. The spatiotemporal dual-stream feature extraction module receives the standardized frame sequence, extracts morphological features, edge features, and echo intensity features through a static feature extraction unit to generate a static morphological feature map, and simultaneously calculates the optical flow field between adjacent frames and extracts the elastic deformation and mobility features of the lesion region through an optical flow dynamic feature extraction unit to generate an optical flow dynamic feature map. The multi-scale lesion detection module outputs a set of candidate lesion regions with a confidence threshold through pyramid pooling and candidate region screening mechanisms. The spatiotemporal feature fusion module performs weighted fusion of static and dynamic features through channel attention and spatial attention mechanisms, and outputs a fused feature vector. The interpretability classification decision module outputs the benign or malignant probability values ​​of lesions and an interpretable heatmap indicating the judgment basis area, and generates a feedback signal to be transmitted to the front-end module to achieve closed-loop optimization.

[0007] The beneficial effects of this invention are as follows: First, this invention simultaneously extracts static morphological features and optical flow dynamic features through a spatiotemporal dual-flow feature extraction module, making full use of the elasticity and mobility information of lesions during dynamic ultrasound scanning. Compared with existing technologies that only use static multimodal images, the accuracy of differentiating between benign and malignant small tumors is improved by 8% to 12%. Second, this invention employs a multi-scale lesion detection module, which enhances the detection capability of lesions of different scales through a pyramid pooling structure, improving the sensitivity of detecting small tumors with a diameter of less than 1 cm by 15% to 20%. Third, this invention generates a gradient-weighted class activation map heatmap through an interpretable classification decision module, intuitively displaying the key image feature regions on which the model's judgment is based, improving the interpretability of the diagnosis and the doctor's trust in the system. Fourth, this invention establishes a closed-loop feedback mechanism from the classification decision module to the feature extraction module and the lesion detection module, realizing adaptive optimization of system parameters and improving the robustness of the system in different application scenarios. Attached Figure Description

[0008] Figure 1 This is the overall architecture diagram of the system of the present invention;

[0009] Figure 2 This is a detailed structural diagram of the spatiotemporal dual-stream feature extraction module;

[0010] Figure 3 This is a detailed structural diagram of the multi-scale lesion detection module. Detailed Implementation

[0011] Please refer to the attached document. Figures 1-3 The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0012] The intelligent screening and benign / malignant differentiation system for small tumors in ultrasound imaging provided by this invention focuses on the screening and risk assessment of early-stage small tumors, and is particularly suitable for differentiating the benign / malignant nature of breast nodules and thyroid nodules with a diameter of less than 1 cm. Figure 1 As shown, the system mainly includes a dynamic ultrasound sequence preprocessing module 1, a spatiotemporal dual-stream feature extraction module 2, a multi-scale lesion detection module 3, a spatiotemporal feature fusion module 4, and an interpretable classification decision module 5. These five modules form a deeply coupled closed-loop collaborative architecture. The output of the previous module serves as the key input of the next module, and the results of the subsequent module can inversely influence the processing parameters of the previous module, thereby achieving synergistic optimization of the overall system performance.

[0013] The dynamic ultrasound sequence preprocessing module 1 is the data entry point for the entire system, responsible for standardizing the raw ultrasound video stream to provide high-quality input data for subsequent feature extraction and lesion detection. This module receives the ultrasound video stream acquired in real time by the ultrasound equipment. The original resolution of the video stream is required to be no less than 512×512 pixels, and the frame rate is required to be no less than 25 frames per second. In a preferred embodiment of the present invention, an ultrasound video stream with a resolution of 1024×768 pixels and a frame rate of 30 frames per second is used as input.

[0014] The dynamic ultrasound sequence preprocessing module 1 includes a region of interest (ROI) extraction unit, a temporal frame sampling unit, a grayscale normalization unit, and a spatial alignment unit. The ROI extraction unit first extracts the region of interest containing the target tissue from each frame of the ultrasound video stream. For breast ultrasound, the ROI includes the breast glandular tissue region; for thyroid ultrasound, the ROI includes the thyroid parenchyma region. The ROI extraction employs a method combining grayscale thresholding and connected component analysis. First, median filtering is applied to the original image to remove speckle noise. Then, adaptive thresholding is used to extract the tissue region. Finally, connected component analysis is used to select the connected region with the largest area as the ROI. Preferably, the ROI also undergoes histogram equalization to enhance image contrast and make the grayscale difference between the lesion area and normal tissue more obvious.

[0015] The temporal frame sampling unit extracts frame sequences from the ultrasound video stream according to a preset temporal frame interval. The setting of the temporal frame interval needs to strike a balance between ensuring sufficient temporal resolution and controlling computational complexity. In this invention, the temporal frame interval ranges from 30ms to 100ms. Preferably, the temporal frame interval is set to 50ms, meaning that one frame is sampled every 1.5 frames from a video stream of 30 frames per second, ultimately forming a standardized frame sequence of approximately 20 frames per second. Experimental verification shows that this sampling interval can effectively capture the elastic response characteristics of the lesion during probe pressure changes, while controlling the complexity of subsequent calculations.

[0016] The gray-level normalization unit standardizes the gray-level values ​​of each sampled image frame. Since different ultrasound devices and scanning parameter settings may cause differences in image gray-level distribution, gray-level normalization can eliminate these differences and improve the system's generalization ability. Gray-level normalization uses a linear transformation method to map the original gray-level values ​​to a standardized range of 0 to 1. The specific calculation method is as follows: Let the original image pixel gray-level value be... The grayscale value range is Then the normalized gray value The calculation formula is as follows:

[0017] ,

[0018] in, This represents the original pixel grayscale value; The minimum gray value in the image; The maximum gray value in the image; The value is the normalized grayscale value, ranging from 0 to 1.

[0019] The spatial alignment unit performs spatial registration on each frame in the frame sequence to eliminate inter-frame displacement caused by slight hand-held probe shaking or patient breathing movements. Spatial alignment adopts a rigid registration method based on mutual information, using the first frame as the reference frame, and subsequent frames are aligned to the coordinate system of the reference frame through translation and rotation transformations. The registered images are uniformly cropped or padded to a standard size of 224×224 pixels, forming a standardized frame sequence that is output to the spatiotemporal dual-stream feature extraction module 2.

[0020] The spatiotemporal dual-stream feature extraction module 2 is one of the core innovative modules of this invention, such as... Figure 2 As shown, this module employs a dual-flow architecture to simultaneously extract static morphological features and optical flow dynamic features. Unlike existing technologies that only process static multimodal images, this invention analyzes the elastic deformation and mobility characteristics of lesion regions in dynamic ultrasound images using optical flow methods. These dynamic features are of significant value in differentiating between benign and malignant small tumors. Malignant tumors, due to their active cell proliferation, dense tissue, and adhesion to surrounding tissues, typically exhibit poor elasticity and mobility; benign tumors, on the other hand, have soft tissue and exhibit better elastic deformation and displacement when probe pressure changes.

[0021] The spatiotemporal dual-stream feature extraction module 2 includes a static feature extraction unit and an optical flow dynamic feature extraction unit. The static feature extraction unit uses a deep residual network to extract features from each frame of the standardized frame sequence. Preferably, ResNet-50 is used as the backbone network. This network effectively solves the gradient vanishing problem during deep network training through residual connections, enabling the extraction of high-level features with rich semantic information. The static feature extraction unit obtains a feature map from the output of the fourth residual block of ResNet-50. This feature map includes morphological features, edge features, and echo intensity features. Morphological features reflect the regularity of the lesion's shape and aspect ratio; malignant tumors typically exhibit irregular shapes and spiky edges. Edge features reflect the clarity and smoothness of the lesion's boundary; the edges of malignant tumors are usually blurred or crab-like. Echo intensity features reflect the uniformity of echoes within the lesion; the echoes within malignant tumors are typically non-uniform. The static feature extraction unit extracts features from frames in the frame sequence at preset static sampling intervals. Preferably, the static sampling interval is set to 3 frames, that is, one frame is extracted every 3 frames for static feature calculation to generate a static morphological feature map sequence.

[0022] The optical flow dynamic feature extraction unit is a key innovation of this invention. It employs a dense optical flow estimation algorithm to calculate the optical flow field between adjacent frames and extracts the elastic deformation and mobility features of the lesion region based on the optical flow field. The optical flow field represents the displacement vector of each pixel in the image from the current frame to the next frame, reflecting the movement and deformation of tissue during probe pressure changes. Preferably, the optical flow dynamic feature extraction unit uses an improved PWC-Net (Pyramid, Warping and Cost Volume Network) to calculate the dense optical flow field. This network achieves coarse-to-fine optical flow estimation through a pyramid structure, resulting in high computational accuracy and efficiency.

[0023] Let the first Frame image is , No. Frame image is Optical flow field The calculation is based on the assumption of constant brightness and spatial smoothness constraints. The temporal frame differential enhancement algorithm proposed in this invention first performs differential preprocessing on adjacent frames to enhance information in dynamically changing regions. The calculation formula for temporal frame differential enhancement is as follows:

[0024] ,

[0025] in, For the first Frame and the Enhanced differential image of frames; For the first Frame image; For the first Frame image; For absolute value operations; For the first The Laplacian operator response of a frame image is used to detect edge regions; The edge suppression coefficient ranges from 0.1 to 0.5, with a preferred value of 0.2. It is an exponential function. This formula reduces the differential response of static edge regions through an edge suppression factor, highlighting information from dynamic motion regions.

[0026] Based on the enhanced differential image, the optical flow dynamic feature extraction unit calculates the optical flow field. ,in This represents the horizontal displacement component. This represents the vertical displacement component. The optical flow elastic feature calculation algorithm proposed in this invention extracts the elastic deformation features of the lesion region from the optical flow field. First, the deformation gradient tensor of the optical flow field is calculated:

[0027] ,

[0028] in, For the first The deformation gradient tensor of the frame; For the horizontal displacement component along Gradient of direction; For the horizontal displacement component along Gradient of direction; For the vertical displacement component along Gradient of direction; For the vertical displacement component along Gradient of direction.

[0029] Based on the deformation gradient tensor, the strain tensor of the elastic deformation characteristics is calculated:

[0030] ,

[0031] in, The strain tensor reflects the degree of local deformation of the tissue; This is the transpose matrix of the deformation gradient tensor. Elastic deformation characteristics. The Frobenius norm is defined as the strain tensor:

[0032] ,

[0033] in, These are the characteristic values ​​of elastic deformation; The Frobenius norm of the strain tensor; For the strain tensor, the first Line number The elements in the column. The larger the elastic deformation characteristic value, the more obvious the deformation of the area when the probe pressure changes, which usually corresponds to benign tissue; the smaller the elastic deformation characteristic value, the more rigid the area is, which may correspond to malignant tumors.

[0034] Mobility characteristics are calculated based on the temporal accumulation of the optical flow field. Assume the frame sequence contains... Frame, center point of lesion area The trajectory features are calculated as follows:

[0035] ,

[0036] in, The center point of the lesion passes through The cumulative position after the next displacement; The coordinates of the center point of the lesion in the initial frame; For the first The frame light flow field at the current position The displacement vector; the position of the next frame. It is obtained by adding the displacement vector to the current position. Mobility characteristics. Defined as the ratio of trajectory length to time:

[0037] ,

[0038] in, This is a mobility characteristic value; The Euclidean distance is the distance from the starting point to the ending point. This represents the frame interval. Mobility eigenvalues ​​reflect the overall displacement of the lesion during dynamic scanning. Benign tumors, due to their clear boundaries with surrounding tissues, typically exhibit larger mobility eigenvalues.

[0039] The optical flow dynamic feature extraction unit encodes elastic deformation features and mobility features into an optical flow dynamic feature map. This feature map has the same spatial dimension as the static morphological feature map, which facilitates subsequent feature fusion processing.

[0040] The multi-scale lesion detection module 3 is responsible for locating suspicious lesion areas from the feature map, such as... Figure 3 As shown, this module employs a feature pyramid structure to enhance the detection capability of lesions at different scales, with specific optimizations for tiny tumors less than 1 cm in diameter. Existing target detection networks lack sufficient sensitivity for small targets, and tiny tumors often occupy only a small area in ultrasound images, making them easily missed. This invention significantly improves the detection sensitivity of tiny lesions through multi-scale feature fusion and adaptive threshold adjustment mechanisms.

[0041] The multi-scale lesion detection module 3 includes a pyramid pooling unit, a candidate region generation unit, and a candidate region filtering unit. The pyramid pooling unit performs multi-scale feature fusion on the static morphological feature map and the optical flow dynamic feature map. Preferably, the pyramid pooling unit adopts a three-layer pyramid structure, with pooling scales of the original scale, half scale, and quarter scale. Let the input feature map size be... ,in For height, For width, Given the number of channels, the feature map dimensions of the three-layer pyramid are respectively... , and Larger-scale feature maps preserve spatial detail, which is beneficial for detecting small lesions; smaller-scale feature maps have a larger receptive field, which is beneficial for detecting larger lesions.

[0042] The multi-scale lesion confidence weighting algorithm proposed in this invention fuses detection results from different scales. Let the... The detection confidence level of the layer pyramid is The scale weight is The confidence level after fusion The calculation is as follows:

[0043] ,

[0044] in, The confidence level for detecting the fused lesions; The number of pyramid layers, in this invention ; For the first The scale weights of the layers, with preferred values ​​as follows: , , Larger-scale layers are given higher weights to improve the sensitivity of detecting small lesions; For the first Layer detection confidence; For the first The confidence threshold of the layer; This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise; To prevent small positive numbers from being divided by zero, the value is taken as... .

[0045] The candidate region generation unit generates candidate regions for suspicious lesions based on an improved region proposal network. This unit includes an anchor box generation subunit and a region proposal subunit. The anchor box generation subunit generates a preset number of anchor boxes with preset aspect ratios at various locations on the multi-scale feature map. Considering that small tumors are usually approximately circular or elliptical, preferably, the aspect ratio of the anchor boxes is set to three types: 1:1, 1:2, and 2:1; the basic size of the anchor boxes is set according to the pyramid hierarchy: 16 pixels for the first layer, 32 pixels for the second layer, and 64 pixels for the third layer. The region proposal subunit calculates the foreground probability and bounding box regression offset for each anchor box, and uses two parallel 3×3 convolutional layers to output the foreground / background classification score and the bounding box coordinate offset, respectively. Anchor boxes with foreground probabilities higher than a preset foreground threshold are output as candidate regions for suspicious lesions; preferably, the foreground threshold is set to 0.3.

[0046] The candidate region screening unit filters suspicious lesion candidate regions according to the lesion detection confidence threshold. The initial value of the lesion detection confidence threshold ranges from 0.5 to 0.8, with a preferred initial value of 0.6. This threshold is not fixed but dynamically adjusted based on the detection feedback signal; the specific adjustment mechanism will be explained in detail in the interpretability classification decision module 5. The candidate region screening unit also uses non-maximum suppression (NMS) to remove candidate regions with excessive overlap. The cross-union ratio (CUI) threshold for NMS is set to 0.5, meaning that when the CUI of two candidate regions is greater than 0.5, the region with higher confidence is retained, and the region with lower confidence is deleted. The filtered set of candidate lesion regions is output to the spatiotemporal feature fusion module 4.

[0047] The spatiotemporal feature fusion module 4 is responsible for fusing static morphological features and optical flow dynamic features. Simple feature splicing or addition cannot fully utilize the complementary information between the two features. This invention adopts a feature fusion method based on attention mechanism, which learns the importance weights of different feature channels and spatial positions to achieve adaptive fusion of static and dynamic features.

[0048] The spatiotemporal feature fusion module 4 includes a channel attention calculation unit, a spatial attention calculation unit, and a feature weighting fusion unit. The channel attention calculation unit learns the importance weights of each feature channel. Static morphological feature maps and optical flow dynamic feature maps each contain multiple feature channels; different channels correspond to different types of feature information, and some channels may contain features more valuable for benign / malignant differentiation. The channel attention calculation unit first performs global average pooling and global max pooling on the feature maps to obtain two channel descriptors. Then, the two descriptors are input into a multilayer perceptron network with shared parameters. Finally, the two outputs are summed and passed through a sigmoid activation function to obtain the channel attention weights.

[0049] The process of calculating channel attention weights in the spatiotemporal dual-stream attention fusion algorithm proposed in this invention is as follows. Let the input feature map be... ,in For the number of channels, and These represent the height and width of the feature map, respectively. Global average pooling and global max pooling are calculated as follows:

[0050] ,

[0051] ,

[0052] in, The channel descriptor obtained from global average pooling; The channel descriptor obtained from global max pooling; Indicates the feature map at location All channel values ​​at that location.

[0053] A multilayer perceptron network with shared parameters for channel descriptor inputs, containing one hidden layer with a number of neurons. ,in The channel compression ratio is set to 16, with an optimal value of 16. The channel attention weights are calculated as follows:

[0054] ,

[0055] in, This is the channel attention weight vector; It is a multilayer perceptron network, containing two fully connected layers. The first fully connected layer will... Dimensional input mapping to Dimension, the second fully connected layer will Dimension Mapping Back The ReLU activation function is used between the two layers; The Sigmoid activation function maps the output value to the range of 0 to 1.

[0056] The spatial attention computation unit learns the importance weights of each spatial location. Features of the lesion region are more important for classifying benign or malignant tumors, while features of the background region have lower value. The spatial attention computation unit first performs average pooling and max pooling along the channel dimension on the channel-weighted feature map to obtain two spatial descriptors. Then, the two descriptors are concatenated along the channel dimension and input into a 7×7 convolutional layer. Finally, the spatial attention weights are obtained by passing the sigmoid activation function. The spatial attention weights are calculated as follows:

[0057] ,

[0058] in, This is the spatial attention weight matrix; This is the channel-weighted feature map. This represents element-wise multiplication of the channel dimension. and These represent average pooling and max pooling along the channel dimension, respectively. This indicates splicing along the channel dimension; This is a 7×7 convolution operation; This is the Sigmoid activation function.

[0059] The feature-weighted fusion unit applies channel attention weights and spatial attention weights to static morphological feature maps. And optical flow dynamic feature map Weighted fusion is performed. This invention employs an adaptive fusion strategy, introducing learnable feature fusion weight coefficients. and The fusion features are calculated as follows:

[0060] ,

[0061] in, To fuse feature maps; and For feature fusion weight coefficients, satisfying The initial values ​​are all set to 0.5, and they are automatically learned through backpropagation during training. and These are the channel attention weights for static and dynamic features, respectively; Spatial attention weights; This represents an element-wise multiplication operation. Preferably, the feature fusion weight coefficients... and The final convergence value ranges from 0.3 to 0.7.

[0062] The fused feature map undergoes global average pooling to compress its spatial dimension, resulting in a one-dimensional fused feature vector, which is output to the interpretable classification decision module 5. For each candidate region in the candidate lesion region set, the spatiotemporal feature fusion module 4 extracts its fused feature vector to form a lesion feature vector set.

[0063] The interpretable classification decision module 5 is the final decision module of the system of this invention. This module not only outputs the benign or malignant classification probability of the lesion, but also generates an interpretable heatmap, which intuitively displays the key imaging feature regions on which the model's judgment is based, providing decision support for doctors. Unlike the black-box models of existing technologies, this invention realizes the visual explanation of the model's decision-making process through gradient-weighted class activation mapping technology, improving the system's credibility and clinical applicability.

[0064] The interpretable classification decision module 5 includes a fully connected classification unit, a gradient-weighted class activation mapping unit, and a closed-loop feedback unit. The fully connected classification unit inputs the fused feature vector into a two-layer fully connected network for classification. The first fully connected layer maps the fused feature vector from its original dimensions to 512 dimensions, using ReLU activation and Dropout regularization, with the Dropout ratio set to 0.5. The second fully connected layer maps the 512-dimensional features to a 2D output, corresponding to the benign and malignant categories, and uses the Softmax activation function to output the probability values ​​for each category. Let the probability value for the malignant category be... The classification decision threshold is The determination rule is: when If the threshold is met, it is classified as a malignant tumor; otherwise, it is classified as a benign tumor. Classification decision threshold. The value ranges from 0.4 to 0.6, with a preferred value of 0.5.

[0065] Gradient-weighted class activation map units generate interpretable heatmaps. The gradient-weighted class activation map generation algorithm proposed in this invention is an improvement upon the Grad-CAM principle. It determines the contribution of each feature channel and spatial location to the classification decision by calculating the gradient of the classification probability relative to the feature map. The specific calculation process is as follows:

[0066] Let the feature map output by the last convolutional layer be... The classification probability is (Regarding the classification of benign and malignant, (Taking the malignancy category), first calculate the gradient of the classification probability with respect to the feature map:

[0067] ,

[0068] in, For the first Feature maps of each channel. Global average pooling is performed on the gradients to obtain the weights for each channel:

[0069] ,

[0070] in, For the first Each channel is a category Importance weights; For the first Each channel feature map at location The value of . The channel weights are weighted and summed with the corresponding feature maps, then processed by the ReLU activation function:

[0071] ,

[0072] in, For category Class activation mapping; The ReLU activation function is used, with negative values ​​set to zero, retaining only regions that positively contribute to classification. The class activation map is then upsampled to the original image resolution using bilinear interpolation, resulting in an interpretable heatmap. In the heatmap, darker colors (larger values) indicate greater contributions to classification decisions, allowing doctors to assess the rationality of the model's judgment criteria.

[0073] The closed-loop feedback unit is one of the key innovations of this invention, realizing a closed-loop feedback mechanism from the classification decision module to the front-end module. The closed-loop parameter adaptive update algorithm proposed in this invention includes two parts: classification feedback signal generation and detection feedback signal generation.

[0074] The classification feedback signal, generated based on the gradient of the classification loss function, is passed to the spatiotemporal dual-stream feature extraction module 2 to update the parameters of the feature extraction network. During the training phase, the cross-entropy loss function is used:

[0075] ,

[0076] in, For classification loss; For unique hot encoding of real labels, benign Malignant ; The category predicted by the model Probability. The classification loss updates the network parameters of the static feature extraction unit and the optical flow dynamic feature extraction unit in the spatiotemporal dual-stream feature extraction module 2 through backpropagation.

[0077] The detection feedback signal, calculated based on the classification accuracy, is transmitted to the multi-scale lesion detection module 3 to adjust the lesion detection confidence threshold. Let the classification accuracy of the most recent batch of samples be... The preset accuracy threshold is The preferred value is 0.85, which is the detection confidence threshold. The update rules are as follows:

[0078] ,

[0079] in, The updated detection confidence threshold; This is the current detection confidence threshold; The threshold adjustment step size ranges from 0.01 to 0.1, with a preferred value of 0.05. This represents the current classification accuracy. This is a preset accuracy threshold. When the classification accuracy falls below the preset threshold, The detection confidence threshold is lowered, allowing more candidate lesions to enter the classification stage and avoiding omissions; when the classification accuracy is higher than the preset threshold, This increases the detection confidence threshold, reduces false positive candidate regions, and improves system efficiency. (Detection confidence threshold) The value range is limited to between 0.3 and 0.9 to prevent the threshold from being too low, resulting in too many candidate regions, or too high, resulting in too few candidate regions.

[0080] Through the aforementioned closed-loop feedback mechanism, the system of this invention can adaptively adjust the sensitivity of lesion detection based on the actual classification effect, thereby controlling the false positive rate while ensuring detection sensitivity and optimizing the overall performance of the system.

[0081] In one specific embodiment of the present invention, the system uses an NVIDIA RTX 3090 GPU as the computing platform and is implemented using the PyTorch deep learning framework. The training dataset contains 1200 dynamic breast ultrasound videos and 800 dynamic thyroid ultrasound videos from a tertiary hospital, with a benign to malignant case ratio of approximately 3:2. All cases were confirmed by pathological biopsy. The video duration is 3 to 5 seconds, the frame rate is 30 frames per second, and the resolution is 1024×768 pixels. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio.

[0082] The model was trained using the Adam optimizer, with an initial learning rate set to... A cosine annealing strategy was used to dynamically adjust the learning rate. The batch size was set to 8, and the number of training epochs was set to 100. An early stopping strategy was adopted during training: training was stopped when the validation set loss no longer decreased for 10 consecutive epochs to avoid overfitting.

[0083] Performance evaluation results on the test set show that the system of this invention achieves a benign / malignant classification accuracy of 93.5%, a sensitivity of 91.2%, a specificity of 95.1%, and an area under the curve of 0.968 for small tumors with a diameter of less than 1 cm. Compared with the method based on static multimodal images disclosed in Chinese invention patent CN119515809A, the classification accuracy of the system of this invention is improved by approximately 10%, especially in terms of sensitivity for detecting small lesions, which is improved by approximately 18%. Interpretable heatmaps, as evaluated by clinicians, show that over 90% of the heatmaps accurately identify the key feature areas of the lesions, highly consistent with the doctors' diagnostic criteria.

[0084] In summary, the intelligent screening and benign / malignant tumor identification system for small tumors in ultrasound imaging provided by this invention achieves joint modeling of static and dynamic features through spatiotemporal dual-stream feature extraction, enhances the detection capability of small tumors through multi-scale lesion detection, achieves adaptive weighting of features through attention fusion mechanism, provides intuitive decision-making basis through interpretable heatmaps, and achieves adaptive optimization of system parameters through closed-loop feedback mechanism. It has significant technical advantages and clinical application value in intelligent screening and benign / malignant tumor identification of small tumors.

Claims

1. A system for intelligent screening and benign / malignant differentiation of small tumors in ultrasound imaging, characterized in that, include: The dynamic ultrasound sequence preprocessing module is used to receive the ultrasound video stream acquired by the ultrasound equipment, extract frames from the ultrasound video stream according to a preset time frame interval, perform grayscale normalization and spatial alignment processing on each extracted frame image, and output a standardized frame sequence. The spatiotemporal dual-stream feature extraction module is used to receive the standardized frame sequence, extract morphological features, edge features and echo intensity features from each frame image through the static feature extraction unit to generate a static morphological feature map, and calculate the optical flow field between adjacent frames through the optical flow dynamic feature extraction unit, and extract the elastic deformation features and mobility features of the lesion area based on the optical flow field to generate an optical flow dynamic feature map. A multi-scale lesion detection module is used to receive the static morphological feature map and the optical flow dynamic feature map, perform multi-scale feature fusion on the static morphological feature map and the optical flow dynamic feature map through a pyramid pooling unit, generate candidate regions of suspicious lesions through a candidate region generation unit based on the fused multi-scale feature map, and filter the candidate regions of suspicious lesions according to a lesion detection confidence threshold through a candidate region filtering unit, and output a set of candidate lesion regions that are higher than the lesion detection confidence threshold. The spatiotemporal feature fusion module is used to receive the candidate lesion region set and the corresponding static morphological feature map and optical flow dynamic feature map, calculate the channel attention weight through the channel attention calculation unit, calculate the spatial attention weight through the spatial attention calculation unit, and perform weighted fusion of the static morphological feature map and optical flow dynamic feature map based on the channel attention weight and the spatial attention weight, and output the fused feature vector.

2. The system according to claim 1, characterized in that, It also includes an interpretable classification decision module, which receives the fused feature vector, outputs the benign or malignant probability value of the lesion through a fully connected layer and a classifier, determines the benign or malignant probability value based on the classification decision threshold, generates an interpretable heatmap indicating the region on which the judgment is based through a gradient-weighted class activation mapping unit, and generates a classification feedback signal and a detection feedback signal based on the benign or malignant probability value. The classification feedback signal is transmitted to the spatiotemporal dual-stream feature extraction module to update the feature extraction parameters, and the detection feedback signal is transmitted to the multi-scale lesion detection module to adjust the lesion detection confidence threshold.

3. The system according to claim 1, characterized in that, The temporal frame interval ranges from 30ms to 100ms; the static feature extraction unit uses a deep residual network to extract the morphological features, edge features, and echo intensity features; the optical flow dynamic feature extraction unit uses a dense optical flow estimation algorithm to calculate the optical flow field. The initial value range of the lesion detection confidence threshold is 0.5 to 0.8; the pyramid pooling unit adopts a three-layer pyramid structure, with pooling scales of the original scale, half scale, and quarter scale; the value range of the feature fusion weight coefficient is 0.3 to 0.

7.

4. The system according to claim 2, characterized in that, The classification decision threshold ranges from 0.4 to 0.6; the spatial resolution of the interpretability heatmap matches the spatial resolution of the input image; when the benign / malignant probability value is higher than the classification decision threshold, it is determined to be a malignant tumor.

5. The system according to claim 1, characterized in that, The optical flow dynamic feature extraction unit obtains the optical flow field by calculating the displacement vector field of pixels between adjacent frames, calculates the deformation gradient tensor of the lesion region based on the optical flow field, extracts the elastic deformation feature from the deformation gradient tensor, calculates the trajectory feature of the lesion region based on the temporal accumulation of the optical flow field, and extracts the mobility feature from the trajectory feature.

6. The system according to claim 1, characterized in that, The channel attention calculation unit obtains channel descriptors by performing global average pooling and global max pooling on the static morphological feature map and the optical flow dynamic feature map, respectively, and inputs the channel descriptors into a shared multilayer perceptron network to calculate channel attention weights. The spatial attention calculation unit obtains a spatial descriptor by performing average pooling and max pooling along the channel dimension on the feature map after being weighted by the channel attention weights, and then calculates the spatial attention weights after processing the spatial descriptor through a convolutional layer.

7. The system according to claim 1, characterized in that, The gradient-weighted class activation mapping unit calculates the gradient of the benign / malignant probability value relative to the feature map of the last convolutional layer, performs global average pooling on the gradient to obtain the weights of each feature channel, and then performs weighted summation with the feature map of the corresponding feature channel before processing by the activation function to generate the interpretable heatmap.

8. The system according to claim 1, characterized in that, The candidate region generation unit includes an anchor box generation subunit and a region suggestion subunit. The anchor box generation subunit generates a preset number of anchor boxes with a preset aspect ratio at each position of the multi-scale feature map. The region suggestion subunit calculates the foreground probability and bounding box regression offset for each anchor box, and outputs the anchor boxes with foreground probabilities higher than a preset foreground threshold as the candidate regions of the suspected lesions.

9. The system according to claim 1, characterized in that, The detection feedback signal is calculated based on the classification accuracy. When the classification accuracy is lower than the preset accuracy threshold, the lesion detection confidence threshold is reduced to increase the number of candidate lesions. When the classification accuracy is higher than the preset accuracy threshold, the lesion detection confidence threshold is increased to reduce the number of candidate lesions.

10. The system according to claim 1, characterized in that, The dynamic ultrasound sequence preprocessing module further includes a region of interest extraction unit, which is used to extract regions of interest containing breast tissue or thyroid tissue from each frame of the ultrasound video stream, and output the regions of interest to the spatiotemporal dual-stream feature extraction module after histogram equalization.

Images