Tumor real-time tracking and benign and malignant detection method based on breast ultrasonic video

Abstract

The invention provides a tumor real-time tracking and benign and malignant detection method based on a mammary gland ultrasonic video, and the method comprises the steps: collecting the mammary gland ultrasonic video in real time, and inputting the mammary gland ultrasonic video into an UltraNet basic detector to generate a detection result of each frame of ultrasonic image; calculating a similarity probability matrix Si, j to reflect the similarity between the ith target and the jth target; performing cascade matching on the target based on the similarity probability matrix Si, j to form a target track pipeline Ti; and carrying out confidence coefficient updating and bounding box position updating on the detection result in each target track pipeline Ti, and finally outputting a breast lesion detection result of each frame of ultrasonic image. The tumor real-time tracking and benign and malignant detection method based on the breast ultrasonic video has the advantages of high calculation efficiency, high precision and strong robustness, can assist a doctor in accurately positioning a focus and performing diagnosis, and is beneficial to early discovery of breast diseases.

Description

Technical Field

[0001] The present invention relates to a method for real-time tumor tracking and benign and malignant detection based on breast ultrasound video, belonging to the technical field of breast ultrasound video detection. Background Art

[0002] Breast cancer is a malignant tumor with a very poor prognosis, seriously endangering human health and becoming a major public health issue worldwide. However, breast ultrasound examinations rely heavily on real-time observation and diagnosis by physicians, requiring a high level of clinical expertise. Inexperienced physicians are prone to misdiagnosis or missed diagnoses. Furthermore, inherent ultrasound imaging issues such as low image contrast, unclear lesion features, and speckle noise can easily cause visual fatigue during long, intense work periods, increasing the risk of misdiagnosis or missed diagnoses.

[0003] With the continuous development of deep learning technology, neural networks have been widely used in many fields. General dynamic target detection models are increasingly being used for lesion detection in medical ultrasound videos. They are generally divided into two categories: target box aggregation-based methods and feature aggregation-based methods. Currently, feature aggregation-based video detection methods rely on dense prediction and cross-frame feature fusion to improve detection accuracy, but the additional aggregation module increases computational complexity, resulting in high inference overhead and difficulty meeting the real-time detection requirements of edge devices. Target box aggregation-based methods rely on the output of the basic detector for cross-frame target association, and the feature extraction capability of the detector directly affects the final aggregation effect. In addition, breast tumor lesions in ultrasound videos vary greatly in size, morphology, and position. Even in continuous video frames, this variation makes it difficult to guarantee the accuracy of lesion classification and positioning by directly using general video target detection models. Summary of the Invention

[0004] The purpose of the present invention is to provide a real-time tumor tracking and benign and malignant detection method based on breast ultrasound video to solve the problems of low computational efficiency and detection accuracy and high false detection rate in the prior art.

[0005] The technical solution of the present invention is:

[0006] A method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video, comprising the following steps:

[0007] S1. Real-time acquisition of breast ultrasound video and input into the UltraNet basic detector to generate detection results for each frame of ultrasound image Among them, I τ represents the τth frame ultrasound image, D i,τ Represents the i-th lesion candidate box detected by the basic detector on the τ-th frame;

[0008] S2, perform similarity judgment on the target detection results in two adjacent frames and calculate the similarity probability matrix S i,j , reflecting the similarity between the i-th target and the j-th target;

[0009] S3, based on similarity probability matrix S i,j Cascade matching is performed on the target to form the target trajectory pipeline T i ;

[0010] S4. For each target trajectory pipeline T i The confidence level and bounding box position of the detection results in the image are updated, and the breast lesion detection results of each frame of ultrasound image are finally output. Among them, the final detection result N of the optimized lesion in the τth frame is i,τ Including classification information cl t , location information pos t and confidence scores

[0011] Furthermore, in step S2, similarity judgment is performed on the target detection results in two adjacent frames, specifically,

[0012] S21, calculate the position information f of each target between two adjacent frames loc , geometric information f geo and semantic information f sem as follows:

[0013] f loc ={IoU,d centers}

[0014] f geo ={ratio w ,ratio h}

[0015]

[0016] Among them, IoU is the i-th target in the t-th frame and the jth target in frame t+1 The intersection-over-union ratio of the bounding boxes of two detected objects, d centers yes and The Euclidean distance between the centers of the bounding boxes of two detected objects, ratio w and ratio h yes and The aspect ratio of the bounding boxes of the two detected objects, f sem yes and Category confidence vectors of two detected targets and The dot product of C is the number of breast tumor categories;

[0017] S22. Introduce the logistic regression model to optimize the detection results and obtain the similarity score LS between any two frames: the i-th detection target in frame t and the j-th detection target in frame t+1:

[0018]

[0019] Among them, Logic represents the logistic regression model;

[0020] S23, the similarity scores LS of all targets between any two frames constitute the similarity probability matrix S i,j .

[0021] Furthermore, step S3 is specifically:

[0022] S31. Create target trajectory pipeline T i ;

[0023] S32, select similarity probability matrix S i,j The detection target pair with the highest similarity score LS is added to the target trajectory pipeline T i , and from the similarity probability matrix S i,j Remove the corresponding rows and columns so that each detected target is matched to only one target trajectory pipeline T i ; If the target detection result in the current frame does not belong to any pipeline, a new target trajectory pipeline is created for tracking management;

[0024] S33. Continuously iterate step S32 until the frame with fewer targets between any two frames is matched first, and there are no remaining targets in the other frame to be matched.

[0025] Furthermore, in step S4, for each target trajectory pipeline T i The confidence and bounding box position of the detection results in are updated, specifically,

[0026] S41, target trajectory pipeline T i The category confidence vector c of all target detection results in t ∈{T i} Take the average value and assign it to the pipeline T i All detection results in the pipeline are obtained to obtain the confidence vector after pipeline update; the maximum value of the updated confidence vector is taken as the confidence score after target trajectory pipeline update

[0027]

[0028] Among them, Mean is the average function, Max is the maximum function;

[0029] S42. Update the confidence score of the target trajectory pipeline The corresponding category index cl t Assigned to the target trajectory pipeline T i All target detection results within;

[0030] S43, pipe the target trajectory that changes with time to T i The coordinates of the detected target in are regarded as a time series with noise, and a one-dimensional Gaussian filter is used to smooth each time series. The smoothed sequence is finally used as the target trajectory pipeline T i The coordinate set of the internal target.

[0031] Furthermore, in step S1, the UltraNet basic detector includes a backbone network Backbone, a neck network Neck and a head network Head. The backbone network Backbone includes a convolution layer Conv, three feature extraction modules arranged in sequence, a lightweight convolution module Block1 and a fast spatial pyramid pooling layer SPPF. Each feature extraction module includes a lightweight convolution module Block and an adaptive wavelet downsampling layer AWFD. The lightweight convolution module Block of the second feature extraction module outputs the first image feature to the neck network Neck, and the lightweight convolution module Block of the third feature extraction module outputs the second image feature to the neck network Neck. The output of the adaptive wavelet downsampling layer AWFD of the third feature extraction module passes through the lightweight convolution module Block1 and the fast spatial pyramid pooling layer SPPF and then outputs the third image feature to the neck network Neck; the neck network Neck outputs three fused features to the head network Head to obtain the detection result.

[0032] Furthermore, in the feature extraction module,

[0033] Lightweight convolution module Block: The first c of the input feature map p The output is a 3×3 partial convolution layer PConv for each channel, and then passes through the first 1×1 point-by-point convolution layer, the batch normalization layer BN, the ReLU activation layer and the second 1×1 point-by-point convolution layer to output the feature X0. The feature X0 is concatenated with the input feature map to output the image feature X l Give the adaptive wavelet downsampling layer AWFD;

[0034] Adaptive wavelet downsampling layer AWFD: the image feature X l Split into features by channel and features feature The wavelet transform is used to decompose the high-frequency component and the low-frequency component. The high-frequency component is output after the residual convolution module to enhance the high-frequency features. The low-frequency components are enhanced using the low-frequency self-attention mechanism. feature Output low-frequency features after passing through the adaptive convolution module The enhanced low-frequency features With low frequency characteristics Perform channel splicing and use convolution with a stride of 1 to increase the channel dimension to obtain low-frequency features The low-frequency features With the enhanced high-frequency features Add pixel by pixel to get the final enhanced feature X l+1 .

[0035] Furthermore, in the neck network Neck, the second and third image features are input into the weighted feature fusion module WFF1, which then outputs the features to the lightweight convolution module Block2. The output of the lightweight convolution module Block2 is then fed together with the first image feature into the weighted feature fusion module WFF2, and then output. The output then passes through the lightweight convolution module Block3 and the convolution operation Conv1, and is then fed together with the output of the lightweight convolution module Block2 into the cascade layer C1 for output. The output then passes through the lightweight convolution module Block4 and the convolution operation Conv2, and is then fed together with the third image feature into the cascade layer C2, and then is output through the lightweight convolution module Block5. In the neck network Neck, the fused feature 1 output by the lightweight convolution module Block3, the fused feature 2 output by the lightweight convolution module Block4, and the fused feature 3 output by the lightweight convolution module Block5 are all fed into the head network Head.

[0036] Furthermore, in the weighted feature fusion module WFF1 and the weighted feature fusion module WFF2, the dynamic low-pass filter DLPF and the dynamic high-pass filter DHPF are introduced to perform weighted fusion on the deep features and shallow features respectively. Specifically,

[0037] (1) Processing deep features: Perform convolution dimensionality reduction operation to obtain a new feature map A K×K low-frequency weighted convolution kernel group is generated using a dynamic low-pass filter (DLPF). In turn Perform weighted convolution on each pixel in Each pixel in the image corresponds to a set of low-frequency weighted convolution kernels. There are H×W groups in total, and four weighted enhanced features are obtained Where i = 0, 1, ..., H-1, j = 0, 1, ..., W-1; Rearrange to form a low-frequency enhancement feature with two times upsampling Among them, PixelShuffle means pixel reshuffle;

[0038] (2) Processing shallow features: Perform convolution dimensionality reduction operation to obtain a new feature map A high-frequency weighted convolution kernel of K×K size is generated using a dynamic high-pass filter (DHPF). In turn Perform weighted convolution operation on each pixel (i, j) to obtain the weight matrix And the weight matrix endowment Get the final high-frequency enhancement feature in, represents the multiplication of corresponding elements in the matrix, where i = 0, 1, ..., H-1, j = 0, 1, ..., W-1;

[0039] (3) Feature fusion: The final high-frequency enhanced features generated and low frequency enhancement features Add pixel by pixel to get the final fusion feature Y l .

[0040] Furthermore, a K×K low-frequency weighted convolution kernel group is generated by a dynamic low-pass filter DLPF. Specifically,

[0041] (11) New feature map for input Perform convolution operation to obtain filter weights Use pixel inverse shuffling to Remodeling, coming soon The height and width are reduced by half, and the channel is expanded by 4 times. The expression is as follows: Among them, the filter weight

[0042] (12) For each set of filter weights Perform Softmax operation to obtain the normalized low-pass filter weights The expression is as follows:

[0043]

[0044] Where Ω refers to the set of (i+p,j+q) spatial positions around pixel (i,j), p,q∈[-1,0,1];

[0045] (13) Take pixel (i, j) as index and set the weights of four low-pass filters at the same position Take out in sequence and get And reshape it to get 4 low-frequency weighted kernel groups of size K×K Among them, K represents the low-frequency weighted kernel size, and the expression is as follows:

[0046]

[0047] Among them, i=0,1,...,H-1,j=0,1,...,W-1, each pixel corresponds to a set of low-frequency weighted convolution kernels There are H×W groups in total.

[0048] Furthermore, a high-frequency weighted convolution kernel of K×K size is generated by a dynamic high-pass filter DHPF. Specifically,

[0049] (21) New feature map for input Perform convolution operation to obtain low-pass filter weights

[0050] (22) Low-pass filter weights Perform Softmax operation to obtain the normalized low-pass filter weights The expression is as follows:

[0051]

[0052] Where Ω refers to the set of (i+p,j+q) spatial positions around pixel (i,j), p,q∈[-1,0,1];

[0053] (23) Take pixel (i, j) as index and K of all pixel positions in 2 The filter weights of the channel are taken out and obtained And Reshape to obtain a low-frequency weighted kernel of size K×K Among them, i=0,1,...,2H-1, j=0,1,...,2W-1, a total of 2H×2W low-frequency weighted kernels

[0054] (24) To ensure the high-frequency characteristics of the weighted kernel, the low-frequency weighted kernel is subtracted from the identity kernel function E. Get the final high-frequency weighted kernel in,

[0055] The present invention provides the following benefits: This method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video offers the advantages of high computational efficiency, high precision, and strong robustness. It can assist physicians in accurately locating lesions and conducting diagnosis, contributing to the early detection of breast diseases and thus improving prognosis. Experimental verification demonstrates that this method effectively improves detection accuracy while successfully reducing computational complexity and the number of model parameters, demonstrating significant advantages in breast ultrasound video tumor detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 1 is a schematic diagram illustrating a method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to an embodiment of the present invention;

[0057] Figure 2 is a network structure diagram of the UltraNet basic detector in the embodiment;

[0058] Figure 3 Schematic diagram illustrating the lightweight convolution module Block in the embodiment;

[0059] Figure 4 1 is a schematic diagram illustrating an adaptive wavelet downsampling layer AWFD in an embodiment;

[0060] Figure 5 is a schematic diagram illustrating the low-frequency attention mechanism in the embodiment;

[0061] Figure 6 2 is a schematic diagram illustrating an adaptive convolution module AdaConv in an embodiment;

[0062] Figure 7 1 is a schematic diagram illustrating a weighted feature fusion module WFF1 in an embodiment;

[0063] Figure 8 is a schematic diagram illustrating a dynamic low-pass filter DLPF in an embodiment;

[0064] Figure 9 is a schematic diagram illustrating a dynamic high-pass filter DHPF in an embodiment;

[0065] Figure 10 2 is a schematic diagram illustrating similarity judgment of target detection results in two adjacent frames in an embodiment;

[0066] Figure 11 are example images of benign and malignant images of the public dataset CVA-BUS used in the embodiments;

[0067] Figure 12 3 is a schematic diagram of the visualization detection results of 6 consecutive frames of images of malignant and benign lesions in the breast ultrasound video in the embodiment. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0069] The embodiment provides a method for real-time tumor tracking and benign and malignant detection based on breast ultrasound video, such as Figure 1 , including the following steps:

[0070] S1. Real-time acquisition of breast ultrasound video and input into the UltraNet basic detector to generate detection results for each frame of ultrasound image Among them, I τ represents the τth frame ultrasound image, D i,τ Represents the i-th lesion candidate box detected by the basic detector on the τ-th frame.

[0071] In step S1, Figure 2 The UltraNet basic detector includes a backbone network Backbone, a neck network Neck and a head network Head. The backbone network Backbone includes a convolution layer Conv, three feature extraction modules arranged in sequence, a lightweight convolution module Block1 and a fast spatial pyramid pooling layer SPPF. Each feature extraction module includes a lightweight convolution module Block and an adaptive wavelet downsampling layer AWFD. The lightweight convolution module Block of the second feature extraction module outputs the first image feature to the neck network Neck, and the lightweight convolution module Block of the third feature extraction module outputs the second image feature to the neck network Neck. The output of the adaptive wavelet downsampling layer AWFD of the third feature extraction module passes through the lightweight convolution module Block1 and the fast spatial pyramid pooling layer SPPF and then outputs the third image feature to the neck network Neck; the neck network Neck outputs three fused features to the head network Head to obtain the detection result.

[0072] like Figure 3 , lightweight convolution module Block: the front c of the input feature map p The output is a 3×3 partial convolution layer PConv for each channel, and then passes through the first 1×1 point-by-point convolution layer, the batch normalization layer BN, the ReLU activation layer and the second 1×1 point-by-point convolution layer to output the feature X0. The feature X0 is concatenated with the input feature map to output the image feature X l The lightweight convolution module Block effectively maintains the feature diversity and greatly reduces the computational complexity of the network. The feature maps between different channels have great similarity. Using conventional convolution to operate on all channels contains many redundant operations. The partial convolution PConv in the lightweight convolution module Block only operates on the first c of the input feature map. pThe convolution operation is performed on the channels, and the remaining channels are directly copied and concatenated with the convolution output, which can reduce redundant operations and greatly reduce the number of floating-point operations per second (FLOPS) of the convolution operation.

[0073] In the backbone network, downsampling is often used to reduce the size of the feature map. Traditional downsampling methods usually use convolution or pooling with a stride of 2, which is similar to interpolation. Although the size of the feature map can be reduced, some useful features are inevitably lost. Due to the inherent problems of low contrast and speckle noise interference in ultrasound images, traditional downsampling is difficult to meet the requirements of this specific scenario and is prone to losing the detailed features of the lesion edge. To address this problem, some studies have attempted to use low-pass filters to smooth the signal and reduce the interference of high-frequency noise. However, although low-pass filters can suppress noise, they also cause useful high-frequency texture feature responses to be weakened. To address this challenge, the present invention proposes an adaptive wavelet downsampling layer AWFD, which uses wavelet transform to decompose the image into high-frequency components and low-frequency components for enhancement processing. While reducing speckle noise interference, it better preserves the edge details and texture features of the image, reduces information loss during the downsampling process, and effectively improves the accuracy of lesion feature extraction in breast ultrasound images.

[0074] like Figure 4 ,The specific implementation steps of the adaptive wavelet downsampling layer AWFD are as follows:

[0075] ①First, image features Perform an average split on one channel to obtain two features with the same number of channels and features in, and will Input discrete wavelet transform WT, Input adaptive convolution module AdaConv. Wavelet transform input feature map Decomposed into four sub-bands, namely low-frequency components and high-frequency components in the horizontal, vertical, and diagonal directions in, Due to the low-frequency components It mainly carries the key contour information in the image and filters out most of the high-frequency noise interference. Therefore, the present invention designs a low-frequency self-attention mechanism LFSA to focus on the low-frequency components. To enhance: in, Represents the enhanced low-frequency features, Represents the low-frequency self-attention mechanism LFSA; and uses the residual convolution module (Residual Block) to Processing: in, Represents the enhanced high-frequency features; Represents the residual convolution module. It effectively improves the accuracy of breast ultrasound image feature expression.

[0076] ② Secondly, the enhanced low-frequency features With low frequency characteristics Perform channel splicing and use convolution with a stride of 1 to increase the number of channels to obtain low-frequency features

[0077]

[0078] ③Finally, the low-frequency features With the enhanced high-frequency features Add pixel by pixel to get the final enhanced feature

[0079]

[0080] As can be seen, the adaptive wavelet downsampling layer (AWFD) proposed in this paper includes two key modules: the low-frequency self-attention mechanism (LFSA) and the adaptive convolution module (AdaConv). It enhances the low-frequency and high-frequency components of the discrete wavelet transform (DWT), integrating the global contour features of the low-frequency components with the local edge detail features of the high-frequency components, effectively improving the accuracy of breast ultrasound image feature representation. The following describes the low-frequency self-attention mechanism (LFSA) and the adaptive convolution module (AdaConv).

[0081] To address the problem that the computational complexity of Transformer increases exponentially with the image size, resulting in low efficiency when processing large-size images, this paper proposes a low-frequency self-attention mechanism LFSA, which significantly reduces the computational complexity by dividing the image into blocks and calculating self-attention within a local window, while maintaining the ability to accurately model low-frequency features. Figure 5 , the implementation steps are as follows:

[0082] 1) Input low frequency components Extract a set of window features:

[0083] Split(X)={X1,X2,…,X n}

[0084] in, It is a set of window feature blocks, and N is the size of the window feature.

[0085] 2) For each window feature block X in turn i Process and generate their own queries key and value

[0086]

[0087] in, Indicates that the window feature block is reshaped. represents a 3×3 depthwise convolution operation, Represents a 1×1 point convolution operation.

[0088] 3) All Query of low-frequency components of a window Combined into the final query And so on, we get the final key Final Value

[0089] 4) On this basis, each window feature block X i The self-attention map A can be obtained as follows:

[0090]

[0091] Among them, α represents a learnable factor of the network, and introduces a method of implicitly encoding global features across channels, avoiding the O(H 2 W 2 ) problem. Specifically, the image is partitioned and self-attention is calculated within a local window, replacing the traditional HW×HW attention map with a more efficient C×C regional attention map. This windowed attention mechanism not only significantly reduces computational overhead but also enhances the low-frequency feature representation of the lesion and the correlation between channels through global cross-channel encoding.

[0092] 5) Add the self-attention map A and the low-frequency component X pixel by pixel to obtain the enhanced low-frequency features

[0093] The adaptive convolution module AdaConv can dynamically perceive the content of different spatial positions and channel positions in the feature map, combine local information with global information, and adaptively filter and enhance different content (foreground and background). The expression of the adaptive convolution module AdaConv is as follows:

[0094]

[0095] Among them, X c,i+p,j+q Represents the features of the input The local window feature, Y c,i,j For X c,i+p,j+qThe corresponding output features, where the subscript (c, i, j) represents the (i, j) pixel in the c-th channel feature map, and S points to the set of spatial positions around the pixel (i, j). The output feature value Y of all (c, i, j) positions is c,i,j The low-frequency features Y that constitute the final output. is the filter weight of the channel and spatial position corresponding to (c,i,j), and the expression is as follows:

[0096]

[0097] Here, V c,i,j represents the local information generated by adaptive convolution on (c,i,j), T c Represents the global context information generated by global pooling. The specific network structure of the adaptive convolution module AdaConv is as follows Figure 6 As shown, V c,i,j and T c Detailed calculation process.

[0098] Describe V using mathematical expressions c,i,j and T c The calculation process is as follows:

[0099] V c,i,j =φ(X c,i+p,j+q )

[0100] Here, φ represents the weight generation function, including the local window feature X c,i+p,j+q Perform plastic transformation Reshape and linear mapping FC.

[0101] T c =φ′(GP(X c,i+p,j+q ))

[0102] Here, GP represents the global pooling operation and φ′ represents the multi-layer linear mapping.

[0103] It can be seen that the adaptive convolution module integrates the local information V c,i,j and global context information T c , dynamically weighting different positions and channels, thereby adaptively maintaining foreground signals and suppressing background noise interference.

[0104] Table 1 lists the detailed parameters of the AWFD-FasterNet backbone network. Conv_k_c_s stands for kernel size k, number of output channels c, and stride s. PConv_k_c_s_r stands for partial convolution with kernel size k, number of output channels c, stride s, and the ratio of channels used in conventional convolution r. h represents the length of the output feature map, w represents the width of the output feature map, BN represents the normalization operation, Acti represents the activation function, FLOPs (G) represents the number of floating-point operations per second (Giga), and Params (M) represents the number of parameters (Mega).

[0105] Table 1 AWFD-FastNet backbone network detailed parameters

[0106]

[0107]

[0108] In the neck network (Neck), the second and third image features are input into the weighted feature fusion module WFF1, which then outputs the features to the lightweight convolution module Block2. The output of lightweight convolution module Block2 is then fed into the weighted feature fusion module WFF2 along with the first image feature. The output then passes through lightweight convolution module Block3 and convolution operation Conv1, and is then fed into cascade layer C1 along with the output of lightweight convolution module Block2. The output then passes through lightweight convolution module Block4 and convolution operation Conv2, and is then fed into cascade layer C2 along with the third image feature, before being output through lightweight convolution module Block5. In the neck network (Neck), fused feature 1 (output from lightweight convolution module Block3), fused feature 2 (output from lightweight convolution module Block4), and fused feature 3 (output from lightweight convolution module Block5) are all fed into the head network (Head).

[0109] In both weighted feature fusion module 1 WFF1 and weighted feature fusion module 2 WFF2, the deep features and shallow features are weightedly fused by introducing dynamic low-pass filter DLPF and dynamic high-pass filter DHPF respectively. The shallow features input to weighted feature fusion module WFF1 are the second image features output by backbone network Backbone, and the deep features are the third image features; the shallow features input to weighted feature fusion module WFF2 are the first image features output by backbone network Backbone, and the deep features are the features output by lightweight convolution module block2. Take weighted feature fusion module 1 WFF1 as an example to illustrate. Figure 7 , specifically,

[0110] (1) Processing deep features: Perform convolution dimensionality reduction operation to obtain a new feature map A K×K low-frequency weighted convolution kernel group is generated using a dynamic low-pass filter (DLPF). In turn Perform weighted convolution on each pixel in Each pixel in the image corresponds to a set of low-frequency weighted convolution kernels. There are H×W groups in total, and four weighted enhanced features are obtained Where i = 0, 1, ..., H-1, j = 0, 1, ..., W-1; Rearrange to form a low-frequency enhancement feature with two times upsampling

[0111] in,

[0112] PixelShuffle means pixel shuffle.

[0113] Among them, the K×K size low-frequency weighted convolution kernel group generated by the dynamic low-pass filter DLPF is like Figure 8 , specifically,

[0114] (11) New feature map for input Perform convolution operation to obtain filter weights Use PixelUnshuffle to Remodeling, coming soon The height and width are reduced by half, and the channel is expanded by 4 times. The expression is as follows: Among them, the filter weight

[0115] (12) For each set of filter weights Perform Softmax operation to obtain the normalized low-pass filter weights The expression is as follows:

[0116]

[0117] Where Ω refers to the set of (i+p,j+q) spatial positions around pixel (i,j), p,q∈[-1,0,1];

[0118] (13) Take pixel (i, j) as index and set the weights of four low-pass filters at the same position Take out in sequence and get And reshape it to get 4 low-frequency weighted kernel groups of size K×K Among them, K represents the low-frequency weighted kernel size, and the expression is as follows:

[0119]

[0120] Among them, i=0,1,...,H-1,j=0,1,...,W-1, each pixel corresponds to a set of low-frequency weighted convolution kernels There are H×W groups in total.

[0121] (2) Processing shallow features: Perform convolution dimensionality reduction operation to obtain a new feature map A high-frequency weighted convolution kernel of K×K size is generated using a dynamic high-pass filter (DHPF). In turn Perform weighted convolution operation on each pixel (i, j) to obtain the weight matrix And the weight matrix endowment Get the final high-frequency enhancement feature in, Represents the multiplication of corresponding elements in the matrix, where i = 0, 1, ..., H-1 and j = 0, 1, ..., W-1.

[0122] Considering that the deep features generated by the backbone network are mostly abstract semantic information and global information, while the shallow features are mostly specific edge texture information and local information, the standard fusion method generally adds the shallow features directly to the upsampled deep features pixel by pixel to achieve the effect of mutual complementation of features. However, due to the performance limitations of the backbone network, the description of texture detail information in the acquired shallow features is sometimes not significant, resulting in poor feature fusion effect. Therefore, to address this problem, the present invention introduces a dynamic high-pass filter DHPF, generates a high-frequency weighted kernel pixel by pixel, performs high-frequency weighting on the shallow features, effectively enhances the edge texture information in the shallow features, and then fuses them with the upsampled deep features to form a more delicate and accurate feature representation.

[0123] A high-frequency weighted convolution kernel of K×K size is generated using a dynamic high-pass filter (DHPF). like Figure 9 , specifically,

[0124] (21) New feature map for input Perform convolution operation to obtain low-pass filter weights

[0125] (22) Low-pass filter weights Perform Softmax operation to obtain the normalized low-pass filter weights The expression is as follows:

[0126]

[0127] Where Ω refers to the set of (i+p,j+q) spatial positions around pixel (i,j), p,q∈[-1,0,1];

[0128] (23) Take pixel (i, j) as index and K of all pixel positions in 2 The filter weights of the channel are taken out and obtained And Reshape to obtain a low-frequency weighted kernel of size K×K Among them, i=0,1,...,2H-1, j=0,1,...,2W-1, a total of 2H×2W low-frequency weighted kernels

[0129] (24) To ensure the high-frequency characteristics of the weighted kernel, the low-frequency weighted kernel is subtracted from the identity kernel function E. Get the final high-frequency weighted kernel in,

[0130] (3) Feature fusion: The final high-frequency enhanced features generated and low frequency enhancement features Add pixel by pixel to get the final fusion feature

[0131] Usually the current standard feature fusion method is to use deep features Figure X l+1 After upsampling, compared with the shallow features Figure X l This is achieved by adding pixel by pixel, which can be summarized as follows: in, Represent the lth layer features and l+1th layer features of the backbone network respectively, Represents the traditional upsampling method (generally using nearest neighbor interpolation or bilinear interpolation). Although standard feature fusion is widely used, this method is prone to data distortion problems. The reason is that a simple interpolation upsampling operation on deep features may cause a single outlier in the deep features to diffuse in the upsampled large-size feature map, resulting in a mutation in the semantic features of adjacent pixels, thereby affecting the continuity and accuracy of the feature representation. In addition, the standard fusion method is to directly add the shallow features to the upsampled deep features pixel by pixel without optimizing the shallow features. To this end, the present invention proposes a weighted feature fusion module WFF, which introduces a dynamic low-pass filter DLPF and a dynamic high-pass filter DHPF to perform weighted fusion on the deep features and shallow features, respectively, to further improve the feature representation of ultrasonic breast tumors.

[0132] The head network consists of three modules, each of which has two branches: one for outputting the target bounding box coordinates and one for outputting the category prediction. Both branches undergo convolution (Conv) and two-dimensional convolution (Conv2d) to generate the bounding box regression loss (Bbox-loss) and classification loss (Cls-loss), respectively. After training and optimization, the final bounding box coordinates and category predictions are obtained.

[0133] In step S1, the UltraNet basic detector improves the backbone network and neck network of the original YOLOv8. The backbone network of the original YOLOv8 uses the CSPDarknet-53 network to extract the features of the input image, but its feature extraction accuracy still has room for improvement. To this end, the present invention proposes an efficient and robust backbone network AWFD-FasterNet. First, taking advantage of the high efficiency and accuracy of FasterNet in feature extraction, the original CSPDarknet-53 network is replaced by FasterNet, and an innovative adaptive wavelet downsampling layer AWFD is designed to replace the traditional convolutional downsampling layer in FasterNet, effectively alleviating the impact of speckle noise on detection accuracy and further improving the feature expression of the network. In the neck network part, the Block module replaces the original C2f module, effectively reducing the number of network parameters. At the same time, by analyzing the characteristics of shallow and deep features, the original upsample-concatenation module was replaced with a weighted feature fusion module (WFF). This performs weighted fusion of high-frequency and low-frequency shallow and deep features, respectively, further enhancing the network's feature representation capabilities and improving the robustness and accuracy of tumor detection in single-frame images. In the Neck network, the weighted feature fusion modules WFF1 and WFF2 have the same structure. Block1, Block2, Block3, Block4, and Block5 also have the same structure as the lightweight convolutional module Block in the backbone network.

[0134] S2, perform similarity judgment on the target detection results in two adjacent frames and calculate the similarity probability matrix S i,j , reflecting the similarity between the i-th target and the j-th target.

[0135] In step S2, the similarity of the target detection results in two adjacent frames is judged, such as Figure 10 , specifically,

[0136] S21, calculate the position information f of each target between two adjacent frames loc , geometric information f geo and semantic information f sem as follows:

[0137] f loc ={IoU,d centers}

[0138] f geo ={ratio w ,ratio h}

[0139]

[0140] Among them, IoU is the i-th target in the t-th frame and the jth target in frame t+1 The intersection-over-union ratio of the bounding boxes of two detected objects, d centers yes and The Euclidean distance between the centers of the bounding boxes of two detected objects, ratio w and ratio h yes and The aspect ratio of the bounding boxes of the two detected objects, f sem yes and Category confidence vectors of two detected targets and The dot product of C is the number of breast tumor categories;

[0141] S22. Introduce the logistic regression model to optimize the detection results and obtain the similarity score LS between any two frames: the i-th detection target in frame t and the j-th detection target in frame t+1:

[0142]

[0143] Among them, Logic represents the logistic regression model;

[0144] S23, the similarity scores LS of all targets between any two frames constitute the similarity probability matrix S i,j .

[0145] In step S2, in order to link the target pipeline between two consecutive frames, the present invention introduces a logistic regression model to optimize the target matching relationship and accurately determine whether each target between two adjacent frames belongs to the same object instance.

[0146] S3, based on similarity probability matrix S i,j Cascade matching is performed on the target to form the target trajectory pipeline T i .

[0147] S31. Create target trajectory pipeline T i ;

[0148] S32, select similarity probability matrix S i,j The detection target pair with the highest similarity score LS is added to the target trajectory pipeline T i , and from the similarity probability matrix S i,j Remove the corresponding rows and columns so that each detected target is matched to only one target trajectory pipeline T i ; If the target detection result in the current frame does not belong to any pipeline, a new target trajectory pipeline is created for tracking management;

[0149] S33. Continuously iterate step S32 until the frame with fewer targets between any two frames is matched first, and there are no remaining targets in the other frame to be matched.

[0150] In step S3, in order to make the dynamic target detection trajectory as stable and continuous as possible, the present invention links the similarities of each pair of targets between two adjacent frames into the trajectory to construct a target trajectory pipeline (tubelets). The target trajectory pipeline construction usually starts with the target detection of the first frame to create a pipeline, and continues to expand in subsequent frames. A new pipeline can be initialized at any frame, but will not be repeated with an existing pipeline. Unlike traditional methods, this method only considers the current frame and the previous frame for matching, and does not rely on future frame information. Since the target detector may have false detections, some instances may be mistakenly linked to non-existent pipelines. In order to reduce the impact of false detections, the present invention sets a threshold for filtering low-confidence links. The threshold is used to filter out low-quality target detection results before the pipeline is built, ensuring that the pipeline construction process can not only ensure the integrity of the target trajectory, but also effectively reduce the interference caused by false matches.

[0151] S4. For each target trajectory pipeline T i The confidence level and bounding box position of the detection results in the image are updated, and the breast lesion detection results of each frame of ultrasound image are finally output. Among them, the final detection result N of the optimized lesion in the τth frame is i,τ Including classification information cl t , location information pos t and confidence scores

[0152] In step S4, for each target trajectory pipeline T i The confidence and bounding box position of the detection results in are updated, specifically,

[0153] S41, target trajectory pipeline T i The category confidence vector c of all target detection results in t ∈{T i} Take the average value and assign it to the pipeline Ti All detection results in the pipeline are obtained to obtain the confidence vector after pipeline update; the maximum value of the updated confidence vector is taken as the confidence score after target trajectory pipeline update

[0154]

[0155] Among them, Mean is the average function, Max is the maximum function;

[0156] S42. Update the confidence score of the target trajectory pipeline The corresponding category index cl t Assigned to the target trajectory pipeline T i All target detection results within;

[0157] S43, pipe the target trajectory that changes with time to T i The coordinates of the detected target in are regarded as a time series with noise, and a one-dimensional Gaussian filter is used to smooth each time series. The smoothed sequence is finally used as the target trajectory pipeline T i The coordinate set of the internal target.

[0158] In step S4, for each trajectory pipeline T i The link information in the pipeline further optimizes the classification and positioning of the target. Typically, target detectors can generate high-precision bounding box coordinates on still images or low-motion frames, but their regression accuracy tends to drop significantly when dealing with defocused or fast-moving targets. The present invention regards the coordinates of the detected targets in the pipeline that changes over time as a noisy time series, and uses a one-dimensional Gaussian filter to smooth each time series. The final smoothed sequence is used as the coordinate set of the target in the pipeline, thereby further improving the stability and accuracy of target detection.

[0159] This real-time tumor tracking and benign and malignant tumor detection method based on breast ultrasound video boasts high computational efficiency, high precision, and strong robustness. It can assist doctors in accurately locating lesions and conducting diagnoses, contributing to the early detection of breast diseases and thus improving prognosis. Experimental validation has shown that this method effectively improves detection accuracy while successfully reducing computational complexity and the number of model parameters, demonstrating significant advantages in breast ultrasound tumor video detection.

[0160] This method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video uses the UltraNet basic detector, which is an improvement on the YOLOv8 detector. First, the backbone network Backbone is improved, with FasterNet replacing the original YOLOv8 backbone network CSPDarknet-53. An innovative adaptive wavelet downsampling layer AWFD is designed to replace the traditional convolutional downsampling layer in FasterNet, effectively alleviating the impact of speckle noise on the detector and optimizing the network's feature expression capabilities. At the same time, the neck network Neck is also improved, with the Block module replacing the original C2f module in YOLOv8 to reduce network parameters. The innovative weighted feature fusion module WFF is designed to replace the original upsample-concat module, further enhancing the network's feature expression capabilities and improving the robustness and accuracy of target detection in single-frame images.

[0161] The adaptive wavelet downsampling layer (AWFD) proposed in this paper uses discrete wavelet transforms to decompose data into low-frequency and high-frequency components. These components are then processed in parallel using a low-frequency self-attention mechanism (LFSA) and a residual convolution module, effectively enhancing the ability to express features in the lesion area. Furthermore, the adaptive convolution module (AdaConv) further smooths speckle noise interference, enabling the network to maintain lesion edge and texture details while reducing noise.

[0162] The weighted feature fusion module WFF proposed in the present invention further improves the feature representation of ultrasonic breast tumors by introducing a dynamic low-pass filter DLPF and a dynamic high-pass filter DHPF to perform weighted fusion on deep features and shallow features respectively. Among them, the dynamic low-pass filter DLPF generates four low-frequency weighted kernels by suppressing the high-frequency components of shallow features, thereby smoothing the deep features, and rearranges the four generated small-sized smoothed deep features to generate a large-sized feature map, which solves the problem of semantic mutation of adjacent pixels when the deep features are directly upsampled. At the same time, the dynamic high-pass filter DHPF performs high-frequency weighting on the shallow features to enhance the detailed information in the shallow features. It is fused with the upsampled deep features to form a more delicate and accurate feature representation. Through this mechanism, the model can more accurately capture the texture information of the lesion edge and improve the stability and accuracy of detection.

[0163] The present invention introduces a classification algorithm based on logistic regression in the post-processing stage to optimize the detection results. Specifically, for all possible detection targets in consecutive frames, feature vectors are constructed based on target position, geometry, semantic information, etc., and a logistic regression model is used to calculate the correlation score (similarity score) between targets, thereby predicting the optimal matching relationship. By establishing inter-frame links, the target trajectory is made as stable and continuous as possible. This method can effectively associate the detection results of the same target in different frames, enhancing the robustness to the dynamic changes of breast lesions.

[0164] The following experimental verification of this method of real-time tumor tracking and benign and malignant detection based on breast ultrasound video is carried out:

[0165] The public dataset CVA-BUS is used, which contains a total of 186 videos. Each frame of each video has been finely annotated, including 113 breast ultrasound malignant videos and 75 breast ultrasound benign videos. The dataset is divided into training, validation sets (154 videos) and test sets (32 videos). There are 21,423 frames of images in the training and validation sets, covering 170 lesions; the test set contains 3,849 frames of images, covering 32 lesions. The experiment not only focuses on the lesion detection task, but also further classifies and analyzes the detected lesions as benign or malignant. The example figure of the public dataset CVA-BUS is as follows Figure 11 As shown, (a)-(c) are three frames of images in a benign video, and (d)-(f) are three frames of images in a malignant video.

[0166] Experimental software and hardware platform: This paper uses YOLOv8 as the benchmark model and conducts experiments based on the Python 3.8 environment, PyTorch1.0 framework, and a 64-bit Ubuntu 22 operating system. The hardware platform is a server equipped with an Intel i9-10980XE CPU, 64G memory, and three NVIDIAGTX 3060 graphics cards.

[0167] Model training setup: During the training of the benchmark model, the public dataset CVA-BUS was first preprocessed, and all images were uniformly scaled to 640×640. To improve the robustness of the model, a dynamic data augmentation strategy was introduced during the data augmentation stage, including blurring, speckle noise, and elastic deformation. These enhancements can simulate the noise and deformation that may occur during ultrasound video acquisition, increase the randomness of the training data, and thus improve the model's adaptability to real-world scenarios. During training, the AdamW optimizer was used, with a batch size of 16 and an initial learning rate of 0.001, and training for 200 epochs. During training, the trend of the loss was monitored and the model results were saved after it stabilized to ensure optimal model performance.

[0168] To train the post-processing logistic regression model, a triplet-based dataset was used. Each triplet consists of an anchor (sample point), a positive (same object instance), and a negative (different object instance) bounding box information, all sourced from the CVA-BUS dataset. Notably, for the link scoring model, each triplet provides both a positive and a negative training sample. Specifically, a set of 50,000 triplet samples was constructed for training and 8,000 for validation. For each triplet sample, a target track was randomly sampled, and a sample point was selected from a random frame within the track as the anchor. Positive examples were selected from a frame within ±25 frames of the sample point. It should be noted that most videos in the CVA-BUS dataset have a frame rate between 20 and 30 frames per second. Negative examples were selected from different object tracks, either randomly from the same video clip as the sample point or from another video. Negative examples may contain objects that belong to the same category as the anchor but are different instances, thus better simulating the complex scenes in actual detection.

[0169] This experiment uses the average precision AP50, the false positive rate of malignant tumors FPR MT , false negative rate of malignant tumors (FNR) MT , parameter quantity Params, computational complexity Gflops and delay are used as evaluation indicators.

[0170] The average precision (AP50) is an indicator that measures the performance of a model by comprehensively considering the precision and recall of the model's prediction results. It is calculated by taking the area under the precision-recall curve (PR curve) when the intersection-over-union (IoU) threshold is 0.5. Its calculation formula is as follows:

[0171]

[0172] Among them, TP is the number of detected targets that are correct targets; FP is the number of targets that are incorrectly detected; FN is the number of false detections and missed detections.

[0173] False positive rate (FPR) of malignant tumors MT It is used to measure the purity of the model's correct prediction of positive samples. The smaller the value, the better the model performance. Specifically, it represents the proportion of all true negative samples that are incorrectly predicted as positive samples. The calculation formula is:

[0174]

[0175] FNR (missed detection rate) MT It is used to measure the purity of the model's correct prediction of negative samples. The smaller the value, the better the model performance. It represents the proportion of all true positive samples that are incorrectly predicted as negative samples. The calculation formula is:

[0176]

[0177] Among them, TN represents the number of correctly predicted negative samples, N represents the number of all true negative samples, and P represents the number of all true positive samples.

[0178] To verify the effectiveness of the embodiment method, on the basis of the basic model, the low-complexity backbone network FasterNet, AWFD, FeaFusion, and post-processing were added in sequence. The post-processing includes target similarity judgment, target pipeline construction, and target information update, namely steps S2-S4. The experimental results are shown in Table 2.

[0179] Table 2 Effects of adding modules in sequence on breast ultrasound lesion detection performance

[0180]

[0181] Table 2 shows that compared to the unmodified baseline model, the introduction of a low-complexity backbone network achieved significant improvements in accuracy and latency. AP50 increased by 5.9%, single-frame processing latency decreased by 1.2 ms, and parameter count and computational complexity were reduced by 2.11 M and 5.4 GFLOPs, respectively. Furthermore, the false detection rate and missed detection rate for malignant tumors decreased by 3.4% and 1.3%, respectively. Building on the low-complexity backbone network, the traditional downsampling layer was replaced with an adaptive wavelet downsampling layer, further improving model accuracy by 0.5% with fewer parameters and computational complexity. Despite a 3 ms increase in latency, the false detection rate and missed detection rate for malignant tumors were significantly improved, decreasing by 5.6% and 2.3%, respectively. Next, the addition of a feature fusion module increased AP50 by 4.1%, and the false detection rate and missed detection rate for malignant tumors were reduced by 5% and 2.9%, respectively. Despite the increase in latency, parameter count, and computational complexity, accuracy was further improved. Finally, after adding the low-complexity post-processing module, the model achieved an AP50 score of 82.3, a 13.9% improvement in detection accuracy compared to the baseline model. The false positive rate and missed detection rate for malignant tumors were reduced by 17.3% and 8.6%, respectively. Furthermore, computational complexity was reduced by 4.8 GFLOPs, and the number of model parameters was reduced by 2.02M. Overall, by gradually incorporating these improved modules, the proposed method successfully reduced computational complexity and model parameter count while improving detection accuracy, validating the effectiveness of each module.

[0182] To verify the importance of each module to the model, Table 3 shows the impact on detection performance when a specific module is missing while the other two modules are present. This experiment allows us to observe the specific impact of the missing module on overall performance.

[0183] Table 3 Effect of a single module on breast ultrasound lesion detection performance

[0184]

[0185] As shown in Table 3, when CSPDarkNet is used as the backbone network instead of the low-complexity AWFD-FasterNet, experimental results show a significant decline in detection performance. Specifically, AP50 decreased by 7%, the false detection rate for malignant tumors increased by 14.9%, and the missed detection rate increased by 7.6%. At the same time, the single-frame processing time, the number of model parameters, and the computational complexity also increased significantly. This demonstrates that the low-complexity backbone network proposed in this paper is crucial for improving model performance and is an indispensable component of model performance.

[0186] When the adaptive wavelet downsampling layer is replaced with a standard convolutional downsampling layer instead of the AWFD, experimental results show that detection performance declines. Although latency, parameter count, and computational complexity are reduced, AP50 decreases by 0.6%, the false positive rate for malignant tumors increases by 1.1%, and the missed detection rate increases by 0.9%. This demonstrates that the AWFD also plays a crucial role in improving model performance.

[0187] When using the standard nearest neighbor interpolation upsampling fusion method instead of feature fusion, experimental results show a significant decline in detection performance, specifically a 3.3% decrease in AP50, a 6.5% increase in the false detection rate for malignant tumors, and a 4.0% increase in the missed detection rate. While there is a slight advantage in latency, the overall performance decline demonstrates that frequency domain feature fusion is crucial for improving model performance.

[0188] When the post-processing module was omitted and only the improved base model with a low-complexity backbone network, adaptive wavelet downsampling, and feature fusion was used, experiments showed a 3.4% reduction in AP50, a 3.3% improvement in the false detection rate for malignant tumors, and a 2.1% increase in the missed detection rate. At the same time, single-frame processing time was reduced by 1.1ms. This result demonstrates that while adding the post-processing module slightly increases latency, the time penalty is negligible, while significantly improving detection accuracy.

[0189] This embodiment's real-time breast ultrasound video-based tumor tracking and benign and malignant tumor detection method was tested on the public CVA-BUS dataset and compared with current state-of-the-art video object detection algorithms. These included classic single-frame object detection algorithms like YOLOX, RTDETR, Faster R-CNN, YOLOv8, and RetinaNet, as well as video detection algorithms like FGFA, SELSA, and MEGA, and breast ultrasound video detection algorithms like CVA-Net, STNet, and UltraDet. Table 4 shows the accuracy and latency performance on the test set.

[0190] Table 4 Comparison of test results between the embodiment method and current advanced methods

[0191] Method Type AP50(%) Latency (ms) YOLOX Image 65.2 16.8 RTDETR Image 66.0 Faster R-CNN Image 49.2 20.3 YOLOv8 Image 68.4 7.4 RetinaNet Image 50.4 18.6 FGFA Video 49.7 29.6 SELSA Video 45.6 MEGA Video 57.2 24.9 CVA-Net Video 65.1 43.2 STNet Video 70.3 UltraDet Video 74.3 32.8 Ours Video 82.3 12.4

[0192] The comparison results in Table 4 show that the method proposed in the embodiment shows significant advantages in the breast ultrasound tumor video detection task, with an AP50 of 82.3% and a single-frame processing delay of only 12.4ms, both of which are better than other video target detection algorithms.

[0193] The embodiment selected 6 consecutive frames of malignant and benign lesions in breast ultrasound videos and performed lesion detection on them. To verify the effectiveness of the embodiment method, the embodiment method was compared with the current advanced video detection algorithms YOLOv8 and UltraDet and real data, and the video detection results were visualized. The experimental results are shown in the figure. Figure 12 As shown, the performance of the embodiment method in the breast ultrasound video lesion detection task is further demonstrated.

[0194] The detailed description of the embodiments is only a specific description of the feasible implementation methods of the present invention and is not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video, characterized by: The following steps are included: S1. Real-time acquisition of breast ultrasound video and input into the UltraNet basic detector to generate detection results for each frame of ultrasound image Among them, I τ represents the τth frame ultrasound image, D i,τ Represents the i-th lesion candidate box detected by the basic detector on the τ-th frame; S2, perform similarity judgment on the target detection results in two adjacent frames and calculate the similarity probability matrix S i,j , reflecting the similarity between the i-th target and the j-th target; S3, based on similarity probability matrix S i,j Cascade matching is performed on the target to form the target trajectory pipeline T i ; S4. For each target trajectory pipeline T i The confidence level and bounding box position of the detection results in the image are updated, and the breast lesion detection results of each frame of ultrasound image are finally output. Among them, the final detection result N of the optimized lesion in the τth frame is i,τ Including classification information cl t , location information pos t and confidence scores 2. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 1, characterized in that: In step S2, the similarity of the target detection results in two adjacent frames is judged, specifically, S21, calculate the position information f of each target between two adjacent frames loc , geometric information f geo and semantic information f sem as follows: f loc ={IoU,d centers } f geo ={reason w reason h } Among them, IoU is the i-th target in the t-th frame and the jth target in frame t+1 The intersection-over-union ratio of the bounding boxes of two detected objects, d centers yes and The Euclidean distance between the centers of the bounding boxes of two detected targets, ratio w and ratio h yes and The aspect ratio of the bounding boxes of the two detected objects, f sem yes and Category confidence vectors of two detected targets and The dot product of C is the number of breast tumor categories; S22. Introduce the logistic regression model to optimize the detection results and obtain the similarity score LS between any two frames: the i-th detection target in frame t and the j-th detection target in frame t+1: Among them, Logic represents the logistic regression model; S23, the similarity scores LS of all targets between any two frames constitute the similarity probability matrix S i,j .

3. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 1, characterized in that: Step S3 is specifically: S31. Create target trajectory pipeline T i ; S32, select similarity probability matrix S i,j The detection target pair with the highest similarity score LS is added to the target trajectory pipeline T i , and from the similarity probability matrix S i,j Remove the corresponding rows and columns so that each detected target is matched to only one target trajectory pipeline T i ; If the target detection result in the current frame does not belong to any pipeline, a new target trajectory pipeline is created for tracking management; S33. Continuously iterate step S32 until the frame with fewer targets between any two frames is matched first, and there are no remaining targets in the other frame to be matched.

4. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to any one of claims 1 to 3, wherein: In step S4, for each target trajectory pipeline T i The confidence and bounding box position of the detection results in are updated, specifically, S41, target trajectory pipeline T i The category confidence vector c of all target detection results in t ∈{T i } Take the average value and assign it to the pipeline T i All detection results in the pipeline are obtained to obtain the confidence vector after pipeline update; the maximum value of the updated confidence vector is taken as the confidence score after target trajectory pipeline update Among them, Mean is the average function, Max is the maximum function; S42. Update the confidence score of the target trajectory pipeline The corresponding category index cl t Assigned to the target trajectory pipeline T i All target detection results within; S43, pipe the target trajectory that changes with time to T i The coordinates of the detected target in are regarded as a time series with noise, and a one-dimensional Gaussian filter is used to smooth each time series. The smoothed sequence is finally used as the target trajectory pipeline T i The coordinate set of the internal target.

5. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 1, wherein: In step S1, the UltraNet basic detector includes a backbone network Backbone, a neck network Neck and a head network Head. The backbone network Backbone includes a convolution layer Conv, three feature extraction modules arranged in sequence, a lightweight convolution module Block1 and a fast spatial pyramid pooling layer SPPF. Each feature extraction module includes a lightweight convolution module Block and an adaptive wavelet downsampling layer AWFD. The lightweight convolution module Block of the second feature extraction module outputs the first image feature to the neck network Neck, and the lightweight convolution module Block of the third feature extraction module outputs the second image feature to the neck network Neck. The output of the adaptive wavelet downsampling layer AWFD of the third feature extraction module passes through the lightweight convolution module Block1 and the fast spatial pyramid pooling layer SPPF and then outputs the third image feature to the neck network Neck; the neck network Neck outputs three fused features to the head network Head to obtain the detection result.

6. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 1, characterized in that: In the feature extraction module, Lightweight convolution module Block: The first c of the input feature map p The output is a 3×3 partial convolution layer PConv for each channel, and then passes through the first 1×1 point-by-point convolution layer, the batch normalization layer BN, the ReLU activation layer and the second 1×1 point-by-point convolution layer to output the feature X0. The feature X0 is concatenated with the input feature map to output the image feature X l Give the adaptive wavelet downsampling layer AWFD; Adaptive wavelet downsampling layer AWFD: the image feature X l Split into features by channel and features feature The wavelet transform is used to decompose the high-frequency component and the low-frequency component. The high-frequency component is output after the residual convolution module to enhance the high-frequency features. The low-frequency components are enhanced using the low-frequency self-attention mechanism. feature Output low-frequency features after passing through the adaptive convolution module The enhanced low-frequency features With low frequency characteristics Perform channel splicing and use convolution with a stride of 1 to increase the channel dimension to obtain low-frequency features The low-frequency features With the enhanced high-frequency features Add pixel by pixel to get the final enhanced feature X l+1 .

7. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 6, characterized in that: In the neck network (Neck), the second and third image features are input into the weighted feature fusion module WFF1, which then outputs the features to the lightweight convolution module Block2. The output of lightweight convolution module Block2 is then fed into the weighted feature fusion module WFF2 along with the first image feature. The output then passes through lightweight convolution module Block3 and convolution operation Conv1, and is then fed into cascade layer C1 along with the output of lightweight convolution module Block2. The output then passes through lightweight convolution module Block4 and convolution operation Conv2, and is then fed into cascade layer C2 along with the third image feature, before being output through lightweight convolution module Block5. In the neck network (Neck), fused feature 1 (output from lightweight convolution module Block3), fused feature 2 (output from lightweight convolution module Block4), and fused feature 3 (output from lightweight convolution module Block5) are all fed into the head network (Head).

8. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 7, characterized in that: In the weighted feature fusion module WFF1 and the weighted feature fusion module WFF2, the dynamic low-pass filter DLPF and the dynamic high-pass filter DHPF are introduced to perform weighted fusion of deep features and shallow features respectively. Specifically, (1) Processing deep features: Perform convolution dimensionality reduction operation to obtain a new feature map A K×K low-frequency weighted convolution kernel group is generated using a dynamic low-pass filter (DLPF). In turn Perform weighted convolution on each pixel in Each pixel in the image corresponds to a set of low-frequency weighted convolution kernels. There are H×W groups in total, and four weighted enhanced features are obtained Where i = 0, 1, ..., H-1, j = 0, 1, ..., W-1; Rearrange to form a low-frequency enhancement feature with two times upsampling Among them, PixelShuffle means pixel reshuffle; (2) Processing shallow features: Perform convolution dimensionality reduction operation to obtain a new feature map A high-frequency weighted convolution kernel of K×K size is generated using a dynamic high-pass filter (DHPF). In turn Perform weighted convolution operation on each pixel (i, j) to obtain the weight matrix And the weight matrix endowment Get the final high-frequency enhancement feature in, represents the multiplication of corresponding elements in the matrix, where i = 0, 1, ..., H-1, j = 0, 1, ..., W-1; (3) Feature fusion: The final high-frequency enhanced features generated and low frequency enhancement features Add pixel by pixel to get the final fusion feature Y l .

9. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 7, characterized in that: A K×K low-frequency weighted convolution kernel group is generated using a dynamic low-pass filter (DLPF). Specifically, (11) New feature map for input Perform convolution operation to obtain filter weights Use pixel inverse shuffling to Remodeling, coming soon The height and width are reduced by half, and the channel is expanded by 4 times. The expression is as follows: Among them, the filter weight (12) For each set of filter weights Perform Softmax operation to obtain the normalized low-pass filter weights The expression is as follows: Where Ω refers to the set of (i+p,j+q) spatial positions around pixel (i,j), p,q∈[-1,0,1]; (13) Take pixel (i, j) as index and set the weights of four low-pass filters at the same position Take out in sequence and get And reshape it to get 4 low-frequency weighted kernel groups of size K×K Among them, K represents the low-frequency weighted kernel size, and the expression is as follows: Among them, i=0,1,...,H-1,j=0,1,...,W-1, each pixel corresponds to a set of low-frequency weighted convolution kernels There are H×W groups in total.

10. The method for real-time tumor tracking and benign and malignant tumor detection based on breast ultrasound video according to claim 7, characterized in that: A high-frequency weighted convolution kernel of K×K size is generated using a dynamic high-pass filter (DHPF). Specifically, (21) New feature map for input Perform convolution operation to obtain low-pass filter weights (22) Low-pass filter weights Perform Softmax operation to obtain the normalized low-pass filter weights The expression is as follows: Where Ω refers to the set of (i+p,j+q) spatial positions around pixel (i,j), p,q∈[-1,0,1]; (23) Take pixel (i, j) as index and K of all pixel positions in 2 The filter weights of the channel are taken out and obtained And Reshape to obtain a low-frequency weighted kernel of size K×K Among them, i=0,1,...,2H-1, j=0,1,...,2W-1, a total of 2H×2W low-frequency weighted kernels (24) To ensure the high-frequency characteristics of the weighted kernel, the low-frequency weighted kernel is subtracted from the identity kernel function E. Get the final high-frequency weighted kernel in,

Images