Mammary gland ultrasonic video lesion detection system based on anatomy perception enhancement network

Abstract

The invention discloses a mammary gland ultrasonic video lesion detection system based on an anatomical perception enhanced network, belongs to the technical field of intelligent medical detection, and particularly relates to the mammary gland ultrasonic video lesion detection system. The invention aims to solve the problem that the existing deep learning auxiliary diagnosis method is low in breast ultrasonic video lesion detection accuracy. The system comprises an image processing main module used for collecting breast ultrasonic video images and preprocessing the collected breast ultrasonic video images to obtain preprocessed images; the self-attention network main module is used for building an anatomical perception enhanced self-attention network model; the training main module uses the preprocessed image to train a built anatomical perception enhanced self-attention network model, and obtains a trained anatomical perception enhanced self-attention network model; and the lesion detection main module is used for loading the trained anatomical perception enhanced self-attention network model, detecting a to-be-detected breast ultrasonic image, and detecting a breast lesion area.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent medical detection, and in particular relates to a breast ultrasound video lesion detection system. Background Art

[0002] Breast tumors include benign cysts, high-risk lesions, precancerous lesions, and malignant tumors, and are common in women aged 35-55 years. At present, the incidence and mortality of breast cancer rank first and second respectively among all female malignant tumors. Early diagnosis of breast cancer is crucial for the long-term survival rate and prognosis of patients. Breast ultrasound has the advantages of low cost, non-invasiveness, no radiation, and relative sensitivity to dense breast tissue. It is an important imaging method for the diagnosis and screening of breast cancer. However, manual analysis of breast ultrasound videos is very time-consuming, resulting in a significant increase in the overall workload of radiologists. Moreover, the inherent characteristics of breast tumors in ultrasound images can easily confuse the identification and diagnosis of lesions, such as user-dependent scanning quality, heterogeneity of lesion tissue, and similarity between lesions and surrounding tissues in texture and contrast; radiologists' interpretation of these inherent characteristics is subjective and largely depends on their professional knowledge and clinical experience. Therefore, automated auxiliary diagnostic methods are necessary to alleviate these problems and assist clinical applications.

[0003] Thanks to the tremendous improvement in computer hardware storage and computing power, auxiliary diagnosis methods based on deep learning are currently the mainstream methods. However, considering the deep learning ultrasound breast lesion detection methods, the existing auxiliary diagnosis methods still have many problems: 1) Auxiliary diagnosis methods usually only give an unattractive prediction probability or category, and cannot provide more valuable information such as diagnostic basis. Especially in the field of high-risk clinical decision-making, this weak interpretability greatly limits the clinical usability of the method. 2) On the other hand, the discriminative features in different breast lesion samples are subtle and diverse, and the method has limited learning ability for these discriminative features. In order to improve the accuracy and usability of auxiliary diagnosis methods, it is very necessary to design an effective breast ultrasound video lesion detection method. Summary of the invention

[0004] In order to solve the problem of low accuracy of existing deep learning-assisted diagnosis methods in breast ultrasound video lesion detection, the present invention proposes a breast ultrasound video lesion detection system based on an anatomical perception enhancement network.

[0005] A breast ultrasound video lesion detection system based on an anatomical perception enhancement network includes an image processing main module, a self-attention network main module, a training main module and a lesion detection main module;

[0006] The image processing main module is used to collect breast ultrasound video images, pre-process the collected breast ultrasound video images, and obtain pre-processed images;

[0007] The self-attention network main module is used to build a self-attention network model with enhanced anatomical perception;

[0008] The main training module uses the preprocessed images to train the built anatomical perception enhanced self-attention network model, and obtains the trained anatomical perception enhanced self-attention network model;

[0009] The lesion detection main module is used to load the trained anatomical perception enhanced self-attention network model, detect the breast ultrasound image to be detected, and detect the breast lesion area.

[0010] The beneficial effects of the present invention are:

[0011] 1. The present invention proposes for the first time a breast ultrasound video lesion detection system based on an anatomical perception enhancement network. The system performs video-level breast lesion detection rather than being limited to the image level, which is close to the actual clinical screening process for breast cancer and improves the availability of auxiliary diagnosis methods.

[0012] 2. An anatomical perception enhancement layer is designed for the proposed breast ultrasound video lesion detection system based on anatomical perception enhancement network, which solves the problem of weak interpretability in existing deep learning-based breast lesion detection. The anatomical perception enhancement layer learns the spatial features of the breast anatomical structure within the same anatomical layer and across anatomical layers, improving the network's feature representation ability and detection accuracy.

[0013] 3. Experiments show that the present invention has good breast lesion detection performance. The average accuracy of the anatomical perception enhancement network for detecting breast lesion areas in ultrasound videos can reach 41.5%, which is at an advanced level. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of the system of the present invention;

[0015] Figure 2 Schematic diagram of the network architecture for anatomical perception enhancement;

[0016] Figure 3a This is the lesion detection result diagram of the original image 1;

[0017] Figure 3b This is the lesion detection result diagram of the original image 2;

[0018] Figure 3c This is the lesion detection result diagram of the original image 3;

[0019] Figure 3dThis is the lesion detection result diagram of the original image 4. DETAILED DESCRIPTION

[0020] Specific implementation method 1: In this implementation method, a breast ultrasound video lesion detection system based on an anatomical perception enhancement network includes:

[0021] Image processing main module, self-attention network main module, training main module and lesion detection main module;

[0022] The image processing main module is used to collect breast ultrasound video images, pre-process the collected breast ultrasound video images, and obtain pre-processed images;

[0023] The self-attention network main module is used to build a self-attention network model with enhanced anatomical perception;

[0024] The main training module uses the preprocessed images to train the built anatomical perception enhanced self-attention network model, and obtains the trained anatomical perception enhanced self-attention network model;

[0025] The lesion detection main module is used to load the trained anatomical perception enhanced self-attention network model, detect the breast ultrasound image to be detected, and detect the breast lesion area.

[0026] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that: the image processing main module is used to collect breast ultrasound video images, pre-process the collected breast ultrasound video images, and obtain the pre-processed images; the specific process is:

[0027] Step A1, obtaining breast ultrasound video images, and extracting video frame images frame by frame as a frame sequence;

[0028] Step A2, cropping the imaging area in the frame sequence obtained in step A1 to obtain a cropped image, and performing data enhancement on the cropped image to obtain a preprocessed image;

[0029] The data enhancement includes random vertical flipping and random horizontal flipping.

[0030] The preprocessed images are used as input to the anatomically-aware enhanced self-attention network model.

[0031] The other steps and parameters are the same as those in the first embodiment.

[0032] Specific implementation method three: This implementation method is different from specific implementation method one or two in that: the self-attention network main module is used to build a self-attention network model with enhanced anatomical perception; the specific process is:

[0033] Step B1: constructing a horizontal perception enhancement layer and a vertical perception enhancement layer, and constructing a spatial perception layer based on the horizontal perception enhancement layer and the vertical perception enhancement layer;

[0034] Step B2: constructing a spatial decoding layer and a cross decoding layer;

[0035] Step B3, constructing an anatomical perception enhanced self-attention network model based on step B1 and step B2;

[0036] Step B4: Construct the loss function of the anatomically-aware enhanced self-attention network model.

[0037] The other steps and parameters are the same as those in the first or second embodiment.

[0038] Specific implementation mode 4: This implementation mode is different from any one of specific implementation modes 1 to 3 in that the horizontal perception enhancement layer and the vertical perception enhancement layer are constructed in step B1; the specific process is as follows:

[0039] Step B11,

[0040] Introducing horizontal embedding pos h Visual words that represent the same anatomical layer in the breast anatomy;

[0041] Introducing vertical embedding pos v Visual words representing the anatomy of the breast across anatomical layers;

[0042] Breast tissue presents a layered anatomical structure in ultrasound, which is skin, subcutaneous tissue, glandular tissue, and muscle tissue from top to bottom. The same anatomical layer refers to the same subcutaneous tissue, glandular tissue, or muscle tissue in the horizontal direction of the image. Cross-anatomical layers refer to the image including multiple anatomical layers in the vertical direction.

[0043] The expression is as follows:

[0044]

[0045]

[0046] in,

[0047] M h Represents the number of visual words in the horizontal direction of the image, and the horizontal perception enhancement layer input feature z h The height dimensions are the same;

[0048] M w Represents the number of visual words in the vertical direction of the image, and the vertical perception enhancement layer input feature z v The width dimensions are the same;

[0049] Dividing features into small pieces of the same size is a visual word; the original size of the feature is Hz W z , divided into M h Horizontal strip H z W z / M h , these small horizontal bars are sight words. Similar vertical bars are also called sight words.

[0050] (r, k) represents the spatial coordinates of each feature embedding;

[0051] H z Represents the height of the input feature of the spatial perception layer; W z Represents the width of the input feature of the spatial perception layer;

[0052] POS h represents the horizontal position embedding, represents the rth horizontal position embedding vector,

[0053] POS v Indicates the vertical position embedding, represents the kth vertical position embedding vector;

[0054] pos represents each individual position embedding vector, represents a set of real numbers, C represents the number of channels;

[0055] Step B12

[0056] Based on horizontal embedding pos h and vertical embedding pos v , calculate the output features of the l-th horizontal perception enhancement layer and the output features of the l-th vertical perception enhancement layer; the expression is:

[0057] The horizontal perception enhancement layer and the vertical perception enhancement layer are two auxiliary branches with horizontal embedding and vertical embedding, respectively, which focus on the spatial correlation within and between layers in the breast anatomical structure;

[0058]

[0059] in,

[0060] Represents the output features of the l-th level perceptual enhancement layer;

[0061] represents the output features of the lth vertical perception enhancement layer;

[0062] represents the output features of the l-1th horizontal perceptual enhancement layer, that is, the input features of the lth horizontal perceptual enhancement layer;

[0063] Represents the intermediate output features of the l-th horizontal perceptual enhancement layer;

[0064] represents the output features of the l-1th vertical perception enhancement layer, that is, the input features of the lth vertical perception enhancement layer;

[0065] Represents the intermediate output features of the l-th vertical perceptual enhancement layer;

[0066] Indicates As input Q and K, Take input V for multi-head attention calculation;

[0067] Indicates As input Q and K, Take input V for multi-head attention calculation;

[0068] FFN stands for feedforward neural network, MultiHeadAttn stands for multi-head attention;

[0069] and It is the horizontal perception enhancement layer and the vertical perception enhancement layer features, FFN is a simple feedforward neural network, and MultiHeadAttn is multi-head attention.

[0070] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0071] Specific implementation mode 5: This implementation mode is different from any one of specific implementation modes 1 to 4 in that in step B1, a spatial perception layer is constructed based on a horizontal perception enhancement layer and a vertical perception enhancement layer; the specific process is as follows:

[0072] 1) For each feature embedded spatial coordinate (r, k), the spatial position embedding is represented by sine and cosine functions of different frequencies: expressed as:

[0073] P(pos,2i)=sin(pos / 10000 2i / d )

[0074] P(pos,2i+1)=cos(pos / 10000 2i / d )

[0075] Spatial position embedding pos s is the set of P,

[0076] pos represents the position embedding vector corresponding to the spatial coordinate (r, k), 0≤i<d, d represents the output dimension of the spatial position embedding;

[0077] P represents the position function, which is used to map pos to the spatial position embedding pos s middle;

[0078] 2) Based on the output features of the first level perception enhancement layer Output features of the lth vertical perception enhancement layer and the output features of the l-1th spatial perception layer Get the output features of the lth spatial perception layer

[0079] The expression is:

[0080]

[0081] in, represents the output features of the l-th spatial perception layer, represents the intermediate output features of the l-1th spatial perception layer,

[0082] Indicates As input Q and K, Take as input V for multi-scale deformable attention calculation;

[0083] Represents the output features of the l-th level perception enhancement layer Output features of the lth vertical perception enhancement layer Output features of the l-1 spatial perception layer The sum of

[0084] MSDeformAttn is multi-scale deformable attention.

[0085] represents the output features of the l-1th spatial perception layer, that is, the input features of the lth spatial perception layer;

[0086] It is also used as the output feature of the anatomical awareness enhancement unit.

[0087] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0088] Specific implementation method 6: This implementation method is different from any one of specific implementation methods 1 to 5 in that the spatial decoding layer and the cross decoding layer are constructed in step B2; the specific process is:

[0089] Set the input of the first decoding unit to the object query (the object query is known and initialized by a random number between 0 and 1), and the input feature of the lth decoding unit to the output feature of the l-1th decoding unit;

[0090] Decoding unit output features Through the spatial decoding layer, the output features of the l-1th spatial decoding layer are obtained

[0091] Will And the output features of the l-th spatial perception layer Input into the cross decoding layer to obtain the output features of the current decoding unit

[0092] The specific process is as follows:

[0093]

[0094] in, represents the output features of the l-1th spatial decoding layer, represents the output feature of the l-1th decoding unit, represents the intermediate output features of the l-th cross decoding layer, represents the output features of the lth cross-decoding layer, and OQ represents object query.

[0095] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0096] Specific implementation method 7: This implementation method is different from any one of specific implementation methods 1 to 6 in that a self-attention network model with enhanced anatomical perception is constructed in step B3; the specific process is as follows:

[0097] The anatomical perception enhanced self-attention network model includes a feature extraction module, an anatomical perception encoding module, and a decoding module;

[0098] The feature extraction module includes:

[0099] Basic unit 1, basic unit 2, basic unit 3, basic unit 4, basic unit 5;

[0100] The basic unit 1 includes the first convolutional layer, the BN layer, the ReLU activation layer, and the maximum pooling layer in sequence;

[0101] The basic unit 2 includes a second convolutional layer, a BN layer, a ReLU activation layer, a third convolutional layer, a BN layer, a ReLU activation layer, a fourth convolutional layer, a BN layer, and a ReLU activation layer in sequence;

[0102] The basic unit 3 sequentially includes a fifth convolutional layer, a BN layer, a ReLU activation layer, a sixth convolutional layer, a BN layer, a ReLU activation layer, a seventh convolutional layer, a BN layer, and a ReLU activation layer;

[0103] The basic unit 4 sequentially includes an eighth convolutional layer, a BN layer, a ReLU activation layer, a ninth convolutional layer, a BN layer, a ReLU activation layer, a tenth convolutional layer, a BN layer, and a ReLU activation layer;

[0104] The basic unit 5 sequentially includes an eleventh convolutional layer, a BN layer, a ReLU activation layer, a twelfth convolutional layer, a BN layer, a ReLU activation layer, a thirteenth convolutional layer, a BN layer, and a ReLU activation layer;

[0105] The anatomical perception encoding module includes in sequence:

[0106] Anatomical perception enhancement unit 1, anatomical perception enhancement unit 2, anatomical perception enhancement unit 3, anatomical perception enhancement unit 4, anatomical perception enhancement unit 5, anatomical perception enhancement unit 6;

[0107] The anatomical perception enhancement unit 1 includes a first horizontal perception enhancement layer, a first vertical perception enhancement layer, and a first spatial perception layer;

[0108] The anatomical perception enhancement unit 2 includes a second horizontal perception enhancement layer, a second vertical perception enhancement layer, and a second spatial perception layer;

[0109] The anatomical perception enhancement unit 3 includes a third horizontal perception enhancement layer, a third vertical perception enhancement layer, and a third spatial perception layer;

[0110] The anatomical perception enhancement unit 4 includes a fourth horizontal perception enhancement layer, a fourth vertical perception enhancement layer, and a fourth spatial perception layer;

[0111] The anatomical perception enhancement unit 5 includes a fifth horizontal perception enhancement layer, a fifth vertical perception enhancement layer, and a fifth spatial perception layer;

[0112] The anatomical perception enhancement unit 6 includes a sixth horizontal perception enhancement layer, a sixth vertical perception enhancement layer, and a sixth spatial perception layer;

[0113] The decoding module comprises in sequence:

[0114] Decoding unit 1, decoding unit 2, decoding unit 3, decoding unit 4, decoding unit 5, decoding unit 6;

[0115] The decoding unit 1 includes a first spatial decoding layer and a first cross decoding layer in sequence;

[0116] The decoding unit 2 includes a second spatial decoding layer and a second cross decoding layer in sequence;

[0117] The decoding unit 3 includes a third spatial decoding layer and a third cross decoding layer in sequence;

[0118] The decoding unit 4 includes a fourth spatial decoding layer and a fourth cross decoding layer in sequence;

[0119] The decoding unit 5 includes a fifth spatial decoding layer and a fifth cross decoding layer in sequence;

[0120] The decoding unit 6 includes a sixth spatial decoding layer and a sixth cross decoding layer in sequence;

[0121] The connection relationship of the anatomical perception enhanced self-attention network model is:

[0122] The preprocessed image is input into the basic unit 1 in the feature extraction module, the output feature of the basic unit 1 is input into the basic unit 2, the output feature of the basic unit 2 is input into the basic unit 3, the output feature of the basic unit 3 is input into the basic unit 4, the output feature of the basic unit 4 is input into the basic unit 5, and the output feature of the basic unit 5 is the output feature of the feature extraction module;

[0123] The feature output from the feature extraction module is input into the anatomical perception enhancement unit 1 in the anatomical perception encoding module, the feature output from the anatomical perception enhancement unit 1 is input into the anatomical perception enhancement unit 2, the feature output from the anatomical perception enhancement unit 2 is input into the anatomical perception enhancement unit 3, the feature output from the anatomical perception enhancement unit 3 is input into the anatomical perception enhancement unit 4, the feature output from the anatomical perception enhancement unit 4 is input into the anatomical perception enhancement unit 5, and the feature output from the anatomical perception enhancement unit 5 is input into the anatomical perception enhancement unit 6;

[0124] The outputs of the horizontal perception enhancement layer and the vertical perception enhancement layer in each anatomical perception enhancement unit are added and connected to the spatial anatomical perception layer;

[0125] The initialized object query and the output features of the anatomical perception enhancement unit 1 are input into the decoding unit 1, the output features of the decoding unit 1 and the output features of the anatomical perception enhancement unit 2 are input into the decoding unit 2, the output features of the decoding unit 2 and the output features of the anatomical perception enhancement unit 3 are input into the decoding unit 3, the output features of the decoding unit 3 and the output features of the anatomical perception enhancement unit 4 are input into the decoding unit 4, the output features of the decoding unit 4 and the output features of the anatomical perception enhancement unit 5 are input into the decoding unit 5, the output features of the decoding unit 5 and the output features of the anatomical perception enhancement unit 6 are input into the decoding unit 6, and the output features of the decoding unit 6 are used as the output features of the self-attention network model of anatomical perception enhancement (breast lesion detection prediction results);

[0126] The initialized object query value is a random number uniformly distributed between 0 and 1.

[0127] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0128] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that the loss function of the self-attention network model with enhanced anatomical perception is constructed in step B4; the specific process is:

[0129] The loss function of the anatomically-aware enhanced self-attention network model uses the lesion detection loss function;

[0130] The lesion detection loss function consists of two parts: classification loss and detection loss;

[0131] The class loss and detection loss are added together as the lesion detection loss function;

[0132] The class loss is defined as:

[0133]

[0134] Among them, n s is the number of samples, c is the number of breast tumor categories, y i,j represents the true category of the i-th sample, represents the predicted category of the anatomical perception enhancement network for the i-th sample;

[0135] The detection loss is defined as:

[0136]

[0137] Among them, n s is the sample size, represents the true bounding box of the i-th sample, represents the predicted bounding box of the anatomical perception enhancement network for the i-th sample, for and The smallest closed convex object of represents a parameter of the anatomical perception enhancement network in the bounding box corresponding to the i-th sample, Indicates that the i-th sample corresponds to a parameter in the real bounding box, B i,b Indicates a parameter in the bounding box corresponding to the i-th sample, x, y are the horizontal and vertical coordinates of the center point of the bounding box, w, h are the width and height of the bounding box.

[0138] The minimum closed convex object contains and The smallest closed convex set of these two sets:

[0139] A convex set means that the line segment between any two points in the set is in this set; a closed set means that the closed set contains all the boundary points of the two points; a minimum set means that it contains and The convex set with the smallest area.

[0140] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0141] Specific implementation method 9: This implementation method is different from specific implementation methods 1 to 8 in that the main training module uses the pre-processed images to train the built anatomical perception enhancement network model to obtain the trained anatomical perception enhancement network model; the specific process is:

[0142] Step C1: Select an optimizer and initialize the training parameters of the anatomical perception enhancement network model; the specific process is as follows:

[0143] The optimizer uses the Adam optimizer;

[0144] Set the random dropout rate to prevent overfitting;

[0145] Set the initial learning rate and learning rate decay rule;

[0146] Set the number of training rounds;

[0147] The decay rule is that the learning is first large and then small during the training process;

[0148] After all training-related hyperparameters (learning rate, decay law, number of training rounds) are set, start training;

[0149] Step C2: Use the preprocessed image data as a training sample set to train the anatomical perception enhancement network model until the set number of training rounds is reached or the lesion detection loss function of the anatomical perception enhancement network model converges, obtain the trained anatomical perception enhancement network model, and save the trained anatomical perception enhancement network model parameters.

[0150] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0151] Specific implementation method ten: This implementation method is different from specific implementation methods one to nine in that the lesion detection main module is used to load the trained anatomical perception enhancement network model, detect the breast ultrasound image to be detected, and detect the breast lesion area; the specific process is:

[0152] Step D1, extracting video frame images frame by frame from the breast ultrasound video to be detected as a breast ultrasound frame sequence to be detected;

[0153] Step D2: input the breast ultrasound frame sequence to be detected processed in step D1 into the trained anatomical perception enhancement network model frame by frame to obtain the breast lesion area detection result.

[0154] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0155] The following examples are used to verify the beneficial effects of the present invention:

[0156] Embodiment 1:

[0157] The specific implementation of the present invention is described below in conjunction with specific embodiments and drawings. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0158] The grayscale breast ultrasound video dataset comes from actual hospital cases, including 390 breast ultrasound videos from 390 patients, including 211 benign cases and 179 malignant cases. These videos include complete tumor scans, and the pathology report results of all patients after biopsy or surgery are used as the basic fact labels of breast lesions. It is randomly divided into training set and test set in a ratio of 8:2. Among them, 313 cases are used as training set to update model parameters; 77 cases are used as test set to test the final model.

[0159] Perform step 1:

[0160] Execute step A1, read 313 grayscale breast ultrasound videos of the training set, and extract video frame images frame by frame as a frame sequence;

[0161] Execute step A2, crop the imaging area in the image processed by step A1, scale the cropped image to a size of 512×512 and perform data enhancement; perform random vertical flipping with a probability of 0.5 and random horizontal flipping with a probability of 0.5.

[0162] Perform step 2:

[0163] Execute step B1 to construct the horizontal perception enhancement layer, vertical perception enhancement layer and spatial perception layer in the anatomical perception enhancement network. Each perception layer is composed of position embedding, feedforward neural network and multi-head attention.

[0164] Execute step B2, construct a spatial decoding layer and a cross decoding layer; the spatial decoding layer is composed of multi-scale deformable attention. The cross decoding layer is composed of a feedforward neural network and multi-scale deformable attention.

[0165] Execute step B3, construct a self-attention network model with enhanced anatomical perception;

[0166] The self-attention network structure of anatomical perception enhancement includes a feature extraction module, an anatomical perception encoding module and a decoding module. According to the order of basic unit 1, basic unit 2, basic unit 3, basic unit 4 and basic unit 5, they are connected in sequence to form a feature extraction module. According to the order of anatomical perception enhancement unit 1, anatomical perception enhancement unit 2, anatomical perception enhancement unit 3, anatomical perception enhancement unit 4, anatomical perception enhancement unit 5 and anatomical perception enhancement unit 6, they are connected in sequence to form an anatomical perception encoding module. According to the order of decoding unit 1, decoding unit 2, decoding unit 3, decoding unit 4, decoding unit 5 and decoding unit 6, they are connected in sequence to form a decoding module.

[0167] Execute step B4 to construct the anatomical perception enhancement network loss function module. The loss function module calculates the loss between the detection result and the true result, and optimizes the overall network based on this. The loss function includes category loss and detection loss.

[0168] The class loss is defined as:

[0169]

[0170] Where c is 2, y i,j represents the true category of the i-th sample, represents the predicted category of the anatomical perception enhancement network for the i-th sample;

[0171] The detection loss is defined as:

[0172]

[0173] in represents the true bounding box of the i-th sample, represents the predicted bounding box of the anatomical perception enhancement network for the i-th sample, for and The smallest closed convex object, B i,b Indicates a parameter in the bounding box corresponding to the i-th sample, x, y, w, h are the horizontal and vertical coordinates of the center point of the bounding box and the width and height of the bounding box respectively.

[0174] Perform step three:

[0175] Execute step C1, select the optimizer, and initialize the training parameters of the anatomical perception enhancement network model. Use the training sample data set to train the built anatomical perception enhancement network model. The training adopts the batch iteration method, the batch size is set to 8, the total number of training rounds is 100 rounds, and one round is defined as the iteration of all training data sets. The optimizer uses Adam, the random inactivation rate is set to 0.1, the initial learning rate is set to 0.0002 and increases and decreases according to the number of training rounds. After every 10 rounds of training, the learning rate decays to half of the original.

[0176] Execute step C2. If the network reaches the set number of training rounds or the average accuracy of the network does not improve within 10 rounds, the network is considered to have converged, and the trained network parameters are saved.

[0177] Execute step 4:

[0178] Execute step D1, and perform the image preprocessing process of step 1 on the 77 images in the test set.

[0179] Execute step D2, input the processed image into the trained network for testing, and do not update the network parameters in this process. After completing the testing of all images, calculate the average precision performance index.

[0180] Table 1 Average accuracy of breast lesion detection

[0181] method Average precision Faster-RCNN 34.5％ DeformDETR 36.9％ DDQ 37.9％ Anatomy-aware augmented network 41.5％

[0182] Table 1 shows the detection results of the proposed anatomical perception enhancement network for breast lesions in ultrasound videos. Faster-RCNN, DeformDETR and DDQ are currently commonly used detection algorithms. Compared with the existing commonly used algorithms, the system of the present invention has a significant improvement in the average accuracy index of breast lesion detection.

[0183] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A breast ultrasound video lesion detection system based on an anatomical perception enhancement network, characterized in that: The system includes an image processing main module, a self-attention network main module, a training main module and a lesion detection main module; The image processing main module is used to collect breast ultrasound video images, pre-process the collected breast ultrasound video images, and obtain pre-processed images; The self-attention network main module is used to build a self-attention network model with enhanced anatomical perception; The main training module uses the preprocessed images to train the built anatomical perception enhanced self-attention network model, and obtains the trained anatomical perception enhanced self-attention network model; The lesion detection main module is used to load the trained anatomical perception enhanced self-attention network model, detect the breast ultrasound image to be detected, and detect the breast lesion area.

2. According to claim 1, a breast ultrasound video lesion detection system based on anatomical perception enhancement network is characterized in that: The image processing main module is used to collect breast ultrasound video images, pre-process the collected breast ultrasound video images, and obtain pre-processed images; the specific process is: Step A1, obtaining breast ultrasound video images, and extracting video frame images frame by frame as a frame sequence; Step A2, cropping the imaging area in the frame sequence obtained in step A1 to obtain a cropped image, and performing data enhancement on the cropped image to obtain a preprocessed image; The data enhancement includes random vertical flipping and random horizontal flipping.

3. The breast ultrasound video lesion detection system based on anatomical perception enhancement network according to claim 2 is characterized in that: The self-attention network main module is used to build a self-attention network model with enhanced anatomical perception; the specific process is as follows: Step B1: constructing a horizontal perception enhancement layer and a vertical perception enhancement layer, and constructing a spatial perception layer based on the horizontal perception enhancement layer and the vertical perception enhancement layer; Step B2: constructing a spatial decoding layer and a cross decoding layer; Step B3, constructing an anatomical perception enhanced self-attention network model based on step B1 and step B2; Step B4: Construct the loss function of the anatomically-aware enhanced self-attention network model.

4. The breast ultrasound video lesion detection system based on anatomical perception enhancement network according to claim 3 is characterized by: In step B1, a horizontal perception enhancement layer and a vertical perception enhancement layer are constructed; the specific process is as follows: Step B11, Introducing horizontal embedding pos h Visual words that represent the same anatomical layer in the breast anatomy; Introducing vertical embedding pos v Visual words representing the anatomy of the breast across anatomical layers; The expression is as follows: in, M h Represents the number of visual words in the horizontal direction of the image; M w Represents the number of visual words in the vertical direction of the image; (r, k) represents the spatial coordinates of each feature embedding; H z Represents the height of the input feature of the spatial perception layer; W z Represents the width of the input feature of the spatial perception layer; POS h represents the horizontal position embedding, represents the rth horizontal position embedding vector, POS v Indicates the vertical position embedding, represents the kth vertical position embedding vector; pos represents each individual position embedding vector, represents a set of real numbers, C represents the number of channels; Step B12 Based on horizontal embedding pos h and vertical embedding pos v , calculate the output features of the lth horizontal perception enhancement layer and the lth vertical perception enhancement layer output features; the expression is: in, Represents the output features of the l-th level perceptual enhancement layer; represents the output features of the lth vertical perception enhancement layer; represents the output features of the l-1th horizontal perceptual enhancement layer, that is, the input features of the lth horizontal perceptual enhancement layer; Represents the intermediate output features of the l-th horizontal perceptual enhancement layer; represents the output features of the l-1th vertical perception enhancement layer, that is, the input features of the lth vertical perception enhancement layer; Represents the intermediate output features of the l-th vertical perceptual enhancement layer; Indicates As input Q and K, Take input V for multi-head attention calculation; Indicates As input Q and K, Take input V for multi-head attention calculation; FFN stands for Feedforward Neural Network and MultiHeadAttn stands for Multi-Head Attention.

5. The breast ultrasound video lesion detection system based on anatomical perception enhancement network according to claim 4 is characterized in that: In step B1, a spatial perception layer is constructed based on the horizontal perception enhancement layer and the vertical perception enhancement layer; the specific process is: 1) For each feature embedded spatial coordinate (r, k), the spatial position embedding is represented by sine and cosine functions of different frequencies: expressed as: P(pos,2i)=sin(pos / 10000 2i / d ) P(pos,2i+1)=cos(pos / 10000 2i / d ) Spatial position embedding pos s is the set of P, pos represents the position embedding vector corresponding to the spatial coordinate (r, k), 0≤i<d, d represents the output dimension of the spatial position embedding; P represents the position function, which is used to map pos to the spatial position embedding pos s middle; 2) Based on the output features of the first level perception enhancement layer Output features of the lth vertical perception enhancement layer and the output features of the l-1th spatial perception layer Get the output features of the lth spatial perception layer The expression is: in, represents the output features of the l-th spatial perception layer, represents the intermediate output features of the l-1th spatial perception layer, Indicates As input Q and K, Take as input V for multi-scale deformable attention calculation; Represents the output features of the l-th level perception enhancement layer Output features of the lth vertical perception enhancement layer Output features of the l-1 spatial perception layer The sum of MSDeformAttn is multi-scale deformable attention.

6. The breast ultrasound video lesion detection system based on anatomical perception enhancement network according to claim 5 is characterized by: In step B2, a spatial decoding layer and a cross decoding layer are constructed; the specific process is as follows: Set the input of the first decoding unit to the object query, and the input features of the lth decoding unit to the output features of the l-1th decoding unit; Decoding unit output features Through the spatial decoding layer, the output features of the l-1th spatial decoding layer are obtained Will And the output features of the l-th spatial perception layer Input into the cross decoding layer to obtain the output features of the current decoding unit The specific process is as follows: in, represents the output features of the l-1th spatial decoding layer, represents the output feature of the l-1th decoding unit, represents the intermediate output features of the l-th cross decoding layer, represents the output features of the lth cross-decoding layer, and OQ represents object query.

7. The breast ultrasound video lesion detection system based on anatomical perception enhancement network according to claim 6 is characterized by: In step B3, a self-attention network model with enhanced anatomical perception is constructed; the specific process is as follows: The anatomical perception enhanced self-attention network model includes a feature extraction module, an anatomical perception encoding module, and a decoding module; The feature extraction module includes: Basic unit 1, basic unit 2, basic unit 3, basic unit 4, basic unit 5; The basic unit 1 includes the first convolutional layer, the BN layer, the ReLU activation layer, and the maximum pooling layer in sequence; The basic unit 2 includes a second convolutional layer, a BN layer, a ReLU activation layer, a third convolutional layer, a BN layer, a ReLU activation layer, a fourth convolutional layer, a BN layer, and a ReLU activation layer in sequence; The basic unit 3 sequentially includes a fifth convolutional layer, a BN layer, a ReLU activation layer, a sixth convolutional layer, a BN layer, a ReLU activation layer, a seventh convolutional layer, a BN layer, and a ReLU activation layer; The basic unit 4 sequentially includes an eighth convolutional layer, a BN layer, a ReLU activation layer, a ninth convolutional layer, a BN layer, a ReLU activation layer, a tenth convolutional layer, a BN layer, and a ReLU activation layer; The basic unit 5 sequentially includes an eleventh convolutional layer, a BN layer, a ReLU activation layer, a twelfth convolutional layer, a BN layer, a ReLU activation layer, a thirteenth convolutional layer, a BN layer, and a ReLU activation layer; The anatomical perception encoding module includes in sequence: Anatomical perception enhancement unit 1, anatomical perception enhancement unit 2, anatomical perception enhancement unit 3, anatomical perception enhancement unit 4, anatomical perception enhancement unit 5, anatomical perception enhancement unit 6; The anatomical perception enhancement unit 1 includes a first horizontal perception enhancement layer, a first vertical perception enhancement layer, and a first spatial perception layer; The anatomical perception enhancement unit 2 includes a second horizontal perception enhancement layer, a second vertical perception enhancement layer, and a second spatial perception layer; The anatomical perception enhancement unit 3 includes a third horizontal perception enhancement layer, a third vertical perception enhancement layer, and a third spatial perception layer; The anatomical perception enhancement unit 4 includes a fourth horizontal perception enhancement layer, a fourth vertical perception enhancement layer, and a fourth spatial perception layer; The anatomical perception enhancement unit 5 includes a fifth horizontal perception enhancement layer, a fifth vertical perception enhancement layer, and a fifth spatial perception layer; The anatomical perception enhancement unit 6 includes a sixth horizontal perception enhancement layer, a sixth vertical perception enhancement layer, and a sixth spatial perception layer; The decoding module comprises in sequence: Decoding unit 1, decoding unit 2, decoding unit 3, decoding unit 4, decoding unit 5, decoding unit 6; The decoding unit 1 includes a first spatial decoding layer and a first cross decoding layer in sequence; The decoding unit 2 includes a second spatial decoding layer and a second cross decoding layer in sequence; The decoding unit 3 includes a third spatial decoding layer and a third cross decoding layer in sequence; The decoding unit 4 includes a fourth spatial decoding layer and a fourth cross decoding layer in sequence; The decoding unit 5 includes a fifth spatial decoding layer and a fifth cross decoding layer in sequence; The decoding unit 6 includes a sixth spatial decoding layer and a sixth cross decoding layer in sequence; The connection relationship of the anatomical perception enhanced self-attention network model is: The preprocessed image is input into the basic unit 1 in the feature extraction module, the output feature of the basic unit 1 is input into the basic unit 2, the output feature of the basic unit 2 is input into the basic unit 3, the output feature of the basic unit 3 is input into the basic unit 4, the output feature of the basic unit 4 is input into the basic unit 5, and the output feature of the basic unit 5 is the output feature of the feature extraction module; The feature output from the feature extraction module is input into the anatomical perception enhancement unit 1 in the anatomical perception encoding module, the feature output from the anatomical perception enhancement unit 1 is input into the anatomical perception enhancement unit 2, the feature output from the anatomical perception enhancement unit 2 is input into the anatomical perception enhancement unit 3, the feature output from the anatomical perception enhancement unit 3 is input into the anatomical perception enhancement unit 4, the feature output from the anatomical perception enhancement unit 4 is input into the anatomical perception enhancement unit 5, and the feature output from the anatomical perception enhancement unit 5 is input into the anatomical perception enhancement unit 6; The initialized object query and the output features of the anatomical perception enhancement unit 1 are input into the decoding unit 1, the output features of the decoding unit 1 and the output features of the anatomical perception enhancement unit 2 are input into the decoding unit 2, the output features of the decoding unit 2 and the output features of the anatomical perception enhancement unit 3 are input into the decoding unit 3, the output features of the decoding unit 3 and the output features of the anatomical perception enhancement unit 4 are input into the decoding unit 4, the output features of the decoding unit 4 and the output features of the anatomical perception enhancement unit 5 are input into the decoding unit 5, the output features of the decoding unit 5 and the output features of the anatomical perception enhancement unit 6 are input into the decoding unit 6, and the output features of the decoding unit 6 are used as the output features of the self-attention network model of anatomical perception enhancement; The initialized object query value is a random number uniformly distributed between 0 and 1.

8. A breast ultrasound video lesion detection system based on an anatomical perception enhancement network according to claim 7, characterized in that: In step B4, the loss function of the self-attention network model with enhanced anatomical perception is constructed; the specific process is: The loss function of the anatomically-aware enhanced self-attention network model uses the lesion detection loss function; The lesion detection loss function consists of two parts: classification loss and detection loss; The class loss and detection loss are added together as the lesion detection loss function; The class loss is defined as: Among them, n s is the number of samples, c is the number of breast tumor categories, y i,j represents the true category of the i-th sample, represents the predicted category of the anatomical perception enhancement network for the i-th sample; The detection loss is defined as: Among them, n s is the sample size, represents the true bounding box of the i-th sample, represents the predicted bounding box of the anatomical perception enhancement network for the i-th sample, for and The smallest closed convex object of represents a parameter of the anatomical perception enhancement network in the bounding box corresponding to the i-th sample, Indicates that the i-th sample corresponds to a parameter in the real bounding box, x, y are the horizontal and vertical coordinates of the center point of the bounding box, w, h are the width and height of the bounding box.

9. The breast ultrasound video lesion detection system based on anatomical perception enhancement network according to claim 8, characterized in that: The training main module uses the preprocessed images to train the built anatomical perception enhancement network model to obtain the trained anatomical perception enhancement network model; the specific process is: Step C1: Select an optimizer and initialize the training parameters of the anatomical perception enhancement network model; the specific process is as follows: The optimizer uses the Adam optimizer; Set the random dropout rate; Set the initial learning rate and learning rate decay rule; Set the number of training rounds; Step C2: Use the preprocessed image data as a training sample set to train the anatomical perception enhancement network model until the set number of training rounds is reached or the lesion detection loss function of the anatomical perception enhancement network model converges, obtain the trained anatomical perception enhancement network model, and save the trained anatomical perception enhancement network model parameters.

10. The breast ultrasound video lesion detection system based on anatomical perception enhancement network according to claim 9, characterized in that: The lesion detection main module is used to load the trained anatomical perception enhancement network model, detect the breast ultrasound image to be detected, and detect the breast lesion area; the specific process is: Step D1, extracting video frame images frame by frame from the breast ultrasound video to be detected as a breast ultrasound frame sequence to be detected; Step D2: input the breast ultrasound frame sequence to be detected processed in step D1 into the trained anatomical perception enhancement network model frame by frame to obtain the breast lesion area detection result.

Images