Ultrasound artifact detection method based on teacher-student network anomaly detection algorithm

Abstract

The present invention discloses an ultrasound artifact detection method based on a teacher-student network anomaly detection algorithm. The method classifies several pre-acquired ultrasound image sequences into artifact-bearing abnormal images and artifact-free normal images. A teacher-student network anomaly detection model is constructed, wherein the output of the teacher network is filtered and then subjected to a single-class bottleneck embedding module, serving as the input to the student network. The teacher-student network anomaly detection model is trained using artifact-free normal images until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network. Ultrasound image sequences acquired in real time are input into the trained anomaly detection model to obtain artifact-free normal images. This method applies the teacher-student network anomaly detection algorithm to ultrasound image artifact detection, boasting high computational efficiency, the ability to effectively detect different types of ultrasound artifacts, and strong robustness.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to an ultrasonic artifact detection method based on a teacher-student network anomaly detection algorithm. Background Art

[0002] Ultrasound imaging is a widely used imaging technique in medical diagnosis. It uses the propagation characteristics of ultrasound waves through tissue to image the internal structures of the human body. However, due to the interference and influence of various factors during the ultrasound imaging process, various artifacts often appear in the images, seriously affecting doctors' ability to accurately diagnose the images. Therefore, how to effectively detect and remove artifacts in ultrasound images has been a research hotspot in this field.

[0003] Existing ultrasound artifact detection methods mainly include methods based on image features, methods based on signal features, and methods based on machine learning. Among them, methods based on image features identify artifacts by analyzing features such as texture and edges in the image, but are significantly affected by image quality. Methods based on signal features detect artifacts by analyzing the statistical characteristics of echo signals, but are sensitive to noise. In recent years, with the development of machine learning technology, artifact detection methods based on machine learning have received widespread attention. Such methods can effectively utilize large amounts of clinical data for modeling and learning, thereby achieving accurate detection of artifacts.

[0004] However, existing machine learning-based artifact detection methods have the following main problems:

[0005] 1) Model training requires a large amount of labeled training data, and data labeling is time-consuming and labor-intensive.

[0006] 2) The model has poor generalization ability and is difficult to deal with small or subtle artifacts;

[0007] 3) The model parameters are large, and training and fine-tuning are time-consuming.

[0008] Therefore, there is an urgent need for a method or system for detecting and identifying artifacts in ultrasound images using unsupervised learning. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide an ultrasound artifact detection method based on a teacher-student network anomaly detection algorithm, which solves the problems in the prior art of large amount of model training data, time-consuming and labor-intensive, and poor generalization ability.

[0010] The present invention adopts the following technical solutions to solve the above technical problems:

[0011] The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm includes the following steps:

[0012] Step 1: Classify several pre-acquired ultrasound image sequences into abnormal images with artifacts and normal images without artifacts;

[0013] Step 2: Construct an anomaly detection model for the teacher-student network. The anomaly detection model includes a teacher network, a single-class bottleneck embedding module, and a student network. The output of the teacher network is filtered and embedded in the single-class bottleneck embedding module before serving as the input of the student network.

[0014] Step 3: Use artifact-free normal images to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network;

[0015] Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image with artifacts removed.

[0016] In step 2, the specific process of building the anomaly detection model for the teacher-student network is as follows:

[0017] Step 2.1: Build a teacher network and use WideResNet50 pre-trained on the ImageNet dataset as the teacher encoder.

[0018] Step 2.2: Construct an adaptive filter module, connected between the teacher network and the single-class bottleneck embedding module. It is designed as a learnable layer and includes a 3x3 convolution kernel for adjusting the filtering parameters.

[0019] Step 2.3: Build a single-class bottleneck embedding module connected between the teacher network and the student network to suppress abnormal features;

[0020] Step 2.4: Construct the student network and use the symmetric inverse structure of the teacher encoder as the structure of the student decoder.

[0021] In step 3, the specific process of training the anomaly detection model of the teacher-student network is as follows:

[0022] Step 3.1. Define the calculation method of the anomaly score map, the loss function of the student network, and the pixel-level anomaly score for artifact localization.

[0023] Step 3.2: Initialize the teacher network using ImageNet pre-trained weights, and randomly initialize the student network and the single-class bottleneck embedding module.

[0024] Step 3.3: Input the images in the training set into the teacher network encoder, obtain the features generated after passing through the three encoder sub-modules, and input them into the adaptive filter for filtering;

[0025] Step 3.4: Input the filtered features into the single-class bottleneck embedding module to obtain fused and compact features; input them into the student decoder to obtain the features generated by the three decoder sub-modules in turn;

[0026] Step 3.5: Use the loss function to calculate the loss value and update the parameters of the teacher network, student network, and single-class bottleneck embedding module through backpropagation;

[0027] Step 3.6: Determine whether all images in the current training set have been traversed. If so, end the training; otherwise, return to step 3.3.

[0028] The anomaly score map is calculated using the following formula:

[0029]

[0030] The loss function of the student network is expressed as follows:

[0031]

[0032] in, f represents the calculated vector representation, C k 、H k 、W k They represent the number of channels, height, and width of the activation vector of the kth layer, h and w represent the width and height of the feature map of the corresponding layer, respectively; K is the number of feature extraction layers.

[0033] The initialization parameter of the adaptive filter module is a trainable tensor α, which represents the starting point of the filter strength, and is based on the teacher network output F teacher The local mean and standard deviation of α are dynamically adjusted.

[0034] For each pixel p, the filter parameter α p Determined by the following formula:

[0035]

[0036] Among them, μ p and σ p are the local mean and standard deviation of pixel p, and is the mean and standard deviation of the entire feature map, and the adjusted filter parameter α is used to filter the feature map F teacher Perform adaptive filtering to obtain the filtered feature map F filtered :F filtered =AdaptiveFilter(F teacher , α).

[0037] The single-class bottleneck embedding module first downsamples the original image multiple times to obtain fused features of different scales, and then inputs the fused feature map into the self-attention module to calculate self-attention.

[0038] The self-attention module includes the generation of queries, keys, and values, as well as an attention-weighted output step, wherein the queries, keys, and values ​​are transformed through linear layers. For each position, the attention weight is calculated and weighted summed to obtain the self-attention output, which is fused with the original output to obtain an enhanced representation.

[0039] It also includes a heat map output step to identify the artifact area, as follows:

[0040] First, obtain the output of the student network and the output of the teacher network, and then compare them to obtain the anomaly score of each pixel;

[0041] Then, the feature map is converted into a heat map based on the calculated anomaly score to show the artifact area in the ultrasound image;

[0042] Finally, a heatmap with artifact regions marked is output.

[0043] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, call all or part of the steps of the method.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. This method applies the teacher-student network anomaly detection algorithm to ultrasound image artifact detection, which can effectively detect different types of ultrasound artifacts and has strong robustness.

[0046] 2. The teacher network can efficiently learn the feature representation of normal tissue from a large amount of unlabeled data, providing valuable knowledge to the student network; the student network uses the knowledge of the teacher network to detect artifacts without the need for a large amount of labeled training data, reducing the workload of data labeling.

[0047] 3. This method adds an adaptive filter module after the teacher network to help clean the data, thereby improving the quality of ultrasound image feature learning; and adds an attention mechanism to the single-class bottleneck embedding module between the teacher and student networks to improve the detection accuracy of small or inconspicuous artifacts.

[0048] 4. This method has high computational efficiency and can realize artifact detection of real-time ultrasound imaging. It can be promoted and applied in corresponding fields and has good market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flowchart of the method of the present invention.

[0050] Figure 2 Schematic diagram of the model training process of the method of the present invention.

[0051] Figure 3 Schematic diagram of the artifact image detection process of the present invention. DETAILED DESCRIPTION

[0052] The structure and working process of the present invention will be further described below with reference to the accompanying drawings.

[0053] The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm includes the following steps:

[0054] Step 1: Classify several pre-acquired ultrasound image sequences into abnormal images with artifacts and normal images without artifacts;

[0055] Step 2: Construct an anomaly detection model for the teacher-student network. The anomaly detection model includes a teacher network, a single-class bottleneck embedding module, and a student network. The output of the teacher network is filtered and embedded in the single-class bottleneck embedding module before serving as the input of the student network.

[0056] Step 3: Use artifact-free normal images to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network;

[0057] Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image with artifacts removed.

[0058] Specific embodiments, such as Figures 1 to 3 As shown,

[0059] The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm includes the following steps:

[0060] Step 1: Classify several pre-acquired ultrasound image sequences into abnormal images with artifacts and normal images without artifacts;

[0061] Step a, pre-collecting a sequence of ultrasound images to be detected, including normal images without artifacts and abnormal images with artifacts;

[0062] Step b: label and classify the collected images; specifically:

[0063] b1: Let represents a set of normal images without artifacts, Represents a set of both normal images and images with artifacts. Use OpenCV to resize the images to a 224*224, 3-channel format.

[0064] b2: For images with artifacts, select them and place them in a folder called "artifacts", and generate the corresponding ground truth binary map to prepare for the subsequent evaluation stage (this step can also be omitted);

[0065] Step 2: Construct a teacher-student network anomaly detection model. The anomaly detection model mainly includes a teacher network, a single-class bottleneck embedding module, and a student network. The output of the teacher network is filtered and then embedded in the single-class bottleneck embedding module before being used as the input of the student network. The teacher network can learn the feature representation of normal tissue from unlabeled image sequences, and the student network uses the knowledge provided by the teacher network to learn to extract normal features. The details are as follows:

[0066] c1: Build a teacher network and use the WideResNet50 pre-trained on the ImageNet dataset as the teacher encoder. The specific configuration is as follows:

[0067] Convolutional layer: 7*7, 64 convolution kernels, stride 2, padding 3, max pooling layer: 3*3, stride 2. Four residual modules, where the convolutional block structure is composed of 1*1, 3*3, and 1*1 convolutional layers plus a 1*1 skip connection layer. The identity block is a superposition of 1*1, 3*3, and 1*1 convolutional layers. The width factors of the four residual blocks are set to k*64, k*128, k*256, and k*512, respectively, with k set to 2, followed by a 7*7 pooling layer and a fully connected layer.

[0068] c2: Adaptive filter module, designed as a learnable layer, includes a 3x3 convolution kernel for adjusting the filter parameters. The parameters of this module are initialized to α, where α is a trainable tensor with an initial value of 0.5, indicating the starting point of the filter strength. The adaptive filter module is based on the output F of the teacher network. teacher The local mean and standard deviation of α are dynamically adjusted. Specifically, for each pixel point p, the filter parameter α p Determined by the following formula:

[0069]

[0070] Among them, μ p and σ p are the local mean and standard deviation of pixel p, and is the mean and standard deviation of the entire feature map. Using the adjusted filter parameter α, the feature map F teacher Perform adaptive filtering to obtain the filtered feature map F filtered :F filtered=AdaptiveFilter(F teacher , α).

[0071] C3: Build a single-class bottleneck embedding module, located between the teacher and student networks, to further suppress the transfer of abnormal features. First, the original image is downsampled multiple times to obtain features at different scales. The downsampling process includes multiple 3x3 convolutional layers with a stride of 2, batch normalization, and ReLU activation functions. Finally, a single 1x1 convolutional layer is used to obtain richer fused features.

[0072] For the fused feature map, self-attention is calculated so that the model can pay more attention to the details of the ultrasound image. This module includes the generation of query, key, value, and an attention weighted output step. The query, key, and value in the self-attention module are transformed through a linear layer. For each position (i, j), the attention weight is calculated and the weighted sum is obtained to obtain the self-attention output, which is compared with the original output F MFF The enhanced representation is obtained by fusion, where λ = 0.5 is a scaling factor used to balance the contribution of the original features and the self-attention features:

[0073] F attention =∑ i，j A ij V j , F enhanced =F MFF +λ·F attention

[0074] A single-class embedding layer is subsequently introduced, and the fourth layer of ResNet is used to implement this function.

[0075] c4: Construct the student network. The student decoder's architecture is a symmetric inverse of the teacher encoder. For example, the downsampling layers in the teacher encoder are replaced by upsampling layers in the student decoder. Since WideResNet is used as the teacher encoder, to implement the corresponding student decoder, deconvolution layers with a kernel size of 2 and a stride of 2 are used as upsampling layers. For each decoder block, upsampling is performed by the first layer. The other components in the original residual layer remain unchanged.

[0076] Step 3: Use normal images without artifacts to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network. Specifically:

[0077] d1: Define the calculation method of the anomaly score map. Let φ represent the result of projecting the original input image I through the single-class bottleneck embedding module. The activation response generated by the corresponding teacher and student networks is Among them E k 、D k denote the encoding module in the k-th teacher network and the decoding module in the student network, respectively. f represents the calculated vector representation, C k 、H k 、W k Denote the number of channels, height, and width of the activation vector of the kth layer, respectively. Using cosine similarity as the loss, a two-dimensional anomaly score map in the vector direction is calculated:

[0078]

[0079] d2: Defines the loss function of the student network, obtained by superimposing the anomaly score maps obtained at multiple scales:

[0080]

[0081] d3: defines the pixel-level anomaly score when localizing artifacts, where ψ represents the upsampling operation implemented using bilinear interpolation:

[0082]

[0083] d4: Define the training process.

[0084] D4.1: Initialize the model. Initialize the teacher network using the ImageNet pre-trained weights (the weights do not change after this), randomly initialize the student network, and the single-class bottleneck embedding module.

[0085] D4.2: Iterate over each image in the training set and execute:

[0086] d4.2.1: Input the image into the teacher network encoder and adaptive filter to obtain the features f1, f2, and f3 generated by the three encoder submodules respectively;

[0087] d4.2.2: Input the features generated in the previous step into the single-class bottleneck embedding module to obtain the fused and compacted features φ;

[0088] d4.2.3: Input the feature φ passed through the single-class bottleneck embedding module into the student decoder to obtain the features g1, g2, and g3 generated by the three decoder sub-modules respectively;

[0089] D4.2.4: Calculate the loss value using the formula for the loss function L and update the parameters of the teacher, student, and single-class bottleneck embedding modules through backpropagation;

[0090] d4.2.5: If the image traversal in the current training set is completed and the loss value converges, the training ends; otherwise, go to step d4.2.1;

[0091] Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image with artifacts removed; specifically:

[0092] e1: Input the image to be tested into the model and first obtain the outputs f1, f2, and f3 of each layer of the teacher network;

[0093] e2: The output of the teacher network is then fed into the adaptive filter, the single-class bottleneck embedding module, and the student network to obtain the outputs g1, g2, and g3 of each layer of the student network;

[0094] e3: According to M mentioned in d1 k (h, w) calculation formula is used to calculate the anomaly score map, that is, the anomaly score of each pixel. The principle is that when the input image to be detected contains artifacts, due to the anomaly filtering effect of the single-class bottleneck layer, the features received and reconstructed by the student remain largely consistent with the normal image, while the features extracted by the teacher contain abnormal information. Therefore, the anomaly score calculated by M is the specific anomaly information detected.

[0095] The above method also includes a heat map output step for identifying the artifact area, as follows:

[0096] Step f: Convert the calculated abnormal score map into a heat map to display the artifact area in the ultrasound image; specifically, for M calculated in step e3, call functions such as cv2.applyColorMap to obtain a heat map;

[0097] Step g: Use the normal representation of the corresponding image output by the student network as the normal picture after removing artifacts; specifically, take the image output by the last module of the student network decoder as the image after removing artifacts.

[0098] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, call all or part of the steps of the method.

[0099] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0100] It should be understood that this solution is not limited to the specific implementation methods described above. Devices and structures not described in detail should be understood to be implemented in a common manner in the art. Any person skilled in the art can, without departing from the scope of this solution, use the methods and technical content disclosed above to make many possible changes and modifications to this solution, or modify it into equivalent embodiments with equivalent changes, without affecting the essence of this solution. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of this solution without departing from the content of this solution are still within the scope of protection of this solution.

Claims

1. An ultrasound artifact detection method based on a teacher-student network anomaly detection algorithm, characterized by: The steps include: Step 1: Classify several pre-acquired ultrasound image sequences into abnormal images with artifacts and normal images without artifacts; Step 2: Construct an anomaly detection model for the teacher-student network. The anomaly detection model includes a teacher network, a single-class bottleneck embedding module, and a student network. The output of the teacher network is filtered and then subjected to the single-class bottleneck embedding module before serving as the input to the student network. The specific process of constructing the anomaly detection model for the teacher-student network is as follows: Step 2.1: Build a teacher network and use WideResNet50 pre-trained on the ImageNet dataset as the teacher encoder. Step 2.2: Construct an adaptive filter module, which is connected between the teacher network and the single-class bottleneck embedding module. It is designed as a learnable layer and includes a 3x3 convolution kernel for adjusting the filtering parameters. The initialization parameter of the adaptive filter module is a trainable tensor α, which represents the starting point of the filtering strength. teacher The local mean and standard deviation of α are dynamically adjusted; Step 2.3: Build a single-class bottleneck embedding module connected between the teacher network and the student network to suppress abnormal features; The single-class bottleneck embedding module first downsamples the original image multiple times to obtain fused features of different scales, and then inputs the fused feature map into the self-attention module to calculate self-attention; Step 2.4: Construct the student network and use the symmetrical inverse structure of the teacher encoder as the structure of the student decoder. Step 3: Use artifact-free normal images to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network; Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image with artifacts removed.

2. The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 1, characterized in that: In step 3, the specific process of training the anomaly detection model of the teacher-student network is as follows: Step 3.

1. Define the calculation method of the anomaly score map, the loss function of the student network, and the pixel-level anomaly score for artifact localization. Step 3.2: Initialize the teacher network using ImageNet pre-trained weights, and randomly initialize the student network and the single-class bottleneck embedding module. Step 3.3: Input the images in the training set into the teacher network encoder, obtain the features generated after passing through the three encoder sub-modules, and input them into the adaptive filter for filtering; Step 3.4: Input the filtered features into the single-class bottleneck embedding module to obtain fused and compact features; Input into the student decoder and obtain the features generated by the three decoder sub-modules in turn; Step 3.5: Use the loss function to calculate the loss value and update the parameters of the teacher network, student network, and single-class bottleneck embedding module through backpropagation; Step 3.6: Determine whether all images in the current training set have been traversed. If so, end the training; otherwise, return to step 3.

3.

3. The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 2, characterized in that: The anomaly score map is calculated using the following formula: The loss function of the student network is expressed as follows: in, f represents the calculated vector representation, C k 、H k 、W k They represent the number of channels, height, and width of the activation vector of the kth layer, h and w represent the width and height of the feature map of the corresponding layer, respectively; K is the number of feature extraction layers.

4. The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 1, characterized in that: For each pixel p, the filter parameter α p Determined by the following formula: Among them, μ p and σ p are the local mean and standard deviation of pixel p, and is the mean and standard deviation of the entire feature map, and the adjusted filter parameter α is used to filter the feature map F teacher Perform adaptive filtering to obtain the filtered feature map F filtered :F filtered =AdaptiveFilter(F teacher , α).

5. The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 1, characterized in that: The self-attention module includes the generation of queries, keys, and values, as well as an attention-weighted output step, wherein the queries, keys, and values ​​are transformed through linear layers. For each position, the attention weight is calculated and weighted summed to obtain the self-attention output, which is fused with the original output to obtain an enhanced representation.

6. The ultrasound artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 1, characterized in that: It also includes a heat map output step to identify the artifact area, as follows: First, obtain the output of the student network and the output of the teacher network, and then compare them to obtain the anomaly score of each pixel; Then, the feature map is converted into a heat map based on the calculated anomaly score to show the artifact area in the ultrasound image; Finally, a heatmap with artifact regions marked is output.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, call the steps of the method according to any one of claims 1 to 6.

Images