Ultrasonic artifact detection method based on teacher-student network anomaly detection algorithm

Abstract

The invention discloses an ultrasonic artifact detection method based on a teacher-student network anomaly detection algorithm, and the method comprises the steps: classifying a plurality of ultrasonic image sequences obtained in advance into abnormal images with artifacts and normal images without artifacts; an anomaly detection model of the teacher-student network is constructed, and the output of the teacher network is used as the input of the student network after sequentially passing through a filtering module and a single-class bottleneck embedding module; training an anomaly detection model of the teacher-student network by using a normal image without artifacts until a student network end has normal feature extraction and reconstruction capabilities consistent with those of the teacher network; and inputting an ultrasonic image sequence obtained in real time into the trained anomaly detection model to obtain a normal image without artifacts. According to the method, the teacher-student network anomaly detection algorithm is applied to ultrasonic image artifact detection, the calculation efficiency is high, different types of ultrasonic artifacts can be effectively detected, and the robustness is high.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to an ultrasonic artifact detection method based on a teacher-student network anomaly detection algorithm. Background Art

[0002] Ultrasound imaging is an imaging technology widely used in medical diagnosis. It uses the propagation characteristics of ultrasound in tissues to image the internal structure of the human body. However, due to the interference and influence of various factors in the ultrasound imaging process, various artificial artifacts often appear in the image, which seriously affects the doctor's correct diagnosis of the image. Therefore, how to effectively detect and remove artifacts in ultrasound images has always been a research hotspot in this field.

[0003] Existing ultrasound artifact detection methods mainly include methods based on image features, methods based on signal features, and methods based on machine learning. Among them, methods based on image features identify artifacts by analyzing features such as texture and edges in the image, but are greatly affected by image quality. Methods based on signal features detect artifacts by analyzing the statistical characteristics of echo signals, but are sensitive to noise. In recent years, with the development of machine learning technology, artifact detection methods based on machine learning have received widespread attention. Such methods can effectively use a large amount of clinical data for modeling and learning, thereby achieving accurate detection of artifacts.

[0004] However, existing artifact detection methods based on machine learning have the following main problems: 1) Model training requires a large amount of labeled training data, and data labeling is time-consuming and labor-intensive; 2) The model has poor generalization ability and is difficult to deal with small or subtle artifacts; 3) The model parameters are large, and training and fine-tuning are time-consuming.

[0005] Therefore, there is an urgent need for a method or system for detecting and identifying artifacts in ultrasound images using unsupervised learning. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide an ultrasonic artifact detection method based on a teacher-student network anomaly detection algorithm, which solves the problems in the prior art of large amount of model training data, time-consuming and labor-intensive, and poor generalization ability.

[0007] The present invention adopts the following technical solutions to solve the above technical problems: The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm comprises the following steps: Step 1, classifying a plurality of ultrasound image sequences acquired in advance into abnormal images with artifacts and normal images without artifacts; Step 2: construct an anomaly detection model of the teacher-student network; the anomaly detection model includes a teacher network, a single-class bottleneck embedding module and a student network, wherein the output of the teacher network is successively filtered and the single-class bottleneck embedding module is used as the input of the student network; Step 3: Use normal images without artifacts to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network; Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image with artifacts removed.

[0008] In step 2, the specific process of building the anomaly detection model of the teacher-student network is as follows: Step 2.1: Build a teacher network and use WideResNet50 pre-trained on the ImageNet dataset as the teacher encoder. Step 2.2: Construct an adaptive filter module, connected between the teacher network and the single-class bottleneck embedding module, designed as a learnable layer, including a 3x3 convolution kernel for adjusting the filtering parameters; Step 2.3: Build a single-class bottleneck embedding module connected between the teacher network and the student network to suppress abnormal features; Step 2.4: Construct the student network and use the symmetric inverse structure of the teacher encoder as the structure of the student decoder.

[0009] In step 3, the specific process of training the anomaly detection model of the teacher-student network is as follows: Step 3.1, define the calculation method of the anomaly score map, the loss function of the student network, and the pixel-level anomaly score for artifact localization; Step 3.2: Use the ImageNet pre-trained weights to initialize the teacher network, and randomly initialize the student network and the single-class bottleneck embedding module; Step 3.3, input the image in the training set into the teacher network encoder, obtain the features generated after passing through the three encoder sub-modules respectively, and input them into the adaptive filter for filtering; Step 3.4, input the filtered features into the single-class bottleneck embedding module to obtain fused and compact features; input them into the student decoder to obtain the features generated by the three decoder sub-modules respectively; Step 3.5: Use the loss function to calculate the loss value, and update the parameters of the teacher network, student network, and single-class bottleneck embedding module through back propagation; Step 3.6: Determine whether all images in the current training set have been traversed. If so, end the training; otherwise, return to step 3.3.

[0010] The anomaly score map is calculated using the following formula: The loss function of the student network is expressed as follows: Among them, QUOTE 、 QUOTE , f represents the calculated vector representation, QUOTE 、 QUOTE Respectively represent the number of channels, height and width of the activation vector of the kth layer, QUOTE Respectively represent the width and height of the feature map of the corresponding layer; QUOTE is the number of feature extraction layers.

[0011] The initialization parameter of the adaptive filter module is a trainable tensor QUOTE , indicating the starting point of the filter strength, according to the teacher network output QUOTE The local mean and standard deviation of QUOTE are dynamically adjusted .

[0012] For each pixel p, the filter parameter QUOTE Determined by the following formula: QUOTE , Among them, QUOTE and QUOTE are the local mean and standard deviation of pixel p, respectively. and QUOTE is the mean and standard deviation of the entire feature map, using the adjusted filter parameter QUOTE , for the feature map QUOTE Perform adaptive filtering to obtain the filtered feature map QUOTE : QUOTE .

[0013] The single-class bottleneck embedding module first downsamples the original image multiple times to obtain fused features of different scales, and then inputs the fused feature map into the self-attention module to calculate the self-attention.

[0014] The self-attention module includes the generation of queries, keys, values, and an attention-weighted output step, wherein the queries, keys, and values ​​are all transformed through linear layers, and for each position, the attention weight is calculated and weighted summed to obtain the self-attention output, which is fused with the original output to obtain an enhanced representation.

[0015] It also includes a heat map output step to identify the artifact area, as follows: First, obtain the output of the student network and the output of the teacher network, and then compare them to obtain the anomaly score of each pixel; Then, the feature map is converted into a heat map based on the calculated anomaly score to show the artifact area in the ultrasound image; Finally, a heatmap with artifact regions marked is output.

[0016] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, call all or part of the steps of the method.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. This method applies the teacher-student network anomaly detection algorithm to ultrasound image artifact detection, which can effectively detect different types of ultrasound artifacts and has strong robustness.

[0018] 2. The teacher network can efficiently learn the feature representation of normal tissue from a large amount of unlabeled data, providing valuable knowledge for the student network; the student network uses the knowledge of the teacher network to detect artifacts without the need for a large amount of labeled training data, reducing the workload of data labeling.

[0019] 3. This method adds an adaptive filter module after the teacher network to help clean the data, thereby improving the quality of ultrasound image feature learning; an attention mechanism is added to the single-class bottleneck embedding module between the teacher and student networks to improve the detection accuracy of small or inconspicuous artifacts.

[0020] 4. This method has high computational efficiency and can realize artifact detection for real-time ultrasound imaging. It can be promoted and applied in corresponding fields and has good market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The figure is a flowchart of the method of the present invention.

[0022] Figure 2 Schematic diagram of the model training process of the method of the present invention.

[0023] Figure 3 Schematic diagram of the artifact image detection process of the present invention. DETAILED DESCRIPTION

[0024] The structure and working process of the present invention will be further described below in conjunction with the accompanying drawings.

[0025] The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm comprises the following steps: Step 1, classifying a plurality of ultrasound image sequences acquired in advance into abnormal images with artifacts and normal images without artifacts; Step 2: construct an anomaly detection model of the teacher-student network; the anomaly detection model includes a teacher network, a single-class bottleneck embedding module and a student network, wherein the output of the teacher network is successively filtered and the single-class bottleneck embedding module is used as the input of the student network; Step 3: Use normal images without artifacts to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network; Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image with artifacts removed.

[0026] Specific embodiments, such as Figures 1 to 3 As shown, The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm comprises the following steps: Step 1, classifying a plurality of ultrasound image sequences acquired in advance into abnormal images with artifacts and normal images without artifacts; Step a, pre-collecting a sequence of ultrasound images to be detected, including normal images without artifacts and abnormal images with artifacts; Step b: label and classify the collected images; specifically: b1: Let QUOTE represents a normal image set without artifacts, QUOTE It represents a set of both normal images and images with artifacts. Use OpenCV to resize the images to a 224*224, 3-channel format. b2: For images with artifacts, select them and put them in a folder called "artifacts", and generate the corresponding ground truth binary map to prepare for the subsequent evaluation stage (this step can also be omitted); Step 2: Construct an anomaly detection model of the teacher-student network; the anomaly detection model mainly includes a teacher network, a single-class bottleneck embedding module and a student network, wherein the output of the teacher network is successively filtered and the single-class bottleneck embedding module is used as the input of the student network; the teacher network can learn the feature representation of normal tissue from the unlabeled image sequence, and the student network uses the knowledge provided by the teacher network to learn to extract normal features; the details are as follows: c1: Build a teacher network and use WideResNet50 pre-trained on the ImageNet dataset as the teacher encoder. The specific configuration is as follows: Convolution layer: 7 * 7, 64 convolution kernels, stride 2, padding 3, maximum pooling layer: 3 * 3, stride 2. Four residual modules, where the convolution block structure is composed of 1*1, 3*3, 1*1 convolution layers plus 1*1 jump connection layers, the identity block is a superposition of 1*1, 3*3, 1*1 convolution layers, the width factors of the four residual blocks are set to k*64, k*128, k*256, k*512, k is set to 2, followed by a 7*7 pooling layer and a fully connected layer.

[0027] c2: Adaptive filter module, designed as a learnable layer, including a 3x3 convolution kernel for adjusting the filter parameters. The parameters of this module are initialized to QUOTE , where QUOTE is a trainable tensor with an initial value of 0.5, indicating the starting point of the filter strength. The adaptive filter module is based on the output of the teacher network. The local mean and standard deviation of QUOTE are dynamically adjusted Specifically, for each pixel p, the filtering parameter QUOTE Determined by the following formula: QUOTE , Among them, QUOTE and QUOTE are the local mean and standard deviation of pixel p, respectively. and QUOTE is the mean and standard deviation of the entire feature map. Use the adjusted filter parameters QUOTE , for the feature map QUOTE Perform adaptive filtering to obtain the filtered feature map QUOTE : QUOTE .

[0028] c3: Construct a single-class bottleneck embedding module, which is located between the teacher network and the student network to further suppress the transmission of abnormal features. First, the original image is downsampled multiple times to obtain features of different scales. The downsampling process includes using multiple 3*3 convolutional layers with a step size of 2, batch normalization, and ReLU activation function, and then a 1*1 convolutional layer is used to obtain richer fusion features.

[0029] For the fused feature map, self-attention is calculated so that the model can pay more attention to the details of the ultrasound image. This module includes the generation of query, key, value, and an attention-weighted output step. The query, key, and value in the self-attention module are transformed through a linear layer. For each position (i, j), the attention weight is calculated and the weighted sum is obtained to obtain the self-attention output, which is QUOTE with the original output The enhanced representation is obtained by fusion, where QUOTE = 0.5 is a scaling factor used to balance the contribution of original features and self-attention features: A single-class embedding layer is subsequently introduced, and the fourth layer of ResNet is used to implement this function.

[0030] c4: Construct the student network. The structure of the student decoder is a symmetric inverse of the teacher encoder. For example, the downsampling layer in the teacher encoder will be replaced by the upsampling layer in the student decoder. Since WideResNet is used as the teacher encoder, in order to implement the corresponding student decoder, we use a deconvolution layer with a kernel size of 2 and a stride of 2 as the upsampling layer. For each decoder block, upsampling is performed by the first layer. The other components in the original residual layer remain unchanged.

[0031] Step 3: Use normal images without artifacts to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network. Specifically: d1: Define the calculation method of the anomaly score map, let QUOTE represents the result after the original input image I is projected by the single-class bottleneck embedding module, then the activation response generated by the corresponding teacher and student networks is QUOTE , where QUOTE 、 QUOTE denote the encoding module in the kth teacher network and the decoding module in the student network, respectively. 、 QUOTE , f represents the calculated vector representation, QUOTE 、QUOTE Represent the number of channels, height and width of the activation vector of the kth layer respectively, and use cosine similarity as the loss to calculate a two-dimensional anomaly score map in the vector direction: d2: Define the loss function of the student network, obtained by superimposing the anomaly score maps obtained at multiple scales: d3: Defines the pixel-level anomaly score when locating artifacts, where QUOTE Represents an upsampling operation implemented using bilinear interpolation: d4: Define the training process.

[0032] d4.1: Initialize the model, use the ImageNet pre-trained weights to initialize the teacher network (the weights will not change after this), randomly initialize the student network and the single-class bottleneck embedding module; d4.2: Iterate over each image in all training sets and execute: d4.2.1: Input the image into the teacher network encoder and adaptive filter to obtain the feature QUOTE generated after passing through the three encoder sub-modules ; d4.2.2: Input the features generated in the previous step into the single-class bottleneck embedding module to obtain the fused and compact features QUOTE ; d4.2.3: QUOTE the features that pass through the single-class bottleneck embedding module Input into the student decoder to obtain the feature QUOTE generated by the three decoder submodules ; d4.2.4: Use the loss function L to calculate the loss value, and update the parameters of the teacher, student, and single-class bottleneck embedding modules through backpropagation; d4.2.5: If the image traversal in the current training set is completed and the loss value converges, the training ends; otherwise, go to step d4.2.1; Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image without artifacts; specifically: e1: Input the image to be tested into the model and obtain the output QUOTE of each layer of the teacher network ; e2: The output of the teacher network is then fed into the adaptive filter, the single-class bottleneck embedding module, and the student network to obtain the output of each layer of the student network QUOTE ; e3: According to the QUOTE mentioned in d1 The calculation formula is used to calculate the anomaly score map, that is, the anomaly score of each pixel. The principle is that when the input image to be detected contains artifacts, due to the abnormal filtering effect of the single-class bottleneck layer, the features received and reconstructed by the student are largely consistent with the normal image, while the features extracted by the teacher contain abnormal information, so the anomaly score calculated by M is the specific abnormal information detected.

[0033] The above method also includes a heat map output step for identifying the artifact area, which is as follows: Step f: convert the calculated abnormal score map into a heat map to display the artifact area in the ultrasound image; specifically, for M calculated in step e3, call cv2.applyColorMap and other functions to obtain the heat map; Step g: Use the normal representation of the corresponding image output by the student network as the normal picture after removing artifacts; specifically, take the image output by the last module of the student network decoder as the image after removing artifacts.

[0034] A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, call all or part of the steps of the method.

[0035] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk, etc., which can store program code.

[0036] It should be understood that the present solution is not limited to the above-mentioned specific implementation methods, and the devices and structures not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present solution without departing from the scope of the technical solution of the present solution, or modify it into an equivalent embodiment with equivalent changes, which does not affect the essential content of the present solution. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present solution without departing from the content of the technical solution of the present solution still falls within the scope of protection of the technical solution of the present solution.

Claims

1. An ultrasonic artifact detection method based on a teacher-student network anomaly detection algorithm, characterized in that: The steps include: Step 1, classifying a plurality of ultrasound image sequences acquired in advance into abnormal images with artifacts and normal images without artifacts; Step 2: construct an anomaly detection model of the teacher-student network; the anomaly detection model includes a teacher network, a single-class bottleneck embedding module and a student network, wherein the output of the teacher network is successively filtered and the single-class bottleneck embedding module is used as the input of the student network; Step 3: Use normal images without artifacts to train the anomaly detection model of the teacher-student network until the student network has the same normal feature extraction and reconstruction capabilities as the teacher network; Step 4: Input the real-time acquired ultrasound image sequence into the trained anomaly detection model to obtain a normal image with artifacts removed.

2. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 1, characterized in that: In step 2, the specific process of building the anomaly detection model of the teacher-student network is as follows: Step 2.1: Build a teacher network and use WideResNet50 pre-trained on the ImageNet dataset as the teacher encoder. Step 2.2: Construct an adaptive filter module, connected between the teacher network and the single-class bottleneck embedding module, designed as a learnable layer, including a 3x3 convolution kernel for adjusting the filtering parameters; Step 2.3: Build a single-class bottleneck embedding module connected between the teacher network and the student network to suppress abnormal features; Step 2.4: Construct the student network and use the symmetric inverse structure of the teacher encoder as the structure of the student decoder.

3. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 2 is characterized in that: In step 3, the specific process of training the anomaly detection model of the teacher-student network is as follows: Step 3.1, define the calculation method of the anomaly score map, the loss function of the student network, and the pixel-level anomaly score for artifact localization; Step 3.2: Use the ImageNet pre-trained weights to initialize the teacher network, and randomly initialize the student network and the single-class bottleneck embedding module; Step 3.3, input the image in the training set into the teacher network encoder, obtain the features generated after passing through the three encoder sub-modules respectively, and input them into the adaptive filter for filtering; Step 3.4: Input the filtered features into the single-class bottleneck embedding module to obtain fused and compact features; Input into the student decoder, and obtain the features generated by the three decoder sub-modules in turn; Step 3.5: Use the loss function to calculate the loss value, and update the parameters of the teacher network, student network, and single-class bottleneck embedding module through back propagation; Step 3.6: Determine whether all images in the current training set have been traversed. If so, end the training; otherwise, return to step 3.

3.

4. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 3 is characterized in that: The anomaly score map is calculated using the following formula: The loss function of the student network is expressed as follows: in, , , f represents the calculated vector representation, , Respectively represent the number of channels, height and width of the activation vector of the kth layer, Respectively represent the width and height of the feature map of the corresponding layer; is the number of feature extraction layers.

5. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 2, characterized in that: The initialization parameter of the adaptive filter module is a trainable tensor , indicating the starting point of the filter strength, according to the teacher network output Dynamic adjustment of the local mean and standard deviation .

6. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 5, characterized in that: For each pixel p, the filter parameters Determined by the following formula: ， in, and are the local mean and standard deviation of pixel p, respectively. and is the mean and standard deviation of the entire feature map, using the adjusted filter parameters , for the feature map Perform adaptive filtering to obtain the filtered feature map : .

7. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 2, characterized in that: The single-class bottleneck embedding module first downsamples the original image multiple times to obtain fused features of different scales, and then inputs the fused feature map into the self-attention module to calculate the self-attention.

8. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 7, characterized in that: The self-attention module includes the generation of queries, keys, values, and an attention-weighted output step, wherein the queries, keys, and values ​​are all transformed through linear layers. For each position, the attention weight is calculated and weighted summed to obtain the self-attention output, which is fused with the original output to obtain an enhanced representation.

9. The ultrasonic artifact detection method based on the teacher-student network anomaly detection algorithm according to claim 1, characterized in that: It also includes a heat map output step to identify the artifact area, as follows: First, obtain the output of the student network and the output of the teacher network, and then compare them to obtain the anomaly score of each pixel; Then, the feature map is converted into a heat map based on the calculated anomaly score to show the artifact area in the ultrasound image; Finally, a heatmap with artifact regions marked is output.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, all or part of the steps of the method described in any one of claims 1 to 9 are called.

Images