A method for detecting against adversarial perturbations ultrasound images

Abstract

The application discloses a detection method for anti-disturbance ultrasonic images, wherein before an ultrasonic image is input into an ultrasonic image diagnosis system, the ultrasonic image is first input into a multi-angle detection module to detect and remove ultrasonic disturbance samples, then the data after removing the disturbance samples is input into the ultrasonic image diagnosis system for processing to obtain a system processing result, a test evaluation module calculates and analyzes different detection method results, then the results are input into a comparison decision module to analyze advantages and disadvantages of the detection methods, and the detection method which can most improve the safety of the ultrasonic image diagnosis system is given. The application combines the frequency domain characteristics of the image, proposes a multi-angle detection method for the disturbance ultrasonic samples, avoids the disturbance ultrasonic samples from being input into the ultrasonic image diagnosis system to affect the system diagnosis effect, and thus effectively improves the safety and reliability of the system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image recognition system security performance improvement, and particularly relates to a detection method for an adversarial perturbation ultrasound image. BACKGROUND

[0002] Ultrasound images have a wide range of applications in the medical field. It has the characteristics of high cost-effectiveness, no danger, strong portability, and can be routinely implemented in many clinical procedures, promoting the development of clinical medicine. In particular, with the development of deep learning, deep learning-based applications in ultrasound medical diagnosis systems are increasing, but the vulnerability of deep learning itself also poses a security threat to these applications. People obtain perturbation ultrasound samples by adding some noise that is not easily perceived by humans to the original ultrasound samples. These perturbation ultrasound samples make the system application error when input into the ultrasound image diagnosis system, leading to the occurrence of medical accidents and reducing the reliability of the system. This paper identifies and removes the perturbation ultrasound samples through a detection-based method to prevent them from being input into the ultrasound image diagnosis system, thereby improving the security of the ultrasound image diagnosis system. SUMMARY

[0003] To solve the problems in the prior art, the application provides a detection method for an adversarial perturbation ultrasound image, which improves the robustness of the ultrasound image diagnosis system.

[0004] To achieve the above-mentioned purpose, the technical solution adopted by the application is as follows:

[0005] A detection method for an adversarial perturbation ultrasound image, before the ultrasound image is input into the ultrasound image diagnosis system, the ultrasound image is first input into a multi-angle detection module for detection and removal of ultrasound perturbation samples, then the data after removing the perturbation samples is input into the ultrasound image diagnosis system for processing to obtain the system processing result, the test evaluation module calculates and analyzes the results of different detection methods, then the results are input into the comparison decision module to analyze the advantages and disadvantages of the detection methods, and the detection method that can most improve the security of the ultrasound image diagnosis system is given.

[0006] Further, the detection method includes a multi-angle ultrasound image detection module, the multi-angle ultrasound image detection module includes frequency domain conversion-based detection, feature distribution-based detection, and adversarial training-based detection;

[0007] The test evaluation module includes calculation and analysis of three aspects of detection rate, reliability, and minimum perceptibility;

[0008] The comparison decision, according to the results of the test evaluation module, gives the detection method that can most improve the security of the system.

[0009] Further, the multi-angle detection module based on the detection method of frequency domain conversion:

[0010] The frequency domain highlights the pattern of noise more than the spatial domain. For speckle noise and artificially added noise of the ultrasound image, the method of converting from the spatial domain to the frequency domain is used to distinguish the clean ultrasound sample and the disturbed ultrasound sample, and the differences in different frequency bands of the two types of samples in the frequency domain are analyzed to provide prior knowledge for the subsequent convolutional neural network to extract frequency domain features.

[0011] First, the original clean ultrasound sample and the ultrasound sample after adding disturbance are converted from the spatial domain to the frequency domain, and the definition of Fourier transform is adopted:

[0012]

[0013]

[0014] Where e is a natural constant, π is the circular constant, M and N are the height and width of the image respectively, f(x, y) is the pixel value of the clean sample at point (x, y), f(x', y') is the pixel value of the disturbed sample at point (x', y'), F(u, v) is the converted frequency value of the clean sample, F(u', v') is the converted frequency value of the disturbed sample, and then the coefficient of variation C.V of the spectrum graph is calculated. C and V are not defined separately, that is, the ratio of the standard deviation to the mean of the spectrum graph, defined as follows:

[0015]

[0016] Where Std is the standard deviation of the spectrum graph, and Mean is the mean of the spectrum graph. After obtaining the coefficient of variation of the clean ultrasound sample and the disturbed ultrasound sample, the threshold that best distinguishes the two types of samples is found, and then the unknown ultrasound sample is judged according to this threshold.

[0017] Further, the multi-angle detection module based on the detection method of feature distribution:

[0018] Speckle noise, as a inherent interference in coherent imaging system, makes the characteristics of ultrasound system image quality are grain appearance, low signal-to-noise ratio and speckle noise; Compared with the speckle noise of the image itself, the disturbance artificially added is usually not perceived by humans; Compared with the speckle noise of the ultrasound image and the noise added to the image by human design (usually according to a certain algorithm), there is a certain difference in the feature space; For clean ultrasound sample Img clean and disturbed ultrasound sample Img pert∈X', where: X is a clean ultrasound sample dataset, and X' is a perturbed ultrasound sample dataset. A feature extraction network is designed to analyze the feature differences between the two types of samples to distinguish them.

[0019] F(f1,f2,...f i ,...f n ) = Net(Img clean ∈X)

[0020] Where: Net(·) is the feature extraction network, F(·) is the overall feature distribution of clean samples, n is the number of categories in the clean ultrasound samples, and f i (i = 1, 2, ..., n) represents the features of each class in the clean sample;

[0021] This method, which combines Transformer and convolutional neural networks, simultaneously extracts spatial and frequency domain features, and then fuses the two types of features. The specific process is as follows:

[0022] (1) Spatial domain feature extraction: For a clean ultrasound sample, it is first divided into sub-blocks with the same width and height. After linear mapping, the samples are input into the Transformer module for spatial domain feature extraction to obtain spatial domain features.

[0023] (2) Frequency Domain Feature Extraction: First, the clean ultrasound samples are Fourier transformed to obtain the image spectrum. Then, the spectrum is decomposed into different frequency bands to obtain spectrums of different frequencies. These spectrums are then fed into a Convolutional Neural Network (CNN) for frequency domain feature extraction. Finally, the features of different frequencies are concatenated to obtain the overall frequency domain features. Compared to directly extracting features from the transformed spectrum, decomposing the spectrum before feature extraction yields better frequency domain features of different frequency bands. After obtaining the spatial and frequency domain features, the two types of features are concatenated to obtain the overall image features. Specifically, for the frequency domain features, different weight coefficients are assigned for concatenation. The mean square error (MSE) of the clean ultrasound samples at different frequency bands is calculated. clean Mean Square Error (MSE) of Perturbed Ultrasonic Samples adv The ratio of MSE clean / MSE adv If the ratio is greater than 1.2 or less than 0.8, the characteristic weight of this frequency band is assigned 0.1; if the ratio is greater than or equal to 0.8 and less than 1.2, the characteristic weight of this frequency band is assigned 0.05. The specific formula structure is as follows:

[0024] F spa =Trans(Img) clean ∈X)F fre =Conv(Fre(fre1,fre2,...fre)n ))

[0025] F total = Connect(F spa , F fre )

[0026] wherein: F spa is the spatial domain feature obtained through the Transformer module; F fre is the frequency domain feature obtained through convolution, Fre(fre1, fre2,...fre n ) is the spectrum of different frequency bands; F total is the overall feature obtained after connecting the spatial domain feature and the frequency domain feature;

[0027] After obtaining the overall feature and the category feature, the feature distribution F of the clean ultrasound sample is obtained by calculating the distance between the feature of a single category and the overall feature, and the distance between the features of different categories; when an unknown ultrasound sample x * is input, the feature f * of the unknown sample is first extracted by the feature extractor, and then it is calculated whether the feature conforms to the feature distribution F; if it conforms to a category feature f i (i = 1, 2,..., n) of the clean ultrasound sample, the sample is a clean ultrasound sample, and if it does not conform to any category feature of the clean ultrasound sample, the sample is a perturbed ultrasound sample;

[0028] After obtaining the feature distribution of the clean ultrasound sample, a binary classifier is trained by maximizing the feature interval between the perturbed ultrasound sample and the clean ultrasound sample to classify the unknown sample, and the loss function is defined as follows:

[0029]

[0030] wherein, is the distance calculation function, F pert is the feature of the perturbed sample, and F clean is the extracted feature of the clean sample.

[0031] Further, the detection method based on adversarial training in the multi-angle detection module:

[0032] The adversarial training is trained by increasing the perturbed ultrasound sample as input, and the model M clean (θ) is trained according to the clean ultrasound sample, and the adversarial training model M AT (θ') is trained by mixing the perturbed ultrasound sample and the clean ultrasound sample; the model M clean (θ) correctly classifies the input clean ultrasound sample, but incorrectly classifies the ultrasound sample after adding the perturbation; the model M AT(θ') will be correct classification of both types of samples; when the input unknown sample, input to the model M clean (θ) get the prediction result Pred1, input to the model M AT (θ') get the prediction result Pred2, if the two prediction results are the same, the unknown sample category is clean ultrasound sample, if the two prediction results are not the same, the unknown sample category is disturbance ultrasound sample.

[0033] Further, the test evaluation module comprises:

[0034] The detection rate test of the detection method analyzes the detection rate of the disturbance sample by different detection methods, and the detection rate detrate is calculated according to the following formula:

[0035]

[0036] The reliability test of the detection method calculates the detection result of each detection method each time by inputting the same batch of samples into different detection methods for multiple times, and if the detection result of each detection is not more than 1% up and down, the detection method is reliable.

[0037] The minimum perceptibility calculation of the detection method obtains the adversarial sample by continuously increasing the disturbance degree for a batch of clean samples:

[0038] Img pert = x i,j + ε i,j (x i,j ∈ Img clean )

[0039] Wherein, Img pert is the obtained disturbance ultrasound sample, x i,j is a clean ultrasound sample Img clean The pixel value at the coordinate (i,j), ε i,j is the change value of the clean ultrasound sample at the coordinate (i,j); the disturbance degree Degpert is obtained by calculating the average value of all absolute values of ε i,j

[0040]

[0041] By inputting the disturbance degree from low to high into the detection method, if the detection method successfully detects the disturbance ultrasound sample, the disturbance is the minimum perceptibility of the detection method; the lower the minimum disturbance perceptibility of the detection method, the better the detection sensitivity of the detection method, and the more perceptible the disturbance ultrasound sample that is not easy to perceive.

[0042] Further, the comparison decision module comprises:

[0043] ​For a batch of unknown ultrasound samples, the comparison decision module first obtains the result of the ultrasound image diagnosis system without being processed by the detection method as the benchmark, then uses a single angle detection method and a variety of detection methods to filter out the disturbance ultrasound samples, inputs the remaining ultrasound samples into the ultrasound image diagnosis system, obtains the degree of improvement of the system safety performance of different detection methods, and comprehensively judges the detection method by combining the results of the analysis of different detection methods in the test evaluation module, and gives the detection method that can most improve the safety of the ultrasound image diagnosis system.

[0044] Compared with the prior art, the present application has the following beneficial effects:

[0045] The present application combines the frequency domain features of images and proposes a multi-angle detection method for disturbance ultrasound samples, avoids the disturbance ultrasound samples from being input into the ultrasound image diagnosis system to affect the system diagnosis effect, and thus effectively improves the safety and reliability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a flowchart of the detection method in the present application;

[0047] Figure 2 is a flowchart of the feature distribution-based detection method in the present application;

[0048] Figure 3 is a flowchart of the adversarial training-based detection method in the present application;

[0049] Figure 4 is a flowchart of the comparison decision module in the present application. DETAILED DESCRIPTION

[0050] The present application will be further described below in conjunction with embodiments.

[0051] Embodiment 1

[0052] As shown in Figures 1-4 , a detection method for adversarial disturbance ultrasound images, before the ultrasound images are input into the ultrasound image diagnosis system, the ultrasound images are first input into the multi-angle detection module for detection and removal of ultrasound disturbance samples, then the data after removal of the disturbance samples is input into the ultrasound image diagnosis system for processing to obtain the system processing result, the test evaluation module calculates and analyzes the results of different detection methods, then inputs the results into the comparison decision module to analyze the pros and cons of the detection methods, and gives the detection method that can most improve the safety of the ultrasound image diagnosis system.

[0053] As a preferred scheme, the detection method comprises:

[0054] Multi-angle ultrasound image detection module, which includes detection based on frequency domain conversion, detection based on feature distribution and detection based on adversarial training;

[0055] Test evaluation module, which includes calculation and analysis of three aspects of detection rate, reliability and minimum perceptibility;

[0056] Contrast decision, which analyzes the results of the test evaluation module and gives the detection method that can best improve the security of the system.

[0057] The detection method based on frequency domain conversion in the multi-angle detection module:

[0058] Compared with the spatial domain, the frequency domain can highlight the pattern of noise. For the speckle noise of the ultrasound image itself and the artificially added noise, the method of converting from the spatial domain to the frequency domain is used to distinguish the clean ultrasound sample and the disturbed ultrasound sample. At the same time, by analyzing the differences in different frequency bands of the two types of samples in the frequency domain, prior knowledge is provided for the subsequent convolutional neural network to extract frequency domain features;

[0059] First, the original clean ultrasound sample and the ultrasound sample after adding disturbance are converted from the spatial domain to the frequency domain, and the definition of Fourier transform is adopted:

[0060]

[0061]

[0062] Where e is a natural constant, π is the circular constant, M and N are the height and width of the image respectively; f(x,y) is the pixel value of the clean sample at point (x,y), f(x',y') is the pixel value of the disturbed sample at point (x',y'); F(u,v) is the frequency value of the converted clean sample, F(u',v') is the frequency value of the converted disturbed sample. Then the coefficient of variation C.V of the spectrum graph is calculated, C and V are not defined separately, that is, the ratio of the standard deviation to the mean of the spectrum graph, which is defined as follows:

[0063]

[0064] Where Std is the standard deviation of the spectrum graph, Mean is the mean of the spectrum graph; After obtaining the coefficient of variation of the clean ultrasound sample and the disturbed ultrasound sample, the threshold that can best distinguish the two types of samples is found, and then the category of the unknown ultrasound sample is judged according to this threshold.

[0065] The detection method based on feature distribution in the multi-angle detection module:

[0066] As an inherent interference in coherent imaging systems, speckle noise makes the image quality of ultrasound system characterized by granular appearance, low signal-to-noise ratio and speckle noise; compared with the speckle noise of the image itself, the disturbance artificially added is usually not perceived by humans; compared with the noise added to the image by artificial design (generally according to an algorithm), the ultrasound image with speckle noise will have certain differences in the feature space; for clean ultrasound samples Img clean ∈X and disturbed ultrasound samples Img pert ∈X', wherein: X is a clean ultrasound sample data set, X' is a disturbed ultrasound sample data set, and a feature extraction network is designed to analyze the feature differences of the two types of samples for distinction:

[0067] F(f1,f2,...f i ,...f n )=Net(Img clean ∈X)

[0068] Wherein: Net(·) is a feature extraction network, F(·) is the overall feature distribution of clean samples, n is the number of categories in clean ultrasound samples, f i (i=1,2,...,n) is the feature of each category in clean samples;

[0069] Based on the method combining Transformer and convolutional neural network, spatial domain features and frequency domain features are extracted, and then the two types of features are fused; the specific process is as follows:

[0070] (1) Spatial domain feature extraction: for clean ultrasound samples, first divide them into sub-blocks with the same width and height, then input them into the Transformer module after linear mapping for feature extraction in the spatial domain to obtain spatial domain features;

[0071] (2) Frequency domain feature extraction: first, the clean ultrasound samples are transformed into frequency spectrum images through Fourier transform, then the frequency spectrum images are decomposed into different frequency bands to obtain spectrum images of different frequencies, then the spectrum images are input into the convolutional neural network CNN for feature extraction in the frequency domain, and finally the features of different frequencies are connected to obtain the overall frequency domain features; compared with directly extracting features from the transformed spectrum images, decomposing the spectrum images and then extracting features can better obtain the frequency domain features of different frequency bands. After obtaining the spatial domain and frequency domain features, the two types of features are connected to obtain the overall features of the image, wherein: for the frequency domain features, different weight coefficients are assigned for connection, and the ratio of the mean square error MSE clean of the clean ultrasound samples of different frequency bands to the mean square error MSE adv of the disturbed ultrasound samples is calculated as the weight coefficient of the frequency domain features. clean / MSE advIf the ratio is greater than 1.2 or the ratio is less than 0.8, the characteristic weight of the frequency band is 0.1, and if the ratio is greater than or equal to 0.8 and less than 1.2, the characteristic weight of the frequency band is 0.05. The specific formula structure is as follows:

[0072] F spa = Trans(Img clean ∈X)F fre = Conv(Fre(fre1,fre2,...fre n ))

[0073] F total = Connect(F spa ,F fre )

[0074] Wherein: F spa is the spatial domain feature obtained through the Transformer module; F fre is the frequency domain feature obtained through convolution, Fre(fre1,fre2,...fre n ) is the spectrum of different frequency bands; F total is the overall feature obtained by connecting the spatial domain feature and the frequency domain feature;

[0075] After obtaining the overall feature and the category feature, the feature distribution F of the clean ultrasound sample is obtained by calculating the distance between the feature of a single category and the overall feature, and the distance between different category features; When inputting an unknown ultrasound sample x * , first extract the feature f * of the unknown sample through the feature extractor, then calculate whether the feature meets the feature distribution F, if it meets the feature f i of a certain category in the clean ultrasound sample (i=1,2,...,n), the sample is a clean ultrasound sample, if it does not meet any category feature in the clean ultrasound sample, the sample is a disturbance ultrasound sample;

[0076] After obtaining the feature distribution of the clean ultrasound sample, a binary classifier is trained by maximizing the feature interval between the disturbance ultrasound sample and the clean ultrasound sample to classify the unknown sample, and the loss function is defined as follows:

[0077]

[0078] Wherein, is the distance calculation function, F pert is the disturbance sample feature, and F clean is the extracted clean sample feature.

[0079] The detection method based on adversarial training in the multi-angle detection module:

[0080] The adversarial training is trained by increasing the perturbation ultrasound sample as the input, and the model M is trained according to the clean ultrasound sample clean (θ), the perturbation ultrasound sample and the clean ultrasound sample are mixed to train the adversarial training model M AT (θ'); the model M clean (θ) correctly classifies the input clean ultrasound sample, but incorrectly classifies the ultrasound sample added with the perturbation; the model M AT (θ') correctly classifies both types of samples; when an unknown sample is input, it is input to the model M clean (θ) to obtain a prediction result Pred1, and input to the model M AT (θ') to obtain a prediction result Pred2, if the two prediction results are the same, the unknown sample is a clean ultrasound sample, if the two prediction results are different, the unknown sample is a perturbation ultrasound sample.

[0081] The test evaluation module comprises:

[0082] The detection rate test of the detection method analyzes the detection rate of the perturbation sample of different detection methods, and the detection rate detrate is calculated according to the following formula:

[0083]

[0084] The reliability test of the detection method, for the same batch of samples, by inputting to different detection methods for multiple times, the detection result of each detection method is calculated each time, if the detection result of each detection is not more than 1% up and down, the detection method is reliable;

[0085] The minimum perceptibility calculation of the detection method, for a batch of clean samples, by continuously increasing the perturbation degree to obtain the adversarial sample:

[0086] Img pert =x i,j +ε i,j (x i,j ∈Img clean )

[0087] Wherein, Img pert is the obtained perturbation ultrasound sample, x i,j is a clean ultrasound sample Img clean The pixel value at the coordinate (i, j), ε i,j is the change value of the clean ultrasound sample at the coordinate (i, j); by calculating the average value of all absolute values of ε i,j The perturbation degree Degpert is obtained:

[0088]

[0089] By inputting the disturbance degree from low to high into the detection method, if the detection method successfully detects the disturbed ultrasonic sample, the disturbance is the lowest perceptibility of the detection method; the lower the lowest disturbance perceptibility of the detection method, the better the detection sensitivity of the detection method, and the more the detection method can perceive the unobvious disturbed ultrasonic sample.

[0090] The comparison decision module comprises:

[0091] For a batch of unknown ultrasonic samples, the comparison decision module first obtains the result of the ultrasonic sample directly input into the ultrasonic image diagnosis system without being processed by the detection method as a benchmark, then filters out the disturbed ultrasonic sample by using a single-angle detection method and a plurality of detection methods in combination, inputs the remaining ultrasonic sample into the ultrasonic image diagnosis system, obtains the improvement degree of the system safety performance of different detection methods, and comprehensively judges the detection method in combination with the analysis result of different detection methods in the test evaluation module to give the detection method which can most improve the safety of the ultrasonic image diagnosis system.

[0092] The above only describes the preferred embodiments of the present application, and it should be noted that those skilled in the art can make some improvements and refinements without departing from the principles of the present application, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method for detecting against adversarial perturbations of ultrasound images, characterized in that, Before the ultrasound image is input into the ultrasound image diagnosis system, the ultrasound image is first input into a multi-angle detection module to detect and remove the ultrasound disturbance samples, then the data after removing the disturbance samples is input into the ultrasound image diagnosis system for processing to obtain a system processing result, a test evaluation module calculates and analyzes the results of different detection methods, and then the results are input into a comparison decision module to analyze the advantages and disadvantages of the detection methods, and a detection method that can most improve the safety of the ultrasound image diagnosis system is given; The detection method comprises: a multi-angle ultrasound image detection module, which comprises frequency domain conversion-based detection, feature distribution-based detection and adversarial training-based detection; a test evaluation module, which comprises calculation and analysis of three aspects of detection rate, reliability and minimum perceptibility; a comparison decision, which analyzes the results of the test evaluation module to give a detection method that can most improve the safety of the system; the frequency domain conversion-based detection method in the multi-angle detection module: Compared with the spatial domain, the frequency domain can highlight the mode of noise more, and for the speckle noise of the ultrasound image itself and the artificially added noise, the method of converting from the spatial domain to the frequency domain is used to distinguish the clean ultrasound samples and the disturbance ultrasound samples, and the differences in different frequency bands of the two types of samples in the frequency domain are analyzed to provide prior knowledge for the extraction of frequency domain features by the subsequent convolutional neural network. First, the original clean ultrasound samples and the ultrasound samples after adding disturbance are converted from the spatial domain to the frequency domain, and the definition of Fourier transform is adopted: Wherein e is a natural constant, π is a circular constant, M and N are the height and width of the image respectively; f(x, y) is the pixel value of the clean sample at point (x, y), f(x', y') is the pixel value of the disturbance sample at point (x', y'); F(u, v) is the frequency value of the clean sample after conversion, F(u', v') is the frequency value of the disturbance sample after conversion, then the coefficient of variation C.V of the spectrum graph is calculated, that is, the ratio of the standard deviation to the mean value of the spectrum graph, which is defined as follows: Wherein Std is the standard deviation of the spectrum graph, and Mean is the mean value of the spectrum graph; after obtaining the coefficient of variation of the clean ultrasound sample and the disturbance ultrasound sample, the threshold value that can best distinguish the two types of samples is found, and then the unknown ultrasound sample is judged according to the threshold value.

2. The detection method for the adversarial disturbance ultrasound image according to claim 1, wherein the feature distribution-based detection method in the multi-angle detection module: For clean ultrasound samples Img clean ∈ X and perturbed ultrasound samples Img pert ∈ X', where: X is a clean ultrasound sample dataset, X' is a perturbed ultrasound sample dataset, the feature difference of the two types of samples is analyzed by a feature extraction network to distinguish: F(f1, f2,... f i ,...f n ) = Net(Img clean ∈ X) wherein: Net() is a feature extraction network, F() is the overall feature distribution of clean samples, n is the number of classes in clean ultrasound samples, f i (i = 1, 2,..., n) is the feature of each class in clean samples; a method combining Transformer and convolutional neural network is used to extract spatial domain features and frequency domain features, and then the two types of features are fused; the specific process is as follows: (1) spatial domain feature extraction: for the clean ultrasound sample, first divide it into subblocks with the same width and height, then after linear mapping, input it into the Transformer module to extract the spatial domain features, and obtain the spatial domain features; (2) Frequency domain feature extraction: first, the clean ultrasound sample is transformed into a frequency spectrum by Fourier transform, then the frequency spectrum is decomposed into different frequency bands, and the frequency spectrum of different frequencies is obtained, then the frequency spectrum is input into the convolutional neural network CNN for frequency domain feature extraction, and finally the features of different frequencies are connected to obtain the overall frequency domain feature, After obtaining the features in the spatial domain and the frequency domain respectively, the two types of features are connected to obtain the overall features of the image, wherein: for the frequency domain features, different weight coefficients are given for connection, and the ratio of the mean square error (MSE) of the clean ultrasonic sample in different frequency bands to the mean square error (MSE) of the disturbed ultrasonic sample is calculated clean adv clean adv If the ratio is greater than 1.2 or the ratio is less than 0.8, the frequency band feature is given a weight of 0.1, and if the ratio is greater than or equal to 0.8 and less than 1.2, the frequency band feature is given a weight of 0.05; the specific formula structure is as follows:​​​ F spa = Trans(Img clean ∈ X) F fre = Conv(Fre(fre1, fre2,...fre n )) F total = Connect(F spa , F fre ) Wherein: F spa is the spatial domain feature obtained by the Transformer module; F fre is the frequency domain feature obtained by convolution, Fre(fre1, fre2,...fre n ) is the spectrum of different frequency bands; F total is the overall feature obtained by connecting the spatial domain feature and the frequency domain feature; After obtaining the overall feature and the category feature, the feature distribution F of the clean ultrasound sample is obtained by calculating the distance between the feature of a single category and the overall feature, and the distance between the features of different categories; when an unknown ultrasound sample x is input * , the feature f of the unknown sample is first extracted by the feature extractor * , and then it is determined whether the feature conforms to the feature distribution F; if the feature conforms to a category feature f i (i = 1, 2,..., n) of the clean ultrasound sample, the sample is a clean ultrasound sample, and if the feature does not conform to any category feature of the clean ultrasound sample, the sample is a disturbed ultrasound sample; After obtaining the feature distribution of the clean ultrasound sample, a binary classifier is trained by maximizing the feature interval between the perturbed ultrasound sample and the clean ultrasound sample to classify unknown samples, and the loss function is defined as follows: wherein, is a distance computation function, F pert is a perturbed sample feature, F clean is a clean sample feature extracted.

3. The detection method for the adversarial perturbed ultrasound image according to claim 1, characterized in that, The detection method based on adversarial training in the multi-angle detection module: The adversarial training is trained by adding a perturbed ultrasound sample as an input, and the model M is trained according to a clean ultrasound sample clean The perturbed ultrasound sample and the clean ultrasound sample are mixed to train the adversarial training model M AT (θ'); the model M clean (θ) correctly classifies the input clean ultrasound sample, but incorrectly classifies the ultrasound sample added with the perturbation; the model M AT (θ') correctly classifies both types of samples; when an unknown sample is input, it is input to the model M clean (θ) to obtain a prediction result Pred1, and input to the model M AT (θ') to obtain a prediction result Pred2, if the two prediction results are the same, the unknown sample is a clean ultrasound sample, if the two prediction results are different, the unknown sample is a perturbed ultrasound sample.

4. The detection method for the adversarial perturbed ultrasound image according to claim 1, characterized in that, The test evaluation module comprises: Detection rate test of the detection method, analyze the detection rate of different detection methods on the perturbed sample, the detection rate detrate calculation formula is as follows: Reliability test of the detection method, for the same batch of samples, by inputting into different detection methods for multiple times, calculate the detection result of each detection method each time, if the detection result is not more than 1% up and down, the detection method is reliable; The lowest perception degree calculation of the detection method, for a batch of clean samples, by increasing the perturbation degree to obtain the adversarial sample: Img pert = x i,j + ε i,j (x i,j ∈ Img clean ) where Img pert is the perturbed ultrasound sample, x i,j is the clean ultrasound sample Img clean is the pixel value at coordinates (i,j), ε i,j is the change value of the clean ultrasound sample at coordinates (i,j); the perturbation degree Degpert is obtained by calculating the average value of all absolute values of ε i,j ​ By inputting the perturbation degree from low to high into the detection method, if the detection method successfully detects the perturbed ultrasound sample, the perturbation is the lowest perception degree of the detection method; the lower the lowest perturbation perception degree of the detection method, the better the detection sensitivity of the detection method, and the more it can perceive the imperceptible perturbed ultrasound sample.

5. The detection method for the adversarial perturbed ultrasound image according to claim 1, characterized in that, The comparison decision module: For a batch of unknown ultrasound samples, the comparison decision module first obtains the result of not being processed by the detection method, which is directly input into the ultrasound image diagnosis system as a benchmark, then uses a single-angle detection method and a combination of multiple detection methods to filter out the perturbed ultrasound sample, and inputs the remaining ultrasound sample into the ultrasound image diagnosis system to obtain the improvement degree of the system safety performance of different detection methods, and combines the analysis results of different detection methods in the test evaluation module to comprehensively judge the detection method, and gives the detection method that can best improve the safety of the ultrasound image diagnosis system.

Images