Ultrasonic image artifact repair method, program product, electronic device and storage medium

Abstract

The application provides an artifact repairing method of an ultrasound image, a program product, an electronic device and a storage medium. The method comprises the following steps: inputting a feature map corresponding to a to-be-repaired ultrasound image and a classification result into an artifact perception attention model to obtain an attention mask output by the artifact perception attention model, wherein the feature map is obtained by performing feature extraction on the to-be-repaired ultrasound image, and the classification result represents a type of artifact in the to-be-repaired ultrasound image; inputting the attention mask into a dynamic condition generator to obtain a deep feature output by the dynamic condition generator; and inputting the deep feature into an ultrasound image generator to obtain a target ultrasound image output by the ultrasound image generator, wherein the target ultrasound image is an image after repairing artifacts in the to-be-repaired ultrasound image. In the above scheme, an artifact repairing algorithm fusing a dynamic condition generation adversarial network and an artifact perception attention model is provided, and the accuracy of repairing artifacts in an ultrasound image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an artifact repairing method of an ultrasound image, a program product, an electronic device and a storage medium. BACKGROUND

[0002] Ultrasound imaging is increasingly widely used in clinical diagnosis due to its advantages of non-invasiveness, real-time, portability, etc. However, the artifact problem in ultrasound images is increasingly prominent, which seriously affects the image quality and diagnostic accuracy. For example, in liver ultrasound, acoustic shadowing may mask tumors, and reverberation artifacts may be mistaken for cysts. These artifacts not only interfere with the identification of lesions by doctors, but also may lead to misdiagnosis or missed diagnosis, thereby affecting clinical decision-making.

[0003] With the increasing demand for ultrasound examination, the artifact problem has become a key factor restricting diagnostic accuracy and efficiency, so it is necessary to remove or repair artifacts in ultrasound images. However, the accuracy of repairing artifacts in ultrasound images in the prior art is low. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide an artifact repairing method of an ultrasound image, a program product, an electronic device and a storage medium, to solve the technical problem of low accuracy of repairing artifacts in ultrasound images in the prior art.

[0005] In a first aspect, the embodiments of the present application provide an artifact repairing method of an ultrasound image, comprising: inputting a feature map corresponding to a to-be-repaired ultrasound image and a classification result into an artifact perception attention model to obtain an attention mask output by the artifact perception attention model, wherein the feature map is obtained by performing feature extraction on the to-be-repaired ultrasound image, and the classification result represents a type of artifact in the to-be-repaired ultrasound image; inputting the attention mask into a dynamic condition generator to obtain a deep feature output by the dynamic condition generator; and inputting the deep feature into an ultrasound image generator to obtain a target ultrasound image output by the ultrasound image generator, wherein the target ultrasound image is an image after repairing artifacts in the to-be-repaired ultrasound image.

[0006] In the above scheme, an artifact repairing algorithm fusing a dynamic condition generation adversarial network and an artifact perception attention model is provided, which automatically identifies artifact regions and repairs artifacts, so that the artifacts in the ultrasound image can be repaired while the anatomical structure of the organ in the ultrasound image is retained, thereby improving the accuracy of repairing artifacts in the ultrasound image.

[0007] In an optional implementation, the artifact-aware attention model comprises a multi-layer perception, a channel attention module, and a spatial attention module, the inputting the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model comprises: inputting the classification result into the multi-layer perception to obtain a conditional embedding feature output by the multi-layer perception; fusing the conditional embedding feature and the feature map and inputting the fused feature map into the channel attention module and the spatial attention module respectively to obtain a channel attention weight map output by the channel attention module and a spatial attention weight map output by the spatial attention module; and fusing the channel attention weight map and the spatial attention weight map to obtain the attention mask. In the foregoing implementation, the artifact-aware attention model can dynamically adjust the weights of the artifact region and the normal tissue region in the feature map, thereby avoiding over-repairing of the normal tissue region, and thus the artifacts in the ultrasound image can be repaired while the anatomical structure of the organ in the ultrasound image is retained.

[0008] In an optional implementation, the dynamic condition generator comprises a residual dense block and a conditional normalization layer, the inputting the attention mask into the dynamic condition generator comprises: inputting the attention mask into the residual dense block to obtain an initial feature output by the residual dense block; and inputting the initial feature into the conditional normalization layer to obtain the deep feature output by the conditional normalization layer. In the foregoing implementation, the dynamic condition generator can achieve targeted repair of different artifact types, i.e., the dynamic condition generator can dynamically adjust the parameters of the ultrasound image generator according to the classification result of the artifact, thereby improving the repair effect of the artifact.

[0009] In an optional implementation, before the inputting the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model, the method further comprises: obtaining an original ultrasound image; inputting the original ultrasound image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or inputting the original ultrasound image into an artifact classification model to obtain the classification result output by the artifact classification model. In the foregoing implementation, the multi-scale pyramid encoder extracts multi-resolution features in the original ultrasound image, and a cross-scale feature fusion mechanism is combined, thereby the repair capability for micro lesions and complex artifacts can be enhanced, and thus the accuracy of repairing the artifacts in the ultrasound image is improved.

[0010] In an optional implementation, the multi-scale pyramid encoder includes three branches, and a hollow convolution module is included in at least one branch. In the above scheme, the introduction of spatial convolution in the multi-scale pyramid encoder can expand the receptive field, thereby enhancing the capture ability of a larger artifact area and improving the accuracy of repairing artifacts in the ultrasound image.

[0011] In an optional implementation, the artifact repairing method for the ultrasound image further includes: obtaining a sample image and label data corresponding to the sample image; inputting a sample feature map corresponding to the sample image and a sample classification result into a neural network model to obtain a predicted image output by the neural network model, and calculating a neural network loss value according to the predicted image and the sample image; inputting the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculating a first adversarial loss value of the global discriminator according to the global discrimination result, and inputting the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculating a second adversarial loss value of the local discriminator according to the local discrimination result; and optimizing the neural network model, the global discriminator and the local discriminator according to the neural network loss value, the first adversarial loss value and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network, wherein the artifact-aware multi-scale fusion network includes the artifact-aware attention model, the dynamic condition generator and the ultrasound image generator. In the above scheme, the neural network model is trained through multi-discriminator collaborative training, and the overall quality and local details of the repaired image are improved through the synergistic effect of the global discriminator and the local discriminator, thereby improving the artifact repairing capability of the artifact-aware multi-scale fusion network obtained through training.

[0012] In an optional implementation, the neural network loss value is equal to a sum of a perception loss value, a multi-scale structural similarity loss value and a classification consistency loss value, wherein the perception loss value represents a pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value represents a structural similarity between the predicted image and the sample image, and the classification consistency loss value represents a probability of an artifact when the predicted image passes through an artifact classification model. In the above scheme, the neural network model is trained through the introduction of the perception loss value, the multi-scale structural similarity loss value and the classification consistency loss value, thereby improving the artifact repairing capability of the artifact-aware multi-scale fusion network obtained through training.

[0013] In second aspect, an embodiment of the present application provides an artifact repair device for an ultrasonic image, comprising: a first input module, used to input a feature map and a classification result corresponding to the ultrasonic image to be repaired into an artifact-aware attention model, to obtain an attention mask output by the artifact-aware attention model, wherein the feature map is obtained by performing feature extraction on the ultrasonic image to be repaired, and the classification result characterizes the type of artifact in the ultrasonic image to be repaired; a second input module, used to input the attention mask into a dynamic condition generator, to obtain a depth feature output by the dynamic condition generator; a third input module, used to input the depth feature into an ultrasonic image generator, to obtain a target ultrasonic image output by the ultrasonic image generator, wherein the target ultrasonic image is an image after repairing the artifact in the ultrasonic image to be repaired.

[0014] In the above scheme, an artifact repair algorithm is provided that integrates a dynamic conditional generative adversarial network and an artifact-aware attention model. The algorithm automatically identifies artifact areas and repairs artifacts, thereby repairing artifacts in ultrasound images while preserving the anatomical structure of organs in ultrasound images, thereby improving the accuracy of repairing artifacts in ultrasound images.

[0015] In an optional embodiment, the artifact-aware attention model includes a multi-layer perceptron, a channel attention module, and a spatial attention module, and the first input module is specifically used to: input the classification result into the multi-layer perceptron to obtain the conditional embedding features output by the multi-layer perceptron; fuse the conditional embedding features with the feature map, and input them into the channel attention module and the spatial attention module respectively to obtain the channel attention weight map output by the channel attention module, and the spatial attention weight map output by the spatial attention module; fuse the channel attention weight map with the spatial attention weight map to obtain the attention mask. In the above scheme, the weights of the artifact area and the normal tissue area in the feature map can be dynamically adjusted by the artifact-aware attention model to avoid over-repairing the normal tissue area, so that the artifacts in the ultrasound image can be repaired while preserving the organ anatomical structure in the ultrasound image.

[0016] In an optional embodiment, the dynamic condition generator includes a residual dense block and a conditional normalization layer, and the second input module is specifically used to: input the attention mask into the residual dense block to obtain the initial features output by the residual dense block; input the initial features into the conditional normalization layer to obtain the depth features output by the conditional normalization layer. In the above scheme, targeted repair of different artifact types is achieved through the dynamic condition generator, that is, the dynamic condition generator can dynamically adjust the parameters of the ultrasound image generator according to the classification results of the artifacts, thereby improving the repair effect of the artifacts.

[0017] In an optional implementation, the artifact repairing apparatus of the ultrasound image further includes: a first acquisition module configured to acquire an original ultrasound image; a fourth input module configured to input the original ultrasound image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or input the original ultrasound image into an artifact classification model to obtain the classification result output by the artifact classification model. In the above scheme, the multi-resolution features in the original ultrasound image are extracted by the multi-scale pyramid encoder, and the cross-scale feature fusion mechanism is combined, so that the repairing capability for micro lesions and complex artifacts can be enhanced, and the accuracy of repairing the artifacts in the ultrasound image is improved.

[0018] In an optional implementation, the multi-scale pyramid encoder includes three branches, and at least one branch includes a hollow convolution module. In the above scheme, the spatial convolution is introduced in the multi-scale pyramid encoder, which can expand the receptive field, thereby enhancing the capturing capability for larger artifact regions, and improving the accuracy of repairing the artifacts in the ultrasound image.

[0019] In an optional implementation, the artifact repairing apparatus of the ultrasound image further includes: a second acquisition module configured to acquire a sample image and label data corresponding to the sample image; a fifth input module configured to input a sample feature map corresponding to the sample image and a sample classification result into a neural network model to obtain a predicted image output by the neural network model, and calculate a neural network loss value according to the predicted image and the sample image; a sixth input module configured to input the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculate a first adversarial loss value of the global discriminator according to the global discrimination result, and input the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculate a second adversarial loss value of the local discriminator according to the local discrimination result; an optimization module configured to optimize the neural network model, the global discriminator and the local discriminator according to the neural network loss value, the first adversarial loss value and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network, wherein the artifact-aware multi-scale fusion network includes the artifact-aware attention model, the dynamic condition generator and the ultrasound image generator. In the above scheme, the neural network model is trained through multi-discriminator collaborative training, and the overall quality and local details of the repaired image are improved through the synergistic effect of the global discriminator and the local discriminator, thereby improving the repairing capability of the artifact-aware multi-scale fusion network obtained by training for artifacts.

[0020] In an optional implementation, the neural network loss value is equal to a sum of a perceptual loss value, a multi-scale structural similarity loss value, and a classification consistency loss value, where the perceptual loss value represents a pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value represents structural similarity of the predicted image and the sample image, and the classification consistency loss value represents a probability of an artifact when the predicted image passes through an artifact classification model. In the foregoing scheme, the neural network model is trained by introducing the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value, so as to improve the artifact repairing capability of the artifact-perception multi-scale fusion network obtained through training.

[0021] In a third aspect, an embodiment of the present application provides a computer program product, including computer program instructions, which, when read and executed by a processor, perform the artifact repairing method of the ultrasonic image according to the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a bus; the processor and the memory complete mutual communication through the bus; the memory stores computer program instructions executable by the processor, and the processor calling the computer program instructions can perform the artifact repairing method of the ultrasonic image according to the first aspect.

[0023] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer program instructions, and the computer program instructions, when executed by a computer, cause the computer to perform the artifact repairing method of the ultrasonic image according to the first aspect.

[0024] In order to make the above objectives, characteristics and advantages of the present application more apparent and easy to understand, the following will specifically describe embodiments of the present application, and the accompanying drawings will be described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without paying creative labor on the basis of these drawings.

[0026] Figure 1 A flowchart of the artifact repairing method of the ultrasonic image according to an embodiment of the present application is provided;

[0027] Figure 2 A schematic diagram of the artifact-perception attention model according to an embodiment of the present application is provided;

[0028] Figure 3 A schematic diagram of a dynamic condition generator provided for an embodiment of the present application;

[0029] Figure 4 A schematic diagram of a multi-scale pyramid encoder provided for an embodiment of the present application;

[0030] Figure 5 A schematic diagram of an artifact-aware multi-scale fusion network provided for an embodiment of the present application;

[0031] Figure 6 A structural block diagram of an artifact repairing device for an ultrasound image provided for an embodiment of the present application;

[0032] Figure 7 A structural block diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0033] Artifacts refer to false information in images that do not conform to the true anatomical structure, which may be caused by equipment, operation or patient factors. Common artifact types include acoustic shadow, reverberation artifact, refraction artifact and sidelobe artifact, etc. These artifacts are very common in clinical images and have various forms, which are easy to be confused with real structures.

[0034] Among them, acoustic shadow is caused by signal attenuation behind a highly reflective interface (such as bone or calcification, etc.), resulting in a hypoechoic region in the rear area; reverberation artifact is caused by multiple reflections, which appears as repeated linear echoes; refraction artifact is caused by the different propagation speeds of sound waves in different media, resulting in image distortion; and sidelobe artifact is caused by probe sidelobe signals, which appears as false echoes outside the main lobe.

[0035] The main causes of artifacts include equipment factors, operation factors and patient factors. Equipment factors such as insufficient probe performance or improper parameter settings (such as gain, frequency optimization, etc.); operation factors such as improper probe angle, pressure or non-standard scanning technique; patient factors such as obesity, intestinal gas interference or tissue heterogeneity (such as fatty liver, etc.). These factors alone or together cause artifacts to appear frequently in images.

[0036] The harm of artifacts cannot be ignored. They not only increase the difficulty of doctors interpreting images, but also may lead to incorrect diagnostic conclusions. For example: acoustic shadow may mask tumors in the liver, reverberation artifact may be mistaken for cysts or stones, refraction artifact may cause errors in lesion positioning, and sidelobe artifact may be mistaken for a small lesion. These errors may delay treatment or lead to unnecessary further examination, increasing the patient's economic and psychological burden, and also reducing the utilization efficiency of medical resources.

[0037] Based on the above problems, the embodiment of the present application provides an artifact repairing algorithm fusing a dynamic condition generator and an artifact perception attention model. The algorithm can automatically identify artifact regions and repair artifacts, thereby repairing artifacts in an ultrasound image while preserving the anatomical structure of the organs in the ultrasound image, and further improving the accuracy of repairing artifacts in the ultrasound image. The technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0038] Please refer to Figure 1 , Figure 1 A flowchart of an ultrasound image artifact repairing method is provided in the embodiment of the present application. The method can be executed by an electronic device, but is not limited thereto. Figure 7 The possible structure of the electronic device is shown, and specific reference can be made to the description of the electronic device below. The ultrasound image artifact repairing method can specifically include the following steps: Figure 7

[0039] Step S101: input the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact perception attention model to obtain the attention mask output by the artifact perception attention model.

[0040] Step S102: input the attention mask into the dynamic condition generator to obtain the deep feature output by the dynamic condition generator.

[0041] Step S103: input the deep feature into the ultrasound image generator to obtain the target ultrasound image output by the ultrasound image generator.

[0042] Specifically, in the above step S101, the ultrasound image to be repaired refers to an ultrasound image that needs to be repaired. The ultrasound image to be repaired can be an ultrasound image confirmed to have artifacts, or an ultrasound image not confirmed to have artifacts. Both types of ultrasound images to be repaired can be processed using the ultrasound image artifact repairing method provided in the embodiment of the present application.

[0043] In addition, the ultrasound image to be repaired can be an original ultrasound image directly output by an ultrasound device, or an ultrasound image obtained by processing the original ultrasound image. The embodiment of the present application does not make specific limitations thereto, and those skilled in the art can make appropriate adjustments according to the actual situation.

[0044] The feature map corresponding to the ultrasound image to be repaired is obtained by feature extraction on the ultrasound image to be repaired. The embodiment of the present application does not make specific limitations to the specific implementation of the feature extraction, and those skilled in the art can make appropriate adjustments according to the actual situation, for example, a convolutional neural network (CNN), a feature extraction algorithm, etc.​

[0045] The classification result corresponding to the to-be-repaired ultrasound image represents a type of artifact in the to-be-repaired ultrasound image, which can be represented as an artifact classification probability. In this embodiment of the present application, the specific implementation of obtaining the above classification result is not specifically limited, and a person skilled in the art can make appropriate adjustments according to the actual situation, for example: CNN, classification algorithm, etc. can be used.

[0046] The artifact-aware attention module (AA-AM) aims to dynamically adjust the weights of the artifact region and the normal tissue region in the feature map; an attention mask is generated by combining the feature map and the classification result to suppress the feature weight of the artifact region and enhance the feature expression of the normal tissue region. The input of the artifact-aware attention module is the feature map corresponding to the to-be-repaired ultrasound image and the classification result, and the output of the artifact-aware attention module is the attention mask.

[0047] It should be noted that the specific architecture of the above artifact-aware attention module is not specifically limited in this embodiment of the present application, and a person skilled in the art can make appropriate adjustments according to the actual situation. For example, the artifact-aware attention module can include at least one of the following: a multilayer perceptron (MLP), a channel attention module, and a spatial attention module.

[0048] It can be understood that the generated attention mask is used to dynamically adjust the weights of each position in the feature map to suppress the feature expression of the artifact region.

[0049] In the above step S102, the dynamic conditional generator (DCG) aims to adaptively adjust the generator parameters according to the artifact type to realize targeted repair of different artifacts. The input of the dynamic conditional generator is the attention mask, and the output of the dynamic conditional generator is the deep feature.

[0050] In the above step S103, the ultrasound image generator is used to generate a target ultrasound image corresponding to the to-be-repaired ultrasound image based on the above deep feature, that is, the target ultrasound image is an image after repairing the artifact in the to-be-repaired ultrasound image. The input of the ultrasound image generator is the deep feature, and the output of the ultrasound image generator is the target ultrasound image.

[0051] In the above scheme, an artifact repairing algorithm fusing a dynamic condition generative adversarial network and an artifact perception attention model is provided. The algorithm can automatically identify artifact regions and repair artifacts, thereby repairing artifacts in an ultrasound image while preserving the anatomical structure of organs in the ultrasound image, and further improving the accuracy of repairing artifacts in the ultrasound image.

[0052] Further, on the basis of the above embodiment, the artifact perception attention model in step S101 is described in detail. In the embodiment of the present application, please refer to Figure 2 , Figure 2 A schematic diagram of an artifact perception attention model provided in the embodiment of the present application can include a multi-layer perception machine, a channel attention module and a spatial attention module.

[0053] The multi-layer perception machine is used to map the artifact classification probability to a feature space, i.e., to map the artifact classification probability to a high-dimensional feature vector to generate a conditional embedding vector. The channel attention module is used to adjust the channel dimension weight of the feature map, and the spatial attention module is used to adjust the spatial position weight of the feature map.

[0054] The artifact perception attention model provided in the embodiment of the present application can dynamically adjust the attention mask according to the classification result, thereby realizing targeted processing of different artifact types. For example, for acoustic shadow artifacts, the artifact perception attention model generates a stronger attention mask to suppress the feature weight of the acoustic shadow region; for reverberation artifacts, the artifact perception attention model generates a weaker attention mask to avoid excessive suppression of normal tissue regions.

[0055] Based on Figure 2 The artifact perception attention model shown in the figure, the above step S101 can specifically include the following steps:

[0056] Step 1), input the classification result into the multi-layer perception machine to obtain the conditional embedding feature output by the multi-layer perception machine.

[0057] Step 2), fuse the conditional embedding feature with the feature map, and input them into the channel attention module and the spatial attention module respectively, to obtain the channel attention weight map output by the channel attention module, and the spatial attention weight map output by the spatial attention module.

[0058] Step 3), fuse the channel attention weight map and the spatial attention weight map to obtain the attention mask.

[0059] Specifically, in the above step 1), the classification result (Classify Probability) is input into a multi-layer perceptron, which processes the classification result and maps it to a new feature space, and outputs a conditional embedding feature (Conditional Embedding Feature).

[0060] In the above step 2), the conditional embedding feature is first fused with the feature map, and then the fusion result is input into a channel attention module and a spatial attention module respectively to obtain a channel attention weight map output by the channel attention module and a spatial attention weight map output by the spatial attention module.

[0061] For the channel attention module, as an implementation, the input fusion result is first subjected to a maximum pooling (MaxPool) and an average pooling (AvgPool) operation to extract feature information from different angles; then the pooled features are respectively sent to a shared multi-layer perceptron (Shared MLP) to transform the features; finally, the results of the two multi-layer perceptron processing are added to generate a channel attention weight map for emphasizing or suppressing features of different channels.

[0062] For the spatial attention module, as an implementation, the input fusion result is first subjected to a convolution layer (ConvLayer), and then subjected to a maximum pooling and an average pooling operation, and finally the two pooling results are spliced together to generate a spatial attention weight map for emphasizing or suppressing features of different spatial positions.

[0063] In the above scheme, the artifact perception attention model can dynamically adjust the weights of the artifact region and the normal tissue region in the feature map, avoid over-repairing the normal tissue region, and thus can repair the artifacts in the ultrasound image while retaining the organ anatomical structure in the ultrasound image.

[0064] Further, on the basis of the above embodiment, the dynamic condition generator in the above step S102 is described in detail. In the embodiment of the present application, please refer to Figure 3 , Figure 3 A schematic diagram of a dynamic condition generator provided in the embodiment of the present application, which can include a residual dense block (Residual Dense Block, RDB) and a conditional normalization layer (Conditional BatchNorm).

[0065] The residual dense block enhances the feature expression capability through dense connection and residual connection. Each residual dense block includes multiple convolution layers. The output of each convolution layer is spliced with the input of the subsequent convolution layer, thereby enhancing the reuse and expression capability of the features. The conditional normalization layer dynamically adjusts the normalization parameters according to the classification result. Specifically, the classification result is mapped to the scaling parameter and the translation parameter of the normalization layer, thereby dynamically adjusting the feature expression of the generator.

[0066] The dynamic condition generator provided in the embodiments can dynamically adjust the generator parameters according to the classification result, thereby realizing targeted repair of different artifact types. For example, for acoustic shadow artifacts, the dynamic condition generator generates stronger repair features to eliminate acoustic shadows; for reverberation artifacts, the dynamic condition generator generates weaker repair features to avoid over-repairing normal tissue regions.

[0067] Based on the above, Figure 3 The step S102 can include the following steps.

[0068] Step 1), input the attention mask into the residual dense block to obtain initial features output by the residual dense block.

[0069] Step 2), input the initial features into the conditional normalization layer to obtain deep features output by the conditional normalization layer.

[0070] Specifically, in the step 1), the attention mask is input into the residual dense block. The attention mask sequentially passes through multiple convolution layers (Conv) and a rectified linear unit (ReLU) activation function. The convolution layer is used to extract the features of the attention mask, and the ReLU is used to perform nonlinear transformation on the output of the convolution layer to increase the expression capability of the model.

[0071] Figure 3 The arcs in the figure represent the skip connection, which directly connects the output of the previous layer to the subsequent layer. In this way, the model can learn features at different levels without increasing the amount of calculation, which helps to alleviate the gradient vanishing problem and improve the efficiency of feature transmission. After multiple convolution and ReLU operations, the features of different layers are combined through a splicing (Concat) operation, and then a 1x1 transpose convolution is performed to obtain the initial features output by the residual dense block. The above transpose convolution is used to adjust the dimension of the feature map.

[0072] In the step 2), the initial features are input into the conditional normalization layer, which performs normalization processing on the data according to the specific condition to obtain deep features. The use of the above conditional normalization layer helps to speed up the model training process and improve the stability of the model.

[0073] In the above scheme, the different artifact types are repaired by the dynamic condition generator, that is, the dynamic condition generator can dynamically adjust the parameters of the ultrasound image generator according to the classification results of the artifacts, thereby improving the repair effect of the artifacts.

[0074] Further, on the basis of the above embodiment, the specific implementation of obtaining the feature map in the above step S101 is introduced. In the embodiment of the present application, the above step S101 can further include the following steps:

[0075] Step 1), obtaining an original ultrasound image.

[0076] Step 2), inputting the original ultrasound image into a multi-scale pyramid encoder to obtain a feature map output by the multi-scale pyramid encoder.

[0077] Specifically, in the above step 1), the original ultrasound image can be an original ultrasound image directly output by an ultrasound device. It should be noted that the specific implementation of obtaining the above original ultrasound image is not specifically limited in the embodiment of the present application, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the original ultrasound image sent by an external device (such as an ultrasound device) can be received; or the original ultrasound image pre-stored locally or in the cloud can be read, etc.

[0078] In the above step 2), the multi-scale pyramid encoder (Multi-Scale Pyramid Encoder) is responsible for extracting multi-resolution features from the original ultrasound image to capture structural information and artifact features of different scales. The input of the multi-scale pyramid encoder is the original ultrasound image, and the output of the multi-scale pyramid encoder is a feature map.

[0079] By using the above multi-scale pyramid encoder, the output feature map is a multi-scale feature. Wherein, the artifact perception attention model can consider the information of the multi-scale feature map when generating the attention mask, thereby enhancing the processing capability for complex artifacts; the dynamic condition generator can consider the information of the multi-scale feature map when generating the repair image, thereby enhancing the processing capability for complex artifacts.

[0080] In the above scheme, the multi-resolution features in the original ultrasound image are extracted by the multi-scale pyramid encoder, and combined with the cross-scale feature fusion mechanism, thereby the repair capability for micro lesions and complex artifacts can be enhanced, and the accuracy of repairing the artifacts in the ultrasound image is improved.

[0081] Further, on the basis of the above embodiment, the multi-scale pyramid encoder in the above step 2) is introduced in detail. In the embodiment of the present application, please refer to Figure 4, Figure 4 A schematic diagram of a multi-scale pyramid encoder provided for an embodiment of the present application can include three parallel branches, each of which extracts features through a convolution layer and restores low-resolution features to the original resolution through an up-sampling operation for feature fusion.

[0082] As an implementation, the three branches in the multi-scale pyramid encoder respectively process original ultrasound images of original resolution (1x), half resolution (0.5x), and quarter resolution (0.25x). The following takes the multi-scale pyramid encoder shown in FIG. 2 as an example to introduce the three branches respectively. Figure 4

[0083] For the first branch (i.e., the branch processing the original resolution), the original ultrasound image is first subjected to a 3x3 convolution operation through a convolution layer with a convolution kernel size of 3x3, a channel number (C) of 64, and a stride of 1, to extract high-resolution features; then, a batch normalization (BN) operation is performed to accelerate model training and improve stability; finally, a ReLU activation function is used to introduce nonlinearity, and the output channel number is 64.

[0084] For the second branch (i.e., the branch processing the half resolution): the original ultrasound image is first subjected to a 3x3 convolution operation through a convolution layer with a convolution kernel size of 3x3, a channel number of 64, and a stride of 2, to reduce the data resolution; then, it enters a dilated convolution module (Dilated Convolution) with a dilated rate of 2 and a convolution kernel size of 3x3, to expand the receptive field without increasing too many parameters; subsequently, batch normalization and ReLU operations are performed; then, it enters another dilated convolution module with a dilated rate of 2 and a convolution kernel size of 3x3, to further extract features; finally, a 1x1 convolution layer with a channel number of 64 is used to adjust the channel dimension.

[0085] For the third branch (i.e., the branch processing the quarter resolution): the original ultrasound image is first subjected to a 3x3 convolution operation through a convolution layer with a convolution kernel size of 3x3, a channel number of 64, and a stride of 4, to reduce the data resolution; then, batch normalization and ReLU operations are performed; then, bilinear up-sampling (BilinearUpSample) is used to restore the data to the original resolution (Scale=4); finally, a 1x1 convolution layer with a channel number of 64 is used to adjust the channel dimension.

[0086] ​For the above three branches, in order to make full use of multi-scale features, a feature cross-connect mechanism (FCC) can be used to splice the output feature maps of the three branches in the channel dimension, and fuse the multi-scale information through a 1x1 convolution operation to generate the final multi-scale fusion feature map.

[0087] As another implementation, a dilated convolution module can be added in any one or more of the above three branches.

[0088] In the above scheme, spatial convolution is introduced in the multi-scale pyramid encoder, which can expand the receptive field and thus enhance the ability to capture larger artifact regions, thereby improving the accuracy of repairing artifacts in ultrasound images.

[0089] Further, on the basis of the above embodiments, the specific implementation of obtaining the classification result in step S101 is introduced. In the embodiments of the present application, the above step S101 can further include the following steps before step S101:

[0090] Step 1), obtaining an original ultrasound image.

[0091] Step 2), inputting the original ultrasound image into an artifact classification model to obtain a classification result output by the artifact classification model.

[0092] Specifically, in the above step 1), the original ultrasound image can be an original ultrasound image directly output by an ultrasound device. It should be noted that the specific implementation of obtaining the above original ultrasound image is not specifically limited in the embodiments of the present application, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the original ultrasound image sent by an external device (such as an ultrasound device) can be received; or the original ultrasound image pre-stored locally or in the cloud can be read, etc.

[0093] In the above step 2), the artifact classification model can use a residual network (Residual Network, ResNet), a densely connected convolutional network (Densely Connected Convolutional Networks, DenseNet), etc., which is not specifically limited in the embodiments of the present application.

[0094] The classification result can include probability values corresponding to different artifact types, wherein the artifact types include acoustic shadow, reverberation artifact, refraction artifact, and sidelobe artifact, etc.

[0095] Further, on the basis of the above-mentioned embodiments, as an implementation manner, after obtaining the original ultrasound image, a residual network (Residual Network, ResNet), a densely connected convolutional network (Densely Connected Convolutional Networks, DenseNet) or the like can be used to perform binary classification on the original ultrasound image to determine whether there is an artifact in the original ultrasound image. If there is an artifact in the original ultrasound image, the artifact repair method for the ultrasound image provided in the embodiments of the present application is performed; otherwise, the artifact repair method for the ultrasound image provided in the embodiments of the present application is not performed.

[0096] Further, on the basis of the above-mentioned embodiments, the embodiments of the present application provide an artifact-aware multi-scale fusion network, please refer to Figure 5 , Figure 5 FIG. 1 is a schematic diagram of an artifact-aware multi-scale fusion network provided in the embodiments of the present application. The artifact-aware multi-scale fusion network includes an artifact-aware attention model, a dynamic condition generator, and an ultrasound image generator. The following describes a specific implementation of training the artifact-aware multi-scale fusion network. In the embodiments of the present application, the artifact repair method for the ultrasound image can further include the following steps:

[0097] Step 1), obtaining a sample image and label data corresponding to the sample image.

[0098] Step 2), inputting the sample feature map corresponding to the sample image and the sample classification result into the neural network model to obtain a predicted image output by the neural network model, and calculating a neural network loss value according to the predicted image and the sample image.

[0099] Step 3), inputting the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculating a first adversarial loss value of the global discriminator according to the global discrimination result, and inputting the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculating a second adversarial loss value of the local discriminator according to the local discrimination result.

[0100] Step 4), optimizing the neural network model, the global discriminator and the local discriminator according to the neural network loss value, the first adversarial loss value and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network.

[0101] Specifically, in the above-mentioned step 1), the specific implementation of obtaining the sample image and the label data is not specifically limited in the embodiments of the present application, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the sample image and the label data sent by an external device can be received; or the sample image and the label data pre-stored locally or in the cloud can be read, etc.

[0102] In the above step 2), the neural network model has the same architecture as the artifact-aware multi-scale fusion network, and the difference is only that the network parameters of the two are different. By optimizing the parameters in the neural network model, the artifact-aware multi-scale fusion network can be obtained.

[0103] By inputting the sample feature map corresponding to the sample image and the sample classification result into the neural network model, a predicted image output by the neural network model can be obtained, and a neural network loss value can be calculated according to the predicted image and the sample image. The neural network loss value represents the difference between the predicted image and the sample image.

[0104] In the above step 3), multi-discriminator collaborative training can be used to improve the overall quality and local details of the repaired image through the synergistic effect of the global discriminator and the local discriminator.

[0105] The global discriminator adopts a fully convolutional network structure, and the input is the entire predicted image, and the output is the authenticity score of the predicted image. As an implementation, the global discriminator can optimize the overall repair capability of the generator through the adversarial loss (Wasserstein GAN Loss). The local discriminator adopts a convolutional network structure with a local receptive field, and the input is a local image block of the artifact region, and the output is the authenticity score of the local image block. As an implementation, the local discriminator can optimize the local repair capability of the generator through the adversarial loss.

[0106] In the above scheme, the neural network model is trained through multi-discriminator collaborative training, and the overall quality and local details of the repaired image are improved through the synergistic effect of the global discriminator and the local discriminator, thereby improving the repair capability of the artifact-aware multi-scale fusion network trained for artifacts.

[0107] Further, on the basis of the above embodiment, the neural network loss value is equal to the sum of the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value.

[0108] Specifically, the perceptual loss value represents the pixel-level difference between the predicted image and the sample image. As an implementation, the Mean Absolute Error (MAE) loss (i.e., L1 loss) can be used. This perceptual loss value is an auxiliary loss added for each scale of the output feature map, which enhances the expression capability of the multi-scale feature. The multi-scale structural similarity loss value represents the structural similarity between the predicted image and the sample image, and the classification consistency loss value represents the probability of artifacts when the predicted image passes through the artifact classification model.

[0109] As an implementation form, the total loss value of the embodiment of the present application can be calculated by using the following formula:

[0110] L total =αL adv +βL per +γL ssim +λL cls ;

[0111] Wherein, L total is the total loss value, L adv is the adversarial loss value, including the first adversarial loss value and the second adversarial loss value, L per is the perceptual loss value, L ssim is the multi-scale structural similarity loss value, L cls is the classification consistency loss value, and α, β, γ, λ are weight coefficients of each loss term.

[0112] As an implementation form, the adversarial loss value L adb may be calculated by using the Wasserstein GAN Loss, the perceptual loss value L per may be calculated by using the L1Loss function, the multi-scale structural similarity loss value L ssim may be calculated by using the SSIM Loss function, and the classification consistency loss value L cls may be calculated by using the CrossEntropyLoss function.

[0113] In the above scheme, the neural network model is trained by introducing the perceptual loss value, the multi-scale structural similarity loss value and the classification consistency loss value, so as to improve the repair ability of the artifact of the artifact perceptual multi-scale fusion network obtained by training.

[0114] Please refer to Figure 6 , Figure 6A structural block diagram of an artifact repair device for an ultrasonic image is provided for an embodiment of the present application. The artifact repair device 600 for an ultrasonic image includes: a first input module 601, used to input the feature map and classification result corresponding to the ultrasonic image to be repaired into an artifact-perceived attention model to obtain an attention mask output by the artifact-perceived attention model, wherein the feature map is obtained by extracting features from the ultrasonic image to be repaired, and the classification result represents the type of artifact in the ultrasonic image to be repaired; a second input module 602, used to input the attention mask into a dynamic condition generator to obtain a depth feature output by the dynamic condition generator; a third input module 603, used to input the depth feature into an ultrasonic image generator to obtain a target ultrasonic image output by the ultrasonic image generator, wherein the target ultrasonic image is an image after the artifact in the ultrasonic image to be repaired is repaired.

[0115] In the above scheme, an artifact repair algorithm is provided that integrates a dynamic conditional generative adversarial network and an artifact-aware attention model. The algorithm automatically identifies artifact areas and repairs artifacts, thereby repairing artifacts in ultrasound images while preserving the anatomical structure of organs in ultrasound images, thereby improving the accuracy of repairing artifacts in ultrasound images.

[0116] Furthermore, based on the above embodiments, the artifact-aware attention model includes a multi-layer perceptron, a channel attention module and a spatial attention module, and the first input module 601 is specifically used to: input the classification result into the multi-layer perceptron to obtain the conditional embedding features output by the multi-layer perceptron; fuse the conditional embedding features with the feature map, and input them into the channel attention module and the spatial attention module respectively to obtain the channel attention weight map output by the channel attention module, and the spatial attention weight map output by the spatial attention module; fuse the channel attention weight map with the spatial attention weight map to obtain the attention mask.

[0117] In the above scheme, the artifact-aware attention model can dynamically adjust the weights of the artifact area and the normal tissue area in the feature map to avoid over-repairing the normal tissue area, so that the artifacts in the ultrasound image can be repaired while preserving the organ anatomical structure in the ultrasound image.

[0118] Furthermore, based on the above embodiment, the dynamic condition generator includes a residual dense block and a conditional normalization layer, and the second input module 602 is specifically used to: input the attention mask into the residual dense block to obtain the initial features output by the residual dense block; input the initial features into the conditional normalization layer to obtain the depth features output by the conditional normalization layer.

[0119] In the above scheme, the dynamic condition generator is used to realize the targeted repair of different artifact types, that is, the dynamic condition generator can dynamically adjust the parameters of the ultrasound image generator according to the classification result of the artifact, thereby improving the repair effect of the artifact.

[0120] Further, on the basis of the above embodiment, the artifact repair device 600 of the ultrasound image further comprises: a first acquisition module for acquiring an original ultrasound image; a fourth input module for inputting the original ultrasound image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or inputting the original ultrasound image into an artifact classification model to obtain the classification result output by the artifact classification model.

[0121] In the above scheme, the multi-scale pyramid encoder is used to extract multi-resolution features in the original ultrasound image, and a cross-scale feature fusion mechanism is combined, so that the repair capability for micro lesions and complex artifacts can be enhanced, and the accuracy of repairing artifacts in the ultrasound image can be improved.

[0122] Further, on the basis of the above embodiment, the multi-scale pyramid encoder comprises three branches, and at least one branch comprises a hollow convolution module.

[0123] In the above scheme, the spatial convolution is introduced into the multi-scale pyramid encoder, which can expand the receptive field, thereby enhancing the capture ability of larger artifact regions, and improving the accuracy of repairing artifacts in the ultrasound image.

[0124] Further, on the basis of the above-mentioned embodiments, the artifact repairing device 600 of the ultrasound image further comprises: a second acquisition module, configured to acquire a sample image and label data corresponding to the sample image; a fifth input module, configured to input a sample feature map corresponding to the sample image and a sample classification result into a neural network model to obtain a predicted image output by the neural network model, and calculate a neural network loss value according to the predicted image and the sample image; a sixth input module, configured to input the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculate a first adversarial loss value of the global discriminator according to the global discrimination result, and input the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculate a second adversarial loss value of the local discriminator according to the local discrimination result; and an optimization module, configured to optimize the neural network model, the global discriminator and the local discriminator according to the neural network loss value, the first adversarial loss value and the second adversarial loss value to obtain an artifact perception multi-scale fusion network, wherein the artifact perception multi-scale fusion network comprises the artifact perception attention model, the dynamic condition generator and the ultrasound image generator.

[0125] In the above scheme, the neural network model is trained through multi-discriminator cooperative training, and then through the cooperative action of the global discriminator and the local discriminator, the overall quality and local details of the repaired image are improved, thereby improving the artifact repairing capability of the trained artifact perception multi-scale fusion network.

[0126] Further, on the basis of the above-mentioned embodiments, the neural network loss value is equal to the sum of a perception loss value, a multi-scale structural similarity loss value and a classification consistency loss value, wherein the perception loss value represents a pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value represents a structural similarity between the predicted image and the sample image, and the classification consistency loss value represents a probability of artifacts when the predicted image passes through an artifact classification model.

[0127] In the above scheme, the neural network model is trained by introducing the perception loss value, the multi-scale structural similarity loss value and the classification consistency loss value, thereby improving the artifact repairing capability of the trained artifact perception multi-scale fusion network.

[0128] Please refer to Figure 7 , Figure 7A structural block diagram of an electronic device is provided for an embodiment of the present application. The electronic device 700 includes at least one processor 701, at least one communication interface 702, at least one memory 703, and at least one communication bus 704. The communication bus 704 is used to realize direct connection communication among the components. The communication interface 702 is used to communicate signaling or data with other node devices. The memory 703 stores machine-readable instructions executable by the processor 701. When the electronic device 700 is running, the processor 701 communicates with the memory 703 through the communication bus 704, and the machine-readable instructions are executed by the processor 701 when called.

[0129] For example, the processor 701 of the embodiment of the present application reads the computer program from the memory 703 through the communication bus 704 and executes the computer program to implement the following method: input the feature map corresponding to the to-be-repaired ultrasonic image and the classification result into the artifact perception attention model to obtain the attention mask output by the artifact perception attention model, wherein the feature map is obtained by performing feature extraction on the to-be-repaired ultrasonic image, and the classification result represents the type of artifacts in the to-be-repaired ultrasonic image; input the attention mask into the dynamic condition generator to obtain the depth feature output by the dynamic condition generator; input the depth feature into the ultrasonic image generator to obtain the target ultrasonic image output by the ultrasonic image generator, wherein the target ultrasonic image is an image after repairing the artifacts in the to-be-repaired ultrasonic image.

[0130] The processor 701 includes one or more, which can be an integrated circuit chip having a processing capability of signals. The processor 701 described above can be a general-purpose processor, including a central processing unit (CPU), a micro controller unit (MCU), a network processor (NP), or other conventional processors; it can also be a special-purpose processor, including a neural network processor (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Moreover, when the processor 701 is multiple, part of them can be general-purpose processors, and the other part can be special-purpose processors.

[0131] The memory 703 includes one or more, which can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), etc.

[0132] It can be understood that Figure 7 The structure shown is only schematic, and the electronic device 700 can also include more or less components than those shown in the figure, or have a different configuration from that shown in the figure. Figure 7 The structure shown is only schematic, and the electronic device 700 can also include more or less components than those shown in the figure, or have a different configuration from that shown in the figure. Figure 7 The structure shown is only schematic, and the electronic device 700 can also include more or less components than those shown in the figure, or have a different configuration from that shown in the figure. Figure 7The components shown in the figure can be implemented in hardware, software, or a combination thereof. In the embodiments of the present application, the electronic device 700 can be, but is not limited to, a desktop computer, a notebook computer, a smart phone, a smart wearable device, a vehicle-mounted device, and the like. In addition, the electronic device 700 can also be a virtual machine or the like. In addition, the electronic device 700 is not necessarily a single device, but can also be a combination of multiple devices, such as a server cluster, and the like.

[0133] The embodiments of the present application also provide a computer program product, comprising a computer program stored on a computer readable storage medium, the computer program comprising computer program instructions, when the computer program instructions are executed by a computer, the computer can execute the steps of the artifact repairing method of the ultrasound image in the above-mentioned embodiments, for example, comprising: step S101: input the feature map corresponding to the to-be-repaired ultrasound image and the classification result into the artifact perception attention model to obtain the attention mask output by the artifact perception attention model. Step S102: input the attention mask into the dynamic condition generator to obtain the depth feature output by the dynamic condition generator. Step S103: input the depth feature into the ultrasound image generator to obtain the target ultrasound image output by the ultrasound image generator.

[0134] The embodiments of the present application also provide a computer readable storage medium, which stores computer program instructions, when the computer program instructions are executed by a computer, the computer executes the artifact repairing method of the ultrasound image in the foregoing method embodiments.

[0135] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There can be another division during actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0136] In addition, the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0137] Further, each functional module in the various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0138] It should be noted that if the functions are realized in the form of software functional modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0139] In this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

[0140] The above only describes the embodiments of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for repairing artifacts of an ultrasound image, characterized in that: include: Inputting a feature map corresponding to the ultrasound image to be repaired and a classification result into an artifact-aware attention model to obtain an attention mask output by the artifact-aware attention model, wherein the feature map is obtained by performing feature extraction on the ultrasound image to be repaired, and the classification result represents the type of artifact in the ultrasound image to be repaired; Inputting the attention mask into a dynamic condition generator to obtain a deep feature output by the dynamic condition generator; Inputting the depth feature into an ultrasonic image generator to obtain a target ultrasonic image output by the ultrasonic image generator, wherein the target ultrasonic image is an image after repairing artifacts in the ultrasonic image to be repaired; The artifact-aware attention model includes a multi-layer perceptron, a channel attention module, and a spatial attention module. Inputting the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model includes: Inputting the classification result into the multi-layer perceptron to obtain the conditional embedding features output by the multi-layer perceptron; The conditional embedding feature is fused with the feature map, and the results are input into the channel attention module and the spatial attention module respectively to obtain the channel attention weight map output by the channel attention module and the spatial attention weight map output by the spatial attention module; Fusing the channel attention weight map with the spatial attention weight map to obtain the attention mask; The dynamic condition generator includes a residual dense block and a conditional normalization layer, and the inputting of the attention mask into the dynamic condition generator to obtain the deep features output by the dynamic condition generator includes: Inputting the attention mask into the residual dense block to obtain the initial features output by the residual dense block; The initial features are input into the conditional normalization layer to obtain the depth features output by the conditional normalization layer.

2. The method for repairing ultrasonic image artifacts according to claim 1, wherein: Before inputting the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model, the method further includes: Acquire original ultrasound images; Inputting the original ultrasound image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or, The original ultrasound image is input into an artifact classification model to obtain the classification result output by the artifact classification model.

3. The method for repairing ultrasonic image artifacts according to claim 2, wherein: The multi-scale pyramid encoder includes three branches, at least one of which includes a dilated convolution module.

4. The method for repairing ultrasonic image artifacts according to any one of claims 1 to 3, characterized in that: Also includes: Acquire a sample image and label data corresponding to the sample image; Inputting the sample feature map corresponding to the sample image and the sample classification result into the neural network model to obtain a predicted image output by the neural network model, and calculating a neural network loss value based on the predicted image and the sample image; Inputting the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculating a first adversarial loss value of the global discriminator based on the global discrimination result; and inputting the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculating a second adversarial loss value of the local discriminator based on the local discrimination result; The neural network model, the global discriminator and the local discriminator are optimized according to the neural network loss value, the first adversarial loss value and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network, wherein the artifact-aware multi-scale fusion network includes the artifact-aware attention model, the dynamic condition generator and the ultrasound image generator.

5. The method for repairing ultrasonic image artifacts according to claim 4, characterized in that: The neural network loss value is equal to the sum of the perceptual loss value, the multi-scale structural similarity loss value and the classification consistency loss value, wherein the perceptual loss value represents the pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value represents the structural similarity between the predicted image and the sample image, and the classification consistency loss value represents the probability of artifacts when the predicted image passes through the artifact classification model.

6. A computer program product, characterized in that The method comprises computer program instructions, which, when read and executed by a processor, executes the method for repairing artifacts of an ultrasound image according to any one of claims 1 to 5.

7. An electronic device, characterized in that: include: processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions that can be executed by the processor, and the processor calls the computer program instructions to execute the method for repairing artifacts of an ultrasound image according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when executed by a computer, enable the computer to perform the method for repairing ultrasound image artifacts according to any one of claims 1 to 5.

Images