Ultrasound image artifact restoration method, program product, electronic equipment and storage medium

Abstract

The invention provides an artifact restoration method for an ultrasonic image, a program product, electronic equipment and a storage medium, and the method comprises the steps: inputting a feature map corresponding to a to-be-restored ultrasonic image and a classification result into an artifact perception attention model, and obtaining an attention mask outputted by the artifact perception attention model, the feature map is obtained by performing feature extraction on the to-be-repaired ultrasonic image, and the classification result represents the type of artifacts in the to-be-repaired ultrasonic image; inputting the attention mask into a dynamic condition generator to obtain a depth feature output by the dynamic condition generator; the depth features are input into an ultrasonic image generator, a target ultrasonic image output by the ultrasonic image generator is obtained, and the target ultrasonic image is an image obtained after artifacts in the ultrasonic image to be repaired are repaired. According to the scheme, the artifact repairing algorithm fusing the dynamic condition generative adversarial network and the artifact perception attention model is provided, and the accuracy of repairing the artifacts in the ultrasonic image is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology. Specifically, it relates to an artifact repair method, program product, electronic device, and storage medium for ultrasonic images. Background Art

[0002] Due to its advantages such as non-invasiveness, real-time, and portability, ultrasonic imaging is increasingly widely used in clinical diagnosis. However, the artifact problem in ultrasonic images has become increasingly prominent, seriously affecting the image quality and diagnostic accuracy. For example, in liver ultrasound, acoustic shadows may mask tumors, and reverberation artifacts may be misinterpreted as cysts. These artifacts not only interfere with doctors' identification of lesions but may also lead to misdiagnosis or missed diagnosis, thus affecting clinical decisions.

[0003] With the increasing demand for ultrasonic examinations, the artifact problem has become a key factor restricting diagnostic accuracy and efficiency. Therefore, it is necessary to remove or repair the artifacts in ultrasonic images. However, the accuracy of repairing artifacts in ultrasonic images in the prior art is relatively low. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide an artifact repair method, program product, electronic device, and storage medium for ultrasonic images to solve the technical problem of relatively low accuracy in repairing artifacts in ultrasonic images in the prior art.

[0005] In a first aspect, the embodiments of this application provide an artifact repair method for ultrasonic images, including: inputting the feature map corresponding to the ultrasonic image to be repaired and the classification result into an artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model, where the feature map is obtained by extracting features from the ultrasonic image to be repaired, and the classification result characterizes the type of artifact in the ultrasonic image to be repaired; inputting the attention mask into a dynamic conditional generator to obtain the depth feature output by the dynamic conditional generator; inputting the depth feature into an ultrasonic image generator to obtain the target ultrasonic image output by the ultrasonic image generator, where the target ultrasonic image is the image after repairing the artifacts in the ultrasonic image to be repaired.

[0006] In the above solution, an artifact repair algorithm that combines a dynamic conditional generative adversarial network and an artifact-aware attention model is provided. Through this algorithm, the artifact area can be automatically identified and the artifacts can be repaired, so that the artifacts in the ultrasonic image can be repaired while retaining the organ anatomical structure in the ultrasonic image, thereby improving the accuracy of repairing the artifacts in the ultrasonic image.

[0007] In an alternative embodiment, the artifact-aware attention model includes a multi-layer perceptron, a channel attention module, and a spatial attention module. The step of inputting the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model includes: inputting the classification result into the multi-layer perceptron to obtain the conditional embedding feature output by the multi-layer perceptron; fusing the conditional embedding feature with the feature map and inputting them into the channel attention module and the spatial attention module respectively to obtain the channel attention weight map output by the channel attention module and the spatial attention weight map output by the spatial attention module; and fusing the channel attention weight map with the spatial attention weight map to obtain the attention mask. In the above solution, the artifact-aware attention model can dynamically adjust the weights of the artifact regions and normal tissue regions in the feature map, avoiding over-repairing the normal tissue regions, so that the artifacts in the ultrasound image can be repaired while retaining the organ anatomical structure in the ultrasound image.

[0008] In an alternative embodiment, the dynamic condition generator includes a residual dense block and a conditional normalization layer. The step of inputting the attention mask into the dynamic condition generator to obtain the depth feature output by the dynamic condition generator includes: inputting the attention mask into the residual dense block to obtain the initial feature output by the residual dense block; and inputting the initial feature into the conditional normalization layer to obtain the depth feature output by the conditional normalization layer. In the above solution, the dynamic condition generator realizes targeted repair for different artifact types, that is, the above dynamic condition generator can dynamically adjust the parameters of the ultrasound image generator according to the classification result of the artifacts, thereby improving the repair effect on the artifacts.

[0009] In an alternative embodiment, before inputting the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model, the method further includes: acquiring the original ultrasound image; inputting the original ultrasound image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or inputting the original ultrasound image into an artifact classification model to obtain the classification result output by the artifact classification model. In the above solution, the multi-scale pyramid encoder extracts multi-resolution features from the original ultrasound image and combines a cross-scale feature fusion mechanism, so as to enhance the repair ability for micro-lesions and complex artifacts, and further improve the accuracy of repairing the artifacts in the ultrasound image.

[0010] In an alternative embodiment, the multi-scale pyramid encoder includes three branches, and at least one branch includes a dilated convolution module. In the above solution, introducing spatial convolution in the multi-scale pyramid encoder can expand the receptive field, thereby enhancing the ability to capture larger artifact regions, and further improving the accuracy of repairing artifacts in ultrasonic images.

[0011] In an alternative embodiment, the method for repairing artifacts in ultrasonic images further includes: obtaining a sample image and label data corresponding to the sample image; inputting the sample feature map and sample classification result corresponding to the sample image into a neural network model to obtain a predicted image output by the neural network model, and calculating a neural network loss value according to the predicted image and the sample image; inputting the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculating a first adversarial loss value of the global discriminator according to the global discrimination result, and inputting the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculating a second adversarial loss value of the local discriminator according to the local discrimination result; optimizing the neural network model, the global discriminator, and the local discriminator according to the neural network loss value, the first adversarial loss value, and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network, where the artifact-aware multi-scale fusion network includes the artifact-aware attention model, the dynamic conditional generator, and the ultrasonic image generator. In the above solution, the neural network model is trained through collaborative training of multiple discriminators, and then through the collaborative action of the global discriminator and the local discriminator, the overall quality and local details of the repaired image are improved, thereby enhancing the artifact repair ability of the trained artifact-aware multi-scale fusion network.

[0012] In an alternative embodiment, the neural network loss value is equal to the sum of the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value, where the perceptual loss value represents the pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value represents the structural similarity between the predicted image and the sample image, and the classification consistency loss value represents the probability of artifacts when the predicted image passes through the artifact classification model. In the above solution, the neural network model is trained by introducing the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value, thereby improving the artifact repair ability of the trained artifact-aware multi-scale fusion network.

[0013] Second aspect, an artifact repair device for ultrasonic images provided by an embodiment of the present application includes: a first input module, configured to input a feature map corresponding to an ultrasonic image to be repaired and a classification result into an artifact perception attention model, and obtain an attention mask output by the artifact perception attention model, where the feature map is obtained by performing feature extraction on the ultrasonic image to be repaired, and the classification result characterizes the type of artifact in the ultrasonic image to be repaired; a second input module, configured to input the attention mask into a dynamic condition generator, and obtain a depth feature output by the dynamic condition generator; a third input module, configured to input the depth feature into an ultrasonic image generator, and obtain a target ultrasonic image output by the ultrasonic image generator, where the target ultrasonic image is an image obtained by repairing the artifact in the ultrasonic image to be repaired.

[0014] In the above solution, an artifact repair algorithm that fuses a dynamic condition generative adversarial network and an artifact perception attention model is provided. Through this algorithm, the artifact area can be automatically identified and repaired, so that the artifact in the ultrasonic image can be repaired while retaining the organ anatomical structure in the ultrasonic image, thereby improving the accuracy of repairing the artifact in the ultrasonic image.

[0015] In an optional implementation manner, the artifact perception attention model includes a multi-layer perceptron, a channel attention module, and a spatial attention module. The first input module is specifically configured to: input the classification result into the multi-layer perceptron, and obtain a conditional embedding feature output by the multi-layer perceptron; fuse the conditional embedding feature with the feature map, and input them into the channel attention module and the spatial attention module respectively, and obtain a channel attention weight map output by the channel attention module, and a spatial attention weight map output by the spatial attention module; fuse the channel attention weight map and the spatial attention weight map to obtain the attention mask. In the above solution, the artifact perception attention model can dynamically adjust the weights of the artifact area and the normal tissue area in the feature map, avoiding over-repairing the normal tissue area, so that the artifact in the ultrasonic image can be repaired while retaining the organ anatomical structure in the ultrasonic image.

[0016] In an optional implementation manner, the dynamic condition generator includes a residual dense block and a conditional normalization layer. The second input module is specifically configured to: input the attention mask into the residual dense block, and obtain an initial feature output by the residual dense block; input the initial feature into the conditional normalization layer, and obtain the depth feature output by the conditional normalization layer. In the above solution, targeted repair of different artifact types is realized through the dynamic condition generator, that is, the above dynamic condition generator can dynamically adjust the parameters of the ultrasonic image generator according to the classification result of the artifact, thereby improving the repair effect of the artifact.

[0017] In an alternative embodiment, the artifact repair device for ultrasonic images further includes: a first acquisition module configured to acquire an original ultrasonic image; a fourth input module configured to input the original ultrasonic image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or, input the original ultrasonic image into an artifact classification model to obtain the classification result output by the artifact classification model. In the above solution, multi-resolution features in the original ultrasonic image are extracted by the multi-scale pyramid encoder, and combined with a cross-scale feature fusion mechanism, thereby enhancing the ability to repair micro-lesions and complex artifacts, and further improving the accuracy of repairing artifacts in ultrasonic images.

[0018] In an alternative embodiment, the multi-scale pyramid encoder includes three branches, and at least one branch includes an atrous convolution module. In the above solution, introducing spatial convolution in the multi-scale pyramid encoder can expand the receptive field, thereby enhancing the ability to capture larger artifact regions, and further improving the accuracy of repairing artifacts in ultrasonic images.

[0019] In an alternative embodiment, the artifact repair device for ultrasonic images further includes: a second acquisition module configured to acquire a sample image and label data corresponding to the sample image; a fifth input module configured to input the sample feature map and sample classification result corresponding to the sample image into a neural network model to obtain a predicted image output by the neural network model, and calculate a neural network loss value according to the predicted image and the sample image; a sixth input module configured to input the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculate a first adversarial loss value of the global discriminator according to the global discrimination result, and input the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculate a second adversarial loss value of the local discriminator according to the local discrimination result; an optimization module configured to optimize the neural network model, the global discriminator, and the local discriminator according to the neural network loss value, the first adversarial loss value, and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network, where the artifact-aware multi-scale fusion network includes the artifact-aware attention model, the dynamic conditional generator, and the ultrasonic image generator. In the above solution, the neural network model is trained through collaborative training of multiple discriminators, and then through the collaborative effect of the global discriminator and the local discriminator, the overall quality and local details of the repaired image are improved, thereby improving the artifact repair ability of the trained artifact-aware multi-scale fusion network.

[0020] In an alternative embodiment, the neural network loss value is equal to the sum of the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value. Among them, the perceptual loss value characterizes the pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value characterizes the structural similarity between the predicted image and the sample image, and the classification consistency loss value characterizes the probability of artifacts when the predicted image passes through the artifact classification model. In the above solution, by introducing the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value to train the neural network model, the artifact repair ability of the trained artifact-aware multi-scale fusion network is improved.

[0021] In a third aspect, an embodiment of the present application provides a computer program product, including computer program instructions. When the computer program instructions are read and run by a processor, they execute the artifact repair method of the ultrasonic image as described in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus; the processor and the memory communicate with each other through the bus; the memory stores computer program instructions executable by the processor, and the processor can execute the artifact repair method of the ultrasonic image as described in the first aspect by calling the computer program instructions.

[0023] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores computer program instructions. When the computer program instructions are run by a computer, the computer is enabled to execute the artifact repair method of the ultrasonic image as described in the first aspect.

[0024] To make the above objects, features, and advantages of the present application more obvious and understandable, specific embodiments of the present application are hereinafter given, and detailed descriptions are made in conjunction with the accompanying drawings as follows. Description of the Drawings

[0025] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a flowchart of an artifact repair method for an ultrasonic image provided by an embodiment of the present application;

[0027] Figure 2 It is a schematic diagram of an artifact-aware attention model provided by an embodiment of the present application;

[0028] Figure 3 Schematic diagram of a dynamic condition generator provided by an embodiment of the present application;

[0029] Figure 4 Schematic diagram of a multi-scale pyramid encoder provided by an embodiment of the present application;

[0030] Figure 5 Schematic diagram of an artifact-aware multi-scale fusion network provided by an embodiment of the present application;

[0031] Figure 6 Block diagram of the structure of an ultrasound image artifact repair device provided by an embodiment of the present application;

[0032] Figure 7 Block diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0033] Artifacts refer to false information in an image that does not conform to the true anatomical structure and may be caused by equipment, operation, or patient factors. Common types of artifacts include acoustic shadows, reverberation artifacts, refraction artifacts, and side lobe artifacts, etc. These artifacts are extremely common in clinical images and have various manifestations, making it easy to confuse with the true structure.

[0034] Among them, an acoustic shadow is caused by signal attenuation behind a high-reflection interface (such as: bone or calcification, etc.), resulting in an anechoic area in the posterior region; a reverberation artifact is caused by multiple reflections and appears as repeated linear echoes; a refraction artifact is caused by the different propagation speeds of sound waves in different media, resulting in image distortion; a side lobe artifact is caused by the side lobe signal of the probe and appears as false echoes outside the main lobe.

[0035] The causes of artifacts mainly include equipment factors, operation factors, and patient factors. Equipment factors such as insufficient probe performance or improper parameter settings (such as: gain, frequency not optimized, etc.); operation factors such as improper probe angle, pressure, or non-standard scanning techniques; patient factors such as obesity, intestinal gas interference, or tissue heterogeneity (such as: fatty liver, etc.). These factors act alone or together, resulting in the frequent appearance of artifacts in the image.

[0036] The harm of artifacts cannot be ignored. They not only increase the difficulty for doctors to interpret images but also may lead to incorrect diagnostic conclusions. For example: an acoustic shadow may cover a tumor in the liver, a reverberation artifact may be mistaken for a cyst or a stone, a refraction artifact may cause incorrect lesion localization, and a side lobe artifact may be mistaken for a minor lesion. These errors may delay treatment or lead to unnecessary further examinations, increasing the economic and psychological burden on patients, and at the same time reducing the utilization efficiency of medical resources.

[0037] Based on the above problems, an artifact repair algorithm that combines a dynamic conditional generative adversarial network and an artifact-aware attention model is provided in an embodiment of the present application. Through this algorithm, the artifact area is automatically identified and the artifacts are repaired, so that the artifacts in the ultrasound image can be repaired while preserving the organ anatomical structure in the ultrasound image, thereby improving the accuracy of repairing the artifacts in the ultrasound image. Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0038] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for repairing artifacts in an ultrasound image provided in an embodiment of the present application. This method can be, but is not limited to, executed by an electronic device. Figure 7 It shows the possible structure of the electronic device. For details, reference can be made to the subsequent description of Figure 7 . Among them, the method for repairing artifacts in the ultrasound image specifically may include the following steps:

[0039] Step S101: Input the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model.

[0040] Step S102: Input the attention mask into the dynamic conditional generator to obtain the depth feature output by the dynamic conditional generator.

[0041] Step S103: Input the depth feature into the ultrasound image generator to obtain the target ultrasound image output by the ultrasound image generator.

[0042] Specifically, in the above step S101, the ultrasound image to be repaired refers to the ultrasound image that needs to be repaired for artifacts. Among them, the ultrasound image to be repaired can be either an ultrasound image confirmed to have artifacts or an ultrasound image whose presence of artifacts has not been confirmed. Both of the above two types of ultrasound images to be repaired can be processed by the method for repairing artifacts in the ultrasound image provided in an embodiment of the present application.

[0043] In addition, the ultrasound image to be repaired can be the original ultrasound image directly output by the ultrasound device or the ultrasound image obtained after processing the original ultrasound image. The embodiments of the present application do not make specific limitations in this regard, and those skilled in the art can make appropriate adjustments according to the actual situation.

[0044] The feature map corresponding to the ultrasound image to be repaired is obtained by performing feature extraction on the ultrasound image to be repaired. Among them, the embodiments of the present application do not make specific limitations on the specific implementation manner of the above feature extraction, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, a convolutional neural network (CNN), a feature extraction algorithm, etc. can be used.

[0045] The classification result corresponding to the ultrasound image to be repaired represents the type of artifacts in the ultrasound image to be repaired, which can be expressed as the artifact classification probability. Among them, the embodiments of the present application do not specifically limit the specific implementation manner of obtaining the above classification result, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, CNN, classification algorithms, etc. can be used.

[0046] The Artifact-Aware Attention Module (AA-AM) aims to dynamically adjust the weights of the artifact regions and normal tissue regions in the feature map; by combining the feature map and the classification result, an attention mask is generated to suppress the feature weights of the artifact regions while enhancing the feature expression of the normal tissue regions. Among them, the input of the artifact-aware attention model is the feature map corresponding to the ultrasound image to be repaired and the classification result, and the output of the artifact-aware attention model is the attention mask.

[0047] It should be noted that the embodiments of the present application do not specifically limit the specific architecture of the above artifact-aware attention model, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the artifact-aware attention model can include at least one of the following: Multilayer Perceptron (MLP), channel attention module, and spatial attention module.

[0048] It can be understood that the generated attention mask is used to dynamically adjust the weights of each position in the feature map to suppress the feature expression of the artifact regions.

[0049] In the above step S102, the Dynamic Conditional Generator (DCG) aims to adaptively adjust the generator parameters according to the type of artifacts to achieve targeted repair of different artifacts. Among them, the input of the dynamic conditional generator is the attention mask, and the output of the dynamic conditional generator is the depth feature.

[0050] In the above step S103, the ultrasound image generator is used to generate a target ultrasound image corresponding to the ultrasound image to be repaired based on the above depth feature, that is, the above target ultrasound image is the image after repairing the artifacts in the ultrasound image to be repaired. Among them, the input of the ultrasound image generator is the depth feature, and the output of the ultrasound image generator is the target ultrasound image.

[0051] In the above solution, an artifact repair algorithm that combines a dynamic conditional generative adversarial network and an artifact-aware attention model is provided. Through this algorithm, the artifact area is automatically identified and the artifacts are repaired, so that the artifacts in the ultrasound image can be repaired while preserving the organ anatomical structure in the ultrasound image, thereby improving the accuracy of repairing the artifacts in the ultrasound image.

[0052] Further, on the basis of the above embodiment, the artifact-aware attention model in the above step S101 is introduced in detail. In the embodiment of the present application, please refer to Figure 2 , Figure 2 which is a schematic diagram of an artifact-aware attention model provided by the embodiment of the present application. The artifact-aware attention model may include a multi-layer perceptron, a channel attention module, and a spatial attention module.

[0053] Among them, the multi-layer perceptron is used to map the artifact classification probability to the feature space, that is, to map the artifact classification probability into a high-dimensional feature vector to generate a conditional embedding vector. The channel attention module is used to adjust the channel dimension weights of the feature map, and the spatial attention module is used to adjust the spatial position weights of the feature map.

[0054] The artifact-aware attention model provided by the embodiment of the present application can dynamically adjust the attention mask according to the classification result, so as to achieve targeted processing of different artifact types. For example: for shadow artifacts, the artifact-aware attention model will generate a strong attention mask to suppress the feature weights in the shadow area; for reverberation artifacts, the artifact-aware attention model will generate a weak attention mask to avoid over-suppressing the normal tissue area.

[0055] Based on Figure 2 the shown artifact-aware attention model, the above step S101 may specifically include the following steps:

[0056] Step 1), input the classification result into the multi-layer perceptron to obtain the conditional embedding feature output by the multi-layer perceptron.

[0057] Step 2), fuse the conditional embedding feature with the feature map, and input them into the channel attention module and the spatial attention module respectively to obtain the channel attention weight map output by the channel attention module and the spatial attention weight map output by the spatial attention module.

[0058] Step 3), fuse the channel attention weight map and the spatial attention weight map to obtain an attention mask.

[0059] Specifically, in the above step 1), the classification result (Classify Probability) is input into a multi-layer perceptron. The multi-layer perceptron processes the classification result, maps it to a new feature space, and outputs the conditional embedding feature.

[0060] In the above step 2), first, the above conditional embedding feature is fused with the feature map. Then, the fused result is respectively input into the channel attention module and the spatial attention module to obtain the channel attention weight map output by the channel attention module and the spatial attention weight map output by the spatial attention module.

[0061] For the channel attention module, as an implementation, the input fused result first undergoes max pooling (MaxPool) and average pooling (AvgPool) operations respectively to extract feature information from different perspectives. Then, the pooled features are respectively fed into a shared multi-layer perceptron (Shared MLP) to transform the features. Finally, the results of the two processed by the multi-layer perceptron are added together to generate the channel attention weight map for emphasizing or suppressing the features of different channels.

[0062] For the spatial attention module, as an implementation, the input fused result first passes through a convolutional layer (ConvLayer), then undergoes max pooling and average pooling operations, and finally, the two pooling results are concatenated together to generate the spatial attention weight map for emphasizing or suppressing the features of different spatial positions.

[0063] In the above solution, the artifact-aware attention model can dynamically adjust the weights of the artifact region and the normal tissue region in the feature map, avoiding over-repairing the normal tissue region. Thus, it can repair the artifacts in the ultrasound image while preserving the organ anatomical structure in the ultrasound image.

[0064] Further, based on the above embodiments, the dynamic condition generator in the above step S102 is introduced in detail. In the embodiments of the present application, please refer to Figure 3 , Figure 3 which is a schematic diagram of a dynamic condition generator provided by the embodiments of the present application. The dynamic condition generator may include a residual dense block (Residual Dense Block, RDB) and a conditional batch normalization layer (Conditional BatchNorm).

[0065] Among them, the residual dense block enhances the feature expression ability through dense connection and residual connection. Each residual dense block contains multiple convolutional layers, and the output of each convolutional layer will be concatenated with the input of subsequent convolutional layers, thereby enhancing the reuse and expression ability of features. The conditional normalization layer dynamically adjusts the normalization parameters according to the classification result. The specific implementation is to map the classification result to the scaling parameter and translation parameter of the normalization layer, thereby realizing the dynamic adjustment of the feature expression of the generator.

[0066] The dynamic conditional generator provided by the embodiment of the present application can dynamically adjust the generator parameters according to the classification result, thereby realizing targeted repair of different types of artifacts. For example, for shadow artifacts, the dynamic conditional generator will generate stronger repair features to eliminate the shadows; for reverberation artifacts, the dynamic conditional generator will generate weaker repair features to avoid over-repairing the normal tissue area.

[0067] Based on Figure 3 the dynamic conditional generator shown, the above step S102 may specifically include the following steps:

[0068] Step 1), input the attention mask into the residual dense block to obtain the initial features output by the residual dense block.

[0069] Step 2), input the initial features into the conditional normalization layer to obtain the deep features output by the conditional normalization layer.

[0070] Specifically, in the above step 1), when inputting the attention mask into the residual dense block, the attention mask sequentially passes through multiple convolutional layers (Conv) and the rectified linear unit (ReLU) activation function. Among them, the convolutional layer is used to extract the features of the attention mask, and ReLU is used to perform non-linear transformation on the output of the convolutional layer to increase the expression ability of the model.

[0071] Figure 3 The arcs in represent skip connections, which directly connect the output of the previous layer to the subsequent layer. In this way, the model can learn features at different levels without increasing too much computational complexity, which helps to alleviate the problem of gradient disappearance and improve the efficiency of feature transmission. After multiple convolutional and ReLU operations, the features of different layers are combined through a concatenation (Concat) operation, and then passed through a 1×1 transposed convolution to obtain the initial features output by the residual dense block. Among them, the above transposed convolution is used to adjust the dimension of the feature map.

[0072] In the above step 2), when inputting the initial features into the conditional normalization layer, this layer will perform normalization processing on the data according to specific conditions to obtain deep features. Among them, adopting the above conditional normalization layer helps to accelerate the model training process and improve the stability of the model.

[0073] In the above solution, targeted repair of different types of artifacts is achieved through a dynamic condition generator, that is, the above dynamic condition generator can dynamically adjust the parameters of the ultrasonic image generator according to the classification results of the artifacts, thereby improving the repair effect of the artifacts.

[0074] Further, on the basis of the above embodiments, the specific implementation manner of obtaining the feature map in the above step S101 is introduced. In the embodiments of the present application, the following steps may further be included before the above step S101:

[0075] Step 1), obtain the original ultrasonic image.

[0076] Step 2), input the original ultrasonic image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder.

[0077] Specifically, in the above step 1), the original ultrasonic image may be the original ultrasonic image directly output by an ultrasonic device. It should be noted that the embodiments of the present application do not specifically limit the specific implementation manner of obtaining the above original ultrasonic image, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the original ultrasonic image sent by an external device (such as an ultrasonic device, etc.) may be received; or, the original ultrasonic image pre-stored locally or in the cloud may be read, etc.

[0078] In the above step 2), the multi-scale pyramid encoder is responsible for extracting multi-resolution features from the original ultrasonic image to capture structural information and artifact features at different scales. Among them, the input of the multi-scale pyramid encoder is the original ultrasonic image, and the output of the multi-scale pyramid encoder is the feature map.

[0079] By adopting the above multi-scale pyramid encoder, the output feature map is a multi-scale feature. Among them, when the artifact-aware attention model generates an attention mask, it can comprehensively consider the information of the multi-scale feature map, thereby enhancing the processing ability for complex artifacts; when the dynamic condition generator generates a repaired image, it can comprehensively consider the information of the multi-scale feature map, thereby enhancing the processing ability for complex artifacts.

[0080] In the above solution, multi-resolution features in the original ultrasonic image are extracted through a multi-scale pyramid encoder, and combined with a cross-scale feature fusion mechanism, so that the repair ability for micro-lesions and complex artifacts can be enhanced, and further the accuracy of repairing artifacts in ultrasonic images can be improved.

[0081] Further, on the basis of the above embodiments, the multi-scale pyramid encoder in the above step 2) is introduced in detail. In the embodiments of the present application, please refer to Figure 4, Figure 4 It is a schematic diagram of a multi-scale pyramid encoder provided by an embodiment of the present application. The multi-scale pyramid encoder may include three parallel branches. Each branch extracts features through a convolutional layer and restores low-resolution features to the original resolution through an upsampling operation for feature fusion.

[0082] As an implementation manner, the three branches in the multi-scale pyramid encoder respectively process the original ultrasound images with original resolution (1×), half resolution (0.5×), and quarter resolution (0.25×). Taking the Figure 4 shown multi-scale pyramid encoder as an example, the three branches will be introduced respectively.

[0083] For the first branch (i.e., the branch processing the original resolution), the original ultrasound image first passes through a convolutional layer with a convolutional kernel size of 3×3, the number of channels (C) of 64, and a stride of 1 to perform a 3×3 convolution operation on the original ultrasound image to extract high-resolution features; then a batch normalization (Batch Normalization, BN) operation is performed to accelerate model training and improve stability; finally, it passes through the ReLU activation function to introduce nonlinearity, and the number of output channels is 64.

[0084] For the second branch (i.e., the branch processing the half resolution): The original ultrasound image first passes through a convolutional layer with a convolutional kernel size of 3×3, the number of channels of 64, and a stride of 2 to reduce the data resolution; then it enters a dilated convolution module with a dilation rate (Rate) of 2 and a convolutional kernel size of 3×3 to expand the receptive field without adding too many parameters; then batch normalization and ReLU operations are performed; then it enters a dilated convolution module with a dilation rate of 2 and a convolutional kernel size of 3×3 to further extract features; finally, it passes through a convolutional layer with a convolutional kernel size of 1×1 and the number of channels of 64 to adjust the channel dimension.

[0085] For the third branch (i.e., the branch processing the quarter resolution): The original ultrasound image first passes through a convolutional layer with a convolutional kernel size of 3×3, the number of channels of 64, and a stride of 4 to reduce the data resolution; then batch normalization and ReLU operations are performed; then the data is restored to the original resolution (Scale = 4) through bilinear upsampling (BilinearUpSample); finally, it passes through a convolutional layer with a convolutional kernel size of 1×1 and the number of channels of 64 to adjust the channel dimension.

[0086] For the above three branches, in order to make full use of multi-scale features, a Feature Cross-Connect (FCC) mechanism can be adopted to splice the output feature maps of the three branches in the channel dimension, and fuse the multi-scale information through 1×1 convolution operations to generate the final multi-scale fusion feature map.

[0087] As another implementation, atrous convolution modules can be added to any one or more of the above three branches.

[0088] In the above solution, introducing spatial convolution in the multi-scale pyramid encoder can expand the receptive field, thereby enhancing the ability to capture larger artifact regions, and further improving the accuracy of repairing artifacts in ultrasonic images.

[0089] Furthermore, based on the above embodiments, the specific implementation manner of obtaining the classification result in the above step S101 is introduced. In the embodiments of the present application, the following steps may further be included before the above step S101:

[0090] Step 1), obtain the original ultrasonic image.

[0091] Step 2), input the original ultrasonic image into the artifact classification model to obtain the classification result output by the artifact classification model.

[0092] Specifically, in the above step 1), the original ultrasonic image may be the original ultrasonic image directly output by the ultrasonic device. It should be noted that the embodiments of the present application do not specifically limit the specific implementation manner of obtaining the above original ultrasonic image, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the original ultrasonic image sent by an external device (such as an ultrasonic device, etc.) can be received; or, the original ultrasonic image pre-stored locally or in the cloud can be read, etc.

[0093] In the above step 2), the artifact classification model can adopt a Residual Network (ResNet), a Densely Connected Convolutional Networks (DenseNet), etc., and the embodiments of the present application do not specifically limit this.

[0094] The classification result may include probability values corresponding to different artifact types, where the artifact types include acoustic shadows, reverberation artifacts, refraction artifacts, sidelobe artifacts, etc.

[0095] Further, based on the above embodiments, as an implementation manner, after obtaining the original ultrasound image, models such as Residual Network (ResNet) and Densely Connected Convolutional Networks (DenseNet) can be used to perform binary classification on the original ultrasound image to determine whether there are artifacts in the original ultrasound image. If there are artifacts in the original ultrasound image, the artifact repair method for the ultrasound image provided in the embodiments of the present application is executed; otherwise, the artifact repair method for the ultrasound image provided in the embodiments of the present application is not executed.

[0096] Further, based on the above embodiments, the embodiments of the present application provide an artifact-aware multi-scale fusion network. Please refer to Figure 5 , Figure 5 which is a schematic diagram of an artifact-aware multi-scale fusion network provided in the embodiments of the present application. The artifact-aware multi-scale fusion network includes an artifact-aware attention model, a dynamic condition generator, and an ultrasound image generator. The following introduces the specific implementation manner of training the above-mentioned artifact-aware multi-scale fusion network. In the embodiments of the present application, the artifact repair method for the ultrasound image may further include the following steps:

[0097] Step 1), obtain a sample image and label data corresponding to the sample image.

[0098] Step 2), input the sample feature map corresponding to the sample image and the sample classification result into a neural network model to obtain a predicted image output by the neural network model, and calculate the neural network loss value according to the predicted image and the sample image.

[0099] Step 3), input the predicted image and the sample image into a global discriminator to obtain a corresponding global discrimination result, and calculate a first adversarial loss value of the global discriminator according to the global discrimination result. And input the predicted image and the sample image into a local discriminator to obtain a corresponding local discrimination result, and calculate a second adversarial loss value of the local discriminator according to the local discrimination result.

[0100] Step 4), optimize the neural network model, the global discriminator, and the local discriminator according to the neural network loss value, the first adversarial loss value, and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network.

[0101] Specifically, in the above step 1), the embodiments of the present application do not specifically limit the specific implementation manner of obtaining the sample image and the label data, and those skilled in the art can make appropriate adjustments according to the actual situation. For example, the sample image and the label data sent by an external device can be received; or, the sample image and the label data pre-stored locally or in the cloud can be read, etc.

[0102] In the above step 2), the neural network model has the same architecture as the artifact-aware multi-scale fusion network, and the only difference is that the network parameters of the two are different. By optimizing the parameters in the neural network model, the artifact-aware multi-scale fusion network can be obtained.

[0103] By inputting the sample feature map corresponding to the sample image and the sample classification result into the neural network model, a predicted image output by the neural network model can be obtained, and the neural network loss value can be calculated based on the predicted image and the sample image. Among them, the above neural network loss value characterizes the difference between the predicted image and the sample image.

[0104] In the above step 3), multi-discriminator collaborative training can be adopted, aiming to improve the overall quality and local details of the restored image through the collaborative action of the global discriminator and the local discriminator.

[0105] Among them, the global discriminator adopts a fully convolutional network structure, with the entire predicted image as the input and the authenticity score of the predicted image as the output. As an implementation, the global discriminator can optimize the overall restoration ability of the generator through the adversarial loss (Wasserstein GAN Loss). The local discriminator adopts a convolutional network structure with a local receptive field, with the local image patch of the artifact area as the input and the authenticity score of the local image patch as the output. As an implementation, the local discriminator can optimize the local restoration ability of the generator through the adversarial loss.

[0106] In the above solution, the neural network model is trained through multi-discriminator collaborative training, and then through the collaborative action of the global discriminator and the local discriminator, the overall quality and local details of the restored image are improved, thereby improving the artifact restoration ability of the trained artifact-aware multi-scale fusion network.

[0107] Furthermore, on the basis of the above embodiments, the above neural network loss value is equal to the sum of the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value.

[0108] Specifically, the perceptual loss value characterizes the pixel-level difference between the predicted image and the sample image. As an implementation, the mean absolute error (MAE) loss (i.e., L1 loss) can be adopted. This perceptual loss value is an auxiliary loss added to the output feature map of each scale to enhance the expression ability of multi-scale features. The multi-scale structural similarity loss value characterizes the structural similarity between the predicted image and the sample image, and the classification consistency loss value characterizes the probability of artifacts when the predicted image passes through the artifact classification model.

[0109] As an implementation, the total loss value of the embodiments of the present application can be calculated using the following formula:

[0110] L total = αL adv + βL per + γL ssim + λL cls ;

[0111] Wherein, L total is the total loss value, L adv is the adversarial loss value, including the first adversarial loss value and the second adversarial loss value, L per is the perceptual loss value, L ssim is the multi-scale structural similarity loss value, L cls is the classification consistency loss value, and α, β, γ, and λ are the weight coefficients of each loss term.

[0112] As an implementation, the above adversarial loss value L adb can be calculated using the Wasserstein GAN Loss, the perceptual loss value L per can be calculated using the L1Loss function, the multi-scale structural similarity loss value L ssim can be calculated using the SSIM Loss function, and the classification consistency loss value L cls can be calculated using the CrossEntropyLoss function.

[0113] In the above solution, by introducing the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value to train the neural network model, the artifact repair ability of the trained artifact perception multi-scale fusion network is improved.

[0114] Please refer to Figure 6 , Figure 6The present application provides a structural block diagram of an artifact repair device for ultrasonic images. The ultrasonic image artifact repair device 600 includes: a first input module 601, configured to input a feature map corresponding to an ultrasonic image to be repaired and a classification result into an artifact-aware attention model, and obtain an attention mask output by the artifact-aware attention model, where the feature map is obtained by performing feature extraction on the ultrasonic image to be repaired, and the classification result represents the type of artifact in the ultrasonic image to be repaired; a second input module 602, configured to input the attention mask into a dynamic conditional generator, and obtain a depth feature output by the dynamic conditional generator; a third input module 603, configured to input the depth feature into an ultrasonic image generator, and obtain a target ultrasonic image output by the ultrasonic image generator, where the target ultrasonic image is an image obtained by repairing the artifact in the ultrasonic image to be repaired.

[0115] In the above solution, an artifact repair algorithm that fuses a dynamic conditional generative adversarial network and an artifact-aware attention model is provided. By this algorithm, the artifact area can be automatically identified and repaired, so that the artifact in the ultrasonic image can be repaired while retaining the organ anatomical structure in the ultrasonic image, thereby improving the accuracy of repairing the artifact in the ultrasonic image.

[0116] Further, based on the above embodiment, the artifact-aware attention model includes a multi-layer perceptron, a channel attention module, and a spatial attention module. The first input module 601 is specifically configured to: input the classification result into the multi-layer perceptron, and obtain a conditional embedding feature output by the multi-layer perceptron; fuse the conditional embedding feature with the feature map, and input them into the channel attention module and the spatial attention module respectively, and obtain a channel attention weight map output by the channel attention module, and a spatial attention weight map output by the spatial attention module; fuse the channel attention weight map and the spatial attention weight map to obtain the attention mask.

[0117] In the above solution, the artifact-aware attention model can dynamically adjust the weights of the artifact area and the normal tissue area in the feature map, avoiding over-repairing the normal tissue area, so that the artifact in the ultrasonic image can be repaired while retaining the organ anatomical structure in the ultrasonic image.

[0118] Further, based on the above embodiment, the dynamic conditional generator includes a residual dense block and a conditional normalization layer. The second input module 602 is specifically configured to: input the attention mask into the residual dense block, and obtain an initial feature output by the residual dense block; input the initial feature into the conditional normalization layer, and obtain the depth feature output by the conditional normalization layer.

[0119] In the above solution, targeted repair of different types of artifacts is achieved through a dynamic condition generator, that is, the above dynamic condition generator can dynamically adjust the parameters of the ultrasonic image generator according to the classification result of the artifacts, thereby improving the repair effect of the artifacts.

[0120] Further, on the basis of the above embodiment, the artifact repair device 600 of the ultrasonic image further includes: a first acquisition module, configured to acquire an original ultrasonic image; a fourth input module, configured to input the original ultrasonic image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or, input the original ultrasonic image into an artifact classification model to obtain the classification result output by the artifact classification model.

[0121] In the above solution, multi-resolution features in the original ultrasonic image are extracted through a multi-scale pyramid encoder, and combined with a cross-scale feature fusion mechanism, so as to enhance the repair ability for small lesions and complex artifacts, and further improve the accuracy of repairing artifacts in the ultrasonic image.

[0122] Further, on the basis of the above embodiment, the multi-scale pyramid encoder includes three branches, and at least one branch includes an atrous convolution module.

[0123] In the above solution, introducing spatial convolution in the multi-scale pyramid encoder can expand the receptive field, thereby enhancing the ability to capture larger artifact regions, and further improving the accuracy of repairing artifacts in the ultrasonic image.

[0124] Further, based on the above embodiments, the artifact repair device 600 for ultrasonic images further includes: a second acquisition module, configured to acquire a sample image and label data corresponding to the sample image; a fifth input module, configured to input a sample feature map corresponding to the sample image and a sample classification result into a neural network model, obtain a predicted image output by the neural network model, and calculate a neural network loss value according to the predicted image and the sample image; a sixth input module, configured to input the predicted image and the sample image into a global discriminator, obtain a corresponding global discrimination result, and calculate a first adversarial loss value of the global discriminator according to the global discrimination result, and input the predicted image and the sample image into a local discriminator, obtain a corresponding local discrimination result, and calculate a second adversarial loss value of the local discriminator according to the local discrimination result; an optimization module, configured to optimize the neural network model, the global discriminator, and the local discriminator according to the neural network loss value, the first adversarial loss value, and the second adversarial loss value, to obtain an artifact-aware multi-scale fusion network, where the artifact-aware multi-scale fusion network includes the artifact-aware attention model, the dynamic conditional generator, and the ultrasonic image generator.

[0125] In the above solution, the neural network model is trained through collaborative training of multiple discriminators. Furthermore, through the collaborative action of the global discriminator and the local discriminator, the overall quality and local details of the repaired image are improved, thereby enhancing the artifact repair ability of the trained artifact-aware multi-scale fusion network.

[0126] Further, based on the above embodiments, the neural network loss value is equal to the sum of the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value, where the perceptual loss value represents the pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value represents the structural similarity between the predicted image and the sample image, and the classification consistency loss value represents the probability of artifacts when the predicted image passes through an artifact classification model.

[0127] In the above solution, the neural network model is trained by introducing the perceptual loss value, the multi-scale structural similarity loss value, and the classification consistency loss value, thereby enhancing the artifact repair ability of the trained artifact-aware multi-scale fusion network.

[0128] Please refer to Figure 7 , Figure 7A structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device 700 includes: at least one processor 701, at least one communication interface 702, at least one memory 703, and at least one communication bus 704. Among them, the communication bus 704 is used to realize the direct connection and communication of these components. The communication interface 702 is used to communicate signaling or data with other node devices. The memory 703 stores machine-readable instructions executable by the processor 701. When the electronic device 700 runs, the processor 701 communicates with the memory 703 through the communication bus 704. When the machine-readable instructions are called by the processor 701, the above-mentioned method for artifact repair of ultrasonic images is executed.

[0129] For example, the processor 701 of the embodiment of the present application reads a computer program from the memory 703 through the communication bus 704 and executes the computer program to implement the following method: input the feature map corresponding to the ultrasonic image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model, where the feature map is obtained by extracting features from the ultrasonic image to be repaired, and the classification result represents the type of artifact in the ultrasonic image to be repaired; input the attention mask into the dynamic condition generator to obtain the depth feature output by the dynamic condition generator; input the depth feature into the ultrasonic image generator to obtain the target ultrasonic image output by the ultrasonic image generator, where the target ultrasonic image is an image obtained by repairing the artifact in the ultrasonic image to be repaired.

[0130] Among them, the processor 701 includes one or more, which can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 701 can be a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP), or other conventional processors; it can also be a dedicated processor, including a neural network processor (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Moreover, when there are multiple processor 701s, a part of them can be general-purpose processors and another part can be dedicated processors.

[0131] The memory 703 includes one or more, which can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0132] It can be understood that Figure 7 The structure shown is only for illustration, and the electronic device 700 may also include more or fewer components than those Figure 7 shown, or have a different configuration from that Figure 7 shown. Figure 7Each component shown in the figure may be implemented by hardware, software, or a combination thereof. In the embodiments of the present application, the electronic device 700 may be, but is not limited to, an entity device such as a desktop computer, a laptop computer, a smart phone, a smart wearable device, a vehicle-mounted device, etc., or may also be a virtual device such as a virtual machine. In addition, the electronic device 700 does not necessarily have to be a single device, and may also be a combination of multiple devices, such as a server cluster, and so on.

[0133] The embodiments of the present application also provide a computer program product, including a computer program stored on a computer-readable storage medium. The computer program includes computer program instructions. When the computer program instructions are executed by a computer, the computer can execute the steps of the method for artifact repair of ultrasonic images in the above embodiments. For example, it includes: Step S101: Input the feature map corresponding to the ultrasonic image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model. Step S102: Input the attention mask into the dynamic condition generator to obtain the depth feature output by the dynamic condition generator. Step S103: Input the depth feature into the ultrasonic image generator to obtain the target ultrasonic image output by the ultrasonic image generator.

[0134] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores computer program instructions. When the computer program instructions are run by a computer, the computer executes the method for artifact repair of ultrasonic images described in the foregoing method embodiments.

[0135] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces. The indirect coupling or communication connection of the devices or units may be in an electrical, mechanical, or other form.

[0136] In addition, the units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] Furthermore, in each embodiment of the present application, each functional module may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0138] It should be noted that if a function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0139] In this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0140] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for repairing artifacts of an ultrasonic image, characterized in that: include: Inputting a feature map corresponding to the ultrasound image to be repaired and a classification result into an artifact-aware attention model to obtain an attention mask output by the artifact-aware attention model, wherein the feature map is obtained by performing feature extraction on the ultrasound image to be repaired, and the classification result represents the type of artifacts in the ultrasound image to be repaired; Inputting the attention mask into a dynamic condition generator to obtain a deep feature output by the dynamic condition generator; The depth feature is input into an ultrasonic image generator to obtain a target ultrasonic image output by the ultrasonic image generator, wherein the target ultrasonic image is an image after repairing artifacts in the ultrasonic image to be repaired.

2. The method for repairing ultrasonic image artifacts according to claim 1, characterized in that: The artifact-aware attention model includes a multi-layer perceptron, a channel attention module, and a spatial attention module. The feature map corresponding to the ultrasound image to be repaired and the classification result are input into the artifact-aware attention model to obtain an attention mask output by the artifact-aware attention model, including: Inputting the classification result into the multi-layer perceptron to obtain conditional embedding features output by the multi-layer perceptron; The conditional embedding feature is fused with the feature map, and the two are respectively input into the channel attention module and the spatial attention module to obtain a channel attention weight map output by the channel attention module and a spatial attention weight map output by the spatial attention module; The channel attention weight map is fused with the spatial attention weight map to obtain the attention mask.

3. The method for repairing ultrasonic image artifacts according to claim 1, characterized in that: The dynamic condition generator includes a residual dense block and a conditional normalization layer, and the attention mask is input into the dynamic condition generator to obtain the deep features output by the dynamic condition generator, including: Inputting the attention mask into the residual dense block to obtain the initial features output by the residual dense block; The initial features are input into the conditional normalization layer to obtain the depth features output by the conditional normalization layer.

4. The method for repairing ultrasonic image artifacts according to claim 1, characterized in that: Before inputting the feature map corresponding to the ultrasound image to be repaired and the classification result into the artifact-aware attention model to obtain the attention mask output by the artifact-aware attention model, the method further includes: Acquire original ultrasound images; Inputting the original ultrasound image into a multi-scale pyramid encoder to obtain the feature map output by the multi-scale pyramid encoder; and / or, The original ultrasound image is input into an artifact classification model to obtain the classification result output by the artifact classification model.

5. The method for repairing ultrasonic image artifacts according to claim 4, characterized in that: The multi-scale pyramid encoder includes three branches, at least one of which includes a dilated convolution module.

6. The method for repairing ultrasonic image artifacts according to any one of claims 1 to 5, characterized in that: Also includes: Acquire a sample image and label data corresponding to the sample image; Inputting the sample feature map corresponding to the sample image and the sample classification result into the neural network model to obtain the predicted image output by the neural network model, and calculating the neural network loss value according to the predicted image and the sample image; Input the predicted image and the sample image into a global discriminator to obtain a corresponding global discriminant result, and calculate a first adversarial loss value of the global discriminator according to the global discriminant result; and input the predicted image and the sample image into a local discriminator to obtain a corresponding local discriminant result, and calculate a second adversarial loss value of the local discriminator according to the local discriminant result; The neural network model, the global discriminator and the local discriminator are optimized according to the neural network loss value, the first adversarial loss value and the second adversarial loss value to obtain an artifact-aware multi-scale fusion network, wherein the artifact-aware multi-scale fusion network includes the artifact-aware attention model, the dynamic condition generator and the ultrasound image generator.

7. The method for repairing ultrasonic image artifacts according to claim 6, characterized in that: The neural network loss value is equal to the sum of the perceptual loss value, the multi-scale structural similarity loss value and the classification consistency loss value, wherein the perceptual loss value represents the pixel-level difference between the predicted image and the sample image, the multi-scale structural similarity loss value represents the structural similarity between the predicted image and the sample image, and the classification consistency loss value represents the probability of artifacts when the predicted image passes through the artifact classification model.

8. A computer program product, characterized in that The method comprises computer program instructions, and when the computer program instructions are read and executed by a processor, the method for repairing artifacts of an ultrasound image according to any one of claims 1 to 7 is executed.

9. An electronic device, characterized in that: include: processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions that can be executed by the processor, and the processor calls the computer program instructions to execute the method for repairing artifacts of an ultrasound image as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a computer, the computer is enabled to perform the method for repairing artifacts of an ultrasound image according to any one of claims 1 to 7.

Images