Deep learning based ultrasound image artifact automatic correction method

Abstract

The application discloses a kind of based on deep learning's ultrasonic image artifact automatic correction method, it is related to image processing technical field, the method includes steps: obtaining the ultrasonic image data to be detected;Based on the feature evaluation value of each divided region in the ultrasonic image data, the artifact sensitivity of the ultrasonic image data under different scales is calculated, and the artifact sensitivity is used to represent the identification sensitivity of artifact feature under different scales;Based on the artifact sensitivity, the attention map corresponding to the artifact region in the ultrasonic image data is generated;The ultrasonic image data and the attention map are input into the preset identification model, and based on the preset identification model, the feature map corresponding to the ultrasonic image data and the attention map are combined by weighting, to obtain the weighted feature map;The artifact identification result of the preset identification model is corrected with the weighted feature map.The application achieves the technical effect of improving the accuracy of model identification artifact.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an automatic correction method for ultrasound image artifacts based on deep learning. Background Technology

[0002] Ultrasound imaging has become an indispensable tool in clinical diagnosis due to its advantages such as non-invasiveness, real-time performance, and low cost. However, its imaging quality is easily affected by various artifacts, such as motion stripe artifacts, which may obscure the true anatomical structure or produce false lesion features.

[0003] In related technologies, hierarchical feature extraction is performed through a multi-scale attention mechanism and coordinated with dynamic weight allocation, achieving a balance between local repair and global consistency in ultrasound artifact removal. However, directly identifying or correcting artifacts on the image based on artifact features in ultrasound images ignores the complex features of artifacts, making it difficult to accurately identify artifacts using a single feature. Summary of the Invention

[0004] The main objective of this application is to provide an automatic correction method for ultrasound image artifacts based on deep learning. This method aims to solve the technical problem in related technologies where artifact identification or correction is performed directly on the image based on artifact features, ignoring the complex features of artifacts and making it difficult to accurately identify artifacts using a single feature.

[0005] To achieve the above objectives, embodiments of this application provide a method for automatic correction of ultrasound image artifacts based on deep learning, comprising: Acquire the ultrasound image data to be tested; Based on the feature evaluation values ​​of each divided region in the ultrasound image data, the artifact sensitivity of the ultrasound image data at different scales is calculated. The artifact sensitivity is used to characterize the sensitivity to the recognition of artifact features at different scales. Based on artifact sensitivity, an attention map corresponding to the artifact region in the ultrasound image data is generated. Ultrasound image data and attention map are input into a preset recognition model. Based on the preset recognition model, the feature map and attention map corresponding to the ultrasound image data are weighted and combined to obtain a weighted feature map. The artifact recognition results of the preset recognition model are corrected using the weighted feature map.

[0006] In one possible implementation of this application, before calculating the artifact sensitivity of the ultrasound image data at different scales based on the feature evaluation values ​​of each divided region in the ultrasound image data, the method further includes: The ultrasound image data is divided into superpixels for each image frame to obtain multiple superpixels for each image frame; Each superpixel is clustered according to different scales to obtain segmented image data at multiple scales, including macro scale, meso scale and micro scale. Based on the feature evaluation values ​​of each segmented region in the ultrasound image data, the artifact sensitivity of the ultrasound image data at different scales is calculated, including: Based on the feature evaluation values ​​of each segmented region in the segmented image data, the artifact sensitivity of the segmented image data at different scales is calculated.

[0007] In one possible implementation of this application, the artifact sensitivity of the segmented image data at different scales is calculated based on the feature evaluation values ​​of each segmented region in the segmented image data, including: Based on the number of regions divided in the image data at each scale and the area difference between the regions, the emphasis coefficient at each scale is calculated. For any given scale, based on the emphasis coefficient and the artifact features of each divided region, the feature evaluation value of each divided region at the current scale is calculated. Based on feature evaluation values, the comparison relationship between image frames at each scale is determined; Based on the feature evaluation values ​​of the same divided region at different scales and the comparison relationship, the artifact sensitivity of the divided image data at different scales is determined.

[0008] In one possible implementation of this application, based on the emphasis coefficient and the artifact features of each divided region, the feature evaluation value of each divided region at the current scale is calculated, including: Calculate the overall distribution characteristics of different divided regions at the current scale, and calculate the internal characteristics of each divided region; Based on the overall distribution characteristics, internal characteristics, and emphasis coefficients, the characteristic evaluation value of each divided region is calculated.

[0009] In one possible implementation of this application, calculating the overall distribution characteristics of different divided regions at the current scale includes: Obtain the first direction of each divided region; For any segmented region, select the first region in the image frame whose directional difference value with the currently segmented region is less than a preset directional threshold; Obtain the total number of all divided regions at the current scale, the first number of the first region, and the average value of the directional angles formed between each first region and the current divided region; Based on the first quantity, the total quantity, and the average angle, the overall distribution characteristics of the different divided regions are calculated.

[0010] In one possible implementation of this application, the internal features of each divided region are calculated, including: Obtain the average edge gradient and edge length of different divided regions at the current scale; Based on the average edge gradient and edge length, the internal features of each divided region are calculated.

[0011] In one possible implementation of this application, determining the correspondence between image frames at each scale based on feature evaluation values ​​includes: Select the maximum feature evaluation value among the feature evaluation values ​​of each image frame; Calculate the standard deviation of the feature evaluation values ​​of all divided regions within each image frame; For any given scale, the comparison relationship between image frames at the current scale is calculated based on the standard deviation and the slope of change of the maximum feature evaluation value of each adjacent image frame.

[0012] In one possible implementation of this application, the artifact sensitivity of the segmented image data at different scales is determined based on the feature evaluation values ​​of the same segmented region at different scales and the comparison relationship, including: For any divided region, obtain the mean value of the feature evaluation value of the current divided region at each scale, and use the mean value as the reference evaluation feature value; Determine the largest eigenvalue among the reference evaluation eigenvalues ​​across all scales; Based on the reference evaluation eigenvalue, the maximum eigenvalue, and the comparison relationship, the artifact sensitivity of each divided region at each scale is calculated.

[0013] In one possible implementation of this application, based on a preset recognition model, the feature map and attention map corresponding to the ultrasound image data are weighted and combined to obtain a weighted feature map, including: Based on the attention mechanism of the preset recognition model, the ultrasound image data is processed to obtain the original feature map; The original feature map and the attention map are fused according to a preset weight ratio to obtain a weighted feature map.

[0014] In one possible implementation of this application, the artifact recognition result of the preset recognition model is corrected using a weighted feature map, including: The weighted feature map is input into the attention module, and the attention module is inserted between the network layers of the preset recognition model; The weighted feature map is processed by an attention module to identify artifacts, and the artifact identification results are output.

[0015] This application provides a deep learning-based automatic artifact correction method for ultrasound images. Compared with related technologies that directly identify or correct artifacts on the image based on artifact features, ignoring the complex features of artifacts and making it difficult to accurately identify artifacts using a single feature, this application acquires ultrasound image data to be detected and calculates the artifact sensitivity of the ultrasound image data at different scales based on the feature evaluation values ​​of each divided region in the ultrasound image data. Then, it generates an attention map corresponding to the artifact region in the ultrasound image data based on the artifact sensitivity. Since the artifact sensitivity characterizes the sensitivity to the recognition of artifact features at different scales, it can highlight the regions with obvious artifact features in the ultrasound image data. Thus, it generates an attention map corresponding to the region of interest in the ultrasound image data. The ultrasound image data and the attention map are then input into a preset recognition model. Based on the preset recognition model, the feature map and attention map corresponding to the ultrasound image data are weighted and combined to obtain a weighted feature map. Therefore, the weighted feature map is more easily recognized by the attention module to identify the corresponding artifact features, thereby enabling the preset recognition model to output accurate artifact recognition results. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the first embodiment of the deep learning-based automatic correction method for ultrasound image artifacts in this application. Figure 2 This is a schematic diagram of the ultrasound image data involved in the deep learning-based automatic correction method for ultrasound image artifacts in this application. Figure 3 This is a schematic diagram of the macroscopically segmented image involved in the deep learning-based automatic correction method for ultrasound image artifacts in this application; Figure 4 This is a schematic diagram of the mesoscale segmentation image involved in the deep learning-based automatic correction method for ultrasound image artifacts in this application. Figure 5 This is a schematic diagram of the segmented image at the microscale involved in the deep learning-based automatic correction method for ultrasound image artifacts in this application; Figure 6 This is a schematic diagram of the execution flow of the second embodiment of the deep learning-based automatic correction method for ultrasound image artifacts in this application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0018] This application provides a deep learning-based automatic ultrasound image artifact correction method. In the first embodiment of this deep learning-based automatic ultrasound image artifact correction method, referring to... Figure 1 The methods include: Step S10: Obtain the ultrasound image data to be detected; Step S20: Based on the feature evaluation values ​​of each divided region in the ultrasound image data, calculate the artifact sensitivity of the ultrasound image data at different scales. The artifact sensitivity is used to characterize the sensitivity of the recognition of artifact features at different scales. Step S30: Based on artifact sensitivity, generate an attention map corresponding to the artifact region in the ultrasound image data; Step S40: Input the ultrasound image data and attention map into the preset recognition model. Based on the preset recognition model, perform a weighted combination of the feature map and attention map corresponding to the ultrasound image data to obtain the weighted feature map. Step S50: Correct the artifact recognition results of the preset recognition model using the weighted feature map.

[0019] This embodiment aims to improve the accuracy of the output artifact recognition results.

[0020] The specific steps are as follows: Step S10: Obtain the ultrasound image data to be detected.

[0021] As an example, the deep learning-based automatic correction method for ultrasound image artifacts can be applied to a deep learning-based automatic correction device for ultrasound image artifacts. The deep learning-based automatic correction device for ultrasound image artifacts belongs to a deep learning-based automatic correction system for ultrasound image artifacts, and the deep learning-based automatic correction system for ultrasound image artifacts belongs to a deep learning-based automatic correction equipment for ultrasound image artifacts.

[0022] As an example, ultrasound image data can be multiple ultrasound image animations determined to have artifacts, obtained from a database. A schematic diagram of ultrasound image data is shown below. Figure 2 As shown, the area pointed to by the arrow is the region where artifacts exist.

[0023] Step S20: Based on the feature evaluation values ​​of each divided region in the ultrasound image data, calculate the artifact sensitivity of the ultrasound image data at different scales. The artifact sensitivity is used to characterize the sensitivity of the recognition of artifact features at different scales.

[0024] As an example, in abdominal ultrasound imaging, reverberation artifacts (manifesting as repeating streaks) often occur due to factors such as sound wave reflection, attenuation, or probe movement. These artifacts may obscure real anatomical structures (such as liver vessels or kidney contours). Through multi-scale attention mechanisms, artifact features at different scales can be dynamically distinguished and corrected in a targeted manner, while preserving the details of real tissue.

[0025] As an example, artifact sensitivity represents the degree of sensitivity to artifact features at different scales. For the same artifact, the sensitivity to artifact features varies at different scales or from different perspectives of feature focus. Therefore, by calculating the artifact sensitivity of ultrasound image data at different scales, we can determine at which scale artifact features are more easily identified, thereby improving the artifact recognition results of the model.

[0026] Prior to step S20, which calculates the artifact sensitivity of ultrasound image data at different scales based on the feature evaluation values ​​of each divided region in the ultrasound image data, the method further includes: The ultrasound image data is divided into superpixels for each image frame to obtain multiple superpixels for each image frame; As an example, superpixel partitioning divides each image frame into several small regions that have similar visual features (such as color, texture, etc.). Each small region is called a superpixel. Compared with directly processing pixels, superpixels can reduce computational complexity while preserving the key structural information of the image. With the initial number of superpixels K=100 and the compactness factor m=10, several superpixels are obtained for each frame of ultrasound image.

[0027] Each superpixel is clustered according to different scales to obtain segmented image data at multiple scales, including macro scale, meso scale and micro scale. Based on the feature evaluation values ​​of each segmented region in the ultrasound image data, the artifact sensitivity of the ultrasound image data at different scales is calculated, including: Based on the feature evaluation values ​​of each segmented region in the segmented image data, the artifact sensitivity of the segmented image data at different scales is calculated.

[0028] As an example, superpixels, as the result of pixel block partitioning at a small scale, are used in multi-scale processing to divide various regions in ultrasound image data into macroscopic (large scale, such as global structure), mesoscopic (medium scale, such as local texture) and microscopic (small scale, such as noise) based on image features.

[0029] As an example, the method of dividing into multiple scales can be as follows: For each superpixel at a small scale in the image frame, perform two clustering operations (the clustering method can be: extract the feature vector composed of the gray mean and gradient standard deviation of the pixels within the superpixel block, and use the difference in feature vectors between superpixel blocks as the clustering distance for clustering). The superpixels are merged into meso- and macro-scale superpixels through two clustering operations respectively. The first clustering corresponds to the pixel block division result at the medium scale, and only adjacent superpixels are allowed to be merged; the second clustering corresponds to the pixel block division result at the large scale, ensuring that the region division results at different scales can reflect the comparison relationship between ultrasound image artifacts and real tissues at different levels.

[0030] As an example, a schematic diagram of image segmentation at macroscopic, mesoscopic, and microscopic scales is shown below. Figure 3 As shown in / 4 / 5, the size of the image regions obtained by dividing the image at different scales is also different.

[0031] As an example, after dividing ultrasound image data into segments according to different scales, segmented image data is obtained, and then the artifact sensitivity of the segmented image data at different scales is calculated.

[0032] Step S30: Based on artifact sensitivity, generate an attention map corresponding to the artifact region in the ultrasound image data.

[0033] As an example, after calculating the artifact sensitivity in the same region at different scales, an attention map is generated based on the artifact sensitivity, which shows the region that the model should focus on when processing ultrasound images.

[0034] Step S40: Input the ultrasound image data and attention map into the preset recognition model. Based on the preset recognition model, perform weighted combination of the feature map and attention map corresponding to the ultrasound image data to obtain the weighted feature map.

[0035] As an example, the preset recognition model can be a neural network model based on deep learning. The preset recognition model includes an attention mechanism. After generating the attention map, the ultrasound image data and the attention map are input into the preset recognition model. The preset recognition model combines the feature map obtained after processing the ultrasound image data by the attention mechanism with the attention map to obtain a weighted feature map.

[0036] The step S40, which involves weightedly combining the feature map and attention map corresponding to the ultrasound image data based on a preset recognition model to obtain a weighted feature map, includes: Based on the attention mechanism of the preset recognition model, the ultrasound image data is processed to obtain the original feature map; As an example, the original feature map is obtained by processing the acquired ultrasound image data through the initial attention mechanism of the deep learning model.

[0037] The original feature map and the attention map are fused according to a preset weight ratio to obtain a weighted feature map.

[0038] As an example, the preset weight ratio can be 1:3, 2:5, etc., and there is no specific limitation.

[0039] As an example, the original feature map and the attention map are fused according to a preset weight ratio, so that the weighted feature map has the characteristics of the attention map, thereby enhancing the model's sensitivity to artifact regions.

[0040] Step S50: Correct the artifact recognition results of the preset recognition model using the weighted feature map.

[0041] As an example, after obtaining the weighted feature map, the weighted feature map is input into the preset recognition model, and then the preset recognition model is used to recognize the weighted feature map, thereby enhancing the model's adaptive correction capability for artifacts and outputting artifact recognition results with higher accuracy.

[0042] The method of correcting the artifact recognition results of the preset recognition model using the weighted feature map includes: The weighted feature map is input into the attention module, and the attention module is inserted between the network layers of the preset recognition model; The weighted feature map is processed by an attention module to identify artifacts, and the artifact identification results are output.

[0043] As an example, the weighted feature map is input into the attention module in the preset recognition model, and the attention module is inserted between the network layers of the model, such as the decoder part in the encoder-decoder structure. During training, a comprehensive loss function is designed, which includes artifact correction loss and other related task losses, to improve the overall performance of the model in multiple tasks.

[0044] The model's performance was evaluated using an independent validation set, with particular attention to the artifact correction effect. Quantitative metrics (such as PSNR (Peak Signal-to-Noise Ratio, the ratio of the maximum possible power of the image signal to the noise power affecting image quality, used to measure the quality difference between the original and processed images) and qualitative observations (such as SSIM (Structural Similarity Index, an index that evaluates image quality based on image structural information, considering three factors: brightness, contrast, and structure)) and qualitative observations (such as visual evaluation) were used to comprehensively evaluate the model's actual performance. Based on the evaluation results, the model was optimized by adjusting the design of hyperparameters, network structure, and attention mechanism to further improve the correction effect.

[0045] This application provides a deep learning-based automatic artifact correction method for ultrasound images. Compared with related technologies that directly identify or correct artifacts on the image based on artifact features, ignoring the complex features of artifacts and making it difficult to accurately identify artifacts using a single feature, this application acquires ultrasound image data to be detected and calculates the artifact sensitivity of the ultrasound image data at different scales based on the feature evaluation values ​​of each divided region in the ultrasound image data. Then, it generates an attention map corresponding to the artifact region in the ultrasound image data based on the artifact sensitivity. Since the artifact sensitivity characterizes the sensitivity to the recognition of artifact features at different scales, it can highlight the regions with obvious artifact features in the ultrasound image data. Thus, it generates an attention map corresponding to the region of interest in the ultrasound image data. The ultrasound image data and the attention map are then input into a preset recognition model. Based on the preset recognition model, the feature map and attention map corresponding to the ultrasound image data are weighted and combined to obtain a weighted feature map. Therefore, the weighted feature map is more easily recognized by the attention module to identify the corresponding artifact features, thereby enabling the preset recognition model to output accurate artifact recognition results.

[0046] Furthermore, referring to Figure 6 Based on the first embodiment of this application, another embodiment of this application is provided. In this embodiment, step S20, which calculates the artifact sensitivity of the segmented image data at different scales based on the feature evaluation values ​​of each segmented region in the segmented image data, includes: Step S21: Based on the number of regions divided in the image data at each scale and the area difference between the regions divided, calculate the emphasis coefficient at each scale. As an example, the region to be divided can be an image region composed of superpixels. The size of the region to be divided varies at different scales, and the number of regions to be divided can be 50, 100, etc., depending on the method of division, and there is no specific limitation.

[0047] As an example, the area difference value can be the area difference between the areas occupied by each divided region. It should be noted that the emphasis coefficient is calculated on a per-image-frame basis, that is, the emphasis coefficient corresponding to each image frame.

[0048] As an example, the emphase coefficient reflects the degree of emphasis on the directional structure of artifacts in the overall distribution at the current scale, and the difference between the emphase coefficient and 1 reflects the degree of emphasis on the internal texture of artifacts at the current scale. In this way, the analysis of artifacts in ultrasound image data at larger scales focuses more on the regularity between the overall structure and less on the internal texture, ensuring that there are analytical differences between scales.

[0049] As an example, artifacts typically occupy a smaller area than the actual tissue region in an ultrasound image. Furthermore, in small-scale segmentation results after superpixel partitioning, the area differences between regions are relatively small initially. However, at larger scales after clustering, because the area of ​​artifacts in ultrasound images is generally relatively small, the area differences between the segmented regions gradually increase. Therefore, the larger the scale of the region segmentation results, the greater the area difference between different segmented regions. At larger scales, the smaller the number of segmented regions, the more important the factor becomes. The calculation method can be: In the formula, This indicates the number of regions that are divided within the same frame of an image at scale a; This represents the maximum area difference between the divided regions at scale a, and th() represents the normalization calculation.

[0050] As an example, when quantifying the feature evaluation values ​​of each divided region at different scales, the emphasis is gradually reduced in capturing internal texture features as the scale increases, while the emphasis is greater in analyzing the patterns between the overall structures, by using the emphasis coefficient.

[0051] Step S22: For any scale, based on the emphasis coefficient and the artifact features of each divided region, calculate the feature evaluation value of each divided region at the current scale. As an example, artifact features include overall distribution features and internal features. Overall distribution features represent the location distribution features and orientation features of the divided region, while internal features are the edge features of the divided region.

[0052] As an example, when quantifying the feature evaluation values ​​of each divided region at different scales, the emphasis is gradually reduced in capturing internal texture features as the scale increases, while the emphasis is greater in analyzing the patterns between the overall structures, by using the emphasis coefficient.

[0053] As an example, feature evaluation value is a metric that quantifies the quality, importance, or effectiveness of image features. It can help us understand the value of image features in a specific task (such as image classification, object detection, image retrieval, etc.). For artifact features, the larger the feature evaluation value, the greater the likelihood that the region is an artifact region.

[0054] Step S22, which calculates the feature evaluation value of each divided region at the current scale based on the emphasis coefficient and the artifact features of each divided region, includes: Calculate the overall distribution characteristics of different divided regions at the current scale, and calculate the internal characteristics of each divided region; As an example, before calculating the feature evaluation value of each divided region, it is necessary to first calculate the overall distribution characteristics and internal characteristics of different divided regions, and then calculate the feature evaluation value using the obtained internal characteristics and overall distribution characteristics.

[0055] The steps for calculating the overall distribution characteristics of different divided regions at the current scale include: Obtain the first direction of each divided region; For any segmented region, select the first region in the image frame whose directional difference value with the currently segmented region is less than a preset directional threshold; As an example, the divided region can be rectangular or irregular in shape. The divided region will have an overall pointing direction. For any divided region, the direction of the long side of its circumscribed rectangle is taken as the direction of the region, that is, the first direction.

[0056] As an example, the directional difference value is the directional deviation between two divided regions. For example, the directional difference value can be a clockwise / counterclockwise difference of 10 degrees, 20 degrees, etc., and the preset directional threshold can be 10 degrees, 20 degrees, etc., without any specific limitation.

[0057] As an example, the first region is the region with a relatively small directional difference value from the currently divided region. When the directional difference value between the first region and the currently divided region is less than a preset directional threshold, they can be identified as regions with similar directions. Thus, all first regions with similar directions to the currently divided region in a certain image frame are selected. When multiple regions with similar directions exist in a certain frame of ultrasound image, it may be caused by the pattern of artifact generation. The more similar the directions are, and the more regular the positional distribution of regions with similar directions is, the more it conforms to the distribution characteristics of stripe artifacts.

[0058] Obtain the total number of all divided regions at the current scale, the first number of the first region, and the average value of the directional angles formed between each first region and the current divided region; Based on the first quantity, the total quantity, and the average angle, the overall distribution characteristics of the different divided regions are calculated.

[0059] As an example, there are mostly directional differences between the first region and the currently divided region. The average angle between the two directions can be determined by extracting the angle between them.

[0060] As an example, taking the currently divided region as region x, the overall distribution characteristics of region x are as follows. The calculation method can be: in, This represents the number of all first regions with similar orientations. This represents the total number of all regions divided at the current scale. This represents the average angle between each first region and the x-direction of region x.

[0061] As an example, the more similar the directions of region x are at the current scale, the more the directional distribution pattern of the stripe artifacts is present in the overall distribution characteristics. The smaller the angle between the directions of regions, the more regular the positional distribution of the regions is, and the more consistent it is with the artifact distribution characteristics.

[0062] The steps for calculating the internal features of each divided region include: Obtain the average edge gradient and edge length of different divided regions at the current scale; Based on the average edge gradient and edge length, the internal features of each divided region are calculated.

[0063] As an example, in smaller-scale region partitioning, the same artifact region may be divided into multiple regions. Therefore, when quantizing internal features based on edge gradients, overlapping edges between regions within the same artifact need to be excluded, as overlapping edges do not have distinguishing features. The edge with a larger gradient in each partitioned region is selected as a reference.

[0064] As an example, internal features The calculation method can be: in, These represent the average gradient of the edge and the edge length, respectively.

[0065] Based on the overall distribution characteristics, internal characteristics, and emphasis coefficients, the characteristic evaluation value of each divided region is calculated.

[0066] As an example, different scales reflect the different levels of artifact characteristics in ultrasound images. Larger scales are more similar to the image and retain tissue structure information, while smaller scales, i.e., the detail layer, focus on the internal information of the region. Therefore, based on the directionality of artifacts and the characteristic of artifacts presenting low-contrast edges in ultrasound images, combined with the emphasis coefficients at different scales and the overall distribution and internal characteristics of the region, the feature evaluation value of each divided region in the ultrasound image data at each scale is determined.

[0067] As an example, taking region x as the partitioned area, the feature evaluation value... The calculation method can be: in, This represents the feature evaluation value of region x at scale a. This represents the weighting coefficient of region x in the overall distribution at scale a. This represents the emphasis coefficient of features within pixel block x at scale a. This represents the overall distribution characteristics of region x; This represents the internal characteristics of region x.

[0068] As an example, the shift in attention from overall distribution characteristics to regional features at different scales, i.e., the change in the emphasis coefficient, yields the feature evaluation values ​​of different regions at each scale.

[0069] As an example, for different regions divided in each image frame of ultrasound image data, the feature evaluation value of the artifact is obtained at each scale based on the characteristics of the actual artifact. At the same scale, the feature evaluation value of the region where the artifact is located is larger than that of other divided regions in a certain image frame. However, between different scales, due to the difference in the focus of different scales, the feature evaluation value of the same artifact region is different.

[0070] Step S23: Based on the feature evaluation values, determine the comparison relationship between image frames at each scale; As an example, at a single scale, among multiple consecutive image frames of ultrasound image data, the more significant the quantification result of artifact features, that is, the more obvious the difference in the feature evaluation values ​​of the divided regions in the image frame, the more sensitive the current scale is to capturing the time-varying features of artifacts. Its weight in the attention mechanism should be greater, and it has a greater positive feedback on the artifact recognition effect: the weighted scale's recognition result of artifacts will further strengthen the model's attention to the features of that scale, thereby improving the overall accuracy of artifact detection.

[0071] As an example, it is necessary to analyze the dynamic comparison relationship of feature evaluation values ​​between image frames at a single scale. Based on the feature evaluation values ​​between frames, the comparison relationship between image frames at each scale is calculated to quantitatively evaluate the scale effectiveness and optimize the attention allocation strategy accordingly.

[0072] Step S23, which determines the comparison relationship between image frames at each scale based on feature evaluation values, includes: Select the maximum feature evaluation value among the feature evaluation values ​​of each image frame; Calculate the standard deviation of the feature evaluation values ​​of all divided regions within each image frame; As an example, the maximum feature evaluation value can be the maximum value among the feature evaluation values ​​of each divided region in a certain image frame.

[0073] For any given scale, the comparison relationship between image frames at the current scale is calculated based on the standard deviation and the slope of change of the maximum feature evaluation value of each adjacent image frame.

[0074] As an example, for several frames of ultrasound images, each segmented region in each frame has a feature evaluation value at a single scale. When the feature evaluation value of the segmented region fluctuates at that scale and the fluctuation returns to a certain value, the maximum value of the feature evaluation value in each frame is selected, and the trend of the maximum feature evaluation value in several frames is analyzed. When the change curve always shows fluctuations within a range and eventually returns to a certain value, the contrast relationship of artifact feature recognition at that scale is stronger.

[0075] As an example, the slope of change refers to the slope of change of the maximum feature evaluation value between two image frames. Taking a scale of 'a' and a region of 'x' as an example, this describes the comparison relationship between image frames at the current scale. The calculation method can be: In the formula, This represents the average of the standard deviations of the feature evaluation values ​​for all regions within each image frame. It represents the sum of the slopes of change of all two adjacent data points in a data sequence of the maximum feature evaluation value across multiple consecutive image frames.

[0076] As an example, the larger the standard deviation of the feature evaluation values ​​of all divided regions within each image frame, the greater the effect of feature recognition of artifacts at that scale; and in consecutive multi-frame images, the smaller the sum of the slopes of the changes of two adjacent data points on the line connecting the changes of the maximum feature evaluation value, the more it indicates that the fluctuation of the feature evaluation value has a certain range of fluctuation and will return to a fixed value, and the stronger the contrast relationship of artifact features at that scale.

[0077] Step S24: Determine the artifact sensitivity of the segmented image data at different scales based on the feature evaluation values ​​of the same segmented region at different scales and the comparison relationship.

[0078] As an example, in the process of learning artifact features in ultrasound images, the same artifact may have different feature evaluation values ​​at different scales. That is, the sensitivity to artifact features varies depending on the focus on different aspects of the same artifact's features. Since the region segmentation result at a large scale is based on the clustering result of superpixels at a small scale, for a certain region at a large scale, it is necessary to compare the feature evaluation values ​​of all regions it contains at a small scale to determine the artifact sensitivity at a certain scale. When the feature evaluation value of the same segmented region is larger at all scales, the sensitivity to artifact feature recognition at that scale is stronger, which means that the segmentation at that scale is more effective for sample learning. Based on the comparison relationship of feature evaluation values ​​of the same region at different scales, the artifact sensitivity at each scale is determined.

[0079] Step S24, which determines the artifact sensitivity of the segmented image data at different scales based on the feature evaluation values ​​and comparison relationships of the same segmented region at different scales, includes: For any divided region, obtain the mean value of the feature evaluation value of the current divided region at each scale, and use the mean value as the reference evaluation feature value; Determine the largest eigenvalue among the reference evaluation eigenvalues ​​across all scales; Based on the reference evaluation eigenvalue, the maximum eigenvalue, and the comparison relationship, the artifact sensitivity of each divided region at each scale is calculated.

[0080] As an example, taking scale a as an example, artifact sensitivity The calculation method can be: In the formula, among all scales, each region at the large scale is selected as the basis for analysis, and the mean of all feature evaluation values ​​of each divided region at each scale is used as its reference feature evaluation value. ; This represents the maximum value of the reference feature evaluation value across all scales. This indicates the comparison relationship between frames at scale a, and th() represents the normalization calculation.

[0081] As an example, the higher the reference feature evaluation value at a certain scale and the stronger its contrast relationship at a single scale, the greater its sensitivity to artifacts.

[0082] In this embodiment, by combining feature evaluation values ​​and the comparison relationship between each image frame, the artifact sensitivity at each scale is calculated, thereby determining the recognition sensitivity of the same divided region to artifact features at each scale, and thus determining the basis for the weight allocation of the attention mechanism.

[0083] Reference Figure 7 , Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0084] like Figure 7 As shown, the deep learning-based automatic ultrasound image artifact correction device may include: a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005.

[0085] Optionally, the deep learning-based automatic ultrasound image artifact correction device may also include a user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, a WiFi module, etc. The user interface may include a display screen and an input submodule such as a keyboard; optional user interfaces may also include standard wired or wireless interfaces. The network interface may include standard wired or wireless interfaces (such as a Wi-Fi interface).

[0086] Those skilled in the art will understand that Figure 7 The structure of the deep learning-based automatic ultrasound image artifact correction device shown does not constitute a limitation on the deep learning-based automatic ultrasound image artifact correction device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0087] like Figure 7 As shown, the memory 1005, serving as a storage medium, may include an operating system, a network communication module, and a deep learning-based automatic ultrasound image artifact correction program. The operating system is a program that manages and controls the hardware and software resources of the deep learning-based automatic ultrasound image artifact correction device, supporting the operation of the deep learning-based automatic ultrasound image artifact correction program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the deep learning-based automatic ultrasound image artifact correction system.

[0088] exist Figure 7 In the deep learning-based automatic ultrasound image artifact correction device shown, the processor 1001 is used to execute the deep learning-based automatic ultrasound image artifact correction program stored in the memory 1005 to implement the steps of the deep learning-based automatic ultrasound image artifact correction method described above.

[0089] The specific implementation of the ultrasound image artifact automatic correction device based on deep learning in this application is basically the same as the embodiments of the ultrasound image artifact automatic correction method based on deep learning described above, and will not be repeated here.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0091] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0093] The above are merely preferred embodiments of this application and do not limit the scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of this application.

[0094] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0095] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for automatic correction of ultrasound image artifacts based on deep learning, characterized in that, The method includes: Acquire the ultrasound image data to be tested; Based on the feature evaluation values ​​of each divided region in the ultrasound image data, the artifact sensitivity of the ultrasound image data at different scales is calculated. The artifact sensitivity is used to characterize the sensitivity to the recognition of artifact features at different scales. Based on the artifact sensitivity, an attention map corresponding to the artifact region in the ultrasound image data is generated. The ultrasound image data and the attention map are input into a preset recognition model. Based on the preset recognition model, the feature map corresponding to the ultrasound image data and the attention map are weighted and combined to obtain a weighted feature map. The artifact recognition results of the preset recognition model are corrected using the weighted feature map.

2. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 1, characterized in that, Before calculating the artifact sensitivity of the ultrasound image data at different scales based on the feature evaluation values ​​of each divided region in the ultrasound image data, the method further includes: The ultrasound image data is divided into superpixels for each image frame to obtain multiple superpixels in each image frame. Each superpixel is clustered according to different scales to obtain segmented image data at multiple scales, including macro scale, meso scale and micro scale. The calculation of artifact sensitivity of the ultrasound image data at different scales based on the feature evaluation values ​​of each divided region in the ultrasound image data includes: Based on the feature evaluation values ​​of each segmented region in the segmented image data, the artifact sensitivity of the segmented image data at different scales is calculated.

3. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 2, characterized in that, The step of calculating the artifact sensitivity of the segmented image data at different scales based on the feature evaluation values ​​of each segmented region in the segmented image data includes: Based on the number of regions in the segmented image data at each scale and the area difference between the regions, the emphasis coefficient at each scale is calculated. For any given scale, based on the emphasis coefficient and the artifact features of each of the divided regions, the feature evaluation value of each divided region at the current scale is calculated. Based on the feature evaluation values, the comparison relationship between image frames at each scale is determined; Based on the feature evaluation values ​​of the same divided region at different scales and the comparison relationship, the artifact sensitivity of the divided image data at different scales is determined.

4. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 3, characterized in that, The calculation of feature evaluation values ​​for each segmented region at the current scale, based on the emphasis coefficient and the artifact features of each segmented region, includes: Calculate the overall distribution characteristics of different divided regions at the current scale, and calculate the internal characteristics of each divided region; Based on the overall distribution characteristics, the internal characteristics, and the emphasis coefficient, the feature evaluation value of each of the divided regions is calculated.

5. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 4, characterized in that, The calculation of the overall distribution characteristics of different divided regions at the current scale includes: Obtain the first direction of each of the divided regions; For any divided region, a first region in the image frame whose directional difference value with the currently divided region is less than a preset directional threshold is selected; Obtain the total number of all divided regions at the current scale, the first number of the first region, and the average value of the directional angles formed between each of the first regions and the current divided region; Based on the first quantity, the total quantity, and the average angle, the overall distribution characteristics of the different divided regions are calculated.

6. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 4, characterized in that, The calculation of the internal features of each of the divided regions includes: Obtain the average edge gradient and edge length of different divided regions at the current scale; Based on the average edge gradient and the edge length, the internal features of each of the divided regions are calculated.

7. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 3, characterized in that, Determining the comparison relationship between image frames at each scale based on the feature evaluation value includes: Select the maximum feature evaluation value among the feature evaluation values ​​of each image frame; Calculate the standard deviation of the feature evaluation values ​​of all divided regions within each image frame; For any given scale, the comparison relationship between image frames at the current scale is calculated based on the standard deviation and the slope of change of the maximum feature evaluation value of each adjacent image frame.

8. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 3, characterized in that, The step of determining the artifact sensitivity of the segmented image data at different scales based on the feature evaluation values ​​of the same segmented region at different scales and the comparison relationship includes: For any of the divided regions, the mean value of the feature evaluation value of the current divided region at each scale is obtained, and the mean value is used as a reference evaluation feature value. Determine the largest eigenvalue among the reference evaluation eigenvalues ​​across all scales; Based on the reference evaluation feature value, the maximum feature value, and the comparison relationship, the artifact sensitivity of each divided region at each scale is calculated.

9. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 1, characterized in that, The step of weightedly combining the feature map corresponding to the ultrasound image data and the attention map based on the preset recognition model to obtain a weighted feature map includes: Based on the attention mechanism of the preset recognition model, the ultrasound image data is processed to obtain the original feature map; The original feature map and the attention map are fused according to a preset weight ratio to obtain a weighted feature map.

10. The method for automatic correction of ultrasound image artifacts based on deep learning as described in claim 1, characterized in that, The step of correcting the artifact recognition result of the preset recognition model using the weighted feature map includes: The weighted feature map is input into the attention module, and the attention module is inserted between the network layers of the preset recognition model; The attention module performs artifact recognition processing on the weighted feature map and outputs the artifact recognition result.

Images