Rib shielding ultrasonic image analysis method and system

Abstract

The invention provides a rib occlusion ultrasonic image analysis method and system. The method comprises the following steps: determining scale feature maps of a target occlusion region in an ultrasonic image of the target occlusion region at different angles under different expansion convolution kernels; determining artifact residual errors of occlusion artifacts in the target occlusion area according to texture differences among the scale feature maps; according to the artifact residual error and the feature descriptor of each scale feature map after noise suppression, determining the positioning contribution degree of each scale feature map to the sound shadow area in the target shielding area; extracting the recognition attention of the shielding gap in the target shielding region from each scale feature map, and determining an enhanced feature map of the shielding gap in the target shielding region through all the recognition attention; and performing shielding compensation on a to-be-enhanced region in the target shielding region through the enhanced feature map and all the positioning contribution degrees to obtain a compensation map of the to-be-enhanced region. By adopting the scheme of the invention, the influence of the sound image artifacts on the analysis of the rib-shielded ultrasonic image can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of ultrasonic image analysis, and more specifically, to a method and system for analyzing ultrasonic images with rib occlusion. Background Art

[0002] Ultrasonic image analysis is a technology that uses ultrasonic technology to obtain images of the internal structure of an object and then processes and interprets them. It is widely used in medical diagnosis (such as abdominal organs, cardiovascular, and obstetrics and gynecology examinations) and industrial non-destructive testing. Its core lies in extracting the structural characteristics or pathological characteristics of the target area by analyzing the grayscale distribution, texture characteristics, boundary contours and other information in the ultrasound image.

[0003] In medical imaging diagnosis, ultrasound imaging is widely used for examination of multiple organs such as the heart, liver, gallbladder, and lungs due to its advantages such as being radiation-free, highly real-time, and low-cost. However, due to the obstruction of rib structures, especially in the chest region, the propagation path of ultrasound waves is often affected by reflection and attenuation from bony structures, resulting in the so-called "acoustic shadow artifact" phenomenon. These artifacts usually appear as abrupt dark bands or areas of information loss in the image, which seriously interfere with the identification and diagnosis of target tissues (such as the lung parenchyma and pleura), reducing the usability and diagnostic accuracy of ultrasound images. Traditional image enhancement or artifact removal methods are mostly based on empirical models or image smoothing processing, and are often unable to accurately distinguish the boundary between tissue texture and occlusion artifacts, which leads to the loss or incorrect enhancement of important lesion information, and the recognition ability of small areas such as the intercostal space is weak. Therefore, how to reduce the impact of acoustic shadow artifacts on ultrasound image analysis of rib occlusion has become a problem facing the industry. Summary of the Invention

[0004] The present application provides a method and system for analyzing an ultrasound image with rib occlusion, which can reduce the impact of acoustic shadow artifacts on the analysis of ultrasound images with rib occlusion.

[0005] In a first aspect, the present application provides a method for analyzing rib-occluded ultrasound images, comprising the following steps: Obtain ultrasound images of the target occluded area at different angles, and then determine the scale feature map of the target occluded area in each ultrasound image under different dilated convolution kernels; Determine the artifact residual of the occlusion artifact in the target occluded area according to the texture difference between the feature maps at each scale; Determining the contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area according to the artifact residual and the feature descriptor of each scale feature map after noise suppression; Based on the attention mechanism, the recognition attention of each scale feature map to the occlusion gap in the target occlusion area is extracted. The occlusion gap in the target occlusion area is coupled and enhanced through all the recognition attentions to obtain the enhanced feature map of the occlusion gap in the target occlusion area. Determine the area to be enhanced in the target occluded area, and perform occlusion compensation on the area to be enhanced in the target occluded area using the enhanced feature map and all positioning contributions to obtain a compensation map of the area to be enhanced.

[0006] In some embodiments, determining the scale feature map of the target occluded region in each ultrasound image under different dilated convolution kernels specifically includes: Preprocessing each ultrasound image to obtain a preprocessed image of each ultrasound image; Select a preprocessed image as the selected preprocessed image, perform layer-by-layer convolution on the selected preprocessed image, and obtain feature maps with different expansion rates; The scale feature map of the target occluded area in the ultrasound image corresponding to the selected preprocessed image under different dilated convolution kernels is constructed through all feature maps; Continue to determine the scale feature maps of the target occluded area in the remaining ultrasound image under different dilated convolution kernels.

[0007] In some embodiments, determining the artifact residual of the occlusion artifact in the target occlusion area according to the texture difference between the scale feature maps specifically includes: Determine the texture differences between feature maps at each scale; All texture differences are fused to obtain texture fusion differences; An artifact residual of an occlusion artifact in the target occlusion area is determined according to the texture fusion difference.

[0008] In some embodiments, determining the contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area according to the artifact residual and the feature descriptor of each scale feature map after noise suppression specifically includes: Determine the feature descriptor of each scale feature map after noise suppression; Associating each feature descriptor with the artifact residual to obtain descriptor association information of the target occluded area; The contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area is determined according to the descriptor association information.

[0009] In some embodiments, extracting the recognition attention of each scale feature map to the occlusion gap in the target occlusion area based on the attention mechanism specifically includes: Selecting a scale feature map as a selected scale feature map to determine an occlusion gap in the target occlusion area; Determining an attention weight of the selected scale feature map for the occlusion gap based on an attention mechanism; Determining, according to the attention weight, the recognition attention of the selected scale feature map to the occlusion gap in the target occlusion area; The remaining scale feature maps are then used to determine the recognition attention of the occlusion gaps in the target occluded region.

[0010] In some embodiments, coupling enhancement is performed on the occlusion gap in the target occlusion region by all recognition attentions to obtain an enhanced feature map of the occlusion gap in the target occlusion region, specifically comprising: determining a potential region of an occlusion gap in the target occluded region based on the artifact residual; All recognition attentions are correlated and fused to obtain a fused attention map; The occlusion gaps in the target occlusion area are enhanced according to the potential area and the fused attention map to obtain an enhanced feature map of the occlusion gaps in the target occlusion area.

[0011] In some embodiments, performing occlusion compensation on the area to be enhanced in the target occlusion area by using the enhanced feature map and all positioning contributions to obtain the compensation map of the area to be enhanced specifically includes: The enhanced feature map is enhanced based on all positioning contributions to obtain an enhanced feature map; Performing fusion compensation on the area to be enhanced in the target occluded area according to the enhanced feature map to obtain a compensated feature map; The compensation map of the area to be enhanced is determined by using the compensation feature map.

[0012] In some embodiments, ultrasound images of the target occluded area at different angles are collected by a color Doppler ultrasound diagnostic apparatus.

[0013] In some embodiments, a deep learning-based denoising model performs noise suppression on each scale feature map.

[0014] In a second aspect, the present application provides a rib-occluded ultrasound image analysis system, comprising: An acquisition module is used to acquire ultrasound images of the target occluded area at different angles, and then determine the scale feature map of the target occluded area in each ultrasound image under different dilated convolution kernels; a processing module for determining an artifact residual of an occlusion artifact in an occluded target area based on texture differences between feature maps at each scale; The processing module is further configured to determine, based on the artifact residual and a feature descriptor of each scale feature map after noise suppression, a contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area; The processing module is further configured to extract the recognition attention of each scale feature map to the occlusion gap in the target occlusion area based on the attention mechanism, and perform coupled enhancement on the occlusion gap in the target occlusion area through all the recognition attentions to obtain an enhanced feature map of the occlusion gap in the target occlusion area; An execution module is used to determine the area to be enhanced in the target occlusion area, and perform occlusion compensation on the area to be enhanced in the target occlusion area through the enhanced feature map and all positioning contributions to obtain a compensation map of the area to be enhanced.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: In the ultrasonic image analysis method and system for rib occlusion provided in the present application, first, ultrasonic images of the target occluded area are obtained at different angles, and then the scale feature maps of the target occluded area under different dilated convolution kernels in each ultrasonic image are determined; the artifact residual of the occlusion artifact in the target occluded area is determined according to the texture difference between each scale feature map; the positioning contribution of each scale feature map to the acoustic shadow area in the target occluded area is determined according to the artifact residual and the feature descriptor of each scale feature map after noise suppression; based on the attention mechanism, the recognition attention of each scale feature map to the occlusion gap in the target occluded area is extracted, and the occlusion gap in the target occluded area is coupled and enhanced through all the recognition attentions to obtain an enhanced feature map of the occlusion gap in the target occluded area; the area to be enhanced in the target occluded area is determined, and the occlusion compensation of the area to be enhanced in the target occluded area is performed through the enhanced feature map and all the positioning contributions to obtain a compensation map of the area to be enhanced.

[0016] It can be seen that in the ultrasound image recognition process of this application, first, the multi-scale feature extraction technology is used to capture the tissue structure characteristics at different resolutions, so as to determine the scale feature map of the target occluded area in each ultrasound image under different dilated convolution kernels, which not only retains the overall morphological characteristics of the large-scale acoustic shadow area, but also obtains the local texture information of the fine structure, providing a hierarchical feature basis for artifact recognition; then, the texture consistency difference between multi-scale features is used to construct a residual mapping, determine the artifact residual, effectively distinguish the real tissue structure from the acoustic artifact (such as the acoustic shadow behind the ribs), and improve the recognition accuracy of the artifact area through nonlinear feature separation; then, combined with the robust features after residual analysis and noise reduction processing, the representation ability of different scale features on the acoustic shadow area is quantified, and the characterization of each scale feature map on the target occluded area is determined. The localization contribution of the acoustic shadow area is determined by establishing a regional weight allocation mechanism based on credibility, thereby reducing the misleading effect of noise interference on the analysis of the occluded area. Furthermore, a channel-spatial dual attention mechanism is used to focus on sound-transmissible areas such as the intercostal space, thereby enhancing the significance of the low-echo effective signal, suppressing background noise in non-critical areas, improving the feature discrimination at the occlusion boundary, and realizing cross-modal fusion of multi-angle feature maps. Attention-guided feature enhancement is used to highlight the weak echo characteristics of the occluded tissue, namely: an enhanced feature map is obtained by coupling enhancement through recognition attention, compensating for the signal attenuation caused by acoustic shadowing, and reconstructing more complete anatomical structure information. Finally, the enhanced feature map and all localization contributions are used to compensate for the occlusion of the target area to be enhanced, obtaining a compensated map of the area to be enhanced. The above scheme can reduce the impact of acoustic shadow artifacts on ultrasound image analysis of rib occlusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an exemplary flow chart of a method for analyzing rib-occluded ultrasound images according to some embodiments of the present application; Figure 2 is a schematic diagram of a scale characteristic map according to some embodiments of the present application; Figure 3 is a schematic diagram of a process for determining artifact residuals according to some embodiments of the present application; Figure 4 is a schematic structural diagram of an ultrasound image analysis system with rib occlusion according to some embodiments of the present application; Figure 5 It is a structural schematic diagram of a computer device for implementing a rib-occluded ultrasound image analysis method according to some embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the technical solution of the present application, the technical solution of the present application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0019] refer to Figure 1 , which is an exemplary flow chart of a method for analyzing an ultrasound image with rib occlusion according to some embodiments of the present application. The method 100 for analyzing an ultrasound image with rib occlusion mainly includes the following steps: In step 101, ultrasonic images of the target occluded area at different angles are obtained, and then the scale feature map of the target occluded area in each ultrasonic image under different dilated convolution kernels is determined.

[0020] In specific implementation, obtaining ultrasound images of the target occluded area at different angles can be achieved in the following manner, namely: using ultrasound equipment (such as a color Doppler ultrasound diagnostic instrument) to collect ultrasound images from different angles of the area where the target is occluded. For example, an ultrasound image is collected at a certain angle (such as 10 degrees, 15 degrees) to ensure that the collected ultrasound image can fully reflect the situation of the target occluded area. In other embodiments, other methods can also be used for collection, which are not limited here.

[0021] It should be noted that the ultrasound images in this application use grayscale mode to display information on the morphology, structure and positional relationship of internal tissues and organs of the human body. Different tissues have different reflection, refraction and absorption characteristics of ultrasound, which are manifested as different grayscale levels in the image.

[0022] In some embodiments, determining the scale feature map of the target occluded region in each ultrasound image under different dilated convolution kernels can be achieved by using the following steps: Preprocessing each ultrasound image to obtain a preprocessed image of each ultrasound image; Select a preprocessed image as the selected preprocessed image, perform layer-by-layer convolution on the selected preprocessed image, and obtain feature maps with different expansion rates; The scale feature map of the target occluded area in the ultrasound image corresponding to the selected preprocessed image under different dilated convolution kernels is constructed through all feature maps; Continue to determine the scale feature maps of the target occluded area in the remaining ultrasound image under different dilated convolution kernels.

[0023] In specific implementation, first, a Gaussian filtering algorithm is used to filter each ultrasound image to obtain a preprocessed image of each ultrasound image, and salt and pepper noise and Gaussian noise in each ultrasound image are removed. Then, a network structure containing convolution layers with different dilation rates (such as 1, 2, and 3) is constructed through a parallel dilated convolution module (ASPP style), and the selected preprocessed image is input into the network structure with different dilation rates in turn to generate feature maps with different dilation rates, wherein the feature map represents the features of the image at different levels of abstraction (i.e., color, brightness, simple edges). Subsequently, tensor splicing in a deep learning framework is used to splice the feature maps at different dilation rates according to the channel dimension, and the spliced map is used as the scale feature map of the target occluded area in the ultrasound image corresponding to the selected preprocessed image under different dilated convolution kernels; in other embodiments, other methods can also be used for determination, which is not limited here.

[0024] It should be noted that the scale feature map in this application represents the feature mapping map of the target occluded area of the ultrasound image at different scales. The scale feature map is an abstract and compressed representation of the ultrasound image features, which converts the pixel information in the ultrasound image into a feature vector with semantic meaning. These feature vectors can represent various features of the occluded area in the image, such as the shape, texture, and color of the object, and can be used to analyze the occluded area.

[0025] In some embodiments, reference Figure 2 As shown in FIG, this figure is a schematic diagram of a scale feature map in some embodiments of the present application, such as Figure 2 As described above, taking the rib occlusion as an example, the white part in the figure is the rib part (ie, the occlusion part), and the black part is the rib gap part (ie, the occlusion gap).

[0026] In step 102, an artifact residual of an occlusion artifact in a target occlusion region is determined based on the texture difference between the scale feature maps; In some embodiments, reference Figure 3 As shown in FIG, this figure is a schematic diagram of a process for determining artifact residuals in some embodiments of the present application. In this embodiment, the artifact residuals of occlusion artifacts in the target occlusion area are determined based on the texture differences between the various scale feature maps, which can be implemented by the following steps: First, in step 1021, the texture differences between the scale feature maps are determined; Next, in step 1022, all texture differences are fused to obtain texture fusion differences; Finally, in step 1023, an artifact residual of the occlusion artifact in the target occlusion area is determined based on the texture fusion difference.

[0027] In a specific implementation, first, a feature extraction model based on deep learning (such as VGG, ResNet) is used to extract texture features of each scale feature map, and the Euclidean distance method is used to combine the texture features of each scale feature map to calculate the texture difference between the scale feature maps, wherein the texture difference represents the difference in texture between the scale feature maps; secondly, in order to avoid the limitations of a single difference metric, the texture differences obtained by different methods are fused. In some embodiments, a weighted average method is used to fuse all texture differences, and the fused result is used as a texture fusion difference, wherein the texture fusion difference includes the weights of each texture difference; then, a texture difference threshold is set by combining historical texture difference data through a cross-validation method, and each weight in the texture fusion difference is compared with the texture difference threshold, and each weight greater than the texture difference threshold is extracted, and each weight is normalized, and the result obtained by the normalization is used as the artifact residual of the occlusion artifact in the target occlusion area; in other embodiments, other methods can also be used for determination, which is not limited here.

[0028] It should be noted that the artifact residual in this application represents the difference between the artifacts generated by occlusion in the ultrasound image and the ideal artifact-free state, and can be used to analyze the occluded area in the ultrasound image.

[0029] In step 103, the contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area is determined based on the artifact residual and the feature descriptor of each scale feature map after noise suppression.

[0030] In some embodiments, determining the contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area based on the artifact residual and the feature descriptor of each scale feature map after noise suppression can be achieved by using the following steps: Determine the feature descriptor of each scale feature map after noise suppression; Associating each feature descriptor with the artifact residual to obtain descriptor association information of the target occluded area; The contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area is determined according to the descriptor association information.

[0031] It should be noted that the artifact residual represents the difference between the ultrasound image and the normal area due to occlusion, which can highlight the location and degree of the possible acoustic shadow area. The feature descriptor of each scale feature map after noise suppression contains feature information at different levels of the image, such as texture, shape, and edge. Feature maps of different scales can capture the features of different details and ranges of the acoustic shadow area. Feature maps with larger scales may be better at describing the overall structure of the acoustic shadow area, while feature maps with smaller scales may be more sensitive to the local details of the acoustic shadow area. By associating the artifact residual with the feature descriptors of each scale feature map, the degree of closeness between the features of each scale feature map and the acoustic shadow area indicated by the artifact residual can be determined. The higher the degree of closeness, the more effective information the scale feature map provides when identifying the acoustic shadow area, and the greater its contribution to the positioning of the acoustic shadow area.

[0032] In a specific implementation, determining the feature descriptor of each scale feature map after noise suppression can be achieved in the following manner, namely: performing noise suppression on each scale feature map based on a deep learning-based denoising model (such as DnCNN), and extracting the feature descriptor of each scale feature map after noise suppression through a scale-invariant feature transformation algorithm, wherein the feature descriptor represents the features around the grayscale extreme points in the scale feature map, such as texture, shape, color, and gradient; in other embodiments, other methods can also be used for determination, which is not limited here.

[0033] In specific implementation, each feature descriptor is associated with the artifact residual to obtain the descriptor association information of the target occlusion area, which can be achieved in the following manner, namely: the correlation value of each feature descriptor and the artifact residual is calculated by the Pearson correlation coefficient, and a weight is assigned to each feature descriptor according to the calculated correlation value. For example, the weight of the feature descriptor with a correlation higher than the threshold is set to a higher value, and the weight of the feature descriptor with a low correlation is given a smaller weight, so that the feature descriptor with a high correlation with the artifact residual obtains a greater weight, and a weighted operation is performed on each feature descriptor, and the values of each dimension of each feature descriptor are multiplied by the corresponding weight, and the weighted feature descriptors are spliced according to the channel dimension, and the spliced vector is used as the descriptor association information of the target occlusion area, wherein the descriptor association information represents the information obtained after associating the feature descriptor domain artifact residual; in other embodiments, other methods can also be used for determination, which is not limited here.

[0034] In a specific implementation, determining the contribution of each scale feature map to the positioning of the sound shadow area in the target occlusion area according to the descriptor association information can be achieved in the following manner, namely: calculating the variance of each weighted feature descriptor in the descriptor association information, the larger the variance, the richer the feature information in the scale feature map, normalizing the variance of each weighted feature descriptor, and using the values obtained by the normalization as the contribution of the corresponding scale feature map to the positioning of the sound shadow area in the target occlusion area; in other embodiments, other methods can also be used for determination, which are not limited here.

[0035] It should be noted that the positioning contribution in this application represents the parameter value of the contribution of the scale feature map to the positioning of the acoustic shadow area in the target occlusion area, and can be used to identify the rib occlusion area.

[0036] In step 104, based on the attention mechanism, the recognition attention of each scale feature map to the occlusion gap in the target occlusion area is extracted, and the occlusion gap in the target occlusion area is coupled and enhanced through all the recognition attentions to obtain an enhanced feature map of the occlusion gap in the target occlusion area.

[0037] In some embodiments, extracting the attention of each scale feature map to the occlusion gap in the target occlusion area based on the attention mechanism can be achieved by the following steps: Selecting a scale feature map as a selected scale feature map to determine an occlusion gap in the target occlusion area; Determining an attention weight of the selected scale feature map for the occlusion gap based on an attention mechanism; Determining, according to the attention weight, the recognition attention of the selected scale feature map to the occlusion gap in the target occlusion area; The remaining scale feature maps are then used to determine the recognition attention of the occlusion gaps in the target occluded region.

[0038] In a specific implementation, first, a semantic segmentation model based on deep learning (such as U-Net) is used to identify occlusion gaps in the target occlusion area from all ultrasound images, where the occlusion gap represents a portion in the target occlusion area that is not completely covered by the occlusion object, such as a gap or hole in the target itself, or a gap between the occlusion object and the target. Then, a spatial and channel dual attention mechanism is constructed: the spatial attention branch applies a convolution operation to the selected scale feature map and generates a spatial weight matrix in combination with a binary mask of the gap position of the occlusion gap. The channel attention branch uses global average pooling to compress the feature dimension of the pixels in the selected scale feature map, and generates a channel weight vector by combining the compressed selected scale feature map through a two-layer fully connected network. Then, the result of multiplying the spatial weight matrix and the channel weight vector is used as the attention weight of the selected scale feature map for the occlusion gap. Finally, the attention weight is element-wise multiplied with the occlusion gap area in the selected scale feature map, and the multiplied image is used as the recognition attention of the selected scale feature map for the occlusion gap in the target occlusion area. In other embodiments, other methods can also be used for determination, which is not limited here.

[0039] It should be noted that the attention weight is multiplied element-by-element with the original feature map. In the spatial dimension, the higher weight value of the attention weight corresponding to the occlusion gap area is multiplied with the element at the same position in the selected scale feature map, so that the features of the occlusion gap area are enhanced, while the feature responses of other areas are relatively weakened due to their lower weight values. In the channel dimension, the channel weight vector is multiplied correspondingly with each channel of the selected scale feature map, highlighting the feature channels that play an important role in identifying occlusion gaps and suppressing the feature responses of irrelevant channels.

[0040] In addition, it should be noted that the recognition attention in this application represents the relevant feature map of the occlusion gap in the target occlusion area, which can be used to analyze the occlusion gap in the occlusion area, thereby improving the recognition accuracy of the occluded target.

[0041] In some embodiments, coupling enhancement of the occlusion gaps in the target occlusion region by all recognition attentions to obtain an enhanced feature map of the occlusion gaps in the target occlusion region can be achieved by the following steps: determining a potential region of an occlusion gap in the target occluded region based on the artifact residual; All recognition attentions are correlated and fused to obtain a fused attention map; The occlusion gaps in the target occlusion area are enhanced according to the potential area and the fused attention map to obtain an enhanced feature map of the occlusion gaps in the target occlusion area.

[0042] It should be noted that the artifact residual determines the area where occlusion gaps may exist, providing a focus range for recognition attention. Within this range, recognition attention captures the characteristics of occlusion gaps more carefully by processing feature maps of different scales, so that the occlusion gap features in these potential areas are further enhanced. By utilizing the advantages of both, the occlusion gap features are deeply mined and strengthened, thereby obtaining an enhanced feature map of each occlusion gap in the target occlusion area.

[0043] In a specific implementation, a deep learning-based segmentation model (such as U-Net) is combined with artifact residuals to identify the occlusion gaps in the target occlusion area, thereby obtaining the potential area of the occlusion gaps in the target occlusion area. For example: the segmentation model is trained, and the manually labeled potential area of the occlusion gaps is used as a label. The cross-entropy loss function is used to calculate the difference between the predicted result and the true label. The network parameters are updated through the back propagation algorithm and stochastic gradient descent, so that the segmentation model learns the mapping relationship between the artifact residuals and the potential area of the occlusion gaps. After the training is completed, the artifact residuals are input to the trained segmentation model, and the segmentation model outputs a predicted binary mask image, in which the area with a pixel value of 1 is the potential area of the occlusion gaps in the identified target occlusion area; in other embodiments, other methods can also be used for determination, which is not limited here.

[0044] In specific implementation, all recognition attentions are associated and fused to obtain a fused attention map, which can be achieved in the following manner: first, the bilinear interpolation method is used to adjust each recognition attention to the same resolution to ensure that they are comparable in spatial dimensions; then, all the adjusted recognition attentions are fused by weighted summation through the fusion strategy of the feature pyramid network (FPN), and the fused map is used as the fused attention map, wherein the fused attention map represents the attention information of the occlusion gap in the target occlusion area after the fusion of recognition attentions of different scales, which can be used to analyze the occlusion gap in the target occlusion area; in other embodiments, other fusion methods can also be used, which are not limited here.

[0045] In a specific implementation, the occlusion gaps in the target occlusion area are enhanced according to the potential area and the fused attention map, and the enhanced feature map of the occlusion gaps in the target occlusion area is obtained, which can be implemented in the following manner, namely: element-by-element multiplication of the fused attention map and the mask of the potential area is performed so that the fused attention map is effective only in the potential area and focuses on the area where the occlusion gaps are located; subsequently, the convolutional layer in the convolutional neural network (CNN) can be used to enhance the multiplied map, and the enhanced map is used as the enhanced feature map of the occlusion gaps in the target occlusion area; in other embodiments, other enhancement methods can also be used, which are not limited here.

[0046] It should be noted that the enhanced feature map in this application represents a feature map after the occlusion gaps in the target occlusion area are enhanced, which can be used to analyze the occlusion gaps in the target occlusion area, reducing the interference of other irrelevant information, and providing a more targeted and effective feature expression for subsequent image analysis, target recognition, and image restoration tasks.

[0047] In step 105, the area to be enhanced in the target occluded area is determined, and occlusion compensation is performed on the area to be enhanced in the target occluded area using the enhanced feature map and all positioning contributions to obtain a compensation map of the area to be enhanced.

[0048] In specific implementation, the following method can be used to determine the area to be enhanced in the target occluded area, namely: a global threshold method (such as the Otsu algorithm) can be used to find a threshold that can maximize the difference between foreground and background pixels, and the value in the artifact residual is compared with the set threshold. The value greater than the threshold is marked in the corresponding area in the ultrasound image, and the marked area is used as the area to be enhanced. In other embodiments, other methods can also be used for determination, which are not limited here.

[0049] It should be noted that the area to be enhanced in the present application refers to the area where the occluded part in the target occluded area needs to be enhanced, which can be used to repair the occluded part in the target occluded area to obtain a complete ultrasound image.

[0050] In some embodiments, occlusion compensation is performed on the area to be enhanced in the target occlusion area using the enhanced feature map and all positioning contributions, and obtaining a compensation map for the area to be enhanced can be achieved by the following steps: The enhanced feature map is enhanced based on all positioning contributions to obtain an enhanced feature map; Performing fusion compensation on the area to be enhanced in the target occluded area according to the enhanced feature map to obtain a compensated feature map; The compensation map of the area to be enhanced is determined by using the compensation feature map.

[0051] It should be noted that by weighting the enhanced feature map according to the positioning contribution, the features closely related to occlusion compensation can be enhanced in a targeted manner, and the influence of these features in subsequent processing can be improved. Based on the enhanced feature map, combined with image restoration and other technologies, the area to be enhanced in the target occluded area is fused and compensated, and the enhanced occlusion gap features are used to fill the part missing due to occlusion. Finally, through edge detection, threshold segmentation and other methods, clear contour information is extracted from the compensated feature map, and the compensation map of the area to be enhanced is obtained.

[0052] In the specific implementation, first, each positioning contribution is used as a weight to multiply the eigenvalue of the corresponding channel of the enhanced feature map, and the multiplied map is used as the enhanced feature map, and a larger weight is given to the positioning contribution with a high degree of weight, so as to enhance the key features related to the occlusion gap; then, an image restoration network based on deep learning (such as Partial Convolutional Networks) can be used to fuse the enhanced feature map with the area to be enhanced, and the fused map is compensated through convolution and deconvolution operations, and the compensated map is used as the compensated feature map. Finally, the outline of the area to be enhanced is identified from the compensated feature map through the threshold segmentation method, and then the continuity and integrity of the outline are optimized through morphological operations (such as expansion and corrosion), and finally the identified and optimized map is used as the compensation map of the area to be enhanced; in other embodiments, other methods can also be used for determination, which is not limited here.

[0053] It should be noted that the compensation map in this application represents a map of the compensation degree of the area to be enhanced in the target occlusion area, which can be used to identify the occlusion part of the target occlusion area, thereby repairing the target occlusion area.

[0054] In addition, in another aspect of the present application, in some embodiments, the present application provides an ultrasound image analysis system for rib occlusion, referring to Figure 4 This figure is a schematic structural diagram of an ultrasound image analysis system with rib occlusion according to some embodiments of the present application. The ultrasound image analysis system with rib occlusion 400 includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described as follows: Acquisition module 401, in this application, acquisition module 401 is mainly used to acquire ultrasound images of the target occlusion area at different angles, and then determine the scale feature map of the target occlusion area in each ultrasound image under different dilated convolution kernels; Processing module 402, in this application, the processing module 402 is used to determine the artifact residual of the occlusion artifact in the target occlusion area according to the texture difference between each scale feature map; It should be noted that the processing module 402 in the present application is further configured to determine the contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area based on the artifact residual and the feature descriptor of each scale feature map after noise suppression; It should be noted that the processing module 402 in the present application is further configured to extract the recognition attention of each scale feature map to the occlusion gap in the target occlusion area based on the attention mechanism, and couple-enhance the occlusion gap in the target occlusion area through all the recognition attentions to obtain an enhanced feature map of the occlusion gap in the target occlusion area; Execution module 403, in this application, execution module 403 is mainly used to determine the area to be enhanced in the target occlusion area, and perform occlusion compensation on the area to be enhanced in the target occlusion area through the enhanced feature map and all positioning contributions to obtain a compensation map of the area to be enhanced.

[0055] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores code, and the processor is configured to obtain the code and execute the above-mentioned rib-occluded ultrasound image analysis method.

[0056] In some embodiments, reference Figure 5 , which is a schematic diagram of the structure of a computer device for implementing the rib-occluded ultrasound image analysis method according to some embodiments of the present application. The rib-occluded ultrasound image analysis method in the above embodiment can be achieved by Figure 5 The computer device 500 shown in FIG. 5 is implemented as shown in FIG. 5 . The computer device 500 includes at least one processor 501 , a communication bus 502 , a memory 503 , and at least one communication interface 504 .

[0057] The processor 501 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of the ultrasound image analysis method for rib occlusion in the present application.

[0058] The communication bus 502 may be used to transmit information between the aforementioned components.

[0059] The memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 503 may be independent and connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0060] The memory 503 is used to store program code for executing the solution of the present application, and is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The method described in the above method embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0061] The communication interface 504 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0062] In a specific implementation, as an example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0063] The aforementioned computer device can be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of this application do not limit the type of computer device.

[0064] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned rib-occluded ultrasound image analysis method.

[0065] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0066] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for analyzing rib-occluded ultrasound images, characterized in that: The steps include: Obtain ultrasound images of the target occluded area at different angles, and then determine the scale feature map of the target occluded area in each ultrasound image under different dilated convolution kernels; Determine the artifact residual of the occlusion artifact in the target occluded area according to the texture difference between the feature maps at each scale; Determining the contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area according to the artifact residual and the feature descriptor of each scale feature map after noise suppression; Based on the attention mechanism, the recognition attention of each scale feature map to the occlusion gap in the target occlusion area is extracted. The occlusion gap in the target occlusion area is coupled and enhanced through all the recognition attentions to obtain the enhanced feature map of the occlusion gap in the target occlusion area. Determine the area to be enhanced in the target occluded area, and perform occlusion compensation on the area to be enhanced in the target occluded area using the enhanced feature map and all positioning contributions to obtain a compensation map of the area to be enhanced.

2. The method according to claim 1, wherein Determining the scale feature map of the target occluded area in each ultrasound image under different dilated convolution kernels specifically includes: Preprocessing each ultrasound image to obtain a preprocessed image of each ultrasound image; Select a preprocessed image as the selected preprocessed image, perform layer-by-layer convolution on the selected preprocessed image, and obtain feature maps with different expansion rates; The scale feature map of the target occluded area in the ultrasound image corresponding to the selected preprocessed image under different dilated convolution kernels is constructed through all feature maps; Continue to determine the scale feature maps of the target occluded area in the remaining ultrasound image under different dilated convolution kernels.

3. The method according to claim 1, wherein The artifact residual of the occlusion artifact in the target occlusion area is determined based on the texture difference between the feature maps of each scale, specifically including: Determine the texture differences between feature maps at each scale; All texture differences are fused to obtain texture fusion differences; An artifact residual of an occlusion artifact in the target occlusion area is determined according to the texture fusion difference.

4. The method according to claim 1, wherein Determining the contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area according to the artifact residual and the feature descriptor of each scale feature map after noise suppression specifically includes: Determine the feature descriptor of each scale feature map after noise suppression; Associating each feature descriptor with the artifact residual to obtain descriptor association information of the target occluded area; The contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area is determined according to the descriptor association information.

5. The method according to claim 1, wherein Based on the attention mechanism, each scale feature map is extracted to recognize the occlusion gaps in the target occlusion area. The specific attention includes: Selecting a scale feature map as a selected scale feature map to determine an occlusion gap in the target occlusion area; Determining an attention weight of the selected scale feature map for the occlusion gap based on an attention mechanism; Determining, according to the attention weight, the recognition attention of the selected scale feature map to the occlusion gap in the target occlusion area; The remaining scale feature maps are then used to determine the recognition attention of the occlusion gaps in the target occluded region.

6. The method according to claim 1, wherein Through all recognition attention, the occlusion gap in the target occlusion area is coupled and enhanced, and the enhanced feature map of the occlusion gap in the target occlusion area is obtained, which specifically includes: determining a potential region of an occlusion gap in the target occluded region based on the artifact residual; All recognition attentions are correlated and fused to obtain a fused attention map; The occlusion gaps in the target occlusion area are enhanced according to the potential area and the fused attention map to obtain an enhanced feature map of the occlusion gaps in the target occlusion area.

7. The method according to claim 1, wherein Performing occlusion compensation on the area to be enhanced in the target occlusion area by using the enhanced feature map and all positioning contributions, and obtaining the compensation map of the area to be enhanced specifically includes: The enhanced feature map is enhanced based on all positioning contributions to obtain an enhanced feature map; Performing fusion compensation on the area to be enhanced in the target occluded area according to the enhanced feature map to obtain a compensated feature map; The compensation map of the area to be enhanced is determined by using the compensation feature map.

8. The method according to claim 1, wherein Ultrasound images of the target occluded area at different angles are collected using a color Doppler ultrasound diagnostic apparatus.

9. The method according to claim 1, wherein The deep learning-based denoising model performs noise suppression on each scale feature map.

10. A rib-occluded ultrasound image analysis system, characterized in that: include: An acquisition module is used to acquire ultrasound images of the target occluded area at different angles, and then determine the scale feature map of the target occluded area in each ultrasound image under different dilated convolution kernels; a processing module for determining an artifact residual of an occlusion artifact in an occluded target area based on texture differences between feature maps at each scale; The processing module is further configured to determine, based on the artifact residual and a feature descriptor of each scale feature map after noise suppression, a contribution of each scale feature map to the positioning of the acoustic shadow area in the target occlusion area; The processing module is further configured to extract the recognition attention of each scale feature map to the occlusion gap in the target occlusion area based on the attention mechanism, and perform coupled enhancement on the occlusion gap in the target occlusion area through all the recognition attentions to obtain an enhanced feature map of the occlusion gap in the target occlusion area; An execution module is used to determine the area to be enhanced in the target occlusion area, and perform occlusion compensation on the area to be enhanced in the target occlusion area through the enhanced feature map and all positioning contributions to obtain a compensation map of the area to be enhanced.

Images