Artifact region mask based information enhancement method, device, equipment and medium

Abstract

The present disclosure provides an artifact region mask-based information enhancement method, device, equipment and medium, comprising: channel splicing a first artifact region mask and a second artifact region mask to generate a metal artifact region mask; applying the metal artifact region mask to an operable vector to obtain a mask representation vector corresponding to a metal artifact region and a mask representation vector corresponding to a non-metal artifact region; determining artifact region branch information based on the mask representation vector corresponding to the metal artifact region and an inoperable vector; determining non-artifact region branch information based on the mask representation vector corresponding to the non-metal artifact region and the inoperable vector; and performing multi-branch information fusion on the artifact region branch information, the non-artifact region branch information and a feature extraction graph to obtain a target artifact removal image. Thus, metal artifacts can be better removed.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of image processing technology, and more specifically, to an information enhancement method, apparatus, device, and medium suitable for an artifact region mask. Background Technology

[0002] Metal artifacts primarily arise from the strong attenuation of X-rays by metallic implants (such as steel plates). X-rays at the metal edges are scattered and hardened, resulting in streak-like, star-shaped, or speckled artifacts in the projected data. Metal artifacts not only obscure the structural details of the surrounding normal tissue, reducing image contrast and spatial resolution, but can also lead to misdiagnosis or missed diagnosis, affecting clinicians' accurate judgment of the condition.

[0003] In related technologies, artifact removal is mainly achieved through spatial self-attention mechanisms, which have extremely high computational complexity, making it difficult to process high-resolution images contaminated with metal. Furthermore, it directly processes all regions of the input image, resulting in incomplete artifact removal in the output image. Summary of the Invention

[0004] The embodiments described herein provide an information enhancement method, apparatus, device, and medium based on artifact region masking that overcomes the aforementioned problems.

[0005] Firstly, according to the present disclosure, an information enhancement method based on artifact region masking is provided, including: A convolution operation is performed on the feature extraction map to generate corresponding operable vectors and non-operable vectors; the operable vectors include query vectors and key vectors, and the non-operable vectors include value vectors; the feature extraction map is obtained by extracting features from the original CT image and normalizing it. The original CT image is subjected to artifact removal using a traditional metal artifact removal algorithm to generate a first artifact region mask; the original CT image is subjected to artifact removal using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask; the first artifact region mask and the second artifact region mask are channel-stitched together to generate a metal artifact region mask. Apply the metal artifact region mask to the operable vector to obtain the mask representation vector corresponding to the metal artifact region and the mask representation vector corresponding to the non-metal artifact region; Based on the mask representation vector corresponding to the metal artifact region and the inoperable vector, determine the artifact region branch information corresponding to the metal artifact region; and based on the mask representation vector corresponding to the non-metal artifact region and the inoperable vector, determine the non-artifact region branch information corresponding to the non-metal artifact region. The artifact region branch information, the non-artifact region branch information, and the feature extraction map are fused to obtain the target artifact removal image.

[0006] Secondly, according to the present disclosure, an information enhancement device based on an artifact region mask is provided, comprising: The first generation module is used to perform a convolution operation on the feature extraction map to generate corresponding operable vectors and non-operable vectors; the operable vectors include query vectors and key vectors, and the non-operable vectors include value vectors; the feature extraction map is obtained by extracting features from the original CT image and normalizing it. The second generation module is used to remove artifacts from the original CT image using a traditional metal artifact removal algorithm to generate a first artifact region mask; to remove artifacts from the original CT image using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask; and to perform channel stitching of the first artifact region mask and the second artifact region mask to generate a metal artifact region mask. The application module is used to apply the metal artifact region mask to the operable vector to obtain the mask representation vector corresponding to the metal artifact region and the mask representation vector corresponding to the non-metal artifact region. The determination module is used to determine the artifact region branch information corresponding to the metal artifact region based on the mask representation vector corresponding to the metal artifact region and the inoperable vector; and to determine the non-artifact region branch information corresponding to the non-metal artifact region based on the mask representation vector corresponding to the non-metal artifact region and the inoperable vector. The fusion module is used to fuse the branch information of the artifact region, the branch information of the non-artifact region, and the feature extraction map to obtain the target artifact removal image.

[0007] Thirdly, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the information enhancement method based on artifact region mask as described in any of the above embodiments.

[0008] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, and when executed by a processor, the computer program implements the steps of the information enhancement method based on artifact region mask as described in any of the above embodiments.

[0009] The information enhancement method based on artifact region masks provided in this application performs convolution operations on the feature extraction map to generate corresponding operable and non-operable vectors. The operable vectors include query vectors and key vectors, while the non-operable vectors include value vectors. The feature extraction map is obtained by feature extraction and normalization of the original CT image. The original CT image is then subjected to artifact removal using a traditional metal artifact removal algorithm to generate a first artifact region mask. A second artifact region mask is generated by performing artifact removal on the original CT image using a deep learning-based metal artifact removal algorithm. The first and second artifact region masks are then combined... Row channels are stitched together to generate a metal artifact region mask. The metal artifact region mask is then applied to operable vectors to obtain mask representation vectors for both metal and non-metal artifact regions. Based on the mask representation vectors for metal and non-operable vectors, artifact region branch information is determined. Similarly, based on the mask representation vectors for non-metal artifact regions, non-artifact region branch information is determined. Multi-branch information fusion is performed on the artifact region branch information, non-artifact region branch information, and feature extraction map to obtain the target artifact-removed image. Thus, a dual-path approach—using traditional removal methods and deep learning methods—is employed to create the metal mask, enhancing the information processing capability of the mask region. By distinguishing between metal and non-metal artifact regions, the ability to remove artifacts from the artifact regions is enhanced while preserving details in the non-artifact regions, resulting in better removal of metal artifacts.

[0010] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure, wherein: Figure 1 This is a flowchart illustrating an information enhancement method based on artifact region masking provided in this disclosure.

[0012] Figure 2 This is a schematic diagram of an information enhancement device based on artifact region mask provided in this disclosure.

[0013] Figure 3 This is a schematic diagram of the structure of a computer device provided in this disclosure.

[0014] It should be noted that the elements in the attached diagram are schematic and not drawn to scale. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0016] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, the statement of “connecting” or “coupling” two or more parts together shall mean that these parts are directly joined together or joined through one or more intermediate components.

[0017] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of the phrase "embodiment" in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0018] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists, A and B exist simultaneously, or B exists. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Terms such as "first" and "second" are only used to distinguish one component (or part of a component) from another component (or another part of a component).

[0019] In the description of this application, unless otherwise stated, "multiple" means two or more (including two), and similarly, "multiple groups" means two or more (including two groups).

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0021] Figure 1 This is a flowchart illustrating an information enhancement method based on artifact region masking provided in this disclosure. Figure 1 As shown, the specific process of the information enhancement method based on artifact region masking includes: S110. Perform a convolution operation on the feature extraction map to generate corresponding operable and non-operable vectors.

[0022] Among them, the operable vectors include the query vector Q and the key vector K, the non-operable vectors include the value vector V, and the feature extraction map is obtained by extracting and normalizing features from the original CT image.

[0023] For example, feature extraction from the original CT image yields a feature map X∈R^(B×C×H×W). Layer normalization of the feature map X yields the normalized feature map X_norm=(X-μ) / √(σ²+ε)×γ+β, which is the extracted feature map. Here, μ and σ² are the mean and variance of the feature along the channel dimension, respectively; ε=1×10⁻⁶. -5 , to prevent division by zero, a very small constant; γ and β are learnable parameters.

[0024] The query vector Q, key vector K, and value vector V are generated through convolution operations: QKV_inter=Conv1×1(X_norm), QKV_enhanced=DWConv3×3(QKV_inter), and Q,K,V=Split(QKV_enhanced,dim=1). The depthwise separable convolution is implemented using grouped convolution with 3C groups, a 3×3 kernel, and padding of 1 to maintain the same spatial dimensions.

[0025] S120. The original CT image is subjected to artifact removal using a traditional metal artifact removal algorithm to generate a first artifact region mask; the original CT image is subjected to artifact removal using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask; the first artifact region mask and the second artifact region mask are channel-stitched together to generate a metal artifact region mask.

[0026] In some embodiments, the original CT image is subjected to artifact removal using a conventional metal artifact removal algorithm to generate a first artifact region mask, including: removing artifacts from the original CT image using a conventional metal artifact removal algorithm to obtain a first artifact-removed image; and generating a first artifact region mask based on the image difference between the original CT image and the first artifact-removed image.

[0027] Traditional metal artifact removal algorithms, such as MAR (Metal Artifact Reduction), utilize threshold segmentation / region growing techniques to identify metal pixels in the original CT image and construct a metal mask. Based on the metal mask, linear / polynomial interpolation is applied in the projection or image domain to repair the projection data or image data of the metal-occluded area, generating a first artifact-removed image. The image difference between the original CT image and the corresponding pixel value in the first artifact-removed image is then obtained by subtracting the pixel value from the pixel value of the original CT image. This image difference reflects the abnormal pixel value areas in the original CT image caused by metal artifacts. A threshold can be set to binarize the image difference between the original CT image and the first artifact-removed image, identifying areas with pixel value differences greater than the set threshold as metal artifact regions, thus generating the first metal artifact mask, mask1.

[0028] The original CT image is subjected to artifact removal using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask. The process includes: removing artifacts from the original CT image using a deep learning-based metal artifact removal algorithm to obtain a second artifact-removed image; and generating a second artifact region mask based on the image difference between the original CT image and the second artifact-removed image.

[0029] In this process, a deep neural network is constructed to remove artifacts from the original CT image, resulting in a second artifact-removed image. The image difference between the two images is obtained by subtracting the corresponding pixel values ​​from the pixel values ​​of the second artifact-removed image. This difference reflects areas of abnormal pixel values ​​in the original CT image caused by metal artifacts. A threshold is set to binarize the image difference between the original CT image and the second artifact-removed image, identifying areas with pixel value differences greater than the threshold as metal artifact regions, thus generating a second metal artifact mask, mask2.

[0030] Therefore, by using traditional methods to provide stable prior information to stabilize the network and accelerate convergence, and then using deep learning methods to gradually optimize and generate higher quality metal artifact masks.

[0031] In some embodiments, the first artifact region mask and the second artifact region mask are channel-stitched to generate a metal artifact region mask, including: channel-stitching the first artifact region mask and the second artifact region mask to obtain a region-stitched mask; and sizing the region-stitched mask based on the original CT image to obtain the metal artifact region mask.

[0032] Specifically, a comprehensive gating weight is generated using a spatial attention network and a global gating network. This comprehensive gating weight is then adaptively fused with the first artifact region mask (mask1) and the second artifact region mask (mask2) to obtain a region stitching mask (mask_fused). Subsequently, the region stitching mask (mask_fused) is upsampled, downsampled, and its dimensions are expanded to obtain a metal artifact region mask (Mask∈R^(B×C×H×W)) used to identify the target regions requiring focused processing, thus restoring the image size to the original CT image size.

[0033] S130. Apply a metal artifact region mask to the operable vector to obtain the mask representation vector corresponding to the metal artifact region and the mask representation vector corresponding to the non-metal artifact region.

[0034] In this process, masks are applied to Q and K in the metal artifact region while V remains intact, as shown in the following formulas (1)-(3).

[0035] Q1_masked=Q⊙Mask+Q⊙(1-Mask)⊙γ(1) K1_masked=K⊙Mask+K⊙(1-Mask)⊙γ(2) V1_original=V(3) In formulas (1) and (2), γ is a learnable temperature parameter.

[0036] Apply a mask to Q and K in the non-metallic artifact region while keeping V intact, as shown in the following formulas (4)-(6).

[0037] Q2_masked=Q⊙(1-Mask) (4) K2_masked=K⊙(1-Mask) (5) V2_original=V(6) The mask representation vectors corresponding to the metallic artifact regions are Q1_masked and K1_masked, and the mask representation vectors corresponding to the non-metallic artifact regions are Q2_masked and K2_masked.

[0038] By masking Q and K, attention is computed only within the target region; V is not masked, preserving information from the entire image and allowing attention to extract features from the whole image, thus avoiding information bottlenecks. This asymmetric design ensures that metallic artifact regions can access information from non-metallic artifact regions as a reference, while the attention distribution is dominated by the target region; non-metallic artifact regions focus on their association with information from the original image, weakening their association with information from metallic artifact regions.

[0039] S140. Based on the mask representation vector and the inoperable vector corresponding to the metal artifact region, determine the artifact region branch information corresponding to the metal artifact region; and based on the mask representation vector and the inoperable vector corresponding to the non-metal artifact region, determine the non-artifact region branch information corresponding to the non-metal artifact region.

[0040] Specifically, the artifact region branch information corresponding to the metallic artifact region can be used to effectively mask or suppress the feature information of the metallic artifact region. The non-artifact region branch information corresponding to the non-metallic artifact region can be used to selectively preserve and enhance the feature information of the non-metallic artifact region, thereby more effectively highlighting the detailed features of the non-metallic artifact region.

[0041] In some embodiments, the branch information of the artifact region corresponding to the metal artifact region is determined based on the mask representation vector and the non-operable vector corresponding to the metal artifact region. This includes: performing channel rearrangement and normalization on the mask representation vector and the non-operable vector corresponding to the metal artifact region according to the channel attention calculation rules; calculating the attention weight corresponding to the metal artifact region based on the mask representation vector; and weighting and aggregating the attention weight corresponding to the metal artifact region based on the non-operable vector to obtain the aggregated feature map corresponding to the metal artifact region; and determining the branch information of the artifact region corresponding to the metal artifact region based on the aggregated feature map and the feature extraction map.

[0042] For example, rearranging Q1_masked, K1_masked, and V1_original into a channel attention calculation form, when using multi-head attention (number of heads h, number of channels per head c=C / h): Q1_masked: B×C×H×W→B×h×c×(H×W); K1_masked: B×C×H×W→B×h×c×(H×W); V1_original: B×C×H×W→B×h×c×(H×W).

[0043] L2 normalization is performed on Q1_masked and K1_masked: Q_norm = Q1_masked / (||Q1_masked||2 + ε), K_norm = K1_masked / (||K1_masked||2 + ε). Normalization is performed in the spatial dimension, ε = 1 × 10 -6 This helps ensure numerical stability.

[0044] Calculate the channel similarity and generate attention weights: Attn1_score = (Q1_norm@K1_norm^T) × τ, Att1 = Softmax(Attn1_score, dim = -1). Here, τ is a learnable temperature parameter with shape (h, 1, 1), initialized to 1. Both Attn1_score and Attn1 have a shape of B × h × c × c, representing the attention weights between c channels. Since Q and K are masked, the Attn1 matrix only has significant weights between channels in the target region.

[0045] Multiply the attention weights by the complete V: Out1_aggr = Attn1@V1_original. Although Attn1 is calculated from the Q and K of the mask (attention is applied to the target region), it acts on the complete V1_original (which contains information from the entire image), enabling the target region to extract reference features from non-target regions. Rearrange Out1_aggr from B×h×c×(H×W) back to B×C×H×W.

[0046] In some embodiments, based on the aggregated feature map and the feature extraction map corresponding to the metal artifact region, the branch information of the artifact region corresponding to the metal artifact region is determined, including: performing a 1×1 convolution on the aggregated feature map corresponding to the metal artifact region, and performing a residual connection between the feature extraction map and the convolved aggregated feature map corresponding to the metal artifact region to obtain a residual connection map corresponding to the metal artifact region; normalizing and processing the residual connection map corresponding to the metal artifact region with a feedforward network to obtain a feedforward network output map corresponding to the metal artifact region; and performing a residual connection between the residual connection map corresponding to the metal artifact region and the feedforward network output map corresponding to the metal artifact region to obtain the branch information of the artifact region corresponding to the metal artifact region.

[0047] Among them, feedforward networks can be used to perform nonlinear transformations and feature enhancements on weighted features.

[0048] Following the example above, the output is projected through a 1×1 convolution and then concatenated with the original input residual: Out1_proj = Conv1×1(Out1_aggr), X1_attn = Out1_proj + X. This residual connection ensures a direct gradient propagation path, avoiding the vanishing gradient problem.

[0049] A second layer normalization is performed on X1_attn, followed by processing through a feedforward network: X1_attn_norm = LayerNorm(X1_attn), X1_expand = Conv1×1(X1_attn_norm), X1_activated = GELU(X1_expand), FFN1_out (i.e., the output graph of the feedforward network) = Conv1×1(X1_activated). Here, the expansion factor r = 2.66, and the activation function can be GELU (Gaussian Error Linear Unit).

[0050] Add the output graph of the feedforward network to X1_attn to obtain the artifact region branch information Y1=FFN1_out+X1_attn, Y1∈R^(B×C×H×W), which has the same shape as X.

[0051] In some embodiments, the non-artifact region branch information corresponding to the non-metallic artifact region is determined based on the mask representation vector and the inoperable vector corresponding to the non-metallic artifact region. This includes: performing channel rearrangement and normalization on the mask representation vector and the inoperable vector corresponding to the non-metallic artifact region using channel attention calculation rules; calculating the attention weight corresponding to the non-metallic artifact region based on the mask representation vector; and weighting and aggregating the attention weight corresponding to the non-metallic artifact region based on the inoperable vector to obtain the aggregated feature map corresponding to the non-metallic artifact region; and determining the non-artifact region branch information corresponding to the non-metallic artifact region based on the aggregated feature map and the feature extraction map.

[0052] For example, rearranging Q2_masked, K2_masked, and V2_original into a channel attention calculation form, when using multi-head attention (number of heads h, number of channels per head c=C / h): Q2_masked: B×C×H×W→B×h×c×(H×W); K2_masked: B×C×H×W→B×h×c×(H×W); V2_original: B×C×H×W→B×h×c×(H×W).

[0053] L2 normalization is performed on Q2_masked and K2_masked: Q_norm = Q2_masked / (||Q2_masked||2 + ε), K_norm = K2_masked / (||K2_masked||2 + ε). Normalization is performed in the spatial dimension, ε = 1 × 10 -6 This helps ensure numerical stability.

[0054] Calculate the channel similarity and generate attention weights: Attn2_score = (Q2_norm@K2_norm^T) × τ, Att2 = Softmax(Attn2_score, dim = -1). Here, τ is a learnable temperature parameter with shape (h, 1, 1), initialized to 1. Both Attn2_score and Attn2 have shapes of B × h × c × c, representing the attention weights between c channels. Since Q and K are masked, the Attn2 matrix only has significant weights between channels in the target region.

[0055] Multiply the attention weights by the complete V: Out2_aggr = Attn2@V2_original. Although Attn2 is calculated from the Q and K of the mask (attention is applied to the target region), it acts on the complete V2_original (which contains information from the entire image), enabling the target region to extract reference features from non-target regions. Rearrange Out2_aggr from B×h×c×(H×W) back to B×C×H×W.

[0056] In some embodiments, the non-artifact region branch information corresponding to the non-metallic artifact region is determined based on the aggregated feature map and the feature extraction map corresponding to the non-metallic artifact region. This includes: performing a 1×1 convolution on the aggregated feature map corresponding to the non-metallic artifact region, and performing a residual connection between the feature extraction map and the convolved aggregated feature map corresponding to the non-metallic artifact region to obtain a residual connection map corresponding to the non-metallic artifact region; normalizing and processing the residual connection map corresponding to the non-metallic artifact region using a feedforward network to obtain a feedforward network output map corresponding to the non-metallic artifact region; and performing a residual connection between the residual connection map corresponding to the non-metallic artifact region and the feedforward network output map corresponding to the non-metallic artifact region to obtain the non-artifact region branch information corresponding to the non-metallic artifact region.

[0057] Among them, feedforward networks can be used to perform nonlinear transformations and feature enhancements on weighted features.

[0058] Following the example above, the output is projected through a 1×1 convolution and then concatenated with the original input residual: Out2_proj = Conv1×1(Out2_aggr), X2_attn = Out2_proj + X. This residual connection ensures a direct gradient propagation path, avoiding the vanishing gradient problem.

[0059] A second layer normalization is performed on X2_attn, followed by processing through a feedforward network: X2_attn_norm = LayerNorm(X2_attn), X2_expand = Conv1×1(X2_attn_norm), X2_activated = GELU(X2_expand), FFN2_out (i.e., the output graph of the feedforward network) = Conv1×1(X2_activated). Here, the expansion factor r = 2.66, and the activation function can be GELU (Gaussian Error Linear Unit).

[0060] Add the output graph of the feedforward network to X2_attn to obtain the non-artifact region branch information Y2=FFN2_out+X2_attn, Y2∈R^(B×C×H×W), which has the same shape as X.

[0061] S150. Multi-branch information fusion is performed on the branch information of artifact regions, branch information of non-artifact regions, and feature extraction map to obtain the target artifact removal image.

[0062] The process involves fusing multi-branch information from the artifact region branch information Y1, the non-artifact region branch information Y2, and the feature extraction map X to obtain Y_fusion=Concat(X,Y1,Y2)#B×3C×H×W. Then, a 1×1 convolution is performed to obtain the target artifact removal image Y_final=Conv1×1_reduce(Y_fusion)#B×3C×H×W→B×C×H×W. This facilitates better identification of metallic artifact regions and provides better constraint on fine linear artifact residues.

[0063] In this embodiment, a convolution operation is performed on the feature extraction map to generate corresponding operable and non-operable vectors. The operable vectors include query vectors and key vectors, while the non-operable vectors include value vectors. The feature extraction map is obtained by extracting and normalizing features from the original CT image. A traditional metal artifact removal algorithm is used to remove artifacts from the original CT image, generating a first artifact region mask. A deep learning-based metal artifact removal algorithm is then used to remove artifacts from the original CT image, generating a second artifact region mask. The first and second artifact region masks are then concatenated to generate a metal artifact mask. The process involves masking the artifact regions. A metal artifact region mask is applied to the operable vectors to obtain mask representation vectors for both metal and non-metal artifact regions. Based on the mask representation vectors for metal and non-operable vectors, the branch information for the metal artifact regions is determined. Similarly, based on the mask representation vectors for non-metal artifact regions, the branch information for the non-metal artifact regions is determined. Multi-branch information fusion is performed on the artifact region branch information, the non-artifact region branch information, and the feature extraction map to obtain the target artifact-removed image. This approach utilizes a dual-path method—combining traditional removal schemes and deep learning—to create the metal mask, enhancing the information processing capability of the mask region. By distinguishing between metal and non-metal artifact regions, the ability to remove artifacts from the metal artifact regions is enhanced while preserving details in the non-artifact regions, thus achieving better removal of metal artifacts.

[0064] In summary, this embodiment achieves differentiated modeling of artifact and non-artifact regions by performing self-attention computation at the channel dimension and introducing a region masking mechanism. The metal artifact region mask employs a soft application strategy, taking into account the blurred boundaries of metal artifact regions and enhancing artifact recognition capabilities; the non-metal artifact region mask employs a hard application strategy, strictly ensuring the correspondence between non-artifact regions and the original image information. The masking network and the metal artifact removal network use the same deep learning network, which avoids training a separate mask sub-network, improving training efficiency, and ensures full utilization of the core module's advantages.

[0065] Figure 2 This is a schematic diagram of an information enhancement device based on an artifact region mask provided in this embodiment. The information enhancement device based on an artifact region mask may include: The first generation module 210 is used to perform convolution operations on the feature extraction map to generate corresponding operable vectors and non-operable vectors; the operable vectors include query vectors and key vectors, and the non-operable vectors include value vectors. The feature extraction map is obtained by extracting features from the original CT image and normalizing it.

[0066] The second generation module 220 is used to remove artifacts from the original CT image using a traditional metal artifact removal algorithm to generate a first artifact region mask; to remove artifacts from the original CT image using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask; and to stitch the first artifact region mask and the second artifact region mask together to generate a metal artifact region mask.

[0067] Application module 230 is used to apply a metal artifact region mask to the operable vector to obtain the mask representation vector corresponding to the metal artifact region and the mask representation vector corresponding to the non-metal artifact region.

[0068] The determination module 240 is used to determine the artifact region branch information corresponding to the metal artifact region based on the mask representation vector and the inoperable vector corresponding to the metal artifact region; and to determine the non-artifact region branch information corresponding to the non-metal artifact region based on the mask representation vector and the inoperable vector corresponding to the non-metal artifact region.

[0069] The fusion module 250 is used to fuse multi-branch information from the artifact region branch information, the non-artifact region branch information, and the feature extraction map to obtain the target artifact removal image.

[0070] In this embodiment, optionally, the determining module 240 is specifically used for: By using channel attention calculation rules, channel rearrangement and normalization are performed on the mask representation vector and non-operable vector corresponding to the metal artifact region. Attention weights corresponding to the metal artifact region are calculated based on the mask representation vector. The attention weights corresponding to the metal artifact region are then weighted and aggregated based on the non-operable vector to obtain the aggregated feature map of the metal artifact region. Based on the aggregated feature map and the feature extraction map, the branch information of the metal artifact region is determined.

[0071] In this embodiment, optionally, the determining module 240 is specifically used for: A 1×1 convolution is performed on the aggregated feature map corresponding to the metal artifact region, and the extracted feature map is residually concatenated with the convolved aggregated feature map corresponding to the metal artifact region to obtain the residual concatenation map corresponding to the metal artifact region. The residual concatenation map corresponding to the metal artifact region is normalized and processed by a feedforward network to obtain the feedforward network output map corresponding to the metal artifact region. The residual concatenation map corresponding to the metal artifact region is residually concatenated with the feedforward network output map corresponding to the metal artifact region to obtain the artifact region branch information corresponding to the metal artifact region.

[0072] In this embodiment, optionally, the determining module 240 is specifically used for: By using channel attention calculation rules, channel rearrangement and normalization are performed on the mask representation vector and non-operable vector corresponding to the non-metallic artifact region. Attention weights corresponding to the non-metallic artifact region are calculated based on the mask representation vector. The attention weights corresponding to the non-metallic artifact region are then weighted and aggregated based on the non-operable vector to obtain the aggregated feature map corresponding to the non-metallic artifact region. Based on the aggregated feature map and feature extraction map corresponding to the non-metallic artifact region, the branch information of the non-artifact region corresponding to the non-metallic artifact region is determined.

[0073] In this embodiment, optionally, the determining module 240 is specifically used for: A 1×1 convolution is performed on the aggregated feature map corresponding to the non-metallic artifact region, and the extracted feature map is residually concatenated with the convolved aggregated feature map corresponding to the non-metallic artifact region to obtain the residual concatenation map corresponding to the non-metallic artifact region. The residual concatenation map corresponding to the non-metallic artifact region is normalized and processed by a feedforward network to obtain the feedforward network output map corresponding to the non-metallic artifact region. The residual concatenation map corresponding to the non-metallic artifact region is residually concatenated with the feedforward network output map corresponding to the non-metallic artifact region to obtain the non-artifact region branch information corresponding to the non-metallic artifact region.

[0074] In this embodiment, optionally, the second generation module 220 is specifically used for: The first artifact region mask and the second artifact region mask are channel-stitched together to obtain a region stitching mask; the region stitching mask is then sized and aligned based on the original CT image to obtain a metal artifact region mask.

[0075] In this embodiment, optionally, the second generation module 220 is specifically used for: The original CT image is processed using a traditional metal artifact removal algorithm to remove artifacts, resulting in a first artifact-removed image. A first artifact region mask is generated based on the image difference between the original CT image and the first artifact-removed image. The original CT image is then processed using a deep learning-based metal artifact removal algorithm to remove artifacts, resulting in a second artifact-removed image. A second artifact region mask is generated based on the image difference between the original CT image and the second artifact-removed image.

[0076] The information enhancement device based on artifact region mask provided in this disclosure can perform the above method embodiments. Its specific implementation principle and technical effects can be found in the above method embodiments, and will not be repeated here.

[0077] This application also provides a computer device. Please refer to the following for details. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0078] The computer device includes a memory 310 and a processor 320 that are interconnected via a system bus. It should be noted that only a computer device with memory 310 and processor 320 is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented alternatively. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0079] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0080] The memory 310 includes at least one type of readable storage medium, including non-volatile memory or volatile memory, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. RAM may include static RAM or dynamic RAM. In some embodiments, the memory 310 may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory 310 may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card equipped on the computer device. Of course, the memory 310 may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory 310 is typically used to store the operating system and various application software installed on the computer device, such as the program code of the method described above. In addition, the memory 310 can also be used to temporarily store various types of data that have been output or will be output.

[0081] Processor 320 is typically used to perform overall operations of a computer device. In this embodiment, memory 310 is used to store program code or instructions, including computer operation instructions, and processor 320 is used to execute the program code or instructions stored in memory 310 or process data, such as program code that runs the methods described above.

[0082] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus system can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0083] Another embodiment of this application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads computer-readable program code stored in the computer-readable medium, enabling the processor to execute the functional actions specified in each step or combination of steps in the above method; and to generate means for implementing the functional actions specified in each block or combination of blocks in the block diagram.

[0084] Computer-readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any suitable combination thereof, wherein the memory is used to store program code or instructions, the program code including computer operation instructions, and the processor is used to execute the program code or instructions of the above-described methods stored in the memory.

[0085] The definitions of memory and processor can be found in the description of the foregoing computer device embodiments, and will not be repeated here.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0087] In the various embodiments of this application, the functional units or modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" as described in this application does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims listing several means, several units of these means may be embodied by the same item of hardware. The use of "first," "second," and "third," etc., does not indicate any order and these words should be interpreted as names. Unless otherwise specified, the steps in the above embodiments should not be construed as limiting the order of execution.

[0090] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An information enhancement method based on artifact region masking, characterized in that, include: Perform a convolution operation on the feature extraction map to generate corresponding operable and non-operable vectors; The operable vectors include query vectors and key vectors, the non-operable vectors include value vectors, and the feature extraction map is obtained by extracting features from the original CT image and normalizing it. The original CT image is subjected to artifact removal using a traditional metal artifact removal algorithm to generate a first artifact region mask; the original CT image is subjected to artifact removal using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask. The first artifact region mask and the second artifact region mask are spliced ​​together to generate a metal artifact region mask. Apply the metal artifact region mask to the operable vector to obtain the mask representation vector corresponding to the metal artifact region and the mask representation vector corresponding to the non-metal artifact region; Based on the mask representation vector corresponding to the metal artifact region and the inoperable vector, determine the artifact region branch information corresponding to the metal artifact region; Furthermore, based on the mask representation vector corresponding to the non-metallic artifact region and the inoperable vector, the non-artifact region branch information corresponding to the non-metallic artifact region is determined; The artifact region branch information, the non-artifact region branch information, and the feature extraction map are fused to obtain the target artifact removal image.

2. The method according to claim 1, characterized in that, Based on the mask representation vector corresponding to the metal artifact region and the inoperable vector, the artifact region branch information corresponding to the metal artifact region is determined, including: The mask representation vector and the inoperable vector corresponding to the metal artifact region are rearranged and normalized using the channel attention calculation rules. The attention weights corresponding to the metal artifact regions are calculated based on the mask representation vectors corresponding to the metal artifact regions; and the attention weights corresponding to the metal artifact regions are weighted and aggregated based on the inoperable vectors corresponding to the metal artifact regions to obtain the aggregated feature map corresponding to the metal artifact regions. Based on the aggregated feature map and the feature extraction map corresponding to the metal artifact region, the artifact region branch information corresponding to the metal artifact region is determined.

3. The method according to claim 2, characterized in that, Based on the aggregated feature map and the extracted feature map corresponding to the metal artifact region, the artifact region branch information corresponding to the metal artifact region is determined, including: Perform a 1×1 convolution on the aggregated feature map corresponding to the metal artifact region, and perform a residual connection between the extracted feature map and the convolved aggregated feature map corresponding to the metal artifact region to obtain the residual connection map corresponding to the metal artifact region. The residual connection graph corresponding to the metal artifact region is normalized and processed by a feedforward network to obtain the feedforward network output graph corresponding to the metal artifact region. By performing a residual connection between the residual connection graph corresponding to the metal artifact region and the output graph of the feedforward network corresponding to the metal artifact region, the branch information of the artifact region corresponding to the metal artifact region is obtained.

4. The method according to claim 1, characterized in that, Based on the mask representation vector corresponding to the non-metallic artifact region and the inoperable vector, the non-artifact region branch information corresponding to the non-metallic artifact region is determined, including: Using channel attention calculation rules, channel rearrangement and normalization are performed on the mask representation vector and the inoperable vector corresponding to the non-metallic artifact region; The attention weights corresponding to the non-metallic artifact regions are calculated based on the mask representation vectors corresponding to the non-metallic artifact regions; and the attention weights corresponding to the non-metallic artifact regions are weighted and aggregated based on the inoperable vectors corresponding to the non-metallic artifact regions to obtain the aggregated feature map corresponding to the non-metallic artifact regions. Based on the aggregated feature map corresponding to the non-metallic artifact region and the feature extraction map, the non-artifact region branch information corresponding to the non-metallic artifact region is determined.

5. The method according to claim 4, characterized in that, Based on the aggregated feature map corresponding to the non-metallic artifact region and the feature extraction map, the non-artifact region branch information corresponding to the non-metallic artifact region is determined, including: Perform a 1×1 convolution on the aggregated feature map corresponding to the non-metallic artifact region, and perform a residual connection between the extracted feature map and the convolved aggregated feature map corresponding to the non-metallic artifact region to obtain the residual connection map corresponding to the non-metallic artifact region. The residual connection graph corresponding to the non-metallic artifact region is normalized and processed by a feedforward network to obtain the feedforward network output graph corresponding to the non-metallic artifact region. By performing a residual connection between the residual connection graph corresponding to the non-metallic artifact region and the feedforward network output graph corresponding to the non-metallic artifact region, the branch information of the non-artifact region corresponding to the non-metallic artifact region is obtained.

6. The method according to claim 1, characterized in that, The first artifact region mask and the second artifact region mask are spliced ​​together to generate a metal artifact region mask, including: The first artifact region mask and the second artifact region mask are spliced ​​together to obtain a region splicing mask; Based on the original CT image, the size of the region stitching mask is aligned to obtain the metal artifact region mask.

7. The method according to claim 1, characterized in that, The original CT image is subjected to artifact removal using a traditional metal artifact removal algorithm to generate a first artifact region mask, including: The original CT image is subjected to artifact removal using a traditional metal artifact removal algorithm to obtain a first artifact-removed image; a first artifact region mask is generated based on the image difference between the original CT image and the first artifact-removed image. The original CT image is subjected to artifact removal using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask, including: The original CT image is subjected to artifact removal using a deep learning-based metal artifact removal algorithm to obtain a second artifact-removed image; a second artifact region mask is generated based on the image difference between the original CT image and the second artifact-removed image.

8. An information enhancement device based on artifact region masking, characterized in that, include: The first generation module is used to perform convolution operations on the feature extraction map to generate corresponding operable and non-operable vectors; The operable vectors include query vectors and key vectors, the non-operable vectors include value vectors, and the feature extraction map is obtained by extracting features from the original CT image and normalizing it. The second generation module is used to remove artifacts from the original CT image using a traditional metal artifact removal algorithm to generate a first artifact region mask; and to remove artifacts from the original CT image using a deep learning-based metal artifact removal algorithm to generate a second artifact region mask. The first artifact region mask and the second artifact region mask are spliced ​​together to generate a metal artifact region mask. The application module is used to apply the metal artifact region mask to the operable vector to obtain the mask representation vector corresponding to the metal artifact region and the mask representation vector corresponding to the non-metal artifact region. The determination module is used to determine the artifact region branch information corresponding to the metal artifact region based on the mask representation vector corresponding to the metal artifact region and the inoperable vector. Furthermore, based on the mask representation vector corresponding to the non-metallic artifact region and the inoperable vector, the non-artifact region branch information corresponding to the non-metallic artifact region is determined; The fusion module is used to fuse the branch information of the artifact region, the branch information of the non-artifact region, and the feature extraction map to obtain the target artifact removal image.

9. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the information enhancement method based on artifact region mask as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the information enhancement method based on artifact region mask as described in any one of claims 1 to 7.

Images