Multi-modal medical image artifact intelligent detection method and system based on deep learning

Abstract

The invention relates to a multi-modal medical image artifact intelligent detection method and system based on deep learning, and the method comprises the steps: receiving a multi-modal original medical image, and carrying out the preprocessing of the original medical image, so as to generate a standardized target medical image; performing global detection based on the target medical image to obtain artifact confidence; when the artifact confidence exceeds a set threshold value, identifying a corresponding artifact type according to a preset multi-classification network, performing pixel-level segmentation positioning on the target medical image based on the artifact type to extract a main artifact area and position, and calculating a severity score according to the main artifact area; according to the method, the interpretable thermodynamic diagram is generated based on the target medical image, and the key features in the interpretable thermodynamic diagram are extracted to generate the structured detection report, so that the problems of dependence on manpower and high modal specificity in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and system for intelligent detection of multimodal medical image artifacts based on deep learning. Background Technology

[0002] In the field of medical imaging technology, artifacts are a key interfering factor affecting image quality and diagnostic accuracy. In medical imaging, artifacts refer to abnormal image signals or structures that are not generated by the human anatomy or physiological function itself, but rather by equipment, operation, or the environment. These abnormal signals appear in multimodal images such as CT, MRI, and PET in specific forms (e.g., stripes, spots, shadows, deformations), and are not directly related to actual anatomical / pathological information. Essentially, they are "noise interference" or "signal distortion" of the image data, directly compromising the authenticity and integrity of the image.

[0003] Due to significant differences in equipment configuration (such as the number of rows in CT scanners, the field strength in MRI scanners, and the detector precision in PET scanners), and the lack of standardized scanning procedures, coupled with individual differences in operator proficiency and parameter setting experience, images of the same anatomical location and disease type exhibit marked variations in key quality indicators such as clarity, contrast, and artifact control. For example, low-field MRI equipment is prone to radiofrequency interference artifacts due to uneven magnetic fields, and improper settings of CT scan window width and level by some operators may mask minute lesions. These quality defects not only increase the difficulty of clinical diagnosis but may also lead to the risk of missed diagnoses and misdiagnoses.

[0004] Current artifact detection relies on manual judgment by staff, which is inefficient and highly subjective. Moreover, existing detection methods are mostly designed for single-modal images and cannot adapt to the differentiated characteristics of multimodal images such as CT, MRI, and PET, thus failing to provide a unified quantitative basis for image quality assessment. Summary of the Invention

[0005] To address the issues of reliance on manual methods and strong modality specificity in existing technologies, this application provides a deep learning-based intelligent detection method and system for multimodal medical image artifacts.

[0006] Firstly, this application provides a deep learning-based intelligent detection method for multimodal medical image artifacts, employing the following technical solution: A deep learning-based intelligent detection method for multimodal medical image artifacts, the method comprising: Receive multimodal raw medical images and preprocess them to generate standardized target medical images; Global detection is performed based on the target medical image to obtain the artifact confidence score, wherein the artifact confidence score is the probability of the existence of an artifact; When the confidence level of an artifact exceeds a set threshold, the corresponding artifact type is identified according to a preset multi-classification network. The artifact type is pre-stored in an artifact knowledge base, and the target medical image is segmented and located at the pixel level based on the artifact type to extract the main artifact region and location, and the severity score is calculated based on the main artifact region. An interpretable heatmap is generated based on the target medical image, and key features are extracted from the interpretable heatmap to determine the artifact types in the artifact knowledge base. A structured detection report is then generated, which includes artifact type, location, and severity score.

[0007] In one possible implementation, the preprocessing of the original medical images to generate standardized target medical images includes: Extract pixel data and modality type from raw medical images; Adapt the corresponding window width and window level according to the modality type; Adjust the pixel data to the predetermined grayscale range; The image size is scaled to a predetermined size to generate a standardized target medical image.

[0008] In one possible implementation, the global detection based on the target medical image to obtain artifact confidence includes: Multi-scale features of the target medical image are extracted by using convolutional layers and residual blocks of a convolutional neural network. These multi-scale features include bottom-level edge and texture features as well as high-level semantic features. The multi-scale feature map is spatially compressed by a global average pooling layer, which transforms the spatial information of each feature channel into global statistical values ​​and generates a one-dimensional feature vector. The feature vectors are weighted and integrated through a fully connected layer, and the output is transformed into a probability distribution by the Softmax activation function. The probability value corresponding to "the existence of artifacts" is the artifact confidence.

[0009] In one possible implementation, the multi-classification network includes a variety of common artifact tasks and other artifact tasks. The step of identifying the corresponding artifact type based on a preset multi-classification network includes: The target features of the target medical image are extracted by using the convolutional layers and residual structure of a multi-classification network. The target features include the edges, textures, and semantics of artifacts. The multi-classification network performs classification operations on the extracted target features and outputs the artifact probability value for each class of artifacts; The artifact probability values ​​of the output artifacts are sorted, the K artifact probability values ​​with the highest probability values ​​are selected, and the artifact type corresponding to the artifact probability value is output. The artifact type is pre-stored in the artifact knowledge base, and the artifact type in the artifact knowledge base corresponds to the artifact task in the multi-classification network.

[0010] In one possible implementation, the pixel-level segmentation and localization of the target medical image based on artifact type to extract the main artifact regions and locations includes: According to the artifact type, the corresponding U-Net pixel-level segmentation model is called, and the artifact probability value of each pixel of the target medical image is predicted by the U-Net pixel-level segmentation model, and a pixel-level probability map is output. The coordinates of each pixel in the pixel-level probability map correspond one-to-one with the spatial position of the target medical image. Morphological operations are performed on pixel-level probability maps to smooth the boundaries of artifact regions; Identify and extract connected regions with artifact probability values ​​higher than a set threshold to generate the main artifact region, and record the spatial location of the main artifact region in the target medical image.

[0011] In one possible implementation, calculating the severity score based on the primary artifact region includes: Obtain the total number of pixels in the target medical image and the total number of artifact pixels in the main artifact region. Calculate the area percentage based on the ratio of the total number of artifact pixels to the total number of pixels. Based on the coverage of the main artifact areas and key anatomical areas, the key structures that affect the image are obtained. The key anatomical areas are pre-marked on the spatial location of the corresponding target medical image. Calculate the severity confidence score, which is the ratio of the sum of the probability values ​​of all artifacts in the main artifact region to the total number of pixels in the main artifact region; The severity score is obtained based on the area proportion, the impact on key structures, and the severity confidence level.

[0012] In one possible implementation, the artifact knowledge base includes motion artifact types, metal artifact types, radio frequency interference types, ring artifact types, aliasing artifact types, chemical shift artifact types, magnetic susceptibility artifact types, truncation artifact types, as well as the image characteristics, causes, detection strategies, and clinical recommendations for each artifact type.

[0013] In one possible implementation, the step of generating an interpretable heatmap based on the target medical image and extracting key features from the interpretable heatmap to generate a structured detection report includes: The gradient-weighted feature map of the last convolutional layer of the model is calculated using the Grad-CAM algorithm, upsampled, and then superimposed on the target medical image to generate an interpretable heatmap. Extract key features from the interpretable heatmap, including brightness, contrast, texture, and shape; By comparing key features with the impact features in the artifact knowledge base, the artifact types in the artifact knowledge base are determined. Generate a structured detection report, which includes artifact type, location, severity score, and clinical recommendations.

[0014] Secondly, this application provides a deep learning-based multimodal medical image artifact intelligent detection system, employing the following technical solution: A deep learning-based intelligent detection system for multimodal medical image artifacts, the system comprising: The preprocessing module receives multimodal raw medical images and preprocesses them to generate standardized target medical images. A global detection module is used to perform global detection based on the target medical image to obtain the artifact confidence score, wherein the artifact confidence score is the probability of the existence of an artifact. The type identification and scoring module is used to identify the corresponding artifact type according to a preset multi-classification network when the confidence of the artifact exceeds a set threshold. The artifact type is pre-stored in the artifact knowledge base, and the target medical image is segmented and located at the pixel level based on the artifact type to extract the main artifact region and location, and the severity score is calculated based on the main artifact region. The report generation module is used to generate an interpretable heatmap based on the target medical image, extract key features from the interpretable heatmap to determine the artifact types in the artifact knowledge base, and generate a structured detection report, wherein the structured detection report includes artifact type, location, and severity score.

[0015] Thirdly, this application provides a deep learning-based intelligent detection device for multimodal medical image artifacts, employing the following technical solution: A deep learning-based intelligent detection device for multimodal medical image artifacts includes: a memory and a processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0016] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: This application provides a deep learning-based intelligent detection method and system for multimodal medical image artifacts. By extracting modality types and adapting window widths and levels, as well as standardizing grayscale and size, it preprocesses original multimodal medical images, achieving unification across devices and multimodal images. Furthermore, through multi-scale feature extraction and global detection features using convolutional networks, it can determine the presence of artifacts. Then, through a multi-classification network, it identifies artifact types. Finally, through multi-dimensional features of severity scoring, it quantifies the impact of artifacts. Finally, through interpretable heatmaps and knowledge base matching features, it generates a structured report containing type, location, score, and suggestions, achieving intelligent, accurate, and interpretable multimodal medical image artifact detection and evaluation throughout the entire process. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the intelligent detection method for multimodal medical image artifacts based on deep learning proposed in this application.

[0018] Figure 2a This is a flowchart illustrating the intelligent detection method for multimodal medical image artifacts based on deep learning proposed in this application.

[0019] Figure 2b This is a flowchart illustrating the intelligent detection method for multimodal medical image artifacts based on deep learning proposed in this application.

[0020] Figure 3 This is a schematic diagram of the structure of a deep learning-based multimodal medical image artifact intelligent detection system according to this application.

[0021] Figure 4 This is a schematic diagram of the structure of a deep learning-based multimodal medical image artifact intelligent detection device according to this application. Detailed Implementation

[0022] To better understand the purpose, technical solutions, and advantages of this application, it has been described and illustrated below with reference to the accompanying drawings and embodiments. However, those skilled in the art should understand that this application can be implemented without these details. In some cases, to avoid obscuring various aspects of this application due to unnecessary description, well-known methods, processes, systems, components, and / or circuits already described at a higher level will not be elaborated upon. It will be apparent to those skilled in the art that various modifications can be made to the embodiments disclosed in this application, and the general principles defined in this application can be applied to other embodiments and application scenarios without departing from the principles and scope of this application. Therefore, this application is not limited to the illustrated embodiments, but conforms to the broadest scope consistent with the scope of protection claimed in this application.

[0023] It should be noted that the descriptions of these embodiments are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] It should be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementations, the processor may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described herein, or combinations thereof.

[0025] When an embodiment is implemented as software, firmware, middleware, or microcode, program code, or code segments, it may be stored in a machine-readable medium, such as a storage component. A code segment may represent a procedure, function, subroutine, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. One code segment can be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or transmitted using any suitable means, including memory sharing, messaging, token passing, network transmission, etc.

[0026] For software implementations, the techniques described herein can be implemented using modules (e.g., programs, functions, etc.) that perform the functions described herein. The software code can be stored in memory units and executed by a processor. The memory units can be implemented within or outside the processor; in the latter case, the memory units can be communicatively coupled to the processor via various methods known in this art.

[0027] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0028] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0029] Traditional artifact detection relies on manual judgment by staff, which is inefficient, highly subjective, and difficult to cover massive amounts of image data. Moreover, existing detection methods are mostly designed for single-modal images and cannot adapt to the differentiated characteristics of multimodal images such as CT, MRI, and PET. Furthermore, they lack the accuracy to identify subtle artifacts and cannot provide an objective and unified quantitative basis for image quality assessment.

[0030] To address the aforementioned technical challenges, this application proposes an intelligent detection solution capable of adapting to multimodal medical images, accurately detecting various artifacts, and providing standardized evaluation of image quality. This solution aims to resolve issues related to uneven image quality, promote mutual recognition of diagnostic results, and optimize the allocation of medical resources.

[0031] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0032] Figure 1 This is a flowchart illustrating a deep learning-based intelligent detection method for multimodal medical image artifacts according to this application; Figure 1 As shown in the embodiments of this application, a method for intelligent detection of multimodal medical image artifacts based on deep learning is disclosed. The method includes: S101. Receive multimodal raw medical images and preprocess the raw medical images to generate standardized target medical images.

[0033] Raw medical images include commonly used clinical modalities such as MRI (Magnetic Resonance Imaging), CT (Computed Tomography), and PET (Positron Emission Tomography). Their raw data can be stored in DICOM (Digital Imaging and Communications in Medicine) format. DICOM parsing can be used to preprocess the raw medical images. Through systematic extraction and conversion operations, interference caused by the inherent differences in imaging principles of different modalities can be eliminated. For example, CT relies on X-ray attenuation, MRI relies on proton resonance, and PET relies on metabolic signals. By unifying the data format, feature distribution, and spatial scale of the raw medical images, standardized target medical images can be generated.

[0034] DICOM parsing can extract metadata and pixel data from raw medical images. Metadata includes modality type, pixel pitch / slice thickness, window width and window level, patient position, etc.

[0035] For example, in a DICOM file: The labels “CT”, “MR”, and “PT” (0008,0060) can be used to obtain the modality type and can be adapted to different modality preprocessing strategies. The (0028,0030) and ((0018,0050) labels identify the pixel spacing and slice thickness, which can map pixel coordinates to physical space coordinates, ensuring the consistency of spatial scale of cross-modal influences; The labels (0028,1050) and (0028,1051) indicate the window width and window level, and specifically highlight the artifacts and tissue features under different modalities; (0018,5100) Labels indicate the patient's position, such as "supine" or "head first," to assist in anatomical structure localization and verify the rationality of the spatial location of artifacts.

[0036] Extracting pixel data involves converting the original pixel matrix into a processable image format and combining the pixel value range from the metadata to convert the original pixel values ​​into physically meaningful numerical values.

[0037] The original pixel matrix corresponding to the (7FE0,0010) tag in the DICOM file is first decompressed from the pixel data in the compressed format (such as JPEG2000) to restore the original two-dimensional / three-dimensional pixel array. Secondly, by combining the pixel value conversion parameters in the metadata, the original pixel values ​​are converted into quantized values ​​with physical meaning. The pixel value conversion parameters are the Rescale Intercept with the (0028,1052) tag and the Rescale Slope with the (0028,1053) tag. Finally, the grayscale range is cropped according to modal characteristics. For example, CT retains -1000~400HU to cover most tissues and artifacts, and normalization, such as scaling to the 0~1 range using Min-Max, is used to eliminate numerical scale differences caused by different equipment and scanning parameters.

[0038] Multimodal raw medical images are converted into standardized target medical images with uniform size, uniform grayscale distribution, and uniform spatial coordinate system. This process retains the specific characteristics of artifacts in each modality, such as the high-density stripes of CT metal artifacts and the signal loss areas of MRI magnetic susceptibility artifacts, while eliminating format and scale barriers between modalities, thus achieving standardized processing of multimodal raw medical images.

[0039] S102. Perform global detection based on the target medical image to obtain the artifact confidence level.

[0040] The system performs global detection on the target medical image and outputs the artifact confidence score. In other words, the deep learning model can extract the global features of the target medical image and output the probability of "the presence of artifacts". This is used to quickly screen whether there are artifacts in the target medical image. The artifact confidence score is the probability of the presence of artifacts. The higher the artifact confidence score, the greater the probability of artifacts. The lower the artifact confidence score, the lower the probability of artifacts.

[0041] For example, if the confidence level of an artifact is lower than the threshold (e.g., <0.3), it is judged as "no artifact" and directly passes the detection, reducing invalid calculations; If the confidence level of the artifact is higher than the threshold (e.g., >0.7), it is determined that "artifacts definitely exist" and subsequent fine-tuning is triggered; If the confidence level of the artifact is in the middle range (e.g., 0.3~0.7), it is marked as "suspected artifact" and manual review is requested.

[0042] Deep learning models can include ResNet50 backbone networks, lightweight convolutional networks, visual Transformers, multimodal fusion networks, etc.

[0043] Among them, the ResNet50 backbone network relies on residual blocks (Bottleneck structure) and skip connections to solve the gradient vanishing problem in deep networks and achieve multi-scale feature extraction. Standardized target medical images are input into the network, and after global average pooling to compress features, the probability of "artifacts exist" is output through fully connected layers + Softmax, i.e. artifact confidence. Lightweight convolutional networks can be depthwise separable convolutions (MobileNet) or composite scaling strategies (EfficientNet), which maintain feature extraction capabilities while reducing the number of parameters. They are suitable for scenarios with limited computing power. Specifically, they can extract multi-scale features from standardized target medical images and output binary classification probabilities (presence / no artifacts) through global pooling and fully connected layers. The artifact confidence is the probability of "artifacts exist". The Visual Transformer (ViT) segments the target medical image into fixed-size image patches and captures global semantic associations through a self-attention mechanism, avoiding the limitations of the local receptive field of convolutional networks. Specifically, ViT splits the target image into a 16×16 patch sequence, and the Transformer encoder learns global features. The probability distribution is output through the classification head, and the artifact confidence is the probability of "the existence of artifacts". The multimodal fusion network can be CrossViT. For multimodal images, it extracts single-modal features through a two-branch network, and then fuses the intermodal associations through cross-attention, such as matching PET metabolic abnormalities with CT structural artifacts, to improve the robustness of cross-modal detection. It outputs the fused binary classification probability, and the artifact confidence reflects the probability of "artifacts exist" in the multimodal joint judgment.

[0044] S103. When the confidence level of the artifact exceeds the set threshold, the corresponding artifact type is identified according to the preset multi-classification network, and the target medical image is segmented and located at the pixel level based on the artifact type to extract the main artifact region and location, and the severity score is calculated based on the main artifact region.

[0045] When the confidence level of an artifact is lower than a set threshold (e.g., >0.7), it is determined that "an artifact definitely exists" and further fine processing is required.

[0046] The first step in fine-grained processing is to determine the type of artifact. For common artifact types, accurate type identification can be achieved through multi-classification networks. Multi-classification networks typically include a feature extraction module, a feature integration module, and a classification output module.

[0047] The feature extraction module is responsible for extracting key features related to artifacts in the target medical image, such as edges, textures, density / signal changes, etc. It belongs to the "perception layer" of the multi-classification network. Common feature extraction modules can be Convolutional Neural Network (CNN), Visual Transformer (ViT) and its variants, hybrid architectures, etc.

[0048] Among them, CNN can be the encoder part of ResNet (ResNet50 / 101), DenseNet, U-Net, etc., which captures local detailed features through multi-layer convolution and pooling operations, such as radial stripes of metal artifacts and blurred edges of motion artifacts, to achieve low-level feature extraction of single-modality images.

[0049] ViT and its variants, such as Swing Transformer and SegViT, segment target medical images into image patches and model global semantic associations through a self-attention mechanism, such as the cross-regional distribution pattern of large-scale artifacts, to capture the overall shape and location features of artifacts. The hybrid architecture can be ConvNeXt (a combination of convolution and Transformer), which combines the local detail sensitivity of CNNs with the global correlation modeling capability of Transformers, balancing the feature extraction requirements of artifacts at different scales.

[0050] The feature integration module fuses and compresses multi-scale features output by the backbone network to enhance the discriminative features of artifact types. For example, "high density + radial distribution" is a core feature combination of metallic artifacts. Common feature integration modules include global pooling, multi-scale feature concatenation, and attention mechanisms.

[0051] Global pooling compresses high-dimensional feature maps into one-dimensional vectors, thus condensing the features of the entire image.

[0052] Multi-scale feature stitching stitches together feature maps from different stages of the backbone network to enrich feature dimensions, such as shallow detail features plus deep semantic features.

[0053] Attention mechanisms use channel attention or spatial attention to dynamically weight and highlight features that are strongly correlated with artifact types, such as strengthening the feature weight of "signal-deficient regions" for MRI magnetic susceptibility artifacts.

[0054] The classification output module maps the integrated features to preset artifact types and outputs the probability distribution of each category.

[0055] Artifact types are pre-stored in an artifact knowledge base, which includes motion artifact types, metal artifact types, radio frequency interference types, ring artifact types, aliasing artifact types, chemical shift artifact types, magnetic susceptibility artifact types, truncation artifact types, as well as the imaging characteristics, causes, detection strategies, and clinical recommendations for each artifact type.

[0056] Typically, a non-linear mapping from features to class probabilities is achieved through 1 to 2 layers of fully connected networks, and then the output is transformed into a probability vector with a sum of 1 through an activation function.

[0057] For example, let's take the identification of artifact types in chest CT images as an example: The feature integration module extracts multi-scale features related to artifact types from CT images, such as detailed texture, edge morphology, and global distribution. By fusing features at different levels, the feature integration module enhances the discriminative features of artifact types. For example, "high density + radial pattern" is the core feature of metallic artifacts. The classification output module maps the integrated features to various artifact types and outputs the probability distribution of each category. The artifact type can be determined based on the probability distribution of each artifact.

[0058] After determining the artifact type, pixel-level segmentation and severity scoring are performed based on the artifact type to achieve spatial localization and quantitative evaluation of the artifact.

[0059] Different types of artifacts have significant differences in morphology and signal / density characteristics. For example, metal artifacts are high-density radial, while motion artifacts are blurred and patchy. Therefore, a segmentation strategy is designed based on the characteristics of artifact types, and a binary mask with the same size as the target medical image is output, where 1 represents artifacts and 0 represents normal tissue, which can obtain the main artifact regions and locations.

[0060] When selecting a segmentation network for target medical images, semantic segmentation networks with an encoder-decoder structure are usually adopted, such as U-Net and its variants (e.g., U-Net++, Attention U-Net), which fuse low-level details (artifact edges) and high-level semantics (artifact regions) through skip connections.

[0061] The encoder extracts multi-scale features of the target medical image, including low-level features to capture the radial edges of metal artifacts and high-level features to locate the overall distribution area of ​​artifacts. Then, the decoder gradually restores the resolution through upsampling, focuses on the artifact region, and performs morphological operations and region filtering on the artifact region to obtain the main artifact regions and their locations.

[0062] Based on the main artifact regions and the target medical image, a severity score can be obtained according to established unified rules. The severity score is an objective and quantifiable standard, which facilitates rapid decision-making and provides data support for image quality control and equipment performance evaluation.

[0063] S104. Generate an interpretable heatmap based on the target medical image and extract key features from the interpretable heatmap to generate a structured detection report.

[0064] Interpretable heatmaps visually demonstrate "why the model identifies a region as an artifact" by highlighting pixel regions in the image that are strongly correlated with artifact detection, thus solving the "black box" problem and providing visual evidence.

[0065] Interpretable heatmaps can be generated using Grad-CAM (gradient-weighted class activation mapping), Score-CAM, and artifact region enhancement heatmap methods.

[0066] Grad-CAM calculates the gradient of the last convolutional feature map of a multi-classification / segmentation network on the artifact type prediction result. By weighting the feature map with gradient weights, it obtains a heatmap of regions strongly correlated with the artifact type; the larger the gradient, the greater the contribution of that region to artifact identification. Combined with the artifact type identification results, a heatmap focusing on the core region of that type of artifact is generated; for example, high-density radial regions of metallic artifacts appear bright red.

[0067] Score-CAM eliminates the need for gradient calculations. It generates more accurate heatmaps by masking and sampling convolutional feature maps to evaluate the contribution of different feature maps to artifact prediction scores.

[0068] Artifact region enhancement heatmaps overlay pixel-level segmentation masks onto the original image and color-code the main segmented artifact regions, such as red for severe areas, yellow for moderate areas, and blue for mild areas, directly visualizing the intensity distribution of artifacts.

[0069] Key clinically relevant features are extracted from interpretability heatmaps to provide data support for structured reporting. These key features can include artifact core region features, artifact intensity distribution features, and clinically relevant features.

[0070] The structured detection report includes artifact type, location, and severity score. The report can be populated into a preset structured template using a template engine and output in compatible clinical formats, including PDF, DICOM, and JSON.

[0071] This application provides a deep learning-based intelligent detection method for multimodal medical image artifacts. Through DICOM parsing and standardized preprocessing, it eliminates format and scale barriers in multimodal images while preserving artifact-specific characteristics. It utilizes multiple models, including ResNet50 and CrossViT, to achieve global artifact confidence detection, rapidly screening for artifact presence. When the confidence level exceeds a threshold, a multi-classification network accurately identifies the artifact type, combining it with a U-Net variant for pixel-level segmentation and localization, and calculating severity scores using multi-dimensional indicators. Finally, it generates an interpretable heatmap and a structured report containing artifact type, location, and score. This method achieves refined detection of multimodal image artifacts throughout the entire process, from "presence screening" to "type-location-severity," providing standardized and quantifiable clinical decision-making support. It balances detection efficiency, accuracy, and interpretability, providing reliable support for image quality control and clinical diagnosis.

[0072] Figure 2a and Figure 2b A flowchart illustrating a deep learning-based intelligent detection method for multimodal medical image artifacts, as provided in one embodiment of this application, is shown below. Figure 2a and Figure 2b As shown, based on the above embodiments, this embodiment includes the following method: S201, Receive raw medical images in multimodal mode.

[0073] In clinical practice, DICOM is the mainstream storage format for multimodal raw medical images, but there are also a few auxiliary formats, such as NIfTI (Neuroimaging Informatics Technology Initiative), Analyze format, RAW format (raw binary format), and JPEG / JPEG 2000 (compressed image format). These auxiliary formats are usually used for scientific research, special equipment compatibility, or non-diagnostic scenarios.

[0074] After receiving the original multimodal medical images, it is determined whether the original medical image format conforms to the set format. The set format can be DICOM format or multiple formats. For original medical images that do not conform to the set format, no further steps or operations are performed. When the original medical images conform to the set format, subsequent steps and operations can continue.

[0075] S202. Extract pixel data and modality type from the original medical image.

[0076] Pixel data belongs to visual information. The raw pixel data can be extracted through the DICOM tag (7FE0,0010) "Pixel Data". If the data is in a compressed format (such as JPEG 2000, RLE), it is first automatically decompressed by a parsing tool to restore it to a two-dimensional (single slice) or three-dimensional (multi-slice sequence) pixel array. Combined with pixel value conversion parameters, the raw pixel values ​​(grayscale values ​​without physical meaning) are converted into quantized values. Specifically, the tags (0028,1052) "Rescale Intercept" and (0028,1053) "Rescale Slope" are extracted and calculated according to the formula: Physical quantized value = raw pixel value × slope + intercept. For example, CT images are converted into HU values ​​(Henness units), and MRI images are converted into signal intensity values.

[0077] Modality is metadata and is used to distinguish between CT, MRI, and PET image types. Modality is precisely extracted using the DICOM standard label (0008, 0060) "Modality". This information is present in the DICOM files output by all clinical devices. The label values ​​correspond to standardized abbreviations, such as "CT" for computed tomography, "MR" for magnetic resonance imaging, and "PT" for positron emission tomography (PET), which are directly mapped to specific modality types.

[0078] S203. Adapt the corresponding window width and window level according to the modal type.

[0079] Based on the modality type, the corresponding window width and window level are adapted. Taking advantage of the differences in the imaging principles of different modal medical images, such as CT relying on X-ray attenuation and MRI relying on proton resonance, the window width and window level parameters are extracted through DICOM metadata or preset according to clinical standards to highlight the contrast between artifacts and normal tissues.

[0080] Window width and window level are used to enhance the visual difference of the target area (artifacts / normal tissue) by adjusting the grayscale display range (window width) and center grayscale value (window level) of the image. The imaging characteristics of different modalities determine their corresponding window parameters. First, the modality type is confirmed by the DICOM tags (0008, 0060), and then the corresponding adaptation strategy is executed.

[0081] (1) When the modal type is "CT": CT uses HU values ​​to quantify tissue density. Window width and window level need to be optimized for different anatomical locations and artifact types. DICOM preset parameters should be extracted first, and clinical standard values ​​should be used when no presets are available.

[0082] A. Read directly from DICOM metadata tags: • Window Center: Labels (0028, 1050); • Window Width: Label (0028, 1051).

[0083] B. Standard adaptation without presets: • Lung CT: Window level -500HU, window width 1500HU; • Head CT: Window level 30HU, window width 80HU; • Skeletal-related CT: Window level 500HU, window width 2000HU.

[0084] (2) When the modal type is "MR": MRI does not have a unified physical unit of quantification; windowing parameters depend on the scanning sequence (T1WI, T2WI, T2*WI, etc.) and need to be adapted in conjunction with sequence information. First, confirm the sequence type (such as "T2WI", "FLAIR", "T2*GRE") through the DICOM label (0008, 103E) "Series Description"; Prioritize reading DICOM preset window parameters (labels same as CT); if no preset is available, adapt according to the sequence. • T1WI sequence (clear soft tissue contrast): window level 300, window width 800; • T2WI sequence (highlight water signal): window level 600, window width 1200; • T2*WI sequence (sensitive to magnetic susceptibility): window level 200, window width 600.

[0085] (3) When the modal type is "PT": PET uses the SUV value (standard uptake value) to reflect metabolic activity, and the window parameter needs to focus on metabolic abnormalities and attenuation correction artifacts. There are no fixed DICOM preset window parameters; they are set according to clinical routines and artifact detection needs.

[0086] • General-purpose SUV window: Window height 2.5, window width 5.0; • Tumor-related PET-CT fusion scenario: window level 3.0, window width 6.0.

[0087] S204. Adjust the pixel data to the predetermined grayscale range.

[0088] The predetermined grayscale range can be set to 0~255, which can be achieved by formula mapping + threshold truncation. Below the lower limit, it is set to 0, and above the upper limit, it is set to 255.

[0089] The adapted window width and window level are applied to the original physical quantification values ​​(HU value of CT, signal intensity of MRI, SUV value of PET), using the following formula: Mapped grayscale value = 255 × (original value - window level + window width / 2) / window width Values ​​exceeding the window width range are truncated according to a threshold (set to 0 for values ​​below the window level - window width / 2, and set to 255 for values ​​above the window level + window width / 2) to ensure that grayscale values ​​in artifact areas are prominent.

[0090] The adjustment process is based on the adapted window width and window level. The original physical values ​​(HU value, signal strength, SUV value) are mapped to a predetermined range of 0~255 through a quantization formula. At the same time, outliers are truncated, and the grayscale range of pixel data is standardized and adjusted in a standardized manner, while highlighting the grayscale characteristics of artifact areas.

[0091] S205. Scale the image size to a predetermined size to generate a standardized target medical image.

[0092] Based on the preprocessed pixel data of multimodal images that have completed window width and window level adaptation, physical value conversion and grayscale mapping, bilinear interpolation or cubic interpolation algorithm is used to uniformly scale the image to a preset fixed size (such as 224×224, 512×512 pixels, set according to the model input requirements).

[0093] During scaling, the spatial proportions of the image and the relative positions of the anatomical structures are kept constant, and the image is scaled to a fixed size while maintaining the aspect ratio to avoid stretching distortion. At the same time, the target image size is ensured to be consistent across all modalities (CT, MRI, PET) to eliminate spatial scale differences caused by different equipment and scanning parameters.

[0094] S206. Extract multi-scale features of the target medical image through the convolutional layers and residual blocks of the convolutional neural network.

[0095] Multi-scale features include bottom-level edge and texture features as well as high-level semantic features.

[0096] Convolutional neural networks with residual structures, such as ResNet50 / 101 and EfficientNet, are selected as the backbone network. Multi-scale feature extraction is achieved through the stacking of multiple convolutional layers and residual blocks. This application uses ResNet50 as the backbone network.

[0097] The bottom convolutional layers (the first two stages) capture fine-grained features such as the edge contours and local textures of artifacts using small-sized convolutional kernels (e.g., 3×3). Middle convolutional layers (the two middle stages): By increasing the receptive field, they extract the morphological features and local distribution patterns of artifacts; High-level convolutional layers (the last stage): solve the gradient vanishing problem through residual connections, integrate global information, and generate semantic features that include the association between artifacts and anatomical structures.

[0098] S207. Spatial compression of multi-scale feature maps is performed through a global average pooling layer, which transforms the spatial information of each feature channel into global statistical values ​​and generates a one-dimensional feature vector.

[0099] Perform global average pooling on the high-dimensional multi-scale feature map (e.g., 7×7×2048) output of the last layer of the backbone network: The average value of all spatial pixels in each feature channel is taken to compress the original two-dimensional feature map (H×W×C) into a one-dimensional feature vector (1×1×C, where C is the number of feature channels).

[0100] This process condenses the artifact-related features of the entire image, preserving the core information of multi-scale features while significantly reducing the computational complexity of subsequent calculations and avoiding the risk of overfitting.

[0101] S208. The feature vectors are weighted and integrated through a fully connected layer, and the output is transformed into a probability distribution by the Softmax activation function. The probability value corresponding to "artifacts exist" is the artifact confidence.

[0102] The one-dimensional feature vector generated by global average pooling is input into 1-2 fully connected layers. Through non-linear transformation, such as ReLU activation, key artifact features are weighted and integrated, such as the combination of "high-density edges + radial distribution" features. The output dimension of the fully connected layer is 2 (corresponding to the two categories of "artifacts present" and "artifacts absent"). The output is normalized to a probability distribution with a sum of 1 by the Softmax activation function, such as [0.85, 0.15].

[0103] The probability value corresponding to "the presence of artifacts" is the artifact confidence level, which is used to determine whether the image needs further fine processing.

[0104] Assumption: When the confidence level of the artifact is >0.7: artifacts are clearly present, proceed to fine detection; When the confidence level of artifacts is between 0.3 and 0.7: the region is uncertain and should be marked for manual review; When the confidence level of the artifact is <0.3: there is no artifact, and the result is passed directly.

[0105] S209. When the artifact confidence exceeds the set threshold, the target features of the target medical image are extracted through the convolutional layers and residual structures of the multi-classification network.

[0106] The target features include the edges, texture, and semantics of the artifacts.

[0107] Multiclassification networks include a variety of common artifact tasks and other artifact tasks.

[0108] When the confidence level of an artifact exceeds the threshold, it is determined that "there is definitely an artifact" and a multi-classification network is activated for type identification.

[0109] The multi-classification network adopts a "backbone network + feature enhancement module" structure: The backbone network adopts convolutional networks with residual structures, such as ResNet101 and ConvNeXt, and further refines feature extraction through convolutional layers and residual blocks, focusing on capturing the type-specific features of artifacts.

[0110] The target features cover three levels: bottom layer (artifact edge clarity, texture density), middle layer (artifact morphology regularity, distribution range), and high layer (artifact interference patterns with anatomical structures, modality-specific semantics, such as abnormal HU values ​​in CT and signal attenuation in T2* sequences of MRI).

[0111] The multi-classification network supports "common artifact tasks" (with 8 preset core artifacts) and "other artifact tasks". Other artifact tasks reserve extended categories to adapt to rare artifacts, ensuring comprehensive type coverage.

[0112] S210. The multi-classification network performs classification operations on the extracted target features and outputs the artifact probability value for each class of artifacts.

[0113] The classification head of the multi-class network consists of fully connected layers. The input is the target feature vector extracted and enhanced by the backbone network. The features are transformed into raw scores corresponding to the number of artifact types through nonlinear mapping (e.g., 8 raw scores are output for 8 artifact types). The raw scores are then subjected to Softmax normalization to output the probability value of each artifact type (the sum is 1), for example, [metal artifact: 0.92, motion artifact: 0.03, others: 0.05].

[0114] S211. Sort the artifact probability values ​​of various types of artifacts in the output, select the K artifact probability values ​​with the highest probability values, and output the artifact type corresponding to the artifact probability value.

[0115] The probability values ​​of all artifact types are sorted from high to low, and the K value is preset to 1~2. In clinical scenarios, the main artifact type is the core, and K=1 is the default value; if multiple types of artifacts are superimposed and their probabilities are all high, K=2.

[0116] If the highest probability value is higher than the confidence threshold (e.g., >0.7), the corresponding artifact type is output directly; if the highest probability value is lower than the threshold (e.g., <0.3), it is marked as "suspected artifact" and manual review is triggered.

[0117] There is a one-to-one correspondence between artifact types and artifact knowledge bases. Artifact types are pre-stored in the artifact knowledge base, and the artifact types in the artifact knowledge base correspond to the artifact tasks in multi-classification networks.

[0118] As shown in Table 1, the artifact knowledge base includes motion artifact types, metal artifact types, radio frequency interference types, ring artifact types, aliasing artifact types, chemical shift artifact types, magnetic susceptibility artifact types, truncation artifact types, as well as the image characteristics, causes, detection strategies, and clinical recommendations for each artifact type.

[0119] Table 1: Artifact Knowledge Base

[0120] S212. Based on the artifact type, call the corresponding U-Net pixel-level segmentation model, and use the U-Net pixel-level segmentation model to predict the artifact probability value of each pixel of the target medical image, and output a pixel-level probability map.

[0121] The artifact knowledge base stores dedicated U-Net segmentation models for various artifacts, such as U-Net for metal artifacts and U-Net for motion artifacts. It can automatically call the appropriate model based on the artifact type.

[0122] The U-Net model adopts an "encoder-decoder + skip connection" structure: the encoder extracts multi-scale features of artifacts, the decoder restores pixel-level resolution through upsampling, and the skip connection fuses low-level edge features and high-level semantic features to improve segmentation accuracy.

[0123] The model predicts the probability value (0~1) of each pixel in the target medical image as an artifact, and outputs a pixel-level probability map that is exactly the same size as the original image. The coordinates of each pixel correspond one-to-one with the spatial position of the original image, thus achieving pixel-level localization of artifacts.

[0124] S213. Perform morphological operations on the pixel-level probability map to smooth the boundaries of artifact regions.

[0125] First, set a probability threshold (e.g., 0.5) for the pixel-level probability map, mark pixels with a probability ≥ 0.5 as candidate artifact pixels, generate a preliminary binarized mask, and then perform morphological operations: Erosion operation: Removes isolated noise points in the mask, such as connected regions of less than 3 pixels, to eliminate segmentation misjudgments; Dilation operation: fills in tiny holes inside the artifact region, smooths the region boundary, and makes the artifact region more complete in shape; The final optimized mask with regularized boundaries and noise filtering is obtained.

[0126] S214. Identify and extract connected regions with artifact probability values ​​higher than a set threshold to generate the main artifact region, and record the spatial position of the main artifact region in the target medical image.

[0127] Based on the morphologically optimized mask, a connected component analysis algorithm is used to identify all connected components with artifact probability values ​​≥ a set threshold. Sort by the area of ​​connected regions, retain the top 3 regions by area as the main artifact regions, and ignore minor interference regions. Record the spatial location of the main artifact areas: including pixel coordinate bounding boxes, top left corner (x1, y1), bottom right corner (x2, y2), center point coordinates (x0, y0), and combine them with the pixel spacing in the DICOM metadata to convert them into physical coordinates (unit: mm) to ensure clinical positioning accuracy.

[0128] S215. Obtain the total number of pixels in the target medical image and the total number of artifact pixels in the main artifact region, and obtain the area ratio based on the ratio of the total number of artifact pixels to the total number of pixels.

[0129] First, count the total number of pixels in the target medical image, such as 224 × 224 = 50176 pixels. Then, count the total number of artifact pixels in all major artifact regions, which is the sum of the number of pixels in each connected region. Finally, calculate the area percentage using the formula: Area percentage = (Total number of artifact pixels / Total number of image pixels) × 100%; For example, if the total number of artifact pixels is 12,544 and the total number of pixels is 50,176, then the area ratio is 25%.

[0130] S216. Based on the coverage of the main artifact areas and key anatomical areas, the key structures that affect the structure are obtained.

[0131] Key anatomical regions are pre-marked on their corresponding spatial locations in the target medical images. These key anatomical regions are pre-registered with the target medical images using anatomical structure template libraries, such as lung lobes, bronchi, and pulmonary nodules from chest CT, and hippocampus, gray matter, and white matter from brain MRI, thus marking their spatial range. Calculate the percentage of overlap between the main artifact region and each key anatomical region: Overlap percentage = (Number of pixels overlapping between artifacts and key regions / Total number of pixels in the key region) × 100%; If the overlap ratio is ≥30%, it is determined to "affect key structures", and the corresponding key anatomical region name is recorded, such as "right upper lobe nodule region" or "left hippocampus". If the overlap of all critical areas is less than 30%, it is determined that "it does not affect the critical structure".

[0132] S217. Calculate the confidence level of severity.

[0133] The severity confidence level is the ratio of the sum of the probability values ​​of all artifacts in the main artifact region to the total number of pixels in the main artifact region.

[0134] First, extract the artifact probability values ​​of all pixels within the main artifact region of the pixel-level probability map. Calculate the sum of the probability values ​​of all pixels within this region and average it with the total number of pixels in the main artifact region to obtain the severity confidence score. For example, if the sum is 10032 and the total number of pixels is 12544, then the severity confidence score = 0.8.

[0135] The severity confidence level reflects the "artifact purity" of the main artifact region. The higher the value, the stronger the certainty that the pixels in the region are artifacts and the more significant the interference.

[0136] S218. The severity score is obtained based on the area proportion, the impact on key structures, and the severity confidence level.

[0137] The severity scoring formula is: Severity = α·area percentage + β·impact on key structures + γ·confidence level Among them, α, β, and γ are weighting coefficients, and their specific values ​​are related to clinical needs, data statistics, and model validation.

[0138] For example, severity rating levels can be set: – 1-3 points: Mild, acceptable; –4-6 points: Moderate, follow-up examination recommended; -7-10 points: Severe, requires rescanning.

[0139] S219. Calculate the gradient-weighted feature map of the last convolutional layer of the model using the Grad-CAM algorithm, upsample it, and overlay it with the target medical image to generate an interpretable heatmap.

[0140] First, the target medical image is input into a multi-classification network, the last convolutional feature map is extracted, the gradient of the feature map for the prediction of artifact type is calculated, and then the feature map is weighted and summed using the gradient mean as the weight to obtain the original heatmap.

[0141] The original heatmap is upsampled to the same size as the target medical image using bilinear interpolation. After normalization (0~255) to adjust the brightness, it is then semi-transparently overlaid on the original image. In the heatmap, red indicates the core area with the highest contribution to artifact detection, orange-yellow indicates the edge area, and normal tissue has no highlighting, which intuitively explains the basis of the model's judgment.

[0142] S220. Extract key features of the interpretable heatmap, including brightness, contrast, texture, and shape.

[0143] Brightness: The average brightness value of the heatmap (reflecting the overall intensity of artifacts), and the proportion of bright areas (brightness > 200); Contrast: The difference in brightness between the highlighted areas and the background areas on a heatmap (reflecting the ability to distinguish artifacts from normal tissue); Texture: Gray-level co-occurrence matrix of heatmap (reflects the texture uniformity of artifact regions, such as the radial texture of metallic artifacts and the blurred texture of motion artifacts); Shape: The aspect ratio and roundness of the bounding rectangle of the highlighted area on the heatmap (reflecting the morphological characteristics of the artifact, such as the roundness of a ring artifact being close to 1).

[0144] S221. Compare key features with impact features in the artifact knowledge base to determine the artifact types in the artifact knowledge base.

[0145] The artifact knowledge base stores feature templates for various artifacts, such as metallic artifacts (high brightness, high contrast, radial texture) and magnetic susceptibility artifacts (low brightness signal missing area, irregular shape). The similarity between the key features of the heatmap and the feature templates of each artifact type in the knowledge base is calculated, such as cosine similarity and Euclidean distance. The artifact type with the highest similarity is taken as the final confirmation result and cross-validated with the type output in step S211. If they match, the type is locked; otherwise, it is marked as "type pending verification," prompting manual confirmation.

[0146] S222, Generate a structured inspection report.

[0147] The structured report includes basic image information (patient ID, modality information, scan time), artifact type (final confirmation result), location (physical coordinates + anatomical location), severity score, clinical recommendations (based on knowledge base matching), and visual evidence (original medical images, interpretable heatmaps, etc.). The report output format is compatible with commonly used clinical standards (PDF, DICOM structured report, JSON) and can be directly embedded into PACS systems.

[0148] Furthermore, once a structured report is generated, it can be automatically pushed out, triggering the image quality control process.

[0149] This application provides a deep learning-based intelligent detection method for multimodal medical image artifacts. Through format verification and DICOM parsing, it accurately extracts pixel data and modal information. Combining modal characteristics with adaptive window width and level, standardized grayscale range, and image size, it eliminates multimodal format and scale barriers. By extracting multi-scale features and calculating artifact confidence, it quickly completes artifact presence screening. When the artifact confidence exceeds a threshold, a multi-classification network accurately identifies the artifact type, and a dedicated U-Net model is used for pixel-level segmentation and localization. A score is then calculated by combining area proportion, key structural influence, and severity confidence. Finally, Grad-CAM generates an interpretable heatmap, cross-validates artifact types, outputs a standardized structured report, and pushes it to the quality control system. This method achieves refined artifact detection throughout the entire process, from "presence screening" to "type-location-severity," balancing efficiency, accuracy, and interpretability, providing reliable support for clinical diagnosis and image quality control.

[0150] Figure 3 This application provides a schematic diagram of the structure of a deep learning-based multimodal medical image artifact intelligent detection system, as shown in one embodiment. Figure 3 As shown, the multimodal medical image artifact intelligent detection system 30 based on deep learning provided in this embodiment includes: The preprocessing module 301 is used to receive multimodal raw medical images and preprocess the raw medical images to generate standardized target medical images. The global detection module 302 is used to perform global detection based on the target medical image to obtain the artifact confidence score, where the artifact confidence score is the probability of the existence of an artifact. The type judgment and scoring module 303 is used to identify the corresponding artifact type according to the preset multi-classification network when the confidence of the artifact exceeds the set threshold, and to perform pixel-level segmentation and localization of the target medical image based on the artifact type to extract the main artifact region and location, and to calculate the severity score based on the main artifact region. The report generation module 304 is used to generate an interpretable heatmap based on the target medical image and extract key features from the interpretable heatmap to generate a structured detection report, wherein the structured detection report includes artifact type, location, and severity score.

[0151] This embodiment provides a deep learning-based multimodal medical image artifact intelligent detection system, which can execute the methods provided in the above-described method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0152] Figure 4 This application provides a schematic diagram of the structure of a deep learning-based multimodal medical image artifact detection device according to an embodiment of the present application. Figure 4 As shown in the figure, this embodiment provides a multimodal medical image artifact intelligent detection device 40 based on deep learning, comprising: The device 40 includes at least one processor 401 and a memory 402. Optionally, the device 40 also includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0153] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0154] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0155] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0156] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0157] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0158] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0159] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0160] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0161] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0162] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0163] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0165] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0167] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A deep learning-based intelligent detection method for multimodal medical image artifacts, characterized in that, The method includes: Receive multimodal raw medical images and preprocess them to generate standardized target medical images; Global detection is performed based on the target medical image to obtain the artifact confidence score, wherein the artifact confidence score is the probability of the existence of an artifact; When the confidence level of an artifact exceeds a set threshold, the corresponding artifact type is identified according to a preset multi-classification network. The artifact type is pre-stored in an artifact knowledge base, and the target medical image is segmented and located at the pixel level based on the artifact type to extract the main artifact region and location, and the severity score is calculated based on the main artifact region. An interpretable heatmap is generated based on the target medical image, and key features are extracted from the interpretable heatmap to determine the artifact types in the artifact knowledge base. A structured detection report is then generated, which includes artifact type, location, and severity score.

2. The method according to claim 1, characterized in that, The preprocessing of the original medical images to generate standardized target medical images includes: Extract pixel data and modality type from raw medical images; Adapt the corresponding window width and window level according to the modality type; Adjust the pixel data to the predetermined grayscale range; The image size is scaled to a predetermined size to generate a standardized target medical image.

3. The method according to claim 2, characterized in that, The global detection based on the target medical image to obtain artifact confidence includes: Multi-scale features of the target medical image are extracted by using convolutional layers and residual blocks of a convolutional neural network. These multi-scale features include bottom-level edge and texture features as well as high-level semantic features. The multi-scale feature map is spatially compressed by a global average pooling layer, which transforms the spatial information of each feature channel into global statistical values ​​and generates a one-dimensional feature vector. The feature vectors are weighted and integrated by a fully connected layer, and the output is transformed into a probability distribution by the Softmax activation function. The probability value corresponding to the existence of artifacts is the artifact confidence.

4. The method according to claim 3, characterized in that, The multi-classification network includes various common artifact tasks and other artifact tasks. The step of identifying the corresponding artifact type based on a preset multi-classification network includes: The target features of the target medical image are extracted by using the convolutional layers and residual structure of a multi-classification network. The target features include the edges, textures, and semantics of artifacts. The multi-classification network performs classification operations on the extracted target features and outputs the artifact probability value for each class of artifacts; The artifact probability values ​​of the output artifacts are sorted, the K artifact probability values ​​with the highest probability values ​​are selected, and the artifact type corresponding to the artifact probability value is output. The artifact type is pre-stored in the artifact knowledge base, and the artifact type in the artifact knowledge base corresponds to the artifact task in the multi-classification network.

5. The method according to claim 4, characterized in that, The pixel-level segmentation and localization of the target medical image based on artifact type to extract the main artifact regions and locations includes: According to the artifact type, the corresponding U-Net pixel-level segmentation model is called, and the artifact probability value of each pixel of the target medical image is predicted by the U-Net pixel-level segmentation model, and a pixel-level probability map is output. The coordinates of each pixel in the pixel-level probability map correspond one-to-one with the spatial position of the target medical image. Morphological operations are performed on pixel-level probability maps to smooth the boundaries of artifact regions; Identify and extract connected regions with artifact probability values ​​higher than a set threshold to generate the main artifact region, and record the spatial location of the main artifact region in the target medical image.

6. The method according to claim 5, characterized in that, The calculation of severity scores based on major artifact regions includes: Obtain the total number of pixels in the target medical image and the total number of artifact pixels in the main artifact region. Calculate the area percentage based on the ratio of the total number of artifact pixels to the total number of pixels. Based on the coverage of the main artifact areas and key anatomical areas, the key structures that affect the image are obtained. The key anatomical areas are pre-marked on the spatial location of the corresponding target medical image. Calculate the severity confidence score, which is the ratio of the sum of the probability values ​​of all artifacts in the main artifact region to the total number of pixels in the main artifact region; The severity score is obtained based on the area proportion, the impact on key structures, and the severity confidence level.

7. The method according to claim 4, characterized in that, The artifact knowledge base includes motion artifact types, metal artifact types, radio frequency interference types, ring artifact types, aliasing artifact types, chemical shift artifact types, magnetic susceptibility artifact types, truncation artifact types, as well as the image characteristics, causes, detection strategies, and clinical recommendations for each artifact type.

8. The method according to claim 7, characterized in that, The process of generating an interpretable heatmap based on the target medical image and extracting key features from the interpretable heatmap to generate a structured detection report includes: The gradient-weighted feature map of the last convolutional layer of the model is calculated using the Grad-CAM algorithm, upsampled, and then superimposed on the target medical image to generate an interpretable heatmap. Extract key features from the interpretable heatmap, including brightness, contrast, texture, and shape; By comparing key features with the impact features in the artifact knowledge base, the artifact types in the artifact knowledge base are determined. Generate a structured detection report, which includes artifact type, location, severity score, and clinical recommendations.

9. A deep learning-based intelligent detection system for multimodal medical image artifacts, characterized in that, The system includes: The preprocessing module receives multimodal raw medical images and preprocesses them to generate standardized target medical images. A global detection module is used to perform global detection based on the target medical image to obtain the artifact confidence score, wherein the artifact confidence score is the probability of the existence of an artifact. The type identification and scoring module is used to identify the corresponding artifact type according to a preset multi-classification network when the confidence of the artifact exceeds a set threshold. The artifact type is pre-stored in the artifact knowledge base, and the target medical image is segmented and located at the pixel level based on the artifact type to extract the main artifact region and location, and the severity score is calculated based on the main artifact region. The report generation module is used to generate an interpretable heatmap based on the target medical image, extract key features from the interpretable heatmap to determine the artifact types in the artifact knowledge base, and generate a structured detection report, wherein the structured detection report includes artifact type, location, and severity score.

10. A deep learning-based intelligent detection device for multimodal medical image artifacts, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

Images