Multimodal medical image quality inspection system and method based on deep learning

Abstract

The application discloses a multi-modal medical image quality inspection system and method based on deep learning, relates to the technical field of medical image processing, and comprises a multi-modal data acquisition and preprocessing module, a deep feature extraction and fusion module, a quality evaluation and artifact detection module, a segmentation and quantitative analysis module, a verification and quality inspection control module, a knowledge retrieval and report generation module, and a result output and visualization module.The application adopts the above-mentioned multi-modal medical image quality inspection system and method based on deep learning, directly predicts the comprehensive quality scores of each mode from the multi-modal deep fusion feature map through a no-reference quality regression network, and simultaneously identifies specific types such as motion artifacts, noise artifacts and beam hardening artifacts, thereby getting rid of the dependence on reference images.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a multimodal medical image quality inspection system and method based on deep learning. Background Technology

[0002] Medical imaging technology has become an indispensable core tool in modern clinical diagnosis, treatment planning, and prognostic assessment. With the continuous advancement of imaging equipment, multiple modalities of imaging data are often required for the same patient—each modality possessing its unique imaging principles and informational advantages: CT offers high resolution for bones and calcifications, MRI provides excellent contrast for soft tissues, PET reflects tissue metabolic activity, and ultrasound offers real-time imaging. Multimodal imaging data can reveal a patient's anatomical structure and pathophysiological state from different dimensions, providing a rich information foundation for accurate diagnosis.

[0003] However, the clinical value of multimodal medical imaging is highly dependent on the quality of the image data. Studies have shown that in over 1.2 million radiological examinations, approximately 2.4% of images have quality issues affecting diagnostic reliability, while in high-precision imaging modalities such as PET, CT, and MRI, this proportion can rise to 6%. The main sources of image quality degradation include motion artifacts, beam hardening artifacts, noise contamination, and image blurring. These quality issues significantly affect subsequent image segmentation accuracy, feature extraction stability, and the reliability of quantitative measurements. Therefore, systematic quality assessment and screening of images before they enter the quantitative analysis and clinical decision-making process is a crucial preliminary step to ensure diagnostic accuracy.

[0004] In existing technologies, the quality control of multimodal medical images has the following main shortcomings: First, traditional quality assessment metrics are difficult to adapt to the clinical needs of multimodal medical imaging. Currently, commonly used clinical metrics such as signal-to-noise ratio, contrast-to-noise ratio, and structural similarity are mainly designed for natural images or general image processing tasks, and are difficult to capture local texture distortions or structural abnormalities that have a significant impact on downstream diagnostic tasks.

[0005] Secondly, existing deep learning methods are mostly designed as single-modal, single-task approaches, lacking the ability to fuse and comprehensively evaluate multiple modalities. Evaluation results often only provide a single quality score, failing to identify and locate artifact types, and even more so lacking a comprehensive assessment of the complementarity and consistency between multimodal images.

[0006] Third, existing medical image analysis systems lack post-segmentation quality verification and invalid result blocking mechanisms. Segmentation models may produce anatomically unreasonable results when faced with images whose distribution is inconsistent with the training data. In the existing pipeline architecture, segmentation results, regardless of quality, are directly passed to downstream modules, leading to erroneous quantification values ​​being written into clinical reports, which may mislead treatment decisions.

[0007] Fourth, there is a lack of effective mechanisms to address the problem of missing modalities in multimodal image fusion. In real clinical scenarios, complete multimodal data is often difficult to obtain due to limitations in scanning time, insufficient equipment availability, or high costs. Existing technologies either simply stitch together available modalities or use computationally complex generative models to complete the data when modalities are missing, neither of which can be stably deployed in a clinical environment. Summary of the Invention

[0008] The purpose of this invention is to provide a multimodal medical image quality inspection system and method based on deep learning, thereby solving the problems mentioned in the background art.

[0009] To achieve the above objectives, this invention provides a deep learning-based multimodal medical image quality inspection system, comprising: The multimodal data acquisition and preprocessing module is used to acquire raw medical image data, perform preprocessing, and output standardized multimodal image data and frequency domain enhanced image data. The deep feature extraction and fusion module includes a CNN local feature extraction submodule, a Transformer global feature extraction submodule, and a cross-modal attention fusion submodule. It is used to extract local texture edge features and global contextual features of each modality image, and output a multimodal deep fusion feature map by adaptively weighting and fusing the multimodal features. The quality assessment and artifact detection module includes a no-reference quality score prediction submodule, an artifact type identification submodule, and a confidence estimation submodule. It is used to generate quality scores for each modality of image based on deep fusion feature maps, identify artifact types and estimate confidence, and output the quality assessment vector of each modality of image and the overall quality passability judgment result. The segmentation and quantification analysis module includes a multimodal segmentation submodule and a quantification measurement submodule, which are used to perform multimodal collaborative segmentation, calculate quantitative measurement values ​​based on the segmentation results, and output structured quantification measurement results; The verification and quality control module includes a segmentation quality verification submodule and a measurement rationality verification submodule. It is used to perform connectivity analysis and anatomical structure existence verification on the spatial integrity and structural consistency of the segmentation output, as well as to compare and verify the quantitative measurement results with the preset physiological reference range, and generate a quality inspection pass signal or a quality inspection fail signal and a corresponding abnormal report based on the verification results. The knowledge retrieval and report generation module includes a knowledge retrieval submodule and a structured report generation submodule. It is used to retrieve the corresponding physiological reference range and clinical guideline information from the external knowledge base based on the quality assessment vector, quantitative measurement results and preset retrieval query statements, and generate a structured clinical quality inspection report. The results output and visualization module is used to visualize and export structured clinical quality inspection reports in a combination of text and graphics.

[0010] The method of a deep learning-based multimodal medical image quality inspection system includes the following steps: S1. Acquire raw medical image data, preprocess the raw medical image data to obtain standardized multimodal image data and frequency domain enhanced image data; S2. Based on standardized multimodal image data and frequency domain enhanced image data, extract local texture edge features and global contextual features of each modality image, and perform adaptive weighted fusion of multimodal features to obtain a multimodal deep fusion feature map; S3. Based on multimodal deep fusion feature maps, predict the comprehensive quality score of each modal image, identify artifact types and estimate prediction confidence, and generate quality assessment vectors and comprehensive quality qualification results for each modal image according to the comprehensive quality score, artifact type and confidence. S4. Based on the comprehensive quality compliance assessment results, perform multimodal collaborative segmentation on the image data that has passed the quality compliance assessment to obtain the segmentation mask of the anatomical structure or lesion area, and then calculate the quantitative measurement value and output the structured quantitative measurement result. S5. Perform connectivity analysis, anatomical structure existence verification, and spatial consistency check on the segmentation mask to generate segmentation quality verification results; compare and verify the quantitative measurement results with the preset physiological reference range to generate measurement rationality verification results; generate quality inspection pass or quality inspection fail signals and corresponding abnormal reports based on the segmentation quality verification results and measurement rationality verification results. S6. Based on the quality assessment vector, quantitative measurement results and preset search query statements, retrieve knowledge information from the external knowledge base, and integrate the quantitative measurement results, quality assessment results and retrieved knowledge information into a structured clinical quality inspection report. S7. Visualize and export structured clinical quality inspection reports in a combination of text and graphics.

[0011] Preferably, S1 includes: S11. Multimodal raw image data acquisition: Acquiring raw medical image data of at least two modalities of the same patient using medical imaging equipment. ,in Indicates modal index, ; S12. Perform offset field correction processing: This involves processing the raw medical image data... The N4ITK offset field correction algorithm is used for preprocessing to suppress low-frequency intensity artifacts in the image, resulting in an offset-corrected image. ; S13. Perform uniform spatial resampling preprocessing: preprocess the offset-corrected image... Resampling to the preset spatial resolution is performed using trilinear interpolation. To obtain spatially standardized images ; S14. Perform discrete wavelet transform frequency domain decomposition and high-frequency noise suppression preprocessing on spatially normalized images: Each axial slice is subjected to a two-dimensional discrete wavelet transform, and a single-layer decomposition is performed using the Daubechies-4 wavelet basis, decomposing the image into low-frequency approximate components and high-frequency detail components in three directions. , , ; S15. Perform high-frequency noise suppression and enhancement reconstruction processing: For high-frequency detail components... , , Soft threshold noise suppression is performed to suppress high-frequency noise while preserving key edge information. The threshold is set as follows: ; ; ; ; In the formula, It is a symbolic function; For the first Modal adaptive noise thresholding based on diagonal high-frequency components The median estimate of the noise level; The denoised high-frequency components , , Approximate components with low frequency Weighted fusion reconstruction is performed to obtain frequency domain enhanced image. : ; In the formula, This is a bilinear interpolation upsampling operation that restores low-frequency components to their original spatial resolution. ; Retain weights for low-frequency structures. Weights are preserved for high-frequency details, determined through cross-validation, to achieve a balance between maintaining the integrity of anatomical structures and suppressing high-frequency noise. S16. Image Normalization: For frequency domain enhanced images Channel-level standardization is performed to obtain standardized multimodal image data. : ; In the formula, For the first Channel mean of modal image, For the first Channel standard deviation of modal images.

[0012] Preferably, S2 includes: S21, CNN Local Feature Extraction: This involves extracting standardized multimodal image data. The input is the CNN local feature extraction submodule, which uses a pre-trained EfficientNet-B0 as the backbone network. The MBConv module within this submodule extracts local texture and edge features from each modality of the image, resulting in local feature maps for each modality. ,in For the first The number of feature channels of a modality , The spatial dimension of the feature map; S22, Transformer Global Feature Extraction: This involves extracting standardized multimodal image data. The input is the Transformer global feature extraction submodule, which uses the Swing Transformer as the backbone network. It extracts the global contextual features of each modality's image through its shifted window self-attention mechanism, thus obtaining the global feature map for each modality. ; S23. Cross-modal attention fusion: integrating local feature maps from various modalities. With global feature map By concatenating the data along the channel dimension, we obtain the complete feature representations of each modality. Then, the multimodal features are adaptively weighted and fused through the cross-modal attention fusion submodule.

[0013] Preferably, S23 includes: S231. Represent the complete features of each modality. Through sharing The convolutional layers are projected onto a unified embedding space to obtain the embedding features. ,in To unify the embedding dimensions; S232, Embedded features of the first modality As a query, the embedding features of the other modalities As keys and values, calculate the cross-modal attention weights: ; ; In the formula, For the first mode to the first The attention weight matrix of the modality. For the first Attention-enhanced features resulting from first-modal query weighting of modal features; S233. Perform a weighted summation of the attention enhancement features across all modalities to obtain a multimodal deep fusion feature map. : ; in, , For the first The adaptive fusion weights of the modalities are calculated through a learnable gating network: ; In the formula, This is a global average pooling operation. It is a multilayer perceptron used to map global features into scalar weights.

[0014] Preferably, S3 includes: S31. Deeply fuse feature maps from multiple modalities. The input is a no-reference quality regression network, which consists of a global average pooling layer and three fully connected layers. The output is a comprehensive quality score for each modality of the image. ; S32. Deeply fuse feature maps from multiple modalities. Input the artifact classification network, which employs a multi-label classification architecture, and output the artifact type probability vector for each modality of image. ,in The total number of preset artifact categories; S33. Deeply fuse feature maps of multiple modalities. The input is a confidence estimation network, which employs a Bayesian neural network architecture. Uncertainty is quantified using Monte Carlo Dropout, and the output is the predictive confidence of the quality scores for each modality. : ; In the formula, For the first The quality score obtained from the Monte Carlo Dropout forward propagation. The number of samples (usually taken as...) ), For variance calculation; S34. Quality scores for each modality. Artifact type probability vector and confidence level Combined into a quality assessment vector ; S35. Based on the quality assessment vector Generate the comprehensive quality pass / fail judgment results for each modal image. ; when Time indicates the first Modal images pass quality compliance assessment when This indicates that the test was not passed. If at least one mode passes the quality compliance assessment, the overall quality compliance assessment is passed.

[0015] Preferably, S4 includes: S41. Based on the overall quality compliance assessment results output by S35, determine whether to perform segmentation and quantitative analysis: If the overall quality assessment is deemed unqualified, skip the subsequent segmentation and quantitative analysis steps and proceed directly to S6 to generate a quality inspection report indicating that the image quality is unqualified and re-acquisition is recommended; if at least one modality passes the quality assessment, proceed to S42. S42, from standardized multimodal image data The modes that pass the quality compliance judgment are selected from the samples. This constitutes a usable modal image set. ;like If a modality is missing, the missing modality compensation mechanism is activated, and S43 is executed. S43, when At the same time, features of missing modalities are compensated by a cross-modal feature generation network. The cross-modal feature generation network employs a conditional generative adversarial network architecture, using deep fusion features from available modalities. Given the condition, generate approximate features for the missing modes. : ; In the formula, For generator networks, This is a random noise vector used to introduce generative diversity; it generates missing modal features. It is concatenated with available modal features to form a complete multimodal feature set; S44. Available modal image data The input is a multimodal segmentation submodule, which adopts a 3DU-Net architecture and performs multimodal collaborative segmentation through an encoder-decoder structure. The encoder fuses information from various modalities through multi-channel input and extracts multi-scale semantic features by downsampling layer by layer. The decoder restores spatial resolution through skip connections and finally outputs a probabilistic segmentation mask for each target structure. ,in Index the target structure. The total number of target structures; The segmentation output of the multimodal segmentation submodule generates a multi-class probability map through a pixel-wise Softmax activation function. ; S45. Probability segmentation mask Using threshold Binarization is performed to obtain a binary segmentation mask. ; S46, Based on binary segmentation mask The quantitative measurement submodule calculates the quantitative measurement values ​​of each target structure and outputs the quantitative measurement results of the structure. .

[0016] Preferably, S5 includes: S51. Connectivity Analysis of Split Masks: Analysis of Binary Split Masks Perform three-dimensional connectivity analysis and calculate the maximum connectivity volume percentage of each target structure. ; like ,in, The threshold for connectivity percentage is taken as follows: If the segmentation result of the target structure is found to have a fragmentation problem, a connectivity anomaly marker is generated. ; S52, Check the binary segmentation mask Does it include the expected anatomical structure? Calculate the total number of voxels for each target structure. ; like ,in, If a preset minimum structure voxel threshold is set based on the physiological size of the target structure, then the target structure is determined to be missing or segmentation failed, and a structure missing marker is generated. ; S53, For target structures that are anatomically adjacent or have spatial constraints Perform spatial consistency verification and calculate its spatial overlap. ; like ,in, The spatial overlap threshold is taken as follows: This indicates that two anatomically non-overlapping structures have abnormally overlapped, generating a spatial consistency aberration marker. ; S54. Based on the above verification results, generate segmentation quality verification results. If any anomaly is marked as 1, the segmentation quality verification fails. S55. Quantize the measurement results output by S46. With respect to the preset physiological reference range Perform comparison and verification: ; In the formula, For the first Measurement values ​​of the target structure, and These are the preset lower and upper limits for the measured value, respectively; if If the measurement result is found to be outside the reasonable range, an abnormal measurement reasonableness marker will be generated. S56. Quality control signal generation: Based on the segmentation quality verification results. and measurement rationality verification results Generate quality control signals : ; when A quality inspection pass signal is generated at the time. It generates a quality inspection failure signal and a corresponding anomaly report in real time; S57. When a quality inspection failure signal is generated, a termination command is sent to S4 to prevent the current segmentation result and quantization measurement result from being propagated to S6. At the same time, the anomaly type and location information are recorded to form a quality inspection anomaly log.

[0017] Preferably, S6 includes: S61. Based on the quality assessment vector output by S3 Quantization measurement results output by S46 Construct knowledge retrieval query statements ; S62, query statement Submit to an external knowledge base for retrieval and return a set of retrieval results. ,in, For the first 1 search result; S63. Regarding the search result set Each result in the dataset has a relevance score calculated. Sort by relevance score in descending order, and select the top 3 results with the highest relevance as reference knowledge. ; S64. The quality assessment vector output by S3 Quantization measurement results output by S46 The measurement rationality verification results output by S55 S56 output quality control signal and reference knowledge output by S63 The input report generation submodule generates structured clinical quality inspection reports. This includes: patient information and image acquisition parameters, image quality scores and artifact identification results for each modality, segmentation and quantification measurement results and comparison with reference ranges, quality inspection conclusions and clinical recommendations.

[0018] Preferably, S7 includes: S71. The structured clinical quality inspection report generated in S64 The system is formatted and rendered according to a preset visualization template to generate a visual quality inspection report page that combines text and graphics. S72, Apply the probability segmentation mask generated in S44 Or binary segmentation mask A semi-transparent pseudo-color overlay is superimposed on the original image to generate a visualization of the segmentation results; S73. The quality score generated in S31 The artifact type probability vector generated by S32 and the confidence level generated by S33 Visualize the data using bar charts, radar charts, or heat maps. S74. Export the visual quality inspection report combining text and images into a standardized format file and store it in the hospital information system or image archiving and communication system.

[0019] Therefore, the present invention employs the above-mentioned deep learning-based multimodal medical image quality inspection system and method, which has the following beneficial effects: (1) This invention predicts the comprehensive quality score of each modality directly from the multimodal deep fusion feature map through a referenceless quality regression network, and simultaneously identifies specific types such as motion artifacts, noise artifacts, and beam hardening artifacts, thus eliminating the dependence on reference images. At the same time, the prediction confidence is estimated by Monte Carlo dropout method, which enables the system to automatically trigger manual review or re-acquisition in low confidence scenarios, effectively reducing the risk of model misjudgment.

[0020] (2) This invention uses the features of each modality as mutual queries and keys, and explicitly models the deep correlation and complementarity between modalities through attention weight matrix to achieve adaptive weighted fusion of multimodal information. When some modalities are missing due to clinical reasons, approximate features of the missing modalities are generated based on available modal features, so that the system can maintain stable performance even when the modalities are incomplete, effectively addressing the common problem of modal missingness in clinical scenarios.

[0021] (3) This invention uses a triple mechanism to systematically verify the anatomical rationality of the segmentation mask through three-dimensional connectivity analysis (with the maximum connected region ratio not less than 0.85 as the passing standard), anatomical structure existence verification, and spatial consistency verification (with the overlap between adjacent structures not exceeding 0.30 as the passing standard). When any verification fails, a quality inspection failure signal is immediately generated and the invalid result is blocked from propagating downstream, fundamentally avoiding the problem of garbage entering and exiting in the existing pipeline system.

[0022] (4) The present invention decomposes the original image into low-frequency approximate components (main anatomical structures) and high-frequency detail components (edge ​​textures). Based on the median of the high-frequency components, the noise level is adaptively estimated and soft threshold shrinkage is performed. While effectively suppressing noise, key edge information is preserved to the maximum extent, which significantly improves the segmentation accuracy of the model in low signal-to-noise ratio scenarios such as low-dose CT and fast MRI.

[0023] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0024] Figure 1 This is a framework diagram of the deep learning-based multimodal medical image quality inspection system of the present invention; Figure 2 This is a flowchart of the deep learning-based multimodal medical image quality inspection method of the present invention. Detailed Implementation

[0025] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0027] Example Please see Figures 1-2 This invention provides a deep learning-based multimodal medical image quality inspection system, comprising: The multimodal data acquisition and preprocessing module is connected to medical imaging equipment at its input end. It is used to acquire raw medical image data of at least two modalities of the same patient. The multimodal medical image data includes, but is not limited to, CT, MRI, PET, ultrasound and X-ray image data. The module performs preprocessing operations on the raw medical image data, such as bias field correction, unified spatial resampling, discrete wavelet transform frequency domain decomposition and high-frequency noise suppression, and outputs standardized multimodal image data and frequency domain enhanced image data. The deep feature extraction and fusion module, whose input is connected to the output of the multimodal data acquisition and preprocessing module, includes a CNN local feature extraction submodule, a Transformer global feature extraction submodule, and a cross-modal attention fusion submodule. It is used to extract the local texture edge features and global contextual features of each modality image, and to perform adaptive weighted fusion of multimodal features through a cross-modal attention mechanism to output a multimodal deep fusion feature map. The quality assessment and artifact detection module, whose input is connected to the output of the deep feature extraction and fusion module, includes a referenceless quality score prediction submodule, an artifact type identification submodule, and a confidence estimation submodule. It is used to generate quality scores for each modality of image based on the deep fusion feature map, identify artifact types and estimate confidence, and output the quality assessment vector of each modality of image and the comprehensive quality pass rate judgment result. The segmentation and quantification analysis module has its input terminals connected to the output terminals of the multimodal data acquisition and preprocessing module and the quality assessment and artifact detection module, respectively. It includes a multimodal segmentation submodule and a quantification measurement submodule, which are used to perform multimodal collaborative segmentation of anatomical structures or lesion areas on image data that has passed the quality qualification judgment, and calculate quantitative measurement values ​​such as volume, thickness, and cell count based on the segmentation results, and output structured quantification measurement results. The verification and quality control module has its inputs connected to the outputs of the quality assessment and artifact detection module and the segmentation and quantitative analysis module, respectively. It includes a segmentation quality verification submodule and a measurement rationality verification submodule. These submodules perform connectivity analysis and anatomical structure existence verification on the spatial integrity and structural consistency of the segmentation output, and compare the quantitative measurement results with preset physiological reference ranges. Based on the verification results, they generate a quality inspection pass signal or a quality inspection fail signal and a corresponding anomaly report. The output of the verification and quality control module is connected to the control terminal of the segmentation and quantitative analysis module. When a quality inspection fail signal is generated, the verification and quality control module sends a termination command to the segmentation and quantitative analysis module to prevent invalid segmentation results from propagating downstream. The knowledge retrieval and report generation module has its input terminals connected to the output terminals of the quality assessment and artifact detection module, the segmentation and quantitative analysis module, and the verification and quality control module, respectively. It includes a knowledge retrieval submodule and a structured report generation submodule. It is used to retrieve the corresponding physiological reference range and clinical guideline information from the external knowledge base based on the quality assessment vector, quantitative measurement results, and preset retrieval query statements, and integrate the quantitative measurement results, quality assessment results, and retrieved knowledge information into a structured clinical quality inspection report. The results output and visualization module connects to the output of the knowledge retrieval and report generation module, and is used to visualize and export structured clinical quality inspection reports in a combination of text and graphics.

[0028] The quality inspection method for a deep learning-based multimodal medical image quality inspection system includes the following steps: S1. Multimodal Medical Image Data Acquisition and Preprocessing: Acquire raw medical image data of at least two modalities from the same patient. Perform bias field correction, unified spatial resampling, discrete wavelet transform frequency domain decomposition, and high-frequency noise suppression on the raw medical image data to obtain standardized multimodal image data and frequency-enhanced image data. This includes the following steps: S11. Multimodal raw image data acquisition: Acquiring raw medical image data of at least two modalities of the same patient using medical imaging equipment. ,in Indicates modal index, Multimodal medical imaging data includes, but is not limited to, CT, MRI T1, MRI-T2, MRI-FLAIR, PET, ultrasound, and X-ray imaging data.

[0029] S12, Offset Field Correction: Correction of raw medical image data The N4ITK offset field correction algorithm is used for preprocessing to suppress low-frequency intensity artifacts in the image, resulting in an offset-corrected image. Its optimization objective is to minimize the following energy function: ; In the formula, For bias field estimation, For true image intensity, The regularization coefficient is . This is the total variational regularization term of the bias field, used to constrain the spatial smoothness of the bias field.

[0030] S13, Uniform Spatial Resampling: Resample the offset-corrected image Resampling to the preset spatial resolution is performed using trilinear interpolation. To obtain spatially standardized images This ensures the consistency of different modal images in spatial dimensions.

[0031] S14. Discrete Wavelet Transform Frequency Domain Decomposition and High-Frequency Noise Suppression: For Spatially Normalized Images Each axial slice is subjected to a two-dimensional discrete wavelet transform, and a single-layer decomposition is performed using the Daubechies-4 wavelet basis. This decomposes the image into low-frequency approximate components and high-frequency detail components in three directions. The specific decomposition formula is as follows: ; ; ; ; In the formula, For the first The low-frequency approximation component of the modality contains the main structural information of the image; , , These are high-frequency detail components in the horizontal, vertical, and diagonal directions, respectively, containing edge and texture detail information of the image; , All are spatial indexes.

[0032] S15. High-frequency noise suppression and enhancement reconstruction: High-frequency detail components , , Soft threshold noise suppression is performed to suppress high-frequency noise while preserving key edge information. The threshold is set as follows: ; ; ; ; In the formula, It is a symbolic function; For the first Modal adaptive noise thresholding based on diagonal high-frequency components The median is used to estimate the noise level; the denoised high-frequency components are then used to... , , Approximate components with low frequency Weighted fusion reconstruction is performed to obtain frequency domain enhanced image. : ; In the formula, This is a bilinear interpolation upsampling operation that restores low-frequency components to their original spatial resolution. ; Retain weights for low-frequency structures. Weights are preserved for high-frequency details, determined through cross-validation, to achieve a balance between maintaining anatomical integrity and suppressing high-frequency noise.

[0033] S16. Image Normalization: For frequency domain enhanced images Channel-level standardization is performed to obtain standardized multimodal image data. : ; In the formula, For the first Channel mean of modal image, For the first Channel standard deviation of modal images.

[0034] S2. Multimodal Image Deep Feature Extraction and Cross-Modal Adaptive Fusion: Based on standardized multimodal image data and frequency-domain enhanced image data, the local texture edge features of each modality are extracted using a CNN local feature extraction submodule, and the global contextual features of each modality are extracted using a Transformer global feature extraction submodule. Finally, the multimodal features are adaptively weighted and fused using a cross-modal attention fusion submodule to obtain a multimodal deep fusion feature map. The steps include: S21, CNN Local Feature Extraction: This involves extracting standardized multimodal image data. The input is the CNN local feature extraction submodule, which uses a pre-trained EfficientNet-B0 as the backbone network. It extracts local texture and edge features of each modality's image through its MBConv module, thus obtaining the local feature maps for each modality. ,in For the first The number of feature channels of a modality , This represents the spatial dimension of the feature map. The computation process of the MBConv module in this step is as follows: ; In the formula, PConv is a pointwise convolution used for channel dimension scaling; DWConv is a depthwise separable convolution used for spatial feature extraction; and SE is a compressed-excited channel attention module used for adaptive recalibration of the feature responses of each channel.

[0035] S22, Transformer Global Feature Extraction: This involves extracting standardized multimodal image data. The input is the Transformer global feature extraction submodule, which uses the Swing Transformer as the backbone network. It extracts the global contextual features of each modality's image through its shifted window self-attention mechanism, thus obtaining the global feature map for each modality. In this step, the self-attention calculation of the Swin Transformer is shown in the following formula: ; In the formula, , , These are the query matrix, key matrix, and value matrix, respectively, obtained by linearly mapping the input image after dividing it into blocks; The dimension of the key vector; This is a relative position offset matrix used to encode spatial position information.

[0036] S23. Cross-modal attention fusion: integrating local feature maps from various modalities. With global feature map By concatenating the data along the channel dimension, we obtain the complete feature representations of each modality. Then, the multimodal features are adaptively weighted and fused through the cross-modal attention fusion submodule. The specific steps are as follows: S231. Represent the complete features of each modality. Through sharing The convolutional layers are projected onto a unified embedding space to obtain the embedding features. ,in To unify the embedding dimensions.

[0037] S232, Embedded features of the first modality As a query, the embedding features of the other modalities As keys and values, calculate the cross-modal attention weights: ; ; In the formula, For the first mode to the first The attention weight matrix of the modality. For the first Modal features are attention-enhanced features weighted by the first modality query.

[0038] S233. Perform a weighted summation of the attention enhancement features across all modalities to obtain a multimodal deep fusion feature map. : .

[0039] In the formula, , For the first The adaptive fusion weights of the modalities are calculated through a learnable gating network: ; In the formula, This is a global average pooling operation. It is a multilayer perceptron used to map global features into scalar weights.

[0040] S3. No-Reference Multimodal Image Quality Assessment and Artifact Detection: Based on multimodal deep fusion feature maps, a no-reference quality regression network predicts the overall quality score of each modality of imagery, an artifact classification network identifies artifact types, and a confidence estimation network estimates and predicts confidence levels. Based on the overall quality score, artifact type, and confidence level, a quality assessment vector for each modality of imagery and an overall quality pass / fail determination result are generated. This includes the following steps: S31. No-reference multimodal quality score prediction: Deeply fusing feature maps of multiple modalities. The input is a no-reference quality regression network, which consists of a global average pooling layer and three fully connected layers. The output is a comprehensive quality score for each modality of the image. : ; In the formula, The Sigmoid activation function compresses the output to... interval; , , This is the weight matrix of the fully connected layer; , , This is the corresponding bias term.

[0041] S32. Artifact Type Recognition: Deeply fusing feature maps from multiple modalities. Input the artifact classification network, which employs a multi-label classification architecture, and output the artifact type probability vector for each modality of image. ,in The total number of preset artifact categories, including but not limited to motion artifacts, noise artifacts, beam hardening artifacts, truncation artifacts, and metal artifacts, is shown in the following formula: ; In the formula, This is the classification layer weight matrix. This is a classification layer bias term.

[0042] S33. Confidence estimation: Deeply fusing feature maps from multiple modalities. The input is a confidence estimation network, which employs a Bayesian neural network architecture. Uncertainty quantification is performed using Monte Carlo Dropout (Bayesian approximate inference based on Monte Carlo dropout). The output is the predictive confidence of the quality scores for each modality. : ; In the formula, For the first The quality score obtained from the Monte Carlo Dropout forward propagation. The number of samples (usually taken as...) ), This is for variance calculation.

[0043] S34. Generation of Comprehensive Quality Assessment Vector: This involves generating quality scores for each modality. Artifact type probability vector and confidence level Combined into a quality assessment vector : .

[0044] S35. Comprehensive Quality Compliance Determination: Based on the quality assessment vector. Generate the comprehensive quality pass / fail judgment results for each modal image. : ; In the formula, The quality score threshold is set at 0.6 (preferably). The artifact probability threshold is set to 0.5 (preferably). The confidence threshold is (preferably 0.7); when Time indicates the first Modal images pass quality compliance assessment when A timeout indicates failure; considering the results of all modalities, if at least one modality passes the quality compliance assessment (i.e., ... If the overall quality is qualified, then the overall quality is deemed to be acceptable.

[0045] S4. Multimodal Image Segmentation and Quantitative Analysis Based on Quality Inspection Gating: Based on the comprehensive quality compliance assessment results, multimodal collaborative segmentation is performed on image data that has passed the quality compliance assessment to obtain segmentation masks for anatomical structures or lesion regions. Quantitative measurements such as volume, thickness, and cell count are calculated based on the segmentation masks, and structured quantitative measurement results are output. This includes the following steps: S41. Quality Inspection Gating Judgment: Based on the overall quality compliance judgment result output by S35, determine whether to perform segmentation and quantitative analysis: If the overall comprehensive quality compliance judgment is unsuccessful (i.e., all modalities fail the quality compliance judgment), If the image quality is not up to standard, the subsequent segmentation and quantification analysis steps are skipped, and the process proceeds directly to S6 to generate a quality inspection report stating "Image quality is unacceptable, re-acquisition recommended"; if at least one modality passes the quality acceptance test... If ), then S42 will be executed.

[0046] S42. Available Modality Filtering: From standardized multimodal image data The modes that pass the quality compliance judgment are selected from the samples. This constitutes a usable modal image set. ;like If a modality is missing, the missing modality compensation mechanism is activated, and S43 is executed.

[0047] S43, Missing Modal Feature Compensation (Conditional Execution): When At the same time, a cross-modal feature generation network is used to compensate for the missing features of the modality; the cross-modal feature generation network adopts a conditional generative adversarial network architecture to deeply fuse features from the available modalities. Given the condition, generate approximate features for the missing modes. : ; In the formula, For generator networks, This is a random noise vector used to introduce generative diversity; it generates missing modal features. It is concatenated with available modal features to form a complete multimodal feature set.

[0048] S44. Multimodal collaborative segmentation: This involves using available modal image data... (And the missing modality compensation features generated by S43, if triggered) Input to the multimodal segmentation submodule. The multimodal segmentation submodule adopts the 3DU-Net architecture and performs multimodal collaborative segmentation through an encoder-decoder structure. The encoder fuses information from various modalities through multi-channel input and extracts multi-scale semantic features by downsampling layer by layer. The decoder restores spatial resolution through skip connections and finally outputs the probability segmentation mask for each target structure. ,in Index the target structure. The target structure is the total number of elements; the segmentation output of 3DU-Net generates multi-class probability maps through a pixel-wise Softmax activation function. : ; In the formula, The position of a voxel in three-dimensional space. For image space domain, The output of the segmentation network is the first Class in position The logit value at that location.

[0049] S45. Binarization segmentation mask generation: Generation of probabilistic segmentation masks. Using threshold Binarization is performed to obtain a binary segmentation mask. : .

[0050] S46. Quantization Measurement Calculation: Based on Binary Segmentation Mask The quantitative measurement values ​​of each target structure are calculated through the quantitative measurement submodule.

[0051] S461. Volume Measurement (Applicable to three-dimensional imaging modalities such as CT and MRI): For the first... Volume of the target structure Perform the calculation: ; In the formula, For belonging to the first The total number of voxels in the target structure , , The images are respectively in , , Physical spacing of voxels in the direction (unit: mm).

[0052] S462. Thickness Measurement (Applicable to OCT and other tomographic imaging modalities): For the first... Thickness of each target structure The calculation is performed using a column-by-column boundary detection method: ; ; In the formula, and The first The target structure is in the horizontal position The pixel coordinates of the upper and lower boundaries at the location, The set of valid boundary detection locations. Axial resolution scaling factor (unit: m / pixel).

[0053] S463, Cell Counting (applicable to two-dimensional modalities such as pathological sections): Counting the number of detected cell structures. Perform the calculation: ; In the formula, For a connected component counting function, apply it to a binary segmentation mask. Statistical analysis was performed on the connected regions; all quantitative measurement results were combined into structural quantitative measurement results. .

[0054] S5. Segmentation Quality Verification and Measurement Rationality Check: Perform connectivity analysis, anatomical structure existence verification, and spatial consistency check on the segmentation mask to generate segmentation quality verification results; compare and verify the quantitative measurement results with the preset physiological reference range to generate measurement rationality check results; generate quality inspection pass or fail signals and corresponding anomaly reports based on the segmentation quality verification results and measurement rationality check results. This includes the following steps: S51. Connectivity Analysis of Split Masks: Analysis of Binary Split Masks Perform three-dimensional connectivity analysis and calculate the maximum connectivity volume percentage of each target structure. : ; In the formula, For the first The set of all connected regions in a binary mask of a target structure For the first A set of voxels of connected regions The number of voxels in a set.

[0055] like ,in, The threshold for connectivity percentage is taken as follows: If the segmentation result of the target structure is found to have a fragmentation problem, a connectivity anomaly marker is generated. .

[0056] S52. Verification of the existence of anatomical structures: Verification of binary segmentation mask Check whether it contains the expected anatomical structures and calculate the total number of voxels for each target structure. ;like ( If a preset minimum structure voxel threshold is set (based on the physiological size of the target structure), then the target structure is determined to be missing or segmentation failed, and a structure missing marker is generated. .

[0057] S53. Spatial Consistency Verification: Verifying the spatial consistency of anatomically adjacent or spatially constrained target structures. Perform spatial consistency verification and calculate its spatial overlap. : ; like ,in, The spatial overlap threshold is taken as follows: This indicates that two anatomically non-overlapping structures have abnormally overlapped, generating a spatial consistency aberration marker. .

[0058] S54. Comprehensive Verification of Segmentation Quality: Based on the verification results above, generate the segmentation quality verification results. : ; If any anomaly is marked as 1, the segmentation quality verification fails.

[0059] S55. Measurement rationality verification: The quantized measurement results output from S46 are verified. With respect to the preset physiological reference range Perform comparison and verification: ; In the formula, For the first Measurement values ​​of the target structure, and These are the preset lower and upper limits for the measured value (based on physiological reference range settings); if If the measurement result is found to be outside the reasonable range, an abnormal measurement reasonableness marker will be generated.

[0060] S56. Quality control signal generation: Based on the segmentation quality verification results. and measurement rationality verification results Generate quality control signals : ; when A quality inspection pass signal is generated at the time. It generates a quality inspection failure signal and a corresponding abnormal report in real time.

[0061] S57. Invalid results are blocked from propagation: When a quality inspection failure signal is generated ( When the current segmentation result and quantization measurement result are transmitted to S4, a termination command is sent to prevent them from being transmitted to the downstream S6. At the same time, the anomaly type and location information are recorded to form a quality inspection anomaly log.

[0062] S6. Knowledge Retrieval and Structured Clinical Quality Inspection Report Generation: Based on the quality assessment vector, quantitative measurement results, and preset search queries, the system retrieves corresponding physiological reference ranges and clinical guideline information from an external knowledge base. The quantitative measurement results, quality assessment results, and retrieved knowledge information are then integrated into a structured clinical quality inspection report. This includes the following steps: S61. Knowledge Retrieval Query Construction: Based on the quality assessment vector output by S3 Quantization measurement results output by S46 Construct knowledge retrieval query statements : ; In the formula, Describe the generation function for the modal type. Generate a function for the target structure name. The function formats the measured value and unit; these three are concatenated to form a natural language query statement.

[0063] S62. External Knowledge Retrieval: This involves retrieving the query statement... The search results are submitted to external knowledge bases via the Tatively search API. These external knowledge bases include medical literature databases, clinical guideline databases, and physiological reference range databases, and the results are returned as a set. ,in For the first Search results.

[0064] S63. Relevance Ranking and Filtering of Search Results: This involves sorting and filtering the search result set... Each result in the dataset has a relevance score calculated. : ; In the formula, For text embedding encoding functions, Calculate cosine similarity; sort by relevance score in descending order, and select the top... The highest relevance result is used as reference knowledge. .

[0065] S64. Generation of Structured Clinical Quality Inspection Reports: This involves generating the quality assessment vector output from S3. Quantization measurement results output by S46 The measurement rationality verification results output by S55 S56 output quality control signal and reference knowledge output by S63 Input report generation submodule The report generation submodule uses a large language model to generate structured clinical quality inspection reports based on preset quality inspection report templates. : ; Structured Clinical Quality Inspection Report This includes: patient information and image acquisition parameters, image quality scores and artifact identification results for each modality, segmentation and quantification measurement results and comparison with reference ranges, quality inspection conclusions (pass / fail) and clinical recommendations.

[0066] S7. Visualized Output of Quality Inspection Results: This involves visually displaying and exporting structured clinical quality inspection reports in a combination of text and graphics. The steps include: S71, Quality Inspection Report Formatting and Rendering: Formats the structured clinical quality inspection report generated in S64. The system is formatted and rendered according to a preset visualization template to generate a visual quality inspection report page that combines text and images.

[0067] S72. Visualization of segmentation results overlay: Visualize the probability segmentation mask generated in S44. Or binary segmentation mask A semi-transparent pseudo-color overlay is superimposed on the original image to generate a visualization of the segmentation results.

[0068] S73. Visualization of Quality Indicators: Visualizing the quality scores generated in S31. The artifact type probability vector generated by S32 and the confidence level generated by S33 Visualize the data using bar charts, radar charts, or heat maps.

[0069] S74. Quality Inspection Report Export and Storage: Export the visual quality inspection report combining text and images into standardized format files (including but not limited to PDF, HTML and JSON formats) and store it in the hospital information system or image archiving and communication system.

[0070] This embodiment uses the quality inspection of multimodal MRI image data of a suspected brain tumor patient in the radiology department of a tertiary hospital as an example to illustrate the specific implementation of the present invention.

[0071] Patient Zhang, male, 56 years old, presented with headache and nausea. The clinician ordered a plain and contrast-enhanced MRI scan of the head. The radiology technician used an MRI scanner to acquire image data in four modalities: T1-weighted, T2-weighted, T2-FLAIR, and T1-contrast-enhanced. Each modality of image was 240×240×155 voxels, with a voxel spacing of 1.0×1.0×1.2 mm. 3 .

[0072] The specific implementation steps are as follows: Step 1: Multimodal image data acquisition and preprocessing.

[0073] Radiology operators will process the raw MR1 data of the above four modalities ( The system imports multimodal data acquisition and preprocessing modules from the quality inspection system. First, it performs N4ITK bias field correction on each modal image to eliminate low-frequency intensity artifacts caused by magnetic field inhomogeneity. Then, all images are uniformly resampled using trilinear interpolation. The standard spatial resolution is then achieved. The system then performs a Daubechies-4 discrete wavelet transform on each axial slice to obtain the low-frequency approximate components. and high-frequency detail components in three directions The system calculates the adaptive noise threshold. (Taking the T2-FLAIR mode as an example) After soft-thresholding shrinkage of the high-frequency components, they are weighted and fused with the low-frequency components at an 8:2 ratio to reconstruct the frequency-domain enhanced image. Finally, channel-level normalization was performed on the enhanced images to obtain standardized multimodal image data. .

[0074] Step 2: Multimodal image depth feature extraction and cross-modal adaptive fusion.

[0075] The system will standardize the images The inputs are respectively the CNN local feature extraction submodule and the Transformer global feature extraction submodule. In the CNN branch, the MBConv module of the EfficientNet-BO network extracts local feature maps of each modality through pointwise convolution, depthwise separable convolution, and SE channel attention. In the Transformer branch, SwinTransformer extracts the global feature map through a shift window self-attention mechanism. The system concatenates the two feature streams along the channel dimension to obtain a complete feature representation. The system then uses the T1 enhanced mode ( For the query, the other three modalities are key-value pairs. Calculate the cross-modal attention weights. Softmax Simultaneously, the system calculates the adaptive fusion weights of each modality through a learnable gating network, obtaining... =0.32, =0.18, =0.21, =0.9 (T1 enhancement receives the highest weight due to its clear visualization of enhanced tumors). The final weighted summation yields the multimodal deep fusion feature map. .

[0076] Step 3: No-reference multimodal image quality assessment and artifact detection.

[0077] The system will The input is a no-reference quality regression network, which outputs a comprehensive quality score for each modality through global average pooling and three fully connected layers. =0.82, =0.73, =0.58, =0.91 (T2-FLAIR received the lowest score due to image blurring caused by slight patient movement). Simultaneously, the artifact classification network output artifact probability vectors for each modality. The motion artifact probability for T2-FLAIR was 0.62 (exceeding the artifact probability threshold of 0.5), while no significant artifacts were detected in the other modalities. The system further performed 30 random forward propagations using the Monte Carlo dropout method to calculate the confidence score for each modality. =0.85, =0.79, =0.62, =0.91. The system makes a comprehensive judgment based on the preset threshold: =0.6, =0.5, =0.7. T1, T2, and T1-enhanced three modes all passed the qualification assessment. The T2-FLAIR test failed because its quality score of 0.58 was below the quality score threshold of 0.6 and its confidence level of 0.62 was below the confidence threshold of 0.7. The system determines that the overall process has passed (a usable mode exists) and continues with subsequent steps.

[0078] Step 4: Multimodal image segmentation and quantitative analysis based on quality inspection gating.

[0079] The system filters available modes based on the quality compliance assessment results. To form a usable modal image set Due to the existence of a mode absence ( The system activates a missing modality compensation mechanism, using the deep fusion features of available modalities as conditions, and generates approximate features of T2-FLAIR through a conditional generative adversarial network. The system then concatenates the available modal features with the generated approximate features to form a complete multimodal feature set, which is then input into the 3D U-Net segmentation network. The segmentation network outputs a probabilistic segmentation mask for brain tumor-related structures, which is then binarized with a 0.5 threshold to obtain binary segmentation masks for each structure. The system further calculates quantitative measurements based on segmentation masks to enhance tumor core volume. Volume of edema area .

[0080] Step 5: Verify the segmentation quality and check the rationality of the measurement.

[0081] The system uses a segmented mask. Perform connectivity analysis and calculate the volume percentage of the largest connected region. (Connectivity percentage threshold) Connectivity is normal. For calculate (Connectivity percentage threshold) to generate connectivity anomaly markers in edema areas. The system checks the total number of each structural element. Much larger than the minimum threshold The structural existence verification passed. The system verified the spatial overlap between the enhanced tumor and edema areas; Overlap(ET, ED) = 0.18 < 0.30 (spatial overlap threshold), indicating no abnormal adhesions. The system will then display the measured values. Compared with the preset physiological reference range The comparison is within the normal range; Reference range The comparison results are also within the normal range.

[0082] Based on the above verification results, due to abnormal connectivity in the edema area, the system generates a quality inspection failure signal. It also sends a termination command to S4 to prevent the current segmentation results and quantization measurements from being propagated to the downstream report generation module. At the same time, it records the following error log: "Poor T2-FLAIR modal quality has caused fragmentation of the edema region segmentation (connectivity 78%). It is recommended to manually review or reacquire the T2-FLAIR sequence."

[0083] Step Six: Knowledge Retrieval and Generation of Structured Clinical Quality Inspection Reports.

[0084] The system constructs a knowledge retrieval query based on the comprehensive quality compliance assessment results and quantitative measurement results: "Brain MRI image quality inspection report: T1, T2, and T1-enhanced modal quality are qualified, T2-FLAIR quality is unqualified (motion artifact, confidence level 0.62). The segmentation results show abnormal connectivity in edema areas (78%). It is recommended to review the delineation of edema areas." The system searches for relevant guidelines and literature on brain tumor MRI segmentation quality control through the Tatively search API, and selects the three most relevant results as reference knowledge. The system integrates the quality assessment results, anomaly detection results, and retrieved knowledge information to generate a structured clinical quality inspection report.

[0085] Step 7: Visualize the quality inspection results.

[0086] The system formats and renders the structured quality inspection report according to a preset template. In a semi-transparent pseudo-color segmentation overlay, enhanced tumor areas with normal connectivity are marked in green, while edema and fragmented areas with abnormal connectivity are highlighted in yellow. The quality inspection conclusion is prominently displayed in the report: "Quality Inspection Failed – Manual review recommended before clinical decision-making." The report also displays the quality scores for each modality using a heatmap and compares the measurements of each structure with reference ranges using a bar chart. The final report is exported in PDF and JSON formats and uploaded to the hospital's image archiving and communication system.

[0087] This embodiment fully demonstrates the entire process of the quality inspection system from image input to quality inspection report output. Through the quality assessment mechanism in S3, the system effectively identifies T2-FLAIR modalities where quality is degraded due to slight motion. Through the quality inspection gating mechanism in S5, the system proactively terminates the process and generates a clear anomaly report when connectivity anomalies are found in the segmentation results, effectively preventing fragmented segmentation results from misleading downstream quantitative analysis and clinical decision-making. In the case of missing T2-FLAIR modalities, the system maintains the normal operation of the segmentation network through the missing modality compensation mechanism (S4), demonstrating robustness to complex clinical scenarios. The entire quality inspection process takes approximately 47 seconds and can run smoothly on a standard image post-processing workstation.

[0088] Therefore, the present invention adopts the above-mentioned deep learning-based multimodal medical image quality inspection system and method, which directly predicts the comprehensive quality score of each modality from the multimodal deep fusion feature map through a referenceless quality regression network, and simultaneously identifies specific types such as motion artifacts, noise artifacts, and beam hardening artifacts, thus eliminating the dependence on reference images.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A deep learning-based multimodal medical image quality inspection system, characterized in that, include: The multimodal data acquisition and preprocessing module is used to acquire raw medical image data, perform preprocessing, and output standardized multimodal image data and frequency domain enhanced image data. The deep feature extraction and fusion module includes a CNN local feature extraction submodule, a Transformer global feature extraction submodule, and a cross-modal attention fusion submodule. It is used to extract local texture edge features and global contextual features of each modality image, and output a multimodal deep fusion feature map by adaptively weighting and fusing the multimodal features. The quality assessment and artifact detection module includes a no-reference quality score prediction submodule, an artifact type identification submodule, and a confidence estimation submodule. It is used to generate quality scores for each modality of image based on deep fusion feature maps, identify artifact types and estimate confidence, and output the quality assessment vector of each modality of image and the overall quality passability judgment result. The segmentation and quantification analysis module includes a multimodal segmentation submodule and a quantification measurement submodule, which are used to perform multimodal collaborative segmentation, calculate quantitative measurement values ​​based on the segmentation results, and output structured quantification measurement results; The verification and quality control module includes a segmentation quality verification submodule and a measurement rationality verification submodule. It is used to perform connectivity analysis and anatomical structure existence verification on the spatial integrity and structural consistency of the segmentation output, as well as to compare and verify the quantitative measurement results with the preset physiological reference range, and generate a quality inspection pass signal or a quality inspection fail signal and a corresponding abnormal report based on the verification results. The knowledge retrieval and report generation module includes a knowledge retrieval submodule and a structured report generation submodule. It is used to retrieve the corresponding physiological reference range and clinical guideline information from the external knowledge base based on the quality assessment vector, quantitative measurement results and preset retrieval query statements, and generate a structured clinical quality inspection report. The results output and visualization module is used to visualize and export structured clinical quality inspection reports in a combination of text and graphics.

2. The method of applying the deep learning-based multimodal medical image quality inspection system as described in claim 1, characterized in that, Includes the following steps: S1. Acquire raw medical image data, preprocess the raw medical image data to obtain standardized multimodal image data and frequency domain enhanced image data; S2. Based on standardized multimodal image data and frequency domain enhanced image data, extract local texture edge features and global contextual features of each modality image, and perform adaptive weighted fusion of multimodal features to obtain a multimodal deep fusion feature map; S3. Based on multimodal deep fusion feature maps, predict the comprehensive quality score of each modal image, identify artifact types and estimate prediction confidence, and generate quality assessment vectors and comprehensive quality qualification results for each modal image according to the comprehensive quality score, artifact type and confidence. S4. Based on the comprehensive quality compliance assessment results, perform multimodal collaborative segmentation on the image data that has passed the quality compliance assessment to obtain the segmentation mask of the anatomical structure or lesion area, and then calculate the quantitative measurement value and output the structured quantitative measurement result. S5. Perform connectivity analysis, anatomical structure existence verification, and spatial consistency check on the segmentation mask to generate segmentation quality verification results; The quantitative measurement results are compared and verified with the preset physiological reference range to generate a measurement rationality verification result; based on the segmentation quality verification result and the measurement rationality verification result, a quality inspection pass signal or a quality inspection fail signal and a corresponding abnormality report are generated. S6. Based on the quality assessment vector, quantitative measurement results and preset search query statements, retrieve knowledge information from the external knowledge base, and integrate the quantitative measurement results, quality assessment results and retrieved knowledge information into a structured clinical quality inspection report. S7. Visualize and export structured clinical quality inspection reports in a combination of text and graphics.

3. The method of the deep learning-based multimodal medical image quality inspection system according to claim 2, characterized in that, S1 includes: S11. Multimodal raw image data acquisition: Acquiring raw medical image data of at least two modalities of the same patient using medical imaging equipment. ,in Indicates modal index, ; S12. Perform offset field correction processing: This involves processing the raw medical image data... The N4ITK offset field correction algorithm is used for preprocessing to suppress low-frequency intensity artifacts in the image, resulting in an offset-corrected image. ; S13. Perform uniform spatial resampling preprocessing: preprocess the offset-corrected image... Resampling to the preset spatial resolution is performed using trilinear interpolation. To obtain spatially standardized images ; S14. Perform discrete wavelet transform frequency domain decomposition and high-frequency noise suppression preprocessing on spatially normalized images: Each axial slice is subjected to a two-dimensional discrete wavelet transform, and a single-layer decomposition is performed using the Daubechies-4 wavelet basis, decomposing the image into low-frequency approximate components and high-frequency detail components in three directions. , , ; S15. Perform high-frequency noise suppression and enhancement reconstruction processing: For high-frequency detail components... , , Soft threshold noise suppression is performed to suppress high-frequency noise while preserving key edge information. The threshold is set as follows: ； ； ； ； In the formula, It is a symbolic function; For the first Modal adaptive noise thresholding based on diagonal high-frequency components The median estimate of the noise level; The denoised high-frequency components , , Approximate components with low frequency Weighted fusion reconstruction is performed to obtain frequency domain enhanced image. : ； In the formula, This is a bilinear interpolation upsampling operation that restores low-frequency components to their original spatial resolution. ; Retain weights for low-frequency structures. Weights are preserved for high-frequency details, determined through cross-validation, to achieve a balance between maintaining the integrity of anatomical structures and suppressing high-frequency noise. S16. Image Normalization: For frequency domain enhanced images Channel-level standardization is performed to obtain standardized multimodal image data. : ； In the formula, For the first Channel mean of modal image, For the first Channel standard deviation of modal images.

4. The method of the deep learning-based multimodal medical image quality inspection system according to claim 3, characterized in that, S2 includes: S21, CNN Local Feature Extraction: This involves extracting standardized multimodal image data. The input is the CNN local feature extraction submodule, which uses a pre-trained EfficientNet-B0 as the backbone network. The MBConv module within this submodule extracts local texture and edge features from each modality of the image, resulting in local feature maps for each modality. ,in For the first The number of feature channels of a modality , The spatial dimension of the feature map; S22, Transformer Global Feature Extraction: This involves extracting standardized multimodal image data. The input is the Transformer global feature extraction submodule, which uses the Swing Transformer as the backbone network. It extracts the global contextual features of each modality's image through its shifted window self-attention mechanism, thus obtaining the global feature map for each modality. ; S23. Cross-modal attention fusion: integrating local feature maps from various modalities. With global feature map By concatenating the data along the channel dimension, we obtain the complete feature representations of each modality. Then, the multimodal features are adaptively weighted and fused through the cross-modal attention fusion submodule.

5. The method of the deep learning-based multimodal medical image quality inspection system according to claim 4, characterized in that: S23 includes: S231. Represent the complete features of each modality. Through sharing The convolutional layers are projected onto a unified embedding space to obtain the embedding features. ,in To unify the embedding dimensions; S232, Embedded features of the first modality As a query, the embedding features of the other modalities As keys and values, calculate the cross-modal attention weights: ； ； In the formula, For the first mode to the first The attention weight matrix of the modality. For the first Attention-enhanced features resulting from first-modal query weighting of modal features; S233. Perform a weighted summation of the attention enhancement features across all modalities to obtain a multimodal deep fusion feature map. : ； in, , For the first The adaptive fusion weights of the modalities are calculated through a learnable gating network: ； In the formula, This is a global average pooling operation. It is a multilayer perceptron used to map global features to scalar weights.

6. The method of the deep learning-based multimodal medical image quality inspection system according to claim 5, characterized in that, S3 includes: S31. Deeply fuse feature maps from multiple modalities. The input is a no-reference quality regression network, which consists of a global average pooling layer and three fully connected layers. The output is a comprehensive quality score for each modality of the image. ; S32. Deeply fuse feature maps from multiple modalities. Input the artifact classification network, which employs a multi-label classification architecture, and output the artifact type probability vector for each modality of image. ,in The total number of preset artifact categories; S33. Deeply fuse feature maps of multiple modalities. The input is a confidence estimation network, which employs a Bayesian neural network architecture. Uncertainty is quantified using Monte Carlo Dropout, and the output is the predictive confidence of the quality scores for each modality. : ； In the formula, For the first The quality score obtained from the Monte Carlo Dropout forward propagation. The number of samples (usually taken as...) ), For variance calculation; S34. Quality scores for each modality. Artifact type probability vector and confidence level Combined into a quality assessment vector ; S35. Based on the quality assessment vector Generate the comprehensive quality pass / fail judgment results for each modal image. ; when Time indicates the first Modal images pass quality compliance assessment when This indicates that the test was not passed. If at least one mode passes the quality compliance assessment, the overall quality compliance assessment is passed.

7. The method of the deep learning-based multimodal medical image quality inspection system according to claim 6, characterized in that, S4 includes: S41. Based on the overall quality compliance assessment result output by S35, determine whether to perform segmentation and quantitative analysis: If the overall quality assessment is deemed unqualified, skip the subsequent segmentation and quantitative analysis steps and proceed directly to S6 to generate a quality inspection report indicating that the image quality is unqualified and re-acquisition is recommended; if at least one modality passes the quality assessment, proceed to S42. S42, from standardized multimodal image data The modalities that pass the quality compliance judgment are selected from the samples. This constitutes a usable modal image set. ;like If a modality is missing, the missing modality compensation mechanism is activated, and S43 is executed. S43, when At the same time, features of missing modalities are compensated by a cross-modal feature generation network. The cross-modal feature generation network employs a conditional generative adversarial network architecture, using deep fusion features from available modalities. Given the condition, generate approximate features for the missing modes. : ； In the formula, For generator networks, This is a random noise vector used to introduce generative diversity; it generates missing modal features. It is concatenated with available modal features to form a complete multimodal feature set; S44. Available modal image data The input is a multimodal segmentation submodule, which adopts a 3DU-Net architecture and performs multimodal collaborative segmentation through an encoder-decoder structure. The encoder fuses information from various modalities through multi-channel input and extracts multi-scale semantic features by downsampling layer by layer. The decoder restores spatial resolution through skip connections and finally outputs a probabilistic segmentation mask for each target structure. ,in Index the target structure. The total number of target structures; The segmentation output of the multimodal segmentation submodule generates a multi-class probability map through a pixel-wise Softmax activation function. ; S45. Probability segmentation mask Using threshold Binarization is performed to obtain a binary segmentation mask. ; S46, Based on binary segmentation mask The quantitative measurement submodule calculates the quantitative measurement values ​​of each target structure and outputs the quantitative measurement results of the structure. .

8. The method of the deep learning-based multimodal medical image quality inspection system according to claim 7, characterized in that, S5 includes: S51. Connectivity Analysis of Split Masks: Analysis of Binary Split Masks Perform three-dimensional connectivity analysis and calculate the maximum connectivity volume percentage of each target structure. ; like ,in, The threshold for connectivity percentage is taken as follows: If the segmentation result of the target structure is found to have a fragmentation problem, a connectivity anomaly marker is generated. ; S52, Check the binary segmentation mask Does it include the expected anatomical structure? Calculate the total number of voxels for each target structure. ; like ,in, If a preset minimum structure voxel threshold is set based on the physiological size of the target structure, then the target structure is determined to be missing or segmentation failed, and a structure missing marker is generated. ; S53, For target structures that are anatomically adjacent or have spatial constraints Perform spatial consistency verification and calculate its spatial overlap. ; like ,in, The spatial overlap threshold is taken as follows: This indicates that two anatomically non-overlapping structures have abnormally overlapped, generating a spatial consistency aberration marker. ; S54. Based on the above verification results, generate segmentation quality verification results. If any anomaly is marked as 1, the segmentation quality verification fails. S55. Quantize the measurement results output by S46. With respect to the preset physiological reference range Perform comparison and verification: ； In the formula, For the first Measurement values ​​of the target structure, and These are the preset lower and upper limits for the measured value, respectively; if If the measurement result is found to be outside the reasonable range, an abnormal measurement reasonableness marker will be generated. S56. Quality control signal generation: Based on the segmentation quality verification results. and measurement rationality verification results Generate quality control signals : ； when A quality inspection pass signal is generated at the specified time. It generates quality inspection failure signals and corresponding anomaly reports in real time; S57. When a quality inspection failure signal is generated, a termination command is sent to S4 to prevent the current segmentation result and quantization measurement result from being propagated to S6. At the same time, the anomaly type and location information are recorded to form a quality inspection anomaly log.

9. The method of the deep learning-based multimodal medical image quality inspection system according to claim 8, characterized in that, S6 includes: S61. Based on the quality assessment vector output by S3 Quantization measurement results output by S46 Construct knowledge retrieval query statements ; S62, query statement Submit to an external knowledge base for retrieval and return a set of retrieval results. ,in, For the first 1 search result; S63. Regarding the search result set Each result in the dataset has a relevance score calculated. Sort by relevance score in descending order, and select the top 3 results with the highest relevance as reference knowledge. ; S64. The quality assessment vector output by S3 Quantization measurement results output by S46 The measurement rationality verification results output by S55 S56 output quality control signal and reference knowledge output by S63 The input report generation submodule generates structured clinical quality inspection reports. This includes: patient information and image acquisition parameters, image quality scores and artifact identification results for each modality, segmentation and quantification measurement results and comparison with reference ranges, quality inspection conclusions and clinical recommendations.

10. The method of the deep learning-based multimodal medical image quality inspection system according to claim 9, characterized in that, S7 includes: S71. The structured clinical quality inspection report generated in S64 The system is formatted and rendered according to a preset visualization template to generate a visual quality inspection report page that combines text and graphics. S72, Apply the probability segmentation mask generated in S44 Or binary segmentation mask A semi-transparent pseudo-color overlay is superimposed on the original image to generate a visualization of the segmentation results; S73. The quality score generated in S31 The artifact type probability vector generated by S32 and the confidence level generated by S33 Visualize the data using bar charts, radar charts, or heat maps. S74. Export the visual quality inspection report combining text and images into a standardized format file and store it in the hospital information system or image archiving and communication system.

Images