Multi-modal medical data conjoint analysis system based on artificial intelligence

Abstract

The invention discloses a multi-modal medical data conjoint analysis system based on artificial intelligence, and particularly relates to the field of data analysis. Comprising a data acquisition module, a data quality evaluation module, a data dynamic association module, a heterogeneity feature fusion module, a model real-time optimization module, a data feedback iteration module and a data abnormal feature detection and analysis module. The data acquisition module is used for acquiring multi-modal medical data and executing modal specificity preprocessing on the multi-modal medical data; the data exception feature detection and analysis module is used for constructing an exception association model based on the optimized model and generating an exception association analysis report; according to the method, the problems that in the prior art, the data quality sensitivity is high, and the stability of an analysis result is poor due to the fact that the method is easily influenced by image artifacts, signal noise and text errors are solved by combining modal specificity preprocessing and quality self-adaptive correction with the quality index and the targeted correction formula.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and more specifically, to a multimodal medical data joint analysis system based on artificial intelligence. Background Technology

[0002] Multimodal medical data comprises three core categories: imaging data, physiological signal data, and text data. Current technologies for processing this type of multimodal medical data often employ a single-modal independent analysis combined with simple feature concatenation. For example, convolutional neural networks (CNNs) are used to extract lesion features from the amygdala region of brain MRI, recurrent neural networks (RNNs) are used to process temporal features from electrocardiogram signals of epilepsy patients, and natural language processing (NLP) models are used to extract textual features such as epileptic seizure frequency and descriptions of abnormal brain structures from electronic medical records. These three types of features are then directly stacked and input into traditional machine learning models for analysis.

[0003] Existing technologies can extract basic features from single-modal medical data related to the brain. For example, CNNs can accurately capture the edge contours and grayscale distribution of lesions in the amygdala region of brain MRI, RNNs can effectively extract the temporal variation patterns of RR intervals in electrocardiogram signals of epilepsy patients, and NLP models can accurately identify keywords such as epilepsy history, descriptions of abnormal brain structures, and medication records in electronic medical records. However, they also have significant shortcomings: First, the intermodal correlation mining is insufficient, and the dynamic correlation between image lesion features and physiological signal changes is not quantified, making it difficult for the analysis results to reflect the intrinsic connections of multimodal data. Second, the ability to handle data heterogeneity is weak. Existing feature splicing methods do not eliminate the differences in dimensional units between modalities, which easily leads to feature redundancy and information conflicts. Third, the data quality is highly sensitive. Image data is easily affected by motion artifacts, physiological signals are easily mixed with electromagnetic noise, and text data may contain spelling errors or ambiguous expressions. Existing methods lack targeted quality correction mechanisms, resulting in poor stability of analysis results. Fourth, the dynamic adaptability is insufficient. After the analysis model is trained, it cannot be adjusted in real time according to newly generated multimodal data, causing the analysis results to lag behind the data update rhythm and failing to meet the needs of long-term continuous data processing.

[0004] To address the shortcomings of existing technologies, such as weak modal correlation, poor heterogeneity handling, quality sensitivity, and insufficient dynamism, there is an urgent need for an AI-based multimodal medical data joint analysis system. This system utilizes seven core components—modality-specific preprocessing, adaptive quality correction, dynamic correlation modeling, heterogeneous feature fusion, real-time model optimization, iterative feedback, and anomaly detection—combined with deep learning and statistical analysis techniques to construct a closed-loop analysis system for multimodal data, enabling in-depth joint analysis of multimodal medical data. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, the present invention provides a multimodal medical data joint analysis system based on artificial intelligence, which solves the problems mentioned in the background art through the following solutions.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multimodal medical data joint analysis system based on artificial intelligence, comprising: Data acquisition module: Acquires multimodal medical data and performs modality-specific preprocessing on the multimodal medical data; Data quality assessment module: Constructs a quality index model based on preprocessed data, quantifies data quality assessment, and generates corrected parameters; Data dynamic association module: Constructs a cross-modal association degree model based on the corrected parameters and generates an association feature matrix; Heterogeneity feature fusion module: Extracts single-modal features based on the correlation feature matrix and performs heterogeneity fusion, and obtains an initial model for multimodal joint analysis through AI model training; Real-time model optimization module: Continuously acquires updated multimodal medical data, associated feature matrices, and fused features, constructs dynamic optimization model optimization coefficients, and generates feature association reports; Data feedback iteration module: Constructs an iterative update model based on the key correlation indicators in the feature correlation report, and performs iterative data updates and storage according to the coefficients; Data anomaly feature detection and analysis module: Based on the optimized model, construct an anomaly correlation model and generate an anomaly correlation analysis report.

[0007] Preferably, the multimodal medical data includes image data, physiological signal data, and text data; the image data specifically includes the mean pixel value of the lesion region, the pixel variance of the lesion region, and the gradient value of the lesion edge; the physiological signal data specifically includes the mean RR interval, ST segment offset, and QRS complex duration; the text data specifically includes the frequency of medical history keywords, semantic similarity of symptom descriptions, and completeness of medication records; the modality-specific preprocessing uses adaptive median filtering to remove image motion artifacts to obtain the preprocessed mean pixel value of the lesion region, uses a 5-layer db4 wavelet basis transform to remove signal noise to obtain the preprocessed mean RR interval, and uses a BERT model for semantic encoding to obtain the preprocessed frequency of medical history keywords and the preprocessed completeness of medication records.

[0008] Preferably, the quality index model includes an image quality index model, a physiological signal quality index model, and a text quality index model; the image quality index model is used to construct an image quality index calculation model to obtain an image quality index; the physiological signal quality index model is used to construct a physiological signal quality index calculation model to obtain a physiological signal quality index; and the text quality index model is used to construct a text quality index calculation model to obtain a text quality index.

[0009] Preferably, the corrected parameters specifically include: if the image quality index is less than 0.6, a generative adversarial network is used to enhance the image to obtain the corrected mean pixel value of the lesion region and the gradient value of the lesion edge; if the physiological signal quality index is less than 0.5, an adaptive Kalman filter is used to complete the signal to obtain the corrected mean RR interval and QRS complex duration; if the text quality index is less than 0.55, a semantic error correction model is used to optimize the text to obtain the corrected frequency of medical history keywords and the completeness of medication records.

[0010] Preferably, the cross-modal correlation model is used to obtain the correlation between image and physiological signals. Image-text correlation and physiological signal-text correlation The associated feature matrix .

[0011] Preferably, the single-modal feature includes image features. Physiological signal characteristics and text features The heterogeneity fusion is used to obtain fusion features. Where M represents the associated feature matrix; the AI ​​model training: fusing features The improved Transformer model is input, with 12 encoder layers, 16 attention heads, and 1024 hidden layer dimensions. It is trained using the cross-entropy loss function to obtain the initial model for multimodal joint analysis.

[0012] Preferably, the dynamic optimization model is used to obtain dynamic optimization coefficients and update the initial model of multimodal joint analysis based on the dynamic optimization coefficients; the feature association report is a professional analysis document that quantifies, interprets the patterns, and verifies the credibility of cross-modal feature associations of three types of medical data—images, physiological signals, and text—based on the updated initial model of multimodal joint analysis.

[0013] Preferably, the iterative update model is used to obtain the iterative update coefficient based on the key correlation indicators; the data iterative update and storage: if the iterative update coefficient is greater than 0.8, the new data, corrected parameters, correlation matrix, and fusion features are stored in the data management module as training samples for the next model; if the iterative update coefficient is less than or equal to 0.8, the data quality assessment module is returned to perform quality correction again, and the correction parameters are adjusted until the iterative update coefficient is greater than 0.8 before being stored in the database, thereby realizing the iterative optimization of data and model.

[0014] Preferably, the anomaly correlation model includes a feature deviation model and an anomaly feature comprehensive index calculation model; the feature deviation model is used to obtain image feature deviation, physiological signal feature deviation, and text feature deviation; the anomaly feature comprehensive index calculation model is used to obtain an anomaly index; the anomaly correlation analysis report: when the anomaly index is greater than 1.2, an anomaly correlation analysis report is generated to clarify the abnormal mode combination and the source of feature deviation.

[0015] The technical effects and advantages of this invention are as follows: 1. This invention combines modality-specific preprocessing with adaptive quality correction, along with a quality index and a targeted correction formula, to obtain standardized, high-quality corrected parameters. This solves the problem of high data quality sensitivity in existing technologies, which are easily affected by image artifacts, signal noise, and text errors, leading to poor stability of analysis results. It achieves the beneficial effect of improving the consistency and reliability of multimodal data and providing a high-quality data foundation for subsequent joint analysis. 2. This invention constructs cross-modal correlation degree and correlation feature matrix through dynamic correlation modeling, and combines the cross-product calculation method of heterogeneous feature fusion to obtain fusion features that reflect the intrinsic relationship between modalities and the initial model for joint analysis. This solves the problems of insufficient intermodal correlation mining, weak data heterogeneity processing capability, and redundancy and information conflict caused by feature splicing in the existing technology. It achieves the beneficial effect of deeply releasing the collaborative value of multimodal data and improving the accuracy of joint analysis. 3. This invention obtains a dynamically updated analytical model and a continuously enriched high-quality sample library through a closed-loop mechanism of real-time optimization coefficient calculation and iterative update of the model. This solves the problem of insufficient dynamic adaptability of existing technologies and the inability of the model to adjust in real time according to new data, which leads to the lag in analysis results. It achieves the beneficial effect of ensuring the real-time performance and timeliness of multimodal data joint analysis and supporting the long-term continuous application of medical data. 4. This invention calculates accurate anomaly correlation analysis reports by using the feature deviation value of anomaly detection and the anomaly comprehensive index. This solves the problem in the prior art where the high sensitivity of data quality leads to the inability to identify anomaly data and the resulting distortion of analysis results. It achieves the beneficial effects of clarifying the source of anomalies, avoiding interference from anomaly data with analysis results, and improving the reliability of multimodal joint analysis and the controllability of data quality. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a schematic diagram of the process structure of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] As attached Figure 1 The system shown is a multimodal medical data joint analysis system based on artificial intelligence, including a data acquisition module, a data quality assessment module, a data dynamic association module, a heterogeneity feature fusion module, a real-time model optimization module, a data feedback iteration module, and a data anomaly feature detection and analysis module.

[0019] As attached Figure 2 The system shown is a multimodal medical data joint analysis system based on artificial intelligence. The functions of each module are as follows: The data acquisition module: acquires multimodal medical data and performs modality-specific preprocessing on the multimodal medical data.

[0020] In this embodiment, it should be specifically noted that the multimodal medical data includes image data, physiological signal data, and text data; the image data specifically includes the average pixel value of the lesion area. Pixel variance of lesion area and the gradient value at the edge of the lesion The physiological signal data specifically includes the mean RR interval. ST segment offset and QRS duration The text data specifically includes the frequency of medical history keywords. symptom description semantic similarity and the completeness of medication records The modality-specific preprocessing employs adaptive median filtering to remove motion artifacts from the image, resulting in the preprocessed mean pixel value of the lesion region. A 5-level db4 wavelet basis transform was used to remove signal noise, and the mean RR interval after preprocessing was obtained. The BERT model was used for semantic encoding to obtain the frequency of preprocessed medical history keywords. and the completeness of medication records after pretreatment The preprocessing methods for multimodal medical data are highly compatible with the data characteristics: MRI data is susceptible to motion artifacts, and 3×3 window adaptive median filtering can accurately remove artifacts while preserving lesion details; ECG signal noise is mainly high-frequency electromagnetic interference, and 5-layer db4 wavelet basis transform can specifically separate signals and noise; electronic medical records require semantic feature extraction, and a BERT model with 768 hidden layer dimensions can cover the complex semantic expressions of medical terms. Parameter calculation is designed based on the physical / semantic attributes of the modal data to ensure that the parameters retain the core information of the original data after preprocessing. Furthermore, the original multimodal medical data has inherent limitations: MRI images contain artifacts, ECG signals are mixed with noise, and electronic medical records have semantic ambiguity issues, which can lead to feature distortion if used directly for analysis; and the three types of data have significant differences in dimensionality and units, making subsequent steps impossible without preprocessing. Therefore, this step is a necessary link between data acquisition and subsequent analysis. Meanwhile, targeted preprocessing transforms the raw data into standardized, high-quality parameters, eliminating the interference of data impurities on the analysis and providing input data in a unified format for subsequent quality assessment and correlation modeling, thus laying the foundation for multimodal joint analysis.

[0021] The data quality assessment module constructs a quality index model based on the preprocessed data, quantifies the data quality, and generates corrected parameters.

[0022] In this embodiment, it should be specifically noted that: the quality index model includes an image quality index model, a physiological signal quality index model, and a text quality index model; the image quality index model is used to construct an image quality index calculation model to obtain the image quality index. The physiological signal quality index model is used to construct a physiological signal quality index calculation model to obtain the physiological signal quality index. The text quality index model is used to construct a text quality index calculation model to obtain the text quality index. The corrected parameters specifically include: if <0.6, Generative Adversarial Network (GAN) was used to enhance the image, and the corrected mean pixel value of the lesion area was obtained. and lesion edge gradient value ;like If the value is less than 0.5, an adaptive Kalman filter is used to complete the signal, and the corrected mean RR interval is obtained. and QRS duration ;like The frequency of medical history keywords was less than 0.55. A semantic error correction model was used to optimize the text, resulting in the corrected frequency of these keywords. Completeness of medication records If the quality index meets the standard If the corrected parameters are consistent with the preprocessed parameters, then the quality index formulas are all constructed based on the preprocessed parameters and incorporate the inherent range of each modality of data to ensure the objective quantification of the evaluation results. The correction methods are precisely matched with the quality issues: poor image quality is enhanced and repaired with GAN to restore lesion details, low physiological signal quality is supplemented with Kalman filtering to complete temporal information, and insufficient text quality is optimized with semantic error correction models to form a logical closed loop of evaluation-correction. Furthermore, preprocessing can only initially remove common impurities and cannot completely guarantee data quality (such as severe motion artifacts, strong noise, and complex text errors). If substandard data is used directly, it will lead to deviations in subsequent correlation modeling and distortion of fusion features, ultimately affecting the reliability of the analysis results. Therefore, this module is a key link in filtering low-quality data and ensuring the accuracy of analysis. By quantifying data reliability through quality assessment and optimizing parameters through adaptive correction, it ensures that the data input to subsequent steps all meet the quality standards, avoids analysis errors caused by differences in data quality, and improves the stability of multimodal joint analysis.

[0023] The data dynamic association module: constructs a cross-modal association degree model based on the corrected parameters and generates an association feature matrix.

[0024] In this embodiment, it should be specifically noted that the cross-modal correlation model is used to obtain the correlation between image and physiological signal. Image-text correlation and physiological signal-text correlation The associated feature matrix The correlation formula integrates and corrects parameters, and uses exponential, logarithmic, and square root functions to balance the influence of parameters on the attenuation of ST segment offset on correlation, which conforms to the characteristics of cross-influence and nonlinear correlation of multimodal data. The 3×3 correlation matrix structurally presents the intramodal (diagonal 1) and intermodal (off-diagonal correlation) relationships, providing a clear mathematical basis for subsequent heterogeneous fusion and designing a mathematical logic that conforms to multimodal correlation analysis. Furthermore, existing technologies often employ independent single-modal analysis and simple splicing, failing to explore the intrinsic connections between modalities (such as the correlation between MRI lesion features and ECG signal changes), resulting in analysis results that cannot reflect the full picture of the data. Without this module, subsequent feature fusion would become an unfounded parameter superposition, losing the core value of multimodal joint analysis. Therefore, this step is a core prerequisite for achieving deep multimodal fusion. By constructing cross-modal correlation degree and correlation matrix, the intrinsic relationship between image, physiological signal and text data is quantified, breaking the limitations of single-modal analysis, providing modal correlation basis for subsequent heterogeneous feature fusion, and making the fused features better reflect the synergistic value of multimodal data.

[0025] The heterogeneity feature fusion module extracts single-modal features based on the correlation feature matrix and performs heterogeneity fusion, and obtains an initial model for multimodal joint analysis through AI model training.

[0026] In this embodiment, it should be specifically noted that the single-modal features include image features. Physiological signal characteristics and text features In the single-modal feature extraction formula, the comma is the standard separator for multi-dimensional feature vectors. Its core function is to distinguish different feature components under the same modality, forming a structured feature vector. The heterogeneous fusion is used to obtain fused features. Where M represents the associated feature matrix; the AI ​​model training: fusing features Inputting an improved Transformer model with 12 encoder layers, 16 attention heads, and 1024 hidden layers, and training it using the cross-entropy loss function, yields the initial model for multimodal joint analysis. The single-modal feature extraction formula enhances the representation of features across different dimensions through nonlinear combinations of parameters (square, cube, logarithm, etc.), avoiding the limitations of a single indicator. The fusion formula, based on the correlation matrix, eliminates heterogeneity through self-multiplication of intra-modal features and cross-production of inter-modal features, achieving feature synergy without the need for weight coefficients. The improved Transformer model (12-layer encoder, 16 attention heads) adapts to high-dimensional fusion features, and the cross-entropy loss function effectively optimizes model parameters, meeting the analytical needs of multimodal data. Furthermore, when single-modal features exist independently, they cannot reflect the synergistic advantages of multimodal data; and intermodal heterogeneity (differences in dimension and unit) can lead to feature redundancy or conflict when directly spliced. If this module is omitted, relying solely on single-modal features cannot achieve deep joint analysis, and there is a lack of a core model capable of handling fused features. Therefore, this step is a crucial link in transforming multimodal data into analytical capabilities. By generating unified-dimensional fusion features through heterogeneous fusion, and then training to obtain the initial model for joint analysis, the transformation from multimodal data to analytical capabilities is realized, providing core analytical tools for subsequent real-time optimization and anomaly detection, and improving the utilization efficiency of multimodal data.

[0027] The real-time model optimization module continuously acquires updated multimodal medical data, associated feature matrices, and fused features, constructs dynamic optimization model optimization coefficients, and generates feature association reports.

[0028] In this embodiment, it is specifically necessary to explain the new correlation degree. , as well as ,in This represents the average pixel value of the lesion region after the new preprocessing and correction. This represents the mean RR interval after the new preprocessing and correction. This represents the gradient value of the lesion margin after new preprocessing and correction. This indicates the duration of the QRS group after the new preprocessing and correction. This indicates the offset of the ST segment after the new preprocessing. This indicates the frequency of medical history keywords after the new preprocessing and correction. This indicates the completeness of the medication records after the new preprocessing and correction. Represents the semantic similarity of the symptom descriptions after the new preprocessing; a new association feature matrix. The dynamic optimization model is used to obtain the dynamic optimization coefficients. The initial model for multimodal joint analysis is updated based on the dynamic optimization coefficients to obtain the optimized model. The feature association report is a professional analysis document that quantifies, interprets patterns, and verifies the credibility of cross-modal feature associations among three types of medical data: images, physiological signals, and text, based on the updated multimodal joint analysis initial model. Specifically, the calculation logic of new parameters is consistent with historical parameters, ensuring the comparability of old and new data; optimization coefficients are quantified by combining the ratio and logarithm of the old and new association degrees, quantifying the optimization effect of new data on the model and avoiding blind updates; the model update formula balances the weights of the old and new models and features, ensuring the stability of the updated model; the feature association report is generated based on quantified association degrees, with specific and traceable content, meeting the rigorous requirements of medical data applications. Medical data is generated in real time and continuously updated; static models cannot adapt to changes in new data, leading to delayed analysis results; and the analysis results must be output in a standardized report format, otherwise they cannot provide effective support for subsequent applications (such as treatment plan support). Therefore, this step is a necessary link to ensure the timeliness and practicality of the analysis. By optimizing and updating the model in real time, we ensure that the model can adapt to new multimodal data and avoid the lag in analysis results. The generated feature association report transforms the abstract model analysis results into specific and applicable quantitative information, providing direct support for the subsequent application of medical data.

[0029] The data feedback iteration module constructs an iterative update model based on the key correlation indicators in the feature correlation report, and performs iterative data updates and storage according to the coefficients.

[0030] In this embodiment, it should be specifically noted that the iterative update model is used to obtain iterative update coefficients based on the key correlation indicators. The data iteration update and storage process is as follows: If the iteration update coefficient is greater than 0.8, the new corrected parameters, correlation matrix, and fusion features are stored in the data management module as training samples for the next model. If the iteration update coefficient is less than or equal to 0.8, the data quality assessment module is returned to perform quality correction, adjusting the correction parameters until the iteration update coefficient is greater than 0.8, at which point the data is stored in the database, achieving iterative optimization of the data and model. The iteration update coefficient, through a combination of the square and square root of the correlation between the new and old data, accurately quantifies the supplementary value of new data to historical data. The decision to store or recalibrate based on the coefficient avoids invalid data consuming resources, aligning with the optimization logic of data management. The closed-loop iteration mechanism (data update, model optimization, and data re-update) conforms to the core principle of data-driven model improvement in machine learning, ensuring that the model and data evolve synchronously. If omitted, new data cannot be included in the database, resulting in outdated model training samples and a lack of high-quality data support for subsequent optimization. Furthermore, the inability to identify low-value data can lead to data redundancy or quality issues, affecting the long-term performance of the model. Therefore, this step is fundamental for achieving continuous optimization of the model and data. By using a closed-loop iterative mechanism to screen and store high-value new data, the sample library of the data management module is enriched, providing a more comprehensive dataset for subsequent model training, promoting the continuous improvement of model accuracy and analytical capabilities, and ensuring the long-term effectiveness of the multimodal joint analysis method.

[0031] The data anomaly feature detection and analysis module: constructs an anomaly correlation model based on the optimized model and generates an anomaly correlation analysis report.

[0032] In this embodiment, it should be specifically noted that: the anomaly correlation model includes a feature deviation model and an anomaly feature comprehensive index calculation model; the feature deviation model is used to obtain image feature deviation. Physiological signal characteristic deviation and text feature bias ,in This represents the average image feature value of similar data in the database. This represents the mean of physiological characteristics for data of the same type. This represents the mean of textual features of similar data; the anomaly feature comprehensive index calculation model is used to obtain the anomaly index. The anomaly correlation analysis report is generated as follows: when the anomaly index is greater than 1.2, an anomaly correlation analysis report is generated, clearly identifying the abnormal modal combination (such as abnormal image-physiological signal correlation) and the source of feature deviation (such as the correlation between ST segment offset and the mean lesion pixel value exceeding the range of historical similar data by 15%). When the anomaly index is less than or equal to 1.2, a normal correlation analysis report is generated. The feature deviation value combines the difference between the current feature and the historical mean, as well as the cross-modal correlation, to avoid misjudgment of single-modal anomalies. The anomaly comprehensive index is calculated through the cross-product of feature deviation values, strengthening the synergistic impact of multimodal anomalies, conforming to the logic that multimodal anomalies require comprehensive judgment. The anomaly threshold is determined based on historical anomaly data statistics. The report only clearly identifies the abnormal modality and the source of feature deviation, meeting the standardized requirements of medical data analysis. Sudden anomalies (such as severe artifacts, signal distortion, and text errors) may occur during the acquisition and preprocessing of multimodal data. If omitted, abnormal data will be mixed into the normal analysis process, leading to distorted correlation reports and affecting the reliability of subsequent applications; moreover, the source of the anomaly cannot be located, making data quality traceability difficult. Therefore, this step is a crucial link in ensuring the reliability of analysis results and controllable data quality. By accurately identifying abnormal correlation features in multimodal data, targeted anomaly analysis reports are generated, providing a clear basis for data quality control, avoiding the impact of abnormal data on analysis results, and providing direction for optimizing subsequent data collection and preprocessing processes, thereby further improving the reliability of multimodal joint analysis.

[0033] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal medical data joint analysis system based on artificial intelligence, characterized in that, include: Data acquisition module: Acquires multimodal medical data and performs modality-specific preprocessing on the multimodal medical data; Data quality assessment module: Constructs a quality index model based on preprocessed data, quantifies data quality assessment, and generates corrected parameters; Data dynamic association module: Constructs a cross-modal association degree model based on the corrected parameters and generates an association feature matrix; Heterogeneity feature fusion module: Extracts single-modal features based on the correlation feature matrix and performs heterogeneity fusion, and obtains an initial model for multimodal joint analysis through AI model training; Real-time model optimization module: Continuously acquires updated multimodal medical data, associated feature matrices, and fused features, constructs dynamic optimization model optimization coefficients, and generates feature association reports; Data feedback iteration module: Constructs an iterative update model based on the key correlation indicators in the feature correlation report, and performs iterative data updates and storage according to the coefficients; Data anomaly feature detection and analysis module: Based on the optimized model, construct an anomaly correlation model and generate an anomaly correlation analysis report.

2. The multimodal medical data joint analysis system based on artificial intelligence according to claim 1, characterized in that: The multimodal medical data includes image data, physiological signal data, and text data. Specifically, the image data includes the mean pixel value of the lesion region, the pixel variance of the lesion region, and the gradient value of the lesion edge. The physiological signal data specifically includes the mean RR interval, ST segment offset, and QRS complex duration. The text data specifically includes the frequency of medical history keywords, semantic similarity of symptom descriptions, and completeness of medication records. The modality-specific preprocessing uses adaptive median filtering to remove image motion artifacts, obtaining the preprocessed mean pixel value of the lesion region. A 5-layer db4 wavelet basis transform is used to remove signal noise, obtaining the preprocessed mean RR interval. A BERT model is used for semantic encoding to obtain the preprocessed frequency of medical history keywords and the preprocessed completeness of medication records.

3. The multimodal medical data joint analysis system based on artificial intelligence according to claim 1, characterized in that: The quality index model includes an image quality index model, a physiological signal quality index model, and a text quality index model; the image quality index model is used to construct an image quality index calculation model to obtain an image quality index; the physiological signal quality index model is used to construct a physiological signal quality index calculation model to obtain a physiological signal quality index; and the text quality index model is used to construct a text quality index calculation model to obtain a text quality index.

4. The multimodal medical data joint analysis system based on artificial intelligence according to claim 3, characterized in that: The corrected parameters specifically include: if the image quality index is less than 0.6, a generative adversarial network is used to enhance the image to obtain the corrected mean pixel value of the lesion region and the gradient value of the lesion edge; if the physiological signal quality index is less than 0.5, an adaptive Kalman filter is used to complete the signal to obtain the corrected mean RR interval and QRS complex duration; if the text quality index is less than 0.55, a semantic error correction model is used to optimize the text to obtain the corrected frequency of medical history keywords and the completeness of medication records.

5. The multimodal medical data joint analysis system based on artificial intelligence according to claim 1, characterized in that: The cross-modal correlation model is used to obtain the correlation between image and physiological signal. Image-text correlation and physiological signal-text correlation The associated feature matrix .

6. The multimodal medical data joint analysis system based on artificial intelligence according to claim 1, characterized in that: The single-modal features include image features. Physiological signal characteristics and text features The heterogeneity fusion is used to obtain fusion features. Where M represents the associated feature matrix; the AI ​​model training: fusing features The improved Transformer model is input, with 12 encoder layers, 16 attention heads, and 1024 hidden layer dimensions. It is trained using the cross-entropy loss function to obtain the initial model for multimodal joint analysis.

7. The multimodal medical data joint analysis system based on artificial intelligence according to claim 1, characterized in that: The dynamic optimization model is used to obtain dynamic optimization coefficients and update the initial model of multimodal joint analysis based on the dynamic optimization coefficients. The feature association report is a professional analysis document that quantifies, interprets the patterns, and verifies the credibility of cross-modal feature associations of three types of medical data: images, physiological signals, and text, based on the updated initial model of multimodal joint analysis.

8. The multimodal medical data joint analysis system based on artificial intelligence according to claim 1, characterized in that: The iterative update model is used to obtain the iterative update coefficient based on the key correlation indicators; the data iterative update and storage: if the iterative update coefficient is greater than 0.8, the new data, corrected parameters, correlation matrix, and fusion features are stored in the data management module as training samples for the next model; if the iterative update coefficient is less than or equal to 0.8, the data quality assessment module is returned to perform quality correction again, and the correction parameters are adjusted until the iterative update coefficient is greater than 0.8 before being stored in the database, thereby realizing the iterative optimization of data and model.

9. The multimodal medical data joint analysis system based on artificial intelligence according to claim 1, characterized in that: The anomaly correlation model includes a feature deviation model and an anomaly feature comprehensive index calculation model; the feature deviation model is used to obtain image feature deviation, physiological signal feature deviation, and text feature deviation; the anomaly feature comprehensive index calculation model is used to obtain an anomaly index; the anomaly correlation analysis report: when the anomaly index is greater than 1.2, an anomaly correlation analysis report is generated to clarify the abnormal mode combination and the source of feature deviation.

Images