Rheumatoid arthritis early-stage AI intelligent diagnosis system based on multi-modal data fusion

The AI-powered intelligent diagnostic system, which integrates multimodal data fusion, addresses the issues of delayed early diagnosis of rheumatoid arthritis (RA) and missed diagnosis of seronegative patients. It enables quantitative assessment of early RA risk and stratified intervention guidance, and is a lightweight device suitable for primary hospitals.

CN121281801APending Publication Date: 2026-01-06THE FIRST AFFILIATED HOSPITAL OF JINZHOU MEDICAL UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202511439433.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-10-10
Publication Date
2026-01-06

AI Technical Summary

Technical Problem

Current technologies for early diagnosis of rheumatoid arthritis suffer from problems such as delays, high rates of missed diagnoses of seronegative patients, and a lack of quantitative standards for risk assessment.

Method used

The AI-powered intelligent diagnostic system employs multimodal data fusion, including multi-source data acquisition, data preprocessing, cross-modal fusion, and dynamic time-series models. Through deep semantic fusion and time-series analysis of serological, ultrasound, and X-ray data, it outputs a quantitative risk score of 0-10.

Benefits of technology

It achieves performance improvement in early RA diagnosis, especially with a detection rate of 92.3% for seronegative patients. The output risk score is strongly correlated with disease progression, supports real-time inference in primary hospitals, and complies with privacy protection regulations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121281801A_ABST
    Figure CN121281801A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-modal data fusion-based rheumatoid arthritis early-stage AI intelligent diagnosis system, belongs to the technical field of medical artificial intelligence, and aims to solve the problems that in the prior art, RA early-stage diagnosis lags behind, the rate of missed diagnosis of serum negative patients is high, and risk assessment lacks quantitative standards. Comprising a multi-source data acquisition module, a data preprocessing module, a cross-modal fusion module, a dynamic time sequence model and a risk assessment module which are connected in sequence, according to the system provided by the invention, a dynamic tracking model of serological change-synovial inflammation-bone structure change is innovatively constructed, and an RA diagnosis window is advanced compared with that of a traditional method (depending on clinical symptoms) by capturing time sequence correlation of the serological change, the synovial inflammation and the bone structure change; aiming at the diagnosis difficulty of serum-negative RA, the system realizes great improvement of diagnosis performance through deep fusion of multi-modal features, serological core indexes, ultrasonic synovial dynamic features and X-ray bone microscopic features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of medical artificial intelligence technology, specifically involving an AI-powered intelligent diagnostic system for early rheumatoid arthritis based on multimodal data fusion. Background Technology

[0002] Rheumatoid arthritis (RA), a systemic autoimmune disease characterized by synovial inflammation, primarily causes progressive joint destruction due to abnormal synovial proliferation. Clinical data shows that without effective intervention within six months of onset (the medically defined intervention window), the annual incidence of irreversible bone erosion is as high as 20%, and the risk of joint dysfunction and disability increases exponentially with disease progression. However, current clinical diagnostic systems face multiple technical bottlenecks, severely hindering the early identification and intervention of RA.

[0003] Traditional RA diagnosis relies heavily on the detection of rheumatoid factor (RF) and anti-cyclic citrullinated peptide (anti-CCP) antibodies, but their sensitivity in the early stages of RA is only 68%–75%. A more significant problem is that approximately 25% of diagnosed RA patients present with double serological negativity (RF < 20 IU / ml and anti-CCP < 5 U / ml).

[0004] Existing imaging techniques all have significant limitations in the early diagnosis of RA. Although hand X-rays are a routine clinical examination, their detection rate of bone erosion in the 12 months prior to onset is only 12% (based on data from the 2022 EULAR Imaging Guidelines), failing to capture pathological changes in the early synovial inflammation stage. High-frequency ultrasound (12-18MHz) can detect inflammatory indicators such as synovial thickening (≥2mm) and blood flow signals, but the manually interpreted intraclass correlation coefficient (ICC) is only 0.68, and there is a lack of unified dynamic quantitative standards (such as the definition of synovial thickening rate and blood flow signal change amplitude), leading to significant differences in test results among different centers (coefficient of variation CV=23.5%). Although MRI is considered the gold standard for assessing synovial inflammation, its cost per examination is as high as 800 yuan, and it is contraindicated for patients with metal implants. Its coverage rate in primary hospitals is less than 15%, making it difficult to use as a routine screening method.

[0005] Publicly available AI diagnostic technologies related to RA, such as the rheumatoid arthritis activity grading device based on multimodal data in Chinese patent application No. 202310755346.1, have the following key shortcomings: they only achieve simple integration of serological and imaging data through feature splicing, without establishing a pathological semantic association between "changes in serological indicators - degree of synovial inflammation - bone structure damage", resulting in a lack of clinical interpretability of the fused features; they cannot capture the dynamic pathological chain of RA progression (such as the continuous process of "doubling of RF titer → aggravation of synovial inflammation → occurrence of bone erosion"), and can only achieve static diagnosis, which does not match the progressive characteristics of the disease; the risk assessment result is a binary classification ("yes / no" RA), without providing a quantitative scoring system, and cannot guide individualized intervention strategies (such as the difference in intervention plans between mild and severe inflammation).

[0006] Therefore, an AI-powered intelligent diagnostic system for early rheumatoid arthritis that integrates multimodal data is needed to address the problems of delayed early diagnosis of RA, high rate of missed diagnosis of seronegative patients, and lack of quantitative standards for risk assessment in existing technologies. Summary of the Invention

[0007] The purpose of this invention is to provide an AI-powered intelligent diagnostic system for early rheumatoid arthritis based on multimodal data fusion, in order to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: an AI-powered intelligent diagnostic system for early rheumatoid arthritis based on multimodal data fusion, comprising a multi-source data acquisition module, a data preprocessing module, a cross-modal fusion module, a dynamic time-series model, and a risk assessment module connected sequentially; wherein: The multi-source data acquisition module is used to simultaneously acquire serological marker data, high-frequency joint ultrasound video data, and hand X-ray data of the subject to be tested. The serological marker data includes, but is not limited to, 12 indicators such as RF, anti-CCP, ESR, and CRP. The high-frequency joint ultrasound video data covers grayscale and energy Doppler signals of 28 joints of both hands. The hand X-ray data is an anteroposterior image of both hands. The data preprocessing module is used to standardize the collected multi-source data, including missing value imputation, feature screening and normalization of serological data, frame alignment, synovial segmentation and dynamic feature extraction of ultrasound video data, and noise reduction and enhancement, bone structure segmentation and micro-feature quantification of X-ray data. The cross-modal fusion module adopts a medical prior-constrained Transformer architecture. By introducing a joint-marker association matrix M to modulate attention weights, it achieves deep semantic fusion of serological feature vectors, ultrasound spatiotemporal feature vectors, and X-ray imaging feature vectors, and outputs a fused feature matrix. The dynamic time series model is built on Bi-LSTM network and medical knowledge graph. It is used to process time series data of 3-12 months. It processes asynchronous data through linear interpolation and embeds pathological rules into LSTM gating parameters based on knowledge distillation to output time series feature vectors. The risk assessment module outputs the RA risk probability and a 0-10 quantitative risk score through a multi-task learning head. The risk score is used to guide stratified intervention strategies.

[0009] The protocol specifies that the serological biomarkers include 12 indicators such as RF (detection range 0-200 IU / ml, CV < 5%), anti-CCP (0-25 U / ml, CV < 4%), AKA, APF, ESR (0-100 mm / h), CRP (0-10 mg / L), IL-6, and TNF-α. The standardization employs a modified Z-score method, specifically calculated as follows:

[0010] in The original detection value. This is the average value for healthy individuals. The standard deviation of the healthy population is represented by 0.1, which is a correction term to avoid a denominator of 0.

[0011] It is further worth noting that the attention calculation of the cross-modal fusion module introduces a joint-marker association matrix M, which is 28×12 dimensional. Matrix element M(i,j) represents the pathological association strength between the i-th joint and the j-th serological marker (value range 0-1), determined based on clinical evidence from the ACR / EULAR guidelines regarding the correlation between joint involvement and serological markers. The specific formula for the attention calculation is as follows:

[0012] Where Q (query matrix), K (key matrix), and V (value matrix) are all matrix representations of multimodal features (with consistent dimensions), where Q and K are used to calculate the correlation strength between features, and V is the feature value to be weighted; The similarity matrix between Q and K is calculated by matrix multiplication, reflecting the original association between different modal features; Scaling factor This is the feature dimension, used to avoid the softmax gradient vanishing due to excessively large matrix element values. softmax: an activation function that normalizes the corrected similarity matrix into attention weights (the sum of the weights is 1).

[0013] Furthermore, it should be noted that the dynamic time series model uses linear interpolation to process asynchronous data. Specifically, for the time misalignment between monthly collected serological data and quarterly collected imaging data, a time axis is constructed on a weekly basis, and the missing feature values ​​at the time points are supplemented through linear interpolation, with an interpolation error of <3%. The time weight decays exponentially with the interval between the collection time and the current time, and the decay formula is as follows:

[0014] in, : The interval between the data collection time and the current time (unit: months); Attenuation coefficient (value 0.2, determined based on clinical data optimization), controls the rate of weight attenuation; : Natural constant.

[0015] In a preferred embodiment, the risk score is calculated by weighting serological characteristic scores, ultrasound characteristic scores, X-ray characteristic scores, and time-series change scores, using the following formula:

[0016] in, This is a feature normalization function that maps input features to the range of 0-1. Normalization: ,in These are the minimum and maximum values ​​of this feature in the training set; S: Serological core feature vector (including anti-CCP, RF, ESR, etc.), weighted by 3 points (reflecting the basic diagnostic value of serological indicators); U: Ultrasound feature vector (including synovial thickness, blood flow signal intensity, etc.), weighted by 2 points (reflecting the real-time state of synovial inflammation); X: X-ray feature vector (including trabecular spacing, cortical bone defect area, etc.), weighted by 2 points (reflecting cumulative damage to bone structure); T: Temporal feature vector (including synovial thickening rate and RF titer change rate within 3 months), weighted by 3 (reflects the disease progression trend and is crucial for early diagnosis); Final risk score (0-10 points): the higher the score, the higher the risk of RA, which directly corresponds to the stratified intervention strategy (0-3 points follow-up, 4-5 points NSAIDs intervention, ≥6 points MTX or biologics treatment).

[0017] Compared with existing technologies, the multimodal data fusion-based AI intelligent diagnostic system for early rheumatoid arthritis provided by this invention has at least the following beneficial effects: (1) The system provided by this invention innovatively constructs a dynamic tracking model of “serological changes → synovial inflammation → bone structure changes”. By capturing the temporal correlation of the three, the diagnostic window for RA is advanced compared to traditional methods (which rely on the appearance of clinical symptoms).

[0018] (2) This invention addresses the diagnostic challenge of serum-negative RA (which accounts for about 25% of RA patients). The system achieves a significant improvement in diagnostic performance through the deep fusion of multimodal features, including serological core indicators, ultrasound synovial dynamic features, and X-ray bone microscopic features.

[0019] (3) The 0-10 dynamic risk score output by the system provided by this invention is strongly correlated with disease progression, providing a quantitative basis for clinical decision-making. Furthermore, the system adopts a lightweight design, supports real-time inference on edge devices, and is compatible with existing equipment in primary hospitals (such as conventional chemiluminescence immunoassay analyzers, high-frequency ultrasound machines, and digital X-ray machines). At the same time, privacy protection is achieved through a federated learning framework, which complies with medical data security standards. Attached Figure Description

[0020] Figure 1 This is a block diagram of the multimodal data fusion-based AI intelligent diagnostic system for early rheumatoid arthritis according to the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to embodiments.

[0022] Please see Figure 1 This invention provides an AI-powered intelligent diagnostic system for early rheumatoid arthritis based on multimodal data fusion, comprising a multi-source data acquisition module, a data preprocessing module, a cross-modal fusion module, a dynamic time-series model, and a risk assessment module connected in sequence; wherein: The multi-source data acquisition module is used to simultaneously acquire serological marker data, high-frequency joint ultrasound video data, and hand X-ray data of the subject to be tested. The serological marker data includes, but is not limited to, 12 indicators such as RF, anti-CCP, ESR, and CRP. The high-frequency joint ultrasound video data covers grayscale and energy Doppler signals of 28 joints of both hands. The hand X-ray data is an anteroposterior image of both hands. Specifically, the multi-source data acquisition module includes a serological testing unit: using a chemiluminescence immunoassay analyzer (such as Rochecobase 602), it detects 12 indicators including RF (detection range 0-200 IU / ml, CV<5%), anti-CCP (0-25 U / ml, CV<4%), ESR (0-100 mm / h), and CRP (0-10 mg / L), and outputs a DICOM format test report; High-frequency ultrasound unit: Equipped with an 18MHz linear probe (such as Philips EPIQ7), it performs dynamic scanning of 28 joints of both hands (including 6 wrist joints, 10 metacarpophalangeal joints, and 12 interphalangeal joints), acquiring grayscale (GS) and energy Doppler (PD) videos (resolution 1280×720, frame rate 30fps, scan duration 15 seconds per joint); X-ray imaging unit: Uses a digital X-ray machine (such as GE Definium 8000) to take bi-hand anteroposterior films (tube voltage 55-65kV, tube current 8-12mA, resolution 512×512), and outputs DICOM format images.

[0023] The data preprocessing module is used to standardize the collected multi-source data, including missing value imputation, feature screening and normalization of serological data, frame alignment, synovial segmentation and dynamic feature extraction of ultrasound video data, and noise reduction and enhancement, bone structure segmentation and micro-feature quantification of X-ray data. Specifically, the serological data processing steps are as follows: a1. Missing value imputation: Multiple imputation method (MICE) based on random forest was adopted, and the imputation accuracy reached 92.3% (28% higher than mean imputation). b1. Feature selection: Core indicators (anti-CCP, RF, ESR, synovial thickness, and degree of bone erosion) are selected through recursive feature elimination (RFE), and the importance weight of the features is verified by Shapley value; c1. Standardization: The data is normalized using the Box-Cox transformation (skewness < 0.3).

[0024] The ultrasound video processing steps are as follows: a2. Inter-frame alignment: Optical flow (Farneback algorithm) is used to correct probe movement, so that the joint area positioning error is less than 1 pixel; b2. Synovial membrane segmentation: Based on the ST-AttentionU-Net model, the synovial membrane region is segmented (Dice coefficient 0.91), and the synovial membrane thickness (mm) and PD signal intensity (0-4 levels, referencing the EULAR standard) are quantified. c2. Dynamic feature extraction: Calculate the synovial thickening rate (mm / month) and blood flow signal change rate (%) within 3 months.

[0025] The X-ray film processing steps are as follows: a3. Noise Reduction and Enhancement: Employs the BM3D algorithm to remove quantum noise, improving contrast by 30%; b3. Bone structure segmentation: Cortical bone and trabecular bone regions were segmented using the ResNet-50+U-Net model (IoU value 0.89). c3. Quantification of microscopic features: Extraction of trabecular spacing (normal range 0.2–0.4 mm) and cortical bone defect area (mm²). 2 12 parameters including )

[0026] The cross-modal fusion module adopts a medical prior-constrained Transformer architecture. By introducing a joint-marker association matrix M to modulate attention weights, it achieves deep semantic fusion of serological feature vectors, ultrasound spatiotemporal feature vectors, and X-ray imaging feature vectors, and outputs a fused feature matrix. Specifically, the Transformer fusion module employing medical prior constraints includes the following steps: a4. Input: S, U, X feature matrices (batch_size×seq_len×feature_dim); b4. Attention mechanism: Introduce joint-marker association matrix M (constructed based on ACR guidelines, such as the association weight of wrist joint and RF is 0.65) to correct self-attention calculation; c4. Output: Fusion feature F (128 dimensions), filtered by entropy-mutual information (retaining features with information entropy > 0.8 and mutual information > 0.35).

[0027] The dynamic time series model is built on Bi-LSTM network and medical knowledge graph. It is used to process time series data of 3-12 months. It processes asynchronous data through linear interpolation and embeds pathological rules into LSTM gating parameters based on knowledge distillation to output time series feature vectors. The risk assessment module outputs the RA risk probability and a 0-10 quantitative risk score through a multi-task learning head. The risk score is used to guide stratified intervention strategies.

[0028] It is worth further specifying that the serological markers include 12 indicators such as RF (detection range 0-200 IU / ml, CV < 5%), anti-CCP (0-25 U / ml, CV < 4%), AKA, APF, ESR (0-100 mm / h), CRP (0-10 mg / L), IL-6, and TNF-α; the standardization adopts a modified Z-score method, and the specific calculation method is as follows:

[0029] in The original detection value. This is the average value for healthy individuals. The standard deviation of the healthy population is represented by 0.1, which is a correction term to avoid a denominator of 0.

[0030] Furthermore, it is worth noting that the attention calculation of the cross-modal fusion module introduces a joint-marker association matrix M. This matrix M is 28×12 dimensional, and the matrix element M(i,j) represents the pathological association strength between the i-th joint and the j-th serological marker (value range 0-1), determined based on clinical evidence from the ACR / EULAR guidelines regarding the correlation between joint involvement and serological markers. The specific formula for the attention calculation is as follows:

[0031] Where Q (query matrix), K (key matrix), and V (value matrix) are all matrix representations of multimodal features (with consistent dimensions), where Q and K are used to calculate the correlation strength between features, and V is the feature value to be weighted; The similarity matrix between Q and K is calculated by matrix multiplication, reflecting the original association between different modal features; Scaling factor This is the feature dimension, used to avoid the softmax gradient vanishing due to excessively large matrix element values. softmax: an activation function that normalizes the corrected similarity matrix into attention weights (the sum of the weights is 1).

[0032] Furthermore, it is worth specifying that the dynamic time series model uses linear interpolation to process asynchronous data. Specifically, for the time misalignment between monthly collected serological data and quarterly collected imaging data, a time axis is constructed on a weekly basis, and the missing feature values ​​at the time points are supplemented through linear interpolation, with an interpolation error of <3%. The time weight decays exponentially with the interval between the collection time and the current time, and the decay formula is as follows:

[0033] in, : The interval between the data collection time and the current time (unit: months); Attenuation coefficient (value 0.2, determined based on clinical data optimization), controls the rate of weight attenuation; : Natural constant.

[0034] Furthermore, it is worth specifying that the risk score is calculated by weighting serological characteristic scores, ultrasound characteristic scores, X-ray characteristic scores, and time-series change scores, using the following formula:

[0035] in, This is a feature normalization function that maps input features to the range of 0-1. Normalization: ,in These are the minimum and maximum values ​​of this feature in the training set; S: Serological core feature vector (including anti-CCP, RF, ESR, etc.), weighted by 3 points (reflecting the basic diagnostic value of serological indicators); U: Ultrasound feature vector (including synovial thickness, blood flow signal intensity, etc.), weighted by 2 points (reflecting the real-time state of synovial inflammation); X: X-ray feature vector (including trabecular spacing, cortical bone defect area, etc.), weighted by 2 points (reflecting cumulative damage to bone structure); T: Temporal feature vector (including synovial thickening rate and RF titer change rate within 3 months), weighted by 3 (reflects the disease progression trend and is crucial for early diagnosis); Final risk score (0-10 points): the higher the score, the higher the risk of RA, which directly corresponds to the stratified intervention strategy (0-3 points follow-up, 4-5 points NSAIDs intervention, ≥6 points MTX or biologics treatment).

[0036] By integrating serological biomarkers, high-frequency joint ultrasound, and hand X-rays, a cross-modal Transformer fusion network with medical prior constraints is constructed. Combined with a Bi-LSTM dynamic temporal model, this enables early prediction and quantitative assessment of RA risk. The system innovatively incorporates a medical knowledge graph into an attention mechanism to solve the multimodal semantic alignment problem. By tracking temporal data for 3-12 months, RA risk can be identified 6-12 months before the onset of clinical symptoms, with a detection rate of 92.3% for serologically negative patients. The output risk score of 0-10 can directly guide stratified intervention. This invention provides a complete solution for the early and accurate diagnosis of RA and has significant clinical value.

[0037] In summary, the system provided by this invention innovatively constructs a dynamic tracking model of "serological changes → synovial inflammation → bone structural changes." By capturing the temporal correlation among these three factors, it advances the diagnostic window for RA compared to traditional methods (which rely on the appearance of clinical symptoms). Addressing the diagnostic challenge of seronegative RA (accounting for approximately 25% of RA patients), this system achieves a significant improvement in diagnostic performance through deep fusion of multimodal features: core serological indicators, dynamic ultrasound synovial features, and X-ray bone microscopic features. The 0-10 dynamic risk score output by the system is strongly correlated with disease progression, providing a quantitative basis for clinical decision-making. Furthermore, this system adopts a lightweight design, supports real-time inference on edge devices, and is compatible with existing equipment in primary hospitals (such as conventional chemiluminescence immunoassay analyzers, high-frequency ultrasound machines, and digital X-ray machines). Simultaneously, it achieves privacy protection through a federated learning framework, complying with medical data security regulations.

[0038] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0039] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-modal data fusion-based AI intelligent diagnosis system for early rheumatoid arthritis, characterized in that: The method comprises a plurality of source data acquisition modules, a data preprocessing module, a cross-modal fusion module, a dynamic timing model and a risk assessment module connected in sequence. The plurality of source data acquisition modules are used for synchronously acquiring serological marker data, high-frequency joint ultrasound video data and hand X-ray data of a to-be-detected object, the serological marker data includes but is not limited to RF, anti-CCP, ESR, CRP12 indicators, the high-frequency joint ultrasound video data covers the gray scale and energy Doppler signal of 28 joints of both hands, and the hand X-ray data is a double-hand orthotopic image; The data preprocessing module is used for standardizing the collected multi-source data, including missing value filling, feature screening and normalization of serological data, inter-frame alignment, synovial membrane segmentation and dynamic feature extraction of ultrasound video data, and denoising enhancement, bone structure segmentation and micro-feature quantization of X-ray data; The cross-modal fusion module adopts a Transformer architecture with medical prior constraints, modulates attention weights by introducing a joint-marker correlation matrix M, realizes deep semantic fusion of serological feature vectors, ultrasound spatio-temporal feature vectors and X-ray image feature vectors, and outputs a fusion feature matrix; The dynamic timing model is constructed based on a Bi-LSTM network and a medical knowledge graph, is used for processing 3-12 month timing data, processes non-synchronous data through linear interpolation, and embeds pathological rules into LSTM gate parameters based on knowledge distillation, and outputs a timing feature vector; The risk assessment module outputs an RA risk probability and a 0-10 quantized risk score through a multi-task learning head, and the risk score is used for guiding a stratified intervention strategy.

2. The multi-modal data fusion-based AI intelligent early diagnosis system for rheumatoid arthritis according to claim 1, characterized in that: The serological markers include RF, anti-CCP, AKA, APF, ESR, CRP, IL-6, TNF-alpha and 12 indicators; the standardization adopts a modified Z-score method, and the specific calculation method is: Wherein x is the original detection value, μ is the mean value of the healthy population, σ is the standard deviation of the healthy population, and 0.1 is a correction term to avoid a denominator of 0.

3. The multi-modal data fusion-based AI intelligent early diagnosis system for rheumatoid arthritis according to claim 1, characterized in that: The attention calculation of the cross-modal fusion module introduces a joint-marker correlation matrix M, the matrix M is 28x12-dimensional, the matrix element M(i,j) represents the pathological correlation strength (value range 0-1) of the i th joint and the j th serological marker, and is determined based on the clinical evidence of "joint involvement and serological index correlation" in the ACR / EULAR guidelines; the specific formula of the attention calculation is: Wherein, Q (query matrix), K (key matrix) and V (value matrix): all are matrix representations of multi-modal features (consistent dimensions), wherein Q and K are used for calculating the correlation strength between features, and V is a feature value to be weighted; QK T : similarity matrix of Q and K is calculated by matrix multiplication, reflecting the original correlation of different modal features; scaling factor, d k is the feature dimension, used to avoid the softmax gradient vanishing caused by too large matrix element values; softmax: an activation function, which normalizes the modified similarity matrix into attention weights (the weight sum is 1).

4. The multi-modal data fusion-based AI intelligent early diagnosis system for rheumatoid arthritis according to claim 1, characterized in that: The dynamic time series model adopts linear interpolation to process non-synchronous data. Specifically, for the time dislocation of serological data collected monthly and imaging data collected quarterly, a time axis is constructed in units of weeks, and the characteristic values of missing time points are supplemented by linear interpolation, with an interpolation error of <3%. The time weight exponentially decays with the interval between the collection time and the current time, and the decay formula is: ω = e -λ·t where t is the interval between the data collection time and the current time (in months); λ is the decay coefficient (value 0.2, determined based on clinical data optimization), which controls the weight decay rate; and e is the natural constant. The risk score is calculated by weighting the serological feature score, ultrasound feature score, X-ray feature score, and time series change score. The specific formula is: Score = 3 × f(S) + 2 × f(U) + 2 × f(X) + 3 × f(T) 5. The multi-modal data fusion-based AI intelligent early diagnosis system for rheumatoid arthritis according to claim 1, characterized in that: S: serological core feature vector (including anti-CCP, RF, ESR, etc.), weight 3 points (reflecting the basic diagnostic value of serological indicators); U: ultrasound feature vector (including synovial membrane thickness, blood flow signal intensity, etc.), weight 2 points (reflecting the real-time state of synovial inflammation); where f(·) is a feature normalization function that maps input features to the range 0-1, specifically min-max normalization: where x min ,x max are the minimum and maximum values of the feature in the training set. X: X-ray feature vector (including trabecular bone spacing, cortical bone defect area, etc.), weight 2 points (reflecting the cumulative damage of bone structure); T: time series feature vector (including synovial membrane thickening rate within 3 months, RF titer change rate, etc.), weight 3 points (reflecting the disease progression trend, which is crucial for early diagnosis); Score: final risk score (0-10 points), the higher the score, the higher the risk of RA, which directly corresponds to the stratified intervention strategy (0-3 points for follow-up, 4-5 points for NSAIDs intervention, ≥6 points for MTX or biologic therapy). ​ ​

Citation Information

Patent Citations

  • Rheumatoid arthritis activity grading device based on multimodal data

    CN116797572B