Intelligent medical diagnosis and decision support method using multi-source data

Abstract

The invention discloses an intelligent medical diagnosis and decision support method using multi-source data. The method comprises the following steps: S1, collecting medical multi-source data; s2, performing data preprocessing on the collected data to obtain standardized data; s3, performing multi-modal processing on the standardized data to obtain image features, text features, structured features and fusion features; s4, constructing medical diagnosis models of different modes; and S5, based on the medical diagnosis model, a corresponding personalized diagnosis result is generated through a dynamic weight adjustment mechanism, and a treatment scheme is generated. According to the method, a dynamic weight adjustment mechanism and a personalized recommendation algorithm based on the knowledge graph are designed, customized diagnosis results and treatment suggestions can be provided for each patient, the importance of various data can be dynamically adjusted according to unique features of each patient, and a real personalized diagnosis and treatment scheme is generated in combination with latest medical knowledge.

Description

Technical Field

[0001] The present invention relates to the field of medical diagnosis technology, and in particular to an intelligent medical diagnosis and decision support method using multi-source data. Background Art

[0002] In the development of medical diagnosis and decision support systems, with the continuous enrichment of medical data and the advancement of technology, a variety of different methods have emerged. Each method has played an important role in its specific historical context, but with the increase in the diversity and complexity of medical data, the limitations of traditional methods have gradually become apparent.

[0003] In the development of medical diagnosis and decision support systems, the following methods are mainly relied on: (1) Single data source diagnosis system, which is the most traditional method and mainly relies on a single type of medical data for diagnosis, relying only on the doctor's clinical examination results or a specific examination (such as X-ray) for diagnosis. (2) Rule-based decision support system, which provides diagnostic suggestions based on preset medical rules and guidelines, such as using IF-THEN rule system to match predefined disease patterns according to symptoms. (3) Statistical methods, using statistical models to analyze patient data, find disease-related patterns, and use logistic regression models to predict the risk of certain diseases. (4) Early machine learning methods, using simple machine learning algorithms to process medical data, such as using decision trees or support vector machines (SVM) for classification diagnosis. These decision-making methods do not fully utilize information, lack flexibility, have low personalization, limited processing capabilities, lack integration, lack decision support, and lack real-time performance.

[0004] In summary, traditional medical diagnosis systems face many challenges in processing complex, dynamic, and multimodal medical data. Therefore, a more intelligent and integrated diagnosis and decision support system is needed to cope with the complexity of modern medical data and achieve more accurate and personalized diagnosis and treatment plans. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an intelligent medical diagnosis and decision support method using multi-source data, comprising the following steps: S1. Collecting medical multi-source data; S2. Preprocessing the collected data to obtain standardized data; S3. Performing multimodal processing on the standardized data to obtain image features, text features, structured features and fusion features; S4. Constructing medical diagnosis models of different modalities; S5. Generate corresponding personalized diagnosis results and generate treatment plans based on the medical diagnosis model through a dynamic weight adjustment mechanism.

[0006] Furthermore, the medical multi-source data in step S1 specifically includes: patient basic information, clinical data, laboratory test data, imaging data, wearable device data, and final treatment results; wherein the patient basic information includes: patient ID, age, and gender; the clinical data includes: patient symptoms, doctor's diagnosis information, and treatment plan.

[0007] Furthermore, the S2 step specifically includes the following sub-steps: S21. Identify and classify the collected medical multi-source data into integer or floating-point data, time series data, text data, and image data; S22. Preprocess the classified data respectively to obtain standardized data; wherein, the S22 step specifically includes the following sub-steps: when the data is integer or floating-point data, fill in the missing values ​​in the data and detect and process the abnormal data therein; when the data is time series data, align the time series data by resampling and interpolation, and when the data from different sources cannot be aligned, fill in the data by interpolation; when the data is text data, delete unnecessary characters by regular expressions and unify the data; when the data is image data, adjust the image size and normalize the pixel values.

[0008] Furthermore, the S3 step specifically includes the following sub-steps: S31. Process the image data using ViT to obtain image features; S32. Perform text data processing on the text data to obtain text features; S33. Process integer or floating-point data and time series data through a fully connected layer to obtain structured features; S34. Map the image features, text features, and structured features to the same feature space to obtain three aligned features; S35. Use the Transformer architecture to fuse the three aligned features.

[0009] Furthermore, the step S35 specifically includes the following sub-steps:

[0010] S351. Image feature F img ∈R d , text features F text ∈R d 、Structural features F structured ∈R dp Concatenate into one input matrix: F input ∈R 3*d , where each row represents the data features of a modality, and d is the feature dimension;

[0011] S352. For each input feature F i , through the linear mapping matrix W Q , W K and W V To generate the corresponding Q, K, V: Q = Finput W Q , K=F input W K , Q=F input W V ; where Q, K, V∈R 3*d , d k It is the dimension of query, key and value, Q, K and V represent query, key and value respectively;

[0012] S353. Calculate the similarity between each modal feature and then perform weighted summation;

[0013] S354. Divide Q, K, and V into multiple heads, each head calculates attention independently, and concatenates the outputs of each head together: MultiHead(Q,K,V) = Concat(Z1,Z2,…,Z H )W O ; Among them, Z h is the self-attention output of each head, W O ∈R Hdk×d The weight matrix used for linear transformation is used to map the concatenated result back to the original dimension;

[0014] S355. Stabilize the error through residual connection and layer normalization, and further extract features through feedforward neural network to output fusion feature Z final .

[0015] Furthermore, the step S353 specifically includes the following sub-steps:

[0016] S3531. Calculate the attention weight, calculate the similarity between the query Q and the key K through the dot product, and scale the result and perform the Softmax operation to obtain the attention weight matrix A: Among them, QKT is the dot product of query and key, indicating the similarity between modal features, is the scaling factor, A∈R 3*3 is the attention weight matrix, which represents the weights between different modal features;

[0017] S3532. Weighted sum value: Use the attention weight matrix A to perform weighted summation on the value V to obtain the fused feature representation Z: Z = AV, where It is the fused feature representation.

[0018] Furthermore, the dynamic weight adjustment mechanism in step S5 specifically includes the following steps: X1. Fine-tuning the medical diagnosis model based on patient age, symptom severity, medical history complexity, time sensitivity, disease complexity, and laboratory test results; X2. Adjusting the data type coefficient according to the data type of the adjustment function; X3. Calculating the final weight adjustment formula based on the adjusted data type coefficients of each type of data; X4. Obtaining the final dynamic adjustment formula based on the final weights of each type of data.

[0019] Further, the adjustment function specifically includes: f_A(A)=1+x1*max(0,An); f_S(S)=1+x1*S; f_H(H)=1+x2*H; f_T(T)=1+x3*T; f_C(C)=1+x4*C; f_L(L)=1+x5*L; wherein A represents the patient's age, S represents the severity of symptoms, H represents the complexity of medical history, T represents time sensitivity, C represents the complexity of the disease, and L represents the laboratory test results;

[0020] The adjustment data type coefficient specifically includes: α_text=f_A(A)*f_S(S)*f_H(H)*f_L(L); α_image=f_S(S)*f_T(T)*f_C(C); α_structured=f_A(A)*f_C(C)*f_L(L); In the above formula, α_text represents the text data type adjustment coefficient, α_image represents the image data type adjustment coefficient, and α_structured represents the structured data type adjustment coefficient.

[0021] Furthermore, the X3 step specifically includes the following sub-steps:

[0022] X31. Calculate the final weight of each data type, adding an artificial adjustment parameter r, which defaults to 1; X32. Calculate the normalization factor; X33. Divide the final weight of each data type by the normalization factor to obtain the final weight of each data type;

[0023] Among them, the final weight adjustment formula is: w_i=(wi0*αi*ri) / (Σ_j(w_j*αj*rj));

[0024] The calculation formula for the final weight of each data type is:

[0025] w_text=(w_text0*f_A(A)*f_S(S)*f_H(H)*f_L(L)*rtext) / Z;

[0026] w_image=(w_image0*f_S(S)*f_T(T)*f_C(C)*rimage) / Z;

[0027] w_structured=(w_structured0*f_A(A)*f_C(C)*f_L(L)*rstructured) / Z;

[0028] Where Z is the normalization factor:

[0029] Z=w_text0*f_A(A)*f_S(S)*f_H(H)*f_L(L)*rtext+w_image0*f_S(S)*f_T(T)*

[0030] f_C(C)*rimage+w_structured0*f_A(A)*f_C(C)*f_L(L)*rstructured;

[0031] In the above formula, i, j represent the index of the data type, i, j∈{text, image, structured}; w_i represents the final adjusted weight of data type i; w i0 represents the initial weight of data type i; r i represents the manually adjusted weight of data type i; α i Indicates the total adjustment factor for data type i.

[0032] Furthermore, the final dynamic adjustment formula in step X4 is: y = w_text*x_text+

[0033] w_image*x_image+w_structured*x_structured.

[0034] The present invention provides an intelligent medical diagnosis and decision support method using multi-source data, which has the following beneficial effects:

[0035] (1) The present invention adopts multi-source data acquisition and fusion technology, combining various types of medical data such as electronic health records, medical images, and genomic data; it can achieve a more comprehensive and accurate diagnosis; it can simultaneously consider the patient's clinical symptoms, imaging characteristics, genetic characteristics and other information, greatly reducing the diagnostic bias and omissions that may be caused by a single data source.

[0036] (2) The present invention has designed a dynamic weight adjustment mechanism and a personalized recommendation algorithm based on knowledge graph, which can provide each patient with customized diagnosis results and treatment suggestions. It will dynamically adjust the importance of various types of data according to the unique characteristics of each patient, and combine the latest medical knowledge to generate a truly personalized diagnosis and treatment plan.

[0037] (3) The present invention combines deep learning and knowledge graph technology: it can effectively learn and adapt to new situations even with relatively little labeled data; this solves the problem of difficulty in obtaining labeled data in the medical field, allowing the system to be applied more quickly to new disease types or medical environments.

[0038] (4) The present invention proposes a new multimodal feature fusion method that can more effectively capture the complex relationships between different types of medical data, which enables the system to understand the patient's condition from multiple perspectives and provide more comprehensive and accurate diagnosis and treatment recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0040] Figure 1 The present invention provides a flow chart of the method. DETAILED DESCRIPTION

[0041] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0042] The following is a detailed description of the implementation method of the present invention in conjunction with the accompanying drawings. Only some embodiments are described, not all embodiments. For the purpose of clarity, representations and descriptions that are not related to the present invention are omitted in the drawings and descriptions.

[0043] In order to have a clearer understanding of the technical features, purposes and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below. Obviously, the implementation cases described are part of the embodiments of the present invention, not all of the embodiments, and cannot be understood as limiting the scope of the implementation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0044] like Figure 1As shown, the present invention provides an intelligent medical diagnosis and decision support method using multi-source data, including the following steps: S1. Collecting medical multi-source data; S2. Preprocessing the collected data to obtain standardized data; S3. Performing multimodal processing on the standardized data to obtain image features, text features, structured features and fusion features; S4. Constructing medical diagnosis models of different modalities; S5. Generate corresponding personalized diagnosis results and generate treatment plans based on the medical diagnosis model through a dynamic weight adjustment mechanism.

[0045] The medical multi-source data in step S1 specifically includes: basic patient information, clinical data, laboratory test data, imaging data, wearable device data, and final treatment results; wherein, the basic patient information includes: patient ID, age, and gender; the clinical data includes: patient symptoms, doctor's diagnosis information, and treatment plan.

[0046] The S2 step specifically includes the following sub-steps:

[0047] S21. Identify and classify the collected medical multi-source data into integer or floating point data, time series data, text data, and image data.

[0048] S22. Preprocess the classified data to obtain standardized data;

[0049] Wherein, step S22 specifically includes the following sub-steps:

[0050] When the data is integer or floating point data, the missing values ​​in the data are filled and the abnormal data are detected and processed. The re-interpolation method is used to handle missing values ​​and the isolation forest algorithm is used to detect and process outliers.

[0051] When the data is time series data, it is aligned through resampling and interpolation. When data from different sources cannot be aligned, it is completed through interpolation.

[0052] When the data is text data, regular expressions are used to delete unnecessary characters and unify the data. Regular expressions are used to delete unnecessary characters and medical vocabularies (such as UMLS and SNOMED CT) are used to unify the terms. Using a specific medical tokenizer (such as SciSpaCy), drug names, diseases, and symptoms can be accurately tokenized, and the tokenizer using BERT will further segment the terms into sub-word units (tokens). Medical text may contain sensitive data such as the patient's name and ID number. Regular expressions can be used to match sensitive information such as phone numbers and addresses, or named entity recognition (NER) can be used to automatically identify and replace, generate the Attention Mask required by the BERT model, and convert the encoding results into the Tensor format required by the BERT model.

[0053] When the data is image data, the image size is adjusted and the pixel values ​​are normalized.

[0054] The S3 step specifically includes the following sub-steps:

[0055] S31. Use Visual Transformer (ViT) to process the image data to obtain image features.

[0056] S32. Use the BERT model to process text data and obtain text features.

[0057] S33. Integer or floating-point data and time series data are processed through a fully connected layer to obtain structured features.

[0058] S34. Use a shared multi-layer perceptron (MLP) or a symmetric network structure to map the image features, text features, and structured features to the same feature space to obtain the three aligned features.

[0059] S35. The three aligned features are fused using the Transformer architecture. When multimodal feature fusion is performed based on the Transformer architecture, the generation of fused features mainly depends on the self-attention mechanism. The self-attention mechanism calculates the similarity weights between the features of each modality and performs weighted summation of information from different modalities, thereby achieving deep interaction between modalities.

[0060] Step S35 specifically includes the following sub-steps:

[0061] S351. Image feature F img ∈R d , text features F text ∈R d 、Structural features F structured∈R dp Concatenate into one input matrix: F input ∈R 3*d , where each row represents the data features of a modality and d is the feature dimension.

[0062] S352. For each input feature F i , through the linear mapping matrix W Q , W K and W V To generate the corresponding Q, K, V: Q = F input W Q , K=F input W K , Q=F input W V ;

[0063] Among them, Q,K,V∈R 3*d , d k These are the query, key, and value dimensions. Q, K, and V represent query, key, and value, respectively.

[0064] S353. Calculate the similarity between each modal feature and then perform weighted summation.

[0065] S354. Divide Q, K, and V into multiple heads, each head calculates attention independently, and concatenates the outputs of each head together: MultiHead(Q,K,V) = Concat(Z1,Z2,…,Z H )W O ;

[0066] Among them, Z h is the self-attention output of each head, W O ∈R Hdk×d The weight matrix used for the linear transformation to map the concatenated result back to the original dimension.

[0067] S355. Stabilize the error through residual connection and layer normalization, and further extract features through feedforward neural network to output fusion feature Z final .

[0068] Step S353 specifically includes the following sub-steps:

[0069] S3531. Calculate the attention weight, calculate the similarity between the query Q and the key K through the dot product, and scale the result and perform the Softmax operation to obtain the attention weight matrix A:

[0070] Among them, QKT is the dot product of query and key, indicating the similarity between modal features, is the scaling factor, A∈R 3*3is the attention weight matrix, which represents the weights between features of different modalities.

[0071] S3532. Weighted sum value: Use the attention weight matrix A to perform weighted summation on the value V to obtain the fused feature representation Z: Z = AV, where It is the fused feature representation.

[0072] Transformer is used to train image features, text features, structural features and fusion features to build a medical diagnosis model that can dynamically adjust the weights of different modalities. The model first integrates the features of each modality through a feature processor, which contains a fully connected layer, layer normalization and activation function to standardize and enhance the representation ability of the features. The processed features are input into the enhanced Transformer encoder, which consists of multiple layers of Transformers, each of which contains a multi-head self-attention mechanism and a feedforward neural network to capture the long-distance dependencies between features and further refine the representation. The multi-head attention mechanism improves the expressiveness of the model in this process and achieves deep fusion of features of different modalities.

[0073] Based on the medical diagnosis model, the dynamic weight adjustment mechanism generates corresponding personalized diagnosis results and treatment plans. It can adaptively adjust various types of data according to the characteristics of different patients to obtain personalized diagnosis results. This dynamic weight adjustment mechanism dynamically adjusts the weights of different data types (text, images, structured data) according to the characteristics of the patient. It considers multiple factors, such as the patient's age, symptom severity, previous medical history, etc., to adjust the weights. The weight adjustment is gradual and is generally divided into two stages.

[0074] The dynamic weight adjustment mechanism in step S5 specifically includes the following steps:

[0075] X1. Fine-tune the medical diagnosis model based on the patient's age, symptom severity, medical history complexity, time sensitivity, disease complexity, and laboratory test results. Age adjustment is to set an age n as the key threshold. For every additional year, the adjustment coefficient increases by x1. Symptom severity, medical history complexity, time sensitivity, disease complexity, laboratory test results, etc. are graded into different levels. For each additional level, the weight increases by x2, x3, x4, x5, and x6 respectively.

[0076] X2. Adjust the data type coefficient according to the data type of the adjustment function. For example, the text data type will be affected by age, symptom severity, medical history complexity, and laboratory tests, the image data type will be affected by symptom severity, time sensitivity, and disease complexity, and the structured data will be affected by age, disease complexity, and laboratory test results.

[0077] X3. Calculate the final weight adjustment formula based on the adjusted data type coefficients of each data type.

[0078] X4. Obtain the final dynamic adjustment formula based on the final weights of various types of data.

[0079] Furthermore, the adjustment function specifically includes: f_A(A)=1+x1*max(0,An); f_S(S)=1+x1*S; f_H(H)=1+x2*H; f_T(T)=1+x3*T; f_C(C)=1+x4*C; f_L(L)=1+x5*L; wherein A represents the patient's age, S represents the severity of symptoms, H represents the complexity of medical history, T represents time sensitivity, C represents the complexity of the disease, and L represents the laboratory test results.

[0080] The adjustment data type coefficients specifically include: α_text=f_A(A)*f_S(S)*f_H(H)*f_L(L); α_image=

[0081] f_S(S)*f_T(T)*f_C(C);α_structured=f_A(A)*f_C(C)*f_L(L);In the above formula, α_text represents the text data type adjustment coefficient, α_image represents the image data type adjustment coefficient, and α_structured represents the structured data type adjustment coefficient.

[0082] Step X3 specifically includes the following sub-steps:

[0083] X31. Calculate the final weight of each data type and add an artificial adjustment parameter r, which defaults to 1.

[0084] X32. Calculate the normalization factor.

[0085] X33. Divide the final weight of each data type by the normalization factor to obtain the final weight of each data type.

[0086] Among them, the final weight adjustment formula is: w_i=(wi0*αi*ri) / (Σ_j(w_j*αj*rj));

[0087] The calculation formula for the final weight of each data type is:

[0088] w_text=(w_text0*f_A(A)*f_S(S)*f_H(H)*f_L(L)*rtext) / Z;

[0089] w_image=(w_image0*f_S(S)*f_T(T)*f_C(C)*rimage) / Z;

[0090] w_structured=(w_structured0*f_A(A)*f_C(C)*f_L(L)*rstructured) / Z;

[0091] Where Z is the normalization factor:

[0092] Z=w_text0*f_A(A)*f_S(S)*f_H(H)*f_L(L)*rtext+w_image0*f_S(S)*f_T(T)*

[0093] f_C(C)*rimage+w_structured0*f_A(A)*f_C(C)*f_L(L)*rstructured;

[0094] In the above formula, i, j represent the index of the data type, i, j∈{text, image, structured}; w_i represents the final adjusted weight of data type i; w i0 represents the initial weight of data type i; r i represents the manually adjusted weight of data type i; α i Indicates the total adjustment factor for data type i.

[0095] The final dynamic adjustment formula in step X4 is:

[0096] y=w_text*x_text+w_image*x_image+w_structured*x_structured.

[0097] In the model inference stage, the dynamically adjusted weights are passed into the model, so that the model can adjust different data sources according to the weights. A personalized diagnosis result is obtained. A new section is added to the generated diagnosis report to display the final weights of various types of data. The transparency of the diagnosis process is increased, allowing doctors to understand how the diagnosis results are obtained, and doctors or experts can manually modify the weights based on manual adjustment parameters.

[0098] By combining medical knowledge graphs, personalized treatment plans can be generated.

[0099] The search rule design of the knowledge graph is mainly divided into: 1. By constructing a medical knowledge graph, a basic medical knowledge graph is obtained. 2. Load the patient's information and use three search methods: direct search based on diagnosis, similarity search based on symptoms, and personalized search based on patient characteristics to obtain multiple treatment plans and disease information. Use attribute filtering-based queries to perform taboo search checks, and filter treatment plans for taboo search checks to ensure treatment safety. Use time attribute queries to perform temporal search rules to ensure the timing of treatment. 3. Integrate the results, and use a weighted scoring method to combine the scores of different dimensions (such as diagnosis, symptoms, patient characteristics, and time) into a comprehensive score. By adjusting the weights of different dimensions, calculate the overall score of each treatment plan. Sort by the overall score to obtain the best treatment plan and disease information.

[0100] The core scoring function of the weighted scoring method, the weight value of the core evaluation function can be adjusted according to the feedback adjustment rule in the fifth step: F(x|D,S,P,T)=∑((w i *AF(x))*S i (x)); where x∈X: candidate treatment options; Diagnostic vector space; S∈{0,1} n :Symptom feature space; Patient feature space; T∈R t : temporal weight space; w i ∈[0,1]: weight coefficient, ∑w i =1;S i (x): Scoring function for each dimension; AF(x): Adjustment factor, doctors or experts can adjust ∑AF(x)=10 according to actual conditions.

[0101] The scene adaptive weight can adjust the benchmark weight set according to different scenarios. Here are two examples of scenarios: W(s) = f(s, W0); where: s∈S: scene type; W0: benchmark weight set; f: scene adjustment function.

[0102] By collecting the final treatment effect and applying the feedback adjustment rule to update the weight value of the core evaluation function, a better evaluation rule is obtained:

[0103] Feedback adjustment rule: Calculate the error based on the difference between the final treatment effect and the predicted effect: Among them: y(t): actual treatment effect; Prediction rating.

[0104] Use the performance metric function L(W) to measure the error of each dimension, so as to optimize each weight: L(W) = ∑λ i *E i2 (t); where: λ i : Error weight coefficient; E i (t): Error in each dimension.

[0105] The weight update rule updates the weights based on the error calculation so that the importance of each dimension is adjusted in the next decision: W(t+1)=N(W'(t+1)); where: N(): normalization function; η: learning rate.

[0106] By taking the derivative of the performance metric function L(W), we calculate the impact of each dimension error on the total error, and thus obtain the gradient used to adjust the weight: in: Partial derivative of the scoring function with respect to the weights.

[0107] Integrate all information into a comprehensive diagnostic report to help doctors make informed medical decisions.

[0108] The risk of many diseases increases with age, especially after age 50. Older patients tend to have more complex health conditions and require more attention to structured data (such as various physiological indicators). For each additional year of age, the adjustment factor increases by 0.5%, which is a gradual, gentle adjustment. The threshold of 50 years is based on the age at which the risk of many chronic diseases begins to increase significantly. The more severe the symptoms, the more comprehensive the diagnostic approach, including greater reliance on imaging data and detailed clinical descriptions. Severity is usually expressed on a scale of 1-5 or 1-10, and the weight increases by 10% for each additional level. This adjustment reflects that in severe cases, doctors will more carefully review all available diagnostic information. Complex medical history often means that more attention needs to be paid to text data, such as detailed medical records. For each additional level of complexity, the adjustment factor increases by 5%, which reflects the importance of medical history in diagnosis, but does not overwhelm other factors. Complexity can be assessed based on the number, type, and interaction of previous diseases. There are also weights adjusted based on time sensitivity, disease complexity, laboratory test results, etc., and then fine-tuned based on expert opinion, and finally standardized to ensure that the weights sum to 1.

[0109] The present invention adopts multi-source data collection and fusion technology, combining various types of medical data such as electronic health records, medical images, and genomic data; it can achieve a more comprehensive and accurate diagnosis; it can simultaneously consider the patient's clinical symptoms, imaging characteristics, genetic characteristics and other information, greatly reducing the diagnostic bias and omissions that may be caused by a single data source.

[0110] The present invention has designed a dynamic weight adjustment mechanism and a personalized recommendation algorithm based on knowledge graph, which can provide each patient with customized diagnosis results and treatment suggestions. It will dynamically adjust the importance of various types of data according to the unique characteristics of each patient, and combine the latest medical knowledge to generate a truly personalized diagnosis and treatment plan.

[0111] The present invention combines deep learning and knowledge graph technology: it can effectively learn and adapt to new situations even with relatively little labeled data; this solves the problem of difficulty in obtaining labeled data in the medical field, allowing the system to be applied to new disease types or medical environments more quickly.

[0112] The present invention proposes a new multimodal feature fusion method that can more effectively capture the complex relationships between different types of medical data, which enables the system to understand the patient's condition from multiple perspectives and provide more comprehensive and accurate diagnosis and treatment recommendations.

[0113] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. An intelligent medical diagnosis and decision support method using multi-source data, characterized in that: The following steps are involved: S1. Collect medical multi-source data; S2. preprocess the collected data to obtain standardized data; S3. Perform multimodal processing on the standardized data to obtain image features, text features, structural features and fusion features; S4. Construct medical diagnosis models of different modalities; S5. Generate corresponding personalized diagnosis results and treatment plans based on the medical diagnosis model through a dynamic weight adjustment mechanism.

2. The intelligent medical diagnosis and decision support method using multi-source data according to claim 1, characterized in that: The medical multi-source data in step S1 specifically includes: basic patient information, clinical data, laboratory test data, imaging data, wearable device data, and final treatment results; wherein, the basic patient information includes: patient ID, age, and gender; the clinical data includes: patient symptoms, doctor's diagnosis information, and treatment plan.

3. The intelligent medical diagnosis and decision support method using multi-source data according to claim 1, characterized in that: The S2 step specifically includes the following sub-steps: S21. Identify and classify the collected medical multi-source data into integer or floating point data, time series data, text data, and image data; S22. Preprocess the classified data to obtain standardized data; Wherein, step S22 specifically includes the following sub-steps: When the data is integer or floating point data, fill in the missing values ​​in the data and detect and process the abnormal data; When the data is time series data, it is aligned through resampling and interpolation. When data from different sources cannot be aligned, it is supplemented through interpolation. When the data is text data, unnecessary characters are deleted through regular expressions and the data is unified; When the data is image data, the image size is adjusted and the pixel values ​​are normalized.

4. The intelligent medical diagnosis and decision support method using multi-source data according to claim 3, characterized in that: The S3 step specifically includes the following sub-steps: S31. Process the image data using ViT to obtain image features; S32. Process the text data to obtain text features; S33. Process the integer or floating point data and time series data through a fully connected layer to obtain structured features; S34. Mapping the image features, text features, and structured features to the same feature space to obtain three aligned features; S35. Use the Transformer architecture to fuse the three aligned features.

5. The intelligent medical diagnosis and decision support method using multi-source data according to claim 4, characterized in that: The S35 step specifically includes the following sub-steps: S351. Image feature F img ∈R d , text features F text ∈R d 、Structural features F structured ∈R d p is concatenated into an input matrix: F input ∈R 3*d , where each row represents the data features of a modality, and d is the feature dimension; S352. For each input feature F i , through the linear mapping matrix W Q , W K and W V To generate the corresponding Q, K, V: Q = F input W Q , K=F input W K , Q=F input W V ; where Q, K, V∈R 3*d , d k It is the dimension of query, key and value, Q, K and V represent query, key and value respectively; S353. Calculate the similarity between each modal feature and then perform weighted summation; S354. Divide Q, K, and V into multiple heads, each head calculates attention independently, and concatenates the outputs of each head together: MultiHead(Q,K,V) = Concat(Z1,Z2,…,Z H )W O ; Among them, Z h is the self-attention output of each head, W O ∈R Hdk×d The weight matrix used for linear transformation is used to map the concatenated result back to the original dimension; S355. Stabilize the error through residual connection and layer normalization, and further extract features through feedforward neural network to output fusion feature Z final .

6. The intelligent medical diagnosis and decision support method using multi-source data according to claim 5, characterized in that: The step S353 specifically includes the following sub-steps: S3531. Calculate the attention weight, calculate the similarity between the query Q and the key K through the dot product, and scale the result and perform the Softmax operation to obtain the attention weight matrix A: Among them, QK T is the dot product of the query and the key, indicating the similarity between modal features, is the scaling factor, A∈R 3*3 is the attention weight matrix, which represents the weights between different modal features; S3532. Weighted sum value: Use the attention weight matrix A to perform weighted summation on the value V to obtain the fused feature representation Z: Z = AV. in, It is the fused feature representation.

7. The intelligent medical diagnosis and decision support method using multi-source data according to claim 1, characterized in that: The dynamic weight adjustment mechanism in step S5 specifically includes the following steps: X1. Fine-tune the medical diagnosis model based on patient age, symptom severity, medical history complexity, time sensitivity, disease complexity, and laboratory test results; X2. Adjust the data type coefficient according to the data type of the adjustment function; X3. Calculate the final weight adjustment formula based on the adjustment data type coefficients of each data type; X4. Obtain the final dynamic adjustment formula based on the final weights of various types of data.

8. The intelligent medical diagnosis and decision support method using multi-source data according to claim 7, characterized in that: The adjustment function specifically includes: f_A(A)=1+x1*max(0,An); f_S(S)=1+x1*S; f_H(H)=1+x2*H; f_T(T)=1+x3*T; f_C(C)=1+x4*C; f_L(L)=1+x5*L; In the formula, A represents the patient's age, S represents the severity of symptoms, H represents the complexity of medical history, T represents time sensitivity, C represents the complexity of the disease, and L represents the laboratory test results; The adjustment data type coefficient specifically includes: α_text=f_A(A)*f_S(S)*f_H(H)*f_L(L); α_image=f_S(S)*f_T(T)*f_C(C); α_structured=f_A(A)*f_C(C)*f_L(L); In the above formula, α_text represents the text data type adjustment coefficient, α_image represents the image data type adjustment coefficient, and α_structured represents the structured data type adjustment coefficient.

9. The intelligent medical diagnosis and decision support method using multi-source data according to claim 8, characterized in that: The X3 step specifically includes the following sub-steps: X31. Calculate the final weight of each data type, adding an artificial adjustment parameter r, which defaults to 1; X32. Calculate the normalization factor; X33. Divide the final weight of each data type by the normalization factor to obtain the final weight of each data type; The final weight adjustment formula is: w_i=(w i0 *α i *r i ) / (Σ_j(w_j*αj*r j )); The calculation formula for the final weight of each data type is: w_text=(w_text0*f_A(A)*f_S(S)*f_H(H)*f_L(L)*r text ) / Z; w_image=(w_image0*f_S(S)*f_T(T)*f_C(C)*r image ) / Z; w_structured=(w_structured0*f_A(A)*f_C(C)*f_L(L)*rstructured) / Z; Where Z is the normalization factor: Z=w_text0*f_A(A)*f_S(S)*f_H(H)*f_L(L)*r text +w_image0*f_S(S)*f_T(T)*f_C(C)*r image +w_structured0*f_A(A)*f_C(C)*f_L(L)*r structured ; In the above formula, i, j represent the index of the data type, i, j∈{text, image, structured}; w_i represents the final adjusted weight of data type i; w i0 represents the initial weight of data type i; r i represents the manually adjusted weight of data type i; α i Indicates the total adjustment factor for data type i.

10. The intelligent medical diagnosis and decision support method using multi-source data according to claim 9, characterized in that: The final dynamic adjustment formula in the X4 step is: y=w_text*x_text+w_image*x_image+w_structured*x_structured.

Images