Construction and application method of multi-mode urinary calculus intelligent diagnosis and treatment model

Abstract

The invention provides a construction and application method of a multi-modal urinary calculus intelligent diagnosis and treatment model, which is an intelligent diagnosis and treatment system based on a large model and a multi-modal data fusion technology, and is used for improving the diagnosis accuracy of urinary calculus diseases, optimizing the treatment effect and improving the diagnosis and treatment efficiency. A model is constructed through a multi-modal data set, intelligent diagnosis of diseases of the urinary calculus is put into the model, clinicians are assisted to rapidly give a diagnosis scheme of the urinary calculus, and the diagnosis and treatment efficiency of the urinary calculus is greatly improved. The method has the technical advantages that diagnosis accuracy is improved, personalized treatment is optimized, automatic diagnosis and treatment processes and medical resources are optimized, and diagnosis and treatment efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical technology, and particularly to a construction of a multi-modal urolithiasis intelligent diagnosis and treatment model, a multi-modal urolithiasis intelligent diagnosis and treatment method, a multi-modal urolithiasis intelligent diagnosis and treatment system, an electronic device, and a computer-readable storage medium. Background Art

[0002] Urolithiasis is a common urinary system disease, and its global incidence rate has been continuously rising. Experts believe that its pathogenesis is complex, and the risk factors include metabolic disorders, infections, and genetics. Common diagnosis and treatment methods include:

[0003] Imaging examination: To confirm the location and size of the stone through CT and ultrasound;

[0004] Laboratory examination: To evaluate the metabolic characteristics through urine analysis and blood tests;

[0005] Medical record analysis: Combining the patient's historical records and symptoms to further infer the cause and treatment plan.

[0006] However, recent studies have shown that there is a high correlation between the formation of urolithiasis and individual metabolic characteristics. Existing clinical research data indicate that calcium oxalate stones account for more than 70% of urolithiasis, and uric acid stones and struvite stones account for about 20% and 10% respectively. However, the treatment plans for these types of stones often need to be determined through comprehensive analysis of multi-modal information in diagnosis and treatment. Therefore, the following problems exist in these traditional methods in practice:

[0007] Weak data fusion ability: Multiple modal data (imaging, laboratory, text) are independent of each other, and the comprehensive analysis ability is insufficient, resulting in incomplete diagnostic results;

[0008] Insufficient diagnostic accuracy: Doctors rely on personal experience for diagnosis, and there are cases of misdiagnosis or missed diagnosis;

[0009] Lack of personalization: General guidelines cannot explain individual differences, resulting in less than ideal treatment effects;

[0010] Limited primary medical resources: Lack of experienced doctors and sufficient equipment, affecting the diagnosis and treatment effects.

[0011] In addition, although in recent years, large model technologies based on Transformer have made remarkable progress in the fields of natural language processing (NLP), image processing (CV), and multi-modal data fusion. However, there is currently no mature and complete intelligent diagnosis and treatment system that applies these technologies to the diagnosis and treatment of urinary calculi. In the traditional diagnosis and treatment methods of urinary calculi, generally, manual diagnosis is mainly carried out by medical staff. Therefore, there is a lack of intelligent diagnosis and treatment assistance methods, and it is impossible to assist doctors in quickly diagnosing urinary calculi and giving corresponding diagnosis and treatment suggestions. As a result, the clinical diagnosis and treatment efficiency of urinary calculi is low. Summary of the Invention

[0012] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions:

[0013] On the one hand, a method for constructing a multi-modal intelligent diagnosis and treatment model for urinary calculi is provided, and the method includes:

[0014] Collect historical medical examination data of a number of urinary calculi patients, including imaging data, text data, and laboratory data related to urinary calculi;

[0015] Extract multi-modal data features from the medical examination data, including: respectively extract corresponding morphological features of urinary calculi, key diagnostic information text features, and laboratory data feature values from the imaging data, text data, and laboratory data;

[0016] Perform feature fusion on the multi-modal data features to obtain multi-modal features;

[0017] Perform feature annotation on the multi-modal features, and the annotation information includes: the symptom level corresponding to the multi-modal features and personalized treatment suggestions recommended according to the multi-modal features;

[0018] After the annotation is completed, a feature set composed of the multi-modal features of a number of urinary calculi patients is obtained;

[0019] Use the feature set as a training set and input it into a preset machine learning model for feature training and learning to obtain the multi-modal intelligent diagnosis and treatment model for urinary calculi;

[0020] Use an independent clinical data set to verify the recognition performance of the multi-modal intelligent diagnosis and treatment model for urinary calculi:

[0021] If the verification is passed, deploy the multi-modal intelligent diagnosis and treatment model for urinary calculi to the background server and put it into application;

[0022] If the verification fails, repeat the above training steps to reconstruct the multi-modal intelligent diagnosis and treatment model for urinary calculi.

[0023] Preferably, extracting the multi-modal data features from the medical examination data includes:

[0024] Using a ResNet or ViT model to extract the morphological features of urinary calculi from the CT and / or ultrasound images of urinary calculi in patients with urinary calculi. The morphological features of urinary calculi include the following features: the location and size of urinary calculi, the density of urinary calculi, and the shape and boundary features of urinary calculi;

[0025] And,

[0026] Using an NLP model or an attention mechanism model to parse the electronic medical record files of patients with urinary calculi and extract the key diagnostic information text features therein. The key diagnostic information text features include diagnostic keywords related to urinary calculi and the association features between the diagnostic keywords and other modal data features;

[0027] And,

[0028] Using an MLP mechanism to extract the laboratory data feature values from the clinical laboratory data of patients with urinary calculi. The laboratory data feature values include at least one of the following features: the pH / uric acid concentration value of urine or blood.

[0029] Preferably, the annotation information further includes:

[0030] Follow-up recommendations recommended according to the multi-modal features of patients with urinary calculi.

[0031] On the other hand, a method for applying a multi-modal intelligent diagnosis and treatment model for urinary calculi is provided. The method includes:

[0032] Collecting the clinical medical examination data of patients with urinary calculi;

[0033] Inputting the clinical medical examination data into a pre-deployed multi-modal intelligent diagnosis and treatment model for urinary calculi, identifying the multi-modal data features and their symptom levels in the clinical medical examination data through the multi-modal intelligent diagnosis and treatment model for urinary calculi, and outputting personalized treatment recommendations matching the multi-modal data features at this symptom level;

[0034] Binding the multi-modal data features with the corresponding personalized treatment recommendations and writing them into a preset diagnostic report to generate a diagnostic report for urinary calculi of this patient with urinary calculi;

[0035] Recommending the diagnostic report for urinary calculi to the medical staff side and / or the patient side.

[0036] Preferably, collecting the clinical medical examination data of patients with urinary calculi includes:

[0037] Build the electronic medical record files of urolithiasis patients in the HIS system in advance;

[0038] Collect the clinical medical examination data of urolithiasis patients related to urolithiasis, upload it to the HIS system and record it in the electronic medical record files of the urolithiasis patients.

[0039] Preferably, after the multi-modal urolithiasis intelligent diagnosis and treatment model identifies the multi-modal data features in the clinical medical examination data and outputs personalized treatment suggestions matching the multi-modal data features, it further includes:

[0040] Extract the urolithiasis diagnosis keywords in the multi-modal data features;

[0041] Build a medical guideline retrieval logic matching the urolithiasis diagnosis keywords;

[0042] Combine the urolithiasis diagnosis keywords and the medical guideline retrieval logic to build a large language model prompt and input it into the preset LLM large language model;

[0043] Through the LLM large language model, based on the large language model prompt, conduct a database search, and retrieve the medical guideline treatment strategies matching the urolithiasis diagnosis keywords from the medical database;

[0044] Output the medical guideline treatment strategies, bind them to the corresponding personalized treatment suggestions, and write them into the preset diagnosis report to generate the urolithiasis diagnosis and treatment report of the urolithiasis patient.

[0045] Preferably, after the multi-modal urolithiasis intelligent diagnosis and treatment model identifies the multi-modal data features in the clinical medical examination data and outputs personalized treatment suggestions matching the multi-modal data features, it further includes:

[0046] Extract the urolithiasis diagnosis keywords in the multi-modal data features;

[0047] Build a medical case retrieval logic matching the urolithiasis diagnosis keywords;

[0048] Combine the urolithiasis diagnosis keywords and the medical case retrieval logic to build a large language model prompt and input it into the preset LLM large language model;

[0049] Through the LLM large language model, based on the large language model prompt, conduct a database search, and retrieve the urolithiasis medical cases matching the urolithiasis diagnosis keywords from the medical case database;

[0050] Output the medical case of urinary calculus and bind it to the corresponding personalized treatment advice, and write it into a preset diagnostic report to generate a urinary calculus diagnosis and treatment report for the urinary calculus patient.

[0051] On the other hand, a multi-modal intelligent diagnosis and treatment system for urinary calculus is provided. The multi-modal intelligent diagnosis and treatment system for urinary calculus is used to implement the application method of the above-mentioned multi-modal intelligent diagnosis and treatment model for urinary calculus. The system includes:

[0052] A data collection module for collecting clinical medical examination data of urinary calculus patients;

[0053] An AI diagnosis and treatment module for inputting the clinical medical examination data into a pre-deployed multi-modal intelligent diagnosis and treatment model for urinary calculus, identifying multi-modal data features and their symptom levels in the clinical medical examination data through the multi-modal intelligent diagnosis and treatment model for urinary calculus, and outputting personalized treatment advice matching the multi-modal data features at the symptom level;

[0054] A report generation module for binding the multi-modal data features to the corresponding personalized treatment advice, and writing them into a preset diagnostic report to generate a urinary calculus diagnosis and treatment report for the urinary calculus patient;

[0055] A report recommendation module for recommending the urinary calculus diagnosis and treatment report to the medical staff side and / or the patient side.

[0056] On the other hand, an electronic device is provided. The electronic device includes: a processor; a memory, and a computer-readable instruction is stored on the memory. When the computer-readable instruction is executed by the processor, any one of the methods in the above-mentioned construction and application methods of the multi-modal intelligent diagnosis and treatment model for urinary calculus is realized.

[0057] On the other hand, a computer-readable storage medium is provided. At least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to realize any one of the methods in the above-mentioned construction and application methods of the multi-modal intelligent diagnosis and treatment model for urinary calculus.

[0058] The beneficial effects brought by the technical solutions provided in the embodiments of the present invention at least include:

[0059] The present invention proposes an intelligent diagnosis and treatment system based on a large model and multi-modal data fusion technology, which is used to improve the diagnostic accuracy of urinary calculus diseases, optimize the treatment effect and improve the diagnosis and treatment efficiency. By constructing a model with a multi-modal data set and putting it into the intelligent diagnosis of urinary calculus diseases, it assists clinicians to quickly give a diagnosis plan for urinary calculus, greatly improving the diagnosis and treatment efficiency of urinary calculus.

[0060] Adopting the present invention, the following technical advantages can be achieved:

[0061] 1. Improved diagnostic accuracy:

[0062] By integrating imaging, medical records, and laboratory data, comprehensive diagnostic conclusions are generated, significantly reducing the misdiagnosis rate and missed diagnosis rate.

[0063] Based on the characteristics of different types of stones, the system can accurately classify calcium oxalate stones, uric acid stones, and other types of stones.

[0064] 2. Optimized personalized treatment:

[0065] Using personalized recommendation algorithms, dynamically adjusted treatment plans are provided to reduce the risk of recurrence.

[0066] Based on the patient's urine pH value, metabolic characteristics, and medical history, the system generates precise personalized treatment recommendations, significantly optimizing the treatment effect.

[0067] 3. Automated diagnosis and treatment process:

[0068] The system automatically generates diagnostic reports and treatment recommendations and provides interpretable results.

[0069] The system can cover the full process services of diagnosis, generation of treatment recommendations, and postoperative follow-up management.

[0070] 4. Optimization of medical resources:

[0071] Provide high-quality auxiliary diagnosis and treatment services for primary medical institutions to improve the problem of uneven distribution of resources.

[0072] The system can significantly improve the diagnosis and treatment efficiency in the environment of primary hospitals with limited resources. Brief description of the drawings

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0074] Figure 1 It is a flowchart of the application method of a multimodal urolithiasis intelligent diagnosis and treatment model provided by an embodiment of the present invention;

[0075] Figure 2 It is a schematic diagram of the construction process of a multimodal urolithiasis intelligent diagnosis and treatment model provided by an embodiment of the present invention;

[0076] Figure 3 It is a block diagram of a multimodal urolithiasis intelligent diagnosis and treatment system provided by an embodiment of the present invention;

[0077] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0078] The following describes the technical solutions in the present invention with reference to the accompanying drawings.

[0079] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two.

[0080] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same. "Of", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same.

[0081] In the embodiments of the present invention, sometimes subscripts such as W1 may be miswritten as non-subscript forms such as W1. When the difference is not emphasized, the meanings they express are the same.

[0082] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0083] The embodiments of the present invention provide a method for constructing and applying a multimodal urolithiasis intelligent diagnosis and treatment model. This method can be implemented by an electronic device, which can be a terminal or a server. As Figure 1 shown in the flowchart of the method for constructing and applying a multimodal urolithiasis intelligent diagnosis and treatment model, the processing flow of this method can include the following steps:

[0084] S1. Collect clinical medical examination data of urolithiasis patients;

[0085] S2. Input the clinical medical examination data into a pre-deployed multimodal urolithiasis intelligent diagnosis and treatment model, identify the multimodal data features and their symptom levels in the clinical medical examination data through the multimodal urolithiasis intelligent diagnosis and treatment model, and output personalized treatment suggestions matching the multimodal data features at this symptom level;

[0086] S3. Bind the multi-modal data features with the corresponding personalized treatment suggestions and write them into a preset diagnostic report to generate a urolithiasis diagnosis and treatment report for the urolithiasis patient;

[0087] S4. Recommend the urolithiasis diagnosis and treatment report to the medical staff side and / or the patient side.

[0088] Regarding the construction and application in the multi-modal urolithiasis intelligent diagnosis and treatment model, please refer to the appendix Figure 2 and its subsequent construction method for understanding.

[0089] The present invention utilizes big data technology to construct a multi-modal urolithiasis intelligent diagnosis and treatment model for identifying and recommending strategies through feature engineering training and deploy it on the background server. Subsequently, it can be used to identify multi-modal data features in the clinical medical examination data of urolithiasis patients, that is, the intelligent diagnosis results of urolithiasis patients, and can recommend personalized treatment suggestions matching the features, and form an intelligent treatment plan for patients in combination with medical knowledge. It can quickly help doctors (or patients) understand the development status of their urolithiasis and provide a reference urolithiasis diagnosis and treatment plan, improving the clinical diagnosis and treatment efficiency of urolithiasis.

[0090] Regarding the clinical medical examination of urolithiasis patients, it can be carried out from aspects such as medical images, case files, and laboratory data (laboratory data such as urine and blood). The corresponding clinical medical examinations can be completed by the corresponding departments and the examination data of urolithiasis patients can be uploaded and saved to the background system. Specifically, each examination department can upload the patient's examination data and store it in the HIS system. The background doctor logs in to the system and can view the various examination data of urolithiasis patients, and it can be understood in combination with the existing hospital's examination data upload and storage plan.

[0091] The implementation plans of each step will be further described below.

[0092] Preferably, the collection of the clinical medical examination data of urolithiasis patients includes:

[0093] Pre-build an electronic medical record file of urolithiasis patients in the HIS system;

[0094] Collect the clinical medical examination data of urolithiasis patients related to urolithiasis, upload it to the HIS system and record it in the electronic medical record file of the urolithiasis patient.

[0095] When the patient seeks medical treatment, they can register on the system and an electronic medical record file will be generated for them in the HIS system. Subsequently, the patient's examination data can be uploaded by each department to the His system and stored in the patient's electronic medical record file, and it can be understood specifically in combination with the above description.

[0096] As Figure 2 shown, preferably, the method for generating the multi-modal urolithiasis intelligent diagnosis and treatment model includes:

[0097] Collect historical medical examination data of a number of urolithiasis patients, including imaging data, text data, and laboratory data related to urolithiasis;

[0098] Extract multi-modal data features from the medical examination data, including: respectively extract the corresponding urolithiasis morphological features, key diagnostic information text features, and laboratory data feature values from the imaging data, text data, and laboratory data;

[0099] Perform feature fusion on the multi-modal data features to obtain multi-modal features;

[0100] Perform feature annotation on the multi-modal features, and the annotation information includes: the symptom level corresponding to the multi-modal features and personalized treatment recommendations based on the multi-modal features;

[0101] After the annotation is completed, a feature set composed of the multi-modal features of a number of urolithiasis patients is obtained;

[0102] Use the feature set as a training set and input it into a preset machine learning model for feature training and learning to obtain the multi-modal urolithiasis intelligent diagnosis and treatment model;

[0103] Use an independent clinical data set to verify the recognition performance of the multi-modal urolithiasis intelligent diagnosis and treatment model:

[0104] If the verification is passed, deploy the multi-modal urolithiasis intelligent diagnosis and treatment model to the background server and put it into application;

[0105] If the verification fails, repeat the above training steps to reconstruct the multi-modal urolithiasis intelligent diagnosis and treatment model.

[0106] The multi-modal urolithiasis intelligent diagnosis and treatment model of the present invention can perform personalized multi-modal feature training and learning based on the urolithiasis symptom characteristics of personalized patients, as well as the corresponding symptom levels and clinical treatment recommendations, so that the model can learn the personalized urolithiasis diagnosis characteristics of a large number of patients (the symptom characteristics, levels, and corresponding personalized diagnosis strategies of urolithiasis in different individuals). Therefore, based on the multi-modal algorithm model, the above-mentioned personalized multi-modal features can be trained and learned to construct an AI model that can identify personalized urolithiasis symptom characteristics and recommend corresponding urolithiasis diagnosis strategies.

[0107] Specifically:

[0108] Collect data

[0109] Morphological features: Extract the morphological features of urinary stones from medical images (such as ultrasound, CT, MRI, etc.), including size, location, shape, quantity, etc.

[0110] Diagnostic information text: Collect text information such as clinicians' diagnostic reports and medical records, including descriptions of stones and descriptions of patients' symptoms.

[0111] Laboratory data: Obtain the laboratory test results of patients' blood, urine, etc., including but not limited to the levels of minerals such as uric acid, calcium, and phosphorus, as well as other biomarkers that may reflect the causes of stones.

[0112] Data preprocessing

[0113] Preprocess the medical images, such as denoising, enhancement, segmentation, etc., to accurately extract the stone features.

[0114] Clean, segment words, remove stop words, etc. from the text data, and may use natural language processing technology (NLP) to extract key information.

[0115] Fuse the morphological features, text features, and laboratory data features to form multi-modal data features, so as to reduce redundant information and improve the model efficiency.

[0116] Considering the possible associations between different features, feature fusion techniques (such as concatenation, weighted sum, deep feature fusion, etc.) can be used to construct a more comprehensive feature representation.

[0117] This embodiment provides a fusion algorithm for multi-modal data feature F fusion :

[0118]

[0119] Among them, the definitions of each letter and character in the formula are shown in Table 1 below:

[0120]

[0121]

[0122] Table 1

[0123] Data preprocessing:

[0124] Morphological features: Normalize the size of the image / video (such as 224×224 resolution) and perform data augmentation (flipping, cropping);

[0125] Text features: Remove stop words after word segmentation and generate a dense matrix through word vector embedding;

[0126] Laboratory data: Fill in missing values and standardize (Z-score).

[0127] Feature extraction and alignment:

[0128] Use multi-threading technology to achieve time synchronization of cross-modal data (error < 50ms);

[0129] Introduce positional encoding to ensure spatial alignment of features in different modalities (e.g., the location of laboratory equipment matches the camera's field of view).

[0130] Dynamic weight optimization:

[0131] Adjust the weight coefficient based on task requirements (e.g., increase the γ value when laboratory data is abnormal);

[0132] Automatically optimize the weight parameters during the training phase through the backpropagation algorithm.

[0133] Feature annotation

[0134] Symptom level annotation: Grade the urinary stone symptoms of each patient according to clinical standards and expert experience, such as mild, moderate, severe, etc.

[0135] Personalized treatment recommendation annotation: Based on current clinical guidelines and expert knowledge, recommend personalized treatment suggestions for each patient according to their multi-modal features, such as drug treatment, surgical stone removal, extracorporeal shock wave lithotripsy, etc.

[0136] Construct an AI model

[0137] According to the task characteristics (classification + recommendation), select a suitable machine learning or deep learning model. For example, a multi-task learning framework can be used to handle both symptom level classification and treatment recommendation.

[0138] Considering the characteristics of multi-modal data, a model that can handle heterogeneous data can be selected, such as a deep neural network (DNN), a convolutional neural network (CNN) combined with a recurrent neural network (RNN), or a Transformer, etc.

[0139] Specific implementation is carried out in combination with subsequent model training.

[0140] In this embodiment, AI algorithm models such as random forest models or decision trees can be used to perform feature engineering and feature training on the historical medical examination data of urolithiasis patients, so as to construct a corresponding multi-modal intelligent diagnosis and treatment model for urolithiasis. For the historical medical examination data of patients, different processing methods can be used to extract the data features of each corresponding modality. In the present invention, multi-modal data mainly includes the following several modality data: for example, text data in medical records, image feature data in urolithiasis medical images, and feature values in laboratory data. Corresponding feature extraction methods can be used to extract the data features of each modality from the corresponding data, so as to perform feature fusion subsequently to obtain the multi-modal data features of the patient. And the administrator can add feature annotations and corresponding personalized treatment suggestions for individual patients according to their multi-modal features. In addition, corresponding other annotation information can be further added, which can be determined according to requirements.

[0141] Model training mainly involves the following stages:

[0142] 1. Data collection

[0143] Image data: CT and ultrasound images in the DICOM format of the hospital.

[0144] Text data: Electronic medical records (EMR), including symptom descriptions, medical histories, and treatment records.

[0145] Laboratory data: Urine and blood analysis results, including pH value, urinary calcium concentration, and uric acid level.

[0146] 2. Feature extraction and data annotation

[0147] Please refer to the following several feature extraction methods:

[0148] 2.1 Text data processing

[0149] Use an NLP model (such as BERT) to parse the medical records and extract key diagnostic information (such as "low back pain", "calcium oxalate stone", "extracorporeal shock wave lithotripsy", and "dynamic health care treatment").

[0150] The text embedding vector passes through the multi-head attention mechanism of Transformer to capture the key features in the text and their potential relevance to other modality data.

[0151] 2.2 Image data processing

[0152] Use ResNet or ViT to process CT and ultrasound images and extract:

[0153] The location and size of the stone (such as "ROI frame coordinates" and "6mm").

[0154] Stone density (HU value).

[0155] Stone shape and boundary features (such as contour features generated by the segmentation model).

[0156] 2.3 Laboratory data processing

[0157] Modularize the urine and blood analysis data:

[0158] Numerical features such as pH value and uric acid concentration.

[0159] Label features classified as "normal" or "abnormal".

[0160] Build a model for numerical features through MLP (Multi-Layer Perceptron) and generate a standardized output embedding vector.

[0161] 2.4 Multimodal data fusion

[0162] Integrate text, imaging, and laboratory features based on a cross-modal Transformer model, learn the deep correlation relationships between modalities, and obtain relevant multimodal features.

[0163] Use the multi-head attention mechanism to perform weighted integration on the multimodal features and generate a unified diagnosis result.

[0164] Generate a joint embedding space after feature fusion to support downstream classification and regression tasks.

[0165] 2.5 Imaging data

[0166] Radiologists annotate the stone location, size, and density to generate a Bounding Box or a segmentation mask.

[0167] Text data annotation: Extract and annotate key diagnostic information in the medical record (such as "calcium oxalate stone").

[0168] Laboratory data annotation: Record numerical features and classify them as normal or abnormal.

[0169] Personalized treatment recommendation annotation: Based on the text, imaging, and laboratory features of each patient, annotate the corresponding personalized treatment recommendations.

[0170] 3. Quality control

[0171] Double review: The annotation results are reviewed by urologists and radiologists respectively.

[0172] Consistency verification: Perform time alignment and consistency checks on the multimodal data of patients.

[0173] 4. Model training:

[0174] Select a machine learning model, such as a random forest model, a decision tree, or a convolutional neural network model like RNN or CNN.

[0175] Here, take the random forest model as an example:

[0176] Training process:

[0177] Single-modal pre-training: Perform pre-training on text, image, and laboratory datasets respectively.

[0178] Multi-modal joint fine-tuning: Fine-tune the model parameters on the multi-modal fusion dataset to optimize the interactive learning between modalities.

[0179] Loss function design:

[0180] For classification tasks, use the cross-entropy loss function (Cross-Entropy Loss).

[0181] For regression tasks, use the mean squared error (MSE) loss function.

[0182] Total loss function for multi-task: The total loss function for multi-task is designed as follows:

[0183] Ltotal = αL classification + βL regression ,

[0184] L classification is the cross-entropy loss (Cross-Entropy Loss) for the classification task,

[0185] L regression is the mean squared error loss (Mean Squared Error, MSE) for the regression task,

[0186] α and β are weight parameters used to balance the contributions of classification and regression tasks and are dynamically adjusted according to experimental settings.

[0187] Algorithm performance verification

[0188] Verification dataset:

[0189] Use an independent clinical dataset covering common stone types (such as calcium oxalate, uric acid stones) and complex cases (such as multiple stones).

[0190] Expand the dataset size to more than 10,000 patients to ensure coverage of diverse case characteristics.

[0191] Performance metrics:

[0192] Classification tasks: Accuracy, Precision, Recall. The verification process can be completed by the administrator. If the verification meets the preset requirements (for example, the precision reaches more than 95%), it is qualified, and the model will be deployed and applied.

[0193] Regression task: Mean Squared Error (MSE).

[0194] Verification methods:

[0195] Comparative experiment: Conduct a comparative analysis with the diagnostic results of traditional doctors.

[0196] Interpretability verification: Use Grad-CAM to generate heatmaps of image features to verify whether the model focuses on key regions; use SHAP to analyze the key decision factors of the text module.

[0197] The training, construction, and application of the random forest model can refer to the following steps:

[0198] 1. Data preparation

[0199] Collect and clean data: First, a dataset for training needs to be prepared, including features and labels. Features are the variables used for prediction, and labels are the results that the model is expected to predict. Ensure that the dataset is cleaned and preprocessed, including handling missing values, outliers, and normalizing features, etc.

[0200] 2. Dataset division

[0201] Training set and test set: Divide the prepared dataset into a training set and a test set. Usually, cross-validation or the hold-out method is used to divide the dataset (for example, in an 8:2 ratio) into a training set and a test set to ensure that the model is verified on unseen data. In this embodiment, an independent clinical dataset is used to verify the recognition performance of the multi-modal urolithiasis intelligent diagnosis and treatment model. Here, it is not necessary to divide the feature set into a training set and a test set, but all the feature sets are input into the model for feature training and learning. The validation set can be prepared independently by the administrator with an independent clinical dataset, and the urolithiasis examination data of clinical example patients are used to verify the model in real time to ensure that the model has clinical verification and recognition capabilities.

[0202] 3. Model selection

[0203] Select the random forest model: Select the random forest as the model. Random forest is an ensemble learning method that makes predictions by constructing multiple decision trees and then combines the results of multiple decision trees to obtain the final prediction result.

[0204] 4. Model training

[0205] Training the Random Forest Model: Use the training set to train the random forest model. During the training process, the random forest randomly selects features and samples to construct multiple decision trees, and uses voting or averaging methods to combine the results of multiple decision trees.

[0206] 5. Model Tuning

[0207] Hyperparameter Tuning: Optimize the trained model, including adjusting hyperparameters (such as the number of trees, maximum depth, etc.), optimizing the model structure, etc., to improve the performance and generalization ability of the model.

[0208] 6. Model Evaluation

[0209] Evaluation Using the Test Set: Use the test set to evaluate the trained model. Usually, metrics such as accuracy, precision, recall, F1-score, etc. are used to evaluate the performance of the model. Cross-validation is an important tool for evaluating the stability and performance of the model.

[0210] 7. Model Application

[0211] Practical Application and Monitoring: When the model passes the evaluation and meets the requirements, it can be used for actual prediction. In practical applications, attention needs to be paid to the deployment and monitoring of the model to ensure the continuous performance of the model.

[0212] Preferably, the annotation information further includes:

[0213] Follow-up suggestions recommended according to the multimodal features of urolithiasis patients.

[0214] After identifying the multimodal features of the patient, the follow-up suggestions corresponding to the features can also be output together, which can be specifically matched and output according to the recommended information annotated by the administrator. For follow-up suggestions, reference can be made to the subsequent embodiments.

[0215] Using the above model for clinical assistant diagnosis can identify the multimodal features of patients, and combine medical guidelines and patient characteristics to dynamically generate personalized diagnosis results and treatment suggestions.

[0216] Preferably, after the multimodal urolithiasis intelligent diagnosis and treatment model identifies the multimodal data features in the clinical medical examination data and outputs personalized treatment suggestions matching the multimodal data features, it further includes:

[0217] Extract the urolithiasis diagnosis keywords from the multimodal data features;

[0218] Construct a medical guideline retrieval logic matching the urolithiasis diagnosis keywords;

[0219] Construct a large language model prompt based on the urolithiasis diagnosis keywords and the medical guideline retrieval logic, and input it into a preset LLM large language model;

[0220] Through the LLM large language model, perform a database search based on the large language model prompt, and retrieve the medical guideline treatment strategies matching the urolithiasis diagnosis keywords from the medical database;

[0221] Output the medical guideline treatment strategies, bind them to the corresponding personalized treatment suggestions, and write them into a preset diagnostic report to generate a urolithiasis diagnosis and treatment report for this urolithiasis patient.

[0222] The model can identify the multi-modal data features in the clinical medical examination data of urolithiasis patients and output personalized treatment suggestions matching the multi-modal data features. In order to provide richer medical diagnosis and treatment suggestions for doctors (or patients), not only the personalized treatment suggestions recommended by the model (the treatment plan recommended for this patient) are adopted here, but also based on the results identified and output by the model, combined with medical guidelines and patient characteristics again, personalized diagnostic results and treatment suggestions are dynamically generated. In this way, on the basis of the personalized treatment suggestions recommended by the model, combined with the characteristic descriptions and treatment suggestions of different urolithiasises in the medical guidelines, the treatment suggestions in the medical guidelines (that is, the medical guideline treatment strategies) are generated. In this way, it is possible to combine the model recommendations and medical guidelines to jointly recommend the diagnosis and treatment plan for urolithiasis, enrich the recommended content, and provide doctors with richer clinical diagnosis and treatment plans.

[0223] Here, the present invention adopts the intelligent assistance function of the large language model, enabling the large language model to retrieve the medical guideline treatment strategies matching the urolithiasis diagnosis keywords from the medical database based on the urolithiasis diagnosis keywords in the multi-modal data features. Specifically:

[0224] The administrator can pre-construct a medical guideline retrieval logic matching the urolithiasis diagnosis keywords, such as "Please query / retrieve the clinical treatment plan for the symptoms and characteristics of early calcium oxalate stones". Subsequently, combine the keywords and the retrieval logic to form a large language model prompt and input it into a preset LLM large language model, enabling the LLM large language model to perform a database search based on the prompt and retrieve the medical guideline treatment strategies matching the urolithiasis diagnosis keywords from the medical database.

[0225] Extracting urolithiasis diagnosis keywords from multi-modal data and retrieving relevant medical guideline treatment strategies from the medical database based on these keywords can be carried out according to the following steps:

[0226] 1. Extract urolithiasis diagnosis keywords

[0227] Data preprocessing: First, preprocess multimodal data (such as text, images, audio, etc.) to ensure data quality and consistency. For text data, operations such as word segmentation, stop word removal, and stemming may be required.

[0228] Feature extraction: Use natural language processing (NLP) techniques, such as word frequency statistics, TF-IDF, word embeddings, etc., to extract keywords related to urolithiasis diagnosis from the preprocessed text data. For image and audio data, specific feature extraction methods may be required, such as convolutional neural networks (CNNs) or audio processing algorithms.

[0229] Keyword screening: Based on medical expertise and experience, screen the extracted keywords and retain the most relevant and representative keywords for urolithiasis diagnosis (such as "low back pain", "calcium oxalate stone", "extracorporeal shock wave lithotripsy", and "dynamic health care treatment").

[0230] 2. Construct the retrieval logic for medical guidelines

[0231] Determine the retrieval scope: Clearly define the scope and source of the medical database to ensure the authority and reliability of the retrieval results.

[0232] Construct the retrieval expression: Based on the extracted keywords for urolithiasis diagnosis, construct a retrieval expression or query statement. This can include combinations of keywords, replacement of synonyms, use of Boolean logical operators, etc. The retrieval time period, retrieval library, and retrieval data range can all be limited in the logic.

[0233] Optimize the retrieval strategy: According to the preliminary retrieval results, adjust the retrieval expression and strategy to improve the accuracy and comprehensiveness of the retrieval results.

[0234] 3. Construct the prompt words for the large language model

[0235] Combine keywords and retrieval logic: Combine the keywords for urolithiasis diagnosis and the retrieval logic for medical guidelines to construct prompt words for input into the large language model (LLM). These prompt words should be able to clearly express our retrieval intent and requirements.

[0236] Optimize the expression of prompt words: According to the characteristics and requirements of the large language model (LLM), optimize and adjust the prompt words to ensure that the model can accurately understand and process these prompt words.

[0237] 4. Input into the large language model (LLM) for database retrieval

[0238] Model selection: Select a large language model (LLM) suitable for the current task, such as BERT, GPT, etc.

[0239] Input prompt: Input the constructed large language model prompt into the LLM large language model.

[0240] Execute retrieval: Use the LLM large language model to search the medical database and obtain the medical guideline treatment strategies that match the keywords for the diagnosis of urinary calculi.

[0241] 5. Process and present the retrieval results

[0242] Result screening: Screen and sort the retrieval results to remove irrelevant or duplicate information.

[0243] Result presentation: Present the screened retrieval results to the user in a clear and readable manner, such as lists, charts, summaries, etc.

[0244] Result interpretation: Interpret and explain the retrieval results based on medical professional knowledge and experience to help the user better understand and apply these medical guideline treatment strategies.

[0245] The retrieved medical guideline treatment strategies can be output and bound to the corresponding personalized treatment suggestions, and written into a preset diagnostic report to generate a urinary calculi diagnosis and treatment report for the urinary calculi patient. Subsequently, doctors can log in to the background to view the treatment suggestions in the diagnostic report.

[0246] Through the above steps, it is possible to effectively extract the keywords for the diagnosis of urinary calculi from multi-modal data, and retrieve relevant medical guideline treatment strategies from the medical database based on these keywords. This can provide accurate diagnostic basis and treatment suggestions for doctors, improving the quality and efficiency of medical services.

[0247] The LLM large language model in this embodiment can be user-customized and developed, or can be called by the background API to request a third-party large language model such as Wenxin Yiyan of Baidu.

[0248] The present invention can also enable the system to combine case analysis of the medical big database to match similar medical records and further optimize the generated suggestions. Specifically:

[0249] Preferably, after the multi-modal urinary calculi intelligent diagnosis and treatment model identifies the multi-modal data features in the clinical medical examination data and outputs personalized treatment suggestions that match the multi-modal data features, it further includes:

[0250] Extract the keywords for the diagnosis of urinary calculi from the multi-modal data features;

[0251] Construct a medical case retrieval logic that matches the keywords for the diagnosis of urinary calculi;

[0252] Combine the above-mentioned urolithiasis diagnosis keywords and the medical case retrieval logic to construct prompt words for the large language model and input them into the preset LLM large language model;

[0253] Through the LLM large language model, based on the prompt words of the large language model, perform a database search, and retrieve urolithiasis medical cases that match the urolithiasis diagnosis keywords from the medical case database;

[0254] Output the urolithiasis medical cases and bind them to the corresponding personalized treatment suggestions, and write them into the preset diagnosis report to generate the urolithiasis diagnosis and treatment report for the urolithiasis patient.

[0255] Based on the application of the above LLM large language model, the present invention can also recommend clinical treatment cases related to the multi-modal data characteristics of the patient to doctors (or patients) for reference by doctors (or patients). For specific understanding, please refer to the method of performing a database search by the above LLM large language model based on the prompt words of the large language model.

[0256] The medical case database here, etc., is a database storing a large amount of medical diagnosis and treatment cases provided by hospitals, etc.

[0257] The corresponding medical case retrieval logic can be constructed in combination with the above-mentioned "medical guideline retrieval logic", and the logic includes the corresponding retrieval scope, case retrieval content keywords, time, etc. Specifically constructed by the administrator.

[0258] Therefore, by adopting the above solution, it is possible to match similar medical records by combining case analysis of the medical big database, and further optimize the generated suggestions.

[0259] The following are cases of using the above intelligent model for clinical intelligent diagnosis and treatment:

[0260] Example 1. Diagnosis of complex cases and generation of personalized treatment plans

[0261] 1. Medical record text: A 50-year-old female with repeated right-sided low back pain and hematuria for 1 month, with a family history of urolithiasis.

[0262] Medical history: Treated with laser lithotripsy for calcium oxalate stones 2 years ago.

[0263] 2. Imaging data:

[0264] CT image: A 6-mm high-density shadow was found in the middle of the right kidney, with mild inflammatory reaction around it.

[0265] 3. Laboratory data:

[0266] Urine pH = 5.8.

[0267] Urine calcium concentration is elevated, and urine citrate is low.

[0268] 4. System Processing:

[0269] Text Processing:

[0270] Extract keywords from the medical record: "low back pain", "history of calcium oxalate stones", "family history". Image Processing:

[0271] Analyze the CT image, identify a 6mm high-density stone in the middle of the right kidney, and mark the area of inflammatory changes. Laboratory Data Processing:

[0272] Extract the characteristics of urine and blood indicators, and evaluate the tendency of stone type in combination with metabolic data.

[0273] Multi-modal Fusion:

[0274] Integrate text, image, and laboratory features, and infer the stone type and cause through a large model. 5. Output:

[0275] Diagnosis Result:

[0276] Calcium oxalate stone in the right kidney, with local inflammation.

[0277] Personalized Treatment Suggestions:

[0278] Short-term: Antibiotic treatment to control inflammation.

[0279] Long-term: Increase water intake and take potassium citrate (to alkalize urine).

[0280] Follow-up: Recheck urine indicators after 2 weeks to evaluate the curative effect.

[0281] 6. System Explanation:

[0282] Heat map of the stone location and size in the CT image.

[0283] Highlight the keywords in the medical record (such as "history of calcium oxalate stones").

[0284] Basis for generating suggestions: Combined with medical guidelines and the patient's metabolic characteristics.

[0285] Example 2. Postoperative Follow-up and Monitoring of Stone Recurrence

[0286] 1. Medical Record Text:

[0287] A 60-year-old male, with a follow-up examination 3 months after surgery, showing no obvious symptoms.

[0288] Medical History: Received laser lithotripsy 3 months ago for the treatment of a 10mm uric acid stone.

[0289] 2. Image Data:

[0290] CT Image: No residual stone was found.

[0291] 3. Laboratory data:

[0292] Urine pH = 6.5 (reached the normal range through alkalization treatment).

[0293] Uric acid level is normal.

[0294] 4. System processing:

[0295] Text processing:

[0296] Extract medical record keywords: "3 months after surgery", "history of uric acid stones".

[0297] Image processing:

[0298] Analyze CT images to confirm no residual stones.

[0299] Laboratory data processing:

[0300] Extract urine and blood indicators to evaluate the uric acid metabolism status.

[0301] 5. Output:

[0302] Postoperative status evaluation:

[0303] Recovery is good, no residual stones.

[0304] Follow-up advice:

[0305] Short term: Continue urine alkalization treatment for 6 months.

[0306] Long term: Recheck urine and uric acid indicators every 6 months to monitor the recurrence risk.

[0307] 6. System explanation:

[0308] Mark the stone-free area on the CT image.

[0309] Model reasoning explanation: The urine pH has reached the normal range, the uric acid concentration is controlled stably, and the recurrence risk is low.

[0310] The present invention uses the intelligent diagnosis and treatment plan of the above model for clinical diagnosis. Through multi-modal data fusion and large model technology, it provides an intelligent and full-process solution for the diagnosis and treatment of urinary stones, significantly improving the diagnostic accuracy and treatment effect. It is applicable to primary and advanced medical institutions and has a wide application prospect.

[0311] Figure 3 It is a block diagram of a multi-modal intelligent diagnosis and treatment system for urinary stones shown according to an exemplary embodiment. This system is used for the construction and application method of a multi-modal intelligent diagnosis and treatment model for urinary stones. Refer to Figure 3 , this system includes a data acquisition module 310, an AI diagnosis and treatment module 320, a report generation module 330, and a report recommendation module 340. Among them:

[0312] A data acquisition module 310, configured to collect clinical medical examination data of patients with urinary calculi;

[0313] An AI diagnosis and treatment module 320, configured to input the clinical medical examination data into a pre-deployed multi-modal intelligent diagnosis and treatment model for urinary calculi, identify multi-modal data features and their symptom levels in the clinical medical examination data through the multi-modal intelligent diagnosis and treatment model for urinary calculi, and output personalized treatment suggestions matching the multi-modal data features at this symptom level;

[0314] A report generation module 330, configured to bind the multi-modal data features with the corresponding personalized treatment suggestions, write them into a preset diagnosis report, and generate a urinary calculi diagnosis and treatment report for this patient with urinary calculi;

[0315] A report recommendation module 340, configured to recommend the urinary calculi diagnosis and treatment report to the medical staff side and / or the patient side.

[0316] For the above-mentioned various modules and their interaction functions, please refer to the corresponding steps in the above method, which will not be elaborated here.

[0317] Figure 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 4 shown, the electronic device may include the above-mentioned Figure 3 multi-modal intelligent diagnosis and treatment system for urinary calculi shown. Optionally, the electronic device 410 may include a first processor 2001.

[0318] Optionally, the electronic device 410 may further include a memory 2002 and a transceiver 2003.

[0319] Wherein, the first processor 2001 is connected to the memory 2002 and the transceiver 2003, such as through a communication bus.

[0320] Next, in combination with Figure 4 each component of the electronic device 410 will be specifically introduced:

[0321] Among them, the first processor 2001 is the control center of the electronic device 410, which can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or can be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0322] Optionally, the first processor 2001 can execute various functions of the electronic device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0323] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 shown in

[0324] In a specific implementation, as an embodiment, the electronic device 410 may also include multiple processors, such as Figure 4 the first processor 2001 and the second processor 2004 shown in

[0325] Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0326] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through the interface circuit of the electronic device 410 ( Figure 4 not shown in the figure), and the embodiments of the present invention do not make specific limitations in this regard.

[0327] The transceiver 2003 is used to communicate with a network device or with a terminal device.

[0328] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 not shown separately in the figure). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0329] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through the interface circuit of the electronic device 410 ( Figure 4 not shown in the figure), and the embodiments of the present invention do not make specific limitations in this regard.

[0330] It should be noted that Figure 4 the structure of the electronic device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0331] In addition, the technical effects of the electronic device 410 can refer to the technical effects of the method for constructing and applying the multi-modal urinary calculus intelligent diagnosis and treatment model described in the above method embodiments, and will not be elaborated here.

[0332] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0333] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for constructing a multimodal intelligent diagnosis and treatment model for urinary stones, characterized in that: The method comprises: Collect historical medical examination data of several patients with urinary stones, including imaging data, text data and laboratory data related to urinary stones; Extracting multimodal data features from the medical examination data, including: extracting corresponding urinary stone morphological features, key diagnostic information text features, and laboratory data feature values ​​from the image data, text data, and laboratory data, respectively; Performing feature fusion on the multimodal data features to obtain multimodal features; Performing feature annotation on the multimodal feature, wherein the annotation information includes: a symptom level corresponding to the multimodal feature and a personalized treatment suggestion recommended according to the multimodal feature; After labeling is completed, a feature set consisting of the multimodal features of several patients with urinary stones is obtained; The feature set is used as a training set and input into a preset machine learning model to perform feature training and learning to obtain the multimodal urinary stone intelligent diagnosis and treatment model; Using independent clinical data sets, the recognition performance of the multimodal urinary stone intelligent diagnosis and treatment model was verified: If the verification is passed, the multimodal urinary stone intelligent diagnosis and treatment model is deployed to the background server and put into use; If the verification fails, the above training steps are repeated to reconstruct the multimodal urinary stone intelligent diagnosis and treatment model.

2. The method for constructing a multimodal intelligent diagnosis and treatment model for urinary stones according to claim 1, characterized in that: The extracting multimodal data features from the medical examination data includes: Using the ResNet or ViT model, extracting urinary stone morphological features from urinary stone CT and / or ultrasound images of patients with urinary stones, the urinary stone morphological features including the following features: urinary stone location and size, urinary stone density, and urinary stone shape and boundary features; as well as, Using an NLP model or an attention mechanism model, parse the electronic medical records of patients with urinary stones and extract key diagnostic information text features therein, wherein the key diagnostic information text features include diagnostic keywords related to urinary stones and association features between the diagnostic keywords and other modal data features; as well as, The laboratory data feature values ​​in the clinical laboratory data of patients with urinary stones are extracted by using the MLP mechanism, and the laboratory data feature values ​​include at least one of the following features: pH / uric acid concentration value of urine or blood.

3. The method for constructing and applying the multimodal intelligent diagnosis and treatment model for urinary stones according to claim 2 is characterized in that: The annotation information also includes: Recommended follow-up recommendations based on the described multimodal characteristics of patients with urinary stones.

4. An application method of the multimodal urinary stone intelligent diagnosis and treatment model according to any one of claims 1 to 3, characterized in that: The method comprises: To collect clinical medical examination data of patients with urinary stones; Inputting the clinical medical examination data into a pre-deployed multimodal urinary calculi intelligent diagnosis and treatment model, identifying the multimodal data features and symptom levels in the clinical medical examination data through the multimodal urinary calculi intelligent diagnosis and treatment model, and outputting personalized treatment recommendations that match the multimodal data features at the symptom level; Binding the multimodal data features with the corresponding personalized treatment recommendations and writing them into a preset diagnosis report to generate a urinary stone diagnosis and treatment report for the urinary stone patient; The urinary stone diagnosis and treatment report is recommended to the medical staff and / or the patient.

5. The application method of the multimodal urinary stone intelligent diagnosis and treatment model according to claim 4 is characterized in that: The clinical medical examination data collected from patients with urinary stones include: Pre-build the electronic medical records of patients with urinary stones in the HIS system; The clinical medical examination data related to urinary stones of the patient are collected, uploaded to the HIS system and recorded in the electronic medical record file of the patient.

6. The application method of the multimodal urinary stone intelligent diagnosis and treatment model according to claim 4 is characterized in that: After the multimodal urinary calculus intelligent diagnosis and treatment model identifies the multimodal data features in the clinical medical examination data and outputs personalized treatment suggestions matching the multimodal data features, it also includes: Extracting urinary stone diagnosis keywords from the multimodal data features; Constructing medical guideline retrieval logic matching the urinary stone diagnosis keywords; Combining the urinary stone diagnosis keywords and the medical guideline retrieval logic, constructing a large language model prompt word and inputting a preset LLM large language model; Through the LLM large language model, a database search is performed based on the large language model prompt words to retrieve medical guideline treatment strategies that match the urinary stone diagnosis keywords from the medical database; The medical guideline treatment strategy is output and bound to the corresponding personalized treatment suggestion, and written into a preset diagnosis report to generate a urinary stone diagnosis and treatment report for the urinary stone patient.

7. The application method of the multimodal urinary stone intelligent diagnosis and treatment model according to claim 4 is characterized in that: After the multimodal urinary calculus intelligent diagnosis and treatment model identifies the multimodal data features in the clinical medical examination data and outputs personalized treatment suggestions matching the multimodal data features, it also includes: Extracting urinary stone diagnosis keywords from the multimodal data features; Constructing medical case retrieval logic matching the urinary stone diagnosis keywords; Combining the urinary stone diagnosis keywords and the medical case retrieval logic, constructing a large language model prompt word and inputting a preset LLM large language model; Using the LLM large language model, based on the large language model prompt words, a library search is performed to retrieve urinary stone medical cases that match the urinary stone diagnosis keywords from the medical case library; The urinary stone medical case is output and bound with the corresponding personalized treatment suggestion, and written into a preset diagnosis report to generate a urinary stone diagnosis and treatment report for the urinary stone patient.

8. A multimodal intelligent diagnosis and treatment system for urinary stones, the multimodal intelligent diagnosis and treatment system for urinary stones is used to implement the application method of the multimodal intelligent diagnosis and treatment model for urinary stones as claimed in any one of claims 4 to 7, characterized in that: The system comprises: A data collection module is used to collect clinical medical examination data of patients with urinary stones; An AI diagnosis and treatment module is used to input the clinical medical examination data into a pre-deployed multimodal urinary calculi intelligent diagnosis and treatment model, identify the multimodal data features and symptom levels in the clinical medical examination data through the multimodal urinary calculi intelligent diagnosis and treatment model, and output personalized treatment recommendations that match the multimodal data features at the symptom level; A report generation module, used to bind the multimodal data features with the corresponding personalized treatment suggestions, and write them into a preset diagnosis report to generate a urinary stone diagnosis and treatment report for the urinary stone patient; The report recommendation module is used to recommend the urinary stone diagnosis and treatment report to the medical staff and / or patient.

9. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images