Medical data intelligent scientific research method and system based on multi-modal model and continuous learning

Abstract

The invention relates to a medical data intelligent scientific research method and system based on a multi-modal model and continuous learning. The method comprises the following steps: adding scientific research tasks and task description; searching medical input data from a medical database, and predicting output data; determining a machine learning model, and performing model training and data analysis to obtain an intelligent prediction model and correlation parameters between the medical input data and the prediction output data; evaluating the intelligent prediction model, and obtaining feedback data of the intelligent prediction model; wherein the machine learning model is a multi-modal model or a single-modal model. Scientific researchers can create scientific research tasks and select medical input data, prediction output data and a machine learning model to accumulate data and scientific research tasks newly added in the future for scientific researchers in the future; a user does not need to try machine learning models one by one, so that the time cost is saved; through evaluation and feedback of the prediction model, model recommendation can be automatically carried out.

Description

Technical Field

[0001] The present invention relates to the technical field of medical data processing, and in particular to an intelligent scientific research method and system for medical data based on multimodal models and continuous learning. Background Art

[0002] The current research methods for hospital medical data usually follow the research paradigm of "topic selection, ethical review, clinical trials, data collection, data processing and analysis, and report writing". In the "data collection, data processing and analysis" stage, it usually involves the use of machine learning technology and statistical methods to study single-modal or multi-modal data, including text (medical history reports, diagnostic reports), images (CT, MRI, color ultrasound images), videos (surgical endoscopes), tables (medication, laboratory tests, questionnaire scores, vital signs detection) and other data. Medical researchers explore the correlation and causality in medical big data based on medical clinical prior knowledge and expert knowledge. First, they confirm the input (independent variable) and output (dependent variable), construct the data set of the "data collection, data processing and analysis" stage, and then use machine learning technology and statistical methods to explain the new knowledge in the data.

[0003] When faced with the above scenarios, existing technologies usually use one-off, proprietary models to study and solve a corresponding "input and output" task, such as: using traditional machine learning models to process tabular data; using NLP models to train and predict medical text report data and tabular data; using NLP+CV models to train and predict medical multimodal data.

[0004] However, the expert knowledge contained in each "input and output" task has low reusability and low automation utilization; the unsatisfactory "input and output" are not fully utilized; the machine learning research tasks of multimodal medical data put forward requirements for code engineering technology. Therefore, traditional medical data research methods often have problems such as single mode, low practicality and flexibility when performing input and output tasks. Summary of the invention

[0005] Based on this, in order to solve the above technical problems, a medical data intelligent scientific research method and system based on multimodal models and continuous learning is provided, which can improve the convenience of medical data scientific research and has high practicality and flexibility.

[0006] A medical data intelligent scientific research method based on multimodal model and continuous learning, the method comprising:

[0007] Obtaining a task adding instruction, adding a scientific research task according to the task adding instruction, and obtaining a task description corresponding to the scientific research task;

[0008] According to the task description, searching for medical input data and predicted output data corresponding to the scientific research task from a medical database;

[0009] Determine a machine learning model, input the medical input data and the predicted output data into the machine learning model for model training and data analysis, and obtain an intelligent prediction model and correlation parameters between the medical input data and the predicted output data;

[0010] Determine an evaluation index, use the evaluation index to evaluate the intelligent prediction model to obtain evaluation data; and obtain feedback data for the intelligent prediction model;

[0011] Storing the intelligent prediction model, correlation parameters, evaluation data, and feedback data in the medical database;

[0012] Wherein, the machine learning model is a multimodal model or a unimodal model.

[0013] In one embodiment, the method further comprises:

[0014] Obtain each medical data input, and add a column name and data description for each medical data;

[0015] Inputting each of the medical data, as well as the column name and data description corresponding to the medical data into the medical database and storing them;

[0016] The formats of the medical data include text, images, videos, and tables.

[0017] In one embodiment, searching for medical input data and predicted output data corresponding to the scientific research task from a medical database according to the task description includes:

[0018] Searching the medical database for a column name and a data description corresponding to the task description as a target column name and a data description;

[0019] The target medical data is determined according to the target column name and data description as medical input data and predicted output data.

[0020] In one embodiment, determining a machine learning model, inputting the medical input data and the predicted output data into the machine learning model for model training and data analysis, includes:

[0021] Obtaining a model selection instruction, and determining a machine learning model according to the model selection instruction;

[0022] Generate a training model data set according to the medical input data and the predicted output data, and divide the training model data set into a training set and a validation set;

[0023] The machine learning model is trained based on the training set and the validation set.

[0024] In one embodiment, the machine learning model includes logistic regression, support vector machine, decision tree, random forest, K nearest neighbor, multi-layer perceptron, convolutional neural network, timing model, language model, cross-modal model.

[0025] In one embodiment, the method further comprises:

[0026] Obtaining an editing instruction, and performing an editing operation on the medical input data and the predicted output data according to the editing instruction to obtain the edited medical input data and the predicted output data;

[0027] Inputting the edited medical input data and predicted output data into the machine learning model for model training and data analysis to obtain an updated intelligent prediction model and the latest correlation parameters between the edited medical input data and predicted output data;

[0028] Using the evaluation index to evaluate the updated intelligent prediction model to obtain updated evaluation data; and obtaining the latest feedback data on the updated intelligent prediction model;

[0029] The updated intelligent prediction model, the latest correlation parameters, the updated evaluation data, and the latest feedback data are stored in the medical database.

[0030] In one embodiment, the method further comprises:

[0031] Obtaining an output task selection instruction, and determining an output task type corresponding to the intelligent prediction model according to the output task selection instruction;

[0032] The intelligent prediction model outputs a prediction result according to the output task type;

[0033] The output task types include classification tasks and regression tasks.

[0034] In one embodiment, using the evaluation index to evaluate the intelligent prediction model includes:

[0035] Determine the corresponding evaluation index according to the output task type;

[0036] When the output task type is a classification task, the intelligent prediction model is evaluated using a first evaluation index;

[0037] When the output task type is a regression task, a second evaluation index is used to evaluate the intelligent prediction model.

[0038] A medical data intelligent scientific research system based on multimodal model and continuous learning, the system comprising:

[0039] A scientific research task adding module, used to obtain a task adding instruction, add a scientific research task according to the task adding instruction, and obtain a task description corresponding to the scientific research task;

[0040] An input and output data determination module, used to search for medical input data and predicted output data corresponding to the scientific research task from a medical database according to the task description;

[0041] A model training and data analysis module, used to determine a machine learning model, input the medical input data and the predicted output data into the machine learning model for model training and data analysis, and obtain an intelligent prediction model and correlation parameters between the medical input data and the predicted output data;

[0042] An evaluation module, used to determine evaluation indicators, use the evaluation indicators to evaluate the intelligent prediction model, obtain evaluation data; and obtain feedback data on the intelligent prediction model;

[0043] A data storage module, used for storing the intelligent prediction model, correlation parameters, evaluation data, and feedback data into the medical database;

[0044] Wherein, the machine learning model is a multimodal model or a unimodal model.

[0045] The above-mentioned intelligent scientific research method and system for medical data based on multimodal models and continuous learning allow researchers to create scientific research tasks and select medical input data, predicted output data, and machine learning models, so as to analyze the correlation between input and output data and train a predictive model, which can accumulate for future researchers, future additional data, and scientific research tasks; and users can understand the correlation between data without trying machine learning models one by one, saving time and cost; by evaluating and providing feedback on the predictive model, model recommendations can be made automatically, reducing the code engineering threshold and time required for different additional data and research tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 An application environment diagram of an intelligent scientific research method for medical data based on a multimodal model and continuous learning in one embodiment;

[0047] Figure 2 A flowchart of an intelligent scientific research method for medical data based on a multimodal model and continuous learning in one embodiment;

[0048] Figure 3 is a structural block diagram of a medical data intelligent scientific research system based on multimodal model and continuous learning in one embodiment;

[0049] Figure 4 is a structural block diagram of a medical data intelligent scientific research system based on multimodal model and continuous learning in another embodiment;

[0050] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] The medical data intelligent research method based on multimodal model and continuous learning provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1 As shown, the application environment includes a computer device 110. The computer device 110 can obtain a task adding instruction, add a scientific research task according to the task adding instruction, and obtain a task description corresponding to the scientific research task; the computer device 110 can search for medical input data and predicted output data corresponding to the scientific research task from the medical database according to the task description; the computer device 110 can determine a machine learning model, input the medical input data and predicted output data into the machine learning model for model training and data analysis, and obtain an intelligent prediction model and correlation parameters between the medical input data and the predicted output data; the computer device 110 can determine an evaluation index, use the evaluation index to evaluate the intelligent prediction model, and obtain evaluation data; and obtain feedback data on the intelligent prediction model; the computer device 110 can store the intelligent prediction model, correlation parameters, evaluation data, and feedback data in the medical database; wherein the machine learning model is a multimodal model or a unimodal model. Among them, the computer device 110 can be, but not limited to, various personal computers, laptops, smart phones, robots, tablet computers and other devices.

[0053] In one embodiment, Figure 2 As shown, a medical data intelligent scientific research method based on multimodal model and continuous learning is provided, including the following steps:

[0054] Step 202, obtain a task adding instruction, add a scientific research task according to the task adding instruction, and obtain a task description corresponding to the scientific research task.

[0055] Among them, the task adding instruction can be triggered by medical researchers or other users in the display interface through the computer device. The computer device can display the editing interface according to the task adding instruction, and the user can add scientific research tasks through the editing interface and add the corresponding task description through the editing interface. The task description can be an explanation of the added scientific research task, and the task description is a free-form, unstructured description.

[0056] Step 204, searching the medical input data and predicted output data corresponding to the scientific research task from the medical database according to the task description.

[0057] Medical databases store various medical data, such as height, weight, blood pressure, heart rate, etc.; and the types of medical data are diverse, such as tables, texts, medical images, etc.

[0058] In one embodiment, a provided intelligent scientific research method for medical data based on multimodal models and continuous learning may also include a process of storing data in a medical database, the specific process including: obtaining each input medical data, and adding a column name and data description for each medical data; inputting each medical data, and the column name and data description corresponding to the medical data into the medical database and storing them; wherein the format of the medical data includes text, image, video, and table.

[0059] The computer equipment is equipped with an intelligent medical data research system based on multimodal models and continuous learning. Based on commonly used machine learning techniques and statistical methods, medical researchers can enter medical data such as text, images, videos, tables, etc. through computer equipment, and can add column names and data descriptions for each type of data.

[0060] In this embodiment, since doctors need to face hundreds of different types of data / column names, doctors can decide to use part of {X-Col-1, X-Col-2...X-Col-N} as independent variables and covariates, and part of {Y-Col-1, Y-Col-2...Y-Col-N} as dependent variables, i.e., "medical input data and predicted output data" based on some medical prior knowledge and observations, and then use machine learning technology to analyze whether the input and output data are related and whether an automated intelligent prediction model can be trained. Among them, {X-Col-1, X-Col-2...X-Col-N} and {Y-Col-1, Y-Col-2...Y-Col-N} refer to different types of data / column names, not to specific data. For example, {X-Col-1, X-Col-2...X-Col-N} = {"height", "weight", "blood pressure", "heart rate"...}, {Y-Col-1, Y-Col-2...Y-Col-N} = {"severity", "concentration of certain data"...}.

[0061] In one embodiment, a medical data intelligent scientific research method based on multimodal models and continuous learning is provided, which may also include a process for confirming input and output data. The specific process includes: searching for column names and data descriptions corresponding to the task description in the medical database as target column names and data descriptions; determining target medical data based on the target column names and data descriptions as medical input data and predicted output data.

[0062] Among them, medical researchers can specify the corresponding column names of medical input data and predicted output data through computer equipment, and allow multiple inputs and multiple outputs at the same time. Medical input data and predicted output data do not limit the number of columns and rows, and medical input data allows different types of data input, including tables, texts, medical images, etc.; output data can be limited to classification and regression tasks, and multiple outputs require users to specify whether different outputs are classification tasks or regression tasks, and the output can be a probability value between 0-1. For example, medical input data and predicted output data can be shown in the following table. Medical input data can include XXX maximum value during XXX treatment, XXX dose at onset (mg), and XXX dose per unit body weight at onset (mg / kg); predicted output data can include efficacy judgment:

[0063]

[0064] Medical researchers can select corresponding medical input data and predicted output data from the medical database through computer equipment according to their own research needs, and then perform subsequent operations such as data correlation analysis.

[0065] Step 206, determine the machine learning model, input the medical input data and the predicted output data into the machine learning model for model training and data analysis, and obtain an intelligent prediction model and correlation parameters between the medical input data and the predicted output data; wherein the machine learning model is a multimodal model or a unimodal model.

[0066] In one embodiment, the machine learning model includes logistic regression, support vector machine, decision tree, random forest, K nearest neighbor, multi-layer perceptron, convolutional neural network, time series model, language model, cross-modal model.

[0067] Medical researchers can select the required machine learning model through computer equipment, or determine the machine learning model based on the model recommendation of the computer equipment. Users can also specify and randomly divide training data and test data, that is, divide the training model data set into training set and validation set, to facilitate subsequent model training. Specifically, users can randomly divide the training model data set into, for example, 70% training set and 30% validation set as needed, and select logistic regression, support vector machine, decision tree, random forest, K nearest neighbor, multi-layer perceptron, convolutional neural network, time series model, language model, cross-modal model and other methods for training.

[0068] In one embodiment, a provided intelligent scientific research method for medical data based on a multimodal model and continuous learning may also include a process for selecting an output task type, the specific process including: obtaining an output task selection instruction, and determining the output task type corresponding to the intelligent prediction model according to the output task selection instruction; the intelligent prediction model outputs a prediction result according to the output task type.

[0069] Among them, the output task types include classification tasks and regression tasks. The output task type can be selected by medical researchers, and different output task types can be selected for different machine learning models.

[0070] Step 208, determine the evaluation index, use the evaluation index to evaluate the intelligent prediction model, obtain evaluation data; and obtain feedback data for the intelligent prediction model.

[0071] There are different evaluation indicators for the corresponding classification and regression tasks based on the current task output, such as accuracy, precision, recall, F1, AUC, root mean square error, mean absolute error, R-Square, etc. The evaluation indicators can be used to evaluate the intelligent prediction model, and can also be used to evaluate various machine learning models. In this embodiment, the support vector machine, random forest, and multi-layer perceptron are evaluated using the evaluation indicators of accuracy, precision, recall, and F1, and the evaluation data obtained are shown in the following table:

[0072]

[0073] In this embodiment, medical researchers can also provide evaluation feedback based on the output of the machine learning model, describing which machine learning models have better performance and higher accuracy during use, thereby obtaining feedback data. The model prediction results and statistical results can be visualized on the computer device.

[0074] Among them, medical researchers can "like / rate" different model results and different statistical methods, add task descriptions for different tasks, and record the scientific research results they focus on, thereby forming feedback data.

[0075] Among them, a single multimodal model, such as one based on a deep neural network or Transformer architecture, can learn text, numerical, and image type information. That is, the multimodal model uses a continuous learning method to learn a large amount of historical scientific research records of users. For example, the multimodal model can learn the relationships shown in the following table:

[0076]

[0077]

[0078] Among them, scientific research methods, description methods, and F1 performance are medical input data; performance ranking and user feedback are predictive output data.

[0079] Step 210, storing the intelligent prediction model, correlation parameters, evaluation data, and feedback data in a medical database.

[0080] After obtaining the evaluation data of the model and the user's feedback data on the model, the intelligent prediction model, correlation parameters, evaluation data, and feedback data can be stored together in the medical database to form historical data in the medical database, which is convenient for searching medical input data and predicted output data next time.

[0081] In this embodiment, medical researchers can view and compare historical training records and adjust their research plans based on the results of past medical input data and predicted output data.

[0082] In one embodiment, a provided intelligent scientific research method for medical data based on a multimodal model and continuous learning may also include a process of editing input and output data and observing model output results, and the specific process includes: obtaining editing instructions, editing medical input data and predicted output data according to the editing instructions, and obtaining edited medical input data and predicted output data; inputting the edited medical input data and predicted output data into a machine learning model for model training and data analysis, and obtaining an updated intelligent prediction model and the latest correlation parameters between the edited medical input data and predicted output data; using evaluation indicators to evaluate the updated intelligent prediction model to obtain updated evaluation data; and obtaining the latest feedback data for the updated intelligent prediction model; storing the updated intelligent prediction model, the latest correlation parameters, the updated evaluation data, and the latest feedback data in a medical database.

[0083] Medical researchers can add and delete rows and columns of input and output in the current research task, add descriptions, summaries, and summaries to the research task results, that is, edit the medical input data and predicted output data, and input the edited medical input data and predicted output data into the machine learning model for model training and data analysis, and provide user feedback for different model methods. Medical researchers can highlight, like, or dislike based on the data analysis results and model training results to provide reference information for subsequent scientific research work.

[0084] Specifically, the computer device can use multimodal model technology to learn the column names, data descriptions, data volumes, data samples, task types, statistical data, model methods, prediction result task descriptions, and "likes / ratings" of historical medical input data and predicted output data, and can provide appropriate machine learning model and statistical method recommendations, similar historical task recommendations, and "medical input data, predicted output data" column name recommendations for new scientific research tasks. In this embodiment, as medical researchers continue to add training tasks and evaluation operations, the multimodal recommendation model is updated with continuous learning technology to provide users with more accurate customized similar recommendations, which can improve the efficiency of scientific research.

[0085] The medical data intelligent scientific research method based on multimodal model and continuous learning provided in the present application learns and fits a large amount of scientific research method input and output (i.e. medical input data, predicted output data), model, training, and prediction result knowledge through continuous learning and continuous updating. When a medical researcher sets a new scientific research task, based on the input and output, task description, etc. given by the new task, the multimodal model plays the role of a recommendation model, and uses the contextual semantic information learned to obtain the low-dimensional dense representation of "{scientific research method, description}...{evaluation index}..." of the new task and the "{scientific research method, description}...{method}...{evaluation index}...{performance ranking}" of those tasks and methods with better performance and better user feedback in the historical scientific research information in the multimodal model, and then uses cosine similarity to calculate their similarity; then recommend several model methods with high similarity and better performance to medical researchers, without trying models with poor performance to train and predict, saving time and cost.

[0086] In this embodiment, the multimodal recommendation model can also make similar task recommendations, "input and output" column name recommendations, and can view historical scientific research tasks and results at any time to provide ideas and guidance for subsequent scientific research tasks.

[0087] An intelligent scientific research method for medical data based on multimodal models and continuous learning provided in this application, by adding functions such as "data type notes, column notes, task description, and scoring / evaluation system after analysis or machine learning training", allows the success and trial and error of medical researchers to be recorded. The "input and output" analysis and machine learning training based on multiple attempts and historical accumulation of medical researchers can be accumulated for future researchers, future newly added data types / columns, and newly added scientific research tasks.

[0088] Furthermore, by using multimodal models and continuous learning technology, we can learn the causal relationship of success and trial and error records and make intelligent recommendations on "inputs, outputs, task types, and task descriptions." When medical researchers try to study certain "inputs and outputs," the multimodal model can intelligently recommend some "inputs and outputs," analysis, or machine learning models to them based on its current analysis and training results.

[0089] The present application provides an intelligent scientific research method for medical data based on multimodal models and continuous learning. On the one hand, it provides a variety of machine learning models that can adapt to the "text, images, tables and other data" of medical units. The model can be single-modal or multi-modal; but more importantly, it uses intelligent recommendation (another separate multimodal model is required to learn the above-mentioned "input, output, task, training results") to intelligently recommend the "input, output, model" expected by the researcher, so as to find suitable analysis and machine learning models more quickly and improve the efficiency of doctors in scientific research.

[0090] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0091] In one embodiment, Figure 3 As shown, a medical data intelligent scientific research system based on multimodal model and continuous learning is provided, including: a scientific research task adding module 310, an input and output data determination module 320, a model training and data analysis module 330, an evaluation module 340 and a data storage module 350, wherein:

[0092] A scientific research task adding module 310 is used to obtain a task adding instruction, add a scientific research task according to the task adding instruction, and obtain a task description corresponding to the scientific research task;

[0093] An input and output data determination module 320, for searching the medical input data and predicting the output data corresponding to the scientific research task from the medical database according to the task description;

[0094] The model training and data analysis module 330 is used to determine the machine learning model, input the medical input data and the predicted output data into the machine learning model for model training and data analysis, and obtain the intelligent prediction model and the correlation parameters between the medical input data and the predicted output data;

[0095] Evaluation module 340, used to determine evaluation indicators, use the evaluation indicators to evaluate the intelligent prediction model, obtain evaluation data; and obtain feedback data on the intelligent prediction model;

[0096] The data storage module 350 is used to store the intelligent prediction model, correlation parameters, evaluation data, and feedback data in a medical database;

[0097] Among them, the machine learning model is a multimodal model or a unimodal model.

[0098] like Figure 4 As shown, in one embodiment, a medical data intelligent scientific research system based on a multimodal model and continuous learning is provided, which also includes: a database creation module 360, which is used to obtain each input medical data and add a column name and a data description for each medical data; each medical data, as well as the column name and data description corresponding to the medical data are input into a medical database and stored; wherein the format of the medical data includes text, image, video, and table.

[0099] In one embodiment, the input and output data determination module 320 is also used to search the medical database for column names and data descriptions corresponding to the task description as target column names and data descriptions; determine the target medical data according to the target column names and data descriptions as medical input data and predicted output data.

[0100] In one embodiment, the model training and data analysis module 330 is also used to obtain a model selection instruction, and determine a machine learning model according to the model selection instruction; generate a training model data set according to medical input data and predicted output data, and divide the training model data set into a training set and a validation set; and perform model training on the machine learning model based on the training set and the validation set.

[0101] In one embodiment, the machine learning model includes logistic regression, support vector machine, decision tree, random forest, K nearest neighbor, multi-layer perceptron, convolutional neural network, time series model, language model, cross-modal model.

[0102] In one embodiment, the model training and data analysis module 330 is also used to obtain editing instructions, edit the medical input data and predicted output data according to the editing instructions, and obtain the edited medical input data and predicted output data; input the edited medical input data and predicted output data into the machine learning model for model training and data analysis, and obtain an updated intelligent prediction model and the latest correlation parameters between the edited medical input data and predicted output data; use evaluation indicators to evaluate the updated intelligent prediction model to obtain updated evaluation data; and obtain the latest feedback data on the updated intelligent prediction model; store the updated intelligent prediction model, the latest correlation parameters, the updated evaluation data, and the latest feedback data in the medical database.

[0103] In one embodiment, the input-output data determination module 320 is also used to obtain an output task selection instruction, and determine the output task type corresponding to the intelligent prediction model according to the output task selection instruction; the intelligent prediction model outputs the prediction result according to the output task type; wherein the output task type includes classification task and regression task.

[0104] In one embodiment, the evaluation module 340 is also used to determine the corresponding evaluation index according to the output task type; when the output task type is a classification task, the first evaluation index is used to evaluate the intelligent prediction model; when the output task type is a regression task, the second evaluation index is used to evaluate the intelligent prediction model.

[0105] An intelligent medical data research system based on multimodal models and continuous learning provided in this application can reduce the code engineering threshold and time required for different new data and research tasks; and the system facilitates users to systematically view historical research tasks and results at any time, providing ideas and guidance for subsequent research tasks; based on multimodal models and continuous learning technology to learn users' historical research records and results, it provides similar recommendations including models, tasks, and data column names for new research tasks, which is convenient and fast.

[0106] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a medical data intelligent scientific research method based on a multimodal model and continuous learning is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0107] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0108] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of an intelligent scientific research method for medical data based on a multimodal model and continuous learning are implemented.

[0109] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of an intelligent scientific research method for medical data based on a multimodal model and continuous learning are implemented.

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0111] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An intelligent scientific research method for medical data based on multimodal models and continuous learning, characterized in that: The method comprises: Obtaining a task adding instruction, adding a scientific research task according to the task adding instruction, and obtaining a task description corresponding to the scientific research task; According to the task description, searching for medical input data and predicted output data corresponding to the scientific research task from a medical database; Determine a machine learning model, input the medical input data and the predicted output data into the machine learning model for model training and data analysis, and obtain an intelligent prediction model and correlation parameters between the medical input data and the predicted output data; Determine an evaluation index, use the evaluation index to evaluate the intelligent prediction model to obtain evaluation data; and obtain feedback data for the intelligent prediction model; Storing the intelligent prediction model, correlation parameters, evaluation data, and feedback data in the medical database; Wherein, the machine learning model is a multimodal model or a unimodal model.

2. The intelligent scientific research method for medical data based on multimodal model and continuous learning according to claim 1, characterized in that: The method further comprises: Obtain each medical data input, and add a column name and data description for each medical data; Inputting each of the medical data, as well as the column name and data description corresponding to the medical data into the medical database and storing them; The formats of the medical data include text, images, videos, and tables.

3. The intelligent scientific research method for medical data based on multimodal model and continuous learning according to claim 2, characterized in that: According to the task description, searching the medical input data and predicted output data corresponding to the scientific research task from the medical database includes: Searching the medical database for a column name and a data description corresponding to the task description as a target column name and a data description; The target medical data is determined according to the target column name and data description as medical input data and predicted output data.

4. The medical data intelligent scientific research method based on multimodal model and continuous learning according to claim 1, characterized in that: Determine a machine learning model, input the medical input data and predicted output data into the machine learning model for model training and data analysis, including: Obtaining a model selection instruction, and determining a machine learning model according to the model selection instruction; Generate a training model data set according to the medical input data and the predicted output data, and divide the training model data set into a training set and a validation set; The machine learning model is trained based on the training set and the validation set.

5. The medical data intelligent research method based on multimodal model and continuous learning according to claim 1, characterized in that: The machine learning models include logistic regression, support vector machine, decision tree, random forest, K nearest neighbor, multi-layer perceptron, convolutional neural network, time series model, language model, and cross-modal model.

6. The medical data intelligent research method based on multimodal model and continuous learning according to claim 1, characterized in that: The method further comprises: Obtaining an editing instruction, and performing an editing operation on the medical input data and the predicted output data according to the editing instruction to obtain the edited medical input data and the predicted output data; Inputting the edited medical input data and predicted output data into the machine learning model for model training and data analysis to obtain an updated intelligent prediction model and the latest correlation parameters between the edited medical input data and predicted output data; Using the evaluation index to evaluate the updated intelligent prediction model to obtain updated evaluation data; and obtaining the latest feedback data on the updated intelligent prediction model; The updated intelligent prediction model, the latest correlation parameters, the updated evaluation data, and the latest feedback data are stored in the medical database.

7. The medical data intelligent scientific research method based on multimodal model and continuous learning according to claim 1, characterized in that: The method further comprises: Obtaining an output task selection instruction, and determining an output task type corresponding to the intelligent prediction model according to the output task selection instruction; The intelligent prediction model outputs a prediction result according to the output task type; The output task types include classification tasks and regression tasks.

8. The medical data intelligent research method based on multimodal model and continuous learning according to claim 7, characterized in that: Using the evaluation index to evaluate the intelligent prediction model includes: Determine the corresponding evaluation index according to the output task type; When the output task type is a classification task, the intelligent prediction model is evaluated using a first evaluation index; When the output task type is a regression task, a second evaluation index is used to evaluate the intelligent prediction model.

9. A medical data intelligent research system based on multimodal model and continuous learning, characterized in that: The system comprises: A scientific research task adding module, used to obtain a task adding instruction, add a scientific research task according to the task adding instruction, and obtain a task description corresponding to the scientific research task; An input and output data determination module, used to search for medical input data and predicted output data corresponding to the scientific research task from a medical database according to the task description; A model training and data analysis module, used to determine a machine learning model, input the medical input data and the predicted output data into the machine learning model for model training and data analysis, and obtain an intelligent prediction model and correlation parameters between the medical input data and the predicted output data; An evaluation module, used to determine evaluation indicators, use the evaluation indicators to evaluate the intelligent prediction model, obtain evaluation data; and obtain feedback data on the intelligent prediction model; A data storage module, used for storing the intelligent prediction model, correlation parameters, evaluation data, and feedback data into the medical database; Wherein, the machine learning model is a multimodal model or a unimodal model.

10. The medical data intelligent research system based on multimodal model and continuous learning according to claim 9, characterized in that: The system also includes: a database creation module, which is used to obtain each input medical data and add a column name and data description for each medical data; input each medical data and the column name and data description corresponding to the medical data into the medical database and store them; wherein the format of the medical data includes text, image, video, and table.

Images