Prediction method, device, equipment and medium for hepatoma cell morphology
By acquiring and processing the target disease association data of the liver and the three-dimensional reconstruction of the graphical data, inputting it into a preset network model, combining 3D graphics pre-training technology and deep learning technology, the accuracy and explanatory prediction of liver tumors in the existing technology are solved. The problem of sexual inadequacy has achieved higher accuracy in predicting liver tumor cell morphology and improved medical level.
Patent Information
- Application Number
- CN202310953451.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-31
- Publication Date
- 2025-02-14
AI Technical Summary
In the prior art, the liver tumor prediction technology has a low accuracy in the recognition of liver tumor cells and is poorly interpretable, resulting in poor prediction effect.
By obtaining the target data, including the target condition association data of the liver and the target three-dimensional reconstruction target three-dimensional graphical data, it is input to the trained preset network model to output the prediction results of liver tumor cell morphology in the future set period of the target user. This method combines 3D graphics pre-training technology and deep learning technology to improve the accuracy and interpretability of predictions.
It improves the accuracy of prediction of liver tumor cell morphology, enhances the interpretability and comprehensibility of the prediction results, and improves the medical level, especially in weaker hospitals such as rural hospitals.
Smart Images
Figure CN119446476A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of medical data processing, and in particular to a method, device, equipment and medium for predicting liver tumor cell morphology. Background Art
[0002] The morphology of liver tumor cells is the most important and direct basis for diagnosing liver tumor malignancy. In the past, automatic diagnosis and recognition methods mainly used 2D (two-dimensional) imaging liver films to make binary predictions of liver tumor malignancy, which had the following problems: (1) 2D films can only provide partial data, and the accuracy of identifying liver tumor cells is low; (2) Existing liver tumor prediction technologies have poor interpretability; that is, existing liver tumor prediction technologies use CT (computer tomography) images as input and use 2D image processing methods to identify liver tumor cells, which generally have problems such as poor prediction effects. Summary of the invention
[0003] The technical problem to be solved by the present disclosure is to overcome the defect that the existing liver tumor prediction technology in the prior art generally has poor prediction effect for identifying liver tumor cells, and to provide a method, device, equipment and medium for predicting liver tumor cell morphology.
[0004] The present invention solves the above technical problems through the following technical solutions:
[0005] The present disclosure provides a method for predicting liver tumor cell morphology, the prediction method comprising:
[0006] Obtain target data based on the cases to be analyzed by the target users;
[0007] The target data includes target disease association data corresponding to the liver, and target three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the target CT data in the case to be analyzed;
[0008] The target data is input into a preset network model that has been trained to output a target prediction result representing the liver tumor cell morphology of the target user within a future set time period.
[0009] Preferably, the step of training the preset network model includes:
[0010] Based on a number of sample cases of a plurality of sample users, a number of sample training data are obtained;
[0011] Each of the sample training data includes sample disease association data corresponding to the liver within a set historical period, and sample three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the sample CT data in the case to be analyzed, and the sample disease association data includes sample disease description data and sample diagnosis data;
[0012] For each of the sample training data, based on the sample disease association data, obtaining a corresponding sample text feature vector;
[0013] Based on the sample three-dimensional graphic data, obtaining a corresponding sample graphic feature vector;
[0014] Mixing the sample text feature vector and the sample graphic feature vector to obtain a first mixed feature vector;
[0015] Decoding the first mixed feature vector to obtain a first prediction result corresponding to the sample training data;
[0016] When the first prediction results corresponding to the sample training data satisfy a first preset condition, it is determined that the preset network model has been trained.
[0017] Preferably, before the step of obtaining target data based on the case to be analyzed of the target user, the step further includes:
[0018] Constructing a first knowledge graph based on a first mapping relationship between the sample disease description data and the corresponding sample diagnosis data;
[0019] Constructing a second knowledge graph based on a second mapping relationship between the sample disease description data and / or the sample diagnosis data, and the sample three-dimensional graphic data;
[0020] The step of obtaining a corresponding sample graphic feature vector based on the sample three-dimensional graphic data comprises:
[0021] Inputting the first knowledge graph into a preset text pre-training model to obtain text pre-training, and obtaining a first training result of distributed expression;
[0022] Using the preset text pre-training model to process the sample disease association data and the first training result to obtain the corresponding sample text feature vector;
[0023] and / or,
[0024] The step of obtaining a corresponding sample graphic feature vector based on the sample three-dimensional graphic data comprises:
[0025] Inputting the second knowledge graph into a preset three-dimensional graphics pre-training model for graphics and training to obtain a second training result of distributed expression;
[0026] The preset three-dimensional graphic pre-training model is used to process the sample three-dimensional graphic data and the second training result to obtain the corresponding sample graphic feature vector.
[0027] Preferably, the target disease association data includes target disease description data and target diagnosis data;
[0028] The step of inputting the target data into a preset network model that has been trained to output a target prediction result representing the liver tumor cell morphology of the target user within a future set period of time includes:
[0029] Outputting a target text feature vector based on the target disease description data and the first knowledge graph using the preset text pre-training model;
[0030] Outputting a target graphic feature vector based on the target three-dimensional graphic data using the preset three-dimensional graphic pre-training model;
[0031] Mixing the target text feature vector and the target graphic feature vector to obtain a target mixed feature vector;
[0032] The target mixed feature vector is decoded to obtain the target prediction result characterizing the liver tumor cell morphology of the target user within the future set time period.
[0033] Preferably, the target disease association data includes target disease description data, and the prediction method further includes:
[0034] If there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, first diagnostic data corresponding to the first case is obtained based on the first knowledge graph, and the first diagnostic data is used as an auxiliary prediction result characterizing the liver tumor cell morphology of the target user within the future set time period.
[0035] Preferably, the step of using the preset text pre-training model to output a target text feature vector based on the target disease description data and the first knowledge graph includes:
[0036] When there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, the preset text pre-training model is used to output the target text feature vector based on the first case corresponding to the first diagnosis data, the target disease description data, the target diagnosis data and the first knowledge graph.
[0037] Preferably, the step of decoding the target mixed feature vector to obtain the target prediction result characterizing the liver tumor cell morphology of the target user within the future set time period includes:
[0038] A preset decoder is used to decode the target mixed feature vector to obtain the target prediction result; wherein the target prediction result includes the second diagnostic data of the target user in the future set time period and several first prediction results of liver tumor cell morphology in different time periods in the future set time period.
[0039] Preferably, the prediction method further comprises:
[0040] When the second preset condition is met, the super-resolution of the set number of layers is increased to perform three-dimensional reconstruction of the liver, so as to update and obtain new target three-dimensional graphic data;
[0041] and / or,
[0042] The area where the tumor cells are located in the liver is marked based on the target three-dimensional graphic data.
[0043] The present disclosure also provides a prediction device for liver tumor cell morphology, the prediction device comprising:
[0044] A data collector, used to obtain target data based on the cases to be analyzed of target users;
[0045] The target data includes target disease association data corresponding to the liver, and target three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the target CT data in the case to be analyzed;
[0046] A data processor is used to input the target data into a preset network model that has been trained to output a target prediction result representing the liver tumor cell morphology of the target user within a future set time period.
[0047] Preferably, the data processor is further used for:
[0048] Based on a number of sample cases of a plurality of sample users, a number of sample training data are obtained;
[0049] Each of the sample training data includes sample disease association data corresponding to the liver within a set historical period, and sample three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the sample CT data in the case to be analyzed, and the sample disease association data includes sample disease description data and sample diagnosis data;
[0050] Constructing a first knowledge graph based on a first mapping relationship between the sample disease description data and the corresponding sample diagnosis data;
[0051] Constructing a second knowledge graph based on a second mapping relationship between the sample disease description data and / or the sample diagnosis data, and the sample three-dimensional graphic data;
[0052] For each of the sample training data, text pre-training is performed based on the sample disease association data and the first knowledge graph, and a corresponding sample text feature vector is obtained based on a preset text pre-training model;
[0053] Performing image pre-training based on the sample three-dimensional graphic data and the second knowledge graph, and acquiring a corresponding sample graphic feature vector based on a preset three-dimensional graphic pre-training model;
[0054] Mixing the sample text feature vector and the sample graphic feature vector to obtain a first mixed feature vector;
[0055] Decoding the first mixed feature vector to obtain a first prediction result corresponding to the sample training data;
[0056] When the first prediction results corresponding to the sample training data satisfy a first preset condition, it is determined that the preset network model has been trained.
[0057] Preferably, the target disease association data includes target disease description data and target diagnosis data;
[0058] The data processor is also used to:
[0059] Outputting a target text feature vector based on the target disease description data and the first knowledge graph using the preset text pre-training model;
[0060] Outputting a target graphic feature vector based on the target three-dimensional graphic data using the preset three-dimensional graphic pre-training model;
[0061] Mixing the target text feature vector and the target graphic feature vector to obtain a target mixed feature vector;
[0062] The target mixed feature vector is decoded to obtain the target prediction result characterizing the liver tumor cell morphology of the target user within the future set time period.
[0063] Preferably, the target disease association data includes target disease description data, and the data processor is further used for:
[0064] If there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, first diagnostic data corresponding to the first case is obtained based on the first knowledge graph, and the first diagnostic data is used as an auxiliary prediction result characterizing the liver tumor cell morphology of the target user within the future set time period.
[0065] Preferably, the data processor is also used to output the target text feature vector using the preset text pre-training model based on the first case corresponding to the first diagnosis data, the target disease description data, the target diagnosis data and the first knowledge graph when there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold.
[0066] Preferably, the data processor is also used to use a preset decoder to decode the target mixed feature vector to obtain the target prediction result; wherein the target prediction result includes the second diagnostic data of the target user in the future set time period and the first prediction result of liver tumor cell morphology in different time periods.
[0067] Preferably, the data processor is further used for:
[0068] When the second preset condition is met, the super-resolution of the set number of layers is increased to perform three-dimensional reconstruction of the liver, so as to update and obtain new target three-dimensional graphic data;
[0069] and / or,
[0070] The area where the tumor cells are located in the liver is marked based on the target three-dimensional graphic data.
[0071] The present disclosure also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for predicting liver tumor cell morphology when executing the computer program.
[0072] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned method for predicting liver tumor cell morphology is implemented.
[0073] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.
[0074] The positive and progressive effects of this disclosure are:
[0075] In the present disclosure, a 3D reconstruction of the liver is performed based on the 2D target CT data in the electronic medical record to obtain 3D graphic data of the 3D organ of the target user's liver. The 3D graphic data in the 3D reconstruction scene can provide richer feature data and clearer morphological display, so that compared with the existing liver tumor prediction technology, the morphology of liver tumor cells is more interpretable and understandable; and the use of 3D graphics pre-training technology and deep learning technology can predict the morphology of liver tumor cells of any user in a period of time in the future, which can effectively improve the accuracy of liver tumor cell morphology prediction and achieve a higher level of diagnosis of liver tumor cell morphology, which can improve the diagnosis level of liver tumors by doctors in hospitals with weaker strength, such as rural hospitals, thereby improving the overall medical level and ensuring the health of more liver tumor patients. It can effectively improve the prediction accuracy of liver tumor cell morphology and at the same time, can perform 3D annotation of liver tumor cells in 3D reconstructed graphics, providing non-professional oncologists and patients with clearer judgment basis and understanding basis than 2D films, thereby laying a good foundation for explainable intelligent diagnosis; in addition, it uses human-machine collaborative technology to build an expert database to achieve diagnosis sharing and knowledge sharing in all fields and all cases, thereby improving the diagnosis level of liver tumors by doctors in weaker hospitals such as rural hospitals. BRIEF DESCRIPTION OF THE DRAWINGS
[0076] Figure 1 is a first flow chart of the method for predicting liver tumor cell morphology according to Example 1 of the present disclosure;
[0077] Figure 2 is a second flow chart of the method for predicting liver tumor cell morphology according to Example 1 of the present disclosure;
[0078] Figure 3 The third flow chart of the method for predicting liver tumor cell morphology of Example 1 of the present disclosure;
[0079] Figure 4 This is a first schematic diagram of the method for predicting liver tumor cell morphology according to Example 1 of the present disclosure;
[0080] Figure 5 is a fourth flow chart of the method for predicting liver tumor cell morphology of Example 1 of the present disclosure;
[0081] Figure 6 is a second schematic diagram of the method for predicting liver tumor cell morphology according to Example 1 of the present disclosure;
[0082] Figure 7 This is a schematic diagram of a module of a device for predicting liver tumor cell morphology according to Example 2 of the present disclosure;
[0083] Figure 8 This is a schematic diagram of the structure of an electronic device according to Embodiment 3 of the present disclosure. DETAILED DESCRIPTION
[0084] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.
[0085] Example 1
[0086] like Figure 1 As shown, this embodiment provides a method for predicting liver tumor cell morphology, comprising:
[0087] S101, acquiring target data based on the case to be analyzed of the target user;
[0088] The target data includes target disease association data corresponding to the liver, and target three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the target CT data in the case to be analyzed;
[0089] The target disease-related data is text data describing the symptoms of the target user, including but not limited to symptom description data, expert diagnosis data and other data expressed in text form; the symptom description text includes blood tests, urine tests, external conditions, etc. at different time periods; the expert diagnosis data includes the disease judgment and treatment measures made based on the symptom description data, such as what treatment method to use.
[0090] Specifically, the symptom description data and expert diagnosis data can be extracted according to the format of the electronic medical record through pattern matching, rule definition and other methods. The target symptom association data and the target three-dimensional graphic data are both time-series data. The corresponding symptom description data, expert diagnosis data and target three-dimensional graphic data are associated one by one through time-series information. Time-series data can better preserve the dynamic changes of the patient's symptoms at different times or periods.
[0091] Based on the medical platform that shares diagnosis and knowledge in all fields and all cases, the electronic medical records to be analyzed of target users are obtained to ensure the efficiency and comprehensiveness of case acquisition, thereby ensuring the accuracy of subsequent predictive processing.
[0092] S102, inputting the target data into the trained preset network model to output a target prediction result representing the liver tumor cell morphology of the target user within a set time period in the future.
[0093] Among them, the target prediction result belongs to the analysis data of the intermediate state of liver tumor cell morphology, which is used for reference by specific personnel such as doctors, patient users, etc.
[0094] In this solution, the liver is 3D reconstructed based on the 2D target CT data in the electronic medical record to obtain 3D graphic data of the 3D organ of the target user's liver. The 3D graphic data in the 3D reconstruction scenario can provide richer feature data and clearer morphological display, making the liver tumor cell morphology more interpretable and understandable than the existing liver tumor prediction technology; and the use of 3D graphics pre-training technology and deep learning technology to predict the morphology of liver tumor cells of any user in the future can effectively improve the accuracy of liver tumor cell morphology prediction and achieve a higher level of diagnosis of liver tumor cell morphology, which can improve the diagnosis level of liver tumors by doctors in hospitals with weaker strength, such as rural hospitals, thereby improving the overall medical level and ensuring the health of more liver tumor patients.
[0095] In one feasible solution, Figure 2 As shown, the steps of training the preset network model include:
[0096] S201, obtaining a plurality of sample training data based on a plurality of sample cases of a plurality of sample users;
[0097] Among them, each sample training data includes sample symptom association data corresponding to the liver within a set historical period, and sample three-dimensional graphic data corresponding to the liver reconstructed based on the sample CT data in the case to be analyzed. The sample symptom association data and the sample three-dimensional graphic data are both time-series data, and the sample symptom association data include sample symptom description data and sample diagnosis data; the corresponding sample symptom description data, sample diagnosis data and sample three-dimensional graphic data are associated one-to-one through time series information, and the time series data can better retain the dynamic changes of the patient's symptoms at different times or time periods, and retain more information for model training.
[0098] Specifically, for example, cases of normal patients and liver tumor patients are collected; wherein, the cases of normal patients are collected by randomly selecting 1,000 cases from patients who have never had liver tumors and have undergone CT scans as negative sample cases, and collecting 1,000 cases from patients with liver tumors as positive sample data.
[0099] Data extraction is performed on the collected positive and negative sample cases, including symptom data, expert diagnosis data, and 3D reconstruction data of CT data. Symptom data mainly includes blood tests, urine tests, external status data at different time periods; expert diagnosis data includes the judgment of the disease and treatment measures made by experts based on symptom data, such as what kind of treatment is needed; 3D reconstruction data of CT data is obtained by 3D reconstruction of CT data based on the patient's early liver tumor using preset multimodal medical imaging processing software. Symptom data and expert diagnosis data are mainly based on the patient's electronic medical records, and data are extracted according to the format of the electronic medical records through pattern matching, rule definition and other methods.
[0100] S202, for each sample training data, based on the sample disease association data, obtain the corresponding sample text feature vector;
[0101] S203, based on the sample three-dimensional graphic data, obtaining a corresponding sample graphic feature vector;
[0102] Among them, the sample disease association data and the sample three-dimensional graphic data are feature vectorized to facilitate subsequent model training processing.
[0103] S204, mixing the sample text feature vector and the sample graphic feature vector to obtain a first mixed feature vector;
[0104] S205, decoding the first mixed feature vector to obtain a first prediction result corresponding to the sample training data;
[0105] For example, a Seq2Seq (an encoding-decoding structure) decoder is used to decode the first mixed feature vector to obtain a decoded feature vector including the expert diagnosis result and the predicted liver tumor cell morphology within a period of time in the future.
[0106] S206: When the first prediction results corresponding to the plurality of sample training data satisfy the first preset condition, it is determined that the preset network model has been trained.
[0107] Among them, the first preset condition includes that for each sample training data, if the difference between the first prediction result and the reference liver tumor cell morphology is less than a set threshold, it is determined that the accuracy of the current network model meets the requirements; otherwise, it is necessary to continue iterative training until a preset network model that meets the requirements is obtained.
[0108] Specifically, the expert diagnosis results in the first prediction result and the future cell morphology of liver cells are jointly learned and trained. The cross entropy is used to calculate the difference between the predicted output and the standard result as the loss function. The gradient descent method is used to update the parameters. The above model is constructed based on PyTorch (a model). The vector size of the vectorized representation is 256, and the hidden layer size is 128. The final output of the model includes two parts: expert diagnosis generation and liver 3D data. When appropriate, a layer of super-resolution reconstruction can be added to improve the clarity and resolution of the generated 3D graphic data, thereby improving the final prediction accuracy and reliability of the liver tumor cell morphology.
[0109] This solution is a liver tumor cell morphology prediction technology based on three-dimensional reconstruction. The three-dimensional reconstruction data of the liver of thousands of actual tumor patients and the doctor's diagnosis are used as training data to build an expert diagnosis-cell morphology library. Combined with 3D graphics pre-training technology and deep learning technology, the future morphology of liver tumor cells is predicted, which can effectively improve the prediction accuracy of liver tumor cell morphology.
[0110] In one feasible solution, Figure 3 As shown, before step S101, it also includes:
[0111] Constructing a first knowledge graph based on a first mapping relationship between the sample disease description data and the corresponding sample diagnosis data;
[0112] Constructing a second knowledge graph based on a second mapping relationship between the sample disease description data and / or the sample diagnosis data and the sample three-dimensional graphic data;
[0113] Step S202 includes:
[0114] S2021. Input the first knowledge graph into a preset text pre-training model to obtain text pre-training, and obtain a first training result of distributed expression;
[0115] Among them, distributed expression of data belongs to existing mature technology and will not be elaborated here.
[0116] S2022, using a preset text pre-training model to process the sample disease association data and the first training result to obtain a corresponding sample text feature vector;
[0117] Step S203 includes:
[0118] S2031, inputting the second knowledge graph into a preset three-dimensional graphics pre-training model for graphics and training to obtain a second training result of distributed expression;
[0119] S2032: Use a preset three-dimensional graphic pre-training model to process the sample three-dimensional graphic data and the second training result to obtain a corresponding sample graphic feature vector.
[0120] Specifically, the preset text pre-training model includes but is not limited to the BERT model (bidirectional encoder representation from transformer).
[0121] The preset 3D graphics pre-trained model includes but is not limited to the Vision Transformer model (a network model).
[0122] Text pre-training is performed on the basis of BERT using expert knowledge graph as input, and image pre-training is performed on the basis of Vision Transformer using 3D graphic knowledge graph as input. Pre-training is an operation independent of the training process, which is used to convert text or graphics into distributed expressions. The pre-training process helps to achieve better results in small sample training, and then ensure the accuracy of the prediction results in subsequent actual prediction scenarios.
[0123] Data extraction is performed on the collected positive and negative sample cases, including symptom data, expert diagnosis data, 3D reconstruction data of CT data, etc. The three types of data are automatically associated through the same patient individual, thus forming relationship pairs of (symptom data, expert diagnosis data), (symptom data, 3D graphics data), and (3D graphics data, expert diagnosis data).
[0124] like Figure 4 As shown, based on the above-obtained (symptom data, expert diagnosis data) relationship pairs, a first knowledge graph or expert diagnosis knowledge graph is constructed, which can quickly locate the relevant diagnosis of experts based on the patient's symptoms; similarly, based on the association mapping of (symptom data, 3D graphics data) and (3D graphics data, expert diagnosis data), a second knowledge graph or 3D graphics knowledge graph is constructed; the knowledge graph finally formed containing multiple modal information is a multimodal knowledge graph, and the above-mentioned multimodal knowledge graph provides basic functions such as similar case query and expert diagnosis basis query.
[0125] In this solution, 3D graphics pre-training technology is used to pre-train the 3D organ of the patient's liver, that is, 3D graphics pre-training technology and deep learning technology are used to predict its future morphology, which can effectively improve the prediction accuracy of liver tumor cell morphology.
[0126] In one feasible solution, Figure 5 As shown, the target disease association data includes target disease description data and target diagnosis data.
[0127] Step S102 includes:
[0128] S1021. Output a target text feature vector based on the target disease description data and the first knowledge graph using a preset text pre-training model;
[0129] S1022, using a preset three-dimensional graphics pre-training model to output a target graphics feature vector based on target three-dimensional graphics data;
[0130] S1023, mixing the target text feature vector and the target graphic feature vector to obtain a target mixed feature vector;
[0131] The hybrid processing includes but is not limited to serially concatenating the target text feature vector and the target graphic feature vector to obtain a target hybrid feature vector.
[0132] S1024, decoding the target mixed feature vector to obtain a target prediction result representing the liver tumor cell morphology of the target user within a set future time period.
[0133] In this solution, 3D graphics, training technology, and deep learning technology are used to predict the liver tumor cell morphology of the target user in the future, thereby effectively improving the prediction accuracy of liver tumor cell morphology.
[0134] In one feasible solution, the target disease association data includes target disease description data, and the prediction method further includes:
[0135] If there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, the first diagnostic data corresponding to the first case is obtained based on the first knowledge graph, and the first diagnostic data is used as an auxiliary prediction result characterizing the liver tumor cell morphology of the target user within a set time period in the future.
[0136] In this solution, a certain number of cases are stored in the preset knowledge base, and these cases are matched with corresponding expert diagnosis results; corresponding to the target symptom description data of the target user, according to the similarity of the patient's symptoms, similarity query is performed on the data in the preset knowledge base to obtain expert diagnosis data, and the similarity threshold is determined according to the average similarity of the symptoms between patients. When the similarity is greater than the threshold, the expert diagnosis data is returned, otherwise it returns empty. When empty is returned, the values of the vectors in the next step of pre-training vectorization are all set to 0, that is, when there are no similar cases in the preset knowledge base, the expert diagnosis data will not be used; when there are similar cases in the preset knowledge base, the expert diagnosis data of the similar cases will be used as auxiliary features to assist doctors in predicting the liver tumor cell morphology of the target user within a set period of time in the future, so as to improve the prediction accuracy.
[0137] In an implementable solution, a preset text pre-training model is used based on the target disease description data and the first knowledge graph, and step S1021 includes:
[0138] When there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, a preset text pre-training model is used to output a target text feature vector based on the first diagnosis data corresponding to the first case, the target disease description data, the target diagnosis data and the first knowledge graph.
[0139] In this solution, when a case similar to the target user's disease is found in the preset knowledge base, the expert diagnosis result corresponding to the first case found is used as the input of the preset text pre-training model, further improving the accuracy of the final prediction result.
[0140] In one feasible solution, step S1024 includes:
[0141] A preset decoder is used to decode the target mixed feature vector to obtain a target prediction result; wherein the target prediction result includes the second diagnosis data of the target user in a future set time period and the first prediction result of the liver tumor cell morphology in different time periods.
[0142] In this scheme, an example is given, such as Figure 6 As shown, the preset decoder decodes corresponding to four decoding modules LSTM (long short-term memory network). Of course, the number of decoding modules can be adjusted according to actual needs. After the preset decoder decodes and processes, expert diagnosis data, liver tumor cell morphology 1, liver tumor cell morphology 2, and liver tumor cell morphology 3 corresponding to the target user in different time periods in the future are obtained respectively, so as to obtain the predicted structure of liver tumor cell morphology more clearly, accurately and reliably.
[0143] In one feasible solution, the prediction method further includes:
[0144] When the second preset condition is met, the super-resolution of the set number of layers is increased to perform three-dimensional reconstruction of the liver, so as to update and obtain new target three-dimensional graphic data;
[0145] In this scheme, a layer of super-resolution reconstruction can be added when appropriate to improve the clarity and resolution of the generated 3D graphic data, thereby improving the final prediction accuracy and reliability of the liver tumor cell morphology.
[0146] In one feasible solution, the prediction method further includes:
[0147] The area where the tumor cells are located in the liver is marked based on the target three-dimensional graphic data.
[0148] In this solution, liver tumor cells can be annotated in 3D, providing non-professional oncologists and patients with a clearer basis for judgment and understanding than 2D films, thereby laying a good foundation for explainable intelligent diagnosis.
[0149] Example 2
[0150] like Figure 7 As shown, the present disclosure also provides a device for predicting liver tumor cell morphology, specifically comprising:
[0151] A data collector 1 is used to obtain target data based on the case to be analyzed of the target user;
[0152] The target data includes target disease association data corresponding to the liver, and target three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the target CT data in the case to be analyzed;
[0153] The target disease-related data is text data describing the symptoms of the target user, including but not limited to symptom description data, expert diagnosis data and other data expressed in text form; the symptom description text includes blood tests, urine tests, external conditions, etc. at different time periods; the expert diagnosis data includes the disease judgment and treatment measures made based on the symptom description data, such as what treatment method to use.
[0154] Specifically, the symptom description data and expert diagnosis data can be extracted according to the format of the electronic medical records through methods such as pattern matching and rule definition. The target symptom association data and the target three-dimensional graphic data are both time-series data. The corresponding symptom description data, expert diagnosis data and target three-dimensional graphic data are associated one by one through time-series information. Time-series data can better preserve the dynamic changes of the patient's symptoms.
[0155] Based on the medical platform that shares diagnosis and knowledge in all fields and all cases, the electronic medical records to be analyzed of target users are obtained to ensure the efficiency and comprehensiveness of case acquisition, thereby ensuring the accuracy of subsequent predictive processing.
[0156] The data processor 2 is used to input the target data into the trained preset network model to output the target prediction result representing the liver tumor cell morphology of the target user within a future set time period.
[0157] Among them, the target prediction result belongs to the analysis data of the intermediate state of liver tumor cell morphology, which is used for reference by specific personnel such as doctors, patient users, etc.
[0158] In this solution, the liver is 3D reconstructed based on the 2D target CT data in the electronic medical record to obtain 3D graphic data of the 3D organ of the target user's liver. The 3D graphic data in the 3D reconstruction scenario can provide richer feature data and clearer morphological display, making the liver tumor cell morphology more interpretable and understandable than the existing liver tumor prediction technology; and the use of 3D graphics pre-training technology and deep learning technology to predict the morphology of liver tumor cells of any user in the future can effectively improve the accuracy of liver tumor cell morphology prediction and achieve a higher level of diagnosis of liver tumor cell morphology, which can improve the diagnosis level of liver tumors by doctors in hospitals with weaker strength, such as rural hospitals, thereby improving the overall medical level and ensuring the health of more liver tumor patients.
[0159] In one feasible solution, the data processor 2 is further configured to:
[0160] Based on a number of sample cases of a plurality of sample users, a number of sample training data are obtained;
[0161] Among them, each sample training data includes sample symptom association data corresponding to the liver within a set historical period, and sample three-dimensional graphic data corresponding to the liver reconstructed based on the sample CT data in the case to be analyzed. The sample symptom association data and the sample three-dimensional graphic data are both time-series data, and the sample symptom association data include sample symptom description data and sample diagnosis data; the corresponding sample symptom description data, sample diagnosis data and sample three-dimensional graphic data are associated one-to-one through time series information, and the time series data can better retain the dynamic changes of the patient's symptoms at different times or time periods, and retain more information for model training.
[0162] Specifically, for example, cases of normal patients and liver tumor patients are collected; wherein, the cases of normal patients are collected by randomly selecting 1,000 cases from patients who have never had liver tumors and have undergone CT scans as negative sample cases, and collecting 1,000 cases from patients with liver tumors as positive sample data.
[0163] Data extraction is performed on the collected positive and negative sample cases, including symptom data, expert diagnosis data, and 3D reconstruction data of CT data. Symptom data mainly includes blood tests, urine tests, external status data at different time periods; expert diagnosis data includes the judgment of the disease and treatment measures made by experts based on symptom data, such as what kind of treatment is needed; 3D reconstruction data of CT data is obtained by 3D reconstruction of CT data based on the patient's early liver tumor using preset multimodal medical imaging processing software. Symptom data and expert diagnosis data are mainly based on the patient's electronic medical records, and data are extracted according to the format of the electronic medical records through pattern matching, rule definition and other methods.
[0164] For each sample training data, based on the sample disease association data, obtain the corresponding sample text feature vector;
[0165] Based on the sample three-dimensional graphic data, obtaining a corresponding sample graphic feature vector;
[0166] Among them, the sample disease association data and the sample three-dimensional graphic data are feature vectorized to facilitate subsequent model training processing.
[0167] Mixing the sample text feature vector and the sample graphic feature vector to obtain a first mixed feature vector;
[0168] Decoding the first mixed feature vector to obtain a first prediction result corresponding to the sample training data;
[0169] For example, a Seq2Seq (an encoding-decoding structure) decoder is used to decode the first mixed feature vector to obtain a decoded feature vector including the expert diagnosis result and the predicted liver tumor cell morphology within a period of time in the future.
[0170] When the first prediction results corresponding to the sample training data satisfy the first preset condition, it is determined that the preset network model has been trained.
[0171] Among them, the first preset condition includes that for each sample training data, if the difference between the first prediction result and the reference liver tumor cell morphology is less than a set threshold, it is determined that the accuracy of the current network model meets the requirements; otherwise, it is necessary to continue iterative training until a preset network model that meets the requirements is obtained.
[0172] Specifically, the expert diagnosis results in the first prediction result and the future cell morphology of liver cells are jointly learned and trained. The cross entropy is used to calculate the difference between the predicted output and the standard result as the loss function. The gradient descent method is used to update the parameters. The above model is built based on PyTorch. The vector size of the vectorized representation is 256, and the hidden layer size is 128. The final output of the model includes two parts: expert diagnosis generation and liver 3D data. When appropriate, a layer of super-resolution reconstruction can be added to improve the clarity and resolution of the generated 3D graphic data, thereby improving the final prediction accuracy and reliability of the liver tumor cell morphology.
[0173] This solution is a liver tumor cell morphology prediction technology based on three-dimensional reconstruction. It uses the three-dimensional reconstruction data of the liver of thousands of actual tumor patients and the doctors' diagnosis as training data to build an expert diagnosis-cell morphology library. It combines 3D graphics pre-training technology and deep learning technology to predict the future morphology of liver tumor cells, which can effectively improve the prediction accuracy of liver tumor cell morphology.
[0174] In one feasible solution, the data processor 2 is further configured to:
[0175] Constructing a first knowledge graph based on a first mapping relationship between the sample disease description data and the corresponding sample diagnosis data;
[0176] Constructing a second knowledge graph based on a second mapping relationship between the sample disease description data and / or the sample diagnosis data and the sample three-dimensional graphic data;
[0177] Inputting the first knowledge graph into a preset text pre-training model to obtain text pre-training, and obtaining a first training result of distributed expression;
[0178] Using a preset text pre-training model to process the sample disease association data and the first training result to obtain a corresponding sample text feature vector;
[0179] The data processor 2 is also used for:
[0180] Inputting the second knowledge graph into a preset three-dimensional graph pre-training model for graph and training to obtain a second training result of distributed expression;
[0181] The sample three-dimensional graphic data and the second training result are processed using a preset three-dimensional graphic pre-training model to obtain a corresponding sample graphic feature vector.
[0182] Specifically, the preset text pre-training model includes but is not limited to the BERT model.
[0183] The preset 3D graphics pre-trained models include but are not limited to the Vision Transformer model.
[0184] Text pre-training is performed on the basis of BERT using expert knowledge graph as input, and image pre-training is performed on the basis of Vision Transformer using 3D graphic knowledge graph as input. Pre-training is an operation independent of the training process, which is used to convert text or graphics into distributed expressions. The pre-training process helps to achieve better results in small sample training, and then ensure the accuracy of the prediction results in subsequent actual prediction scenarios.
[0185] Data extraction is performed on the collected positive and negative sample cases, including symptom data, expert diagnosis data, 3D reconstruction data of CT data, etc. The three types of data are automatically associated through the same patient individual, thus forming relationship pairs of (symptom data, expert diagnosis data), (symptom data, 3D graphics data), and (3D graphics data, expert diagnosis data).
[0186] Based on the above-obtained (symptom data, expert diagnosis data) relationship pairs, a first knowledge graph or expert diagnosis knowledge graph is constructed, which can quickly locate the relevant diagnosis of experts based on the patient's symptoms; similarly, based on the association mapping of (symptom data, 3D graphics data) and (3D graphics data, expert diagnosis data), a second knowledge graph or 3D graphics knowledge graph is constructed; the knowledge graph finally formed containing multiple modal information is a multimodal knowledge graph, which provides basic functions such as similar case query and expert diagnosis basis query.
[0187] In this solution, 3D graphics pre-training technology is used to pre-train the 3D organ of the patient's liver, that is, 3D graphics pre-training technology and deep learning technology are used to predict its future morphology, which can effectively improve the prediction accuracy of liver tumor cell morphology.
[0188] In one practicable solution, the target disease-related data includes target disease description data and target diagnosis data;
[0189] The data processor 2 is also used for:
[0190] A preset text pre-training model is used to output a target text feature vector based on the target disease description data and the first knowledge graph;
[0191] Using a preset 3D graphics pre-training model to output a target graphics feature vector based on target 3D graphics data;
[0192] Mixing the target text feature vector and the target graphic feature vector to obtain a target mixed feature vector;
[0193] The hybrid processing includes but is not limited to serially concatenating the target text feature vector and the target graphic feature vector to obtain a target hybrid feature vector.
[0194] The target mixed feature vector is decoded to obtain a target prediction result representing the liver tumor cell morphology of the target user within a set time period in the future.
[0195] In this solution, 3D graphics, training technology, and deep learning technology are used to predict the liver tumor cell morphology of the target user in the future, thereby effectively improving the prediction accuracy of liver tumor cell morphology.
[0196] In an implementable solution, the target disease association data includes target disease description data, and the data processor 2 is further used for:
[0197] If there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, the first diagnostic data corresponding to the first case is obtained based on the first knowledge graph, and the first diagnostic data is used as an auxiliary prediction result characterizing the liver tumor cell morphology of the target user within a set time period in the future.
[0198] In this solution, a certain number of cases are stored in the preset knowledge base, and these cases are matched with corresponding expert diagnosis results; corresponding to the target symptom description data of the target user, according to the similarity of the patient's symptoms, similarity query is performed on the data in the preset knowledge base to obtain expert diagnosis data, and the similarity threshold is determined according to the average similarity of the symptoms between patients. When the similarity is greater than the threshold, the expert diagnosis data is returned, otherwise it returns empty. When empty is returned, the values of the vectors in the next step of pre-training vectorization are all set to 0, that is, when there are no similar cases in the preset knowledge base, the expert diagnosis data will not be used; when there are similar cases in the preset knowledge base, the expert diagnosis data of the similar cases will be used as auxiliary features to assist doctors in predicting the liver tumor cell morphology of the target user within a set period of time in the future, so as to improve the prediction accuracy.
[0199] In an implementable scheme, the data processor 2 is also used to output the target text feature vector based on the first diagnosis data corresponding to the first case, the target disease description data, the target diagnosis data and the first knowledge graph using a preset text pre-training model when there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold.
[0200] In this solution, when a case similar to the target user's disease is found in the preset knowledge base, the expert diagnosis result corresponding to the first case found is used as the input of the preset text pre-training model, further improving the accuracy of the final prediction result.
[0201] In an implementable scheme, the data processor 2 is also used to use a preset decoder to decode the target mixed feature vector to obtain a target prediction result; wherein the target prediction result includes the second diagnostic data of the target user in a future set time period and the first prediction result of liver tumor cell morphology in different time periods.
[0202] In this scheme, an example is given, such as Figure 6 As shown, the preset decoder decodes corresponding to four decoding modules LSTM. Of course, the number of decoding modules can be adjusted according to actual needs. After the preset decoder decodes and processes, expert diagnosis data, liver tumor cell morphology 1, liver tumor cell morphology 2, and liver tumor cell morphology 3 corresponding to the target user in different time periods in the future are obtained respectively, so as to obtain the predicted structure of liver tumor cell morphology more clearly, accurately and reliably.
[0203] In one feasible solution, the data processor 2 is further configured to:
[0204] When the second preset condition is met, the super-resolution of the set number of layers is increased to perform three-dimensional reconstruction of the liver, so as to update and obtain new target three-dimensional graphic data;
[0205] In this scheme, a layer of super-resolution reconstruction can be added when appropriate to improve the clarity and resolution of the generated 3D graphic data, thereby improving the final prediction accuracy and reliability of the liver tumor cell morphology.
[0206] In one feasible solution, the area where the tumor cells are located in the liver is marked based on the target three-dimensional graphic data.
[0207] In this solution, liver tumor cells can be annotated in 3D, providing non-professional oncologists and patients with a clearer basis for judgment and understanding than 2D films, thereby laying a good foundation for explainable intelligent diagnosis.
[0208] Example 3
[0209] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the method in the above embodiment is implemented when the processor executes the program. Figure 8 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0210] like Figure 8 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).
[0211] The bus 33 includes a data bus, an address bus, and a control bus.
[0212] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .
[0213] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0214] The processor 31 executes various functional applications and data processing by running the computer programs stored in the memory 32, such as the method in the above-mentioned embodiment of the present disclosure.
[0215] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 8 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.
[0216] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.
[0217] Example 4
[0218] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method in the above embodiment are implemented.
[0219] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.
[0220] In a possible implementation manner, the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps in the method of the above embodiment.
[0221] Among them, the program code for executing the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.
[0222] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.
Claims
1. A method for predicting liver tumor cell morphology, characterized in that: The prediction method comprises: Obtain target data based on the cases to be analyzed by the target users; The target data includes target disease association data corresponding to the liver, and target three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the target CT data in the case to be analyzed; The target data is input into a preset network model that has been trained to output a target prediction result representing the liver tumor cell morphology of the target user within a future set time period.
2. The method for predicting liver tumor cell morphology according to claim 1, characterized in that: The step of training the preset network model includes: Based on a number of sample cases of a plurality of sample users, a number of sample training data are obtained; Each of the sample training data includes sample disease association data corresponding to the liver within a set historical period, and sample three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the sample CT data in the case to be analyzed, and the sample disease association data includes sample disease description data and sample diagnosis data; For each of the sample training data, based on the sample disease association data, obtaining a corresponding sample text feature vector; Based on the sample three-dimensional graphic data, obtaining a corresponding sample graphic feature vector; Mixing the sample text feature vector and the sample graphic feature vector to obtain a first mixed feature vector; Decoding the first mixed feature vector to obtain a first prediction result corresponding to the sample training data; When the first prediction results corresponding to the sample training data satisfy a first preset condition, it is determined that the preset network model has been trained.
3. The method for predicting liver tumor cell morphology according to claim 2, characterized in that: Before the step of obtaining target data based on the case to be analyzed of the target user, the method further includes: Constructing a first knowledge graph based on a first mapping relationship between the sample disease description data and the corresponding sample diagnosis data; Constructing a second knowledge graph based on a second mapping relationship between the sample disease description data and / or the sample diagnosis data, and the sample three-dimensional graphic data; The step of obtaining a corresponding sample graphic feature vector based on the sample three-dimensional graphic data comprises: Inputting the first knowledge graph into a preset text pre-training model to obtain text pre-training, and obtaining a first training result of distributed expression; Using the preset text pre-training model to process the sample disease association data and the first training result to obtain the corresponding sample text feature vector; and / or, The step of obtaining a corresponding sample graphic feature vector based on the sample three-dimensional graphic data comprises: Inputting the second knowledge graph into a preset three-dimensional graphics pre-training model for graphics and training to obtain a second training result of distributed expression; The preset three-dimensional graphic pre-training model is used to process the sample three-dimensional graphic data and the second training result to obtain the corresponding sample graphic feature vector.
4. The method for predicting liver tumor cell morphology according to claim 3, characterized in that: The target disease association data includes target disease description data and target diagnosis data; The step of inputting the target data into a preset network model that has been trained to output a target prediction result representing the liver tumor cell morphology of the target user within a future set period of time includes: Outputting a target text feature vector based on the target disease description data and the first knowledge graph using the preset text pre-training model; Outputting a target graphic feature vector based on the target three-dimensional graphic data using the preset three-dimensional graphic pre-training model; Mixing the target text feature vector and the target graphic feature vector to obtain a target mixed feature vector; The target mixed feature vector is decoded to obtain the target prediction result characterizing the liver tumor cell morphology of the target user within the future set time period.
5. The method for predicting liver tumor cell morphology according to claim 4, characterized in that: The target disease association data includes target disease description data, and the prediction method further includes: If there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, first diagnostic data corresponding to the first case is obtained based on the first knowledge graph, and the first diagnostic data is used as an auxiliary prediction result characterizing the liver tumor cell morphology of the target user within the future set time period.
6. The method for predicting liver tumor cell morphology according to claim 5, characterized in that: The step of using the preset text pre-training model to output a target text feature vector based on the target disease description data and the first knowledge graph includes: When there is a first case in the preset knowledge base whose similarity with the target disease description data is greater than a set threshold, the preset text pre-training model is used to output the target text feature vector based on the first case corresponding to the first diagnosis data, the target disease description data, the target diagnosis data and the first knowledge graph.
7. The method for predicting liver tumor cell morphology according to claim 6, characterized in that: The step of decoding the target mixed feature vector to obtain the target prediction result characterizing the liver tumor cell morphology of the target user within the future set time period includes: A preset decoder is used to decode the target mixed feature vector to obtain the target prediction result; wherein the target prediction result includes the second diagnostic data of the target user in the future set time period and several first prediction results of liver tumor cell morphology in different time periods in the future set time period.
8. The method for predicting liver tumor cell morphology according to any one of claims 1 to 7, characterized in that: The prediction method further comprises: When the second preset condition is met, the super-resolution of the set number of layers is increased to perform three-dimensional reconstruction of the liver, so as to update and obtain new target three-dimensional graphic data; and / or, The area where the tumor cells are located in the liver is marked based on the target three-dimensional graphic data.
9. A device for predicting liver tumor cell morphology, characterized in that: The prediction device comprises: A data collector, used to obtain target data based on the cases to be analyzed of target users; The target data includes target disease association data corresponding to the liver, and target three-dimensional graphic data corresponding to the liver reconstructed three-dimensionally based on the target CT data in the case to be analyzed; A data processor is used to input the target data into a preset network model that has been trained to output a target prediction result representing the liver tumor cell morphology of the target user within a future set time period.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for predicting liver tumor cell morphology according to any one of claims 1 to 8 is implemented.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for predicting liver tumor cell morphology according to any one of claims 1 to 8 is implemented.