Traditional Chinese medicine auxiliary syndrome differentiation method, system and device based on artificial intelligence technology and medium

Abstract

The invention relates to a traditional Chinese medicine auxiliary dialectical method, system and device based on an artificial intelligence technology and a medium, and the method comprises the steps: processing obtained inquiry data for a retrospective patient queue, obtaining a joint word embedding representation of the inquiry data, forming an overall inquiry feature in combination with a basic category feature, and carrying out the processing of the overall inquiry feature; after processing, adding the data into an inquiry data set; for the prospective patient queue, obtaining question and answer pairs through questionnaires, processing the question and answer pairs and adding the question and answer pairs into an inquiry data set; annotating the image data of the patient, and adding the annotated image data into an inspection data set; and constructing and training to obtain an interrogation auxiliary diagnosis large model and an inspection auxiliary model, predicting the condition of the patient through at least one round of interrogation traditional Chinese medicine dialectical judgment in combination with the inspection diagnosis result, and taking the last round of interrogation traditional Chinese medicine dialectical result as an auxiliary dialectical result. According to the invention, through integration of the inquiry data and the inspection data, an efficient auxiliary diagnosis model is constructed, and intelligence and precision of traditional Chinese medicine syndrome differentiation treatment are realized.

Description

Technical Field

[0001] The present invention relates to the field of medical assistance technology, and in particular to a TCM-assisted syndrome differentiation method, system, equipment and medium based on artificial intelligence technology. Background Art

[0002] With the advancement of science and technology, artificial intelligence has become a key force in promoting the digital transformation of many industries. In the medical field, especially in the traditional Chinese medicine industry, the integrated application of artificial intelligence technology is regarded as an important way to achieve the digitalization and intelligent upgrading of the industry. The traditional Chinese medicine diagnosis process mainly relies on the four-diagnosis method of "looking, smelling, asking, and touching", which is deeply trusted by traditional Chinese medicine practitioners of all generations.

[0003] However, in the diagnosis practice of traditional Chinese medicine, the diagnosis of the disease mainly relies on the combination of "looking, smelling, asking, and palpating". This diagnosis method relies heavily on the doctor's personal experience and skills, so there is a strong subjectivity and individual differences. Since this diagnostic method is difficult to quantify and analyze, it limits the accuracy and repeatability of TCM diagnosis to a certain extent.

[0004] In addition, in the initial diagnosis stage, patients are often unwilling to fully cooperate with doctors for reasons of privacy protection, which may result in doctors being unable to obtain complete and accurate information from the four diagnostic methods in a timely manner, thus affecting the efficiency and accuracy of diagnosis and treatment. Therefore, how to overcome these limitations in traditional Chinese medicine diagnosis methods and improve the efficiency and accuracy of diagnosis and treatment has become an urgent problem to be solved in the current Chinese medicine industry. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a TCM-assisted syndrome differentiation method, system, device and medium based on artificial intelligence technology, which solves the technical problems that traditional TCM diagnosis methods rely on the personal experience and skills of doctors, have subjectivity and individual differences, are difficult to quantify and analyze, and have low patient cooperation in the initial diagnosis stage, affecting the efficiency and accuracy of diagnosis and treatment.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] In a first aspect, an embodiment of the present invention provides a TCM-assisted syndrome differentiation method based on artificial intelligence technology, comprising:

[0010] For retrospective patient cohorts with historical cases, the consultation data obtained from the historical case database is classified, question-answer pairs are configured and coded, and a subset of consultation questions is extracted to form a consultation matrix and coded. The joint word embedding representation of the consultation data is obtained, and the basic category features extracted from the consultation data are combined to form the overall consultation features, which are added to the consultation data set after coding and annotation.

[0011] For prospective patient cohorts without historical cases, question-answer pairs were obtained through questionnaires and added to the consultation data set after annotation processing;

[0012] Based on expert knowledge, the collected patient image data is annotated and added to the visual diagnosis data set;

[0013] Build a graph neural network and a convolutional neural network, and train them based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively, to obtain a large medical consultation-assisted diagnosis model and a visual diagnosis-assisted model;

[0014] Based on the large model of auxiliary diagnosis by questioning and the auxiliary model of inspection, the patient's condition is predicted through at least one round of TCM syndrome differentiation judgment through questioning and combined with the diagnosis and treatment results of inspection, and the results of TCM syndrome differentiation in the last round of questioning are used as the auxiliary syndrome differentiation results of the patient.

[0015] Optionally, for a retrospective patient cohort with historical cases, the consultation data obtained from the historical case database is classified, question-answer pairs are configured and coded, and a subset of the consultation questions is extracted to form a consultation matrix and coded, and a joint word embedding representation of the consultation data is obtained. The basic category features extracted from the consultation data are combined to form an overall consultation feature. After coding and annotation, the consultation data set is added to include:

[0016] Obtain the consultation data of the retrospective patient cohort from the historical case database and divide the consultation data into multiple consultation categories;

[0017] Determine the question-answer pairs for each category based on expert knowledge to form a set of medical questions, and represent each question and the corresponding answer in the set of medical questions by unique hot encoding;

[0018] Extract at least one question from the set of medical questions represented by one-hot encoding to form a question subset for questioning the patient;

[0019] Convert each medical question in the question subset into an embedded representation, convert the corresponding patient-selected answer into an embedded representation, and concatenate the embedded representations of the medical question and the patient-selected answer to form a feature vector embedded representation of a single question-answer pair;

[0020] The feature vector embedding representations of multiple single question-answer pairs are stacked according to the sample dimension to form a question matrix. The question matrix is ​​encoded using an encoder model based on the attention mechanism to obtain a joint word embedding representation of the question data.

[0021] Extract basic category features including consultation time, consultation results, basic information, current medical history, past medical history, life history, and family genetic history from the patient's historical medical records, encode at least one type of information in the basic category features, and combine the joint word embedding representation of the consultation data to form the overall consultation features;

[0022] Based on expert knowledge, the TCM syndrome differentiation guidelines corresponding to the overall consultation characteristics are labeled, and the labeled consultation feature samples are added to the consultation data set.

[0023] Optionally, for prospective patient cohorts without historical cases, question-answer pairs are obtained through questionnaires and added to the consultation data set after annotation processing, including:

[0024] A questionnaire was distributed to a prospective cohort of patients without historical electronic medical records;

[0025] Collect the patient's answers to the questions in the questionnaire to form a question-answer pair containing the medical question and the answer;

[0026] Based on expert knowledge, the TCM syndrome differentiation guidelines corresponding to the questions and answers are annotated, and the annotated question and answer pairs are used as sample points and added to the consultation data set.

[0027] Optionally, the collected patient image data is annotated based on expert knowledge and added to the visual diagnosis data set, including:

[0028] Collecting patient image data from at least one image acquisition device to establish a visual diagnosis patient image database;

[0029] For each patient image in the image database, automatic annotation is performed based on the pre-trained image processing model according to the expert knowledge in the field of traditional Chinese medicine;

[0030] After all patient images are labeled, the labeled inspection features and diagnosis result samples are added to the inspection data set.

[0031] Optionally, a graph neural network and a convolutional neural network are constructed, and are trained based on a graph data set and a visual diagnosis data set converted from a medical consultation data set, respectively, to obtain a medical consultation-assisted diagnosis model and a visual diagnosis-assisted diagnosis model, including:

[0032] Divide the samples in the consultation data set into multiple sub-datasets, each sub-dataset contains a fixed number of patient sample points;

[0033] The sample points in each sub-dataset are constructed into a graph structure representation, where the nodes of the graph structure represent the patient sample points, and the edges of the graph structure represent the similarity or implicit relationship between the sample points;

[0034] Construct a patient consultation feature association encoder based on the attention mechanism in the graph neural network model to predict the association strength of the consultation feature embedding representation between any two patient consultation sample points in the graph structure representation;

[0035] Using the graph neural network model, combined with the patient inquiry feature association encoder, each graph structure representation is processed in turn. The graph neural network model has a multi-layer structure, and each layer realizes the information transmission and aggregation of patient features, and finally obtains the hidden layer embedding vector of each patient sample.

[0036] After being processed by a multi-layer graph neural network, a linear fully connected layer is used to convert the hidden layer embedding vector into the corresponding diagnosis and diagnosis program prediction result;

[0037] The loss value is calculated based on the real labels of the patient consultation samples, and the model parameters and the associated encoder model parameters are updated based on the average loss of the whole graph nodes to obtain a trained consultation-assisted diagnosis model.

[0038] A convolutional neural network is used to learn and train the image knowledge in the visual diagnosis data set, and an visual diagnosis auxiliary model is obtained that can process any image data in the same format as the visual diagnosis data set or new image data input by the user online.

[0039] Optionally, based on the large-scale diagnosis model assisted by medical questioning and the auxiliary model of visual diagnosis, the patient's condition is predicted through at least one round of Chinese medicine syndrome differentiation judgment by medical questioning, combined with the diagnosis and treatment results of visual diagnosis, and the Chinese medicine syndrome differentiation results of the last round of medical questioning are used as the auxiliary syndrome differentiation results of the patient, including:

[0040] Determine the total number of consultation rounds, and in each round, extract at least one question from the set of consultation questions represented by one-hot encoding for the patient to answer;

[0041] In the first round of consultation, according to the medical record information and consultation answers entered by the patients, the overall consultation characteristics of the first round are obtained;

[0042] Determine whether the patient simultaneously enters image data;

[0043] If the patient enters the image data, it will be processed by the visual diagnosis auxiliary model to calculate the patient's symptom prediction, which will serve as the prior syndrome differentiation program vector for the patient's consultation;

[0044] If the patient has not entered the image data, the anonymized and noised image selected by the patient in the visual diagnosis database will be used as the reference image and processed by the visual diagnosis auxiliary model to obtain the corresponding prior syndrome differentiation program vector;

[0045] If the patient has not entered image data and has not selected a reference image, the prior syndrome differentiation program vector is initialized to a zero vector;

[0046] In the first round of consultation, the overall consultation characteristics, the prior syndrome differentiation program vector and at least one patient consultation sample extracted from the consultation data set are used as inputs of the consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of the first round of consultation;

[0047] In the indirect rounds from the second round of consultation to the last round, only the consultation-assisted diagnosis model is used. In each indirect round, the overall consultation characteristics of this round are obtained according to the answers entered by the patient, and the consultation-assisted diagnosis model in the previous consultation round outputs the TCM syndrome differentiation results as the prior syndrome differentiation program vector in this round. The overall consultation characteristics of this round and the prior syndrome differentiation program vector are used together as the input of the consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of this round.

[0048] In the last round of consultation, the overall consultation characteristics of the current round, the prior syndrome differentiation program vector encoded from the TCM syndrome differentiation results of the previous round of consultation, and at least one patient consultation sample extracted from the consultation data set are used as input to a large consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of the last round of consultation, and the TCM syndrome differentiation results of the last round of consultation are used as the auxiliary syndrome differentiation results of the patient.

[0049] Optionally, based on the large model of medical consultation auxiliary diagnosis and the auxiliary model of visual diagnosis, the patient's condition is predicted through at least one round of medical consultation and TCM syndrome differentiation judgment, combined with the diagnosis and treatment results of visual diagnosis, and the last round of medical consultation and TCM syndrome differentiation results are used as the auxiliary syndrome differentiation results of the patient, and further includes:

[0050] Introduce the expert experience database to annotate the auxiliary syndrome differentiation results and the corresponding recommended prescriptions, form data pairs containing syndrome differentiation results and diagnostic prescriptions, and construct the corresponding annotated data set;

[0051] Construct a prescription decision tree model, determine the auxiliary syndrome differentiation results as input features, the corresponding diagnosis results or prescriptions as output targets, select information gain as the split basis for decision tree training, and recursively apply the split basis to generate nodes and branches of the decision tree;

[0052] The prescription decision tree model is trained using labeled data sets to learn the association pattern between pathogenesis and prescription, and finally output a reference prescription.

[0053] In a second aspect, an embodiment of the present invention provides a TCM-assisted syndrome differentiation system based on artificial intelligence technology, comprising:

[0054] The consultation data set output module is used to classify the consultation data obtained from the historical case library for retrospective patient cohorts with historical cases, configure and encode question-answer pairs, and extract a subset of consultation questions to form a consultation matrix and perform encoding processing, obtain a joint word embedding representation of the consultation data, combine the basic category features extracted from the consultation data to form an overall consultation feature, and add it to the consultation data set after encoding and labeling; and, for prospective patient cohorts without historical cases, obtain question-answer pairs through questionnaires, and add them to the consultation data set after labeling processing;

[0055] The visual diagnosis data set output module is used to annotate the collected patient image data based on expert knowledge and add it to the visual diagnosis data set;

[0056] The model building and training module is used to build a graph neural network and a convolutional neural network, which are trained based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively, to obtain a large medical consultation-assisted diagnosis model and a visual diagnosis-assisted model;

[0057] The auxiliary syndrome differentiation and judgment module is used to predict the patient's condition through at least one round of Chinese medicine syndrome differentiation and judgment through consultation based on the large model of auxiliary diagnosis through consultation and the auxiliary model of inspection through inspection, and use the results of the last round of Chinese medicine syndrome differentiation and judgment through consultation as the auxiliary syndrome differentiation results of the patient.

[0058] In a third aspect, an embodiment of the present invention provides a TCM-assisted syndrome differentiation device based on artificial intelligence technology, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the TCM-assisted syndrome differentiation method based on artificial intelligence technology as described above.

[0059] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having computer executable instructions stored thereon, which, when executed by a processor, implements the TCM-assisted syndrome differentiation method based on artificial intelligence technology as described above.

[0060] (III) Beneficial effects

[0061] The beneficial effects of the present invention are:

[0062] First, the present invention processes the consultation data in a sophisticated way, including classifying the consultation data of a retrospective patient cohort with historical cases, configuring coded question-answer pairs, and extracting subsets to form a consultation matrix, and combining basic category features to form an overall consultation feature. This effectively solves the problem that traditional Chinese medicine diagnosis methods rely on the personal experience and skills of doctors, are subjective and have individual differences, and are difficult to quantify and analyze, thereby providing a rich and accurate data foundation for subsequent model training.

[0063] For prospective patient cohorts without historical cases, the present invention obtains question-answer pairs through questionnaires, and also adds them to the consultation data set after annotation, ensuring the comprehensiveness and diversity of the data. In addition, the present invention also annotates the collected patient image data based on expert knowledge and adds them to the visual diagnosis data set. This step allows the traditional Chinese medical diagnosis method of visual diagnosis to be combined with artificial intelligence technology, providing more comprehensive and objective information for auxiliary diagnosis.

[0064] On this basis, the present invention constructs a graph neural network and a convolutional neural network, which are trained based on the graph data set converted from the interview data set and the visual diagnosis data set, respectively, to obtain the interview-assisted diagnosis large model and the visual diagnosis-assisted model. These two models can learn modal, multi-dimensional, and multi-level expert TCM diagnostic knowledge, providing doctors with powerful auxiliary tools to improve the consistency and accuracy of diagnosis.

[0065] Finally, based on the large-scale diagnosis-aided model of medical questioning and the auxiliary model of visual inspection, the present invention predicts the patient's condition through at least one round of medical questioning and Chinese medicine syndrome differentiation judgment, combined with the diagnosis and treatment results of visual inspection, and uses the Chinese medicine syndrome differentiation results of the last round of medical questioning as the patient's final medical questioning result. This process realizes the organic combination of medical questioning and visual inspection, gives full play to the advantages of Chinese medicine syndrome differentiation and treatment, and improves the accuracy and reliability of patient condition prediction. At the same time, through multiple rounds of medical questioning and Chinese medicine syndrome differentiation judgment, the diagnosis results can be gradually refined and improved, providing patients with more accurate and personalized diagnosis and treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of a flow chart of a TCM-assisted syndrome differentiation method based on artificial intelligence technology proposed in an embodiment of the present invention;

[0067] Figure 2 A specific flow chart of step S1 of a TCM-assisted syndrome differentiation method based on artificial intelligence technology proposed in an embodiment of the present invention;

[0068] Figure 3 A specific flow chart of step S2 of a TCM-assisted syndrome differentiation method based on artificial intelligence technology proposed in an embodiment of the present invention;

[0069] Figure 4A schematic diagram of a specific flow chart of step S3 of a TCM-assisted syndrome differentiation method based on artificial intelligence technology proposed in an embodiment of the present invention;

[0070] Figure 5 A specific flow chart of step S4 of a TCM-assisted syndrome differentiation method based on artificial intelligence technology proposed in an embodiment of the present invention;

[0071] Figure 6 This is a specific flow chart of step S5 of a TCM-assisted syndrome differentiation method based on artificial intelligence technology proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0073] like Figure 1 As shown, an embodiment of the present invention proposes a TCM-assisted syndrome differentiation method based on artificial intelligence technology, comprising: for a retrospective patient cohort with historical cases, the medical consultation data obtained from the historical case library are classified, question-answer pairs are configured and coded, and a subset of medical consultation questions are extracted to form a medical consultation matrix and coded to obtain a joint word embedding representation of the medical consultation data, and the overall medical consultation features are formed by combining the basic category features extracted from the medical consultation data, and the overall medical consultation features are added to the medical consultation data set after coding and labeling; for a prospective patient cohort without historical cases, question-answer pairs are obtained through a questionnaire, After being labeled, the data are added to the medical consultation data set; based on expert knowledge, the collected image data of the patients are labeled and added to the visual diagnosis data set; a graph neural network and a convolutional neural network are constructed, and they are trained based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively, to obtain a large model for medical consultation-assisted diagnosis and a visual diagnosis-assisted model; based on the large model for medical consultation-assisted diagnosis and the visual diagnosis-assisted model, at least one round of Chinese medicine syndrome differentiation judgment through medical consultation is combined with the diagnosis and treatment results of visual diagnosis to predict the patient's condition, and the results of the last round of Chinese medicine syndrome differentiation through medical consultation are used as the auxiliary syndrome differentiation results of the patient.

[0074] First, the present invention processes the consultation data in a sophisticated way, including classifying the consultation data of a retrospective patient cohort with historical cases, configuring coded question-answer pairs, and extracting subsets to form a consultation matrix, and combining basic category features to form an overall consultation feature. This effectively solves the problem that traditional Chinese medicine diagnosis methods rely on the personal experience and skills of doctors, are subjective and have individual differences, and are difficult to quantify and analyze, thereby providing a rich and accurate data foundation for subsequent model training.

[0075] For prospective patient cohorts without historical cases, the present invention obtains question-answer pairs through questionnaires, and also adds them to the consultation data set after annotation, ensuring the comprehensiveness and diversity of the data. In addition, the present invention also annotates the collected patient image data based on expert knowledge and adds them to the visual diagnosis data set. This step allows the traditional Chinese medical diagnosis method of visual diagnosis to be combined with artificial intelligence technology, providing more comprehensive and objective information for auxiliary diagnosis.

[0076] On this basis, the present invention constructs a graph neural network and a convolutional neural network, which are trained based on the graph data set converted from the interview data set and the visual diagnosis data set, respectively, to obtain the interview-assisted diagnosis large model and the visual diagnosis-assisted model. These two models can learn modal, multi-dimensional, and multi-level expert TCM diagnostic knowledge, providing doctors with powerful auxiliary tools to improve the consistency and accuracy of diagnosis.

[0077] Finally, based on the large-scale diagnosis-aided model of medical questioning and the auxiliary model of visual inspection, the present invention predicts the patient's condition through at least one round of medical questioning and Chinese medicine syndrome differentiation judgment, combined with the diagnosis and treatment results of visual inspection, and uses the Chinese medicine syndrome differentiation results of the last round of medical questioning as the patient's final medical questioning result. This process realizes the organic combination of medical questioning and visual inspection, gives full play to the advantages of Chinese medicine syndrome differentiation and treatment, and improves the accuracy and reliability of patient condition prediction. At the same time, through multiple rounds of medical questioning and Chinese medicine syndrome differentiation judgment, the diagnosis results can be gradually refined and improved, providing patients with more accurate and personalized diagnosis and treatment plans.

[0078] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0079] Specifically, the embodiment of the present invention provides a TCM-assisted syndrome differentiation method based on artificial intelligence technology, which is characterized by comprising:

[0080] S1. For retrospective patient cohorts with historical cases, the consultation data obtained from the historical case database is classified, question-answer pairs are configured and coded, and a subset of consultation questions is extracted to form a consultation matrix and encode it to obtain a joint word embedding representation of the consultation data. The basic category features extracted from the consultation data are combined to form an overall consultation feature, which is then added to the consultation data set after encoding and labeling.

[0081] Furthermore, if Figure 2 As shown, step S1 includes:

[0082] S11. Obtain consultation data of a retrospective patient cohort from a historical case database, and divide the consultation data into multiple consultation categories.

[0083] S12. Determine the question-answer pairs for each category based on expert knowledge to form a set of medical questions, and represent each question and corresponding answer in the set of medical questions by unique-hot encoding.

[0084] S13. Extract at least one question from the set of medical questions represented by one-hot encoding to form a question subset for questioning the patient.

[0085] S14. Convert each medical question in the question subset into an embedded representation, convert the corresponding answer selected by the patient into an embedded representation, and concatenate the embedded representations of the medical question and the answer selected by the patient to form a feature vector embedded representation of a single question-answer pair.

[0086] S15. The feature vector embedding representations of multiple single question-answer pairs are stacked according to the sample dimension to form a question matrix, and the question matrix is ​​encoded using an encoder model based on the attention mechanism to obtain a joint word embedding representation of the question data.

[0087] S16. Extract basic category features including consultation time, consultation results, basic information, current medical history, past medical history, life history, and family genetic history from the patient's historical medical records, encode at least one type of information in the basic category features, and combine the joint word embedding representation of the consultation data to form an overall consultation feature. The basic information includes gender, age, marital status, ethnicity, occupation, etc.

[0088] One-hot encoding is used to represent general situations, standard medical encoding is used to represent current medical history and family genetic disease history, and binary encoding is used to represent specific content in life history.

[0089] S17. Based on expert knowledge, the TCM syndrome differentiation guidelines corresponding to the overall medical consultation characteristics are labeled, and the labeled medical consultation feature samples are added to the medical consultation data set.

[0090] In one embodiment, for a retrospective patient cohort with historical electronic medical records, the present invention first divides the acquired consultation data into C types of consultation information categories, where C = {cold and heat, pain, sweat, head and body, chest and abdomen, ears and eyes, sleep, diet, and defecation}.

[0091] Based on expert knowledge, a number of question-answer pairs are set for each category, with a total of K. The definition is as follows: The question set is Q = {q 1 ,q 2 , ..., q K}. At the same time, each question q kThere are at most M possible answers. Therefore, for each question q k and its corresponding answer can be represented independently using one-hot encoding. The encoding is as follows: Define the problem q defined by one-hot encoding k Represented as q k ∈R K , the question q k The mth answer is coded as a km ∈R M .

[0092] In the process of consulting a single patient, in order to improve the efficiency of the consultation, a small number of N questions are selected from the total number of K questions to ask the patient, forming a subset of questions: Q * = {q 1 * ,q 2 , ..., q N *}. For the question set Q * The kth medical question q selected in k * , define its embedding representation as q k * ; At the same time, use a km * Represents the answer chosen by the patient, and defines its embedding as a km * The medical question and the answer selected by the patient are concatenated to form a feature vector embedding representation of a single question-answer pair:

[0093] z=[q k * , a km ]∈R (K+M) ;

[0094] Afterwards, for a sequence of N extracted questions, they are stacked according to the sample dimension to form a question matrix. Specifically, during the stacking process, each row of the generated question matrix represents the feature vector embedding representation z of a question-answer pair. 1 ,z 2 …Each column represents a feature dimension value of a question-answer pair feature vector, and the following matrix is ​​finally obtained:

[0095] Z=[z 1 , z 2 ,...,z N ]∈R N×(K+M) ;

[0096] Next, the attention-based encoder model is used to encode the inquiry matrix Z to obtain the joint word embedding representation of the entire inquiry information:

[0097] z * =Encoder(Z);

[0098] Wherein, Encoder(·) can be any general word embedding encoder model, or an encoder model specially trained for the present invention. Using the encoder to perform feature fusion on the global question-answering process can capture the global dependency between the question and the answer, and improve the model's ability to understand the question information.

[0099] Furthermore, for a single patient sample j, extract the consultation time t contained in its electronic medical records j , Consultation results Basic information j , History of present illness j 、Past medical history j 、Life history j , family history of genetic diseases j The seven categories of items in each consultation period are taken as the overall consultation characteristics of patient sample i.

[0100]

[0101] Among them, for the general classification situation g j Use one-hot encoding. For the history of present illness h j , family history of genetic diseases j Using standard medical coding, for life history j Whether smoking, drinking, etc. are represented by binary encoding. It is obtained by processing the corresponding content in the historical cases that matches the consultation set C according to the encoding method described above.

[0102] The overall consultation characteristics x for a sample point j , based on expert knowledge, the eight TCM syndrome differentiation guidelines of “exterior, interior, cold, heat, deficiency, excess, yin, and yang” corresponding to the sample characteristics are annotated. Among them, expert knowledge can be the historical syndrome differentiation results recorded in the electronic medical records. It can also be re-annotated after collecting data samples. Specifically, for x j , will be marked as:

[0103] y i ={y 表 ,y 里 ,y 寒 ,y 热 ,y 虚 ,y 实 ,y 阴 ,y 阳}∈R 8 ;

[0104] Among them, y i It has eight dimensions, each corresponding to one of the above-mentioned TCM syndrome differentiation guidelines. i Each dimension in corresponds to the presence, severity or specific category of a dialectical program. 寒 =0 means the patient has no cold syndrome, y 热 =1 indicates that the patient has heat syndrome. i The value range and boundary value division of each element in can be completed based on the empirical knowledge of Chinese medicine diagnosis. For example, deficiency syndrome is divided into qi deficiency, blood deficiency, yang deficiency, and yin deficiency according to experience, which can be divided into y 虚 ∈{0, 1, 2, 3, 4}.

[0105] Afterwards, the labeled medical inquiry feature samples (x i ,y i ) Add the consultation data set D 问 For the convenience of expression, the symbol (x * ,y * ) represents the consultation data set D 问 Any sample point in .

[0106] S2. For prospective patient cohorts without historical cases, question-answer pairs are obtained through questionnaires and added to the consultation data set after annotation.

[0107] Furthermore, if Figure 3 As shown, step S2 includes:

[0108] S21. Questionnaires were distributed to a prospective cohort of patients without historical electronic medical records.

[0109] S22. Collect the patients' responses to the questions in the questionnaire to form question-answer pairs including medical questions and answers.

[0110] S23. Annotate the TCM syndrome differentiation guidelines corresponding to the questions and answers based on expert knowledge, and add the annotated question and answer pairs as sample points to the medical consultation data set.

[0111] For prospective cohort patients without historical electronic medical records, we will record the questions and answers online in the form of questionnaires, and annotate the eight TCM syndrome differentiation guidelines of "exterior, interior, cold, heat, deficiency, excess, yin, and yang" corresponding to the question and answer characteristics based on expert knowledge. The obtained data will be added to the consultation data set D 问 .

[0112] S3. Annotate the collected patient image data based on expert knowledge and add it to the visual diagnosis data set.

[0113] Furthermore, if Figure 4 As shown, step S3 includes:

[0114] S31. Collect patient image data from at least one image acquisition device to establish a visual diagnosis patient image database.

[0115] S32. For each patient image in the image database, automatic labeling is performed based on the pre-trained image processing model according to the expert knowledge in the field of traditional Chinese medicine.

[0116] S33. After all patient images are labeled, the labeled inspection features and diagnosis result samples are added to the inspection data set.

[0117] In one embodiment, a patient image database for visual diagnosis is established by collecting image data such as the patient's consciousness, eyesight, complexion, body shape, lips, tongue coating, and tongue image. For each patient image in the image database, the eight visual diagnosis results of TCM syndrome differentiation guidelines are annotated based on expert knowledge. In this process, the existing image processing model based on convolutional neural network can be used for automatic annotation, or the image annotation can be performed manually. Specifically, for the jth image f in the patient image database, j , and get the labeling result:

[0118] y j ={y 表 ,y 里 ,y 寒 ,y 热 ,y 虚 ,y 实 ,y 阴 ,y 阳}∈R 8 ;

[0119] Afterwards, the marked inspection features and the diagnostic result samples (f j ,y j ) Add the visual diagnosis data set D 望 For the convenience of expression, the symbol (f * ,y * ) represents the consultation data set D 问 Any sample point in .

[0120] S4. Construct a graph neural network and a convolutional neural network, and train them based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively, to obtain a large model for medical consultation-assisted diagnosis and a visual diagnosis-assisted model.

[0121] Furthermore, if Figure 5 As shown, step S4 includes:

[0122] S41. Divide the samples in the medical consultation data set into a plurality of sub-data sets, each sub-data set containing a fixed number of patient sample points.

[0123] S42. Construct the sample points in each sub-data set into a graph structure representation, wherein the nodes represented by the graph structure represent the patient sample points, and the edges represented by the graph structure represent the similarity or implicit relationship between the sample points.

[0124] S43. Construct a patient consultation feature association encoder based on the attention mechanism in the graph neural network model to predict the association strength of the consultation feature embedding representation between any two patient consultation sample points in the graph structure representation.

[0125] S44. Use the graph neural network model in combination with the patient question feature association encoder to process each graph structure representation in turn. The graph neural network model has a multi-layer structure, and each layer realizes the information transmission and aggregation of patient features, and finally obtains the hidden layer embedding vector of each patient sample.

[0126] S45. After being processed by the multi-layer graph neural network, a linear fully connected layer is used to convert the hidden layer embedding vector into the corresponding diagnosis and differentiation program prediction result.

[0127] S46. Calculate the loss value based on the real label of the patient consultation sample, and update the model parameters and the associated encoder model parameters based on the average loss of the whole graph nodes to obtain a trained consultation-assisted diagnosis model.

[0128] S47. Use a convolutional neural network to learn and train the image knowledge in the visual diagnosis data set to obtain a visual diagnosis auxiliary model that can process any image data in the same format as the visual diagnosis data set or new image data input by the user online.

[0129] In step S4, the present invention represents the consultation data of multiple patients as a graph structure, takes the associations between patients as edges, constructs a graph data set suitable for graph neural network processing, and processes the potential association representations between the consultation samples of multiple patients through the graph neural network. In the clinical application stage of the model, when judging the condition and syndrome of a single patient, the specific consultation results of the patient will be used to avoid the traditional method of processing the consultation results of a single patient, which cannot fully utilize the association information between multiple similar patients, resulting in strong subjectivity, weak consistency, and large individual differences in diagnosis and treatment.

[0130] In a specific embodiment, the medical consultation data set D 问 Complete model training. In the early preparation process of training, firstly, D 问 The samples in are divided into M sub-datasets, each database contains a fixed number of V sample points. Each sub-dataset is denoted as It can be expressed as follows:

[0131]

[0132] For each sub-dataset Construct its sample points into a graph structure representation in Represents the set of nodes (i.e., patient sample points) of the graph, Represents the edge set of the graph, that is, the similarity or implicit relationship between sample points. The specific expression is as follows:

[0133]

[0134] For the graph Each node v in * , whose question embedding feature is represented as X * , the label of the consultation result is y * . Because of the figure In the graph neural network model, the connection between each patient consultation sample point is usually implicit, and there is a lack of universal similarity measurement rules. Therefore, the invention will construct a patient consultation feature association encoder based on the attention mechanism in the graph neural network model, aiming at Any two patient consultation sample points v j , v k , predict and output the corresponding question feature embedding representation x j , x k The scalar value of the correlation strength between jk The feature association encoder model structure is as follows:

[0135] e jk =LeakyReLU(aT[W en x j ||W en x k ]);

[0136] The activation function LeakyReLu(x) used is defined as the nonlinear representation LeakyReLu(x) = max(αx, x) for any input x; α is a constant parameter. en and a are learnable parameters in the encoder model.

[0137] After that, the Softmax normalization function is used to calculate the relationship between any two patient consultation sample points v j , v k Characteristics of the interview X j , X k The attention score α between jk :

[0138]

[0139] During the training phase, the patient question feature association encoder model will participate in the forward propagation together with the downstream diagnosis and treatment model, and realize the model parameter W through the back propagation method based on the loss function of the downstream model. en An update with a.

[0140] The invention uses a graph neural network to infer the patient's diagnosis and treatment results based on the historical patient TCM syndrome differentiation information and the multi-patient disease association information, that is, the TCM syndrome differentiation status posterior information prediction. The TCM syndrome differentiation information represents the "exterior, interior, cold, heat, deficiency, excess, yin, and yang" category prediction results based on the initial understanding of the patient before the consultation. The multi-patient disease association information is determined by the attention score between the patient samples. In the training phase, the database D 问 Each subgraph in Iteratively train the AI ​​consultation decision model. Specifically, the model has a total of 6 graph neural network layers. The first layer will directly process the patient sample data as follows:

[0141]

[0142] in, is the learnable parameter of the first layer of the graph neural network. Specifically, for each sample point v in the graph j , Will achieve v j The transfer and aggregation between the global patient features, and α jk Used to effectively control the sample point v j For the sample point v k The degree of aggregation of information. For patient sample v j After the graph neural network completes the patient feature information transmission and aggregation operation, the hidden layer representation is obtained. b j is the sample point v j The high-dimensional embedding representation corresponding to the prior dialectical program result is j Have the same structure.

[0143] After that, the sample point v j The corresponding output of the first layer It will be used as the input of the second layer and processed by the six-layer graph neural network in sequence according to this rule. In the second to sixth layers, the patient sample data is processed as follows:

[0144]

[0145] Among them, s=2, 3, .., 6 is the index of the middle layer graph neural network layer; is the learnable parameter of the s-th layer graph neural network; For sample v j The hidden layer output representation obtained after processing the sth layer. After processing by the 6-layer graph neural network, each patient sample v j The corresponding hidden layer embedding vector A linear fully connected layer will be used to obtain the corresponding diagnosis and diagnosis program prediction results:

[0146]

[0147] Getting the prediction results Then, based on the patient consultation sample v j The true label y j Calculate the mean square error loss value and update the parameters based on the average loss of all nodes in the graph and the associated encoder model parameters W en With a.

[0148] For ease of representation, the multi-patient-associated interview-assisted diagnosis model based on graph neural network trained by the method recorded in the above process is defined as the following function:

[0149]

[0150] Among them, the function Model 问 Represents a large model of a trained neural network, used to assist in diagnosis and treatment decision-making; Represents the model input, which is a set of V nodes Any set of corresponding patient sample consultation features; represent Contains each sample point x j The corresponding a priori dialectical program vector b j The set composed of. represents the output of the model, i.e. Each element x in j With prior knowledge j Prediction results of the medical consultation outline corresponding to the joint medical consultation information The set composed of.

[0151] Therefore, referring to the above-mentioned model input and output mode and training method, in terms of usage, the multi-patient-associated medical consultation-assisted diagnosis large model provided by the present invention will use the medical consultation feature set of multiple (specifically V) patients as output, combined with the global patient feature association relationship, and simultaneously output the prediction results of the "exterior, interior, cold, heat, deficiency, excess, yin, and yang" categories for each patient.

[0152] In addition, the present invention also provides a method for using the visual diagnosis data set D 望Assisted consultation model Method. Using convolutional neural network to 望 The convolutional neural network can be a basic convolutional neural network or a specially optimized complex deep convolutional neural network model, that is, the algorithm and implementation of the convolutional neural network are not constrained. The visual diagnosis auxiliary model based on the graph convolutional neural network algorithm after training is defined as the following function:

[0153]

[0154] Among them, the function Model 望 represents the visual diagnosis auxiliary model obtained after training. * Representation and database D 望 Any image data of the same format in the database D 望 It can be existing data in the , or it can be new data entered by users online after deployment. Represents the patient input image f * The prediction classification results of "exterior, interior, cold, heat, deficiency, excess, yin, yang" are compared with the vector and b j Have the same structure.

[0155] S5. Based on the large model of medical questioning-assisted diagnosis and the auxiliary model of inspection-assisted diagnosis, predict the patient's condition through at least one round of medical questioning-assisted TCM syndrome differentiation and judgment, combined with the diagnosis and treatment results of inspection-assisted diagnosis, and use the results of the last round of medical questioning-assisted TCM syndrome differentiation as the auxiliary syndrome differentiation results of the patient.

[0156] Furthermore, if Figure 6 As shown, step S5 includes:

[0157] S51. Determine the total number of consultation rounds, and in each round, extract at least one question from the set of consultation questions represented by one-hot encoding for the patient to answer.

[0158] S52. In the first round of consultation, based on the medical record information and consultation answers entered by the patient, obtain the overall consultation characteristics of the first round.

[0159] S53: Determine whether the patient simultaneously enters image data.

[0160] S54. If the patient enters the image data, it will be processed by the visual diagnosis auxiliary model to calculate the patient's condition prediction to serve as the prior syndrome differentiation program vector for the patient's medical consultation; if the patient does not enter the image data, the anonymized and noise-added picture selected by the patient in the visual diagnosis database will be used as the reference image, and will be processed by the visual diagnosis auxiliary model to obtain the corresponding prior syndrome differentiation program vector; if the patient does not enter the image data and does not select a reference picture, the prior syndrome differentiation program vector will be initialized to a zero vector.

[0161] S55. In the first round of consultation, the overall consultation characteristics, the prior syndrome differentiation program vector and at least one patient consultation sample extracted from the consultation data set are simultaneously used as inputs of the consultation-assisted diagnosis model to predict the results of the first round of TCM syndrome differentiation.

[0162] S56. In the indirect rounds from the second round of consultation to the last round, only the consultation-assisted diagnosis model is used. In each indirect round, the overall consultation characteristics of this round are obtained according to the answers entered by the patients, and the TCM syndrome differentiation results output by the consultation-assisted diagnosis model in the previous consultation round are used as the prior syndrome differentiation program vector in this round. The overall consultation characteristics of this round and the prior syndrome differentiation program vector are used together as the input of the consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of this round.

[0163] S57. In the last round of consultation, the overall consultation characteristics of the current round, the prior syndrome differentiation program vector encoded from the TCM syndrome differentiation results of the previous round of consultation, and at least one patient consultation sample extracted from the consultation data set are used as inputs to a large consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of the last round of consultation, and the TCM syndrome differentiation results of the last round of consultation are used as the auxiliary syndrome differentiation results of the patient.

[0164] In a specific embodiment, when providing online TCM diagnosis and treatment services, in order to protect patient privacy, patients usually only provide limited picture information. In addition, during a consultation, the patient's understanding and answers to the consultation content may be unclear or even misleading. To address this issue, the core idea is to predict the patient's "exterior, interior, cold, heat, deficiency, excess, yin, and yang" TCM syndrome identification program status based on the results of multiple rounds of questionnaires, supplemented by the results of visual diagnosis.

[0165] There are M rounds of consultations. In each round, N questions are extracted from the encoded question set Q for the patient to answer. During the answering process of any patient, the processing method described in the third step is used to obtain the joint word embedding vector representation z of the consultation information of this round * , as the basic general processing method for each round. Specifically, in each round, there are the following differentiated operations:

[0166] In the first round, the patient's medical history information entered in the online system is combined with the above-described processing method to obtain the overall consultation feature x that will be used in the first round. * At the same time, if the patient has input image data f * , then first pass it through the visual diagnosis auxiliary model Model 望 Processing, calculation of patient condition prediction Afterwards, the symptoms will be predicted by visual examination. As the priori syndrome differentiation program vector b for the patient's consultation * , that is, in the system At the same time, if the patient does not enter the picture information, the patient can 望 Select some pictures that have been anonymized and denoised as reference pictures and obtain the corresponding data b * If the patient does not enter the image information and does not select a reference image, b will be * Initialized to the zero vector.

[0167] After that, x * With b * As a large model for auxiliary diagnosis 问 Input information, predict the patient's first round of consultation results To facilitate distinction, Expressed as Indicates the results of the first round of consultation. 问 The consultation results of V patients are required to be input as common input to construct a graph structure representation. 问 V-1 patient consultation samples are drawn from * With b * Also as a Model 问 The graph embedding input information.

[0168] In the second to the Mth round of consultation, only the consultation-assisted diagnosis model Model is used 问 For the sake of distinction, the prediction of the mth round of consultation results is expressed as In each round m, the patient answers the questions in this round to obtain the overall consultation characteristics x of this round * After that, the model Model in the previous round of consultation 问 The results of TCM diagnosis and treatment As the a priori dialectical program vector b in this round * , x * With b * As a model in this round of consultation 问The input information of the TCM consultation model is used to obtain the TCM syndrome differentiation results of this round of consultation. Through multiple rounds of question-and-answer sessions, the model can gradually gain a deeper understanding of the patient's condition, and use the results of the previous round of consultations to assist the current round of consultations, thereby improving the accuracy and consistency of the consultation decision-making model.

[0169] Furthermore, in the results of the last round (Mth round) of consultation, first, based on the answers entered by the patient in this round, combined with the set of one-hot encoded consultation questions, the consultation data is extracted and encoded to generate the overall consultation features of the current round. Subsequently, the TCM syndrome differentiation results of the consultation are obtained from the output of the large consultation-assisted diagnosis model of the previous round, and encoded as a priori syndrome differentiation program vector. Next, the overall consultation features of the current round, the prior syndrome differentiation program vector, and the patient consultation samples (if any) associated with the current consultation features are taken as input and passed into the large consultation-assisted diagnosis model for inference. The model outputs the TCM syndrome differentiation results of the current round of consultation It will be used as a TCM auxiliary clinical decision-making system to predict the final results of the eight TCM syndrome differentiation guidelines of "exterior, interior, cold, heat, deficiency, excess, yin, and yang" for patients, and assist on-site diagnostic doctors or patients themselves to clarify the pathogenesis and syndrome manifestations. Based on the patient's complaints, TCM doctors can determine treatment principles or provide prescriptions online or offline based on their own basic TCM diagnostic theories or clinical experience.

[0170] In addition, after step S5, it also includes: introducing an expert experience library, annotating the auxiliary syndrome differentiation results and the corresponding recommended prescriptions, forming a data pair including the syndrome differentiation results and the diagnostic prescriptions, and constructing a corresponding annotated data set; constructing a prescription decision tree model, determining the auxiliary syndrome differentiation results as input features, the corresponding diagnostic results or prescriptions as output targets, selecting information gain as the splitting basis for decision tree training, and recursively applying the splitting basis to generate nodes and branches of the decision tree; using the annotated data set to train the prescription decision tree model, learning the association pattern between pathogenesis and prescriptions, and finally outputting a reference prescription.

[0171] It includes the following sub-steps:

[0172] (1) Diagnostic data collection and data annotation: TCM experts annotate the TCM syndrome differentiation results and the corresponding recommended prescriptions to form a "syndrome differentiation result-diagnostic prescription" data pair and construct a corresponding annotated data set.

[0173] (2) Constructing a prescription decision tree model: Based on the syndrome differentiation results (input features) and the corresponding diagnostic results or prescriptions (output targets), the prescription decision tree is trained using the information gain algorithm (in addition to information gain, there are many other splitting criteria to choose from, such as the Gini index, mean square error, etc.) to learn the association pattern between pathogenesis and prescription.

[0174] (3) Use of prescription decision tree: The application steps of the decision tree model are as follows: Based on steps S1-S5, first obtain the syndrome differentiation judgment based on the patient's case information and the results of the medical interview (such as the expected diagnosis results). After that, the obtained syndrome differentiation results are used as input and input into the prescription decision tree model. The decision tree will generate the corresponding prescription according to the trained rules. In this way, the correlation between the patient's syndrome (syndrome differentiation results) and the prescription is established, thereby providing the patient with auxiliary diagnostic direction and prescription reference, but this method cannot be used as a basis for diagnosis or a final prescription plan.

[0175] It should be emphasized that the present invention establishes a data-driven association model between pathogenesis and prescription based on rule-based or decision tree methods, allowing patients to obtain the correct diagnosis results based on the results of the consultation. The online use of this association model can provide patients with diagnostic direction and prescription references based on the consultation results to a certain extent without the intervention of a doctor, but it cannot be used as the basis for diagnosis and the final prescription plan.

[0176] In addition, an embodiment of the present invention provides a TCM-assisted syndrome differentiation system based on artificial intelligence technology, comprising:

[0177] The consultation data set output module is used to classify the consultation data obtained from the historical case library for retrospective patient cohorts with historical cases, configure and encode question-answer pairs, and extract a subset of consultation questions to form a consultation matrix and perform encoding processing, obtain a joint word embedding representation of the consultation data, combine the basic category features extracted from the consultation data to form an overall consultation feature, and add it to the consultation data set after encoding and labeling; and, for prospective patient cohorts without historical cases, obtain question-answer pairs through questionnaires, and add them to the consultation data set after labeling processing;

[0178] The visual diagnosis data set output module is used to annotate the collected patient image data based on expert knowledge and add it to the visual diagnosis data set;

[0179] The model building and training module is used to build a graph neural network and a convolutional neural network, which are trained based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively, to obtain a large medical consultation-assisted diagnosis model and a visual diagnosis-assisted model;

[0180] The auxiliary syndrome differentiation and judgment module is used to predict the patient's condition through at least one round of Chinese medicine syndrome differentiation and judgment through consultation based on the large model of auxiliary diagnosis through consultation and the auxiliary model of inspection through inspection, and use the results of the last round of Chinese medicine syndrome differentiation and judgment through consultation as the auxiliary syndrome differentiation results of the patient.

[0181] Furthermore, an embodiment of the present invention provides a TCM-assisted syndrome differentiation device based on artificial intelligence technology, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the TCM-assisted syndrome differentiation method based on artificial intelligence technology as described above.

[0182] The device is equipped with at least one processor as the core of operation and control, which is responsible for executing instructions stored in the memory to complete the relevant calculation and processing tasks of TCM-assisted syndrome differentiation.

[0183] Next, the memory closely connected to the processor is used to store a series of instructions that can be executed by the processor. These instructions are the specific implementation of the TCM-assisted syndrome differentiation method based on artificial intelligence technology. They are carefully designed to ensure that the processor can perform efficient TCM syndrome differentiation analysis according to the established logic and algorithm.

[0184] Therefore, the TCM auxiliary syndrome differentiation device provided in the embodiment of the present invention realizes the automated execution of the TCM syndrome differentiation method based on artificial intelligence technology through the collaborative work of the processor and the memory, provides powerful technical support for TCM clinical practice, and helps to improve the accuracy and efficiency of TCM syndrome differentiation.

[0185] Furthermore, an embodiment of the present invention provides a computer-readable medium having computer executable instructions stored thereon, and when the executable instructions are executed by a processor, the TCM-assisted syndrome differentiation method based on artificial intelligence technology as described above is implemented.

[0186] The embodiment of the present invention also provides a computer-readable medium on which a series of computer-executable instructions are stored. When these instructions are executed by the processor, they can implement the above-mentioned TCM-assisted syndrome differentiation method based on artificial intelligence technology. Specifically, this computer-readable medium, as an information storage carrier, contains carefully designed algorithms and logics, which are the core of the TCM-assisted syndrome differentiation method based on artificial intelligence technology. By reading and executing these instructions, the processor can automatically complete the relevant analysis and processing work of TCM syndrome differentiation, thereby improving the intelligence level and accuracy of TCM syndrome differentiation.

[0187] In summary, the embodiments of the present invention provide a method, system, device and medium for TCM-assisted syndrome differentiation based on artificial intelligence technology, and the overall process is as follows:

[0188] First, construct and clarify the entry point for collecting medical data of the four diagnostic methods of TCM, namely "looking, listening, asking and palpating". The result of "questioning" is used as the main basis for analysis, and "looking" is used to assist decision-making. It does not involve the use of "listening" and "palpating" medical data. In addition, a TCM diagnosis and treatment set is set, including a questioning data set and a looking data set.

[0189] Secondly, the medical consultation information is divided into multiple categories, and several question-answer pairs are set up for each category. The questions and answers are represented using one-hot encoding to form a feature vector embedding representation of a single question-answer pair, and then a medical consultation matrix is ​​constructed. The medical consultation matrix is ​​encoded using an encoder model based on the attention mechanism to obtain the joint word embedding representation of the entire medical consultation information.

[0190] Next, a retrospective consultation cohort and a prospective consultation cohort were constructed from the case database, and multiple items such as consultation time, consultation results, and general conditions were extracted as the overall consultation characteristics of the patient sample, and the corresponding TCM syndrome differentiation guidelines were annotated based on expert knowledge.

[0191] For prospective cohort patients without historical electronic medical records, the medical questions and answer pairs were recorded in the form of questionnaires, annotated with the guidelines of TCM syndrome differentiation, and the obtained data were added to the medical consultation data set.

[0192] Afterwards, a visual diagnosis database was constructed to collect patients' image data, and the visual diagnosis results of their TCM syndrome differentiation guidelines were annotated based on expert knowledge for subsequent model training.

[0193] Then, the consultation data of multiple patients are represented as a graph structure, and a patient consultation feature association encoder based on the attention mechanism is constructed to predict the association strength between the output patient consultation feature embedding representations, and the potential association representation between multiple patient consultation samples is processed through a graph neural network. Using a graph neural network, based on the historical patient TCM syndrome differentiation outline a priori information and the association information of multiple patients' conditions, the patient's diagnosis and treatment results, that is, the TCM syndrome differentiation outline status posterior information prediction, are inferred.

[0194] Furthermore, a convolutional neural network is used to learn the image knowledge in the visual diagnosis data set to obtain a visual diagnosis auxiliary model.

[0195] Finally, through the results of multiple rounds of questionnaires, supplemented by the results of visual examination, the patient's TCM syndrome differentiation program status is predicted. The results of the last round of TCM syndrome differentiation are used as the final prediction results for the patient, assisting the on-site diagnostic doctor or the patient himself to clarify the pathogenesis and symptom manifestations.

[0196] The present invention realizes the efficient use of consultation data through a series of sophisticated data processing steps, broadens the source channels of consultation data, and ensures the wide applicability of the method. At the same time, it makes full use of expert knowledge to annotate patient image data, thereby improving the reliability and accuracy of visual diagnosis data. In terms of model construction, the method uses the powerful ability of deep learning to obtain a large consultation-assisted diagnosis model and a visual diagnosis-assisted model with high accuracy and generalization ability. Finally, through at least one round of Chinese medicine syndrome differentiation judgment through consultation, the patient's condition is predicted in combination with the diagnosis and treatment results of visual diagnosis, thereby improving the accuracy and reliability of the prediction of the patient's condition.

[0197] Therefore, the present invention assists doctors in improving the consistency and accuracy of diagnosis by learning modal, multi-dimensional, and multi-level expert TCM diagnostic knowledge. At the same time, it provides patients with a privacy-protected online initial diagnosis channel, helps patients make preliminary judgments on their own conditions and symptoms, and provides a reference for subsequent online diagnosis and treatment.

[0198] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0200] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0201] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A TCM-assisted syndrome differentiation method based on artificial intelligence technology, characterized in that: include: For retrospective patient cohorts with historical cases, the consultation data obtained from the historical case database is classified, question-answer pairs are configured and coded, and a subset of consultation questions is extracted to form a consultation matrix and coded. The joint word embedding representation of the consultation data is obtained, and the basic category features extracted from the consultation data are combined to form the overall consultation features, which are added to the consultation data set after coding and annotation. For prospective patient cohorts without historical cases, question-answer pairs were obtained through questionnaires and added to the consultation data set after annotation processing; Based on expert knowledge, the collected patient image data is annotated and added to the visual diagnosis data set; Build a graph neural network and a convolutional neural network, and train them based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively, to obtain a large medical consultation-assisted diagnosis model and a visual diagnosis-assisted model; Based on the large model of auxiliary diagnosis by questioning and the auxiliary model of inspection, the patient's condition is predicted through at least one round of TCM syndrome differentiation judgment through questioning and combined with the diagnosis and treatment results of inspection, and the results of TCM syndrome differentiation in the last round of questioning are used as the auxiliary syndrome differentiation results of the patient.

2. The TCM-assisted syndrome differentiation method based on artificial intelligence technology as claimed in claim 1, characterized in that: For retrospective patient cohorts with historical cases, the consultation data obtained from the historical case database is classified, question-answer pairs are configured and coded, and a subset of consultation questions is extracted to form a consultation matrix and coded. The joint word embedding representation of the consultation data is obtained, and the basic category features extracted from the consultation data are combined to form the overall consultation features. After coding and annotation, the consultation data set is added, including: Obtain the consultation data of the retrospective patient cohort from the historical case database and divide the consultation data into multiple consultation categories; Determine the question-answer pairs for each category based on expert knowledge to form a set of medical questions, and represent each question and the corresponding answer in the set of medical questions by unique hot encoding; Extract at least one question from the set of medical questions represented by one-hot encoding to form a question subset for questioning the patient; Convert each medical question in the question subset into an embedded representation, convert the corresponding patient-selected answer into an embedded representation, and concatenate the embedded representations of the medical question and the patient-selected answer to form a feature vector embedded representation of a single question-answer pair; The feature vector embedding representations of multiple single question-answer pairs are stacked according to the sample dimension to form a question matrix. The question matrix is ​​encoded using an encoder model based on the attention mechanism to obtain a joint word embedding representation of the question data. Extract basic category features including consultation time, consultation results, basic information, current medical history, past medical history, life history, and family genetic history from the patient's historical medical records, encode at least one type of information in the basic category features, and combine the joint word embedding representation of the consultation data to form the overall consultation features; Based on expert knowledge, the TCM syndrome differentiation guidelines corresponding to the overall consultation characteristics are labeled, and the labeled consultation feature samples are added to the consultation data set.

3. The TCM-assisted syndrome differentiation method based on artificial intelligence technology as claimed in claim 1, characterized in that: For prospective patient cohorts without historical cases, question-answer pairs were obtained through questionnaires and added to the consultation data set after annotation processing, including: A questionnaire was distributed to a prospective cohort of patients without historical electronic medical records; Collect the patient's answers to the questions in the questionnaire to form a question-answer pair containing the medical question and the answer; Based on expert knowledge, the TCM syndrome differentiation guidelines corresponding to the questions and answers are annotated, and the annotated question and answer pairs are used as sample points and added to the consultation data set.

4. The TCM-assisted syndrome differentiation method based on artificial intelligence technology as claimed in claim 1, characterized in that: The collected patient image data is annotated based on expert knowledge and added to the visual diagnosis data set, including: Collecting patient image data from at least one image acquisition device to establish a visual diagnosis patient image database; For each patient image in the image database, automatic annotation is performed based on the pre-trained image processing model according to the expert knowledge in the field of traditional Chinese medicine; After all patient images are labeled, the labeled inspection features and diagnosis result samples are added to the inspection data set.

5. The TCM-assisted syndrome differentiation method based on artificial intelligence technology as claimed in claim 1, characterized in that: A graph neural network and a convolutional neural network are constructed and trained based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively. The obtained large medical consultation-assisted diagnosis model and visual diagnosis-assisted diagnosis model include: Divide the samples in the consultation data set into multiple sub-datasets, each sub-dataset contains a fixed number of patient sample points; The sample points in each sub-dataset are constructed into a graph structure representation, where the nodes of the graph structure represent the patient sample points, and the edges of the graph structure represent the similarity or implicit relationship between the sample points; Construct a patient consultation feature association encoder based on the attention mechanism in the graph neural network model to predict the association strength of the consultation feature embedding representation between any two patient consultation sample points in the graph structure representation; Using the graph neural network model, combined with the patient inquiry feature association encoder, each graph structure representation is processed in turn. The graph neural network model has a multi-layer structure, and each layer realizes the information transmission and aggregation of patient features, and finally obtains the hidden layer embedding vector of each patient sample. After being processed by a multi-layer graph neural network, a linear fully connected layer is used to convert the hidden layer embedding vector into the corresponding diagnosis and diagnosis program prediction result; The loss value is calculated based on the real labels of the patient consultation samples, and the model parameters and the associated encoder model parameters are updated based on the average loss of the whole graph nodes to obtain a trained consultation-assisted diagnosis model. A convolutional neural network is used to learn and train the image knowledge in the visual diagnosis data set, and an visual diagnosis auxiliary model is obtained that can process any image data in the same format as the visual diagnosis data set or new image data input by the user online.

6. The TCM-assisted syndrome differentiation method based on artificial intelligence technology as claimed in claim 2, characterized in that: Based on the large-scale model of medical consultation-assisted diagnosis and the auxiliary model of visual inspection, the patient's condition is predicted through at least one round of medical consultation and TCM syndrome differentiation judgment, combined with the diagnosis and treatment results of visual inspection. The results of the last round of medical consultation and TCM syndrome differentiation are used as the auxiliary syndrome differentiation results of the patient, including: Determine the total number of consultation rounds, and in each round, extract at least one question from the set of consultation questions represented by one-hot encoding for the patient to answer; In the first round of consultation, according to the medical record information and consultation answers entered by the patients, the overall consultation characteristics of the first round are obtained; Determine whether the patient simultaneously enters image data; If the patient enters the image data, it will be processed by the visual diagnosis auxiliary model to calculate the patient's symptom prediction, which will serve as the prior syndrome differentiation program vector for the patient's consultation; If the patient has not entered the image data, the anonymized and noised image selected by the patient in the visual diagnosis database will be used as the reference image and processed by the visual diagnosis auxiliary model to obtain the corresponding prior syndrome differentiation program vector; If the patient has not entered image data and has not selected a reference image, the prior syndrome differentiation program vector is initialized to a zero vector; In the first round of consultation, the overall consultation characteristics, the prior syndrome differentiation program vector and at least one patient consultation sample extracted from the consultation data set are used as inputs of the consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of the first round of consultation; In the indirect rounds from the second round of consultation to the last round, only the consultation-assisted diagnosis model is used. In each indirect round, the overall consultation characteristics of this round are obtained according to the answers entered by the patient, and the consultation-assisted diagnosis model in the previous consultation round outputs the TCM syndrome differentiation results as the prior syndrome differentiation program vector in this round. The overall consultation characteristics of this round and the prior syndrome differentiation program vector are used together as the input of the consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of this round. In the last round of consultation, the overall consultation characteristics of the current round, the prior syndrome differentiation program vector encoded from the TCM syndrome differentiation results of the previous round of consultation, and at least one patient consultation sample extracted from the consultation data set are used as input to a large consultation-assisted diagnosis model to predict the TCM syndrome differentiation results of the last round of consultation, and the TCM syndrome differentiation results of the last round of consultation are used as the auxiliary syndrome differentiation results of the patient.

7. The TCM-assisted syndrome differentiation method based on artificial intelligence technology as claimed in claim 1, characterized in that: Based on the large-scale model of medical consultation-assisted diagnosis and the auxiliary model of visual diagnosis, the patient's condition is predicted through at least one round of medical consultation-assisted Chinese medicine syndrome differentiation and judgment, combined with the diagnosis and treatment results of visual diagnosis. After the last round of medical consultation-assisted Chinese medicine syndrome differentiation results are used as the auxiliary syndrome differentiation results of the patient, it also includes: Introduce the expert experience database to annotate the auxiliary syndrome differentiation results and the corresponding recommended prescriptions, form data pairs containing syndrome differentiation results and diagnostic prescriptions, and construct the corresponding annotated data set; Construct a prescription decision tree model, determine the auxiliary syndrome differentiation results as input features, the corresponding diagnosis results or prescriptions as output targets, select information gain as the split basis for decision tree training, and recursively apply the split basis to generate nodes and branches of the decision tree; The prescription decision tree model is trained using labeled data sets to learn the association pattern between pathogenesis and prescription, and finally output a reference prescription.

8. A TCM-assisted syndrome differentiation system based on artificial intelligence technology, characterized in that: include: The consultation data set output module is used to classify the consultation data obtained from the historical case library for retrospective patient cohorts with historical cases, configure and encode question-answer pairs, and extract a subset of consultation questions to form a consultation matrix and perform encoding processing, obtain a joint word embedding representation of the consultation data, combine the basic category features extracted from the consultation data to form an overall consultation feature, and add it to the consultation data set after encoding and labeling; and, for prospective patient cohorts without historical cases, obtain question-answer pairs through questionnaires, and add them to the consultation data set after labeling processing; The visual diagnosis data set output module is used to annotate the collected patient image data based on expert knowledge and add it to the visual diagnosis data set; The model building and training module is used to build a graph neural network and a convolutional neural network, which are trained based on the graph data set converted from the medical consultation data set and the visual diagnosis data set, respectively, to obtain a large medical consultation-assisted diagnosis model and a visual diagnosis-assisted model; The auxiliary syndrome differentiation and judgment module is used to predict the patient's condition through at least one round of Chinese medicine syndrome differentiation and judgment through consultation based on the large model of auxiliary diagnosis through consultation and the auxiliary model of inspection through inspection, and use the results of the last round of Chinese medicine syndrome differentiation and judgment through consultation as the auxiliary syndrome differentiation results of the patient.

9. A TCM-assisted syndrome differentiation device based on artificial intelligence technology, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the TCM-assisted syndrome differentiation method based on artificial intelligence technology as described in any one of claims 1-7.

10. A computer-readable medium having computer-executable instructions stored thereon, characterized in that: When the executable instructions are executed by the processor, the TCM-assisted syndrome differentiation method based on artificial intelligence technology as described in any one of claims 1 to 7 is implemented.

Images