Internet hospital diagnosis and treatment method and system for contemporary AI fusion medicine

Abstract

The invention relates to an internet hospital diagnosis and treatment method and system for contemporary AI fusion medicine, and the method comprises the steps: constructing a traditional medicine AI diagnosis disease sequence model and a modern medicine AI diagnosis disease sequence model in the system in advance, and constructing a traditional medicine AI diagnosis disease sequence and a modern medicine AI diagnosis disease sequence of a patient based on the models, further constructing a contemporary AI fusion medical disease sequence of the patient, and before a treatment scheme is determined, selecting a disease needing to be treated and a treatment mode from the contemporary AI fusion medical disease sequence by the patient, so that a contemporary AI fusion medical system performs artificial intelligence processing and determines an auxiliary treatment scheme.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of internet medical treatment, and particularly relates to an AI diagnosis and treatment method and system fusing traditional medicine and modern medicine. BACKGROUND

[0002] Traditional medicine is a national medicine formed on the basis of the study of human diseases and their transmission rules from the perspective of balance and holism.

[0003] Modern medicine is a western medicine system for studying human diseases from the perspective of precise diagnosis and treatment based on modern science and emphasizing laboratory research and scientific verification.

[0004] Traditional medicine emphasizes holism and has the advantage of holistic treatment, but lacks accurate mechanism analysis and is limited by empirical diagnosis and treatment mode; modern medicine, which is developed on the basis of reductionism, has strong vitality, but it cannot explain the nonlinear characteristics of complex diseases such as senile diseases, chronic diseases and multi-system metabolic diseases by decomposing the human body into isolated organs and molecular pathways.

[0005] The separation of traditional medicine and modern medicine constitutes the deep difficulty of contemporary medical practice, and the two medical systems each have their own government, causing waste of medical resources and bottleneck of curative effect. SUMMARY

[0006] The application aims to provide an internet hospital diagnosis and treatment method and system of contemporary AI fusion medicine, which fuses traditional medicine and modern medicine based on the fusion algorithm of the application. On the one hand, the diagnosis results of traditional medicine and the diagnosis results of modern medicine are mutually superimposed and supplemented, so that the contemporary AI fusion medicine can more accurately and comprehensively reflect the state of human diseases and health in disease diagnosis; on the other hand, the diagnosis and treatment methods of traditional medicine for complex diseases and the diagnosis and treatment methods of modern medicine for complex diseases can be combined, so that the contemporary AI fusion medicine can achieve the effect of accurate treatment and holistic treatment in complex disease diagnosis and treatment.

[0007] To achieve the above-mentioned purpose, the application adopts the following technical solutions.

[0008] According to one aspect of the application, an internet hospital diagnosis and treatment method of contemporary AI fusion medicine is provided, which is applied to an internet hospital diagnosis and treatment system of contemporary AI fusion medicine and includes:

[0009] (1) starting online consultation of a patient and a digital human doctor;

[0010] (2) Collecting multi-modal original mixed medical data of a patient, including online detection medical data of the patient collected by a digital human physician guiding the patient to use an intelligent wearable device and / or a smart phone, and offline detection medical data of the patient uploaded by the patient, such as images, biochemical data, genetic data, and physical examination data;

[0011] (3) Calling an intelligent data classification model (A class) to perform modal automatic classification on the multi-modal original mixed medical data, to obtain medical data and / or medical feature data that have completed modal classification;

[0012] Before the modal automatic classification, a binary classification model is first called to perform binary classification of traditional medical data and modern medical data on the multi-modal original mixed medical data, and then the traditional medical data and the modern medical data are respectively subjected to modal automatic classification;

[0013] (4) Calling a modal exclusive preprocessing engine (B class) to perform modal exclusive preprocessing on the medical data and / or medical feature data that have completed modal classification, to obtain medical data and / or medical feature data of traditional medicine and modern medicine that have completed modal exclusive preprocessing;

[0014] (5) Calling disease diagnosis prediction models (C class and E class) to respectively perform artificial intelligence disease diagnosis prediction on the medical data and / or medical feature data of traditional medicine and modern medicine that have completed modal exclusive preprocessing, to respectively obtain disease diagnosis results of traditional medicine and modern medicine of the patient;

[0015] (6) Calling a disease sequence generator (D class and F class) and a disease sequence fusion engine to perform artificial intelligence processing on the disease diagnosis results of traditional medicine and modern medicine of the patient, to automatically generate a traditional medicine disease sequence and a modern medicine disease sequence of the patient, and to automatically fuse the traditional medicine disease sequence and the modern medicine disease sequence to generate a contemporary AI fusion medical disease sequence;

[0016] The disease sequence generation and fusion algorithm of the traditional medicine and the modern medicine is as follows:

[0017] An organ weight knowledge graph is constructed, and an exhaustive method is used to list human organs / organ systems and disease names of traditional medicine and modern medicine (among them, the names of viscera of traditional medicine are converted into organ / organ system names expressed in modern medicine according to the mapping relationship between the viscera and the organs of modern medicine, and correspondingly, the types / syndromes of traditional medicine are converted into diseases expressed in modern medicine according to the mapping relationship between the types / syndromes and the diseases of modern medicine), weight distribution is performed according to the importance of the human organs / organ systems to human health, and a first weight distribution result is obtained;

[0018] The weight rule of human organ / organ system disease staging is established, and weight distribution is performed according to the staging of the human organ / organ system, to obtain a second weight distribution result;

[0019] The weight values in the first weight distribution result and the weight values in the second weight distribution result correspond to each other, the first comprehensive weight distribution result of each disease in traditional medicine and the second comprehensive weight distribution result of each disease in modern medicine are calculated by multiplying the first weight distribution result and the second weight distribution result;

[0020] Based on the weight value size in the first comprehensive weight distribution result and the weight value size in the second comprehensive weight distribution result, each AI diagnosed disease is sorted in descending order according to the order from large to small, i.e. according to the severity of the disease;

[0021] Thus, the traditional medicine AI diagnosed disease sequence model and the modern medicine AI diagnosed disease sequence model are obtained respectively;

[0022] After determining the diagnosis result of one or more diseases of the patient diagnosed by traditional medicine AI, the data of each disease and its disease stage in the traditional medicine AI diagnosis result of the patient are obtained, and the traditional medicine AI diagnosed disease sequence of the patient is obtained based on the traditional medicine AI diagnosed disease sequence model in the contemporary AI fusion medical system;

[0023] After determining the diagnosis result of one or more diseases of the patient diagnosed by modern medicine AI, the data of each organ / organ system disease and its disease stage in the modern medicine AI diagnosis result of the patient are obtained, and the modern medicine AI diagnosed disease sequence of the patient is obtained based on the modern medicine AI diagnosed disease sequence model in the contemporary AI fusion medical system;

[0024] After obtaining the traditional medicine AI diagnosed disease sequence and the modern medicine AI diagnosed disease sequence of the patient, the contemporary AI fusion medical disease sequence of the patient is constructed, and the construction method is as follows:

[0025] The traditional medicine AI diagnosed disease sequence and the modern medicine AI diagnosed disease sequence of the same patient are merged, and the items of the same and different disease names in the two sequences are defined as the same disease items and different disease items of the respective sequences. The same disease items of the respective sequences in the two sequences are merged and included in the initial sequence of the contemporary AI fusion medical disease of the patient. The different disease items of the respective sequences in the two sequences are respectively included in the initial sequence of the contemporary AI fusion medical disease of the patient.

[0026] Weight distribution is performed on all disease items in the initial sequence of the contemporary AI fusion medical disease of the patient, and the weight distribution includes:

[0027] weighting the weight value of the first comprehensive weight of each same disease item in the traditional medicine AI diagnosed disease sequence of the patient with the weight value of the second comprehensive weight of each same disease item in the modern medicine AI diagnosed disease sequence of the patient (wherein the weight distribution between the weight value of the first comprehensive weight and the weight value of the second comprehensive weight is distributed according to a preset distribution rule, which can be optimized in time as appropriate), and setting the weight value obtained by the weighting calculation as the weight value of each same disease item in the contemporary AI fusion medical disease initial sequence of the patient;

[0028] setting the weight value of the first comprehensive weight of each different disease item in the traditional medicine AI diagnosed disease sequence of the patient as the weight value of the corresponding disease item in the contemporary AI fusion medical disease initial sequence of the patient;

[0029] setting the weight value of the second comprehensive weight of each different disease item in the modern medicine AI diagnosed disease sequence of the patient as the weight value of the corresponding different disease item in the contemporary AI fusion medical disease initial sequence of the patient;

[0030] based on the size of the weight value of each disease item in the contemporary AI fusion medical disease initial sequence of the patient, arranging each AI diagnosed disease in descending order according to the size from large to small, i.e. according to the severity of the disease;

[0031] thereby obtaining the contemporary AI fusion medical disease sequence of the patient;

[0032] (7) According to the required treatment of the disease and its treatment method selected by the patient in the contemporary AI fusion medical disease sequence, calling the treatment scheme generator artificial intelligence to process the disease and its related medical data and / or medical feature data, automatically generating the disease treatment scheme of the contemporary AI fusion medicine.

[0033] According to another aspect of the present application, a contemporary AI fusion medical internet hospital diagnosis and treatment system is provided, comprising:

[0034] Patient and digital physician interaction module, medical data acquisition module, data classification service (service A) module, traditional medical data preprocessing service (service B1) module and modern medical data preprocessing service (service B2) module, traditional medical disease diagnosis service (service C) module, traditional medical disease sequence generation service (service D) module, modern medical disease diagnosis service (service E) module, modern medical disease sequence generation service (service F) module, disease sequence merging service (service merge) module, patient medical selection service (service Treatment Selector) module, treatment plan generation service (service G / H / I) module, medical resource pre-evaluation service (service J) module, medical insurance billing service (service K) module, payment service (service Payment) module and AI scheduling engine module;

[0035] The overall architecture of the system is an event-driven multi-microservice architecture, and the system is provided with a message middleware (such as Kafka or Rabbit MQ), each service model calling link is a consumer service, each service listens to one or more topics, receives upstream data, processes and sends to the next topic in the message queue topic;

[0036] An API gateway is used to interact with patients and process patient-side requests, and internal services can communicate through gRPC or REST;

[0037] A distributed tracking (such as Jaeger) is used to monitor the whole link.

[0038] The present application has the following advantages:

[0039] 1. The digital physician can guide the patient to use the medical data acquisition device, especially to use the smart wearable device and / or smart phone at home to accurately collect the multi-modal original mixed medical data of the patient and guide the patient to upload the multi-modal original mixed medical data of offline detection such as image, biochemical, gene and physical examination;

[0040] 2. The multi-modal original mixed medical data can be classified into traditional medical data and modern medical data;

[0041] 3. The multi-modal original mixed medical data can be automatically classified and exclusively preprocessed according to the mode;

[0042] 4. The disease diagnosis results and disease sequences of traditional medicine and modern medicine can be automatically generated;

[0043] 5. The disease sequences of traditional medicine and modern medicine can be automatically fused to generate contemporary AI fusion medical disease sequences;

[0044] 6. The present application can automatically generate a disease treatment plan for modern AI integrated medicine according to the patient's choice of the required treatment of the disease and its treatment method in the modern AI integrated medical disease sequence;

[0045] 7. The present application can be widely used in countries and regions with both traditional medicine and modern medicine. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 The event-driven micro-service architecture of the diagnosis and treatment system of the present application (wherein traditional medicine is taken as an example of traditional Chinese medicine, modern medicine is taken as western medicine, and integrated medicine is taken as an example of traditional Chinese medicine combined with western medicine);

[0047] Figure 2 The whole process calling timing diagram of the diagnosis and treatment system of the present application (wherein traditional medicine is taken as an example of traditional Chinese medicine, and modern medicine is taken as western medicine); DETAILED DESCRIPTION

[0048] In order to more clearly illustrate the object, technical solution and advantages of the present application, the specific embodiments of the present application will be described in detail in the present application.

[0049] Unless specifically defined, the technical and scientific terms used in the present application should be regarded as having the standard meanings familiar to those skilled in the art. The terms such as "first", "second", "A type", "B type", "service A", "service B" and the like in the present application document are used to distinguish different parts, and not to indicate any specific order, quantity or importance level. The expressions such as "include", "contain" and the like mean that the listed items include but are not limited to the elements immediately following, and the existence of other elements is allowed.

[0050] In the present application, when describing that the system calls certain models to process related medical data through a pre-set model calling module, these models may actually be described as certain engines at times, and these models may sometimes refer to a system or certain algorithms and algorithm analysis for processing related medical data in certain cases.

[0051] In addition, in some embodiments of the present application, in addition to some technical solution contents related to the present application, some medical standards, prior art knowledge graph / database / mapping tool / algorithm / model / system / software and the like may also be involved in order to better illustrate and understand the technical solutions or technical effects of the present application.

[0052] In some embodiments, the present application is that before starting the online consultation of the patient with the digital human doctor, the system uses a pre-set digital human doctor personalized recommendation model to perform digital human doctor personalized recommendation based on the patient's preference;

[0053] The Python code of the digital human doctor personalized recommendation model implementation scheme is as follows:

[0054]

[0055] In some embodiments, the diagnosis and treatment conversation between the patient and the digital human doctor is carried out in a system multi-terminal purpose terminal diagnosis and treatment room, which includes:

[0056] A smart phone, a smart bracelet, a smart watch, a body weight instrument, etc. linked to a terminal of a modern AI fusion medical system are used as interactive devices or main detection devices, and are usually used in a purpose terminal diagnosis and treatment room in a family scene;

[0057] A desktop computer, a tablet computer, a body weight instrument, an electronic stethoscope, an electronic sphygmomanometer, an electronic thermometer, a vision detection instrument, a hearing detection instrument, etc. linked to a terminal of a modern AI fusion medical system are used as interactive devices or main detection devices, and are usually used in a target terminal diagnosis and treatment room in a school medical room scene, an enterprise and institution medical room scene, a community medical service agency medical room scene, and a drug sales store scene;

[0058] A desktop computer, a tablet computer, a body weight instrument, an electronic stethoscope, an electronic sphygmomanometer, an electronic thermometer, a vision detection instrument, a hearing detection instrument, an odor detection instrument, a pulse instrument, a blood glucose meter, an electrocardiograph, etc. linked to a terminal of a modern AI fusion medical system are used as interactive devices or main detection devices, and are usually used in a purpose terminal diagnosis and treatment room in a health and care agency scene, a physiotherapy agency scene, a nursing agency scene, a health consultation agency scene, and a drug sales store scene;

[0059] A purpose terminal diagnosis and treatment room in a clinic in a clinic scene is linked to a modern AI fusion medical system by corresponding interactive devices and detection devices;

[0060] A purpose terminal diagnosis and treatment room in a clinic in a clinic scene is linked to a modern AI fusion medical system by corresponding interactive devices and detection devices;

[0061] A purpose terminal diagnosis and treatment room in a clinic in a clinic scene is linked to a modern AI fusion medical system by corresponding interactive devices and detection devices.

[0062] In some embodiments, the online consultation conversation method between the patient and the digital human doctor includes:

[0063] Based on the superior performance of large-scale large language models (such as Wenxin Yiyang, Tongyi Qianwen, ChatGPT, Deepseek-R1, etc.) that are excavated by prompting engineering, a small-scale large language model is established to imitate learning the knowledge contained in the large-scale large language model, the small-scale large language model is supervised and fine-tuned, a medical consultation interaction small-scale large language private model is formed, and the digital human doctor and the patient and his / her assistant realize medical consultation interaction based on the medical consultation interaction small-scale large language private model, including:

[0064] Collect a small amount of high-quality medical consultation corpus (including traditional medical consultation, modern medical consultation, preventive medical consultation, and psychological counseling dialogue), rewrite part of the medical consultation corpus, insert the dialogue content of the auxiliary person participating in the dialogue, insert the dialogue content of the digital person (doctor) guiding the patient or auxiliary person to describe the symptoms and express what detection equipment there is and how to use the detection equipment correctly, form a multi-person multi-round dialogue medical consultation case corpus, manually annotate the corpus, that is, summarize the question strategy and its reply strategy for each question and its reply content in the consultation corpus topic, and manually annotate each question strategy and its reply strategy to obtain the corresponding strategy annotated corpus;

[0065] Construct a Markov chain simulating manual annotation of strategy labels by counting the strategy labels in the collected strategy annotated corpus;

[0066] Obtain multi-channel collected medical consultation multi-round dialogue data (including but not limited to crawling medical consultation multi-round dialogue data in each corresponding website), rewrite part of the medical consultation corpus, insert the dialogue content of the auxiliary person participating in the dialogue, insert the dialogue content of the digital person (doctor) guiding the patient or auxiliary person to describe the symptoms and express what detection equipment there is and how to use the detection equipment correctly, and form a data set, combine the data set after deduplication with the strategy generated based on the Markov chain to obtain a large number of rich medical consultation topics and reply strategies;

[0067] Based on the given question and strategy (group), first, through strategy matching, find similar corpus to the current strategy group in the strategy matching annotated corpus as a candidate dialogue case, that is, calculate the minimum edit distance between the current strategy group and the strategy group in the strategy annotated corpus, and then filter out all dialogue cases with a minimum edit distance less than a certain threshold as candidate dialogue cases; then use the BERT encoder to encode the candidate dialogue cases and the question into semantic vectors, and calculate the cosine similarity between them to find the most matched dialogue case as a sample, which is provided to a large-scale large language model as a multi-round dialogue reference case to better guide the large-scale large language model to generate a reply;

[0068] Use a large-scale large language model to ask and answer itself or use one or more large-scale large language models matched by the number of dialogue persons to interact in dialogue, and automatically generate high-quality medical consultation two-person or multi-person multi-round dialogue data with strategy labels;

[0069] Use a set of vector space distance loss functions to train the small-scale large language model, form a medical consultation interactive small-scale large language model, and generate a question reply based on the medical consultation interactive small-scale large language model, and realize doctor-patient consultation interaction.

[0070] In some embodiments, another conversation method for online consultation between a patient and a digital human doctor includes:

[0071] Establishing a retrieval enhancement generation technology assisted large language model to enable the assisted large language model to master more medical knowledge and more medical consultation (including traditional medical consultation, modern medical consultation, preventive medical consultation, and psychological counseling conversation) interaction methods;

[0072] The large language model includes but is not limited to Wenxin Yiyang, Tongyi Qianwen, Deepseek-R1, ChatGPT, etc.

[0073] The retrieval enhancement generation technology assisted large language model is constructed as follows:

[0074] An ontology editor such as Protege is applied to construct a knowledge graph containing rich medical knowledge, including medical literature and case studies, authoritative medical textbooks, expert consensus reports, disease diagnosis manuals, drug databases, and various information sources; a knowledge graph containing rich medical consultation interaction cases is constructed;

[0075] A retrieval model is constructed using a vector space model information retrieval technology;

[0076] A large language model is used to generate a question query, i.e., to generate current input information;

[0077] An intent recognition model is used to identify the intent of the current input information and obtain a result;

[0078] A preset prompt word generation model is used to generate a prompt word based on the current input information and the intent, and the preset prompt word generation model is trained based on a large amount of medical consultation dialogue information and corresponding prompt words;

[0079] The question query (current input information) and the prompt word are input into the retrieval model to retrieve relevant medical knowledge and relevant medical consultation interaction cases;

[0080] The retrieved medical knowledge segments and medical consultation interaction case segments are inserted into the generative large language model to complete the question reply of the generated question query;

[0081] Thus, the doctor-patient consultation interaction is realized.

[0082] In some embodiments, before the medical consultation interaction small-scale large language private model or the retrieval enhancement generation technology assisted large language model performs a question reply, a multi-person dialogue audio role recognition model is used to identify the patient identity or the assistant identity of the person in the conversation, and then a corresponding question reply is performed;

[0083] The method for establishing a multi-person dialogue audio role recognition model is as follows:

[0084] The UBM-GMM model is obtained by training the labeled speech data using the UBM-GMM algorithm; the speech pauses are determined according to the speech part and the mute part of the multi-person dialogue audio data to be recognized, the audio is segmented according to the speech pauses, and the first audio segment data after segmentation is obtained; the BIC method is used to perform second segmentation on the data with a first audio segment duration greater than 5 seconds, to determine the segmentation points of the real speakers, and the audio data is divided according to the segmentation points to obtain second audio segments, and then part of the speech samples with clustering labels are extracted as samples and put into the UBM-BMM model for quasi-pair training to obtain a quasi-pair training model, and the identity in the second audio segment is determined according to the quasi-pair training model; according to the identity recognition, the speech segments of the same identity are aggregated, the speech of the same identity is classified, and the speech data of each person in the dialogue is output.

[0085] In some embodiments, the digital physician is driven by voice, and the generation method and working process of the voice-driven digital physician include:

[0086] The voice data and video data of the target person during the inquiry are collected by the source terminal of the modern AI fusion medical system multi-terminal and preprocessed to obtain a preprocessed video, including text data, style data, face feature data, morphological feature data, and voice feature data of the target person, and head feature data and torso feature data, and are sent to the system cloud server;

[0087] A digital physician image library is established, and the establishment method of the image library includes: collecting the static front image and background picture of the target person by the source terminal of the modern AI fusion medical system multi-terminal and sending them to the system cloud server, and the static front image and background picture of the target person are stored as a preset or customized digital physician image in the digital physician image library;

[0088] The system server performs digital physician head modeling and digital physician torso modeling using the received preprocessed video, head feature data, and torso feature data;

[0089] The Muse Talk model is used as an initial head model, and the training duration, training accuracy, and loss function of the initial head model are set;

[0090] The preprocessed video and the head feature data are input into the initial head model, and mouth shape synchronous training is performed according to the training duration and the training accuracy, until the loss function of the trained initial head model converges to the training accuracy, and the digital head model is obtained according to the trained initial head model;

[0091] The original trunk model including a variational autoencoder and a NeRF network connected with each other is used to input the preprocessed video and the trunk feature data into the original trunk model for audio and trunk action association training, so as to obtain a digital physician trunk model;

[0092] When the contemporary AI fusion medical system server detects that the patient and the assistant enter the destination terminal examination room of the multi-terminal system, the pre-established digital physician image library is called, the preselected digital physician image or the digital physician image customized by the user in advance is pushed to the destination terminal examination room interactive interface, and a two-person or multi-person multi-round voice dialogue of doctor-patient interaction is started;

[0093] After calling the medical inquiry interactive small-scale large language private model inserted with the multi-person dialogue audio role recognition model or calling the retrieval enhancement generation technology assisted large language model inserted with the multi-person dialogue audio role recognition model, the question voice input by the patient and / or the assistant is acquired, and the corresponding reply text is output for the patient and the assistant after audio role recognition of the question voice;

[0094] The reply text is converted into initial voice, the tone of the target character audio is extracted from the target character audio data of the digital physician, the initial voice is converted again according to the tone of the target character audio, and the reply voice of the target character is obtained;

[0095] The reply voice is input into the digital physician head model and the digital physician trunk model respectively, the corresponding digital physician head video and digital physician trunk video are generated, and the digital physician head video and digital physician trunk video are video spliced and image high-definition processed to obtain a digital physician video;

[0096] The system cloud server encodes the digital physician video according to the transmission code rate of the digital physician video and sends the encoded digital physician video to the destination terminal examination room interactive interface, so as to realize the medical inquiry interaction between the digital physician and the patient.

[0097] In some embodiments, another generation method and working process of the voice-driven digital physician includes:

[0098] The voice data and video data of the target character during the inquiry are collected by the source terminal of the contemporary AI fusion medical system multi-terminal, and are preprocessed to obtain preprocessed video, including text data, style data, facial feature data, morphological feature data, sound feature data of the target character, head feature data and trunk feature data, and are sent to the system server;

[0099] When the system server is instructed by the destination terminal of the multi-terminal of the contemporary AI fusion medical system to generate a digital human physician, the system server sends the preprocessed video received from the source terminal, the text data, style data, facial feature data, morphological feature data, and voice feature data of the target human, and the received head feature data and torso feature data to the destination terminal after determining that the destination terminal has the generation capability;

[0100] Establishing a digital human physician image library, the method for establishing the image library comprises: collecting static front images and background pictures of a target person by a source terminal of a multi-terminal of a contemporary AI fusion medical system and sending them to a system cloud server, the static front images and background pictures of the target person are stored as pre-set or customized digital human physician images in the digital human physician image library;

[0101] The destination terminal uses the received preprocessed video, head feature data, and torso feature data to perform digital human physician head modeling and digital human physician torso modeling;

[0102] Using the Muse Talk model as an initial head model, and setting the training duration, training accuracy, and loss function of the initial head model;

[0103] Inputting the preprocessed video and the head feature data into the initial head model, performing mouth shape synchronous training according to the training duration and training accuracy until the loss function of the trained head model converges to the training accuracy, and obtaining a digital human head model according to the trained initial head model;

[0104] Using an original torso model including a variational autoencoder and a NeRF network connected to each other, inputting the preprocessed video and the torso feature data into the original torso model to perform audio and torso motion correlation training, and obtaining a digital human physician torso model;

[0105] When the system cloud server of the contemporary AI fusion medical system detects that a patient and an assistant enter a destination terminal of a system multi-terminal examination room, calling a pre-established digital human physician image library, pushing pre-set selectable digital human physician images or digital human physician images customized by a user in advance to an interactive interface of the destination terminal examination room, and starting a multi-round voice conversation of two or more people for doctor-patient interaction;

[0106] Calling a medical inquiry interaction small-scale large language private model with a multi-person dialogue audio role recognition model inserted or calling a retrieval enhancement generation technology assisted large language model with a multi-person dialogue audio role recognition model inserted, obtaining question voice input by a patient and / or an assistant, performing audio role recognition on the question voice, and outputting corresponding reply text for the patient and the assistant respectively;

[0107] Convert the reply text into initial speech, and extract the tone of the target character audio from the target character audio data of the digital human doctor. The initial speech is converted again according to the tone of the target character audio to obtain the reply speech of the target character;

[0108] The reply speech is input into the digital human doctor head model and the digital human doctor torso model, respectively, to drive the generation of corresponding digital human doctor head video and digital human doctor torso video, and the digital human doctor head video and digital human doctor torso video are video spliced and image high-definition processed to obtain the digital human doctor video generated by the destination terminal in local driving digital human doctor, and presented in the destination terminal diagnosis room interactive interface, realizing the medical inquiry interaction between digital human doctor and patient.

[0109] In some embodiments, the traditional medical inquiry content of the patient and the digital human doctor includes "Ten Questions Song": one question about cold and heat, two questions about sweat, three questions about head and body, four questions about stool, five questions about diet, six questions about chest, seven deafness, eight thirst, nine questions about old diseases, and ten questions about causes; The modern medical inquiry content includes system review ROS (Review Of Systems).

[0110] In some embodiments, the digital human doctor guides the patient to use the corresponding medical data acquisition equipment (such as smart wearable devices and / or smart phones used at home) to collect and obtain the patient's multi-modal original mixed medical data related to the disease during the inquiry process;

[0111] After obtaining the multi-modal original mixed medical data of the patient, the system calls the intelligent data classification model (A class) through the preset model calling module to automatically classify the obtained multi-modal original mixed medical data by mode, and obtains the medical data and / or medical feature data of the patient after completing the mode classification;

[0112] The Python code of the intelligent data classification model (A class) implementation scheme is as follows (wherein traditional medicine takes traditional Chinese medicine as an example, and modern medicine is referred to as western medicine):

[0113]

[0114] In some embodiments, before calling the intelligent data classification model to automatically classify the multi-modal original mixed medical data by mode, a two-classification model is called to two-classify the data into traditional medical data and modern medical data, and then the multi-modal classification is performed on the traditional medical data and the modern medical data, respectively. Taking the two-classification of traditional Chinese medicine and western medicine medical data as an example, the two-classification model is constructed as follows:

[0115] In the data preparation stage, 10000 Chinese medicine and western medicine medical text samples are collected and data labeling is completed;

[0116] Feature engineering, extract traditional Chinese medicine (TCM) and Western medicine (WM) specific terms as keyword features, such as TCM specific terms like qi, yin and yang, five elements, meridians, acupoints, and Chinese medicine names, and WM specific terms like antibiotics, surgical names, Western medicine chemical names, and anatomy terms;

[0117] Model selection and training, combining traditional features and deep learning models, such as BERT embedding + custom features → classifier;

[0118] Model selection and training, combining traditional features and deep learning models, such as BERT embedding + custom features → classifier;

[0119] API deployment, using Fast API or Flask to build RESTful API, inputting raw medical text and outputting classification results and confidence;

[0120] Mixed content processing, considering that the text contains mixed content of TCM and WM, a "mixed" category can be set, and the output probability is not only hard classification.

[0121] In some embodiments, after obtaining the completed modal classification medical data and / or medical feature data of the patient, the system calls the preset modal-specific preprocessing engine (class B) to perform modal-specific preprocessing on the obtained completed modal classification medical data and / or medical feature data, forming traditional and modern medical data and / or medical feature data suitable for subsequent artificial intelligence processing through a preset model calling module;

[0122] The implementation scheme Python code of the modal-specific preprocessing engine (class B) is as follows (wherein traditional medicine is taken as an example of TCM, and modern medicine is referred to as WM):

[0123]

[0124]

[0125] In some embodiments, the video (inspection / pulse) medical data preprocessing implementation scheme Python code of the present application is as follows (wherein traditional medicine is taken as an example of TCM):

[0126]

[0127] In some embodiments, the audio (auscultation) medical data preprocessing implementation scheme Python code is as follows:

[0128]

[0129]

[0130] In some embodiments, the signal class (PPG / ECG) medical data preprocessing implementation Python code is as follows:

[0131]

[0132] In some embodiments, the digital physician guides the patient and his / her caregivers to use the video acquisition device (including a smart phone adaptation) linked to the system to collect the patient's face, tongue, eye, and skin lesion videos. The system calls the modal-specific preprocessing engine according to the detected acquisition device and the user terminal's capability boundary parameters, and automatically adapts the following video preprocessing methods: one method, the smart phone automatically adapts infrared camera in low light environment, ensuring that the video taken in low light environment can still maintain clarity; the second method, preprocessing the video based on the preset video enhancement model; the third method, based on the smart phone adaptation three camera shooting and based on the preset three-dimensional reconstruction model preprocessing video; the fourth method, based on the smart phone adaptation three camera shooting and based on the preset video enhancement and three-dimensional reconstruction joint modeling preprocessing video; thus obtaining clearer, containing more physiological information facial video and video frame image, tongue video and video frame image, eye video and video frame image, skin lesion video and video frame image;

[0133] The computer video algorithm model such as YoLov7 backbone network is called for data preprocessing to detect facial image and eye image features, obtain facial complexion state data, including: red, pale, yellow, cyan and swelling; obtain facial skin condition data, including: dry, oily, mixed, sensitive, acne, dull, loose, pigmentation and dermatitis; and obtain eye white color data;

[0134] The computer vision algorithm model such as YoLov5s6 is called for tongue detection, the U-Net network model for data preprocessing is called for tongue segmentation, the network classification model such as Mobilenet V3 for data preprocessing is called for tongue image feature recognition, obtain tongue quality data, including: old and tender, fat and thin, pointy, cracked and tooth marks; and obtain moss quality data, including: thick and thin, wet and dry, rotten and greasy, peeling and true and false;

[0135] The computer vision algorithm model YoLov7 backbone network for data preprocessing is called for skin lesion image feature detection, obtain skin lesion color data, skin lesion shape data and skin lesion distribution data;

[0136] In some embodiments, the digital human physician guides the patient and his or her assistant to use the video capture device (including a smart phone adapter) linked to the system to capture the video of the patient's wrist, obtain the video frame image of the wrist, call the data pre-processing model, i.e., the preset wrist flex artery positioning model to obtain the wrist image labeled with the precise positioning point of the wrist flex artery, and send the image to the doctor-patient interaction interface of the user terminal (including a smart phone adapter), in which the image of the digital human physician and the wrist image labeled with the flex artery positioning point can be displayed on the screen, and the patient or his or her assistant can compare the patient's real wrist with the wrist image labeled with the flex artery positioning point on the screen, and manually label the flex artery positioning point on the patient's real wrist; further, the digital human physician guides the patient or his or her assistant to use the video capture device (including a smart phone adapter) linked to the system to capture the flex artery pulse video of the flex artery positioning point, and the system automatically adapts the following video processing methods according to the detected capability boundary parameters of the capture device and the user terminal by the modal exclusive pre-processing engine: the first method, the smart phone automatically adapts infrared camera in low light environment to ensure that the video taken in low light environment can still be clear; the second method, calling the data pre-processing model, i.e., the preset video enhancement model to process the video; the third method, calling the data pre-processing model, i.e., the preset three-dimensional reconstruction model to process the video; the fourth method, calling the data pre-processing model, i.e., the preset joint modeling of video enhancement and three-dimensional reconstruction to process the video, obtaining clearer flex artery pulse video containing more physiological information; the following methods are used to further process the flex artery pulse video data to obtain the patient's pulse syndrome data or pulse pattern data: the first method, calling the data pre-processing model, i.e., the pulse video feature extraction model to obtain the pulse video feature vector, calling the data pre-processing model, i.e., the preset graph convolution network to mine the relationship between the pulse pattern label data (such as 28 kinds of pulse) and non-image features (such as age, gender, height, body shape, constitution, season, etc.), and to perform multi-label classification of pulse to obtain pulse relationship feature vector, calling the data pre-processing model, i.e., the preset data fusion model to obtain the patient's multi-modal pulse fusion feature data; calling the data pre-processing model, i.e., the preset pulse pattern recognition model to obtain the patient's pulse pattern data; the second method, calling the data pre-processing model, i.e., the preset video micro-motion amplified pulse video digital information data extraction model to obtain the patient's flex artery pulse digital information data, calling the data pre-processing model, i.e., the preset neural network model and pulse data analysis algorithm model to obtain the patient's pulse syndrome data.Third method, in the comparison of the patient's real wrist with the wrist image in the screen with the labeled radial artery positioning point by the digital human physician guiding the patient or his assistant, after the patient and his assistant manually labeling the radial artery positioning point on the patient's real wrist, the digital human physician guides the patient and his assistant to use the pulse signal acquisition device (including smart wearable device adaptation) to measure the pulse beat data based on the photoelectric sensor or pressure sensor or multi-modal sensing array on the device, and call the data processing model, that is, the preset pulse beat information feature extraction model to obtain the pulse symbol type data of the patient; the fourth method, in the comparison of the patient's real wrist with the wrist image in the screen with the labeled radial artery positioning point by the digital human physician guiding the patient or his assistant, and after the patient and his assistant manually labeling the radial artery positioning point on the patient's real wrist, the digital human physician guides the patient and his assistant to use the pulse signal acquisition device (including smart wearable device adaptation) to measure the pulse beat data based on the pressure sensor on the device, and after converting the pressure signal into an electrical signal, generate a human pulse symbol change curve, call the data preprocessing model, that is, the preset pulse data analysis algorithm model to analyze the pulse beat data, and according to the analysis results of the four dimensions of pulse position, pulse number, pulse type and pulse potential, obtain the pulse symbol type data of the patient's 28 pulses;

[0137] In some embodiments, the digital human physician can guide the patient and his assistant to use the sound acquisition device (including smart phone adaptation) connected with the system to collect the vocal color data of the patient's voice, including clear, fuzzy and hoarse conditions; the vocal tone data, including normal and abnormal conditions; the vocal loudness data, including smooth, irregular and intermittent conditions.

[0138] In some embodiments, the digital human physician guides the patient and his assistant to use the smell signal acquisition device (such as smart wearable device with electronic nose) connected with the system to collect the smell signals of different parts of the patient's body (such as oral cavity, nasal cavity, underarm part) and secretions and excretions, and the system calls the data preprocessing model, that is, the preset smell signal classification model, to process respectively to obtain the smell type data of different parts of the body and secretions and excretions in the smell diagnosis, including sour smell, turbid smell, stench smell, fishy smell, rancid smell, rotten apple smell, garlic smell, urine stench smell (ammonia smell), urine smell.

[0139] In some embodiments, the digital physician guides the patient and his / her assistant to use the video acquisition device (including a smart phone adapter) connected to the system to collect the patient's facial video, eye movement video, and skin lesion video, and the system calls a data preprocessing model according to the detected capability boundary parameters of the acquisition device and the user terminal, and automatically adapts the following video processing methods: one method, calling a data preprocessing model, i.e., a preset video enhancement model to process the video; the second method, calling a data preprocessing model, i.e., a preset three-dimensional reconstruction model to process the video; the third method, calling a data preprocessing model, i.e., a preset joint modeling of video enhancement and three-dimensional reconstruction to process the video; thereby obtaining clearer facial video data, eye movement video data, and skin lesion video data containing more physiological information.

[0140] In some embodiments, the digital physician guides the patient and his / her assistant to use the video acquisition device (including a smart phone adapter) connected to the system to collect the patient's facial video, eye movement video, and skin lesion video, and the system calls a data preprocessing model according to the detected capability boundary parameters of the acquisition device and the user terminal, and automatically adapts the following video processing methods: one method, calling a data preprocessing model, i.e., a preset video enhancement model to process the video; the second method, calling a data preprocessing model, i.e., a preset three-dimensional reconstruction model to process the video; the third method, calling a data preprocessing model, i.e., a preset joint modeling of video enhancement and three-dimensional reconstruction to process the video; thereby obtaining clearer facial video data, eye movement video data, and skin lesion video data containing more physiological information.

[0141] In some embodiments, the digital physician guides the patient and his / her caregivers to collect data using the smart wearable devices or smart phones connected with the system, including medical data such as photoplethysmogram (PPG) or single-lead electrocardiogram (ECG), electroencephalogram (EEG), heart rate, blood pressure, blood glucose, blood oxygen saturation, sweat content (e.g., glucose concentration, lactic acid concentration, pH value), body temperature, respiratory rate, exercise intensity, sleep quality, etc. The data preprocessing model, i.e., the preset PPG and ECG signal conversion model and ECG lead diffusion generation model, is called to convert PPG into ECG through the preset signal conversion model. In addition, the ECG collected by the smart wearable devices and smart phones is usually single-lead ECG, and therefore, the ECG lead diffusion generation model is migrated to the collected single-lead ECG data through the preset ECG lead diffusion generation model using transfer learning, and multi-lead (e.g., 8-lead, 12-lead) ECG signal reconstruction is performed on the collected single-lead ECG data to obtain reconstructed multi-lead ECG data.

[0142] In some embodiments, the offline detection medical data uploaded by the patient includes:

[0143] Medical imaging data, such as chest X-ray, magnetic resonance imaging (MRI), CT device generated deep tissue cross-sectional image, gastrointestinal endoscopy image, blood vessel image, etc.

[0144] Genetic sequencing data

[0145] Physical examination data, such as 18 items of blood routine, i.e., WBC, LYN, GRAN, RBC, HGB, HCT, MCV, MCH, MCHC, RDW, PLT, MPV, PDW, PCT, MON, LRR%, RPR%, MPR%; 10 items of urine routine, i.e., SG, PH, LEU, NIT, PRO, GLU, KET, UBG, U-BiL, ERY; 11 items of liver function, i.e., ALT, GOT, GGT, TPO, ALB, GLO, A / G, ALP, LDH, TBS, and direct bilirubin; 4 items of blood lipid, i.e., total cholesterol, triglyceride, HDL-Cholesterol, LDL-Cholesterol; 3 items of kidney function, i.e., BUN, CR, UA; calcium Ca content, phosphorus P content, iron Fe / SI content; 7 items of tumor, i.e., CEA, a-FA / AFP, PSA, CA15-3, CA19-9, CA125, TSGF; and 15 kinds of trace elements in blood or hair, i.e., calcium Ca, magnesium Mg, zinc Zn, iron Fe, copper Cu, manganese Mn, strontium Sr, chromium Cr, molybdenum Mo, cobalt Co, selenium Se, nickel Ni, lead Pb, cadmium Cd, aluminum Al.

[0146] In some embodiments, after obtaining the patient uploaded medical image data without image analysis, the system calls the specialized AI image analysis commercial software / model through the preset model calling module to perform image analysis on the image, and obtains the patient's image analysis medical data.

[0147] The AI medical image analysis commercial software / model is listed as follows:

[0148] 1. Tencent Yijing, which can be applied to CT lung slice analysis, early cancer screening, etc.

[0149] 2. Infervision, which can be used for lung nodule detection, DR chest X-ray diagnosis, etc.

[0150] 3. ShuKun Technology, which can be applied to coronary CTA analysis, pulmonary embolism detection, etc.

[0151] 4. Aidoc, which can be applied to CT / MRI emergency case priority triage.

[0152] 5. Zebra MedicalVision, which can be applied to CT / X image analysis.

[0153] In some embodiments, after obtaining the modality-specific preprocessed medical data and / or medical feature data of traditional medicine and modern medicine, the system calls the preset disease diagnosis prediction model (C and E) through the preset model calling module to perform artificial intelligence processing on the medical data and / or medical feature data, and obtains the disease diagnosis results of traditional medicine and modern medicine of the patient respectively.

[0154] The implementation scheme Python code of the disease diagnosis prediction model (C and E) is as follows (wherein traditional medicine takes traditional Chinese medicine as an example, and modern medicine is referred to as western medicine):

[0155]

[0156]

[0157] In some embodiments, when the disease diagnosis prediction model is called to perform artificial intelligence disease diagnosis prediction on the medical data and / or medical feature data of traditional medicine and modern medicine which have completed modality-specific preprocessing, the high specificity model is preferentially called in combination with the patient's complaint and available data types, and other models are selected as supplementary auxiliary models.

[0158] For example, when the patient complains of heart and lung conditions, and the data type is heart and lung auscultation sound, the high specificity heart and lung sound disease diagnosis model is called, for example, the following heart and lung sound disease diagnosis model, the architecture of which includes: constructing heart sound and lung sound data set and heart sound algorithm model and lung sound algorithm model, pre-training heart sound algorithm model and lung sound algorithm model using heart sound and lung sound data set, using data enhancement classification algorithm to input the heart sound and lung sound data collected by the stethoscope into the algorithm model, and outputting the diagnosis classification.

[0159] In some embodiments, after obtaining the traditional medicine and modern medicine disease diagnosis results of the patient respectively, the system automatically generates the traditional medicine disease sequence and the modern medicine disease sequence of the patient by calling the preset disease sequence generator (D class and F class) and disease sequence fusion engine through the preset model calling module for artificial intelligence processing, and automatically fuses the traditional medicine disease sequence and the modern medicine disease sequence to generate the contemporary AI fusion medicine disease sequence.

[0160] The implementation scheme Python code of the disease sequence generator (D class and F class) is as follows (only the implementation scheme code of the disease sequence generator D class for traditional medicine is shown here, because the implementation scheme for modern medicine, i.e., western medicine, is the same as that for traditional medicine, and the F class is omitted; wherein the traditional medicine takes traditional Chinese medicine as an example):

[0161]

[0162]

[0163] In some embodiments, the syndromes / syndromes of traditional medicine in the present application are converted into disease names expressed in modern medicine according to the mapping relationship thereof with the disease names of modern medicine. For example, taking traditional Chinese medicine as an example, the establishment of the mapping relationship can refer to the applicable national standard (GB / T 15657-2021) of Chinese medicine syndrome classification and code, traditional Chinese medicine syndrome-western medicine disease knowledge graph (TCM-syndrome-Disease KG), DISEASES / TCM database, ICD-11-TM mapping tool (WHO traditional medicine module), etc.

[0164] In some embodiments, when merging the traditional medicine AI diagnosis disease sequence and the modern medicine AI diagnosis disease sequence of the same patient, the same disease item involved in each sequence is merged, that is, the weight value of the first comprehensive weight of each same disease item in the traditional medicine AI diagnosis disease sequence of the patient is weighted and calculated with the weight value of the second comprehensive weight of each same disease item in the modern medicine AI diagnosis disease sequence of the patient, wherein the weight distribution between the weight value of the first comprehensive weight and the weight value of the second comprehensive weight is distributed according to a preset distribution rule; when the distribution rule is preset, considering that modern medicine has the advantage of accuracy in disease diagnosis, a larger weight can be assigned to the "weight value of the second comprehensive weight of each same disease item in the modern medicine AI diagnosis disease sequence of the patient", and a smaller weight can be assigned to the "weight value of the first comprehensive weight of each same disease item in the traditional medicine AI diagnosis disease sequence of the patient", for example, the former is assigned a weight of 0.8, and the latter is assigned a weight of 0.2.

[0165] In some embodiments, after generating the AI diagnosis disease sequence of traditional medicine and the AI diagnosis disease sequence of modern medicine, the two disease sequences are merged, and the implementation scheme Python code is as follows (wherein traditional medicine takes traditional Chinese medicine as an example, modern medicine is referred to as western medicine, and integrated medicine takes traditional Chinese medicine combined with western medicine as an example):

[0166]

[0167]

[0168] In some embodiments, after the patient selects the required treatment disease and its treatment method in the contemporary AI integrated medicine disease sequence, the system calls the treatment scheme generator (G / H / I class) through a preset model calling module, and the artificial intelligence processes the disease and its related medical data and / or medical feature data to generate a scheme for treating the disease in the contemporary AI integrated medicine;

[0169] The implementation method Python code of the treatment scheme generator (G / H / I class) is as follows (wherein traditional medicine takes traditional Chinese medicine as an example, modern medicine is referred to as western medicine, and integrated medicine takes traditional Chinese medicine combined with western medicine as an example)

[0170]

[0171]

[0172] In some embodiments, when the system calls the treatment plan generator to process the patient's required treatment of the disease in the AI-fused medical disease sequence and its treatment method, and generates a treatment plan, a deep learning end-to-end model, a pre-trained LLM fine-tuning system (such as TCM-GPT, Chat Doctor, Med-PaLM), a retrieval-augmented generation (RAG) system (such as UpTo Date+GPT), a knowledge graph-driven system based on disease-treatment plan graph path reasoning (such as TCM-KG, OncoGraph), and other systems can be called, such as models calling RAG+knowledge graph hybrid architecture, to ensure that the generated treatment plan has both evidence-based and explainable properties.

[0173] In some embodiments, the treatment plan generated by the treatment plan generator includes: a surgical plan, a physical therapy plan, a chemotherapy plan, a prescription and dispensing plan, a nursing plan, a rehabilitation and health care plan, a psychological therapy plan, a diet therapy plan, a chronic disease treatment and health conditioning package plan for the elderly, etc.

[0174] In some embodiments, after generating the disease treatment plan of the contemporary AI-fused medicine, according to the multi-dimensional needs of the patient for medical resources (including the priority needs for medical quality or medical expenses), the system calls the medical resource evaluation model (class J) through the preset model calling module to extract the required resources and obtain the candidate resources in the medical resources registered in the system client, and performs multi-dimensional scoring to generate the preferred disease treatment plan of the contemporary AI-fused medicine for the patient.

[0175] The implementation scheme of the medical resource evaluation model (class J) is as follows (wherein traditional medicine is taken as an example of traditional Chinese medicine, and modern medicine is referred to as western medicine):

[0176]

[0177] In some embodiments, the medical resources of the contemporary AI-fused medicine system client include:

[0178] Offline treatment service resources: treatment service resources for implementing surgical therapy treatment plans, physical therapy treatment plans, chemotherapy treatment plans, nursing plans, rehabilitation and health care plans, and any combination of the above plans;

[0179] Online and offline medical product resources: Western medicine, health products, equipment, appliances; Chinese medicine, health products, equipment, appliances; Indian medicine, health products, equipment, appliances; Arab medicine, health products, equipment, appliances; Persian medicine, health products, equipment, appliances; Latin American medicine, health products, equipment, appliances; Japanese medicine, health products, equipment, appliances; Korean medicine, health products, equipment, appliances; North Korean medicine, health products, equipment, appliances; Vietnamese medicine, health products, equipment, appliances.

[0180] In some embodiments, the medical / health / health care / psychological counseling institutions that have settled in the contemporary AI fusion medical system client are allowed to set up the client's diagnosis and treatment room according to the preset rules and provide corresponding services to patients using part or all of the services of the contemporary AI fusion medical system.

[0181] In some embodiments, after sending one or more alternative disease treatment options of the contemporary AI fusion medicine or multiple alternative sub-disease treatment options of each disease treatment option of the patient to the patient, the system sends the disease treatment option or sub-disease treatment option selected by the patient to the doctor / pharmacist prescription review verification library according to the disease treatment option or sub-disease treatment option selected by the patient, so that the doctor / pharmacist can prescribe, review and verify the treatment option or sub-treatment option in a preset order receiving / order grabbing manner. During the prescription review and verification process, the doctor / pharmacist can also interact with the patient, and then send the verified treatment option or sub-treatment option to the patient.

[0182] In some embodiments, after obtaining the medical data or medical feature data of the patient in traditional medicine, the medical data or medical feature data of the patient in modern medicine, the contemporary AI fusion medical disease diagnosis result of the patient, the disease sequence of the patient in contemporary AI fusion medicine, the disease selected by the patient for treatment and the corresponding treatment option or sub-treatment option, the system automatically makes an information table for each patient and uploads it to the block chain node for storage.

[0183] In some embodiments, after obtaining the preferred disease treatment option of the contemporary AI fusion medicine, the system calls the treatment option collaborative optimization model to collaboratively optimize the preferred disease treatment option through a preset model calling module.

[0184] The Python code for implementing the treatment option collaborative optimization model is as follows (wherein traditional medicine is taken as an example of traditional Chinese medicine, modern medicine is taken as an example of Western medicine, and fusion medicine is taken as an example of traditional Chinese medicine combined with Western medicine):

[0185]

[0186] In some embodiments, after generating the preferred disease treatment plan of the patient's contemporary AI fusion medicine, the system invokes the medical insurance billing model (K class) through the preset model calling module to bill the preferred disease treatment plan;

[0187] The implementation scheme Python code of the medical insurance billing model (K class) is as follows:

[0188]

[0189] In some embodiments, the whole process calling sequence of the contemporary AI fusion medical internet hospital diagnosis and treatment system of the application is as shown in Figure 2

[0190] In some embodiments, the service model of the contemporary AI fusion medical internet hospital diagnosis and treatment system of the application includes (traditional medicine takes traditional Chinese medicine as an example, modern medicine is called western medicine, and fusion medicine takes traditional Chinese and western medicine combination as an example):

[0191] Class A model: data classification model input: patient uploaded original mixed data (including traditional Chinese medicine four diagnostic data, western medicine detection data, image, biochemistry, gene, physical examination, etc.)

[0192] Output: classify the data into two categories of traditional Chinese medicine data and western medicine data

[0193] Class B model: modal exclusive preprocessing

[0194] Including two branches: traditional Chinese medicine data preprocessing and western medicine data preprocessing

[0195] Traditional Chinese medicine data preprocessing: including preprocessing of tongue diagnosis, face diagnosis, pulse diagnosis and other data (such as video enhancement, three-dimensional reconstruction, etc.) Western medicine data preprocessing: including image reading, biochemical index standardization, gene data annotation, etc.

[0196] Class C model: traditional Chinese medicine disease diagnosis prediction

[0197] Input: preprocessed traditional Chinese medicine data

[0198] Output: predict traditional Chinese medicine diseases and stages

[0199] Class D model: traditional Chinese medicine disease sequence generation input: output of class C model (disease and stage) and preset traditional Chinese medicine disease sequence model with organ / organ system weight and stage weight label

[0200] Output: traditional Chinese medicine disease sequence sorted by severity

[0201] Class E model: western medicine disease diagnosis prediction

[0202] Input: preprocessed western medicine data ​

[0203] Output: Predicted Western medical diseases and stages

[0204] Class F model: Western medical disease sequence generation input: Output of class E model (diseases and stages) and preset Western medical disease sequence model with organ / organ system weight and stage weight label

[0205] Output: Western medical disease sequence sorted by severity

[0206] Merge model: merge TCM disease sequence and Western medical disease sequence into integrated TCM and Western medical disease sequence rule: two-way diagnosis is mutually superimposed and complementary (such as disease merger of the same organ, comprehensive weight weighted calculation, etc.)

[0207] According to the patient's choice of one or more diseases needing treatment and the selected treatment method, call the corresponding model to generate a treatment plan: G: Integrated TCM and Western medical treatment plan model

[0208] H: TCM treatment plan model

[0209] I: Western medical treatment plan model

[0210] Input: preprocessed data (TCM and / or Western medicine) and selected diseases

[0211] Output: Treatment plan (may include drugs, surgery, rehabilitation recommendations, etc.)

[0212] J class model: medical resource pre-evaluation model input: resources required in the treatment plan (such as drugs, hospitals, equipment, etc.) and patient quality requirements and price requirements output: evaluation results (such as recommended hospitals, pharmacies, etc.)

[0213] K class model: medical insurance billing model

[0214] Input: treatment plan and medical resources

[0215] Output: cost details (including medical insurance payment and personal payment portion).

[0216] Payment and logistics: after the patient pays, trigger logistics distribution

[0217] In some embodiments, the message queue topics of the contemporary AI fusion medicine internet hospital diagnosis and treatment system of the present application include (traditional medicine takes TCM as an example, modern medicine is called Western medicine, and fusion medicine takes integrated TCM and Western medicine as an example):

[0218] • patient_raw_data: raw data

[0219] • classified_data: Classification data output by the A-class model (divided into tcm_data and western_data)

[0220] • preprocessed_tcm_data: TCM data preprocessed by the B-class model

[0221] • preprocessed_western_data: Western medicine data preprocessed by the B-class model

[0222] • tcm_diagnosis: TCM disease diagnosis output by the C-class model

[0223] • tcm_disease_sequence: TCM disease sequence output by the D-class model

[0224] • western_diagnosis: Western medicine disease diagnosis output by the E-class model

[0225] • western_disease_sequence: Western medicine disease sequence output by the F-class model

[0226] • integrated_disease_sequence: Integrated disease sequence

[0227] • treatment_plan_request: Patient-selected treatment request (including selected disease and treatment method)

[0228] • treatment_plan: Treatment plan generated by the G / H / I-class model

[0229] • resource_evaluation: Resource evaluation output by the J-class model

[0230] • billing: Fee output by the K-class model

[0231] In some embodiments, as shown in Figure 1 , the microservices of the contemporary AI-fused medical internet hospital diagnosis and treatment system of the present application include (traditional medicine takes TCM as an example, modern medicine is called Western medicine, and fusion medicine takes TCM-WM integration as an example): Service A: Data classification service

[0232] Listen: Patient raw data

[0233] Send: Classification data (contains two fields of TCM data and Western medicine data)

[0234] Service B1: TCM data preprocessing service

[0235] Listen: Classified data (extracting TCM data)

[0236] Send: Preprocessed TCM data

[0237] Service B2: Western medicine data preprocessing service

[0238] Listen: Classified data (extracting Western medicine data)

[0239] Send: Preprocessed Western medicine data

[0240] Service C: TCM disease diagnosis service

[0241] Listen: Preprocessed TCM data

[0242] Send: TCM disease diagnosis

[0243] Service D: TCM disease sequence generation service

[0244] Listen: TCM disease diagnosis

[0245] Send: TCM disease sequence

[0246] Service E: Western medicine disease diagnosis service

[0247] Listen: Preprocessed Western medicine data

[0248] Send: Western medicine disease diagnosis

[0249] Service F: Western medicine disease sequence generation service

[0250] Listen: Western medicine disease diagnosis

[0251] Send: Western medicine disease sequence

[0252] Service merge: Disease sequence merge service

[0253] Listen: TCM disease sequence and Western medicine disease sequence

[0254] Send: Merged disease sequence

[0255] Service: Patient selection service (interacts with patients through API gateway)

[0256] Listen: Merged disease sequence (and waits for patient selection)

[0257] Send: Patient-selected treatment request (contains patient-selected disease and treatment method) Service G / H / I: Treatment plan generation service (calls different models based on treatment method) Listen: Patient-selected treatment request

[0258] If the treatment method is "integrated traditional Chinese and Western medicine": Call G-class model

[0259] If treatment method is "TCM": invoke H-class model

[0260] If treatment method is "Western medicine": invoke I-class model

[0261] Send: treatment plan

[0262] Service J: medical resource pre-evaluation service

[0263] Listen: treatment plan

[0264] Send: resource evaluation

[0265] Service K: medical insurance billing service

[0266] Listen: resource evaluation

[0267] Send: medical expense payment service: listen to medical expenses and wait for payment to be completed, then trigger logistics.

Claims

1. An internet hospital diagnosis and treatment method of the current AI fusion medicine, characterized in that, The contemporary AI fusion medical internet hospital diagnosis and treatment method comprises: (1) starting online consultation of patients and digital human doctors; (2) collecting multi-modal original mixed medical data of patients, including offline detection medical data uploaded by patients, such as images, biochemistry, genes, physical examination, etc., and online detection medical data of patients collected by digital human doctors using intelligent wearable devices and / or smart phones; (3) calling an intelligent data classification model (A class) to automatically classify the multi-modal original mixed medical data by modalities, to obtain medical data and / or medical feature data that have completed modality classification; Before the automatic modality classification, a binary classification model is first called to perform binary classification of traditional medical data and modern medical data, and then the traditional medical data and the modern medical data are respectively automatically classified by modalities; (4) calling a modality-specific preprocessing engine (B class) to perform modality-specific preprocessing on the medical data and / or medical feature data that have completed modality classification, to obtain medical data and / or medical feature data of traditional medicine and modern medicine that have completed modality-specific preprocessing; (5) calling disease diagnosis prediction models (C class and E class) to respectively perform artificial intelligence disease diagnosis prediction on the medical data and / or medical feature data of traditional medicine and modern medicine that have completed modality-specific preprocessing, to respectively obtain disease diagnosis results of traditional medicine and modern medicine of the patient; (6) calling a disease sequence generator (D class and F class) and a disease sequence fusion engine to perform artificial intelligence processing on the disease diagnosis results of traditional medicine and modern medicine of the patient, to automatically generate a traditional medicine disease sequence and a modern medicine disease sequence of the patient, and to automatically fuse the traditional medicine disease sequence and the modern medicine disease sequence to generate a contemporary AI fusion medical disease sequence; The disease sequence generation and fusion algorithm of traditional medicine and modern medicine is as follows: An organ weight knowledge graph is constructed, and an exhaustive method is used to list human organs / organ systems and disease names of traditional medicine and modern medicine (among them, the names of viscera of traditional medicine are converted into organ / organ system names expressed in modern medicine according to the mapping relationship between them and modern medicine organs, and correspondingly, the types / syndromes of traditional medicine are converted into diseases expressed in modern medicine according to the mapping relationship between them and modern medicine diseases), weight distribution is performed according to the importance of the human organs / organ systems to human health, to obtain a first weight distribution result; A disease staging weight rule of human organs / organ systems is established, and weight distribution is performed according to the staging of the human organs / organ systems, to obtain a second weight distribution result; The weight values in the first weight distribution result and the weight values in the second weight distribution result correspond to each other, the first weight distribution result is multiplied by the second weight distribution result, to obtain a first comprehensive weight distribution result of each disease in traditional medicine and a second comprehensive weight distribution result of each disease in modern medicine; Based on the weight value size in the first comprehensive weight distribution result and the weight value size in the second comprehensive weight distribution result, each AI-diagnosed disease is sorted in descending order according to the order from large to small, i.e. according to the order of the severity of the disease; Thus, the traditional medicine AI-diagnosed disease sequence model and the modern medicine AI-diagnosed disease sequence model are obtained respectively; After determining the diagnosis result of one or more diseases of the patient diagnosed by traditional medicine AI, the data of each disease and its disease stage in the traditional medicine AI diagnosis result of the patient are obtained, and based on the traditional medicine AI-diagnosed disease sequence model in the contemporary AI fusion medical system, the traditional medicine AI-diagnosed disease sequence of the patient is obtained; After determining the diagnosis result of one or more diseases of the patient diagnosed by modern medicine AI, the data of each organ / organ system disease and its disease stage in the modern medicine AI diagnosis result of the patient are obtained, and based on the modern medicine AI-diagnosed disease sequence model in the contemporary AI fusion medical system, the modern medicine AI-diagnosed disease sequence of the patient is obtained; After obtaining the traditional medicine AI-diagnosed disease sequence and the modern medicine AI-diagnosed disease sequence of the patient, the contemporary AI fusion medical disease sequence of the patient is constructed, and the construction method is as follows: The traditional medicine AI-diagnosed disease sequence and the modern medicine AI-diagnosed disease sequence of the same patient are merged, and the items that are the same and different in the two sequences are defined as the same disease items and different disease items of the respective sequences. The same disease items of the respective sequences in the two sequences are merged and included in the initial sequence of the contemporary AI fusion medical disease of the patient. The different disease items of the respective sequences in the two sequences are respectively included in the initial sequence of the contemporary AI fusion medical disease of the patient. The weight distribution of all disease items in the initial sequence of the contemporary AI fusion medical disease of the patient includes: The weight value of the first comprehensive weight of each same disease item in the traditional medicine AI-diagnosed disease sequence of the patient and the weight value of the second comprehensive weight of each same disease item in the modern medicine AI-diagnosed disease sequence of the patient are calculated by weighting (wherein the weight distribution between the weight value of the first comprehensive weight and the weight value of the second comprehensive weight is distributed according to a pre-set distribution rule, which can be optimized as appropriate and timely), and the weight value obtained by the weighting calculation is set as the weight value of each same disease item in the initial sequence of the contemporary AI fusion medical disease of the patient. The weight value of the first comprehensive weight of each different disease item in the traditional medicine AI-diagnosed disease sequence of the patient is set as the weight value of the corresponding disease item in the initial sequence of the contemporary AI fusion medical disease of the patient. The weight value of the second comprehensive weight of each different disease item in the modern medicine AI-diagnosed disease sequence of the patient is set as the weight value of the corresponding different disease item in the initial sequence of the contemporary AI fusion medical disease of the patient. According to the size of the weight value of each disease item in the initial sequence of the patient's contemporary AI fusion medical diseases, each AI diagnosed disease is arranged in descending order according to the severity of the disease from large to small; Thus, the patient's contemporary AI fusion medical disease sequence is obtained; (7) According to the required treatment disease and its treatment mode selected by the patient in the contemporary AI fusion medical disease sequence, the treatment scheme generator artificial intelligence processes the disease and its related medical data and / or medical feature data to automatically generate the contemporary AI fusion medical disease treatment scheme.

2. The Internet hospital diagnosis and treatment method of contemporary AI fusion medicine according to claim 1, characterized in that, Before starting the online consultation between the patient and the digital human doctor, the system calls the preset digital human doctor personalized recommendation model through the preset model calling module, and performs digital human doctor personalized recommendation based on the patient's preference. 3.The internet hospital diagnosis and treatment method of modern AI fusion medicine according to claim 1, characterized in that, After generating the contemporary AI fusion medical disease treatment scheme, according to the multi-dimensional demand of the patient for medical resources, the system calls the medical resource evaluation model (J class) through the preset model calling module to extract the required resources and obtain the candidate resources in the medical resources registered in the system client, and performs multi-dimensional scoring to generate the patient's contemporary AI fusion medical preferred disease treatment scheme. 4.The internet hospital diagnosis and treatment method of modern AI fusion medicine according to claim 3, characterized in that, After obtaining the contemporary AI fusion medical preferred disease treatment scheme, the system calls the treatment scheme collaborative optimization model through the preset model calling module to collaboratively optimize the preferred disease treatment scheme. 5.The internet hospital diagnosis and treatment method of modern AI fusion medicine according to claim 3, characterized in that, After generating the patient's contemporary AI fusion medical preferred disease treatment scheme, the system calls the medical insurance billing model (K class) through the preset model calling module to bill the preferred disease treatment scheme.

6. An internet hospital diagnosis and treatment system of the present age AI fusion medicine, characterized in that, The diagnosis and treatment system comprises a patient and digital human doctor interaction module, a medical data acquisition module, a data classification service (service A) module, a traditional medical data preprocessing service (service B1) module and a modern medical data preprocessing service (service B2) module, a traditional medical disease diagnosis service (service C) module, a traditional medical disease sequence generation service (service D) module, a modern medical disease diagnosis service (service E) module, a modern medical disease sequence generation service (service F) module, a disease sequence merging service (service merge) module, a patient medical selection service (service Treatment Selector) module, a treatment scheme generation service (service G / H / I) module, a medical resource pre-evaluation service (service J) module, a medical insurance billing service (service K) module, a payment service (service Payment) module and an AI scheduling engine module; The overall architecture of the system is an event-driven multi-microservice architecture, and the system sets a message middleware (such as Kafka or RabbitMQ), each service model calling link is a consumer service, each service listens to one or more topics, receives upstream data, processes and sends to the next topic in the message queue topic; Use API gateway to interact with patients and process patient requests, internal services can communicate through gRPC or REST; Use distributed tracking (such as Jaeger) to monitor the whole link. 7.The internet hospital diagnosis and treatment system of modern AI integrated medicine according to claim 6, characterized in that, The service model of the system includes (traditional medicine, for example, Chinese medicine, modern medicine, called Western medicine, integrated medicine, for example, traditional Chinese and Western medicine) : A type model: data classification model Input: raw mixed data uploaded by patients (including traditional Chinese medicine four diagnostic data, Western medicine test data, imaging, biochemistry, gene, physical examination, etc.) Output: classify the data into two categories of traditional Chinese medicine data and Western medicine data B type model: modal specific preprocessing Including two branches: traditional Chinese medicine data preprocessing and Western medicine data preprocessing Traditional Chinese medicine data preprocessing: including tongue diagnosis, face diagnosis, pulse diagnosis, etc. Data preprocessing (such as video enhancement, three-dimensional reconstruction, etc.) Western medicine data preprocessing: including image reading, biochemical index standardization, gene data annotation, etc. C type model: traditional Chinese medicine disease diagnosis prediction Input: preprocessed traditional Chinese medicine data Output: predict traditional Chinese medicine disease and stage D type model: traditional Chinese medicine disease sequence generation Input: the output of the C type model (disease and stage) and the preset traditional Chinese medicine disease sequence model with organ / organ system weight and stage weight label Output: traditional Chinese medicine disease sequence sorted by severity E type model: Western medicine disease diagnosis prediction Input: preprocessed Western medicine data Output: predict Western medicine disease and stage F type model: Western medicine disease sequence generation Input: the output of the E type model (disease and stage) and the preset Western medicine disease sequence model with organ / organ system weight and stage weight label Output: Western medicine disease sequence sorted by severity Merged model: merge traditional Chinese medicine disease sequence and Western medicine disease sequence into traditional Chinese and Western medicine combined disease sequence Rule: two-way diagnosis is mutually superimposed and complementary (such as disease combination of the same organ, comprehensive weight weighted calculation, etc.) G, H, I type model: treatment plan generation According to the patient's choice of one or more diseases needing treatment and the choice of treatment method, call the corresponding model to generate the treatment plan: G: Traditional Chinese and Western medicine treatment plan model H: Traditional Chinese medicine treatment plan model I: Western medicine treatment plan model Input: preprocessed data (traditional Chinese medicine and / or Western medicine) and selected disease Output: treatment plan (may include drugs, surgery, rehabilitation suggestions, etc.) J type model: medical resource pre-evaluation model Input: resources required in the treatment plan (such as drugs, hospitals, equipment, etc.) and patient quality demand and price demand Output: evaluation results (such as recommended hospitals, drugstores, etc.) K type model: medical insurance billing model Input: treatment plan and medical resources 8.The internet hospital diagnosis and treatment system of modern AI fusion medicine according to claim 6, characterized in that Output: cost details (including medical insurance payment and personal payment) , The message queue topic includes (traditional medicine, for example, Chinese medicine, modern medicine, called Western medicine, integrated medicine, for example, traditional Chinese and Western medicine) : · Raw data · Classified data output by A type model (divided into traditional Chinese medicine data and Western medicine data) · Preprocessed traditional Chinese medicine data by B type model · Preprocessed Western medicine data by B type model · Traditional Chinese medicine disease diagnosis output by C type model · Traditional Chinese medicine disease sequence output by D type model · Western medicine disease diagnosis output by E type model · Western medicine disease sequence output by F type model · Merged disease sequence · Patient's treatment request (including selected disease and treatment method) · Treatment plan generated by G / H / I type model · Resource evaluation output by J type model • Cost of the K-class model output. 9.The internet hospital diagnosis and treatment system of modern AI fusion medicine of claim 6, wherein, The microservices of the system include (traditional medicine is exemplified by traditional Chinese medicine, modern medicine is called Western medicine, and integrated medicine is exemplified by traditional Chinese and Western medicine integration): Service A: Data Classification Service Listen: Patient raw data Send: Classified data (contains two fields of traditional Chinese medicine data and Western medicine data) Service B1: Traditional Chinese Medicine Data Preprocessing Service Listen: Classified data (extract traditional Chinese medicine data) Send: Preprocessed traditional Chinese medicine data Service B2: Western Medicine Data Preprocessing Service Listen: Classified data (extract Western medicine data) Send: Preprocessed Western medicine data Service C: Traditional Chinese Medicine Disease Diagnosis Service Listen: Preprocessed traditional Chinese medicine data Send: Traditional Chinese medicine disease diagnosis Service D: Traditional Chinese Medicine Disease Sequence Generation Service Listen: Traditional Chinese medicine disease diagnosis Send: Traditional Chinese medicine disease sequence Service E: Western Medicine Disease Diagnosis Service Listen: Preprocessed Western medicine data Send: Western medicine disease diagnosis Service F: Western Medicine Disease Sequence Generation Service Listen: Western medicine disease diagnosis Send: Western medicine disease sequence Service Merge: Disease Sequence Merge Service Listen: Traditional Chinese medicine disease sequence and Western medicine disease sequence Send: Merged disease sequence Service: Patient Selection Service (interacts with patients through API gateway) Listen: Merged disease sequence (and waits for patient selection) Send: Patient-selected treatment request (contains patient-selected disease and treatment method) Service G / H / I: Treatment Plan Generation Service (different models are called according to the treatment method) Listen: Patient-selected treatment request If the treatment method is "traditional Chinese and Western medicine integration": call the G-class model If the treatment method is "traditional Chinese medicine": call the H-class model If the treatment method is "Western medicine": call the I-class model Send: Treatment plan Service J: Medical Resource Pre-evaluation Service Listen: Treatment plan Send: Resource evaluation Service K: Medical Insurance Billing Service Listen: Resource evaluation Send: Medical expenses Payment service: listens to medical expenses and waits for payment to be completed, then triggers logistics.

Images