Digital patient inquiry interaction system based on 3D digital organ model

Abstract

According to the digital patient interrogation interaction system based on the 3D digital organ model, digital patients are introduced into an interrogation training system for clinical medical staff, and a standard medical interrogation question database, an interrogation answer database and a medical standard interrogation database are systematically established; the method comprises the following steps: establishing a plurality of interrogation question logics and feedback logics, performing structured processing according to symptoms / cases, interrogation key points and a logic sequence to form a plurality of interrogation question logics and feedback logics of different training dimensions, crosswise establishing a mesh relation graph between the logics, and performing evaluation and assessment on interrogation questions, the logic sequence and the like in the interrogation process according to a certain weight to achieve the purpose of interrogation practical training. The system can be used on terminal equipment such as a cloud server, a PC terminal, a mobile terminal and a robot, a display screen and AR / VR equipment, is beneficial to training in various modes, is not limited by time and regions, and is convenient to use.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional simulation technology and its application technology, and in particular to a digital patient consultation interactive system based on a 3D digital organ model. Background Art

[0002] Clinical physicians, medical staff and other medical personnel need to undergo professional skills internship and training after they officially enter their posts. Skill internship and training are important links in the transition from medical students to doctors and medical staff. Generally, they need to go through theoretical re-learning (such as doctor-patient communication, professional ethics, case prescription writing and reading, infection and aseptic technology, first aid, clinical professional knowledge), training in combination of theory and practice, teaching and learning with clinical patient examples, and training with simulated human bodies (such as rubber men) and expert patients (experts are simulated as patients). The practical training part is a key link for clinical medical personnel to learn and improve their own clinical experience, and it is also an indispensable key link for long-term accumulation of clinical consultation and diagnosis experience. Even after they officially enter the medical profession, they still face such a training process.

[0003] At present, the training of clinical medical personnel is generally carried out in hospitals and hospital training centers, through the use of simulated patients or diseased bodies, such as rubber men or formalin-soaked human bodies, or by practicing doctors in the hospital (usually experienced ones) acting as simulated patients to have medical consultation conversations with the trainees, training them in the diagnosis process of various diseases and lesions, and correcting their mistakes after the training, to help them accumulate clinical experience and improve their level.

[0004] However, at present, medical resources are still scarce, and the demand gap for medical staff is still relatively large. The training of clinical medical staff through traditional consultation and training methods is far from meeting the demand. Even if practicing doctors in hospitals are used as simulated patients, they usually rely on their own experience for the symptoms and problems involved in the consultation process, and it is difficult to provide trainees with systematic response and teaching for various symptoms. At the same time, when dealing with increasingly complex complications and newly emerging lesions, it is difficult to carry out effective training by relying solely on traditional consultation and training methods. Summary of the invention

[0005] The purpose of the present invention is to provide a digital patient consultation training system, method and equipment, which can realize the consultation training for medical personnel through simulated digital patients.

[0006] Other objects of the present invention are to combine VR technology, artificial intelligence technology and speech recognition technology to further enhance the practicality and scope of application of the medical consultation training system, so that it can be used in different forms and in different medical training processes, so as to help improve the systematicness and continuity of systematic medical staff medical consultation training.

[0007] To achieve the above object, the present invention provides a digital patient consultation training system, comprising:

[0008] A medical inquiry data receiving module is configured to receive medical inquiry data input by an external user;

[0009] A medical inquiry recognition module is configured to perform semantic recognition processing on the medical inquiry data and convert it into a standard medical inquiry question;

[0010] The consultation interaction processing module is configured to generate the digital patient's feedback answer content according to the standard medical consultation questions and the consultation feedback logic;

[0011] The consultation training evaluation module is configured to record and evaluate the consultation process for a certain disease during the training process and output the evaluation results.

[0012] In a further embodiment, the medical consultation training system further comprises:

[0013] The digital patient engine module is equipped with a 3D digital organ model of the human body and is configured to display and feedback the demeanor, facial expressions and / or body movements corresponding to the answers of different cases and the digital patient's consultation interactions.

[0014] In a further embodiment, the aforementioned medical inquiry data receiving module includes a voice module, which is configured to obtain the medical inquiry voice input by the user, perform text conversion, and obtain the medical inquiry data.

[0015] In a further embodiment, the aforementioned voice module is further configured to convert the feedback and answer content of the digital patient into voice data for feedback to the user for voice question and answer.

[0016] In a further embodiment, the medical consultation training system further comprises:

[0017] The medical question database is configured to construct a set of standard medical question sets oriented toward symptoms;

[0018] The medical inquiry recognition module is configured to perform keyword matching and recognition from the standard medical inquiry question set library based on the semantic recognition results to generate standard medical inquiry questions.

[0019] In a further embodiment, the medical inquiry recognition module includes an intelligent word segmentation module and a pattern recognition module. The intelligent word segmentation module is used to perform word segmentation processing on the medical inquiry data, and the pattern recognition module is used to perform keyword matching and recognition on the word segmentation results to generate standard medical inquiry questions.

[0020] In addition, the medical question database also constructs a set of synonymous medical questions.

[0021] In a further embodiment, the medical question database is constructed with a keyword library, a synonym library and a semantic recognition pattern library, and deep learning is used for recognition when the system is used.

[0022] In a further embodiment, the medical consultation training system further comprises:

[0023] The consultation feedback database is configured to have a digital patient consultation question feedback content set and feedback logic oriented to symptoms;

[0024] The consultation interaction processing module is configured to generate the digital patient's feedback answer content according to the digital patient's consultation question feedback content set and the consultation feedback logic.

[0025] In a further embodiment, the medical consultation feedback database is constructed with symptoms as the guide, and is structured according to symptoms, medical consultation points, and medical consultation logic to form a multi-dimensional medical consultation question feedback tree logic, and cross-establishes a network logic between each other.

[0026] In a further embodiment, the medical consultation training system further comprises:

[0027] The consultation and assessment database is set up to construct an assessment system with a tree-like logic based on the consultation questions and feedback;

[0028] The medical consultation training evaluation module is configured to compare the medical consultation questions of a medical consultation process of a certain disease with the symptoms, medical consultation points and medical consultation logic of the assessment system and make an evaluation.

[0029] In a further embodiment, the medical consultation assessment database is constructed with assessment weights for symptoms, medical consultation points, and medical consultation logic.

[0030] According to a second aspect of the present disclosure, a digital patient consultation training method is also provided, comprising the following steps:

[0031] During the consultation training process, receiving consultation data input by external users;

[0032] Perform semantic recognition processing on the medical consultation data and convert it into standard medical consultation questions;

[0033] Generate digital patient feedback based on standard medical consultation questions and consultation feedback logic;

[0034] Record the consultation process for a certain disease during the training process, make judgments, and output the evaluation results.

[0035] In a further embodiment, the medical consultation training method further comprises:

[0036] A digital patient engine module with a 3D digital organ model of the human body is configured. During the consultation training process, the demeanor, facial expressions and / or body movements corresponding to the answers of the digital patient are displayed and fed back according to different cases and the consultation interaction.

[0037] In a further embodiment, in the medical consultation training method:

[0038] Voice input is used to input questions for medical consultation, and the digital patient's feedback on the answer is provided through voice feedback.

[0039] In a further embodiment, in the medical consultation training method:

[0040] Perform semantic recognition processing on the medical inquiry data, including intelligent word segmentation processing on the medical inquiry data and keyword recognition processing on the word segmentation processing results, perform keyword matching recognition from a medical inquiry question database, and generate standard medical inquiry questions;

[0041] The medical question database is constructed with a set of standard medical question sets oriented towards symptoms.

[0042] In a further embodiment, the medical question database is also constructed with a set of synonymous medical questions.

[0043] In a further embodiment, the medical question database is constructed with a keyword library, a synonym library and a semantic recognition pattern library, and deep learning is used for recognition when the system is used.

[0044] In a further embodiment, in the medical consultation training method:

[0045] In the process of generating the digital patient's feedback answer content, the digital patient's feedback answer content is generated according to the consultation feedback database and the consultation feedback logic;

[0046] The consultation feedback database is constructed with a digital patient consultation question feedback content set and feedback logic oriented towards symptoms.

[0047] In a further embodiment, the medical consultation feedback database is constructed with symptoms as the guide, and is structured according to symptoms, medical consultation points, and medical consultation logic to form a multi-dimensional medical consultation question feedback tree logic, and cross-establishes a network logic between each other.

[0048] In a further embodiment, the evaluation of the medical consultation training process comprises the following steps:

[0049] The questions involved in the diagnosis process of a certain disease are judged based on the symptoms, key points of the diagnosis, and logic of the diagnosis in a preset assessment system.

[0050] In a further embodiment, the symptoms, key points of the medical consultation, and the medical consultation logic in the preset assessment system are set with different weights.

[0051] According to a third aspect of the disclosure of the present invention, a digital patient consultation training terminal is also provided, comprising:

[0052] Display screen;

[0053] speaker;

[0054] microphone;

[0055] Memory;

[0056] one or more processors;

[0057] One or more modules, which are stored in the memory and configured to be executed by the one or more processors, and the one or more modules include an instruction set for performing the following processing:

[0058] During the consultation training process, receiving consultation data input by external users;

[0059] Perform semantic recognition processing on the medical consultation data and convert it into standard medical consultation questions;

[0060] Generate digital patient feedback based on standard medical consultation questions and consultation feedback logic;

[0061] Record the consultation process for a certain disease during the training process, make judgments, and output the evaluation results.

[0062] In a further embodiment, the one or more modules include an instruction set for performing the following processing:

[0063] During the consultation training process, a digital patient engine module with a 3D digital organ model of the human body is used to display the demeanor, facial expressions and / or body movements corresponding to the digital patient's consultation interactions and feedback answers on the display screen.

[0064] In a further embodiment, the instruction set for performing semantic recognition processing on the medical inquiry data and converting it into a standard medical inquiry question further includes an instruction set for executing the following processing:

[0065] Intelligent word segmentation processing of medical consultation data;

[0066] The word segmentation results are processed for keyword recognition, and keyword matching and recognition are performed from a medical question database to generate standard medical question database; the medical question database is constructed with a set of standard medical question databases oriented to symptoms.

[0067] In a further embodiment, the instruction set for recording the consultation process for a certain disease during the training process and making judgments further includes an instruction set for executing the following processing:

[0068] The questions involved in the diagnosis process of a certain disease are judged based on the symptoms, key points of the diagnosis, and logic of the diagnosis in a preset assessment system.

[0069] According to a fourth aspect of the present disclosure, a cloud server is also provided, comprising:

[0070] A network interface is configured to perform data interaction with a communication base station through a communication link;

[0071] The cloud server is configured as follows:

[0072] In response to the performing of the consultation training, receiving input consultation data;

[0073] Perform semantic recognition processing on the medical consultation data and convert it into standard medical consultation questions;

[0074] Generate digital patient feedback based on standard medical consultation questions and consultation feedback logic;

[0075] Record the consultation process for a certain disease during the training process, make judgments, and output the evaluation results.

[0076] In a further embodiment, the cloud server is configured to:

[0077] During the consultation training process, a digital patient engine module with a 3D digital organ model of the human body is used to display the demeanor, facial expressions and / or body movements corresponding to the digital patient's consultation interactions and feedback answers on the display screen.

[0078] In a further embodiment, the cloud server is configured to:

[0079] Intelligent word segmentation processing of medical consultation data;

[0080] The word segmentation results are processed for keyword recognition, and keyword matching and recognition are performed from a medical question database to generate standard medical question database; the medical question database is constructed with a set of standard medical question databases oriented to symptoms.

[0081] In a further embodiment, the cloud server is configured to:

[0082] The questions involved in the diagnosis process of a certain disease are judged based on the symptoms, key points of the diagnosis, and logic of the diagnosis in a preset assessment system.

[0083] According to a fifth aspect of the present invention, a digital patient consultation training and assessment system is also provided, comprising:

[0084] at least one electronic device; and

[0085] The aforementioned cloud server that exchanges data with the electronic device via a communication network;

[0086] The electronic device has:

[0087] A display screen configured to execute text content and / or a 3D presentation of a digital patient;

[0088] An audio input device configured to collect user voice information;

[0089] An audio output device configured to play user voice information.

[0090] In a particularly preferred solution, the electronic device includes one of a handheld electronic device, a PC, a VR (virtual reality) device, an AR (augmented reality) device, and a holographic projection device.

[0091] It can be seen from the digital patient consultation training method and system of one or more schemes described above, as well as the cloud server, terminal and consultation assessment system for digital patient consultation training, that the significant beneficial effects of the present invention are:

[0092] Traditional online doctors and medical websites (as well as corresponding web applications and mobile apps) treat doctors as simulated intelligent doctors. For example, through online doctors, online hospitals, online diagnosis and treatment, etc., combined with new technologies of big data and cloud services, remote, real-time online pre-diagnosis, hospital appointments and diversion functions are realized. However, these methods all use medical data as doctors' empirical data to liberate the role of doctors. However, such systems or online doctors and medical websites cannot solve the clinical or practical consultation training for interns and medical staff. Before the professional ability of doctors is fully achieved, they cannot be systematically and targetedly trained and assessed.

[0093] The solution proposed in the present invention aims to solve the above-mentioned problems by introducing digital patients into a medical consultation training system for interns or newly employed doctors. The digital patient is a digitized 3D human body model used for medical education, training and assessment of doctors and students. By systematically establishing a standard medical consultation question database, a consultation answer database and a medical standard consultation database (assessment system), and structurally processing the medical standard consultation database according to symptoms, consultation points and logical order, a tree directory structure of multiple training dimensions is formed, and a network relationship diagram is cross-established between each other. The consultation questions, logical order, etc. in the consultation process are judged and assessed according to certain weights to achieve the purpose of practical training.

[0094] At the same time, in the medical consultation training system, both the medical questions and the answers can adopt structured and standardized semantic recognition solutions, use deep learning methods to identify and parse the medical questions, establish a keyword, synonym and semantic pattern library, and conduct further deep learning when the user uses the system for training, so as to autonomously improve the adaptability and semantic recognition rate.

[0095] In terms of system deployment and practical training methods, we also display digital patients in 3D, including 3D digital organ models of the human body, which can display corresponding demeanor, facial expressions and body movements according to different cases and voice dialogues, further enhancing the authenticity of the practical training and the on-site experience.

[0096] The training schemes described in the various embodiments of the present invention are system solutions for medical consultation training and assessment using digital patients. They can be deployed on terminal devices such as cloud servers, PCs, mobile terminals, and robots. They can also be viewed and used in a variety of ways, such as flat-panel displays, 3D displays, virtual reality, augmented reality, and holographic projections. They are conducive to conducting training in various modes, are not restricted by time and region, and are easy to use.

[0097] It should be understood that all combinations of the aforementioned concepts and the additional concepts described in more detail below can be considered as part of the inventive subject matter of the present disclosure as long as such concepts are not mutually inconsistent. In addition, all combinations of the claimed subject matter are considered as part of the inventive subject matter of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For clarity, not every component is labeled in every figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings, in which:

[0099] Figure 1 It is an exemplary system block diagram of a digital patient consultation training system.

[0100] Figure 2 It is an exemplary processing flow chart of the digital patient consultation training method.

[0101] Figure 3 It is a block diagram of the electronic medical record module of another example of the digital patient consultation training system.

[0102] Figure 4 It is an exemplary system block diagram of a digital patient consultation training terminal.

[0103] Figure 5 is an example diagram of a cloud server of an embodiment.

[0104] Figure 6 The diagram is an example of a medical consultation training and assessment system including a cloud server according to an embodiment.

[0105] Figure 7 is another exemplary diagram of an electronic device with which some embodiments of the present invention may be implemented. DETAILED DESCRIPTION

[0106] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0107] Various aspects of the present invention are described in this disclosure with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the present disclosure are not necessarily intended to include all aspects of the present invention. It should be understood that the various concepts and embodiments introduced above, as well as those described in more detail below, can be implemented in any of many ways, because the concepts and embodiments disclosed by the present invention are not limited to any implementation. In addition, some aspects disclosed by the present invention can be used alone or in any appropriate combination with other aspects disclosed by the present invention.

[0108] Some embodiments of the present invention provide a novel digital patient consultation training system. In some implementations, the system can be deployed on a handheld terminal (mobile terminal), a PC (e.g., a desktop PC terminal, a laptop PC terminal), a cloud server, a robot, or other terminals or devices, and the system can be interacted with through a display screen (especially a touch screen), a voice acquisition device, an audio playback device, a mouse, a keyboard, and other types of interactive devices, for clinical training personnel to use for practical training, such as watching the human posture, expression, and condition / symptom displayed by a virtual digital patient on the front end of the application through a display screen, inputting and expressing consultation questions to the system through the front end voice module, and then the digital patient gives feedback, and the above consultation and answer are repeated until the consultation process is completed.

[0109] During the trainees' questioning and the digital patient's answers, the system will also record the content of the questions and answers, evaluate and assess the key points of the questions, and achieve the purpose of training and assessment.

[0110] In some medical consultation training systems, an electronic medical record function can also be set, for example, through an external trigger button or an internally set automatic / active trigger function, the patient's electronic medical record will be saved according to the set rules. The electronic medical record function can also set an assessment option in some embodiments, and evaluate and assess the electronic medical record according to the set assessment criteria to achieve the purpose of practical training and assessment.

[0111] Figure 1 The process 100 of the digital patient consultation training method is exemplarily shown, which includes the following process: during the consultation training process, receiving consultation data input by external users; performing semantic recognition processing on the consultation data and converting them into standard medical consultation questions; generating the feedback answer content of the digital patient according to the standard medical consultation questions and consultation feedback logic; recording and judging the consultation process for a certain disease during the consultation training process, and outputting the evaluation results.

[0112] The process 100 starts with initially receiving a case or disease for consultation training selected by a user, at S101. In some embodiments, the selection can be made by receiving a user's voice command input from the outside, or by receiving a user's command input from an external input device such as a touch screen, a mouse, or a keyboard. In some embodiments, the aforementioned cases include disease cases such as pneumonia, bronchitis, coronary heart disease, and influenza, and the aforementioned diseases include fever, chest tightness, abdominal pain, and stomach bloating.

[0113] After a case or disease is selected, the process begins to enter training at S102.

[0114] After entering the training process, the process begins to receive the external input of the inquiry data input, at S103. In some embodiments, for example, the inquiry data collected by the external voice device is received. Of course, in other embodiments, the inquiry data collected and input by other types of interactive devices can also be used. In the following embodiments, we take voice inquiry data as an example for exemplary explanation. In some embodiments, the aforementioned inquiry data contains the initial inquiry information for a certain case or disease. For the sake of convenience, the inquiry training of the case will be used as an example in the following embodiments. Inquiry data, for example, in some examples, includes inquiry data with keywords such as one or more parts of the human body, human feelings, time points, time periods, etc., such as "Where do you feel uncomfortable?", "When did your stomach bloating start and how long did it last?", "What's the situation with your stomach?", etc., in order to expect the digital patient to give preliminary feedback on possible cases.

[0115] In some embodiments, S103 may also include text processing of the consultation data, such as converting the consultation voice data into text data through speech-text recognition and conversion recognition, so as to facilitate the subsequent semantic recognition, at S104. Of course, in other examples, we can also integrate it into the process S104 for implementation.

[0116] Then, the process performs semantic recognition processing on the received medical inquiry data, i.e., voice data, at S104, and converts it into a standard medical inquiry question. In some embodiments, the aforementioned semantic recognition processing includes intelligent word segmentation processing of the medical inquiry data and keyword recognition processing of the word segmentation processing result, performing keyword matching recognition from a medical inquiry question database, and generating a standard medical inquiry question. The medical inquiry question database is constructed with a set of standard medical inquiry questions oriented to symptoms.

[0117] In some embodiments, for example, the medical interview data is expressed in the form of standard or relatively standard medical interview questions, such as "When did your stomach bloating start and how long did it last?", "How long have you had a fever and what was the highest temperature?"; in other examples, the medical interview data is expressed in a non-standard or incompletely standard manner, such as "How many days have you had stomach pain?", "Where does your arm hurt?", etc. For medical interview questions expressed in a non-standard or incompletely standard manner, we use the constructed medical interview question database to perform guided conversion of standard medical interview questions.

[0118] In some embodiments, the medical question database stores a set of standard medical question sets oriented towards symptoms; these question sets are processed through sampling learning or learning, clustering and standardization of medical big data to form a tree-shaped, fishbone-shaped or other logic-oriented question set oriented towards symptoms.

[0119] In some exemplary implementations of process S104, intelligent word segmentation processing of medical consultation data includes intelligent word segmentation processing of converted text data. In some embodiments, intelligent word segmentation processing is performed by using a word segmentation algorithm such as a dictionary-based word segmentation algorithm or a statistical-based machine learning algorithm to extract key words, especially key words about body parts, symptoms, feelings, time points and durations.

[0120] The keyword recognition process includes performing keyword matching recognition from the standard medical question set library according to the semantic recognition result to generate a standard medical question. In some embodiments, the aforementioned matching recognition process can be performed according to the set rules, for example, using matching algorithms such as exact matching, fuzzy matching (broad matching), phrase matching, and multi-keyword matching to match the standard medical question, and the standard medical question of the matching result is used as the final conversion result. In some examples, we select the matching result that matches the standard medical question that is closest.

[0121] In some embodiments, the construction of the aforementioned medical question database also includes a keyword library and a semantic recognition pattern library. During the medical question training process, the recognition and conversion of medical questions are carried out using deep learning methods, and a vocabulary of synonyms, related words, modified words, and core words is established. When used, it is strengthened through deep learning methods to improve the adaptability and recognition rate of semantic recognition processing.

[0122] In some optional embodiments, the medical question database also includes a set of synonymous medical questions, which is established based on a set of standard medical questions and a vocabulary of synonyms, related words, variant words, and core words, so as to further improve the efficiency and adaptability of recognition and conversion.

[0123] The standard medical consultation questions obtained through the aforementioned conversion are expected to be answered by the digital patient to achieve interaction with the consultation training user.

[0124] Then, after the converted standard medical consultation questions are determined (at S104), the digital patient generates the digital patient's feedback answer content according to the standard medical consultation questions and the consultation feedback logic. In some embodiments, the digital patient is configured to provide feedback by querying and matching the feedback content data set in the consultation feedback database. In some embodiments, the consultation feedback database is constructed with the digital patient's consultation question feedback content set and feedback logic oriented to symptoms.

[0125] In some optional examples, the consultation feedback database is constructed with a symptom-oriented, structured processing according to symptoms, consultation points, and consultation logic sequence to form a multi-dimensional consultation question feedback tree logic, and cross-establishes a mesh logic between each other to form a tree logic for question feedback. For example, for cases of viral colds and influenza, a structured question tree logic and corresponding feedback logic are established based on consultation points such as fever, repeated high fever, runny nose, purulent secretions in the respiratory tract, red and swollen tonsils / pharyngeal pain, diarrhea, infection of the surrounding population, and clinical detection of white blood cell counts. For viral colds and influenza, some symptoms between the two are the same or similar, and others are significantly different, so a cross-multi-dimensional consultation question feedback tree logic is formed, and the digital patient generates feedback answer content through keyword matching and consultation feedback logic.

[0126] In some embodiments, the construction of the consultation feedback logic can be carried out through sampling learning of offline clinical patients' consultation process and problem feedback or through learning, clustering and standardization of medical big data to form a tree-like logic of symptom-oriented consultation questions and their feedback.

[0127] As an example, for example, in the consultation training for viral colds, based on the consultation question "How long have you had a fever, and what is the highest temperature?", a comprehensive match is performed from the consultation question feedback content set in the consultation feedback database to determine the content of the feedback. For example, the feedback content is "I have had a fever for 3 days, and the highest temperature is 40 degrees", or the feedback is "I have had a fever for 3 days, and my head hurts a lot", and the feedback answer content for the consultation question is determined by such expressions.

[0128] In some embodiments, the digital patient's feedback and answers are converted from digital to analog by the voice module and then played back to the consultation training user through an external speaker. The digital patient and its feedback and expressions will be described in more detail in the following description.

[0129] After receiving such feedback, the consultation training user will decide and continue the consultation in the aforementioned manner, and the digital patient will provide feedback in the aforementioned manner, and the consultation and feedback will be repeated until the consultation process is completed.

[0130] The completion process referred to above refers to the consultation training user actively completing the consultation training, or completing the consultation training process according to the set training time, or according to the satisfaction of the set consultation end trigger conditions.

[0131] During the execution of process 100, it is determined whether the consultation training is terminated in S106. In some embodiments, the determination of the termination of the consultation training depends on whether the aforementioned completion process is triggered, such as receiving information input from the consultation training user to actively complete the consultation training, or completing the consultation training process according to the set training time, or according to the satisfaction of the set consultation termination trigger condition.

[0132] If the consultation training is completed, in S107, the consultation training is evaluated. In some embodiments, after the consultation training process is completed, the consultation questions of a certain symptom / case in the consultation training process are evaluated, and the evaluation results are output. It is particularly preferred that the consultation questions of a certain symptom / case in the consultation process are compared with the symptoms, consultation points, and consultation logic of the evaluation system and evaluated.

[0133] In some implementations, the evaluation and assessment is performed based on a medical consultation and assessment database, wherein the medical consultation and assessment database is configured to construct an assessment system having a tree-like logic based on the feedback of medical consultation questions.

[0134] The construction of the aforementioned assessment system, according to the preset rules, establishes automatic judgment of the questions asked during the questioning process, the order of the questions asked, whether the questions are on point, and the differential diagnosis (for example, the diagnosis results of the case, symptom, and course of the disease). As an optional method, the construction of the assessment system can be formed through sampling and learning of offline clinical medical big data or processing by deep learning, clustering, and standardization to form a symptom-oriented assessment system for the questions asked, the order of the questions asked, whether the questions are on point, and the differential diagnosis.

[0135] In some embodiments, we also set different weights for assessment items such as the key points of case / symptom consultation, consultation sequence logic, etc. in the assessment system, and assess the consultation training process according to different weights.

[0136] At this point, the consultation training process for a case is completed.

[0137] In combination with the above process 100, in some embodiments, the digital patient can also be presented to the consultation training users in 3D during the consultation training. For example, a digital patient engine module is configured with a 3D digital organ model of the human body, which has a 3D digital organ model of the human body and can display corresponding demeanor, facial expressions and body movements according to different cases, symptoms and voice dialogues.

[0138] In some specific implementation schemes, when the digital patient's interactive feedback on medical questions and answers is obtained (at S104), the digital patient's demeanor, facial expressions and / or body movements corresponding to the medical answers are fed back to the medical user through display terminals / devices such as flat display screens, 3D display screens, touch screens, holographic projections, VR glasses, etc., so as to provide a medical consultation training simulation scene in a more realistic way, and interact with the nearly physical medical teaching human body model and trainees.

[0139] In the actual clinical consultation process, the record of medical history is equally important. In other embodiments of the present disclosure, in the consultation training process, an electronic medical record trigger can also be set. According to the manual trigger of the consultation training user, or according to the automatic save setting, the medical history of the consultation case / symptom is recorded and stored. For example, in the interactive process, in response to the digital patient's answers to the consultation questions, the user who is undergoing the consultation training can input the patient's medical history through manual text input, voice input, or a combination of the two, to form a medical history record. After the training is completed or ended, these medical history records will form a complete consultation medical record of the cases / symptoms involved in this consultation training process, that is, an electronic medical record.

[0140] In a further embodiment, after the consultation training is completed, the medical history storage of the electronic medical record is also assessed and evaluated. As an optional example, the medical history storage of the electronic medical record is matched and evaluated with the medical record record standard of the medical history assessment library, and the evaluation result is output.

[0141] The medical record recording standards of the medical history assessment database can be deeply learned and mined based on clinical medicine big data, and / or processed using machine learning, clustering and standardization to obtain medical history recording standards and keywords for cases / symptoms.

[0142] Combining the above implementation plans, Figure 2 The digital patient consultation training system 200 and the schematic diagram of its interaction with the consultation user in various aspects of the present invention are exemplarily shown. According to the disclosure of the present invention, the digital patient consultation training system 220 realizes the interaction between the questioning and answering feedback with the consultation training user through the user interaction input and output interface 210. In some embodiments, the user interaction input and output interface 210 includes but is not limited to the following methods: video input / output interface, plane / 3D display interface, holographic display interface, VR / AR display interface, voice input / output interface and keyboard and mouse interface.

[0143] During consultation training, users can obtain and observe 3D digital patients and their corresponding demeanor, facial expressions and body movements based on different cases, symptoms and voice dialogues through the display interface, obtain possible consultation prompts and help during consultation training, and obtain other items that need to be displayed or may be displayed.

[0144] The aforementioned display interfaces, especially the non-exhaustive flat / 3D display interfaces, holographic display interfaces, and VR / AR display interfaces, can be independent display devices, such as flat-panel display devices, 3D display devices, holographic projection devices, etc., or can be display modules that are part of electronic terminal devices, such as the display screen of a handheld smartphone, the display screen of a VR device, and the display screen of an AR device.

[0145] The consultation training user can express and input the consultation questions through the voice input / output interface or the keyboard and mouse interface, and interact with the consultation training system 200 to achieve the practical training purpose of the consultation training. As described above, combined with the display of the 3D digital patient, the more realistic simulation of the consultation scene effect plays a better role in the authenticity and on-site experience of the consultation training.

[0146] Figure 2 The exemplary illustrated medical consultation training system 220 comprises a medical consultation data receiving module 221 , a medical consultation identification module 222 , a medical consultation interaction processing module 223 , and a medical consultation training evaluation module 224 .

[0147] The inquiry data receiving module 221 is configured to receive the inquiry data input by an external user. The inquiry recognition module 222 is configured to perform semantic recognition processing on the inquiry data and convert it into standard medical inquiry questions. The inquiry interaction processing module 223 is configured to generate the feedback answer content of the digital patient according to the standard medical inquiry questions and the inquiry feedback logic. The inquiry training evaluation module 224 is configured to record and evaluate the inquiry process for a certain disease during the training process and output the evaluation results.

[0148] In some embodiments, the consultation training system further comprises:

[0149] The digital patient engine module 228 is configured with a 3D digital organ model of the human body, and is configured to display and feedback the demeanor, facial expressions and / or body movements corresponding to the answers of different cases and the digital patient's consultation interactions.

[0150] In some embodiments, the medical consultation data receiving module 221 includes a voice module, which is configured to obtain the medical consultation voice input by the user, convert it into text, and obtain medical consultation data, that is, medical consultation question data. These medical consultation question data in text format are processed into standard medical consultation questions.

[0151] In some embodiments, the aforementioned voice module may be further configured to convert the digital patient's feedback and answer content into voice data for feedback to the user for voice question and answer.

[0152] In some embodiments, the consultation training system further comprises:

[0153] The medical question database 225 is configured to construct a set of standard medical question sets oriented toward symptoms;

[0154] The medical inquiry recognition module 222 is configured to perform keyword matching and recognition from the standard medical inquiry question set library according to the semantic recognition result, and generate standard medical inquiry questions.

[0155] In some optional implementation schemes, the medical consultation identification module 222 includes an intelligent word segmentation module and a pattern recognition module. The intelligent word segmentation module is used to perform word segmentation processing on the medical consultation data, and the pattern recognition module is used to perform keyword matching and recognition on the word segmentation results to generate standard medical consultation questions. Preferably, the medical consultation question database is also constructed with a synonymous medical consultation question set. The medical consultation question database is constructed with a keyword library, a synonym library, and a semantic recognition pattern library, and is recognized by a deep learning method when the system is used.

[0156] In some embodiments, the consultation training system further comprises: a consultation feedback database 226, which is configured to have a digital patient consultation question feedback content set and feedback logic oriented toward symptoms;

[0157] The aforementioned consultation interaction processing module 223 is configured to generate the digital patient's feedback answer content according to the digital patient's consultation question feedback content set and the consultation feedback logic.

[0158] As an optional embodiment, the consultation feedback database 226 is constructed with a symptom-oriented structure that is structured according to symptoms, consultation points, and consultation logic to form a multi-dimensional consultation question feedback tree logic, and cross-establishes a network logic between each other.

[0159] In some embodiments, the medical consultation training system further comprises:

[0160] The diagnosis and assessment database 227 is configured to construct an assessment system based on a tree-like logic of diagnosis and assessment question feedback;

[0161] The consultation training evaluation module 224 is configured to compare the consultation questions of a certain disease with the symptoms, consultation points, and consultation logic of the assessment system and make an assessment.

[0162] Among some optional solutions, the construction of the medical consultation assessment library has different weights set for assessment items such as the key points of medical consultation for cases / symptoms, and the logic of medical consultation sequence, and the medical consultation training process is assessed according to different weights.

[0163] In some optional examples, the medical consultation training system further includes an electronic medical record module 230 to enhance the medical history recording and evaluation of the medical consultation system. Figure 3 The electronic medical record module has a medical record input module 231 and a medical history generation module 232. Through case input and medical history generation, the medical history record of the digital patient during the medical history training process is obtained, which can be assessed and evaluated together or separately after the medical history training is completed or ended.

[0164] In some solutions, the medical record input module is configured to receive voice and / or text input of the medical history record of the digital patient, for example, the interview training personnel input and record the symptoms and conditions answered by the digital patient through the keyboard, mouse, voice button (trigger, or through the microphone, etc.) voice input component, etc. The medical record input module receives these records and stores them. In a more advantageous solution, the medical record can be identified and stored according to the rules and regulations of clinical medical records.

[0165] The medical history generation module is used to generate a medical history of a digital patient based on the medical history of the digital patient after the medical history training is over or completed, so as to facilitate assessment and / or evaluation, enhance the systematic nature of medical history training and assessment, and simulate and practice the process of clinical medical history and record keeping.

[0166] In some embodiments, the medical consultation training system further includes a medical record evaluation module 240, which is configured to evaluate and assess the medical records.

[0167] The application modules (or programs, instruction sets) in the process features and systems described in the above embodiments can be implemented as a software process of a set of instructions recorded on a designated readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (such as one or more processors, processor cores or other processing units), these instructions enable one or more processing units to perform the indicated action process. Exemplary forms of computer-readable media include, but are not limited to, CD-ROMs, flash drives, random access memory (RAM), disk drives, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), etc. Computer-readable media do not include carrier waves and electrical signals transmitted wirelessly or connected by wire.

[0168] It should be understood that in the above and below descriptions of this specification, "software" is intended to include firmware residing in a read-only memory or an application stored in a magnetic memory, and "firmware" or "program" can be read into a memory to be processed by a processor. In other schemes, multiple software functions (inventions) can be implemented as sub-parts (subroutines) of a larger program while retaining different software invention functions (inventions). In other embodiments, multiple software functions (inventions) can also be implemented as independent programs. Finally, when these independent programs and subroutines are installed and run on one or more electronic systems, the software program defines the specific implementation of one or more specific machines that execute and implement the software program operations.

[0169] Applications of some embodiments may operate on mobile devices (also referred to as handheld electronic devices, such as mobile smartphones and tablet computers), which typically have independent input and output interfaces, processing units, storage units, data buses, and peripheral device interfaces. Figure 4 The mobile device 300 includes one or more processing units, a touch screen, a speaker, a microphone, a memory, and a peripheral interface.

[0170] The power supply system provides power supply for the entire mobile device. What is not shown is that the power supply system has a battery body, a charging circuit, an interface, and a discharging circuit.

[0171] The peripheral interface is coupled to various sensors and subsystems, including a camera subsystem, a wireless communication subsystem, an audio subsystem, an I / O subsystem, etc. The peripheral interface can realize communication between the processing unit and various peripheral devices, for example, a motion sensor (such as a gyroscope) or a positioning device (such as a GPS device) is coupled to the peripheral interface to facilitate the control and operation of the motion sensor and the positioning device, and to obtain working data (such as posture data, position data).

[0172] The camera subsystem is coupled to one or more optical sensors (eg, CCD, CMOS, etc.). The camera subsystem coupled to the optical sensors facilitates camera functions, such as acquisition of images and / or capture of video data.

[0173] The wireless communication subsystem is coupled to the peripheral interface, and the wireless communication subsystem is used to facilitate communication functions. In some embodiments, the wireless communication subsystem includes a radio frequency transmitter and receiver, and an optical receiver and transmitter. In some schemes, these transmitters and receivers are configured to work on one or more communication networks, such as GSM, CDMA, Wi-Fi, Bluetooth networks, etc.

[0174] The audio subsystem is coupled to a speaker to output speech (eg, outputting speech feedback content of a digital patient). Additionally, the audio subsystem is coupled to a microphone to support speech functions, such as speech collection and recognition, digital recording, and the like.

[0175] The I / O subsystem involves the transmission between the input / output peripheral devices (such as touch screens, displays) and the data bus of the processing unit through the peripheral interface. The I / O subsystem includes a touch screen controller and other input controllers to facilitate the transmission between the input / output peripheral devices and the data bus of the processing unit.

[0176] like Figure 4 As shown, the touch screen controller is coupled to the touch screen. The touch screen controller can use one of a variety of touch-sensitive control sensing technologies to detect contact and movement on the touch screen. Other input controllers are coupled to other output / control devices, such as one or more buttons, especially, for example, a start training button, an end training button, an electronic medical record button, a consultation question button, etc.

[0177] The memory controller is coupled to the memory. In some embodiments, the memory includes volatile memory, non-volatile memory, or a combination thereof, and / or any other form of memory. The memory stores an operating system (OS). The operating system includes instructions for processing basic system services and for performing hardware-related tasks. The memory also includes: communication instructions for facilitating communication of one or more additional devices; GUI instructions for facilitating graphical user interface processing; image processing instructions for facilitating image-related processing and functions; input processing instructions for facilitating input-related processing (such as touch); audio processing instructions for facilitating audio-related processing; camera instructions for facilitating camera-related processes and functions. The above instructions are merely exemplary, and in some embodiments, the memory may also include additional other instructions, if necessary, such as call processing instructions for facilitating calls.

[0178] The instructions identified above need not be implemented as separate software programs or modules. In some embodiments, such as in software and / or hardware, one or more signal processing or specialized integrated circuits are included to implement various functions of the mobile device.

[0179] Although Figure 4 The components shown in the figure are shown as independent components, but it should be understood that two or more of them can be integrated into one or more integrated circuits. In addition, two or more components can be coupled together by one or more communication buses. Of course, those skilled in the art can also Figure 4 The functions described in the embodiment are split into two or more integrated circuits.

[0180] Combination Figure 4The digital patient consultation training process 100 and the consultation training system 200 described in the above disclosure of the present invention can be deployed separately on an independent terminal of this type, making it an independently usable digital patient consultation training terminal. After the terminal is powered on and the processing unit runs the process 100 or the consultation training system 200, interaction is achieved through interactive interfaces such as display screens (especially touch screens), speakers, and microphones to realize consultation training.

[0181] In this embodiment, one or more modules (software modules, programs) are stored in the memory and configured to be executed by one or more processing units, and the one or more modules include instruction sets that perform the following processes:

[0182] During the consultation training process, receiving consultation data input by external users;

[0183] Perform semantic recognition processing on the medical consultation data and convert it into standard medical consultation questions;

[0184] Generate digital patient feedback based on standard medical consultation questions and consultation feedback logic;

[0185] Record the consultation process for a certain disease during the training process, make judgments, and output the evaluation results.

[0186] In some embodiments, one or more modules include a set of instructions to perform the following processes:

[0187] During the consultation training process, a digital patient engine module with a 3D digital organ model of the human body is used to display the demeanor, facial expressions and / or body movements corresponding to the digital patient's consultation interactions and feedback answers on the display screen.

[0188] In some embodiments, the instruction set for performing semantic recognition processing on the medical inquiry data and converting the information into standard medical inquiry questions further includes an instruction set for performing the following processing:

[0189] Intelligent word segmentation processing of medical consultation data;

[0190] The word segmentation results are processed for keyword recognition, and keyword matching and recognition are performed from a medical question database to generate standard medical question database; the medical question database is constructed with a set of standard medical question databases oriented to symptoms.

[0191] In some embodiments, the instruction set for recording and judging the consultation process for a certain disease during the training process further includes an instruction set for performing the following processing:

[0192] The questions involved in the diagnosis process of a certain disease are judged based on the symptoms, key points of the diagnosis, and logic of the diagnosis in a preset assessment system.

[0193] In other scenarios, the applications of some embodiments may operate on other electronic devices, such as personal computers, desktop computers, laptop computers, VR devices, AR devices, etc. Such electronic devices typically have input and output interfaces, processing units, storage units, data buses, and networks. Figure 5 4 is an exemplary architectural diagram of such an electronic device 400. The mobile device 400 includes one or more processing units, storage devices, GUI, system memory, network, etc. Such an electronic device includes a computer-readable medium and an interface for other types of computer-readable rings.

[0194] The electronic device 400 includes a bus, a system memory, a storage unit, a GUI unit, a network, a read-only memory, a permanent storage device, an input device, and an output device.

[0195] The bus refers to a bus that communicatively connects all systems, peripheral devices, and chipsets of multiple internal devices of an electronic device. For example, the bus communicatively connects a processing unit with a GPU, a system memory, and a read-only memory, a permanent storage device.

[0196] The processing unit retrieves instructions to be executed and data to be processed from these memory units in order to perform the process of the present invention. In some embodiments, some instructions are transmitted to and executed by the GPU. The GPU can offload various computing instructions or supplement the image processing provided by the processing unit.

[0197] Read-only memory stores static data and instructions needed by the processing unit. Permanent storage devices are read-write memory devices that still store instructions and data when the electronic device is turned off, and are non-volatile memories.

[0198] The system memory is a volatile memory that stores some of the data and instructions required by the processing unit during operation, such as a random access memory.

[0199] The bus also connects input devices and output devices. Input devices such as keyboard and mouse input devices, cameras, button switches, button control panels, microphones, or other devices for receiving voice commands. Output devices display images or other output data generated by electronic devices. Output devices may include printers, speakers, displays, etc.

[0200] The bus also couples electronic devices to a network through a network adapter.

[0201] From the above description, the digital patient consultation training process 100 and the consultation training system 200 described in the above disclosure of the present invention can be deployed separately on an independent terminal of this type, making it a digital patient consultation training terminal that can be used independently.

[0202] In other embodiments, the digital patient consultation training process 100 and the consultation training system 200 described above in the present invention can also be deployed on the cloud server 600, and each user terminal is connected to the cloud server 600 through a network to achieve distributed use.

[0203] like Figure 6 As shown, the electronic devices 500 of each user terminal, such as mobile terminals (such as smart phones, etc.), PC computers (such as laptops or desktop computers), VR glasses devices and other different electronic devices, are connected to the cloud server 600 through a network link to form a digital patient consultation training and assessment system. When the system is running, the electronic device 500 of the terminal acts as a user control terminal, which can collect information, display information, and control the start, progress and end of the consultation training. After the training starts, the electronic device of the terminal collects consultation questions and sends them to the cloud server 600, and receives the consultation feedback answer content and 3D display of the digital patient sent by the cloud server, plays the answer content through its speaker, and displays the corresponding demeanor, expression and body movements of the digital patient through the display screen.

[0204] It should be understood that the electronic device 500 here may be a device including but not limited to the aforementioned disclosed mobile device 00 and electronic device 400, as well as other types of electronic devices, which generally have a display device configured to execute text content and / or 3D display of a digital patient; an audio input device configured to collect user voice information; and an audio output device configured to play user voice information.

[0205] The cloud server 600 deployed receives the consultation control sent by the terminal's electronic device through the network, as well as the voice consultation data collected after the training starts, and operates in accordance with the aforementioned disclosed process 100 and training system 200, and sends the digital patient's consultation feedback answer content and 3D display information to the terminal's electronic device 500, thereby realizing the interaction and implementation of consultation training.

[0206] According to the above disclosure, such cloud server 600 is configured as follows:

[0207] In response to the performing of the consultation training, receiving input consultation data;

[0208] Perform semantic recognition processing on the medical consultation data and convert it into standard medical consultation questions;

[0209] Generate digital patient feedback based on standard medical consultation questions and consultation feedback logic;

[0210] Record the consultation process for a certain disease during the training process, make judgments, and output the evaluation results.

[0211] Figure 7 An exemplary architecture of a cloud server is shown. As shown, the cloud server 600 includes a processor and a storage medium connected to a processor bus, a network interface, and a memory, wherein the storage medium stores a database, an operating system (OS), and application components of the medical consultation training system 200, and the network interface is configured to perform data interaction with a communication base station through a communication link. The memory is a non-volatile memory, preferably a mass memory, for storing program instructions and data information used by the processor. The storage medium also stores a temporary storage area (not shown) required for system operation, which is used to provide part of the program instructions and data information used when the cloud server is running.

[0212] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A digital patient consultation interactive system based on a 3D digital organ model, characterized in that: include: The digital patient engine module is equipped with a 3D digital organ model of the human body, which is used to display the digital patient's demeanor, expression and / or body movements according to different cases and consultation interaction content; A medical inquiry data receiving module, used for receiving medical inquiry data input by an external user; The consultation interaction processing module is used to generate the digital patient's feedback answer content according to the consultation data, and display the corresponding demeanor, expression and / or body movement through the digital patient engine module; The display module is used to display 3D digital organ models and interactive consultation content through display screens, AR / VR devices or holographic projection devices.

2. The digital patient consultation interactive system based on 3D digital organ model according to claim 1 is characterized in that: The digital patient engine module also includes: A voice module, used for converting the digital patient's feedback answers into voice data and playing it through a speaker; The dynamic display module is used to dynamically adjust the demeanor, expression and / or body movements of the 3D digital organ model according to the interactive content of the consultation.

3. The digital patient consultation interactive system based on 3D digital organ model according to claim 1 is characterized in that: The medical inquiry data receiving module comprises: The voice input module is used to receive the voice consultation data input by the user and convert it into text data; The text input module is used to receive text consultation data directly input by the user.

4. The digital patient consultation interactive system based on 3D digital organ model according to claim 1 is characterized in that: The display module supports multiple display modes, including: A flat display screen for displaying a two-dimensional view of the 3D digital organ model; AR / VR devices to display immersive views of 3D digital organ models; Holographic projection equipment is used to display holographic images of 3D digital organ models.

5. The digital patient consultation interactive system based on 3D digital organ model according to claim 1 is characterized in that: The system further comprises: A consultation feedback database is used to store the digital patient consultation question feedback content set and feedback logic oriented to symptoms; The consultation interaction processing module generates the digital patient's feedback answer content based on the consultation feedback database and feedback logic.

6. The digital patient consultation interactive system based on 3D digital organ model according to claim 1, characterized in that: The system further comprises: A medical record input module, used to receive digital patient medical history records input by a user; The medical consultation record generation module is used to generate the medical consultation record of the digital patient based on the medical history record of the digital patient.

7. The digital patient consultation interactive system based on 3D digital organ model according to claim 1 is characterized in that: The system is deployed on a cloud server, PC, mobile terminal or robot terminal device.

8. The digital patient consultation interactive system based on 3D digital organ model according to claim 1 is characterized in that: The system optimizes the display effect and consultation interaction logic of the 3D digital organ model through deep learning.

9. The digital patient consultation interactive system based on 3D digital organ model according to claim 1, characterized in that: The system supports data synchronization and interaction of multiple terminal devices, including handheld electronic devices, PCs, virtual reality devices, augmented reality devices and holographic projection devices.

10. The digital patient consultation interactive system based on 3D digital organ model according to claim 1, characterized in that: The system is used for consultation training, medical education and skill assessment of clinical medical staff.

Images