Active inquiry method and system, electronic equipment and storage medium

Abstract

The invention provides an active inquiry method and system, electronic equipment and a storage medium, and relates to the technical field of medical inquiry, and the method can support multi-mode reply of a patient, provides a more comprehensive and flexible interaction mode for the patient, breaks through the limitation that a traditional inquiry system only depends on text interaction, and remarkably improves the inquiry efficiency. Through multiple rounds of interaction with the patient, the patient is actively guided to provide information, a personalized inquiry process can be realized, the inquiry time is greatly shortened, and the reception efficiency and the inquiry accuracy of the doctor are improved. Through the forms of charts, graphs, dynamic time axes and the like, in combination with an augmented reality technology, immersive disease condition display experience can be provided for patients and doctors of the doctor terminal, so that the patients can more intuitively know own disease conditions and inquiry progress, the sense of participation and trust of the patients to the medical process are enhanced, and the medical treatment efficiency is improved. And the doctor can understand the condition of the patient more quickly and accurately, so that the probability of misdiagnosis and missed diagnosis is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical consultation, and in particular to an active consultation method, system, electronic equipment and storage medium. Background Art

[0002] With the development of informatization in the medical field, medical question-answering systems are gradually being widely used. Through medical question-answering systems, patients’ condition description information is obtained to help doctors quickly diagnose, thereby reducing the doctor’s consultation time and improving consultation efficiency.

[0003] However, the existing medical Q&A system can only obtain the basic information and main complaint of the patient. It can neither provide doctors with accurate and reliable information about the patient's condition nor meet the consultation needs of complex conditions. Doctors still need to determine the patient's condition information by asking the patient, which will prolong the consultation time and reduce the doctor's consultation efficiency. Moreover, since the existing medical Q&A system cannot obtain in-depth information about the patient's condition, it also increases the risk of misdiagnosis and missed diagnosis by doctors.

[0004] Therefore, there is an urgent need to provide an active medical consultation method. Summary of the invention

[0005] The present invention provides an active medical consultation method, system, electronic device and storage medium to solve the defects existing in the related technology.

[0006] The present invention provides an active medical consultation method, which is applied to an active medical consultation system. The active medical consultation system stores a human body part map, including: Based on the patient terminal, initialization questions are presented to the patient; receiving an initial response from the patient to the initialization question; Based on the initial response, a medical knowledge graph and a deep learning model are applied to perform multiple rounds of interaction with the patient to determine the patient's condition description information; the initial response and the patient's response during the multiple rounds of interaction are all multimodal responses, and the multimodal response includes at least one of an image and a response based on extended reality technology, and is obtained by assisting the patient in responding based on the human body part map; Based on the condition description information, auxiliary diagnosis information is generated, and based on at least one of text, charts, graphics, dynamic timeline and extended reality technology, the condition description information and the auxiliary diagnosis information are visualized on the patient terminal and the doctor terminal respectively.

[0007] According to an active medical consultation method provided by the present invention, the image in the multimodal response is determined based on the following steps: Based on the patient terminal, displaying the human body part map to the patient; A first selected area of ​​the patient on the human body part map is received, and the first selected area is used as an image in the multimodal response.

[0008] According to an active medical consultation method provided by the present invention, the patient terminal supports an extended reality function, or the active medical consultation system is connected to an extended reality device; The extended reality technology-based response is determined based on the following steps: Acquire a human body image; the human body image is obtained based on the patient terminal or the extended reality device, or is pre-stored in the active consultation system; The human body part map is superimposed and displayed with the human body image, and a second selected area of ​​the patient on the superimposed display result is received as a response based on the extended reality technology.

[0009] According to an active consultation method provided by the present invention, based on the initial reply, a medical knowledge graph and a deep learning model are applied to perform multiple rounds of interaction with the patient to determine the patient's condition description information, including: Based on the deep learning model, selecting relevant knowledge of the initial response from the medical knowledge graph; Based on the relevant knowledge, the deep learning model is applied to generate guided questions, and the guided questions are presented to the patient, and the interaction continues until the condition description information is determined.

[0010] According to an active medical consultation method provided by the present invention, the condition description information includes static information and dynamic information; The display interface of the active medical consultation system is configured with a slider combination control; The slider combination control is used to display the change amount of the dynamic information.

[0011] An active medical consultation method provided by the present invention further includes: receiving first feedback information from the patient regarding the multiple rounds of interaction process, and / or second feedback information from the patient regarding a diagnosis result; the diagnosis result is determined by a doctor at the doctor terminal based on the condition description information and / or the auxiliary diagnosis information; Based on the first feedback information and / or the second feedback information, the multi-round interaction process is adjusted and the medical knowledge graph is optimized.

[0012] According to an active medical consultation method provided by the present invention, the step of presenting an initialization question to a patient comprises: presenting scenario information to the patient; receiving a target scene selected by the patient; The initialization question and the contents of the multiple rounds of interactions all correspond to the target scenario.

[0013] The present invention also provides an active medical consultation system, wherein the active medical consultation system stores a human body part map, including: A visualization module, used to display initialization questions to patients based on the patient terminal; A multimodal interaction module, configured to receive an initial response from the patient to the initialization question; A multi-round interaction module, for performing multi-round interactions with the patient based on the initial response, applying a medical knowledge graph and a deep learning model, and determining the patient's condition description information; the initial response and the patient's response during the multi-round interaction are both multimodal responses, and the multimodal response includes at least one of an image and a response based on extended reality technology, and is obtained by assisting the patient in responding based on the human body part map; A data analysis and diagnosis module, used to generate auxiliary diagnosis information based on the condition description information; The visualization module is also used to visualize the condition description information and the auxiliary diagnosis information on the patient terminal and the doctor terminal based on at least one of text, charts, graphics, dynamic timeline and extended reality technology.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the active medical consultation method as described above is implemented.

[0015] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the active medical consultation method as described above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the active medical consultation method as described above is implemented.

[0017] The active consultation method, system, electronic device and storage medium provided by the present invention can support the multimodal reply of the patient, provide a more comprehensive and flexible interaction mode for the patient, enable the patient to describe the condition more naturally and comprehensively, break through the limitation that the traditional consultation system only relies on text interaction, avoid the problem that the patient cannot accurately describe the condition due to lack of medical knowledge, and significantly improve the efficiency of the consultation. Through multiple rounds of interaction with the patient, the patient is actively guided to provide information, and a personalized consultation process can be realized, which reduces the time wasted by the doctor's inquiry and the patient's description, greatly shortens the consultation time, and improves the doctor's reception efficiency and the accuracy of the consultation. Moreover, the obtained condition description information can be made more comprehensive, accurate and reliable. Through charts, graphics, dynamic timelines and other forms, combined with extended reality technology, an immersive condition display experience can be provided for patients and doctors, so that patients can more intuitively understand their own condition and consultation progress, enhance the patient's sense of participation and trust in the medical process, make the consultation process more intuitive and efficient, and enable doctors to understand the patient's condition more quickly and accurately, and reduce the probability of misdiagnosis and missed diagnosis. Auxiliary diagnostic information can provide doctors with powerful auxiliary decision-making support, reduce their workload and improve the utilization efficiency of medical resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of the active medical consultation method provided by the present invention.

[0020] Figure 2 It is a structural schematic diagram of the active medical consultation system provided by the present invention.

[0021] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In the traditional medical consultation process, there are many problems in the information exchange between patients and doctors. On the one hand, patients may not be able to accurately and comprehensively describe key information such as symptoms and medical history to doctors due to nervousness, lack of medical knowledge, and limited ability to describe their own conditions. On the other hand, doctors may find it difficult to obtain sufficient and accurate patient information in a short period of time due to their busy work, which affects the accuracy and efficiency of diagnosis.

[0024] Although there are medical question-and-answer systems currently, they cannot meet the needs of complex medical consultations and cannot provide doctors with accurate and reliable patient condition information. This not only prolongs the doctor's consultation time and reduces the doctor's consultation efficiency, but also affects the doctor's accurate judgment of the patient's condition, increasing the risk of misdiagnosis and missed diagnosis. Therefore, an active consultation method is provided in an embodiment of the present invention.

[0025] Figure 1 FIG. 1 is a flow chart of an active consultation method provided in an embodiment of the present invention, such as Figure 1 As shown, the active medical inquiry method can be applied to an active medical inquiry system, wherein the active medical inquiry system stores a human body part map, including: S1, based on the patient terminal, presents the initialization questions to the patient; S2, receiving the patient’s initial responses to the initialization questions; S3, based on the initial response, applying the medical knowledge graph and the deep learning model, performing multiple rounds of interaction with the patient to determine the patient's condition description information; the initial response and the patient's response during the multiple rounds of interaction are all multimodal responses, and the multimodal response includes at least one of an image and a response based on extended reality technology, and is obtained by assisting the patient in responding based on the human body part map; S4, based on the condition description information, auxiliary diagnosis information is generated, and based on at least one of text, charts, graphics, dynamic timeline and extended reality technology, the condition description information and the auxiliary diagnosis information are respectively visualized on the patient terminal and the doctor terminal.

[0026] Specifically, the active medical consultation method provided in the embodiment of the present invention is executed by an active medical consultation system, and the active medical consultation system can be deployed on the information server of the hospital.

[0027] The active consultation system can be connected to the doctor workstations of various departments and the patient terminals through the hospital's internal network to ensure the stability of the active consultation system and the security of data transmission. Here, a hospital application (APP) can be installed on the patient terminal, and the patient accesses the active consultation system through the active consultation function in the hospital application.

[0028] The active consultation system may store a human body part map, which is a commonly used tool in medical education and clinical practice for displaying the anatomical structure and organ distribution of various parts of the human body. The human body part map may include the head, neck, chest, abdomen, pelvis, and limbs.

[0029] First, step S1 is performed to display initialization questions to the patient based on the patient terminal. The patient terminal can be a terminal device held by the patient, such as a smart phone, a tablet or a laptop. Before or during the consultation, the patient can open the active consultation function in the hospital application installed on the patient terminal to access the active consultation system.

[0030] The patient can enter basic information, such as age, gender, medical history, etc. After that, the active consultation system can display initialization questions to the patient through the patient terminal to start the consultation process. The initialization questions are pre-built questions in the active consultation system, such as "Where do you feel uncomfortable?", "Please describe your symptoms", etc.

[0031] Then, step S2 is executed, and the patient can see the initialization question through the patient terminal, and respond to the initialization question to obtain an initial response. Here, the active consultation system can support multiple interaction methods such as text, voice, image, and extended reality (Extended Reality, XR). The initial response can be a multimodal response, which can be the content of the patient's response to the initialization question through one or more interaction methods, and can include at least one of text, image, voice, and response based on extended reality technology. The text refers to the text form of the content uploaded by the patient for answering the initialization question, the image refers to the image form of the content uploaded by the patient and obtained with the assistance of the human body part map for answering the initialization question, the voice refers to the voice form of the content uploaded by the patient for answering the initialization question, and the response based on extended reality technology refers to the content uploaded by the patient through a device that supports the extended reality function and obtained with the assistance of the human body part map for answering the initialization question.

[0032] Here, extended reality technology may include virtual reality (VR) technology and augmented reality (AR) technology. The device supporting the extended reality function may be either a patient terminal or an extended reality device used in conjunction with an active consultation system. The extended reality device may include VR devices and AR devices. VR devices may include VR head-mounted devices, VR glasses, etc. AR devices may include AR head-mounted devices, AR glasses, AR mirrors, AR projection devices, etc.

[0033] The active consultation system can display the human body part map to patients through 3D models, dynamic images and other display methods. It can also superimpose the human body part map with the human body image through devices that support extended reality functions, so that patients can clearly know the location of each body part.

[0034] Patients can use the human body part map to answer initial questions, for example, by clicking on the part of the human body that needs to be examined in the map, the active inquiry system will intercept the position selected by the patient to obtain an image. Patients can also click on the part of the body that needs to be examined in the superimposed display results, so that the active inquiry system will intercept the position selected by the patient to obtain a response based on the extended display technology.

[0035] Then execute step S3, use the initial response, apply the medical knowledge graph and deep learning model, conduct multiple rounds of interaction with the patient, and determine the patient's condition description information.

[0036] Here, non-text in the multimodal response can be converted into text. If the multimodal response includes an image, the image can be analyzed to extract key information from the image, such as symptom information contained in the image. If the multimodal response includes speech, the speech can be recognized and converted into text.

[0037] After that, the text in the multimodal response and the converted text can be input into the deep learning model. The deep learning model can parse the input text and call the medical knowledge graph. Using the parsing results, the multimodal response can be matched and inferred with the knowledge in the medical knowledge graph, and the target knowledge related to the multimodal response in the medical knowledge graph can be extracted. This target knowledge can be used to dynamically generate more accurate guided questions.

[0038] For example, when the patient's multimodal response is "headache" through text input or voice input, the active consultation system automatically generates guiding questions such as "How long did the headache last?", "What is the severity of the headache (mild, moderate, severe)?", "Where is the headache located?"

[0039] For example, when the patient's multimodal response is input through voice, "I have been coughing and having a slight fever in the past few days," the proactive consultation system automatically generates guiding questions such as "Does the cough produce phlegm?" and "How high is the fever?"

[0040] Through the guiding questions, the system can interact with the patient in multiple rounds, and the patient's condition information can be continuously improved by the active consultation system during the multiple rounds of interaction. Based on the relevant information obtained during the multiple rounds of interaction, the patient's condition description information can be determined.

[0041] Here, the patient's responses during multiple rounds of interaction are all multimodal responses, which may include at least one of text, image, voice, image, and response based on extended reality technology. For the images in the multimodal response and the response based on extended reality technology, both can be obtained by showing the patient a human body part map to assist the patient in replying. For example, the patient can click on the human body part map or the superimposed display result of the human body part map and the human body image, and the click result can be used as the patient's reply.

[0042] Furthermore, the deep learning model can be used to select knowledge related to the patient's click results in the medical knowledge graph, and use the selected knowledge to apply the deep learning model to generate the next round of questions, which are presented to the patient for him to continue to respond to. This interaction continues until the patient's condition description information is mastered.

[0043] It is understandable that the medical knowledge graph can be stored in the active consultation system and can be obtained through the experience of medical experts. The medical knowledge graph is used to represent the relationship between diseases, symptoms, test results, treatment plans, etc. The medical knowledge graph can be updated regularly to ensure the accuracy and timeliness of medical information. The medical knowledge graph not only contains relevant knowledge about common diseases, but also covers relevant knowledge about special situations such as rare diseases, which can provide solid data support for the intelligent consultation of the active consultation system.

[0044] The deep learning model can be a large language model, or a model obtained by optimizing a pre-trained model through a private database, which is not specifically limited here.

[0045] The private database may be constructed by using conversation data obtained by simulating multiple rounds of conversations between a patient agent and a doctor agent.

[0046] The condition description information may include static information and dynamic information. The static information may include the patient's symptoms, medical history, etc., and the dynamic information may include the progress of the consultation, symptom change trends, etc.

[0047] After that, step S4 is executed. The active consultation system can use the deep learning model to conduct in-depth analysis of the condition description information, mine potential condition information, and search for corresponding diseases, test results, treatment plans and other diagnostic knowledge from the medical knowledge graph based on the condition description information and potential condition information. Auxiliary diagnosis information can be generated based on the diagnostic knowledge. The auxiliary diagnosis information may include a list of suspected diseases of the patient, the probability that the patient suffers from each predicted disease, the basis for diagnosis, and testing recommendations. Among them, the predicted disease refers to the disease that the patient suffers from in advance, which can be one or more diseases in the medical knowledge graph that have a high degree of match with the condition description information and potential condition information.

[0048] By using at least one of text, charts, graphics, dynamic timelines and extended reality technology, the disease description information and auxiliary diagnosis information can be visualized on the patient terminal and the doctor terminal respectively.

[0049] Here, the doctor terminal can be a terminal device held by the doctor, on which a doctor workbench is installed. The doctor can access the active consultation system by opening the workbench, and the active consultation system can automatically display the patient's condition description information and auxiliary diagnosis information. The doctor can quickly understand the patient's condition based on the patient's condition description information and auxiliary diagnosis information, and make further diagnosis and treatment plans.

[0050] Display in the form of text, charts, and graphics can be directly realized through the patient terminal and the doctor terminal. With text, static information and dynamic information can be directly described. With charts, graphics, dynamic timelines, and extended reality technology, static information and dynamic information can be displayed in a diversified manner. Charts can display structured data, including bar charts, line charts, etc. Graphics can display unstructured data, including information charts, schematic diagrams, etc. The dynamic timeline can present a timeline of symptom changes for patients, such as when the cough started, when the fever appeared, and changes in the severity of symptoms.

[0051] Extended reality technology can provide patients with an immersive disease display experience. The visualization achieved by extended reality technology can be achieved through devices that support extended reality functions. For the visualization of static information, the body parts involved in the static information can be displayed in the form of 3D models, and the symptoms can be marked with different colors, shapes, etc. For example, if the patient's symptom is redness and swelling of the arm, the arm can be displayed through a 3D model, and the arm area can be marked in the form of an enlarged red color. For the visualization of dynamic information, the symptom evolution trajectory can be drawn on the 3D model through a spatiotemporal symptom recorder.

[0052] For example, symptoms can be visualized through severity bar graphs and duration line graphs. Medical history can be visualized through events at each time point on the medical history timeline. Consultation progress can be visualized through progress bars, current stages, remaining steps, etc. Symptom change trends can be visualized through line graphs and bar graphs of changes for each symptom. Auxiliary diagnostic information can be visualized through timeline heat maps, disease probability radar maps, and examination recommendation association maps.

[0053] The active consultation method provided in the embodiment of the present invention is applied to the active consultation system. In this method, the multimodal response of the patient can be supported, and a more comprehensive and flexible interaction method is provided for the patient, so that the patient can describe the condition more naturally and comprehensively, breaking through the limitation of the traditional consultation system that only relies on text interaction, avoiding the problem that the patient cannot accurately describe the condition due to lack of medical knowledge, and significantly improving the efficiency of the consultation. Through multiple rounds of interaction with the patient, the patient is actively guided to provide information, and a personalized consultation process can be realized, which reduces the time wasted by the doctor's inquiry and the patient's description, greatly shortens the consultation time, and improves the doctor's reception efficiency and the accuracy of the consultation. Moreover, the obtained condition description information can be made more comprehensive, accurate and reliable. Through charts, graphics, dynamic timelines and other forms, combined with extended reality technology, an immersive condition display experience can be provided for patients and doctors, so that patients can more intuitively understand their condition and consultation progress, enhance the patient's sense of participation and trust in the medical process, make the consultation process more intuitive and efficient, and enable doctors to understand the patient's condition more quickly and accurately, reducing the probability of misdiagnosis and missed diagnosis. Auxiliary diagnostic information can provide doctors with powerful auxiliary decision-making support, reduce their workload and improve the utilization efficiency of medical resources.

[0054] On the basis of the above embodiment, the active medical consultation system stores a human body part map; the image in the multimodal reply is determined based on the following steps; Based on the patient terminal, displaying the human body part map to the patient; A first selected area of ​​the patient on the human body part map is received, and the first selected area is used as an image in the multimodal response.

[0055] Specifically, the patient can select a first selected area on the human body part map by clicking on the part, and the active medical consultation system receives the first selected area and captures the first selected area as an image in the multimodal response given by the patient.

[0056] In the embodiment of the present invention, by providing a human body part map to the patient, the patient can more quickly determine the location of the symptom, thereby avoiding errors in the condition description information due to inaccurate description by the patient.

[0057] On the basis of the above embodiment, the active medical consultation system stores a human body part map; the patient terminal supports an extended reality function, or the active medical consultation system is connected to an extended reality device, and the extended reality device stores the human body part map; The extended reality technology-based response is determined based on the following steps: Acquire a human body image; the human body image is obtained based on the patient terminal or the extended reality device, or is pre-stored in the active consultation system; The human body part map is superimposed and displayed with the human body image, and a second selected area of ​​the patient on the superimposed display result is received as a response based on the extended reality technology.

[0058] Specifically, when the patient and the active consultation system interact based on extended reality technology, if the patient terminal supports the extended reality function, the interaction can be carried out through the patient terminal. If the patient terminal does not support the extended reality function, and the active consultation system is connected to the supporting extended reality device, the interaction can be carried out through both the patient terminal and the extended reality device.

[0059] Furthermore, the active medical consultation system can obtain a human body image. The human body image can be one of a virtual human body model image, a physical human body model image, and a patient's body image. If the human body image is a virtual human body model image, the human body image can be an image of a virtual human body model pre-stored in the active medical consultation system. If the human body image is a physical human body model image, the human body image can be obtained by photographing the physical human body model through a camera on a patient terminal or an extended reality device. If the human body image is an image of a patient's body, the human body image can be obtained by photographing the patient's body through a camera on a patient terminal or an extended reality device.

[0060] After that, the human body part map can be superimposed on the human body image so that the patient can clearly know the human body part corresponding to each area in the human body image. Furthermore, the patient can select the part that needs to be consulted in the superimposed display result. The active consultation system receives the second selected area of ​​the patient on the superimposed display result and intercepts it as the patient's reply based on the extended reality technology.

[0061] For example, if the patient's gesture points to the head, the active consultation system can identify the direction of the head pointed by the patient's gesture through the camera of the patient's terminal or extended reality device, and then provide the patient with guided questions related to the head (i.e., neurological problems), such as headache-related questions ("How long has the headache lasted?", "How severe is the headache?"), etc.

[0062] In the embodiment of the present invention, an immersive interactive experience can be provided to the patient by interacting with the patient through extended reality technology.

[0063] On the basis of the above embodiment, based on the initial response, the medical knowledge graph and deep learning model are applied to perform multiple rounds of interaction with the patient to determine the patient's condition description information, including: Based on the deep learning model, selecting relevant knowledge of the initial response from the medical knowledge graph; Based on the relevant knowledge, the deep learning model is applied to generate guided questions, and the guided questions are presented to the patient, and the interaction continues until the condition description information is determined.

[0064] Specifically, during multiple rounds of interactions with patients, the active consultation system can first use the deep learning model to select relevant knowledge for the initial response from the medical knowledge graph. If the initial response is the first selected area, the deep learning model can be used to select knowledge related to the first selected area from the medical knowledge graph. For example, if the first selected area is the head, the relevant knowledge can be diseases corresponding to different headache durations and different headache degrees.

[0065] After that, the deep learning model can be applied using the relevant knowledge to generate guided questions, and the guided questions can be presented to the patient, and the interaction can continue until the description of the condition is determined. For example, the guided questions can be questions related to headaches ("How long did the headache last?", "How severe is the headache?"), etc.

[0066] The standard for determining the condition description information may be that one or more diseases can be clearly identified in the medical knowledge graph through the condition description information.

[0067] In an embodiment of the present invention, by combining a medical knowledge graph and a deep learning model, detailed and accurate description information of the disease can be quickly determined.

[0068] Based on the above embodiment, the condition description information includes static information and dynamic information; the display interface of the active consultation system is configured with a slider combination control; the slider combination control is used to display the change amount of the dynamic information.

[0069] Specifically, a slider combination control may be configured on the display interface of the active medical consultation system, and the slider combination control may interact with the sliding combination control to display the change amount of dynamic information.

[0070] For example, if the patient's input is "the pain increases from 3 points to 7 points", the sliding combination control can synchronously display the scale changes from 3 points to 7 points.

[0071] In the embodiment of the present invention, the slider combination control can enable patients and doctors to observe the changes in dynamic information more intuitively.

[0072] Based on the above embodiment, it also includes: receiving first feedback information from the patient regarding the multiple rounds of interaction process, and / or second feedback information from the patient regarding a diagnosis result; the diagnosis result is determined by a doctor at the doctor terminal based on the condition description information and / or the auxiliary diagnosis information; Based on the first feedback information and / or the second feedback information, the multi-round interaction process is adjusted and the medical knowledge graph is optimized.

[0073] Specifically, the patient can give first feedback information for multiple rounds of interaction during the active consultation process. The patient can also give second feedback information for the diagnosis result after completing the doctor's diagnosis and treatment. Here, the diagnosis result can be determined by the doctor at the doctor terminal based on the condition description information and the auxiliary diagnosis information. Both the first feedback information and the second feedback information can be the patient's evaluation information.

[0074] After receiving the first feedback information and / or the second feedback information from the patient, the active consultation system can adjust the multi-round interaction process and optimize the medical knowledge graph according to the first feedback information and / or the second feedback information, thereby improving the accuracy of subsequent active consultations and patient satisfaction, thereby achieving self-optimization and continuous improvement of the active consultation system.

[0075] Based on the above embodiment, the presenting of the initialization question to the patient includes: presenting scenario information to the patient; receiving a target scene selected by the patient; The initialization question and the contents of the multiple rounds of interactions all correspond to the target scenario.

[0076] Specifically, after the patient enters basic information and before displaying initialization questions to the patient, the active consultation system can display scenario information to the patient. The scenario information may be a department scenario that supports active consultation, such as an emergency scenario, a pediatric scenario, a chronic disease scenario, etc.

[0077] Patients can select the target scenario in which they need to be treated, and the subsequent initialization questions and the content of multiple rounds of interactions all correspond to the target scenario.

[0078] For example, if the target scenario is an emergency scenario, the patient touches the right abdomen in front of the AR mirror, and the active consultation system automatically highlights the liver and gallbladder area; the patient inputs "pain after meals" through voice, and the active consultation system can mark the gallbladder projection area on the 3D model. Furthermore, the active consultation system can generate disease probability radar maps and examination recommendation association maps in real time. The disease probability radar map can include cholecystitis probability 65% ​​(orange warning), gallstone probability 28% (yellow prompt), etc., and the examination recommendation association map can include ultrasound (★★★), blood routine (★★), etc.

[0079] For another example, if the target scenario is a pediatric scenario, the patient, that is, the child or parent, can access the active consultation system on the patient terminal and paint the fever area on the display interface of the active consultation system (the facial redness pattern can be displayed). The patient can also add a voice description "fever for 3 days, rash for 2 days". The active consultation system generates a timeline heat map and a disease probability radar map in real time. The timeline heat map can include the temporal relationship between fever and rash, and the disease probability radar map can include the probability of hand, foot and mouth disease (82%), the probability of chickenpox (15%), etc.

[0080] Based on the above embodiments, Figure 2 As shown, an active medical consultation system is also provided in an embodiment of the present invention. The active medical consultation system stores a human body part map, including: A visualization module 21, for displaying initialization problems to the patient based on the patient terminal; A multimodal interaction module 22, configured to receive an initial response from the patient to the initialization question; A multi-round interaction module 23 is used to perform multi-round interactions with the patient based on the initial response, using a medical knowledge graph and a deep learning model to determine the patient's condition description information; the initial response and the patient's response during the multi-round interaction are both multimodal responses, and the multimodal response includes at least one of an image and a response based on extended reality technology, and is obtained by assisting the patient in responding based on the human body part map; A data analysis and diagnosis module 24, used to generate auxiliary diagnosis information based on the condition description information; The visualization module 21 is also used to visualize the condition description information and the auxiliary diagnosis information on the patient terminal and the doctor terminal based on at least one of text, charts, graphics, dynamic timeline and extended reality technology.

[0081] On the basis of the above embodiment, the active medical consultation system provided in the embodiment of the present invention stores a human body part map; the multimodal interaction module is specifically used for: Based on the patient terminal, displaying the human body part map to the patient; A first selected area of ​​the patient on the human body part map is received, and the first selected area is used as an image in the multimodal response.

[0082] On the basis of the above embodiment, the active medical consultation system provided in the embodiment of the present invention stores a human body part map; the patient terminal supports an extended reality function, or the active medical consultation system is connected to an extended reality device; The multimodal interaction module is also specifically used for: Acquire a human body image; the human body image is obtained based on the patient terminal or the extended reality device, or is pre-stored in the active consultation system; The human body part map is superimposed and displayed with the human body image, and a second selected area of ​​the patient on the superimposed display result is received as a response based on the extended reality technology.

[0083] On the basis of the above-mentioned embodiment, in the active medical consultation system provided in the embodiment of the present invention, the multi-round interaction module is specifically used for: Based on the deep learning model, selecting relevant knowledge of the initial response from the medical knowledge graph; Based on the relevant knowledge, the deep learning model is applied to generate guided questions, and the guided questions are presented to the patient, and the interaction continues until the condition description information is determined.

[0084] Based on the above embodiment, in the active medical consultation system provided in the embodiment of the present invention, the condition description information includes static information and dynamic information; The display interface of the active medical consultation system is configured with a slider combination control; The slider combination control is used to display the change amount of the dynamic information.

[0085] On the basis of the above embodiment, the active medical consultation system provided in the embodiment of the present invention further includes a patient feedback module for: receiving first feedback information from the patient regarding the multiple rounds of interaction process, and / or second feedback information from the patient regarding a diagnosis result; the diagnosis result is determined by a doctor at the doctor terminal based on the condition description information and / or the auxiliary diagnosis information; Based on the first feedback information and / or the second feedback information, the multi-round interaction process is adjusted and the medical knowledge graph is optimized.

[0086] On the basis of the above-mentioned embodiment, in the active medical consultation system provided in the embodiment of the present invention, the visualization module is further used for: presenting scenario information to the patient; receiving a target scene selected by the patient; The initialization question and the contents of the multiple rounds of interactions all correspond to the target scenario.

[0087] Specifically, the functions of each module in the active medical consultation system provided in the embodiment of the present invention correspond one-to-one to the operating procedures of each step in the above-mentioned method embodiment, and the effects achieved are also consistent. Please refer to the above-mentioned embodiment for details, and no further details will be given in the embodiment of the present invention.

[0088] Figure 3An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the active consultation method provided in the above embodiments.

[0089] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0090] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the active consultation method provided in the above embodiments.

[0091] In another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the active consultation method provided in the above embodiments. The computer-readable storage medium can be either a non-transitory computer-readable storage medium or a transient computer-readable storage medium, which is not specifically limited here.

[0092] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An active medical consultation method, characterized in that: Applied to an active medical consultation system, the active medical consultation system stores a human body part map, including: Based on the patient terminal, initialization questions are presented to the patient; receiving an initial response from the patient to the initialization question; Based on the initial response, a medical knowledge graph and a deep learning model are applied to perform multiple rounds of interaction with the patient to determine the patient's condition description information; the initial response and the patient's response during the multiple rounds of interaction are all multimodal responses, and the multimodal response includes at least one of an image and a response based on extended reality technology, and is obtained by assisting the patient in responding based on the human body part map; Based on the condition description information, auxiliary diagnosis information is generated, and based on at least one of text, charts, graphics, dynamic timeline and extended reality technology, the condition description information and the auxiliary diagnosis information are visualized on the patient terminal and the doctor terminal respectively.

2. The active medical consultation method according to claim 1, characterized in that: The image in the multimodal reply is determined based on the following steps: Based on the patient terminal, displaying the human body part map to the patient; A first selected area of ​​the patient on the human body part map is received, and the first selected area is used as an image in the multimodal response.

3. The active medical consultation method according to claim 1, characterized in that: The patient terminal supports an extended reality function, or the active medical consultation system is connected to an extended reality device; The extended reality technology-based response is determined based on the following steps: Acquire a human body image; the human body image is obtained based on the patient terminal or the extended reality device, or is pre-stored in the active consultation system; The human body part map is superimposed and displayed with the human body image, and a second selected area of ​​the patient on the superimposed display result is received as a response based on the extended reality technology.

4. The active medical consultation method according to claim 1, characterized in that: Based on the initial response, applying the medical knowledge graph and the deep learning model, performing multiple rounds of interactions with the patient, and determining the patient's condition description information, including: Based on the deep learning model, selecting relevant knowledge of the initial response from the medical knowledge graph; Based on the relevant knowledge, the deep learning model is applied to generate guided questions, and the guided questions are presented to the patient, and the interaction continues until the condition description information is determined.

5. The active medical consultation method according to claim 1, characterized in that: The condition description information includes static information and dynamic information; The display interface of the active medical consultation system is configured with a slider combination control; The slider combination control is used to display the change amount of the dynamic information.

6. The active medical consultation method according to claim 1, characterized in that: Also includes: receiving first feedback information from the patient regarding the multiple rounds of interaction process, and / or second feedback information from the patient regarding the diagnosis result; The diagnosis result is determined by the doctor of the doctor terminal based on the condition description information and / or the auxiliary diagnosis information; Based on the first feedback information and / or the second feedback information, the multi-round interaction process is adjusted and the medical knowledge graph is optimized.

7. The active medical consultation method according to any one of claims 1 to 6, characterized in that: The initialization questions presented to the patient previously included: presenting scenario information to the patient; receiving a target scene selected by the patient; The initialization questions and the contents of multiple rounds of interactions all correspond to the target scenario.

8. An active medical consultation system, characterized in that: The active medical consultation system stores a human body part map, including: A visualization module, used to display initialization questions to patients based on the patient terminal; A multimodal interaction module, configured to receive an initial response from the patient to the initialization question; A multi-round interaction module, for performing multi-round interactions with the patient based on the initial response, applying a medical knowledge graph and a deep learning model, and determining the patient's condition description information; the initial response and the patient's response during the multi-round interaction are both multimodal responses, and the multimodal response includes at least one of an image and a response based on extended reality technology, and is obtained by assisting the patient in responding based on the human body part map; A data analysis and diagnosis module, used to generate auxiliary diagnosis information based on the condition description information; The visualization module is also used to visualize the condition description information and the auxiliary diagnosis information on the patient terminal and the doctor terminal based on at least one of text, charts, graphics, dynamic timeline and extended reality technology.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the active medical consultation method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the active consultation method according to any one of claims 1 to 7 is implemented.

Images