Method, system, device and medium for displaying diagnosis and treatment information during virtual consultation

Abstract

The present invention discloses a method, system, device and medium for displaying diagnosis and treatment information during virtual medical consultation, wherein the method for displaying diagnosis and treatment information during virtual medical consultation comprises acquiring patient information collected according to a first virtual reality device worn by the patient, establishing a virtual patient corresponding to the patient based on the patient information, acquiring doctor action information collected according to a second virtual reality device worn by the doctor, and identifying the diagnosis and treatment information prescribed by the doctor during the virtual medical consultation according to the doctor action information, and displaying the diagnosis and treatment information in real time during the virtual patient's virtual medical consultation, thereby solving the problems of low efficiency and poor interaction effect of existing online medical consultations, making the consultation stage more transparent and humane, and greatly improving the patient's medical experience.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual medical consultation, and in particular to a method, system, device and medium for displaying medical treatment information during virtual medical consultation. Background Art

[0002] At the current stage, digital medicine is generally used in scenarios such as precise and intelligent appointments, electronic medical records, interconnection and mutual recognition, medical payment, online application for nucleic acid testing, and smart first aid, and is combined with facilities and data around the hospital to optimize processes, and realize more intelligent medical scenarios such as intelligent triage, parking appointments, intelligent planning of accompanying visits, intelligent priority visits, and home health follow-up. These applications have improved the efficiency of seeing a doctor, simplified the process of seeing a doctor, and allowed doctors to understand the medical history and current situation in advance, analyze whether to see a doctor, and thus reduce the number of times patients go in and out of the hospital. However, these methods still require going to the hospital, and there are still risks. Although there are online visits, doctors and patients still cannot communicate face to face, and the existing online visits cannot display the doctor's written medical advice information and drug information in real time, which affects the effective communication of the visit, and thus lacks the experience of immersive medical treatment. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defect in the prior art that diagnosis and treatment information cannot be displayed in real time during online consultation, and to provide a method, system, device and medium for displaying diagnosis and treatment information during virtual consultation.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] The present invention provides a method for displaying diagnosis and treatment information during a virtual consultation, the method comprising:

[0006] Acquire patient information collected by a first virtual reality device worn by the patient, and establish a virtual patient corresponding to the patient based on the patient information;

[0007] Acquiring doctor movement information collected by a second virtual reality device worn by the doctor, and identifying diagnosis and treatment information prescribed by the doctor during the virtual consultation process based on the doctor movement information;

[0008] The diagnosis and treatment information is displayed in real time during the virtual patient's virtual consultation process.

[0009] Preferably, the medical information includes consultation records and / or drug prices.

[0010] Preferably, the step of displaying the diagnosis and treatment information in real time includes:

[0011] Obtaining information about medications prescribed by the doctor during the virtual consultation;

[0012] The corresponding drug price is displayed based on the drug information; wherein the drug price includes the drug price in the hospital where the doctor is located and / or the reference price of the drug in other regions or other hospitals.

[0013] Preferably, after obtaining the drug information prescribed by the doctor during the virtual consultation process, the diagnosis and treatment information display method further includes:

[0014] Acquire similar drugs of the drug based on a similar drug recommendation model, wherein the drug price also includes drug prices of the similar drugs;

[0015] The similar drug recommendation model is trained by preset drug training data, and the preset drug training data includes multiple drugs and similar drugs for each drug.

[0016] Preferably, the diagnosis and treatment information display method further includes:

[0017] Obtaining medical insurance information of the patient;

[0018] A drug recommendation is made for the patient based on the medical insurance information and the drug price.

[0019] Preferably, when the medical information includes consultation records, the medical information display method further includes:

[0020] Receive feedback information from the patient on the consultation record, and modify the consultation record according to the feedback information.

[0021] Preferably, the patient information includes at least one of a real person image, micro-expressions, micro-movements and physical signs, and the step of establishing a virtual patient corresponding to the patient based on the patient information includes:

[0022] Modeling based on a real-life image of the patient to construct a basic virtual character;

[0023] At least one of the micro-expressions, micro-movements and physical signs of the patient is mapped to the basic virtual character to perform real-time rendering to obtain the corresponding virtual patient.

[0024] The present invention also provides a diagnosis and treatment information display system for virtual consultation, the diagnosis and treatment information display system comprising:

[0025] A virtual patient establishment module, used to obtain patient information collected by a virtual reality device, and establish a virtual patient corresponding to the patient based on the patient information;

[0026] A virtual doctor diagnosis and treatment module, used to collect the doctor's action information using the virtual reality device, and identify the diagnosis and treatment information prescribed by the doctor during the virtual consultation process according to the action information; wherein the action information includes body movements and / or verbal movements;

[0027] The diagnosis and treatment information display module is used to display the diagnosis and treatment information in real time during the virtual patient's virtual consultation process.

[0028] Preferably, the medical information includes consultation records and / or drug prices.

[0029] Preferably, the diagnosis and treatment information display module is also used to obtain the drug information prescribed by the doctor during the virtual consultation process;

[0030] The corresponding drug price is displayed based on the drug information; wherein the drug price includes the drug price in the hospital where the doctor is located and / or the reference price of the drug in other regions or other hospitals.

[0031] Preferably, the diagnosis and treatment information display module is also used to obtain similar drugs of the drug based on the similar drug recommendation model, and the drug price also includes the drug price of the similar drug;

[0032] The similar drug recommendation model is trained by preset drug training data, and the preset drug training data includes multiple drugs and similar drugs for each drug.

[0033] Preferably, the diagnosis and treatment information display system further includes a drug recommendation module;

[0034] The drug recommendation module is used to obtain the medical insurance information of the patient;

[0035] A drug recommendation is made for the patient based on the medical insurance information and the drug price.

[0036] Preferably, the diagnosis and treatment information display module is further used to receive feedback information from the patient on the consultation record, and modify the consultation record according to the feedback information.

[0037] Preferably, the patient information includes at least one of a real person image, micro-expressions, micro-movements and physical signs, and the virtual patient building module is further used to model based on the real person image of the patient to construct a basic virtual character;

[0038] At least one of the micro-expressions, micro-movements and physical signs of the patient is mapped to the basic virtual character to perform real-time rendering to obtain the corresponding virtual patient.

[0039] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for displaying diagnosis and treatment information for virtual consultation as described above is implemented.

[0040] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for displaying diagnosis and treatment information for virtual consultation as described above is implemented.

[0041] The positive and progressive effects of the present invention are: by acquiring patient information collected by a first virtual reality device worn by the patient, a virtual patient corresponding to the patient is established based on the patient information, the doctor's action information collected by a second virtual reality device worn by the doctor is acquired, and the diagnosis and treatment information prescribed by the doctor during the virtual consultation process is identified based on the doctor's action information, and the diagnosis and treatment information is displayed in real time during the virtual patient's virtual consultation process, thereby solving the problems of low efficiency and poor interaction effect of existing online consultations, making the consultation stage more transparent and humane, and greatly improving the patient's consultation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a first flow chart of the method for displaying diagnosis and treatment information during a virtual consultation according to Example 1 of the present invention.

[0043] Figure 2 This is a schematic diagram of a user wearing a virtual display device in the present invention.

[0044] Figure 3 This is a second flow chart of the method for displaying diagnosis and treatment information during a virtual consultation according to Example 1 of the present invention.

[0045] Figure 4 This is a third flow chart of the method for displaying diagnosis and treatment information during a virtual consultation according to Embodiment 1 of the present invention.

[0046] Figure 5 This is a fourth flow chart of the method for displaying diagnosis and treatment information during a virtual consultation according to Example 1 of the present invention.

[0047] Figure 6 This is a module diagram of a diagnosis and treatment information display system for virtual consultation according to Example 2 of the present invention.

[0048] Figure 7 This is a schematic diagram of the structure of an electronic device for implementing a method for displaying diagnosis and treatment information during a virtual consultation according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0049] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0050] Example 1

[0051] As digital intelligent technology continues to penetrate into all walks of life, digital medical technology has gradually been promoted. Then the concept of online medical treatment was proposed. Although there is online medical treatment, doctors and patients still cannot communicate face to face, and the existing online medical treatment cannot display the doctor's prescription information and drug information in real time, which affects the effective communication of the medical treatment, and thus lacks the experience of immersive medical treatment. Based on this, this embodiment provides a method embodiment of a method for displaying medical treatment information during virtual medical treatment. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0052] The diagnosis and treatment information display method provided in this embodiment can be executed in a mobile terminal, a computer terminal or a similar computing device.

[0053] Specifically, this embodiment provides a method for displaying diagnosis and treatment information during a virtual consultation, such as Figure 1 As shown, the diagnosis and treatment information display method of this embodiment includes:

[0054] S101. Acquire patient information collected by a first virtual reality device worn by a patient, and establish a virtual patient corresponding to the patient based on the patient information.

[0055] S102: Obtain doctor's motion information collected by the second virtual reality device worn by the doctor, and identify the diagnosis and treatment information prescribed by the doctor during the virtual consultation process based on the doctor's motion information.

[0056] It should be noted that the first virtual reality device and the second virtual reality device of the present embodiment may be devices that can display virtual scenes or real scenes or scenes combined with virtual reality, including but not limited to devices based on head-mounted display devices such as AR (Augmented Reality), VR (Virtual Reality), and MR (Mixed Reality) in combination with various sensor devices, positioning tracking devices, motion capture devices, and interactive devices. Figure 2A schematic diagram of a user wearing a virtual device is shown. In addition, the first virtual reality device and the second virtual reality device of this embodiment can also simulate various human perceptions, such as cold, heat, touch, smell, pain, etc. Furthermore, in addition to realizing the above functions, the virtual reality device can also be improved according to user needs. For example, the second virtual reality device worn by the doctor also has voice recognition and text recognition to recognize and convert the doctor's action information in reality, such as written text information or voice information, into diagnosis and treatment information during the virtual consultation process. Thereby, the virtual consultation of this embodiment can make the communication between patients and doctors more real and rich through multi-faceted interaction of voice, action, and body sensation.

[0057] Optionally, in this embodiment, a virtual hospital can be modeled according to the configuration of the physical hospital, and the virtual hospital has the same configuration as the physical hospital; of course, first of all, it is necessary to obtain authorization from the hospital, and the entity is the hospital modeling. The configuration of the physical hospital can be roughly based on the basic needs of the physical hospital, such as registration, diagnosis, payment, etc., as well as the setting of departments and diagnosis rooms, and corresponding guidance instructions, etc., which can be determined according to the needs. Of course, the above is not limited to the establishment of virtual hospitals, and it can also be a variety of places where medical interaction scenarios can be applied, such as medical outpatient departments, medical stations, pharmacies, etc. of various institutions. Furthermore, virtual patients can choose the corresponding virtual doctor for medical treatment in the virtual hospital according to their medical needs, and because it is a virtual medical treatment, it can avoid hospital congestion, long waiting time for patients to see the doctor, and serious crowding of patients, thereby improving the patient's medical experience.

[0058] S103. Displaying diagnosis and treatment information in real time during the virtual patient's virtual consultation process.

[0059] In an optional implementation, the diagnosis and treatment information of this embodiment includes but is not limited to consultation records and / or drug prices. It should be noted that the real-time display of diagnosis and treatment information during the virtual consultation process can be displayed according to the patient's needs or according to the doctor's instructions. For example, when the patient needs to know the consultation record, the consultation record will be displayed in real time during the virtual consultation process; when the patient needs to know the price of the drug, the drug price will be displayed in real time during the virtual consultation process; and when the doctor wants to communicate with the patient about the consultation record and drug price, the consultation record and drug price will be displayed in real time during the virtual consultation process. Therefore, this embodiment can start from the user and display the corresponding diagnosis and treatment information according to the user's needs and instructions, so as to improve the user experience.

[0060] The interactive process of the above-mentioned virtual medical consultation process includes the patient and the doctor entering the same virtual space through the first virtual reality device and the second virtual reality device respectively, the patient sees the doctor's virtual stand-in (virtual doctor) through the first virtual reality device, and the doctor sees the patient's virtual stand-in (virtual patient) through the second virtual reality device, and the patient and the doctor can communicate through body movements and voice.

[0061] This embodiment obtains patient information collected by a first virtual reality device worn by the patient, establishes a virtual patient corresponding to the patient based on the patient information, obtains doctor movement information collected by a second virtual reality device worn by the doctor, and identifies the diagnosis and treatment information prescribed by the doctor during the virtual consultation process based on the doctor movement information, and displays the diagnosis and treatment information in real time during the virtual patient's virtual consultation process, thereby changing the patient's consultation method in the prior art, improving the efficiency of consultation, simplifying the consultation process, avoiding the energy-consuming process of going to the hospital for consultation and wasting travel time and waiting time, and improving the user experience.

[0062] Furthermore, there are many third-party drug price inquiry websites on the Internet, which can query drug prices, but they lack real-time and promotion, and most people basically do not understand or use them. Because many hospital doctors use electronic medical records to record patient medical history and prescribe medicines immediately during consultations, their screens are always facing away from patients, resulting in opacity during consultations, causing patients to have to listen to doctors. Even if some hospitals are humane and patients can view the consultation display screen, they do not understand the overall trend of drug prices, whether the drugs are reimbursed, the course of drug treatment, and the medical expenses after reimbursement. As a result, they always have concerns about drug prices during consultations, which affects effective communication during consultations.

[0063] Based on the above content, as an optional implementation of this embodiment, Figure 3 As shown, step S103 of this embodiment includes:

[0064] S1031. Obtain information on medications prescribed by a doctor during a virtual consultation.

[0065] S1032: Display the corresponding drug price based on the drug information, wherein the drug price includes the drug price in the doctor's hospital and / or the reference price of the drug in other regions or other hospitals.

[0066] In an optional implementation manner, the diagnosis and treatment information display method of this embodiment further includes:

[0067] Similar drugs of a drug are obtained based on a similar drug recommendation model, and the drug price also includes the drug price of similar drugs.

[0068] Among them, the similar drug recommendation model is trained by preset drug training data, and the preset drug training data includes multiple drugs and similar drugs for each drug.

[0069] Through the above steps, when a doctor prescribes a drug, this embodiment can not only display the drug information of the drug in real time, but also display the drug price of the drug in different regions and different hospitals. In addition, the preset drug training data can be used to train a similar drug recommendation model to recommend similar drugs of the drug, and then the drug information of similar drugs and the drug price of similar drugs can be displayed, thereby realizing unified management of drugs, reducing doctors' drug memory ability, improving the transparency of medical prices, and reducing patients' fear of drug prices.

[0070] In an alternative embodiment, Figure 4 As shown, the diagnosis and treatment information display method of this embodiment also includes:

[0071] S301. Obtain the patient's medical insurance information.

[0072] S302. Recommend drugs to patients based on medical insurance information and drug prices.

[0073] This embodiment can recommend drugs to patients based on the reimbursement conditions of the patient's insurance information type and the drug price, so that patients can obtain the most suitable drugs, further reduce patients' fear of drug prices, and supervise doctors from the patient's perspective, avoiding the problem of indiscriminate prescription and reimbursement.

[0074] As another optional implementation of this embodiment, when the diagnosis and treatment information includes consultation records, the diagnosis and treatment information display method of this embodiment further includes:

[0075] Receive patient feedback on consultation records and modify consultation records based on the feedback.

[0076] Optionally, the feedback information received from the patient on the consultation record can be input by voice, text, sign language, etc. During the consultation, the patient can watch the doctor's consultation record in real time and make corrections at any time, and the doctor can modify the consultation record according to the feedback information.

[0077] In an optional implementation, the diagnosis and treatment information display method of this embodiment can also build intelligent diagnosis models and intelligent treatment models by learning disease diagnosis and treatment knowledge and clinical case data, and conduct diagnostic calculation experiments and treatment calculation experiments to obtain intelligent diagnosis results and intelligent treatment plans to help doctors improve the accuracy of virtual consultation results during the virtual consultation process.

[0078] As an optional implementation of this embodiment, the patient information includes but is not limited to at least one of a real person image, micro-expression, micro-motion and physical sign, such as Figure 5 As shown, step S101 includes:

[0079] S1011. Modeling is performed based on the patient's real-life image to construct a basic virtual character.

[0080] S1012: Map at least one of the patient's micro-expressions, micro-movements, and physical signs to a basic virtual character to perform real-time rendering to obtain a corresponding virtual patient.

[0081] In the process of establishing the virtual patient, the face and body 3D modeling is first performed based on the real person image, some key points of the body parts are selected to realize the image design, and the key points of the body are identified and mapped to the model for binding. The more key points there are, the more detailed the model is. The key point changes in the shape, expression, eyes, gestures, etc. are captured by using motion capture equipment or specific cameras and image recognition. According to the fine behavior, redirection is performed, and different driving methods are used for the actions, eyes, fingers, voice, etc., so as to finally realize the real-time virtual digital human behavior. Optionally, taking a VR device and a wearable device as an example, the VR device communicates with the wearable device, the wearable device simulates the corresponding environmental state of the human body according to the virtual environment parameters provided by the VR device, the VR device receives the human body parameters fed back by the wearable device and simulates the corresponding virtual human body state in the virtual environment, the wearable device includes a coat, the coat includes an environmental simulation unit and a human body parameter acquisition unit, the environmental simulation unit and the human body parameter acquisition unit both communicate with the VR device, the coat simulates the corresponding environmental state of the human body according to the virtual environment parameters provided by the VR device, the VR device receives the human body parameters fed back by the coat and simulates the corresponding virtual human body state in the virtual environment. Accordingly, the above process for establishing a virtual patient is also applicable to establishing a virtual doctor, which will not be elaborated here.

[0082] In an optional implementation, virtual technology can also be used to realistically display human features. This can be applicable to the modeling of later computer-driven virtual digital humans, and to the training of various driving models. Parameters such as voice and lip shape can be input into the model, and digital humans can be displayed through 3D rendering. This can also be applicable to patients with depression and other conditions to achieve mental health treatment.

[0083] Through virtual technology, a virtual person is created in real time based on at least one of real-life images, micro-expressions, micro-movements and physical signs. The virtual person can capture facial expressions and body movements at all times. During the virtual medical consultation, the doctor can judge the patient's psychological changes through micro-expressions and micro-movements, combined with the heart rate sensor on the virtual device, and provide real-time comfort and reassurance, thereby improving the spiritual communication between doctors and patients, allowing patients to trust doctors more and reduce their prejudice against doctors.

[0084] The method for displaying medical information during virtual medical consultations in this embodiment realizes the real-time display of drug prices and consultation records that patients are concerned about during virtual medical consultations through the above steps, solving the problems of low efficiency and poor interaction effects of existing online medical consultations, making the medical consultation stage more transparent and humane, and greatly improving the patient's medical experience.

[0085] Example 2

[0086] The present invention also provides a diagnosis and treatment information display system for virtual consultation, such as Figure 6 As shown, the diagnosis and treatment information display system of this embodiment includes:

[0087] The virtual patient establishment module 1 is used to obtain patient information collected by the virtual reality device and establish a virtual patient corresponding to the patient based on the patient information.

[0088] The virtual doctor diagnosis and treatment module 2 is used to collect the doctor's movement information using virtual reality equipment, and identify the diagnosis and treatment information prescribed by the doctor during the virtual consultation process based on the movement information; wherein the movement information includes body movements and / or verbal movements.

[0089] It should be noted that the first virtual reality device and the second virtual reality device of this embodiment can be a device that can display virtual scenes or real scenes or scenes combined with virtual reality, including but not limited to AR (Augmented Reality), VR (Virtual Reality), MR (Mixed Reality) and other head-mounted display devices, combined with various sensor devices, positioning tracking devices, motion capture devices, and interactive devices. In addition, the first virtual reality device and the second virtual reality device of this embodiment can also simulate various human perceptions, such as cold, hot, touch, smell, pain, etc. Further, in addition to realizing the above functions, the virtual reality device can also be improved according to user needs. For example, the second virtual reality device worn by the doctor also has voice recognition and text recognition to recognize and convert the doctor's action information in reality, such as written text information or voice information, into diagnosis and treatment information during the virtual consultation process. Thereby, the virtual consultation of this embodiment can be made to interact in all directions from sound, action, and body sensation, making the communication between patients and doctors more real and rich.

[0090] Optionally, in this embodiment, a virtual hospital can be modeled according to the configuration of the physical hospital, and the virtual hospital has the same configuration as the physical hospital; of course, first of all, it is necessary to obtain authorization from the hospital, and the entity is the hospital modeling. The configuration of the physical hospital can be roughly based on the basic needs of the physical hospital, such as registration, diagnosis, payment, etc., as well as the setting of departments and diagnosis rooms, and corresponding guidance instructions, etc., which can be determined according to the needs. Of course, the above is not limited to the establishment of virtual hospitals, and it can also be a variety of places where medical interaction scenarios can be applied, such as medical outpatient departments, medical stations, pharmacies, etc. of various institutions. Furthermore, virtual patients can choose the corresponding virtual doctor for medical treatment in the virtual hospital according to their medical needs, and because it is a virtual medical treatment, it can avoid hospital congestion, long waiting time for patients to see the doctor, and serious crowding of patients, thereby improving the patient's medical experience.

[0091] The diagnosis and treatment information display module 3 is used to display the diagnosis and treatment information in real time during the virtual patient's virtual consultation process.

[0092] In an optional implementation, the diagnosis and treatment information of this embodiment includes but is not limited to consultation records and / or drug prices. It should be noted that the real-time display of diagnosis and treatment information during the virtual consultation process can be displayed according to the patient's needs or according to the doctor's instructions. For example, when the patient needs to know the consultation record, the consultation record will be displayed in real time during the virtual consultation process; when the patient needs to know the price of the drug, the drug price will be displayed in real time during the virtual consultation process; and when the doctor wants to communicate with the patient about the consultation record and drug price, the consultation record and drug price will be displayed in real time during the virtual consultation process. Therefore, this embodiment can start from the user and display the corresponding diagnosis and treatment information according to the user's needs and instructions, so as to improve the user experience.

[0093] The interactive process of the above-mentioned virtual medical consultation process includes the patient and the doctor entering the same virtual space through the first virtual reality device and the second virtual reality device respectively, the patient sees the doctor's virtual stand-in (virtual doctor) through the first virtual reality device, and the doctor sees the patient's virtual stand-in (virtual patient) through the second virtual reality device, and the patient and the doctor can communicate through body movements and voice.

[0094] This embodiment obtains patient information collected by a first virtual reality device worn by the patient, establishes a virtual patient corresponding to the patient based on the patient information, obtains doctor movement information collected by a second virtual reality device worn by the doctor, and identifies the diagnosis and treatment information prescribed by the doctor during the virtual consultation process based on the doctor movement information, and displays the diagnosis and treatment information in real time during the virtual patient's virtual consultation process, thereby changing the patient's consultation method in the prior art, improving the efficiency of consultation, simplifying the consultation process, avoiding the energy-consuming process of going to the hospital for consultation and wasting travel and waiting time, and improving the user experience.

[0095] Furthermore, there are many third-party drug price inquiry websites on the Internet, which can query drug prices, but lack real-time and promotion, and most people basically do not understand or use them. Because many doctors use electronic medical records to record patient medical history and prescribe medicines immediately during consultations, their screens are always facing away from patients, resulting in opacity during consultations, causing patients to have to listen to doctors. Even if some hospitals are humane and patients can view the consultation display screen, they do not understand the overall trend of drug prices, whether the drugs are reimbursed, the course of drug treatment, and the medical expenses after reimbursement. As a result, they always have concerns about drug prices during consultations, which affects effective communication during consultations.

[0096] Based on the above content, as an optional implementation of this embodiment, the diagnosis and treatment information display module 3 of this embodiment is also used to obtain the drug information prescribed by the doctor during the virtual consultation process. The corresponding drug price is displayed based on the drug information; wherein the drug price includes the drug price of the drug in the doctor's hospital and / or the reference price of the drug in other regions or other hospitals.

[0097] In an optional implementation, the diagnosis and treatment information display module 3 of this embodiment is also used to obtain similar drugs of the drug based on the similar drug recommendation model, and the drug price also includes the drug price of the similar drugs.

[0098] Among them, the similar drug recommendation model is trained by preset drug training data, and the preset drug training data includes multiple drugs and similar drugs for each drug.

[0099] Among them, the similar drug recommendation model is trained by preset drug training data, and the preset drug training data includes multiple drugs and similar drugs for each drug.

[0100] Through the above process, when a doctor prescribes a drug, this embodiment can not only display the drug information of the drug in real time, but also display the drug price of the drug in different regions and different hospitals. In addition, the preset drug training data can be used to train a similar drug recommendation model to recommend similar drugs of the drug, and then the drug information of similar drugs and the drug price of similar drugs can be displayed, thereby realizing unified management of drugs, reducing doctors' drug memory ability, improving the transparency of medical prices, and reducing patients' fear of drug prices.

[0101] In an optional implementation, the diagnosis and treatment information display system of this embodiment also includes a drug recommendation module 4.

[0102] The drug recommendation module is used to obtain the patient's medical insurance information.

[0103] Make drug recommendations for patients based on medical insurance information and drug prices.

[0104] This embodiment can recommend drugs to patients based on the reimbursement conditions of the patient's insurance information type and the drug price, so that patients can obtain the most suitable drugs, further reduce patients' fear of drug prices, and supervise doctors from the patient's perspective, avoiding the problem of indiscriminate prescription and reimbursement.

[0105] As another optional implementation of this embodiment, the diagnosis and treatment information display module 3 of this embodiment is also used to receive feedback information from the patient on the consultation record, and modify the consultation record according to the feedback information.

[0106] Optionally, the feedback information received from the patient on the consultation record can be input by voice, text, sign language, etc. During the consultation, the patient can watch the doctor's consultation record in real time and make corrections at any time, and the doctor can modify the consultation record according to the feedback information.

[0107] In an optional implementation, the diagnosis and treatment information display method of this embodiment can also build intelligent diagnosis models and intelligent treatment models by learning disease diagnosis and treatment knowledge and clinical case data, and conduct diagnostic calculation experiments and treatment calculation experiments to obtain intelligent diagnosis results and intelligent treatment plans to help doctors improve the accuracy of virtual consultation results during the virtual consultation process.

[0108] As an optional implementation of this embodiment, the patient information includes at least one of real-life images, micro-expressions, micro-movements and physical signs, and the virtual patient establishment module 1 is also used to model based on the patient's real-life images to construct a basic virtual character.

[0109] At least one of the patient's micro-expressions, micro-movements, and physical signs is mapped to a basic virtual character to perform real-time rendering to obtain a corresponding virtual patient.

[0110] In the process of establishing the virtual patient, the face and body 3D modeling is first performed based on the real person image, some key points of the body parts are selected to realize the image design, and the key points of the body are identified and mapped to the model for binding. The more key points there are, the more detailed the model is. The key point changes in the shape, expression, eyes, gestures, etc. are captured by using motion capture equipment or specific cameras and image recognition. According to the fine behavior, redirection is performed, and different driving methods are used for the actions, eyes, fingers, voice, etc., so as to finally realize the real-time virtual digital human behavior. Optionally, taking a VR device and a wearable device as an example, the VR device communicates with the wearable device, the wearable device simulates the corresponding environmental state of the human body according to the virtual environment parameters provided by the VR device, the VR device receives the human body parameters fed back by the wearable device and simulates the corresponding virtual human body state in the virtual environment, the wearable device includes a coat, the coat includes an environmental simulation unit and a human body parameter acquisition unit, the environmental simulation unit and the human body parameter acquisition unit both communicate with the VR device, the coat simulates the corresponding environmental state of the human body according to the virtual environment parameters provided by the VR device, the VR device receives the human body parameters fed back by the coat and simulates the corresponding virtual human body state in the virtual environment. Accordingly, the above process for establishing a virtual patient is also applicable to establishing a virtual doctor, which will not be elaborated here.

[0111] In an optional implementation, virtual technology can also be used to realistically display human features. This can be applicable to the modeling of later computer-driven virtual digital humans, and to the training of various driving models. Parameters such as voice and lip shape can be input into the model, and digital humans can be displayed through 3D rendering. This can also be applicable to patients with depression and other conditions to achieve mental health treatment.

[0112] Through virtual technology, a virtual person is created in real time based on at least one of real-life images, micro-expressions, micro-movements and physical signs. The virtual person can capture facial expressions and body movements at all times. During the virtual medical consultation, the doctor can judge the patient's psychological changes through micro-expressions and micro-movements, combined with the heart rate sensor on the virtual device, and provide real-time comfort and reassurance, thereby improving the spiritual communication between doctors and patients, allowing patients to trust doctors more and reduce their prejudice against doctors.

[0113] The medical information display system for virtual medical consultations in this embodiment realizes the real-time display of drug prices and consultation records that patients are concerned about during the virtual medical consultation process, solving the problems of low efficiency and poor interaction effects in existing online medical consultations, making the medical consultation stage more transparent and humane, and greatly improving the patient's medical experience.

[0114] Example 3

[0115] Figure 7This is a schematic diagram of the structure of an electronic device provided in Example 3 of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the method for displaying diagnosis and treatment information for virtual consultation in Example 1 is implemented. Figure 7 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0116] like Figure 7 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0117] The bus 33 includes a data bus, an address bus, and a control bus.

[0118] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0119] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0120] The processor 31 executes various functional applications and data processing by running the computer programs stored in the memory 32, such as the method for displaying diagnosis and treatment information for virtual consultation in Example 1 of the present invention.

[0121] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model generating device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. Figure 7 As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via the bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0122] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0123] Example 4

[0124] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method for displaying diagnosis and treatment information during virtual consultation in Example 1 are implemented.

[0125] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0126] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps in the method for displaying diagnosis and treatment information for virtual consultation in Example 1.

[0127] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0128] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A method for displaying diagnosis and treatment information during virtual consultation, characterized in that: The diagnosis and treatment information display method comprises: Acquire patient information collected by a first virtual reality device worn by the patient, and establish a virtual patient corresponding to the patient based on the patient information; Acquiring doctor movement information collected by a second virtual reality device worn by the doctor, and identifying diagnosis and treatment information prescribed by the doctor during the virtual consultation process based on the doctor movement information; The diagnosis and treatment information is displayed in real time during the virtual patient's virtual consultation process.

2. The method for displaying diagnosis and treatment information during virtual consultation according to claim 1, characterized in that: The medical information includes consultation records and / or drug prices.

3. The method for displaying diagnosis and treatment information during virtual consultation according to claim 2, characterized in that: The step of displaying the diagnosis and treatment information in real time comprises: Obtaining information about medications prescribed by the doctor during the virtual consultation; The corresponding drug price is displayed based on the drug information; wherein the drug price includes the drug price in the hospital where the doctor is located and / or the reference price of the drug in other regions or other hospitals.

4. The method for displaying diagnosis and treatment information during virtual consultation according to claim 3, characterized in that: After obtaining the drug information prescribed by the doctor during the virtual consultation process, the diagnosis and treatment information display method further includes: Acquire similar drugs of the drug based on a similar drug recommendation model, wherein the drug price also includes drug prices of the similar drugs; The similar drug recommendation model is trained by preset drug training data, and the preset drug training data includes multiple drugs and similar drugs for each drug.

5. The method for displaying diagnosis and treatment information for virtual consultation according to claim 3 or 4, characterized in that: The diagnosis and treatment information display method further includes: Obtaining medical insurance information of the patient; A drug recommendation is made for the patient based on the medical insurance information and the drug price.

6. The method for displaying diagnosis and treatment information during virtual consultation according to claim 2, characterized in that: When the diagnosis and treatment information includes a consultation record, the diagnosis and treatment information display method further includes: Receive feedback information from the patient on the consultation record, and modify the consultation record according to the feedback information.

7. The method for displaying diagnosis and treatment information during virtual consultation according to claim 1, characterized in that: The patient information includes at least one of a real person image, micro-expressions, micro-movements and physical signs, and the step of establishing a virtual patient corresponding to the patient based on the patient information includes: Modeling based on a real-life image of the patient to construct a basic virtual character; At least one of the micro-expressions, micro-movements and physical signs of the patient is mapped to the basic virtual character to perform real-time rendering to obtain the corresponding virtual patient.

8. A diagnosis and treatment information display system for virtual consultation, characterized in that: The diagnosis and treatment information display system comprises: A virtual patient establishment module, used to obtain patient information collected by a virtual reality device, and establish a virtual patient corresponding to the patient based on the patient information; A virtual doctor diagnosis and treatment module, used to collect the doctor's action information using the virtual reality device, and identify the diagnosis and treatment information prescribed by the doctor during the virtual consultation process according to the action information; wherein the action information includes body movements and / or verbal movements; The diagnosis and treatment information display module is used to display the diagnosis and treatment information in real time during the virtual patient's virtual consultation process.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the method for displaying diagnosis and treatment information for virtual consultation as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for displaying diagnosis and treatment information for virtual consultation as described in any one of claims 1 to 7 is implemented.

Images