Medical consultation method, system, related equipment and computer program product

Abstract

The invention discloses a medical consultation method and system, related equipment and a computer program product, and aims to collect and manage multi-modal information of a patient and improve the consultation efficiency. Through more than one configured AI intelligent agent, real-time multi-round dialogues with consultation doctors can be carried out. By means of an artificial intelligence algorithm, man-machine multi-round dialogues are achieved, multi-modal information of a patient is understood, and targeted suggestions are given. The diagnosis accuracy is improved, and the working efficiency of doctors is also improved. In addition, the method can also record and manage the consultation process, for the consultation report filling process after the consultation is finished, an operation object can be supported to quote the opinions of each AI agent in each round of dialogue process in the consultation period, the doctor does not need to memorize in detail or repeatedly switch pages to check, and the consultation report writing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of digital medical health technology, and more specifically, to a medical consultation method, system, related equipment and computer program product. Background Art

[0002] In current medical practice, hospitals generally build information platforms to facilitate doctors to record patient information, medication information prescribed by doctors, etc. during offline consultations.

[0003] For some difficult diseases, multidisciplinary diseases, etc. (for example, hepatobiliary and pancreatic diseases), actual medical practice usually requires joint consultation by multiple doctors. In current medical practice, doctors usually conduct offline consultations, and the consultation process gives consultation results such as diagnosis results and treatment plans based on the experience of each doctor. The doctor representative will organize the consultation records in writing.

[0004] The current consultation method relies on professional doctor resources and has extremely strict professional requirements for doctors. In actual medical scenarios, doctors with rich experience are very scarce. Under the condition of tight professional doctor resources, it may not be possible to complete the consultation process well. In addition, the process of offline doctors sorting out consultation records also requires doctors to frequently review the recordings, videos and other meeting materials of the consultation process to improve the consultation records, which takes a long time. Summary of the invention

[0005] In view of the above problems, the present application provides a medical consultation method, system, related equipment and computer program product to solve at least part of the above problems. The specific solutions are as follows:

[0006] In a first aspect, a medical consultation system is provided, comprising:

[0007] Patient management module, used to collect and manage multimodal information of patients;

[0008] The AI ​​consultation module is used to conduct multiple rounds of real-time dialogues with the consulting doctor. During each round of dialogue, one or more configured AI agents understand the consulting doctor's speech based on the patient's multimodal information and generate response information through reasoning;

[0009] The consultation management module is used to record and manage the consultation process, wherein the consultation process includes the consultation report filling process after the consultation. The consultation management module manages the consultation report filling process, including:

[0010] In response to an instruction by the operation subject to quote a consultation opinion in a consultation report filling interface, a consultation opinion list is displayed, wherein the consultation opinion list includes the opinion of each AI agent in each round of dialogue process;

[0011] In response to the operation of the operating object selecting the target AI agent opinion in the consultation opinion list, the target AI agent opinion is referenced in the consultation report filling interface.

[0012] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the process in which the consultation management module manages the consultation report filling process further includes:

[0013] In response to an instruction by the operator to reference test information on a consultation report filling interface, a test information list is displayed, wherein the test information list includes various laboratory test results completed by the patient;

[0014] In response to the operation of the operating object selecting a target test result in the test information list, the target test result is referenced in the consultation report filling interface.

[0015] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the process in which the consultation management module manages the consultation report filling process further includes:

[0016] In response to an instruction of the operation subject to reference medical record information in the consultation report filling interface, display a medical record information list, wherein the medical record information list includes the patient's medical record information;

[0017] In response to the operation of the operating object selecting a target medical record segment in the medical record information list, the target medical record segment is referenced in the consultation report filling interface.

[0018] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the AI ​​consultation module is further used to:

[0019] After outputting the response information, in response to the operation of the operating object to view the details of the response information, a details page is displayed, and the details page includes: the logical reasoning process of each AI agent participating in the reasoning in the current round of dialogue process, and / or, the arguments and evidence given by each AI agent, wherein multiple AI agents can execute the reasoning process in parallel or serially.

[0020] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the consultation process further includes at least one of the following processes: consultation application, consultation doctor invitation, consultation review, consultation process and follow-up process after the consultation;

[0021] The consultation management module manages the consultation process, including:

[0022] The operations and data generated in each process of the consultation process are output and displayed in the form of a list.

[0023] In a possible design, in another implementation of the first aspect of the embodiment of the present application, the method further includes:

[0024] The consultation log recording module is used to record and count the speeches of the consulting doctors and the opinions of each AI agent during the consultation process.

[0025] In a second aspect, a medical consultation method is provided, comprising:

[0026] In response to the speech content input by the consulting doctor during the multi-round dialogue process, one or more configured AI agents understand the speech content based on the patient's multimodal medical information, generate response information by reasoning, and record the speech content of the consulting doctor and the opinions of each AI agent during the multi-round dialogue process;

[0027] In response to an instruction by the operation subject to quote a consultation opinion in a consultation report filling interface, a consultation opinion list is displayed, wherein the consultation opinion list includes the opinion of each AI agent in each round of dialogue process;

[0028] In response to the operation of the operating object selecting the target AI agent opinion in the consultation opinion list, the target AI agent opinion is referenced in the consultation report filling interface.

[0029] In a possible design, in another implementation of the second aspect of the embodiment of the present application, the method further includes:

[0030] In response to an instruction by the operator to reference test information on a consultation report filling interface, a test information list is displayed, wherein the test information list includes various laboratory test results completed by the patient;

[0031] In response to the operation of the operating object selecting a target test result in the test information list, the target test result is referenced in the consultation report filling interface.

[0032] In a possible design, in another implementation of the second aspect of the embodiment of the present application, the method further includes:

[0033] In response to an instruction of the operation subject to reference medical record information in the consultation report filling interface, display a medical record information list, wherein the medical record information list includes the patient's medical record information;

[0034] In response to the operation of the operating object selecting a target medical record segment in the medical record information list, the target medical record segment is referenced in the consultation report filling interface.

[0035] In a possible design, in another implementation of the second aspect of the embodiment of the present application, the method further includes:

[0036] After outputting the response information, in response to the operation of the operating object to view the details of the response information, a details page is displayed, and the details page includes: the logical reasoning process of each AI agent participating in the reasoning in the current round of dialogue process, and / or, the arguments and evidence given by each AI agent, wherein multiple AI agents can execute the reasoning process in parallel or serially.

[0037] In a third aspect, an electronic device is provided, comprising: a memory and a processor;

[0038] The memory is used to store programs;

[0039] The processor is used to execute the program to implement the medical consultation method described in any one of the second aspects of the present application.

[0040] In a fourth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the medical consultation method described in any one of the aforementioned second aspects of the present application is implemented.

[0041] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the medical consultation method described in any one of the aforementioned second aspects of the present application.

[0042] By means of the above technical scheme, the present application provides a medical consultation system integrating human and machine, which can collect and manage the multimodal information of patients through the patient management module, realize the integration of the multimodal information of patients, and improve the consultation efficiency. In addition, the consultation system of the present application is also provided with an AI consultation module, which is pre-configured with one or more AI agents, and can conduct real-time multi-round dialogues with the consulting doctor. In each round of dialogue, the AI ​​agent understands the speech content of the consulting doctor based on the multimodal information of the patient, and generates response information by reasoning. By introducing the AI ​​agent, it is possible to realize multi-round dialogues between humans and machines, understand the multimodal information of patients, and give targeted opinions, including but not limited to: treatment plan recommendations, disease diagnosis opinions, etc. By using AI agents to assist the doctor's consultation process, the accuracy of diagnosis is improved, and the work efficiency of doctors and the utilization efficiency of medical resources are improved. Furthermore, the consultation system is also provided with a consultation management module, which can record and manage the consultation process and improve the work efficiency of doctors. Among them, for the consultation report filling process after the consultation, the consultation management module can support the operation object to reference the opinions of each AI agent in each round of dialogue during the consultation, without the need for doctors to remember the details or repeatedly switch pages to view, thereby improving the efficiency of consultation report writing. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0044] Figure 1 A schematic diagram of a hardware architecture for implementing the medical consultation method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a terminal structure provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a server structure provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a process of a medical consultation method provided in an embodiment of the present application;

[0048] Figure 5 An example of a human-computer dialogue interface diagram for a consultation process is provided;

[0049] Figure 6 An example of a consultation report filling interface diagram is shown;

[0050] Figure 7 An example of an interface diagram for citing inspection information during the consultation report filling process is shown;

[0051] Figure 8 An example of an interface diagram for citing medical record information during consultation report filling process;

[0052] Fig. 9 A schematic diagram of a consultation process provided in an embodiment of the present application;

[0053] Fig.10 A schematic diagram of the structure of a medical consultation system provided in an embodiment of the present application;

[0054] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0057] This application provides a medical consultation method that can be applied to Figure 1 The hardware architecture shown in FIG. 1 may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 A server is included as an example for explanation).

[0058] The terminal 100 or the server 200 can be used alone to execute the medical consultation method provided in the embodiment of the present application. In addition, the terminal 100 and the server 200 can also be used in collaboration to execute the medical consultation method provided in the embodiment of the present application.

[0059] Next describe Figure 1 The product form of the mid-terminal 100;

[0060] The terminal 100 in the embodiment of the present application may be a mobile phone, a computer, a medical robot, etc., and the embodiment of the present application does not impose any limitation on this.

[0061] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0062] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160, a speaker 161, a microphone 162, an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0063] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0064] Among them, other input devices 132 can receive input data and so on.

[0065] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 can be used to display various interactive interfaces, processing results, etc. in the medical consultation method.

[0066] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0067] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0068] The memory 120 may be used to store software codes related to the medical consultation method, the processor 170 may execute the steps of the medical consultation method, and may also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to implement corresponding functions.

[0069] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0070] In the embodiment of the present application, the radio frequency unit 110 can send data to the server 200 and receive the processing result sent by the server 200. Exemplarily, the radio frequency unit 110 sends the received speech content of the consulting doctor to the server 200, and the server 200 calls the AI ​​agent to generate response information and returns the response information to the terminal 100 for output display.

[0071] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0072] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0073] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0074] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0075] Next describe Figure 1 The product form of the server 200;

[0076] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0077] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0078] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0079] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0080] The memory 204 may be used to store software codes related to the medical consultation method, and the processor 202 may execute the steps of the medical consultation method and may also schedule other units to implement corresponding functions.

[0081] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0082] The medical consultation method and system provided in this application can be applied to consultation scenarios for various diseases, especially consultation scenarios for difficult diseases, such as consultation scenarios for hepatobiliary and pancreatic diseases.

[0083] Traditional consultation methods face the following shortcomings:

[0084] 1. Tight doctor resources. When it comes to difficult diseases, the diagnosis is complex and cross-disciplinary, which places extremely high demands on doctors. In particular, doctors with rich experience are very scarce and unevenly distributed. This will also affect the accuracy of the final consultation results.

[0085] 2. Information is scattered and difficult to integrate. Consultation information is usually collected by the attending physician and then synchronized with other doctors during offline consultations. The information is scattered and incomplete.

[0086] 3. The consultation process is not standardized and cumbersome, which is not conducive to improving the doctor's work efficiency. Especially when sorting out the consultation records after the consultation, the doctor needs to rely on memory or frequently refer to the audio and video recordings of the consultation process to improve the consultation records, which is time-consuming and laborious.

[0087] To this end, the present application provides a medical consultation method and system to at least solve some of the above-mentioned problems.

[0088] The present application embodiment provides a medical consultation method, which is illustrated by applying the method to a computer device. The computer device may be Figure 1 The terminal 100 or the system consisting of the terminal 100 and the server 200. Figure 4 , the medical consultation method specifically includes the following steps:

[0089] Step S100, in response to the speech content input by the consulting doctor during the multi-round dialogue process, one or more configured AI agents understand the speech content based on the patient's multimodal medical information, infer and generate response information, and record the speech content of the consulting doctor and the opinions of each AI agent during the multi-round dialogue process.

[0090] This application can pre-train and configure one or more AI agents, and the AI ​​agents can be trained based on large model technology. Different AI agents have different capabilities for handling different tasks. The capabilities of AI agents include but are not limited to: doctor speech comprehension, agent self-organization, machine language generation, medical decision-making and diagnosis, treatment plan recommendation, etc.

[0091] The AI ​​agent of the present application may have the ability to process multimodal information in order to understand the patient's multimodal information.

[0092] Among them, the multimodal information of patients may include basic information of patients, medical history information (such as admission records, physical examinations, surgical information, etc.), laboratory test information, imaging examination information, etc. By collecting multimodal information of patients, the efficiency of consultation and the accuracy of consultation results can be improved.

[0093] The medical consultation method of the present application supports multi-round dialogue capabilities between the consulting doctor and the machine, wherein the machine can call on the capabilities of the AI ​​agent, understand the content of the doctor's speech, and infer and generate response information.

[0094] During multiple rounds of dialogue, one or more AI agents can be called to perform logical reasoning based on the doctor's speech in the current round, and multiple AI agents can perform logical reasoning in parallel or serially to obtain the final response information.

[0095] In this step, the content of the consulting doctors' speeches during multiple rounds of dialogue and the opinions of each AI agent can be recorded for subsequent query applications.

[0096] Reference Figure 5 , which illustrates a scenario of human-computer dialogue. The consulting doctor can be any one or more doctors participating in the consultation. They can input the speech content in various forms such as text and voice, and the machine will give the response information, corresponding to Figure 5 At the same time, the application method also supports multiple modes of output of response information, such as text mode output, voice broadcast output, etc.

[0097] Step S110, in response to the instruction of the operation object to quote the consultation opinion in the consultation report filling interface, display the consultation opinion list, wherein the consultation opinion list includes the opinion of each AI agent in each round of dialogue process.

[0098] Specifically, after the consultation, a consultation report is generally required. The consultation report may include patient information, consultation host, consultation results, etc. Among them, the consultation results are relatively important. Traditionally, doctors organize the consultation results based on their own memories or by consulting audio and video recordings of the consultation process. In order to improve the efficiency of consultation report preparation, this application can provide a consultation opinion reference interface in the consultation report filling interface, refer to Figure 6 As shown, the interface may provide a "reference consultation opinion" control, which can be triggered to initiate an instruction to reference consultation opinions, and the system responds to the instruction and displays a list of consultation opinions, such as Figure 6 Shown on the right.

[0099] In this step, the operation object may be the person who prepares the consultation report, who may be a doctor participating in the consultation, or other staff.

[0100] The consultation opinion list may include the opinions of each AI agent in each round of dialogue during the multi-round human-computer dialogue. In addition, it may also include the understanding results of the doctor's input speech content.

[0101] By outputting and displaying the consultation opinion list, an operator can select a target content segment to be referenced.

[0102] Step S120, in response to the operation of the operating object selecting the target AI agent opinion in the consultation opinion list, the target AI agent opinion is referenced in the consultation report filling interface.

[0103] Specifically, when the operator selects the target AI agent opinion in the consultation opinion list, the target AI agent opinion can be referenced to the consultation report filling interface, such as introducing it into the consultation result column of the consultation report filling interface, thereby eliminating the operator's manual search and writing process and improving work efficiency. At the same time, by citing the AI ​​agent opinion in the consultation report, strong supporting evidence for the conclusion can be provided in the consultation report, improving the information richness and quality of the consultation report.

[0104] The present application provides a medical consultation method of human-machine fusion, which can conduct multiple rounds of real-time dialogue with the consulting doctor through one or more pre-configured AI agents. In each round of dialogue, the AI ​​agent understands the content of the consulting doctor's speech based on the patient's multimodal information, and generates response information by reasoning. By introducing AI agents, artificial intelligence algorithms can be used to achieve multiple rounds of human-machine dialogue, understand the patient's multimodal information, and give targeted opinions, including but not limited to: treatment plan recommendations, disease diagnosis opinions, etc. By using AI agents to assist the doctor's consultation process, the accuracy of diagnosis is improved, and the doctor's work efficiency and the utilization efficiency of medical resources are improved. Furthermore, for the consultation report filling process after the consultation, the operation object is supported to quote the opinions of each AI agent in each round of dialogue during the consultation, without the need for doctors to remember in detail or repeatedly switch pages to view, thereby improving the efficiency of consultation report writing.

[0105] By using the consultation method of this application, doctors can actively ask questions to AI robots, which will respond based on the multimodal information of patients through the fusion of multiple AI agents, thereby inspiring doctors' diagnosis and treatment inspiration. Multiple rounds of debate-style dialogue interaction gradually fill in the information gaps of diagnosis and treatment plans, and jointly draw up the most suitable treatment plan for patients.

[0106] In some possible implementations, the consultation method of the present application may further include recording the speeches of the consulting doctors and the opinions of each AI agent during the consultation process in the form of a consultation log to facilitate subsequent query and call.

[0107] In some possible implementations, considering that the consultation report may also need to include the patient's test information, in order to facilitate the reference of the test information, such as Figure 6 As shown, the consultation report filling interface can further be provided with a "reference test" control, and the operator can trigger the control to initiate an instruction to reference test information. The present application system responds to the instruction and further displays a test information list, which includes the results of various laboratory tests completed by the patient.

[0108] like Figure 7 , illustrates an interface diagram for referencing test information, which may include several test results that the patient has completed.

[0109] The operator can select the target test result in the test information list, and then refer to the target test result in the consultation report filling interface, which saves the operator from the process of looking up relevant materials and improves work efficiency.

[0110] In some possible implementations, considering that the consultation report may also need to include the patient's medical record information, in order to facilitate the reference of medical record information, such as Figure 6As shown, the consultation report filling interface can further be provided with a "reference medical record" control, and the operator can trigger the control to initiate a command to reference medical record information. In response to the command, the present application system further displays a medical record information list, which contains the patient's medical record information.

[0111] like Figure 7 , illustrates an interface diagram for referencing medical record information, which may include various medical record information of patients.

[0112] The operator can select the target medical record segment in the medical record information list, and then refer to the target medical record segment in the consultation report filling interface, which saves the operator from the process of looking up relevant materials and improves work efficiency.

[0113] In some possible implementations, the method of the present application can further support doctors to view the details of the response information given by the AI ​​agent during the human-computer dialogue. Figure 5 As shown in the figure, after the AI ​​agent outputs the response information, a "View Details" control can be added to the dialogue interface for the response information. The doctor can trigger the control to enter the details interface, where the doctor can view the details corresponding to the response information of this round. This includes but is not limited to: the logical reasoning process of each AI agent participating in the reasoning of the current round of dialogue process, the arguments and evidence given by each AI agent, etc.

[0114] The specific implementation process may include:

[0115] After outputting the response information, in response to the operation object's operation to view the details of the response information, a details page is displayed, which includes but is not limited to: the logical reasoning process of each AI agent participating in the reasoning in the current round of dialogue process, the arguments and evidence given by each AI agent, etc.

[0116] Among them, multiple AI agents can execute the reasoning process in parallel or serially.

[0117] The consultation method provided in the embodiment of the present application can output the details of the response information to the user, including the logical reasoning process, arguments and evidence of each AI agent, which can facilitate doctors and researchers to understand the reasoning logic of the AI ​​agent in detail.

[0118] In some possible implementations, the medical consultation method of the present application may also support the collection and management of multimodal information of patients.

[0119] Among them, multimodal information includes basic patient information, medical record information (such as admission records, physical examinations, surgical information, etc.), laboratory test information, imaging examination information, etc.

[0120] The process of managing multimodal patient information includes but is not limited to:

[0121] Patient information editing: When patient information is added or modified, the new or modified information can be edited to ensure the sustainability of the system.

[0122] Adding new patients: Doctors or other staff can import new patients by editing or importing files in the corresponding format.

[0123] Patient file list display: Patient files can be displayed in list form, where the file list includes but is not limited to: patient name, department, attending physician and other basic information for easy query and positioning.

[0124] In some possible implementations, the medical consultation method of the present application may also support recording and managing the consultation process. The consultation process may include: consultation application, consultation doctor invitation, consultation review, consultation process, consultation report filling process after the consultation, follow-up process, etc.

[0125] Reference Fig. 9 , which illustrates a consultation process.

[0126] The consultation application process may include selecting the patient, selecting the urgency of the consultation, selecting the consulting doctor team, setting the expected consultation time, filling in the consultation location, the patient's brief medical history, the purpose of the consultation, etc.

[0127] After the consultation application is submitted, the invited doctor can see this information. At the same time, the reviewing doctor can also receive the application and review whether the consultation can begin. If rejected, the reason for rejection can be further filled in.

[0128] Once the consultation application is approved, the consultation can begin, and the speeches of the doctor and the AI ​​agent will be recorded during the consultation.

[0129] After the consultation, the designated doctor can fill in the consultation report, which includes but is not limited to: the consultation host, consultation results, whether to choose a follow-up visit, the follow-up doctor, and the expected follow-up time, etc. If a follow-up visit is chosen, the follow-up doctor is required to fill in the follow-up report.

[0130] In one possible implementation, the operations and data generated in each process of the consultation process can be output and displayed in the form of a list, and the doctor can view the list and the details of the corresponding operation content.

[0131] Further optionally, the consultation method of the present application may also manage the consulting doctor team, specifically, it may maintain the consulting doctor team information, including the team name, team members, team profile, etc.

[0132] The following is a description of a medical consultation system provided in an embodiment of the present application. The medical consultation system described below and the medical consultation method described above can be referenced to each other.

[0133] In some embodiments of the present application, a medical consultation system is further introduced, combining Fig.10 As shown, the medical consultation system 100 may include the following modules:

[0134] Patient management module 10, AI consultation module 11 and consultation management module 12.

[0135] The patient management module 10 is used to collect and manage multimodal information of patients;

[0136] The AI ​​consultation module 11 is used to conduct multiple rounds of dialogue with the consulting doctor in real time. In each round of dialogue, one or more configured AI agents understand the content of the consulting doctor's speech based on the patient's multimodal information and generate response information by reasoning;

[0137] The consultation management module 12 is used to record and manage the consultation process, wherein the consultation process includes the consultation report filling process after the consultation. The consultation management module manages the consultation report filling process, including:

[0138] In response to an instruction by the operation subject to quote a consultation opinion in a consultation report filling interface, a consultation opinion list is displayed, wherein the consultation opinion list includes the opinion of each AI agent in each round of dialogue process;

[0139] In response to the operation of the operating object selecting the target AI agent opinion in the consultation opinion list, the target AI agent opinion is referenced in the consultation report filling interface.

[0140] In a possible implementation, the process of the consultation management module 12 managing the consultation report filling process may also include:

[0141] In response to an instruction by the operator to reference test information on a consultation report filling interface, a test information list is displayed, wherein the test information list includes various laboratory test results completed by the patient;

[0142] In response to the operation of the operating object selecting a target test result in the test information list, the target test result is referenced in the consultation report filling interface.

[0143] In a possible implementation, the process of the consultation management module 12 managing the consultation report filling process may also include:

[0144] In response to an instruction of the operation subject to reference medical record information in the consultation report filling interface, display a medical record information list, wherein the medical record information list includes the patient's medical record information;

[0145] In response to the operation of the operating object selecting a target medical record segment in the medical record information list, the target medical record segment is referenced in the consultation report filling interface.

[0146] In a possible implementation, the AI ​​consultation module 11 may also be used for:

[0147] After outputting the response information, in response to the operation of the operating object to view the details of the response information, a details page is displayed, and the details page includes: the logical reasoning process of each AI agent participating in the reasoning in the current round of dialogue process, and / or, the arguments and evidence given by each AI agent, wherein multiple AI agents can execute the reasoning process in parallel or serially.

[0148] In a possible implementation, the process of the consultation management module 12 managing the consultation process may include:

[0149] The operations and data generated in each process of the consultation process are output and displayed in list form.

[0150] In a possible implementation, the medical consultation system of the present application may further include:

[0151] The consultation log recording module is used to record and count the speeches of the consulting doctors and the opinions of each AI agent during the consultation process.

[0152] The medical consultation system provided in the embodiment of the present application can be opened and logged in in a PC browser in the form of a web application. In addition, it can also be run in a terminal in the form of an application APP.

[0153] The present application also provides an electronic device in an embodiment. Fig.11 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, computers, medical robots, etc. Fig.11 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0154] like Fig.11As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the medical consultation method of the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0155] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig.11 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0156] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any medical consultation method provided in the embodiment of the present application.

[0157] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any medical consultation method provided in the embodiment of the present application.

[0158] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0159] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0160] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0161] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0162] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A medical consultation system, characterized in that: include: Patient management module, used to collect and manage multimodal information of patients; The AI ​​consultation module is used to conduct multiple rounds of real-time dialogues with the consulting doctor. During each round of dialogue, one or more configured AI agents understand the consulting doctor's speech based on the patient's multimodal information and generate response information through reasoning; The consultation management module is used to record and manage the consultation process, wherein the consultation process includes the consultation report filling process after the consultation. The consultation management module manages the consultation report filling process, including: In response to an instruction by the operation subject to quote a consultation opinion in a consultation report filling interface, a consultation opinion list is displayed, wherein the consultation opinion list includes the opinion of each AI agent in each round of dialogue process; In response to the operation of the operating object selecting the target AI agent opinion in the consultation opinion list, the target AI agent opinion is referenced in the consultation report filling interface.

2. The system according to claim 1, characterized in that The process of the consultation management module managing the consultation report filling process also includes: In response to an instruction by the operator to reference test information on a consultation report filling interface, a test information list is displayed, wherein the test information list includes various laboratory test results completed by the patient; In response to the operation of the operating object selecting a target test result in the test information list, the target test result is referenced in the consultation report filling interface.

3. The system according to claim 1, characterized in that The process of the consultation management module managing the consultation report filling process also includes: In response to an instruction of the operation subject to reference medical record information in the consultation report filling interface, display a medical record information list, wherein the medical record information list includes the patient's medical record information; In response to the operation of the operating object selecting a target medical record segment in the medical record information list, the target medical record segment is referenced in the consultation report filling interface.

4. The system according to claim 1, characterized in that The AI ​​consultation module is also used for: After outputting the response information, in response to the operation of the operating object to view the details of the response information, a details page is displayed, and the details page includes: the logical reasoning process of each AI agent participating in the reasoning in the current round of dialogue process, and / or, the arguments and evidence given by each AI agent, wherein multiple AI agents can execute the reasoning process in parallel or serially.

5. The system according to claim 1, characterized in that The consultation process also includes at least one of the following processes: consultation application, consultation doctor invitation, consultation review, consultation process and follow-up process after the consultation; The consultation management module manages the consultation process, including: The operations and data generated in each process of the consultation process are output and displayed in the form of a list.

6. The system according to any one of claims 1 to 5, characterized in that: Also includes: The consultation log recording module is used to record and count the speeches of the consulting doctors and the opinions of each AI agent during the consultation process.

7. A medical consultation method, characterized in that: include: In response to the speech content input by the consulting doctor during the multi-round dialogue process, one or more configured AI agents understand the speech content based on the patient's multimodal medical information, generate response information by reasoning, and record the speech content of the consulting doctor and the opinions of each AI agent during the multi-round dialogue process; In response to an instruction by the operation subject to quote a consultation opinion in a consultation report filling interface, a consultation opinion list is displayed, wherein the consultation opinion list includes the opinion of each AI agent in each round of dialogue process; In response to the operation of the operating object selecting the target AI agent opinion in the consultation opinion list, the target AI agent opinion is referenced in the consultation report filling interface.

8. The method according to claim 7, characterized in that Also includes: In response to an instruction by the operator to reference test information on a consultation report filling interface, a test information list is displayed, wherein the test information list includes various laboratory test results completed by the patient; In response to the operation of the operating object selecting a target test result in the test information list, the target test result is referenced in the consultation report filling interface.

9. The method according to claim 7, characterized in that: Also includes: In response to an instruction of the operation subject to reference medical record information in the consultation report filling interface, display a medical record information list, wherein the medical record information list includes the patient's medical record information; In response to the operation of the operating object selecting a target medical record segment in the medical record information list, the target medical record segment is referenced in the consultation report filling interface.

10. The method according to claim 7, characterized in that Also includes: After outputting the response information, in response to the operation of the operating object to view the details of the response information, a details page is displayed, and the details page includes: the logical reasoning process of each AI agent participating in the reasoning in the current round of dialogue process, and / or, the arguments and evidence given by each AI agent, wherein multiple AI agents can execute the reasoning process in parallel or serially.

11. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the medical consultation method as described in any one of claims 7 to 10.

12. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the medical consultation method as described in any one of claims 7 to 10 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the medical consultation method as described in any one of claims 7 to 10 is implemented.

Images