Method and system for providing medical treatment service and storage medium

Abstract

The embodiment of the invention provides a method and system for providing medical treatment service and a storage medium. The method is executed by at least one processor and comprises the following steps of: performing registration inquiry on a patient through a patient terminal of the patient so as to determine a doctor registered by the patient; based on the department of the doctor, performing pre-inquiry on the patient through the patient terminal to generate a pre-inquiry record of the patient; displaying interface elements related to the pre-inquiry record through a consulting room terminal in a consulting room in response to detecting that the patient starts inquiry, so as to display the pre-inquiry record to the doctor and the patient; the method comprises the following steps: generating an initial diagnosis record based on perception information collected by perception equipment in a consulting room in a treatment process of a patient, and carrying out registration inquiry on the patient through a patient terminal of the patient: displaying a first virtual character through the patient terminal to carry out registration inquiry, the step of pre-inquiry on the patient through the patient terminal comprises the step of displaying the second virtual character through the patient terminal to carry out pre-inquiry.

Description

Cross-references

[0001] This application is a divisional application for a Chinese application with an application date of November 29, 2024, application number 202411742797.2, and invention name “A method, system and storage medium for providing medical treatment services”. The Chinese application claims priority to the international application with application number PCT / CN2024 / 109057 filed on July 31, 2024, and the entire contents of the above application are incorporated herein by reference. Technical Field

[0002] This specification relates to the field of medical services, and in particular to a method, system and storage medium for providing medical consultation services. Background Art

[0003] Outpatient services play an important role in the hospital system. They are of great significance in promoting early diagnosis, improving disease management, reducing hospitalization needs, providing health education and consultation, facilitating patient access, and alleviating pressure on the medical system. Outpatient services involve multiple links such as registration, waiting, medical treatment, and medication collection. However, there are many problems with current outpatient services, including patients having difficulty getting a suitable number, patients waiting too long for medical treatment, inefficient services, and fragmented and lacking continuity in the medical service process.

[0004] Therefore, it is necessary to provide a system, method and storage medium for providing medical treatment services, so as to improve the efficiency of medical treatment and the quality of medical treatment services. Summary of the invention

[0005] One aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized by comprising: presenting a first interface element to a doctor and a patient via a first terminal in a consulting room, wherein the first interface element at least includes the electronic health record of the patient; obtaining a control instruction issued by the target patient and / or the doctor based on the perception information collected by one or more sensing devices in the consulting room during the consultation process, wherein the control instruction is used to retrieve at least a portion of the electronic health record; in response to the control instruction, retrieving at least a portion of the electronic health record, and presenting at least a portion of the electronic health record via the first terminal.

[0006] Another aspect of the embodiments of the present specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized by comprising: making a first inquiry to the patient through the patient terminal to determine the doctor with whom the patient is registered; based on the doctor's department, controlling the patient terminal to make a second inquiry to the patient to generate a pre-consultation record for the patient; in response to detecting that the patient has started a consultation, controlling the consulting room terminal to display the pre-consultation record to the doctor and the patient on an interface element; generating an initial diagnosis record based on the perception information collected by the perception device in the consulting room during the patient's consultation process.

[0007] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized by comprising: making a first inquiry to the patient through a patient terminal to determine the doctor with whom the patient is registered; making a second inquiry to the patient through the patient terminal based on the doctor's department to generate a pre-consultation record for the patient; in response to detecting that the patient has started a consultation, controlling the patient terminal and the doctor terminal to display the pre-consultation record to the patient and the doctor on an interface element respectively; generating an initial diagnosis record based on the perception information collected by the patient terminal and the doctor terminal during the patient's consultation process.

[0008] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized by comprising: presenting a first interface element to a target user through at least one terminal; the first interface element is related to the electronic health record of a patient receiving medical consultation services; the target users include at least the patient and the doctor; based on the perception information, obtaining a control instruction initiated by at least one of the target users; the perception information is collected based on one or more perception devices during the consultation process; the control instruction is used to present at least a portion of the medical data; and in response to the control instruction, retrieving at least a portion of the medical data and presenting at least a portion of the medical data through the at least one terminal.

[0009] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized in that it includes: performing a registration inquiry on the patient through the patient terminal of the patient to determine the doctor with whom the patient is registered; based on the department of the doctor, performing a pre-consultation inquiry on the patient through the patient terminal to generate a pre-consultation record of the patient; in response to detecting that the patient has started a consultation, displaying interface elements related to the pre-consultation record through the clinic terminal in the clinic to show the pre-consultation record to the doctor and the patient; generating an initial diagnosis record based on the perception information collected by the perception device in the clinic during the patient's consultation, wherein: performing a registration inquiry on the patient through the patient terminal of the patient includes: performing the registration inquiry by displaying a first virtual character through the patient terminal, and performing a pre-consultation inquiry on the patient through the patient terminal includes: performing the pre-consultation inquiry by displaying a second virtual character through the patient terminal.

[0010] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized in that it includes: registering and inquiring the patient through the patient terminal of the patient to determine the doctor with whom the patient is registered; based on the doctor's department, performing a pre-consultation inquiry on the patient through the patient terminal to generate a pre-consultation record for the patient; in response to detecting that the patient has started a consultation, displaying interface elements related to the pre-consultation record through the patient terminal and the doctor's doctor terminal respectively to show the pre-consultation record to the patient and the doctor; generating an initial diagnosis record based on the perception information collected by the patient terminal and the doctor terminal during the patient's consultation, wherein: the registering and inquiring the patient through the patient terminal of the patient includes: performing the registration inquiry through displaying a first virtual character through the patient terminal, and the pre-consultation inquiry on the patient through the patient terminal includes: performing the pre-consultation inquiry through displaying a second virtual character through the patient terminal.

[0011] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized in that it includes: presenting a first interface element to a target user through at least one terminal, the first interface element is related to the electronic health record of a patient receiving medical consultation services, and the target users include at least the patient and the doctor; based on the perception information, obtaining a control instruction initiated by at least one of the target users, the perception information is collected by the perception device during the consultation process, and the control instruction is used to retrieve at least a part of the electronic health record; and in response to the control instruction, retrieving at least a part of the electronic health record and presenting it to the target user through the at least one terminal, wherein the at least one terminal presents at least a part of the electronic health record to the target user using extended reality technology.

[0012] Another aspect of the embodiments of this specification provides a method for providing registration services. The method is executed by at least one processor, and is characterized by comprising: obtaining the patient's chief complaint through the patient's patient terminal or the hospital's registration terminal; determining at least one candidate department based on the patient's chief complaint; performing registration inquiries through displaying a virtual character through the patient terminal or the registration terminal based on the patient's chief complaint and the at least one candidate department; and determining the doctor based on the data collected by the patient terminal or the registration terminal in the registration inquiry.

[0013] Another aspect of the embodiments of this specification provides a method for providing a pre-consultation service. The method is executed by at least one processor, and is characterized in that it includes: based on the department where the doctor registered by the patient is located, displaying a virtual character to conduct a pre-consultation inquiry to the patient through the patient terminal of the patient; generating the pre-consultation record based on the data collected by the patient terminal in the pre-consultation inquiry; in response to detecting that the patient has started the consultation, presenting interface elements related to the pre-consultation record of the patient through the clinic terminal in the clinic, wherein the processor is configured with a pre-consultation agent corresponding to the pre-consultation service of the department, the pre-consultation inquiry is executed using the pre-consultation agent, and the pre-consultation agent stores the knowledge data corresponding to the department and can achieve self-evolution using artificial intelligence technology.

[0014] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized by comprising: acquiring perception information in real time during the patient's consultation process, wherein the perception information is collected by a perception device in the consulting room during the consultation process; during the consultation process, using a consultation agent to process the perception information to generate consultation suggestions, wherein the consultation suggestions include at least one of supplementary inquiry suggestions, physical examination suggestions, prescription suggestions, and treatment plan suggestions; after the consultation process is completed, using the consultation agent to process the perception information to generate an initial diagnosis record, wherein the consultation agent uses artificial intelligence technology to achieve self-evolution.

[0015] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized by comprising: receiving a remote medical consultation request initiated by a patient, wherein the remote medical consultation request is related to a remote medical companion; responding to the remote medical consultation request and detecting that the patient has started medical consultation, and displaying a real-time image of the remote medical companion to the patient and the doctor via at least one terminal device using extended reality technology.

[0016] Another aspect of the embodiments of this specification provides a method for providing medical consultation services. The method is executed by at least one processor, and is characterized in that it includes: generating a health monitoring plan based on the patient's target diagnosis record, and the health monitoring plan is related to at least one of medication monitoring, health habit monitoring, and physiological data monitoring; generating health monitoring instructions based on the health monitoring plan, and the health monitoring instructions include at least one of medication instructions, health habit instructions, and physiological data monitoring instructions; sending the health monitoring instructions to a monitoring device to obtain the patient's health monitoring information from the monitoring device; updating the health monitoring plan based on the health monitoring information, wherein the processor is configured with a post-diagnosis agent, and the method is executed by the post-diagnosis agent, and the post-diagnosis agent realizes self-evolution based on artificial intelligence technology.

[0017] Another aspect of the embodiments of this specification provides a system for providing medical consultation services. The system includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute at least part of the computer instructions to implement a method for providing medical consultation services.

[0018] Another aspect of the embodiments of the present specification provides a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes a method for providing medical consultation services. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0020] Figure 1 is a block diagram of an exemplary medical service system according to some embodiments of the present application;

[0021] Figure 2 is a schematic diagram of an exemplary medical service system according to some embodiments of the present application;

[0022] Figure 3 is a schematic diagram of an exemplary hospital support platform according to some embodiments of the present application;

[0023] Figure 4 is an exemplary module diagram of a medical consultation system according to some embodiments of this specification;

[0024] Figure 5 is a schematic diagram of an exemplary medical consultation process according to some embodiments of this specification;

[0025] Figure 6 is a schematic diagram of an exemplary process of determining a doctor to be registered with a patient according to some embodiments of this specification;

[0026] Figure 7 is a schematic diagram of a path planning and path guidance service according to some embodiments of this specification;

[0027] Figure 8 is a schematic diagram of an exemplary process of providing pre-consultation services according to some embodiments of this specification;

[0028] Fig. 9 is a schematic diagram of an exemplary process of initiating a second inquiry according to some embodiments of this specification;

[0029] Fig.10 is a schematic diagram of a process of presenting a virtual character through an XR device according to some embodiments of this specification;

[0030] Fig.11 is a schematic diagram of an exemplary medical consultation system according to some embodiments of the present specification;

[0031] Fig.12 is an exemplary flowchart of displaying medical data to a target user during a medical consultation process according to some embodiments of this specification;

[0032] Fig.13is a flowchart of an exemplary process of providing medical consultation services based on perception information according to some embodiments of this specification;

[0033] Fig.14 is a schematic diagram of an exemplary medical consultation interface according to some embodiments of this specification;

[0034] Fig.15 is a flowchart of an exemplary process of generating an initial diagnosis record according to some embodiments of the present specification;

[0035] Fig.16 is a schematic diagram of an exemplary process of providing remote medical escort services according to some embodiments of this specification;

[0036] Fig.17 is a schematic diagram of an exemplary process of providing health monitoring services according to some embodiments of this specification;

[0037] Fig.18 is a schematic diagram of an exemplary medical consultation process according to some embodiments of this specification;

[0038] Fig.19 It is a schematic diagram of an exemplary medical consultation process according to some embodiments of the present specification. DETAILED DESCRIPTION

[0039] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0040] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0041] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0042] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0043] Figure 1 is a block diagram of an exemplary medical service system 100 according to some embodiments of the present application.

[0044] The medical service system 100 can also be called a meta-hospital system. It is built based on a variety of innovative technologies, including metaverse technology, XR technology (for example, augmented reality (AR) technology, virtual reality (VR) technology, mixed reality (MR) technology, etc.), AI technology, digital twin technology, IOT technology, data circulation technology (for example, blockchain technology, data privacy computing technology), spatial computing technology, image rendering technology, etc.

[0045] like Figure 1 As shown, the medical service system 100 may include a physical hospital 110, a virtual hospital 130, a user space application 120, and a hospital support platform 140. In some embodiments, the hospital support platform 140 may map data related to the physical hospital 110 to the virtual hospital 130 corresponding to the physical hospital 110, and provide user services to related users of the physical hospital 110 through the user space application 120.

[0046] Physical hospital 110 refers to a hospital that exists in the physical world and has tangible attributes.Herein, healthcare facilities that provide medical, surgical, and psychiatric care and treatment to people are collectively referred to as hospitals.

[0047] like Figure 1 As shown, the physical hospital 110 may include multiple physical entities. For example, the multiple physical entities may include departments, users, hardware equipment, user services, public areas, medical service processes, etc., or any combination thereof.

[0048] Departments are specialized units or departments that provide specific types of medical care, treatment, and services. Each department may focus on a particular area of ​​medicine and may be staffed by health care professionals with expertise in that area. For example, departments may include outpatient departments, inpatient departments, surgical departments, support departments (e.g., registration departments, pharmacy departments), internal medicine, surgery, specialty medicine, child health departments, etc., or any combination thereof.

[0049] The user may include any user associated with the physical hospital 110 (or referred to as a related user of the physical hospital 110). For example, the user may include a patient (or a part of a patient (e.g., an organ)), a companion, a visitor of a patient, a hospital staff of the physical hospital 110, a supplier of the physical hospital 110, an application developer of the physical hospital 110, etc., or any combination thereof. The hospital staff of the physical hospital 110 may include medical service providers (e.g., doctors, nurses, technicians, etc.), hospital administrators, support staff, or the like, or any combination thereof. Exemplary hospital administrators may include department nursing administrators, clinical administrators, department directors, hospital directors, hospital administrators, functional administrators, or the like, or any combination thereof.

[0050] The hardware devices may include hardware devices located in the physical hospital 110 and / or hardware devices that communicate with hardware devices in the physical hospital 110. Exemplary hardware devices may include terminal devices, medical service devices, sensing devices, basic devices, etc., or any combination thereof.

[0051] The terminal device may include a terminal device that interacts with a user of the medical service system 100. For example, the terminal device may include a terminal device that interacts with a patient (also referred to as a patient terminal), a terminal device that interacts with a patient's doctor (also referred to as a doctor terminal), a terminal device that interacts with a nurse (also referred to as a nurse terminal), a terminal device that interacts with a remote visitor (also referred to as a remote terminal device), or a public terminal of a hospital (e.g., a consulting room terminal, a bedside terminal device, a terminal device in a waiting area, an intelligent surgical terminal), etc., or any combination thereof. In the present application, unless clearly obtained from the context or otherwise specified by the context, terminal devices owned by a user and terminal devices provided to a user by the physical hospital 110 are collectively referred to as terminal devices of the user or terminal devices that interact with the user.

[0052] The terminal device may include a mobile terminal, an XR device, a smart wearable device, etc. The mobile terminal may include a smart phone, a personal digital assistant (PDA), a display, a gaming device, a navigation device, a handheld terminal (POS), a tablet computer, etc., or any combination thereof.

[0053] XR devices may include devices that allow users to participate in extended reality experiences. For example, XR devices may include VR components, AR components, MR components, etc., or any combination thereof. In some embodiments, XR devices may include XR helmets, XR glasses, XR patches, stereo headphones, etc., or any combination thereof. For example, XR devices may include Google Glass TM 、Oculus Rift TM 、GearVR TM 、AppleVisionproTM Etc. Specifically, the XR device may include a display component on which virtual content can be presented and / or displayed. In some embodiments, the XR device may further include an input component. The input component enables user interaction between the user and the virtual content (e.g., a virtual surgical environment) displayed by the display component. For example, the input component may include a touch sensor, a microphone, an image sensor, etc., which is configured to receive user input, which can be provided to the XR device and used to control the virtual world by changing the visual content presented on the display component. The input component may include a handle, a glove, a stylus, a console, etc.

[0054] Smart wearable devices may include smart bracelets, smart shoes and socks, smart glasses, smart helmets, smart watches, smart clothes, smart backpacks, smart accessories, etc., or any combination thereof. In some embodiments, smart wearable devices may obtain physiological data (e.g., heart rate, blood pressure, body temperature, etc.) of the user.

[0055] The medical service equipment may be configured to provide medical services to the patient. For example, the medical service equipment may include examination equipment, nursing equipment, treatment equipment, etc., or any combination thereof.

[0056] The inspection device may be configured to provide inspection services to the patient, for example, to collect the patient's inspection data. Exemplary inspection data may include heart rate, respiratory rate, body temperature, blood pressure, medical imaging data, body fluid test reports (e.g., blood test reports), etc., or any combination thereof. Accordingly, the inspection device may include a vital sign monitor (e.g., a blood pressure monitor, a blood glucose meter, a heart rate meter, a thermometer, a digital stethoscope, etc.), a medical imaging device (e.g., a computer tomography (CT) device, a digital subtraction angiography (DSA) device, a magnetic resonance (MR) device, etc.), a laboratory device (e.g., a routine blood test device, etc.), or any combination thereof.

[0057] Nursing equipment can be configured to provide nursing services to patients and / or assist medical service providers in providing nursing services. Exemplary nursing equipment can include hospital beds, patient care robots, smart nursing vehicles, smart medicine boxes, smart wheelchairs, etc.

[0058] The therapeutic device can be configured to provide therapeutic services to patients and / or assist medical service providers in providing therapeutic services. Exemplary therapeutic devices can include surgical devices, radiation therapy devices, physical therapy devices, etc., or any combination thereof.

[0059] The perception device may be configured to collect perception information related to the environment in which it is located. For example, the perception device may include an image sensor, a sound sensor, etc. The image sensor may be configured to collect image data in the physical hospital 110, and the sound sensor may be configured to collect voice signals in the physical hospital 110. In some embodiments, the perception device may be an independent device or may be integrated into another device. For example, the sound sensor may be part of a medical service device or a terminal device.

[0060] The infrastructure can be configured to support data transmission, storage and processing. For example, the infrastructure can include networks, computer room facilities, computing equipment, computing chips, storage devices, etc.

[0061] In some embodiments, at least a portion of the hardware devices of the physical hospital 110 are IoT devices. IoT devices refer to devices with sensors, processing capabilities, software, and other technologies that connect and exchange data with other devices and systems via the Internet or other communication networks. For example, one or more medical service devices and / or sensing devices of the physical hospital 110 are IoT devices and are configured to transmit collected data to the hospital support platform 140 for storage and / or processing.

[0062] User services may include any services provided to users by the hospital support platform 140. For example, user services include medical services provided to patients and / or accompanying persons, support services provided to staff of the physical hospital 110 and / or suppliers of the physical hospital 110, etc. In some embodiments, user services may be provided to patients, doctors, and hospital administrators through the user space application 120, which will be described in detail in the following description.

[0063] Public areas refer to shared spaces accessible to users (or portions of users) in the physical hospital 110. For example, public areas may include a reception area (eg, front desk), a waiting area, a corridor, or any combination thereof.

[0064] The medical service process refers to the process of providing corresponding medical services to patients. The medical service process usually includes several links and / or steps, and the user needs to go through these links and / or steps to obtain the corresponding medical services. Exemplary medical service processes may include outpatient processes, inpatient processes, surgical processes, or similar processes, or any combination thereof. In some embodiments, the medical service process may include medical service processes corresponding to different departments, different diseases, etc. In some embodiments, a preset data collection protocol can be set and the standard links involved in the medical service process and how to collect data related to the medical service process can be specified.

[0065] The user space application 120 provides users with access to user services provided by the hospital support platform 140. The user space application 120 can be an application, a plug-in, a website, a small program, or any other suitable form. For example, the user space application 120 is an application installed on a user terminal device, and the application includes a user interface for the user to initiate a request and receive the corresponding service.

[0066] In some embodiments, the user space application 120 may include different applications corresponding to different types of users. For example, the user space application 120 includes a patient space application corresponding to the patient, a medical space application corresponding to the doctor, a management space application corresponding to the manager, etc., or any combination thereof. The user services provided by the patient space application, the medical space application, and the management space application are also referred to as patient space services, medical space services, and management space services, respectively. Exemplary patient space services include registration services, path guidance services, pre-consultation services, remote medical services, hospitalization services, discharge services, etc. Exemplary medical space services include scheduling services, surgical planning services, surgical simulation services, patient management services, remote ward rounds services, remote outpatient services, etc. Exemplary manager space services include monitoring services, medical service evaluation services, equipment parameter setting services, service parameter setting services, resource scheduling services, etc.

[0067] In some embodiments, the patient space application, the medical space application, and the management space application may be integrated into one user space application 120, and the user space application 120 may be configured to provide access portals for each type of user (e.g., patients, medical service providers, managers, etc.). As an example only, a specific user may have a corresponding account that can be used to log in to the user space application, view corresponding diagnosis and treatment data, and obtain corresponding user services.

[0068] According to some embodiments of the present application, by providing user space applications for different types of users, each type of user can easily obtain various user services that he / she may need on his / her corresponding user space application. In addition, currently users usually need to install various applications to obtain different user services, which leads to poor user experience and high development costs. Therefore, the user space application of the present application can improve user experience, improve service quality and efficiency, enhance service security, and reduce development or operation costs.

[0069] In some embodiments, the user space application 120 can be configured to provide relevant users of the physical hospital 110 with an access portal for interacting with the virtual hospital 130. For example, through the user space application 120, the user can enter instructions for retrieving the digital content of the virtual hospital 130 (e.g., hardware equipment, patient organs, digital twin models of public areas), view the digital content, and interact with the digital content. As another example, through the user space application 120, the user can communicate with a virtual character representing an intelligent body. In some embodiments, the public terminal of the hospital can install a tube space application, and the administrator account of the department corresponding to the public terminal can be logged in in the tube space application. Users can receive user services through the tube space application installed in the public terminal.

[0070] The virtual hospital 130 is a digital twin (ie, a virtual representation or virtual copy) of the physical hospital 110 and is used to simulate, analyze, predict, and optimize the operating status of the physical hospital 110. For example, the virtual hospital 130 may be a real-time digital copy of the physical hospital 110.

[0071] In some embodiments, the virtual hospital 130 may be presented to the user using digital technology. For example, when the relevant user interacts with the virtual hospital 130, at least a portion of the virtual hospital 130 may be presented to the relevant user using XR technology. As an example only, at least a portion of the virtual hospital 130 may be superimposed on the relevant user's real-world view using MR technology.

[0072] In some embodiments, the virtual hospital 130 may include a digital twin of a physical entity associated with the physical hospital 110. A digital twin refers to a virtual representation of a physical entity (e.g., a virtual copy, a mapping, a digital simulator). A digital twin can reflect and predict the state, behavior, and performance of a physical entity in real time. For example, the virtual hospital 130 may include a digital twin of at least a portion of the medical services, departments, users, hardware equipment, user services, public areas, medical service processes, etc. of the physical hospital 110. The digital twin of a physical entity may be in a variety of forms, including models, images, graphics, text, numerical values, and the like. For example, a digital twin may be a virtual hospital corresponding to a physical hospital, virtual personnel (e.g., virtual doctors, virtual nurses, and virtual patients) corresponding to personnel entities (e.g., doctors, nurses, and patients), virtual equipment (e.g., virtual imaging equipment and virtual scalpels) corresponding to medical service equipment (e.g., imaging equipment and scalpels), and the like.

[0073] In some embodiments, the digital twin may include one or more first digital twins and / or one or more second digital twins. The state of each first digital twin may be updated based on the update of the state of the corresponding physical entity. For example, in the process of mapping data related to the physical hospital 110 to the virtual hospital 130, one or more first digital twins may be updated. One or more second digital twins may be updated by at least one of the user space applications 120, and the update of each second digital twin may result in an update of the state of the corresponding physical entity. In other words, when the corresponding physical entity changes its state, the first digital twin may be updated accordingly; when the second digital twin is updated, the state of the corresponding physical entity also changes accordingly. For example, one or more first digital twins may include digital twins of public areas, medical services, users, hardware devices, etc., and one or more second digital twins may include digital twins of hardware devices, user services, medical service processes, etc. It should be understood that the digital twin may be a first digital twin or a second digital twin.

[0074] According to some embodiments of the present application, by generating a virtual hospital 130 that includes a digital twin of physical entities related to the physical hospital 110, the physical hospital 110 (including hardware equipment, users, user services, medical service processes, etc.) can be simulated and tested in a safe and controllable environment. Through virtual reality linkage (for example, real-time interaction between the physical hospital 110 and the virtual hospital 130), various medical scenarios can be predicted and responded to more accurately, thereby improving the quality and efficiency of medical services. In addition, the use of XR technology and virtual reality integration technology makes the interaction between relevant users more natural and intuitive, providing a more comfortable and efficient medical environment, thereby enhancing the user experience.

[0075] In some embodiments, the virtual hospital 130 may further include an intelligent agent that achieves self-evolution based on data related to the physical hospital 110 and AI technology.

[0076] An agent refers to an agent that acts in an intelligent manner. For example, an agent may include a computing / software entity that can autonomously learn and evolve, and perceive and analyze data to perform specific tasks and / or achieve specific goals (e.g., medical service processes). Through AI technology (e.g., reinforcement learning, deep learning, etc.), the agent can continuously learn and self-optimize in interaction with the environment. In addition, the agent can collect and analyze massive data (e.g., relevant data of the physical hospital 110) through big data technology, and mine patterns and learning rules from the data to optimize the decision-making process, thereby identifying environmental changes in uncertain or dynamic environments, responding quickly, and making reasonable judgments. For example, the agent can autonomously learn and evolve based on AI technology to adapt to changes in the physical hospital 110. As an example only, the agent can be built based on NLP technology (e.g., large language models, etc.), and can automatically learn and autonomously update through a large amount of language text (e.g., hospital business data and patient feedback information) to improve the quality of user services provided by the physical hospital 110.

[0077] In some embodiments, agents may include different types of agents corresponding to different medical service processes, different user services, different departments, different diseases, different hospital positions (e.g., nurses, doctors, technicians, etc.), different links in the medical service process, etc. Agents of a specific type are used to process tasks corresponding to a specific type. In some embodiments, an agent may correspond to different medical service processes (or different medical services, or different departments, or different diseases, or different hospital locations). In some embodiments, agents may operate with reference to basic configuration data (e.g., dictionaries, knowledge graphs, templates, etc.) of the departments and / or diseases corresponding to the agents. In some embodiments, multiple agents may collaborate and share information through network communications to jointly complete complex tasks.

[0078] In some embodiments, the configuration of the agent can be set. For example, basic configuration data used by the agent in operation can be set. The basic configuration data can include dictionaries, knowledge databases, templates, etc. For another example, the use rights of the agent can be set for different users. In some embodiments, the administrator of the physical hospital 110 can set the configuration of the agent through the management space application.

[0079] In some embodiments, the agent can be integrated into or deployed on a hardware device. For example, the agent corresponding to the inpatient service can be integrated into a hospital bed or a presentation device of a hospital bed. In some embodiments, the agent can be integrated into or deployed on an embodied intelligent robot. An embodied intelligent robot refers to a robotic system that combines physical presence (embodiment) with intelligent behavior (cognition). An embodied intelligent robot can be configured to interact with the real world in a way that mimics or supplements human capabilities, using physical form and cognitive functions to perform tasks, make decisions, and adapt to the environment. By leveraging artificial intelligence and sensor technology, embodied intelligent robots can operate autonomously, interact with the environment, and continuously improve performance. For example, an embodied intelligent robot can be configured with an agent corresponding to a surgical service and assist doctors in performing surgery.

[0080] In some embodiments, at least a portion of user services may be provided based on an agent. For example, at least a portion of user services may be provided to relevant users based on a processing result, wherein the processing result is generated by at least one of the agents based on data related to the physical hospital 110. As an example only, the data related to the physical hospital 110 may include data related to a medical service process of the physical hospital 110, the agent may include an agent corresponding to the medical service process, and user services may be provided to relevant users of the medical service process by processing the data using the agent corresponding to the medical service process.

[0081] The hospital support platform 140 may be configured to provide technical support to the medical service system 100. For example, the hospital support platform 140 may include computing hardware and software to support innovative technologies, including XR technology, AI technology, digital twin technology, data circulation technology, etc. In some embodiments, the hospital support platform 140 may include at least a storage device for data storage and a processing device for data calculation.

[0082] In some embodiments, the hospital support platform 140 can support the interaction between the physical hospital 110 and the virtual hospital 130. For example, the processing device of the hospital support platform 140 can obtain data related to the physical hospital 110 from the hardware device and map the data related to the physical hospital 110 to the virtual hospital 130. For example, the processing device of the hospital support platform 140 can update a part of the digital twin in the virtual hospital 130 (e.g., one or more first digital twins) based on the obtained data, so that each part of the digital twin in the virtual hospital 130 can reflect the updated state of the corresponding physical entity in the physical hospital 110. Based on this digital twin that is constantly updated with the corresponding physical entity, the user can understand the state of the physical entity related to the physical hospital 110 in real time, thereby realizing the monitoring and evaluation of the physical entity. As another example, the intelligent agent corresponding to the data related to the physical hospital 110 can be trained and / or updated based on the data related to the physical hospital 110, thereby self-evolving and self-learning.

[0083] In some embodiments, the hospital support platform 140 may support and / or provide user services to relevant users of the physical hospital 110. For example, in response to receiving a user service request from a user, a processing device of the hospital support platform 140 may provide a user service corresponding to the service request. For another example, in response to detecting that a user service needs to be provided to a user, the processing device of the hospital support platform 140 may control a physical entity or a virtual entity corresponding to the user service to provide the user service. For example, in response to detecting that a patient is admitted to a hospital ward, the processing device of the hospital support platform 140 may control a smart nursing vehicle to guide a nurse to the hospital ward to perform an admission examination on the patient.

[0084] In some embodiments, at least a portion of user services may be provided to the relevant user based on the interaction between the relevant user and the virtual hospital 130. Interaction refers to the interaction or influence (e.g., conversation, behavior, etc.) between the relevant user and the virtual hospital 130. For example, the interaction between the relevant user and the virtual hospital 130 may include the interaction between the relevant user and the digital twin in the virtual hospital 130, the interaction between the relevant user and the intelligent agent, the interaction between the relevant user and the virtual character, etc., or any combination thereof.

[0085] In some embodiments, at least a portion of user services may be provided to the relevant user based on the interaction between the relevant user and at least one of the digital twins. For example, an update instruction for a second digital twin input by a relevant user may be received through the user space application 120, and the corresponding physical entity of the second digital twin may be updated according to the update instruction. As another example, a user may view a first digital twin of a physical entity (e.g., a 3D digital twin model of a patient's organ or a hardware device) through the user space application 120 to understand the state of the physical entity. Optionally, the user may change the display angle, display size, etc. of the digital twin.

[0086] In some embodiments, the processing device of the hospital support platform 140 can present a virtual character corresponding to the intelligent agent through a user space application, interact with the relevant user, and provide at least part of the user service to the relevant user based on the interaction between the relevant user and the virtual character.

[0087] In some embodiments, the hospital support platform 140 may have a five-layer structure, including a hardware device layer, an interface layer, a data processing layer, an application development layer, and a service layer. Figure 3 In some embodiments, the hardware devices of the physical hospital 110 may be part of the hospital support platform 140 .

[0088] According to some embodiments of the present application, a virtual hospital corresponding to the physical hospital can be established by comprehensively integrating various internal and external resources of the physical hospital (e.g., medical service equipment, hospital personnel, medicines and consumables, etc.). The virtual hospital can reflect the real-time status (e.g., changes, updates, etc.) of the physical entities related to the physical hospital, thereby enabling monitoring and evaluation of the physical entities. This integration can provide accurate data support for the operation and intelligent decision-making of medical services. In addition, through the virtual hospital, users related to medical services can jointly establish an open and shared ecosystem, thereby promoting the innovation and improvement of medical services.

[0089] In addition, it is possible to provide full-life cycle healthcare services for patients that are linked inside and outside the hospital. The perspective of medical services has expanded from simple disease treatment to cover the entire life cycle of patients, including prevention, diagnosis, treatment, rehabilitation, health management, etc. By establishing linkages inside and outside the hospital, physical hospitals can better integrate online and offline resources to provide patients with comprehensive and continuous medical and health services. For example, through remote monitoring and online medical consultations, patients' health conditions can be followed up in real time, treatment plans can be adjusted in a timely manner, and treatment effects can be improved.

[0090] Figure 2 is a schematic diagram of an exemplary medical service system 200 according to some embodiments of the present application.

[0091] like Figure 2 As shown, the medical service system 200 may include a processing device 210, a network 220, a storage device 230, one or more medical service devices 240, one or more sensing devices 250, one or more patient terminals 260 of a patient 261, and one or more doctor terminals 270 of a doctor 271 associated with the patient 261. In some embodiments, the components in the medical service system 200 may be interconnected and / or communicated via wireless connections, wired connections, or a combination thereof. The connections between the components of the medical service system 200 may be variable.

[0092] The processing device 210 may process data and / or information obtained from the storage device 230, the medical service device 240, the sensing device 250, the patient terminal 260, and / or the doctor terminal 270. For example, the processing device 210 may map data related to a physical hospital to a virtual hospital corresponding to the physical hospital, and provide user services to the patient 261 and the doctor 271 through the patient terminal 260 and / or the doctor terminal 270, respectively, by processing the data related to the physical hospital. As another example, the processing device 210 may maintain a digital smart object, and provide user services to the patient 261 and the doctor 271 through the patient terminal 260 and / or the doctor terminal 270, respectively, by allowing the digital smart object to participate in processing the data related to the physical hospital.

[0093] In some embodiments, the processing device 210 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processing device 210 may be located locally or remotely from the medical service system 200. In some embodiments, the processing device 210 may be implemented on a cloud platform. For example, a cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud cloud, a multi-cloud, etc., or a combination thereof.

[0094] In some embodiments, the processing device 210 may include one or more processors (e.g., a single-core processor or a multi-core processor). For illustration purposes only, only one processing device 210 is described in the medical service system 200. However, it should be noted that the medical service system 200 in the present application may also include multiple processing devices. Therefore, as in the present application, the operations and / or method steps performed by one processing device 210 may also be performed jointly or individually by multiple processing devices.

[0095] The network 220 may include any suitable network capable of facilitating information and / or data exchange for the healthcare service system 200. The network 220 may be or include a wired network, a wireless network (e.g., an 802.11 network, a Wi-Fi network), a Bluetooth™ network, a near field communication (NFC) network, etc., or any combination thereof.

[0096] The storage device 230 may store data, instructions, and / or any other information. In some embodiments, the storage device 230 may store data obtained from other components of the healthcare delivery system 200. In some embodiments, the storage device 230 may store data and / or instructions that the processing device 210 may execute or use to execute the exemplary methods described in this application.

[0097] In some embodiments, the data stored in the storage device 230 may include multimodal data. Multimodal data may include data in multiple forms (e.g., images, graphics, videos, text, etc.), various types of data, data obtained from different sources, data related to different medical services (e.g., diagnosis, surgery, rehabilitation, etc.), and data related to different users (e.g., patients, medical staff, managers, etc.). For example, the data stored in the storage device 230 may include medical data of the patient 261 reflecting the health status of the patient 261. For example, the medical data may include an electronic health record of the patient 261. An electronic health record refers to an electronic file that records various types of patient data (e.g., basic information, examination data, imaging data). For example, an electronic health record may include a three-dimensional model of multiple organs and / or tissues of the patient 261.

[0098] In some embodiments, the storage device 230 may include a mass storage device, a removable storage device, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the storage device 230 may include a data lake and a data warehouse. Figure 3 Describe in detail.

[0099] The medical service device 240 can be used to provide or assist medical services. Figure 2 As shown, the medical service equipment 240 includes a clinic terminal 240-1, a hospital bed 240-2, an intelligent surgery terminal 240-3, an intelligent nursing vehicle 240-4, an intelligent wheelchair 240-5, etc., or any combination thereof.

[0100] The clinic terminal 240-1 refers to a terminal device configured in the clinic for use by doctors and patients in the medical outpatient process. For example, the clinic terminal 240-1 may include one or more of a screen, a sound output component, an image sensor, or a sound sensor. The screen of the clinic terminal 240-1 may display a consultation interface, and data may be displayed on the consultation interface to facilitate communication between doctors and patients. Exemplary data may include electronic health records (or part thereof), pre-consultation records, medical images, 3D organ models, examination results, consultation recommendations, etc.

[0101] The hospital bed 240-2 refers to a bed in a hospital ward that can support inpatients and provide user services to patients. The hospital bed 240-2 may include a bed, a bedside terminal device, a bedside inspection device, a sensor, etc., or any combination thereof. The bedside terminal device may include an XR device, a display device, a mobile device, etc., or any combination thereof. In some embodiments, the hospital bed 240-2 may be controlled by an intelligent agent corresponding to an inpatient service, wherein the hospital bed may also be referred to as an intelligent hospital bed or a meta-hospital bed.

[0102] The intelligent surgical terminal 240-3 refers to a device configured with an intelligent agent for assisting surgery, and is controlled by an intelligent agent corresponding to the surgical service. The intelligent surgical terminal 240-3 can perceive the interaction (e.g., conversation, behavior, etc.) between the medical service provider, the patient, and the intelligent agent, and obtain the data captured by the sensing device 250, thereby providing surgical assistance. In some embodiments, the intelligent surgical terminal 240-3 can be configured to perform risk warnings for surgical operations based on the intelligent agent configured therein, generate surgical records of the surgical process, etc.

[0103] The smart nursing car 240-4 refers to a nursing car with an automatic driving function that can assist in the treatment and care of patients. For example, the smart nursing car 240-4 can be configured to guide a nurse to a hospital ward to conduct an admission examination on a patient. In some embodiments, the smart nursing car 240-4 can be controlled by an intelligent agent (e.g., an intelligent agent corresponding to an inpatient service, a nursing intelligent agent). In some embodiments, the smart nursing car 240-4 may include a trolley, a presentation device, one or more inspection devices and / or nursing tools, a sensing device (such as an image sensor, a GPS sensor, a sound sensor, etc.), etc. In some embodiments, the smart nursing car 240-4 may be configured to obtain relevant treatment and nursing information of the patient, and generate physical examination data, nursing data, etc. The physical examination data may include the patient's vital signs data. Nursing data may include detailed records of nursing operations, such as nursing time, nursing operators, nursing measures, patient reactions, etc.

[0104] Smart wheelchair 240-5 refers to a transportation device for intelligently picking up and dropping off patients. In some embodiments, smart wheelchair 240-5 can be configured to perform autonomous navigation through integrated sensors and maps, locate the patient's position using radio frequency identification devices (RFID), Bluetooth or Wi-Fi signals, and identify the patient through biometric technology. In some embodiments, smart wheelchair 240-5 can be controlled by an intelligent agent (e.g., an intelligent agent corresponding to inpatient services, an intelligent agent corresponding to surgical services). In some embodiments, smart wheelchair 240-5 can be configured to generate data (e.g., a record of the content of the interaction between the intelligent agent and the patient) by sensing interaction data through a built-in camera / sensor.

[0105] The sensing device 250 may be configured to collect sensing information related to the environment in which it is located. In some embodiments, the sensing device 250 may include a sensing device in the physical hospital 110. For example, the sensing device 250 may include an image sensor 250-1, a sound sensor 250-2, a temperature sensor, a humidity sensor, and the like.

[0106] The patient terminal 260 may be a terminal device that interacts with the patient 261. In some embodiments, the patient terminal 260 may include a mobile terminal 260-1, an XR device 260-2, a smart wearable device 260-3, etc. The doctor terminal 270 may be a terminal device that interacts with the doctor 271. In some embodiments, the doctor terminal 270 may include a mobile terminal 270-1, an XR device 270-2, etc. In some embodiments, the patient 261 may access user space applications (e.g., patient space applications) through the patient terminal 260, and the doctor 271 may access user space applications (e.g., medical space applications) through the doctor terminal 270. In some embodiments, the patient 261 and the doctor 271 may communicate remotely with each other through the patient terminal 260 and the doctor terminal 270, thereby providing telemedicine services, such as remote outpatient services, remote ward rounds services, remote follow-up services, etc.

[0107] The sensing device 250, the patient terminal 260, and the doctor terminal 270 may be configured as data sources to provide information to the medical service system 200. For example, these devices may transmit the collected data to the processing device 210, and the processing device 210 may provide user services based on the received data.

[0108] It should be noted that the description of the above-mentioned medical service systems 100 and 200 is intended to be illustrative, not to limit the scope of the present application. Many alternatives, modifications and variations will be apparent to those skilled in the art. The features, structures, methods and other features of the exemplary embodiments herein may be combined in various ways to obtain additional and / or optional exemplary embodiments. For example, the medical service system 200 may include one or more additional components, such as terminal devices of other users, public terminal devices of hospitals, etc. As another example, two or more components of the medical service system 200 may be integrated into a single component.

[0109] Figure 3 is a schematic diagram of an exemplary hospital support platform 300 according to some embodiments of the present application.

[0110] like Figure 3As shown, the hospital support platform 300 may include a hardware layer 310 (also referred to as a hardware module), an interface layer 320 (also referred to as an interface module), a data processing layer 330 (also referred to as a data processing module), an application development layer 340 (also referred to as an application development module), and a service layer 350 (also referred to as a service module). It should be understood that the "layers" and "modules" in this application are only used to logically divide the components of the hospital support platform and are not intended to be limiting.

[0111] The hardware layer 310 can be configured to provide a hardware foundation for the interaction between the real world and the digital world, and can include one or more hardware devices related to hospital operations. Exemplary hardware devices can include medical service devices, sensor devices, terminal devices, and basic devices.

[0112] The interface layer 320 may be connected to the hardware layer 310 and the data processing layer 330. The interface layer 320 may be configured to obtain data collected by the hardware devices of the hardware layer 310 and send the data to the data processing layer 330 for storage and / or processing. The interface layer 320 may also be configured to control at least a portion of the hardware devices of the hardware layer 310. In some embodiments, the interface layer 320 may include a hardware interface and a software interface (e.g., a data interface, a control interface).

[0113] The data processing layer 330 may be configured to store and / or process data. The data processing layer 330 may include a processing device, and a plurality of data processing units may be configured on the processing device. The data processing layer 330 may be configured to obtain data from the interface layer 320 and process the data through at least one data processing unit to implement user services related to hospital business.

[0114] The data processing unit may include various preset algorithms for implementing data processing. In some embodiments, the data processing layer 330 may include a processing device (e.g., Figure 2 The data processing unit may be configured on the processing device. In some embodiments, the data processing unit may include an XR unit configured to process data using XR technology to implement XR services, an AI unit (e.g., an intelligent unit) configured to process data using AI technology to implement AI services, a digital twin unit configured to process data using digital twin technology to implement digital twin services, a data circulation unit configured to process data using data circulation technology (e.g., blockchain technology, data privacy computing technology) to implement data circulation services, etc.

[0115] In some embodiments, the data processing layer 330 may also include a data center configured to store data. In some embodiments, the data center may adopt a lake warehouse integrated architecture, which may include a data lake and a data warehouse. The data lake can be used to store large amounts of data in a tamper-proof manner. The data warehouse can be used to store index data corresponding to the data in the data lake. The data stored in the data lake may include native (or original) data collected by hardware devices, derived data generated based on native data, and the like. In some embodiments, the data in the data lake can be processed by a processing device (e.g., processing device 210).

[0116] The application development layer 340 may be configured to support application development, publishing, subscription, etc. The application development layer 340 is also referred to as the ecosystem layer. In some embodiments, the application development layer 340 may be configured to provide an open interface for application developers to access or call at least a portion of the data processing unit and use at least a portion of the data processing unit to develop applications. Figure 3 As shown, the application development layer 340 can provide development toolkits, application markets, multi-tenant operation platforms, cloud official websites, workspaces, and other supporting toolkits to assist developers in their work.

[0117] The service layer 350 may be configured so that users related to hospital business can access user services related to hospital business through user space applications.

[0118] The present application provides a hospital support platform designed for the comprehensive management of various resources within the hospital, including hardware resources, software resources, and data resources. In certain embodiments, the platform further integrates a data processing unit capable of supporting advanced technologies, such as artificial intelligence, XR, digital twins, and blockchain. These advanced technologies are used to improve the service efficiency and quality of the healthcare industry. For example, artificial intelligence technology enables the autonomous evolution and continuous optimization of hospital operations, while XR and digital twin technologies facilitate the creation and maintenance of virtual hospitals. The virtual hospital can interact with users to provide an immersive and novel service experience. In addition, the platform also includes an application development layer for granting access to these advanced technologies to third-party developers in the healthcare industry. This access promotes an open ecosystem, promotes the development and innovation of applications, and thus promotes the advancement of medical services.

[0119] Figure 4 4 is a block diagram of an exemplary medical consultation system 400 according to some embodiments of this specification. Figure 4 As shown, the medical consultation system 400 may include a registration module 410, a waiting module 420, a consultation module 430 and / or a post-consultation service module 440. In some embodiments, Figure 4One or more modules shown in FIG. 2 may be implemented by the processing device 210 .

[0120] The registration module 410 can be used to perform operations related to the registration link and / or provide services related to the registration link, such as obtaining / establishing electronic health records, intelligent registration services, route planning and route guidance, etc. For a detailed description of the registration module 410, please refer to the description related to the registration link 510 below.

[0121] The waiting module 420 can be used to perform operations related to the waiting phase and / or provide services related to the waiting phase, such as pre-consultation services, force feedback / temperature feedback, etc. For a detailed description of the waiting module 420, please refer to the description related to the waiting phase 520 below.

[0122] The consultation module 430 can be used to perform relevant operations of the consultation link and / or provide relevant services of the consultation link, such as preview of the clinic on the same day, display of medical data, provision of consultation advice, generation of diagnosis records, remote accompanying services, etc. For a detailed description of the consultation module 430, please refer to the description related to the consultation link 530 below.

[0123] The post-diagnosis service module 440 can be used to perform relevant operations of the post-diagnosis link and / or provide relevant services of the post-diagnosis link, for example, providing medication services, providing examination services, health monitoring, etc. For a detailed description of the post-diagnosis service module 440, please refer to the description related to the post-diagnosis link 540 below.

[0124] In some embodiments, the registration module 410, the waiting module 420, the consultation module 430 and / or the post-diagnosis service module 440 can be implemented on the same or different processing devices. In some embodiments, the medical consultation system 400 can include one or more other modules and / or omit one or more modules described above. In some embodiments, a module can be split into multiple modules, and multiple modules can also be combined into one module.

[0125] Figure 5 FIG. 5 is a schematic diagram of an exemplary medical consultation process 500 according to some embodiments of the present specification. Figure 5 As shown, the medical consultation process 500 includes multiple links such as registration link 510, waiting link 520, consultation link 530 and post-consultation link 540. In some embodiments, the processing device 210 and / or the medical consultation system 400 can perform operations involved in each link of the medical consultation process 500.

[0126] The patient can make an appointment with a doctor in the registration process 510. The relevant operations of the registration process 510 can be performed by the registration module 410. Figure 5As shown, the related operations of the registration link 510 include obtaining / establishing electronic health records, intelligent registration, route planning and route guidance, etc. In some embodiments, the patient can initiate a registration request through a third terminal (also referred to as a patient terminal) or a registration terminal of the hospital. After receiving the registration request, the registration module 410 can perform one or more operations corresponding to the registration link 510. In some embodiments, the patient can initiate a registration request through a third terminal or a registration terminal at the hospital site, or initiate a registration request through a third terminal at other places outside the hospital. In some embodiments, a patient space application is installed on the third terminal, and the patient can initiate a registration request through the patient space application.

[0127] In some embodiments, after receiving the patient's registration request, the registration module 410 can obtain or create the patient's electronic health record. For example, if the patient's electronic health record is already stored in the hospital's storage device, the registration module 410 can obtain the patient's electronic health record, otherwise the registration module 410 can create the patient's electronic health record.

[0128] Electronic health records are electronic files that record various types of patient data. Patient data includes one or more of basic patient information, genetic information, medical history information, health management information, etc. Basic patient information may include the patient's identity ID (ID number, medical insurance card number, etc.), name, age, gender, height, weight, and occupation. Genetic information includes family medical history information and genetic testing information. Medical record information includes information generated from the patient's historical medical visits, such as chief complaints, registration records, pre-consultation records, diagnosis records, hospitalization records, surgical records, pathological test results, imaging data, etc. Health management information is related to behavioral habits that affect health (such as living habits that are beneficial and detrimental to health). In some embodiments, the electronic health record includes a three-dimensional model of the patient, a three-dimensional model of multiple organs and / or tissues of the patient, etc., which are used to intuitively display the patient's health / medical condition. In some embodiments, health management information may also include health monitoring information of the patient. For a detailed description of health monitoring information, see Fig.17 and its related description.

[0129] In some embodiments, the information recorded in the electronic health record is multimodal. For example, the electronic health record contains information in various forms such as text, pictures, graphics, and voice. For another example, the electronic health record may include various types of information related to multiple dimensions and / or multiple aspects, such as basic information, physical examination data, imaging data, etc. In some embodiments, the electronic health record can be updated based on the progress of the patient's medical treatment process. For example, when the patient registers, the electronic health record may include the patient's basic information; after the patient completes the registration, the electronic health record may be updated to further include the patient's chief complaint and registration record; after the patient completes the visit in the clinic, the electronic health record may be updated to further include the diagnosis record; when the patient is at home, the electronic health record may be updated to further include the patient's health monitoring information.

[0130] In some embodiments, the registration module 410 can obtain patient data and establish an electronic health record of the patient based on the patient data. In some embodiments, the registration module 410 can determine at least part of the patient data (such as basic information) based on the patient input. Patient input includes text input, voice input, etc. Patient input can be obtained from a third terminal or a registration terminal. In some embodiments, the registration module 410 can obtain at least part of the patient data by reading data stored in a storage device, calling relevant interfaces and / or other methods. For example, the registration module 410 can obtain the patient's medical record information and electronic physical examination report from the hospital and physical examination center databases respectively. For another example, the registration module 410 can obtain the patient's exercise information for the past month from a smart terminal authorized by the patient (such as a mobile phone, smart watch).

[0131] In some embodiments, before obtaining patient data, the registration module 410 may first use a third terminal or a registration terminal to authenticate the patient and obtain the patient's authorization. Identity authentication may be performed through biometric recognition (such as face recognition, fingerprint recognition, palm print recognition, voice print recognition, and behavior recognition, etc.), identity document chip recognition (for example, identity card chip recognition, medical card chip recognition), and / or password verification.

[0132] In some embodiments, after receiving a registration request from a patient, the registration module 410 can provide the patient with an intelligent registration service, which is used to match the patient with a registration department, a registration doctor, and a registration time period.

[0133] When the patient has a clear registration department, the corresponding registration department can be directly selected; when the patient does not have a clear registration department, the registration module 410 can provide the patient with intelligent registration services based on the patient's chief complaint, the examination report uploaded by the patient and / or the historical patient information stored in the storage device to determine the recommended department that matches the patient's medical needs. After the patient determines the registration department based on the recommended department, the registration module 410 can generate a registration link and send it to the patient. After the patient clicks the registration link, he can further select the registration doctor and registration time period, or directly provide the patient with a registration recommendation plan on the original interface.

[0134] When the patient has a clear registered doctor, the corresponding registered doctor can be directly selected; when the patient does not have a clear registered doctor, the registration module 410 can recommend a registered doctor to the patient based on the patient's chief complaint, and / or display the doctor information of the registered department to the patient to help the patient choose a registered doctor.

[0135] When the patient has a clear registration time, the corresponding registration time period can be directly selected; when the patient does not have a clear registration time, the registration module 410 can recommend a registration time period with fewer appointments to the patient based on the appointment information of the registered doctor. In some embodiments, when the patient registers on-site at the hospital, the registration module 410 can recommend the nearest registration time period for the patient to sign in on time based on the patient's current location and the location of the registration department. In some embodiments, when the patient selects multiple registration departments and / or registered doctors, the registration module 410 can recommend multiple registration departments and / or registered doctors based on the appointment records, treatment efficiency, department location and other information of the registration departments and / or registered doctors, The corresponding registration time periods are recommended to the patient, so that the patient can complete the medical treatment services of multiple registration departments and / or registered doctors in the shortest time.

[0136] In some embodiments, after the patient determines the doctor and registration time period, the registration module 410 can send a check-in credential (such as a QR code or a key) and appointment information to the patient, and update the patient's electronic health record based on the registration record.

[0137] In some embodiments, the patient can make a remote registration through a third terminal at other places outside the hospital. After registration, the registration module 410 can send inquiry information about the consultation method to the third terminal. For example, the inquiry information is used to ask the patient whether to go to the hospital for consultation or to visit remotely. When the patient chooses to go to the hospital for consultation, the registration module 410 can recommend a suitable departure time to the patient through the third terminal based on the appointment information, the patient's current location and the location information of the consulting room. When the patient chooses to visit remotely, the registration module 410 can remind the patient before the appointment time through the third terminal.

[0138] In some embodiments, during the registration process, the processing device 210 (e.g., an agent configured on the processing device) can actively ask the patient whether he needs to receive telemedicine consultation services. When the patient chooses to receive telemedicine consultation services, the registration module 410 can recommend a suitable consultation time to the patient through the third terminal based on the registration information.

[0139] In some embodiments, the registration module 410 can use the third terminal or the registration terminal to conduct a first inquiry (also referred to as a registration inquiry) to the patient to determine the patient's registered doctor. For more information on determining the registered doctor, see Figure 6 and its related description.

[0140] The route planning service is used to provide patients with route suggestions for going to the registered doctor's office. Specifically, the registration module 410 can generate a planned route to the office based on the patient's current location and the location information of the registered doctor's office. In some embodiments, when the patient registers on-site at the hospital, the registration module 410 can generate a planned route. In some embodiments, when the patient registers remotely and arrives at the hospital on the appointment date, the registration module 410 can generate a planned route. For more information about route planning, see Figure 7 and its related description.

[0141] The path guidance service is used to guide the patient to the registered doctor's office based on the planned path. In some embodiments, the registration module 410 can display guidance information to the patient through the patient's third terminal (such as an XR device). For more description of path guidance, please refer to Figure 7 and its related description.

[0142] In some embodiments, the registration module 410 can provide relevant services for the registration process through the hospital's registration terminal. The hospital's registration terminal is a terminal set up on site at the hospital, and may include one or more of a display screen, an input device (such as a keyboard), a sound output device, a sound sensor, and an XR device. Exemplarily, after the patient arrives at the hospital's registration terminal, the registration module 410 can interact with the patient through the display screen, sound output device, sound sensor, etc. on the registration terminal to provide the patient with intelligent registration services, path planning services, and other services. For another example, after the patient arrives at the hospital's registration terminal, he can wear the XR device provided by the registration terminal, and the registration module 410 can provide the patient with intelligent registration, path planning, path guidance, and other services through the XR device. After the patient completes the visit, the XR device can be returned to the hospital.

[0143] In some embodiments, the registration module 410 can provide services related to the registration process through the patient's third terminal. Specifically, the registration module 410 can interact with the patient through the patient's third terminal to provide the patient with services such as intelligent registration, route planning, and route guidance. For example, the registration module 410 can provide the patient with remote intelligent registration services through the patient's third terminal; after the patient arrives at the hospital on the appointment date, the registration module 410 can provide the patient with route planning services and route guidance services through the patient's third terminal.

[0144] In some embodiments, the registration module 410 can receive monitoring data collected by one or more monitoring devices (e.g., infrared monitoring) in the registration hall, and monitor the state of the registration hall based on the monitoring data. For example, the registration module 410 can monitor the congestion level of the registration hall and monitor whether a dangerous event has occurred in the registration hall.

[0145] The patient can wait for and prepare for the visit in the waiting phase 520. In the waiting phase 520, the patient can be located at the hospital site (such as in the waiting area at the door of the registered doctor's office), or can be located at other places outside the hospital (such as at the patient's home). The relevant operations of the waiting phase 520 can be performed by the waiting module 420. Figure 5 As shown, the relevant operations of the waiting stage 520 include pre-consultation, force feedback / temperature feedback, etc.

[0146] The pre-consultation service is used to conduct preliminary inquiries to the patient to obtain relevant information before the patient enters the consulting room for formal consultation. Specifically, when the patient is waiting for consultation, the waiting module 420 can conduct pre-consultation inquiries with the patient through the patient's third terminal or the waiting terminal configured in the waiting area to reduce the patient's anxiety while waiting, and generate a pre-consultation record for the doctor's reference to improve the doctor's consultation efficiency.

[0147] Force feedback / temperature feedback is used to provide comforting feedback to patients. Specifically, when a patient is waiting for a consultation, after the waiting module 420 detects the patient's nervousness, fear, anxiety and other emotions, force feedback and / or temperature feedback can be applied to the patient through the patient's wearable device, so that the patient can feel actions such as handshakes and hugs, thereby comforting the patient's negative emotions.

[0148] For more information about pre-diagnosis, force feedback, and temperature feedback, see Figure 8 and its related description.

[0149] In some embodiments, the waiting module 420 can provide relevant services for the waiting link through the hospital's waiting terminal. Similar to the registration terminal, the hospital's waiting terminal is a terminal set up on-site at the hospital, and may include one or more of a display screen, a sound output device, a sound sensor, an XR device, and a wearable device. For example, after the patient arrives at the waiting area of ​​the registration department, the waiting module 420 can interact with the patient through the waiting terminal set up in the waiting area to provide the patient with pre-consultation services. For another example, after the patient arrives at the waiting area of ​​the consulting room, he can wear the wearable device provided by the waiting terminal, and the waiting module 420 can provide the patient with force feedback / temperature feedback services through the wearable device.

[0150] In some embodiments, the waiting module 420 can provide relevant services in the waiting process through the hospital's XR device. For example, after the patient wears the XR device of the registration terminal and arrives at the waiting area, the waiting module 420 can provide the patient with pre-consultation services through the XR device. In some embodiments, the waiting terminal includes an XR device, and the waiting module 420 can provide the patient with pre-consultation services through the XR device of the waiting terminal.

[0151] In some embodiments, the waiting module 420 can receive monitoring data collected by one or more monitoring devices (e.g., infrared monitoring) in the waiting area, and monitor the status of the waiting area based on the monitoring data. For example, the waiting module 420 can monitor the waiting time of waiting patients and monitor whether a dangerous event has occurred in the waiting area.

[0152] In some embodiments, the waiting module 420 can provide services related to the waiting link through the patient's third terminal. For example, the waiting module 420 can remotely provide pre-consultation services to patients waiting for online consultation through the patient's third terminal. For another example, the waiting module 420 can provide pre-consultation services to patients waiting in the hospital through the patient's third terminal.

[0153] The patient can communicate with the registered doctor in the consultation link 530 to receive medical consultation services. In the consultation link 530, the patient can be located in the clinic to receive on-site consultation services, or can be located in other places outside the hospital (such as at the patient's home) to receive remote consultation services. Specifically, the relevant operations of the consultation link 530 can be performed by the consultation module 430. Figure 5 As shown, the related operations of the consultation link 530 include preview of the outpatient clinic on the same day, display of medical data, provision of consultation advice, generation of diagnosis records, remote accompanying services, etc.

[0154] The outpatient clinic preview of the day is used to show the doctor relevant information about the patients waiting to be seen on the same day. For example, the consultation module 430 can show the doctor the categories of patients who are registered or waiting for the day before the doctor starts the consultation. Among them, the categories of patients may include first-visit patients, follow-up patients, etc. When there are follow-up patients, the consultation module 430 can show the doctor the changes in the condition of the follow-up patients based on the information of the follow-up patients. In some embodiments, the doctor can browse the outpatient clinic preview of the day through the first terminal in the consulting room or his own second terminal (such as an XR device). In some embodiments, the doctor's second terminal is installed with a medical space application, and the doctor can obtain the outpatient clinic preview of the day through the medical space application. Optionally, before the doctor needs to provide outpatient services, the medical space application can send a prompt to the doctor to remind the doctor to check the outpatient clinic preview of the day.

[0155] Medical data display is used to display the patient's medical data, such as electronic health records, to doctors, patients and / or remote patients. Specifically, the consultation module 430 can synchronously display the patient's medical data to doctors, patients and / or remote patients through at least one terminal, and update the display method and / or display content of the electronic health record according to the doctor's, patient's and / or remote patients' interactive operations on the medical data, so that doctors, patients and / or remote patients can communicate. For more descriptions of medical data display, see Fig.12 and its related description.

[0156] Providing consultation advice refers to providing the doctor with consultation advice that can be used as a reference during the consultation process. Exemplary consultation advice may include supplementary inquiry advice, physical examination advice, prescription advice, treatment plan advice, etc. Specifically, the consultation module 430 can generate consultation advice for the doctor based on the perception information collected by the perception device during the consultation process, so that the doctor can adjust the diagnosis method, diagnosis results and prescription to improve the efficiency and accuracy of the consultation. For more descriptions on providing consultation advice, please refer to the relevant description of subprocess 1310, which will not be repeated here.

[0157] The diagnostic record generation is used to record the patient's condition information diagnosed by the doctor, the doctor's medical advice, etc., to assist the doctor in generating the patient's diagnostic record. Specifically, the consultation module 430 can generate an initial diagnostic record based on the sensory information collected by the sensory device during the consultation process, and generate a target diagnostic record based on the doctor's feedback information on the initial diagnostic record, thereby reducing the doctor's paperwork and improving the efficiency and accuracy of diagnostic record generation. For more descriptions of diagnostic record generation, see subprocess 1320 and Fig.15 The relevant description will not be repeated here.

[0158] The remote medical escort service is used to provide immersive medical escort services for patients and remote medical escorts. For example, the consultation module 430 can display a virtual medical room space that simulates a real medical room, a real-time image of the doctor in the medical room, and a real-time image of the patient to the remote medical escort. For another example, the consultation module 430 can display the real-time image of the remote medical escort through the first terminal in the medical room. For another example, the consultation module 430 can display the real-time image of the remote medical escort to the doctor and the patient respectively through the doctor's second device and the patient's third device. For another example, the consultation module 430 can obtain force feedback and / or temperature feedback from the remote medical escort through a wearable device worn by the remote medical escort, and apply corresponding force feedback and / or temperature feedback to the patient through the wearable device worn by the patient. For more descriptions of the remote medical escort service, please refer to subprocesses 1330 and Fig.16 The relevant description will not be repeated here.

[0159] In some embodiments, the consultation module 430 may also provide other services related to the consultation process 530, such as assisting doctors in completing workflow-related tasks such as calling numbers and approving requests for accompanying patients.

[0160] In some embodiments, the consultation module 430 can provide consultation-related services to doctors and patients through the first terminal in the consulting room. The first terminal is a terminal set up at the consulting room, and may include one or more of a display screen, a sound output device, a sound sensor, an XR device, and a wearable device. For example, after the patient enters the consulting room, the consultation module 430 can display electronic health records, consultation recommendations, pre-consultation records, diagnosis records, real-time images of remote accompanying persons, etc. to the doctor and / or patient through the large screen of the first terminal. For another example, the patient can wear a wearable device of the first terminal and receive force feedback from the remote accompanying person through the wearable device. For another example, the doctor and the patient can wear an XR device of the first terminal, and the consultation module 430 can display various types of information to the doctor and / or the patient through the XR device of the first terminal. In some embodiments, the XR device worn by the patient may also be an XR device configured on the registration terminal. For more descriptions of the content displayed on the first terminal, see Fig.14 and its related description.

[0161] In some embodiments, the consultation module 430 can provide the doctor and the patient with relevant services of the consultation link through the doctor's second terminal and the patient's third terminal. For example, the consultation module 430 can display electronic health records, consultation recommendations, diagnosis records, real-time images of remote accompanying persons, etc. to the doctor and the patient through the doctor's second terminal and the patient's third terminal respectively. At least part of the content presented by the second terminal and the third terminal is synchronized. For example, the electronic health records presented on the second terminal and the third terminal are synchronized. When the doctor or the patient updates the display method or display content of the electronic health record through their respective terminals, the electronic health record on the other terminal will also be updated accordingly. In some embodiments, the consultation module 430 can further provide the remote accompanying person with relevant services of the consultation link through the remote accompanying person's fourth terminal (such as an XR terminal). The fourth terminal may also be referred to as an accompanying person terminal.

[0162] When the patient finishes the consultation and communication with the doctor, the patient may enter the post-consultation link 540. The relevant operations of the post-consultation link 540 may be performed by the post-consultation service module 440. Figure 5 As shown, the related operations of the post-diagnosis link 540 include providing medication pickup services, providing examination services, health monitoring, etc.

[0163] The medication pickup service is used to assist patients in getting the medications prescribed by doctors. For example, the medication pickup service is used to assist patients in paying medication fees, making appointments with pharmacies to pick up medications, and guiding patients to pharmacies. The examination service is used to assist patients in receiving examinations requested by doctors. For example, the examination service is used to assist patients in paying examination fees, making appointments with examination departments, and guiding patients to examination departments.

[0164] In some embodiments, in response to detecting that the consultation process is over, the post-consultation service module 440 can determine the target service to be provided to the patient after the consultation process, and make an appointment for the patient with the target business department that provides the target service. The post-consultation service module 440 can detect whether the consultation process is over in a variety of ways. For example, when it is detected that the doctor has submitted the patient's target diagnosis record or the doctor calls the next patient, the post-consultation service module 440 can determine that the patient's consultation process is over.

[0165] In some embodiments, the post-diagnosis service module 440 can determine the target service required by the patient based on the target treatment prescription in the target diagnosis record. For example, the target service can be a medication pickup service, and the target business department can be a pharmacy. When the patient needs medication pickup service, the post-diagnosis service module 440 can send the patient's medication prescription information to the pharmacy and make an appointment to pick up the medication. For another example, the target service can be an examination service, and the target business department can be an examination department. When the patient needs an examination service, the post-diagnosis service module 440 can send the patient's examination prescription information to the examination department and make an appointment for the examination. Optionally, after making an appointment with the target business department, the post-diagnosis service module 440 can send the appointment information to the patient's third terminal. In some embodiments, after detecting that the consultation process is over, the post-diagnosis service module 440 can first send a fee payment reminder to the patient's third terminal; after the patient pays the fee for the target service, the post-diagnosis service module 440 makes an appointment with the target business department.

[0166] In some embodiments, the post-diagnosis service module 440 can provide the patient with a path planning service and a path guidance service to the target business department based on the patient's current location and the location of the target business department. The path planning service and path guidance service provided to the patient to the target business department are similar to the path planning service and path guidance service of the registration link 510, and will not be repeated here.

[0167] The health monitoring service is used to continuously monitor the patient's health status after the consultation process is completed. For a detailed description of the health monitoring service, please refer to Fig.17 And the related description will not be repeated here.

[0168] In some embodiments, one or more operations in the medical treatment process 500 can be performed by an intelligent agent corresponding to the medical treatment service / process. An intelligent agent is a computing entity that can learn and evolve autonomously, and can perceive and analyze data to perform specific tasks and / or achieve specific goals. Through AI technologies such as reinforcement learning and deep learning, the intelligent agent can continuously learn and evolve itself in interaction with the environment. In addition, the intelligent agent can use big data technology to collect and analyze massive amounts of information, and mine patterns and learn laws from the data to optimize the decision-making process, so that it can identify environmental changes, respond quickly, and make reasonable judgments in uncertain or dynamic environments.

[0169] In some embodiments, different links of the medical consultation process 500 may share a common agent, or have corresponding agents respectively. For example, the relevant operations of the registration link 510, the waiting link 520, the consultation link 530 and the post-consultation link 540 can be performed by the registration agent, the pre-consultation agent, the consultation agent and the post-consultation agent respectively. For another example, the intelligent registration service in the registration link 510 can be performed by the registration agent, the pre-consultation service in the waiting link 520 can be performed by the pre-consultation agent, and the path planning and path guidance in the registration link 510 and the post-consultation link 540 can be performed by the travel agent. For another example, all links or operations in the medical consultation process 500 can be performed by the same medical consultation service agent.

[0170] In some embodiments, different departments and / or different diseases, and different hospitals may correspond to different agents. The medical services corresponding to a specific department and / or a specific disease can be implemented by the agent corresponding to the department and / or the disease. In some embodiments, the agent can refer to the knowledge data of the corresponding department and / or disease during operation, such as a dictionary, a knowledge graph, a template, etc. Optionally, the knowledge data can be set by a hospital manager (such as a department director). In some embodiments, multiple agents can collaborate with each other and share information through network communication to jointly complete complex tasks.

[0171] Figure 6 It is a schematic diagram of an exemplary process for determining a doctor to be registered with a patient according to some embodiments of the present specification. Figure 6 The process 600 shown can be executed in the registration link 510.

[0172] Step 610, obtaining the patient's chief complaint.

[0173] The patient's chief complaint is the patient's self-description of the condition and / or symptoms. For example, the patient's chief complaint may include an introduction to the site of the disease, the time of disease development, the intensity of symptoms, the frequency of symptoms, related life events, the purpose of medical treatment, etc. In some embodiments, the form of the patient's chief complaint may include but is not limited to text, voice, pictures, gestures, etc. For example, the patient's chief complaint may include the patient's text description, voice description, and sign language description of the condition. For another example, the patient's chief complaint may include a picture of the patient's lesion. For another example, the patient's chief complaint may include the patient's description of the pain level.

[0174] In some embodiments, the registration module 410 can obtain the patient's chief complaint through the patient's third terminal or the hospital's registration terminal. Exemplarily, the registration module 410 can obtain the patient's chief complaint in text form through the keyboard (physical keyboard, screen keyboard and / or virtual keyboard) of the third terminal or the registration terminal. For another example, the registration module 410 can obtain the patient's chief complaint in voice form through the sound sensor of the third terminal or the registration terminal. In another example, the registration module 410 can obtain the patient's chief complaint through the image sensor of the third terminal or the registration terminal, and recognize the patient's gesture in the video, and further determine the patient's chief complaint in sign language form based on the patient's gesture. In another example, the registration module 410 can take a picture of the patient's lesion through the image sensor of the third terminal or the registration terminal.

[0175] Step 620, determining at least one candidate department based on the patient's complaint.

[0176] Candidate departments are departments that are preliminarily determined based on the patient's condition and / or symptoms.

[0177] Specifically, the registration module 410 can extract the first keyword of the patient's complaint. The first keyword can include key words that summarize the condition and / or symptoms, such as the diseased part, symptoms, time, intensity, frequency, etc. Exemplarily, the registration module 410 can extract the first keyword in the patient's complaint in text form through a keyword extraction algorithm. The keyword extraction algorithm may include but is not limited to the TF / IDF algorithm, the Topic Model algorithm, the textrank algorithm, the rake algorithm, etc. For another example, the registration module 410 can identify the first keyword in the patient's complaint in voice form through speech recognition technology. The speech recognition technology may include but is not limited to automatic speech recognition (ASR) technology, computer speech recognition technology, speech to text recognition (STT) technology, etc. For another example, the registration module 410 can identify the part, symptoms, etc. in the patient's complaint in the form of a picture through image recognition technology as the first keyword. The image recognition technology may include but is not limited to image feature extraction technology, target detection technology, target recognition technology, etc. For another example, the registration module 410 can identify the first keyword in the patient's complaint in gesture form through gesture recognition technology. Gesture recognition technology may include gesture trajectory recognition technology, gesture posture recognition technology, gesture analysis technology, etc.

[0178] Further, the registration module 410 can determine at least one candidate department based on the first keyword. For example, the registration module 410 can determine at least one candidate department by retrieving a preset keyword-department comparison table. The preset keyword-department comparison table can be generated in advance based on medical knowledge, doctor experience, historical registration records and other information, which records various types of chief complaint keywords and their corresponding departments. For another example, the registration module 410 can input the first keyword into a department determination model to determine at least one candidate department. The department determination model is a pre-trained machine learning model that can process the model input to output a recommended candidate department. In some embodiments, the preset keyword-department comparison table and / or the department determination model can be learned from historical registration records by the intelligent agent corresponding to the medical consultation process.

[0179] Step 630: Based on at least one candidate department, control the third terminal or the registration terminal to conduct a first inquiry to the patient.

[0180] The first inquiry is used to further clarify the patient's medical needs, so as to determine a matching registration department and / or registration doctor for the patient. The first inquiry may also be referred to as a registration inquiry. In some embodiments, the first inquiry may include multiple rounds of dialogue. Each round of dialogue may include one inquiry and one patient answer.

[0181] In some embodiments, the registration module 410 may determine the first inquiry content of the first inquiry based on the patient's main complaint and at least one candidate department.

[0182] The first inquiry content may also be referred to as the registration inquiry content. The first inquiry content includes at least the content of the first round of inquiries. Exemplarily, the registration module 410 may input the patient's chief complaint and at least one candidate department into the first inquiry model, and the first inquiry model outputs the content of the first round of inquiries. In some embodiments, the first inquiry model may include but is not limited to a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a long short-term memory network (LSTM) model, a BERT model, a ChatGPT model, etc. In some embodiments, the first inquiry model may be acquired based on the first sample training set training. Among them, the first training sample set may include multiple first training samples and multiple first training labels corresponding thereto. The first training sample may include a sample patient's chief complaint and a sample candidate department, and the first training label may include the content of the sample's first round of inquiries. The first training sample and the first training label may be based on historical registration records and / or determined manually by the user. In some embodiments, the first training sample can be input into the initial model, the value of the loss function can be determined based on the data output by the initial first query model and the first training label, and the parameters of the initial model can be iteratively updated based on the value of the loss function until the value of the loss function reaches a preset value or the number of iterations reaches a preset number, thereby obtaining a trained first query model.

[0183] For another example, the registration module 410 can obtain the keywords corresponding to each candidate department based on the preset keyword-department comparison table, and determine the difference words between the keywords corresponding to any two candidate departments, and then determine the content of the first round of inquiries based on the difference words. For example, if the keywords corresponding to orthopedics are "local pain, fever" and the keywords corresponding to vascular department are "leg pain, coldness", the difference words "fever" and "coldness" between the keywords corresponding to orthopedics and vascular department can be determined, and the content of the first round of inquiries "Is the leg hot or cold?" can be determined based on the difference words.

[0184] In some embodiments, for subsequent rounds of inquiries in the first inquiry, the registration module 410 can use the second inquiry model to determine the corresponding first inquiry content based on the patient's chief complaint, at least one candidate department, and historical rounds of dialogue. The second inquiry model may include a CNN model, an RNN model, an LSTM model, a BERT model, a ChatGPT model, and the like. In some embodiments, the second inquiry model can be obtained based on the second sample training set training. Among them, the second training sample set may include multiple second training samples and multiple second training labels corresponding thereto. The second training sample may include a sample patient's chief complaint, a sample candidate department, and a sample historical round of dialogue, and the second training label may include the content of the sample's current round of inquiry. The second training sample and the second training label may be based on historical registration records and / or determined manually by the user. The training process of the second inquiry model is similar to that of the first inquiry model, and will not be repeated here.

[0185] In some embodiments, the registration module 410 can use the third terminal or the registration terminal to make the first inquiry through text display, voice output, etc. For example, the first inquiry content is displayed based on the screen of the third terminal or the registration terminal. For another example, the first inquiry content is played based on the microphone of the third terminal or the registration terminal.

[0186] In some embodiments, the registration module 410 can control the third terminal or the registration terminal to display the first virtual character to conduct the first inquiry. The first virtual character is a digital character image with specific appearance characteristics, tone characteristics, etc., which can communicate with the patient and assist the patient in registration. Specifically, the registration module 410 can display the first virtual character through the screen of the third terminal or the registration terminal, and play the first inquiry content through the microphone of the third terminal or the registration terminal. At the same time, the first virtual character can simulate the expression, action, etc. of human speech, giving the patient a real communication experience.

[0187] In some embodiments, the first virtual character has a preset appearance feature. For example, the first virtual character may be a nurse. In some embodiments, the registration module 410 may determine the appearance feature of the first virtual character based on the basic information of the patient. The basic information of the patient may include the age, gender, occupation, etc. of the patient. Specifically, the registration module 410 may determine the appearance feature of the first virtual character based on the basic information of the patient, and generate the corresponding first virtual character based on the appearance feature. As an example only, the appearance feature may include body shape feature, skin feature, appearance feature, clothing feature, etc. For example, for a 70-year-old female patient, the first virtual character may be a virtual female nurse with strong affinity. For another example, for a patient whose occupation is a doctor, the first virtual character may be a professional virtual doctor. In some embodiments, the registration module 410 may determine the first virtual character based on the basic information of the patient from a plurality of candidate first virtual characters. In some embodiments, the first virtual character may initiate a first inquiry to the patient based on the sound feature. For a detailed description of the sound feature, please refer to the relevant description of step 820.

[0188] In some embodiments, the third terminal or the registration terminal may include an XR device, and the registration module 410 may control the XR device to present a first virtual character that performs a first inquiry. The XR device may display the first virtual character within the patient's field of view and play the first inquiry content. The patient's field of view may present the real world within the patient's field of view, or present a virtual background. In some embodiments, the registration module 410 may synchronously display the first inquiry content within the patient's field of view through the XR device. Fig.10 Schematic diagram of the process of presenting a virtual character through an XR device according to some embodiments of this specification. Fig.10 As shown, the XR device can display a virtual character 1010 (such as a first virtual character) within the patient's field of view, and synchronously display inquiry content 1020 (for example, a first inquiry content).

[0189] In some embodiments, the registration module 410 can end the first inquiry based on a first preset condition. The first preset condition can be that the number of dialogue rounds reaches a threshold, such as 10 rounds. The first preset condition can be that the patient enters a preset answer or the inquiry content is a preset content. For example, when the patient enters the answer "end" or "no other symptoms", the first preset condition is met. For another example, when the content of the previous round of inquiry is "no other questions, thank you!", the first preset condition is met.

[0190] In some embodiments of this specification, a first virtual character is used to conduct a first inquiry to a patient, and the patient's sense of interaction is enhanced through the anthropomorphic first virtual character without increasing labor costs, thereby improving the quality and efficiency of the registration service. In some embodiments of this specification, a machine learning model (such as a first inquiry model and / or a second inquiry model) is used to determine the first inquiry content of the first inquiry. Compared with using a pre-set inquiry content, the method in this specification can improve the efficiency and quality of inquiries, and improve the accuracy of the subsequently determined registration department and / or registration doctor.

[0191] Step 640, based on the first data collected by the third terminal or the registration terminal in the first inquiry, determine the doctor that the patient wants to register with.

[0192] The first data may include various types of data input by the patient via the third terminal or the registration terminal, such as voice data, text data, and image input. In some embodiments, the third terminal or the registration terminal is equipped with a sound sensor, and the first data can be collected by the sound sensor. Specifically, the registration module 410 can determine the confidence of each candidate department based on the first data, the patient's chief complaint, and at least one candidate department. The confidence of the candidate department is a numerical value that evaluates the degree of match between the candidate department and the patient's condition / symptoms. The higher the confidence of the candidate department, the higher the degree of match with the patient's condition / symptoms.

[0193] For example, for each candidate department, the registration module 410 can determine the keyword of the candidate department based on a preset keyword-department comparison table; the registration module 410 can further determine the confidence of the candidate department based on the number and frequency by counting the number and frequency of keywords of the candidate department appearing in the transcribed text of the first data and the patient's chief complaint. For another example, the trained department determination model can be used to process the transcribed text of the first data, the patient's chief complaint, and at least one candidate department, and the department determination model outputs the confidence of each candidate department.

[0194] Further, the registration module 410 can determine at least one recommended department based on the confidence of at least one candidate department, so that the patient can select at least one registration department based on the at least one recommended department. For example, the registration module 410 can determine the candidate department with a confidence greater than a preset threshold as a recommended department, and control the third terminal or the registration terminal to display the recommended department, and the patient determines the registration department from them. After the registration department is determined, the third terminal or the registration terminal can display the registration link corresponding to the registration department, and the patient can further select the registration doctor and registration time by clicking the registration link through the terminal.

[0195] In some embodiments, the registration module 410 can display the basic information and available appointment time of each registered doctor corresponding to the registered department to the patient based on the third terminal or the registration terminal. As an example only, the registration module 410 can control the XR device of the third terminal or the registration terminal to display the virtual character of the registered doctor, and the virtual character introduces the basic information and available appointment time of the corresponding registered doctor to the patient. In some embodiments, the registration module 410 can recommend registered doctors and registration time periods to the patient.

[0196] In some embodiments of the present specification, by executing the first inquiry to communicate with the patient, the patient's medical needs are fully understood, thereby improving the accuracy of the registration suggestions provided to the patient, improving the registration efficiency, and improving the patient's interactive experience.

[0197] In some embodiments, process 600 may be executed by an agent corresponding to the medical consultation service / medical consultation process. For example, an intelligent registration service may be provided through a registration agent. The registration agent may understand and process natural language and images, allowing patients to describe symptoms in their own words, accurately assessing the patient's condition and providing high-confidence registration recommendations. At the same time, the registration agent may continuously improve the accuracy and efficiency of the intelligent registration service by learning from knowledge data and historical service data, thereby improving the user experience.

[0198] In some embodiments, the hospital administrator can configure the registration agent, for example, through the management space application installed on the administrator's terminal. The configuration of the registration agent includes the configuration of general parameters, which are related to hardware configuration, concurrent request processing, cache strategy, security settings, model parameters, error handling and logging, performance monitoring and alarm, version management, etc. The configuration of the registration agent also includes the configuration of business parameters. Business parameters are related to optional departments, doctor lists and number sources, and inquiry rounds.

[0199] Figure 7 Schematic diagram of route planning and route guidance services according to some embodiments of the present specification. Route planning and route guidance are related to the registration link 510.

[0200] After the patient has registered with the corresponding doctor, the registration module 410 can generate a planned path based on the patient's current location and the location information of the doctor's corresponding consulting room. The patient's current location is the patient's current location at the moment. The registration module 410 can obtain the patient's current location based on the positioning information of the third terminal or the registration terminal. The planned path is the path that guides the patient to the consulting room. In some embodiments, the registration module 410 can determine the path from the patient's current location to the consulting room based on the real-time three-dimensional map of the hospital. The real-time three-dimensional map of the hospital is a three-dimensional map that reflects the layout and real-time status of the hospital.

[0201] In some embodiments, when a patient has made appointments with multiple doctors and needs to go to multiple clinics, the registration module 410 can determine the order of visiting multiple clinics and the planned path through these clinics based on one or more of the patient's current location, registration records, location information of multiple clinics, and other information. For example, the registration module 410 can determine the estimated waiting time for the patient corresponding to each clinic based on the clinic doctor's daily visit record and the patient's registration record. For a detailed description of determining the estimated waiting time for the patient, please refer to the relevant description of step 810. Furthermore, the registration module 410 can sort the multiple clinics in descending order according to the estimated waiting time, and then determine the planned path of the sorted clinics.

[0202] After the planned path is generated, the registration module 410 can control the patient's third terminal or the registration terminal to present guidance information related to the planned path. The guidance information is information that instructs the patient to move along the planned path. For example, the guidance information may include a guidance gesture, a guidance speech, etc. of a virtual guide character (e.g., the first virtual character). For another example, the guidance information may include arrows, text, etc.

[0203] In some embodiments, the third terminal or the registration terminal includes an XR device, and the registration module 410 enables the XR device to overlay guidance information on the patient's real-world field of view through AR technology or MR technology. That is, when the patient wears the XR device, he can see the real world around him and the guidance information at the same time. The patient's real-world field of view is the real world within the patient's gaze range. The patient's gaze range changes with the patient's head movement, and the corresponding patient's real-world field of view also changes accordingly.

[0204] For example, Figure 7 As shown, when the patient's head is in the direction shown in the figure, the corresponding patient's real-world field of view may include the real hospital scene within the dotted frame, and the registration module 410 may superimpose the virtual guide character on the real-world field of view corresponding to the dotted frame, and guide the patient to move along the planned path through the gestures and words of the virtual guide character. For another example, the registration module 410 may superimpose a virtual arrow on the real-world field of view corresponding to the dotted frame, thereby guiding the patient to move along the planned path. When the patient moves along the planned path, the patient's real-world field of view changes in real time, and the registration module 410 may superimpose the guidance information on the real-world field of view that changes in real time to guide the patient.

[0205] In some embodiments of this specification, providing path planning for patients and providing path guidance services based on path planning can help patients quickly reach the clinic and improve the efficiency of patients' visits. On the other hand, in some embodiments, AR / MR technology is used for path guidance, so that guidance information can be presented to patients in a more accurate manner, improving guidance accuracy without affecting patients' perception of the real world.

[0206] Figure 8 It is a schematic diagram of an exemplary process for providing pre-consultation services according to some embodiments of the present specification. Figure 8 The process 800 shown can be executed in the waiting stage 520 .

[0207] Step 810, determining the second inquiry content of the second inquiry based on the doctor's department.

[0208] The second inquiry may also be referred to as a pre-inquiry inquiry, which is used to conduct preliminary inquiries to the patient before the formal inquiry. The second inquiry may include multiple rounds of inquiries. The second inquiry content may also be referred to as the pre-inquiry inquiry content. The second inquiry content may include the inquiry content of each round of inquiry. Alternatively, the second inquiry content only includes the inquiry content of the first round of inquiry.

[0209] In some embodiments, the waiting module 420 can obtain the pre-consultation record template corresponding to the doctor's department, and determine the second inquiry content based on the pre-consultation record template. For example, the pre-consultation record template records various types of patient information that need to be collected in the pre-consultation, and the waiting module 420 can use each type of information as the content of a round of inquiry. Optionally, the doctor can personalize and modify the pre-consultation record template.

[0210] In some embodiments, the waiting module 420 can obtain the patient's known information (such as electronic health records, chief complaints, etc.), and determine the missing information that has not been collected in the pre-consultation record template by comparing the pre-consultation record template with the known information. For example, if the known information includes the patient's family medical history, the missing information does not need to include the family medical history. For another example, if the patient's chief complaint includes the patient's medical history, the missing information does not need to include the medical history. In some embodiments, the missing information can be determined based on the patient's basic information. For example, for male patients, the omission does not need to include menstrual history and reproductive history. Further, the waiting module 420 can determine the second inquiry content based on the missing information. Specifically, the inquiry content of each round of inquiry includes questions about one or more missing information. For example, the waiting module 420 can compare the patient's electronic health record and the pre-consultation record template, and determine that the allergy history and pain level in the pre-consultation record template are not included in the electronic health record, then the allergy history and pain level are missing information. The waiting module 420 can determine that the second inquiry content includes questions about the allergy history and questions about the pain level. In some embodiments, before conducting the preliminary consultation, the waiting module 420 may first display the electronic health record to the patient through the patient's third terminal, and then determine the second inquiry content after the patient confirms that the electronic health record is correct.

[0211] In some embodiments, the waiting module 420 may use a third inquiry model to determine the second inquiry content based on the doctor's department and the known information of the patient. The third inquiry model may include a CNN model, an RNN model, an LSTM model, a BERT model, a ChatGPT model, etc. In some embodiments, the third inquiry model may include a missing information determination model and a first inquiry content determination model.

[0212] The missing information determination model is used to process the known information of the doctor's department and the patient to output the missing information. In some embodiments, the missing information determination model can be obtained based on the third training sample set training. Among them, the third training sample set can include multiple third training samples and multiple third training labels corresponding thereto. The third training sample can include the known information of the sample patient and the corresponding sample doctor's department, and the third training label can include sample missing information. The third training sample set can be determined based on historical pre-consultation records or manually determined by the user. In some embodiments, the waiting module 420 can determine the sample missing information based on the knowledge database corresponding to the third training sample and the sample doctor's department. Among them, the knowledge database can include the consultation specifications of the corresponding department, for example, symptom description specifications, diagnosis specifications, prescription specifications, medical advice specifications, etc. Specifically, the waiting module 420 can traverse the third training sample based on the knowledge database, and determine that the content not involved in the third training sample in the knowledge database is the sample missing information.

[0213] The first inquiry content determination model is used to output the second inquiry content based on the patient's missing information. The first inquiry content determination model is used to process the missing information to output the second inquiry content of each round of inquiry in the second inquiry or the second inquiry content of the first round of inquiry. The first inquiry content determination model can be acquired based on the fourth training sample set training. Among them, the fourth training sample set may include multiple fourth training samples and multiple fourth training labels corresponding thereto. The fourth training sample may include sample missing information, and the fourth training label may include sample inquiry content. The fourth training sample set may be determined based on historical pre-consultation records or manually determined by the user.

[0214] Step 820: Based on the second inquiry content, control the patient's third terminal to conduct a second inquiry to the patient.

[0215] In some embodiments, after the patient registers, the waiting module 420 can determine the patient's estimated waiting time. For example, the estimated waiting time can be the time difference between the current moment and the patient's registration time period. For another example, the estimated waiting time can be determined based on the doctor's day visit record and the patient's registration record. The patient's registration record may include the patient's appointment time period. The doctor's day visit record is a record that reflects the doctor's day visit situation. As an example only, the waiting module 420 can determine the number of waiting patients who visited the doctor earlier than the patient and the doctor's average visit time for each patient based on the doctor's day visit record and the patient's registration record, and use the product of the number and the average visit time as the estimated waiting time.

[0216] In some embodiments, in response to determining that the estimated waiting time is greater than the first preset time threshold, the waiting module 420 may initiate a second inquiry to the patient through the patient's third terminal or display a suggestion for a second inquiry. For example, if the patient's estimated waiting time of 10 minutes is greater than 5 minutes, the waiting module 420 may initiate a second inquiry to the patient through the patient's third terminal. In some embodiments of the present specification, determining to initiate a second inquiry based on the patient's estimated waiting time can ensure that the pre-consultation time is sufficient, and avoid the doctor calling the patient during the pre-consultation process, resulting in the patient missing the appointment.

[0217] In some embodiments, in response to determining that the estimated waiting time is less than the second preset time threshold, the waiting module 420 may initiate a second inquiry to the patient through the patient's third terminal or display a suggestion for the second inquiry. The second preset time threshold is greater than the first preset time threshold. For example, when it is detected that the time from the current moment to the registration time period is less than 24 hours (i.e., the estimated waiting time is less than 24 hours), the third terminal may display a suggestion for the second inquiry to the patient (such as making a suggestion to the patient through a virtual character), thereby promptly reminding the patient to conduct a pre-consultation.

[0218] In some embodiments, the waiting module 420 may detect that the patient actively initiates a pre-consultation request through a third terminal. In response to the pre-consultation request, the waiting module 420 may initiate a second inquiry to the patient through the patient's third terminal. In some embodiments, a patient space application is installed on the third terminal, and the second inquiry is performed using the patient space application. The patient space application is an interactive entrance between the patient and the third terminal. The patient can obtain various medical services through the patient space application.

[0219] In some embodiments, a second virtual character can be presented through a third terminal, and the second virtual character is used to conduct a second inquiry based on the second inquiry content. The second virtual character is a digital character image with specific appearance characteristics, tone characteristics and other characteristics, and can communicate with the patient to conduct a preliminary consultation on the patient. Specifically, the waiting module 420 can display the second virtual character through the screen of the third terminal, and play the second inquiry content through the sound output device of the third terminal. At the same time, the second virtual character can simulate the expression and movements of human speech to give the patient a real communication experience. For example, the third terminal may include an XR device, and the waiting module 420 may present the second virtual character through the XR device. Just as an example, Fig.10 As shown, the XR device can display a virtual character 1010 (such as a second virtual character) within the patient's field of view, and synchronously display inquiry content 1020 (for example, second inquiry content).

[0220] In some embodiments, the second virtual character has preset appearance features. In some embodiments, the appearance features of the second virtual character can be determined based on the optical image of the doctor with whom the patient is registered. Specifically, the waiting module 420 can extract the appearance features of the doctor (such as body shape information, clothing information) from the optical image of the doctor to be registered, and generate a second virtual character with the same or similar appearance features as the doctor based on the appearance features of the doctor. In some embodiments, the appearance features of the second virtual character can be determined based on the basic information of the patient. For a detailed description of determining the appearance features of the second virtual character based on the basic information, please refer to the relevant description of determining the appearance features of the first virtual character based on the basic information in step 630. In some embodiments, the waiting module 420 can select a suitable virtual character as the second virtual character from the virtual character library based on the appearance features of the doctor and / or the basic information of the patient.

[0221] In some embodiments, as described above, the second inquiry includes multiple rounds of inquiry, and the second inquiry content may include the inquiry content of each round of inquiry in the second inquiry. Fig. 9 The process 900 shown is executed.

[0222] like Fig. 9As shown, for the first round of inquiries, the waiting module 420 can control the third terminal to conduct the first round of inquiries based on the corresponding inquiry content.

[0223] For each current round of inquiries (referred to as the current inquiries for short) except the first round of inquiries, the waiting module 420 can adjust the inquiry content of the current inquiry (referred to as the current inquiry content for short) based on the second data collected before the current inquiry, so that the inquiry content is more in line with the patient's condition. Specifically, the waiting module 420 can determine the semantic information and emotional information of the patient's historical answers based on the second data collected before the current inquiry. The second data may include voice data, image data, text data, and other data collected by a third terminal. For example, the second data can be collected by a sound sensor of the third terminal after detecting the sound. The historical answer is the patient's answer to the historical round of inquiries. For example, if the current inquiry is the third round of inquiry, the historical answer may include the patient's answer to the first round of inquiry and the second round of inquiry.

[0224] The semantic information of the historical answers represents the content of the historical answers. The emotional information of the historical answers represents the patient's emotions when answering (for example, calm, nervous, anxious, afraid, confused, irritable, etc.). The waiting module 420 can perform text transcription, voice content recognition, etc. on the second data to determine the semantic information. The waiting module 420 can analyze the voice content, tone, intonation, speech speed and other features of the second data to determine the emotional information. In some embodiments, the waiting module 420 can first extract the patient's target voice signal from the second data, and then extract semantic information and emotional information from the target voice signal.

[0225] Continue to refer Fig. 9 , the waiting module 420 can adjust the current inquiry content based on semantic information and emotional information. For example, when the patient's emotional information is "nervous" or "afraid", the waiting module 420 can add soothing words to the current inquiry content. For another example, when the patient's emotional information is "doubtful", the waiting module 420 can add explanatory words to the current inquiry content. For another example, when the semantic information shows that the patient has not clearly answered the historical inquiry, the waiting module 420 can adjust the current inquiry content to repeat the historical inquiry, thereby guiding the patient to clearly answer the historical inquiry. The originally determined current inquiry content can be used as the inquiry content of the next round of inquiries.

[0226] In some embodiments of the present specification, the current inquiry content is adjusted based on the semantic information and emotional information of the patient's historical answers, so that the current inquiry content can be adjusted in time according to the patient's condition, thereby improving the service quality of the pre-consultation.

[0227] In some embodiments, in addition to adjusting the current inquiry content, the sound features used for the inquiry can also be adjusted in real time based on the patient's condition. Sound features include speech speed features, tone features, intonation features, volume features, etc. Fig. 9 As shown, the waiting module 420 can determine the sound characteristics of the current inquiry based on the semantic information and emotional information of the patient's historical answers, and control the third terminal to perform the current inquiry based on the adjusted inquiry content and sound characteristics. Specifically, the waiting module 420 can determine the sound characteristics of the current round of inquiries based on the semantic information and emotional information of the patient's historical answers according to a preset correspondence. Among them, the preset correspondence can characterize the correspondence between semantic information, emotional information and sound characteristics. As an example only, the preset correspondence can characterize that when the semantic information is a positive answer and the emotional information is calm, the speech speed feature is medium, the tone feature is polite, the intonation feature is calm, and the volume feature is medium.

[0228] In some embodiments of the present specification, based on the semantic information and emotional information of the patient's historical answers, the sound features used in the second inquiry are adjusted in real time, which can better take into account the patient's emotional changes, thereby enhancing the personification effect of the second virtual character and improving the quality of pre-consultation services.

[0229] In some embodiments, Fig. 9 As shown, the waiting module 420 can further obtain the patient's physiological state information. The patient's physiological state information can reflect the patient's real-time physiological state. The physiological state information may include the patient's physiological parameter values, such as heart rate, pulse, respiratory rate, etc. The physiological state information may also include information related to the patient's posture, limb behavior, facial expression, muscle state, etc. In some embodiments, the patient's physiological state information can be obtained using a wearable device worn by the patient. For example, the patient's physiological parameter values ​​can be collected by a physiological sensor integrated on the wearable device. In some embodiments, the patient's physiological state information can be collected by an image sensor in the patient's environment. For example, the patient's posture, facial expression, etc. can be collected by a monitoring device in the waiting area.

[0230] Further, the waiting module 420 can adjust the current inquiry content based on semantic information, emotional information and physiological state information. Specifically, the waiting module 420 can update the patient's emotional information based on the patient's physiological state information. It can be understood that the patient's inner emotions may not be fully expressed through the patient's answers. Therefore, the patient's emotional information can be updated or corrected based on the patient's physiological state information. For example, assuming that the patient is determined to be in a calm state based on the second voice information, but the patient's physiological state information shows that the patient is in a nervous state (such as a heart rate exceeding a preset threshold), the consultation module 430 can correct the patient's emotional information to a nervous state. Further, the candidate module 420 can adjust the current inquiry content based on the semantic information and the updated emotional information.

[0231] In some embodiments of the present specification, by further considering the patient's physiological state information, the patient's emotional information can be captured more accurately, so that the adjustment of the second inquiry content is more accurate, thereby improving the service quality of the pre-consultation.

[0232] like Fig. 9 As shown, in some embodiments, the waiting module 420 can determine feedback parameters based on at least a portion of the semantic information, emotional information, and physiological state information, and control the wearable device to apply feedback to the patient based on the feedback parameters. The feedback may include at least one of force feedback and temperature feedback. The feedback parameters are used to control the way the feedback is applied, such as the type of feedback, the part of the body on which the feedback acts, the size of the feedback, etc. In some embodiments, the waiting module 420 can determine the patient's emotions and emotional levels based on at least a portion of the semantic information, emotional information, and physiological state information, and then determine the feedback parameters based on the emotions and emotional levels. For example, the patient's emotions can be used to determine the type of feedback and the part of the body on which the feedback acts, and the patient's emotional level can be used to determine the size of the feedback.

[0233] In some embodiments of the present specification, feedback parameters are determined based on semantic information, emotional information, and physiological state information, and the wearable device is controlled to provide feedback to the patient according to the feedback parameters, which can timely soothe the patient's negative emotions and thus improve the quality of pre-consultation services.

[0234] In some embodiments, the waiting module 420 can end the second inquiry based on a second preset condition. The second preset condition can be that the number of remaining missing information is 0. The second preset condition can be that the time difference between the current time and the patient's estimated consultation time is less than a threshold. The second preset condition can also be similar to the first preset condition. For example, the second preset condition can be that the number of rounds of inquiry performed is equal to a threshold. For another example, the second preset condition can be that the current inquiry content or the patient's answer is a preset content.

[0235] In some embodiments, the second inquiry content determined in step 810 only includes the inquiry content of the first round of inquiries. The current inquiry content of each current inquiry except the first round of inquiries can be determined during the second inquiry. For example, in the current inquiry, the waiting module 420 can input the inquiry content of the historical inquiry, the patient's historical answers, the patient's known information, etc. into the second inquiry content determination model, and the second inquiry content determination model outputs the current inquiry content. The second inquiry content determination model can be obtained based on the fifth sample training set training. Among them, the fifth training sample set may include multiple fifth training samples and corresponding multiple fifth training labels. The fifth training sample may include the inquiry content of the sample historical inquiry, the sample historical answers of the sample patient, and the sample known information of the sample patient, and the fifth training label may include the content of the current round of inquiry of the sample. The fifth training sample set can be determined based on the historical pre-inquiry record or manually determined by the user. The training process of the second inquiry content determination model is similar to that of the first inquiry model, and will not be repeated here.

[0236] In some embodiments of this specification, by executing the second inquiry to communicate with the patient, the various types of information about the patient are fully understood, thereby improving the accuracy of the pre-consultation record and providing more accurate reference information for subsequent consultations. In some embodiments of this specification, a second virtual character is used to conduct a second inquiry to the patient, and the patient's sense of interaction is enhanced through the anthropomorphic second virtual character without increasing labor costs, thereby improving the quality and efficiency of the pre-consultation service. In some embodiments of this specification, a machine learning model (such as a missing information determination model, a first inquiry content determination model, and a second inquiry content determination model) is used to determine the second inquiry content of the second inquiry, making the second inquiry more natural and flexible.

[0237] Step 830: Generate a pre-inquiry record based on the second data collected by the third terminal in the second inquiry.

[0238] The second data includes voice data, text data, image data, etc. input by the patient through the third terminal in the second inquiry. The pre-consultation record is used to record the patient information collected in the second inquiry (i.e., the pre-consultation inquiry). Optionally, some known information of the patient can also be recorded in the pre-consultation record. In some embodiments, the pre-consultation record is generated according to a preset template. The preset template can be a template corresponding to the doctor's department, or it can be a template set by the doctor. In some embodiments, the pre-consultation record can be presented to the doctor for reference when the patient is visiting the doctor. For specific descriptions, please refer to Fig.11 and its related description.

[0239] For example, when the second data includes a voice signal, the waiting module 420 may first transcribe the second data into text, and then extract the second keyword from the text through a keyword extraction algorithm. Further, the waiting module 420 may convert the second keyword into a medical term. Further, the waiting module 420 may obtain multiple template fields in the pre-inquiry record template, retrieve the content corresponding to each template field from the hospital terminology, and fill it into the corresponding position of the pre-inquiry record template. The conversion of the second keyword may be based on a term conversion model or a knowledge dictionary. For a detailed description of the knowledge dictionary, see step 1520. The term conversion model is used to convert the spoken description into medical terms, which may be obtained based on the sixth sample training set training. Among them, the sixth training sample set may include multiple sixth training samples and multiple sixth training labels corresponding thereto. The sixth training sample may include sample keywords, and the sixth training label may include corresponding sample medical terms. The sixth training sample set may be based on historical pre-inquiry records or determined manually by the user. The training process of the term conversion model is similar to that of the first inquiry model, and will not be repeated here.

[0240] In some embodiments of the present specification, pre-consultation services are provided to patients while they are waiting for their appointments, and pre-consultation records are generated. This can, on the one hand, reduce the negative emotions of patients during the waiting period, and on the other hand, improve the doctor's consultation efficiency through pre-consultation records.

[0241] In some embodiments, the second inquiry can be performed via other terminals other than the third terminal. For example, a patient can wait in a waiting area of ​​a hospital, and a waiting terminal is configured in the waiting area, and the waiting terminal can be used to make a second inquiry to the patient. The process of making a second inquiry to the patient using other terminals is similar to the process of making a second inquiry to the patient using the third terminal, and will not be repeated here.

[0242] In some embodiments, process 800 can be executed by an agent corresponding to the medical consultation service / medical consultation process. For example, a pre-consultation service can be provided by a pre-consultation agent. The pre-consultation agent can understand and process multimodal data such as natural language and images, and has knowledge data (such as dictionaries, knowledge graphs, knowledge databases, templates) and patient data corresponding to the consultation department to provide patients with personalized and professional pre-consultation services. On the other hand, the pre-consultation agent can convert the data collected in the pre-consultation inquiry into accurate medical terms, making it convenient for doctors to view structured pre-consultation reports during consultations and reducing doctors' paperwork. In addition, the pre-consultation agent can achieve self-evolution and self-learning based on artificial intelligence technology, and can continuously improve the accuracy and efficiency of pre-consultation services by learning knowledge data and historical service data, thereby improving user experience.

[0243] In some embodiments, the hospital administrator can configure the pre-diagnosis agent, for example, through the management space application installed on the administrator's terminal. The configuration of the pre-diagnosis agent includes the configuration of general parameters, which are related to hardware configuration, concurrent request processing, cache strategy, security settings, model parameters, error handling and logging, performance monitoring and alarm, version management, etc. The configuration of the pre-diagnosis agent also includes the configuration of business parameters. Business parameters are related to department dictionary configuration, answers to common questions, inquiry rounds, pre-diagnosis key fields, structured rules, etc.

[0244] Fig.11 1 is a schematic diagram of an exemplary medical consultation system 1100 according to some embodiments of this specification. The medical consultation system 1100 can be used to provide user services related to the consultation link 530 to the target user. Fig.11 As shown, the medical consultation system 1100 may include a processing device 210, a terminal 1110, a sensing device 1120, and a physical examination device 1130. The processing device 210 may be connected to the terminal 1110, the sensing device 1120, and the physical examination device 1130. The target users may include patients and doctors who provide medical consultation services to the patients. In some cases, the target users may also include remote companions.

[0245] The sensing device 1120 can sense the surrounding environment in which it is located to collect sensing information. Exemplary sensing devices 1120 may include various sensor devices such as sound sensors, image sensors, temperature sensors, humidity sensors, etc. The sensing device 1120 may be an independent device, for example, a monitoring device installed in a consulting room. The sensing device 1120 may also be integrated into other devices. For example, a sound sensor may be integrated into at least one terminal 1110. In some embodiments, the sensing device 1120 may transmit the collected sensing information to the processing device 210 via a network (such as real-time transmission). In some embodiments, the sensing device 1120 may be an Internet of Things device.

[0246] When the medical consultation system 1100 is used to provide on-site medical consultation services, the sensing device 1120 may be located in the consulting room to sense the consulting room environment. When the medical consultation system 1100 is used to provide remote medical consultation services, the sensing device 1120 may include a first sensing device located in the doctor's environment and a second sensing device located in the patient's environment. The first sensing device is used to collect sensing information related to the doctor's environment. The second sensing device is used to collect second sensing information related to the patient's environment.

[0247] In some embodiments, the sensing device 1120 further includes a third sensing device located in the environment where the remote patient is located, for collecting sensing information related to the environment where the remote patient is located.

[0248] The physical examination device 1130 performs a health examination on the patient to collect the patient's physical examination data. Exemplary physical examination devices 1130 may include, but are not limited to, stethoscopes, sphygmomanometers, thermometers, heart rate monitors, blood oxygen saturation meters, etc. The physical examination device 1130 may be an independent device or may be integrated into other devices (such as wearable devices). In some embodiments, the physical examination device 1130 may transmit the collected physical examination data to the processing device 210 via a network (such as real-time transmission). In some embodiments, the physical examination device 1130 may be an Internet of Things device.

[0249] When the medical treatment system 1100 is used to provide on-site medical treatment services, the physical examination device 1130 can be located in the consulting room, and the doctor can operate the physical examination device 1130 to perform a health examination on the patient. When the medical treatment system 1100 is used to provide remote medical treatment services, the physical examination device 1130 may include a device located in the patient's environment, and the patient can operate the physical examination device 1130 to perform a health examination on the patient. Alternatively, the doctor can remotely control the physical examination device 1130 through a second terminal (such as a second XR device) to perform a health examination on the patient. In some embodiments, the physical examination device 1130 can be worn on the patient as a wearable device to continuously collect the patient's physical examination data during the consultation process.

[0250] Terminal 1110 is a device for interacting with a target user. In some embodiments, terminal 1110 may include at least one of first terminal 1111, second terminal 1112, third terminal 1113, and / or fourth terminal 1114. During the consultation process, at least one terminal may be used to present data to the corresponding target user.

[0251] The first terminal 1111 is a terminal that the hospital equips for the consulting room. The first terminal 1111 may include one or more of a display screen, a sound output device, a sound sensor, an image sensor, etc. The second terminal 1112, the third terminal 1113, and the fourth terminal 1114 are terminals for interacting with doctors, patients, and remote accompanying persons, respectively. The second terminal 1112, the third terminal 1113, and the fourth terminal 1114 may include one or more of a mobile phone, a computer device, a wearable device, an XR device, etc. Fig.11 As shown, the second terminal 1112 may include a second XR device, the third terminal 1113 may include a first XR device, and the fourth terminal 1114 may include a third XR device.

[0252] When a patient receives on-site medical consultation services in a consulting room, a consultation interface can be presented to the patient and the doctor at the same time through the first terminal 1111, and a variety of data can be presented on the consultation interface to facilitate communication between the patient and the doctor. In some embodiments, patient data, such as medical data related to the patient, can be presented on the consultation interface. Exemplary medical data include electronic health records (or a portion thereof), pre-consultation records, medical images, three-dimensional organ models, examination results, etc. In some embodiments, the data presented on the consultation interface is multimodal, including data in various forms such as images, graphics, videos, and text.

[0253] Fig.14 is a schematic diagram of an exemplary medical consultation interface according to some embodiments of this specification. Fig.14 As shown, the consultation interface 1400 may include a first interface element 1410 related to the patient's electronic health record, a second interface element 1420 related to the remote accompanying consultation service, a third interface element 1430 related to medical documents, and a fourth interface element 1440 related to consultation recommendations. It should be understood that the consultation interface 1400 is only for illustrative purposes, and it may include one or more other interface elements, or omit one or more Fig.14 The interface elements shown in .

[0254] Furthermore, the content and / or form presented in the consultation interface 1400 may change as the consultation process progresses. For example, the doctor and / or the patient may issue a control instruction to adjust the content presented in the consultation interface 1400. The control instruction may be a voice control instruction, a gesture control instruction, a touch control instruction, a control instruction inputted with an input device (such as a mouse, a keyboard, etc.), or other forms of instructions. In some embodiments, the hospital administrator may configure the consultation interface 1400, for example, through a management space application installed on the administrator terminal.

[0255] The first interface element 1410 is an icon corresponding to the electronic health record, or is used to present the content of the electronic health record. During the consultation process, the doctor and / or the patient can call at least a part of the electronic health record by issuing a control command, and present it through the first interface element 1410. For a detailed description of presenting at least a part of the electronic health record, please refer to Fig.12 and its related description.

[0256] The second interface element 1420 is an icon corresponding to the remote medical companion service, or is used to present the screen of the remote medical companion. When the patient's request for remote medical companion service is approved by the doctor, the patient can communicate with the remote medical companion during the consultation process and view the screen of the remote medical companion through the second interface element 1420. For a detailed description of the remote medical companion service, please refer to Fig.16 and its related description.

[0257] The third interface element 1430 is an icon corresponding to a medical document, or is used to present medical documents such as pre-consultation records, diagnosis records (initial consultation records, target consultation records), etc. During the consultation process, when it is detected that the patient has started the consultation, the pre-consultation record can be presented through the third interface element 1430; when the doctor and the patient have completed the communication, the third interface element 1430 can be updated to present the initial diagnosis record; when the doctor confirms the modification of the initial diagnosis record, the third interface element 1430 can be updated to present the target diagnosis record.

[0258] In some embodiments, the consultation module 430 can determine that the patient has started the consultation based on a call instruction. The call instruction is an instruction from the doctor to start the next patient consultation. The doctor can actively initiate the call instruction through the first terminal or the second terminal. Alternatively, after the doctor submits the target diagnosis record of the previous patient, the consultation module 430 can automatically generate a call instruction. In some embodiments, when the patient receives on-site medical consultation services, the consultation module 430 can determine that the patient has started the consultation based on the positioning information of the patient's third terminal. Alternatively, the consultation module 430 can obtain an image of the patient entering the consulting room based on a sensing device (for example, an image sensor) to determine that the patient has started the consultation. When the patient receives remote medical consultation services, the consultation module 430 can obtain remote consultation confirmation messages sent by the doctor and the patient through the doctor's second terminal and the patient's third terminal, respectively, to determine that the patient has started the consultation. For more descriptions of pre-consultation records and diagnosis records, see Fig.13 and its related description.

[0259] The fourth interface element 1440 is an icon corresponding to the consultation suggestion, or is used to present the content of the consultation suggestion. During the consultation process, after the consultation suggestion is generated based on the perception information, the consultation suggestion can be presented through the fourth interface element 1440. For a detailed description of the consultation suggestion, please refer to Fig.13 and its related description.

[0260] In some embodiments, various data can be synchronously presented to patients and doctors through the second terminal 1112 and the third terminal 1113, respectively. For example, doctors and patients can wear XR devices respectively, and through mixed reality and augmented reality technology, doctors and patients can synchronously share various data while perceiving the real physical world. Among them, the second terminal 1112 and the third terminal 1113 can present content similar to the medical consultation interface. When a remote patient attends the medical consultation, the fourth terminal 1114 can present the remote patient with a real-time picture of the clinic scene and content similar to that presented on the medical consultation interface.

[0261] When a patient receives a telemedicine consultation service, a virtual consulting room space may be presented to the doctor through the first terminal 1111 or the second terminal 1112, and the virtual consulting room space may be presented to the patient through the third terminal 1113. For the convenience of description, the following description is based on an example of presenting data to the doctor through the second terminal 1112. For example, the patient's third terminal includes a first XR device, and the doctor's second terminal includes a second XR device, and the first XR device and the second XR device may present the virtual consulting room space to the patient and the doctor, respectively.

[0262] In some embodiments, various data can be synchronously presented to doctors and patients through the second terminal 1112 and the third terminal 1113, respectively. For example, the first XR device and the second XR device allow patients and doctors to synchronously share various data while perceiving the virtual consulting room space. Among them, the second terminal 1112 and the third terminal 1113 can present content similar to the consultation interface. When a remote patient participates in the consultation process, the virtual consulting room space and various data can be presented to the remote patient through the third XR device of the fourth terminal 1114.

[0263] The virtual clinic space is a virtual space for the participants of the consultation to interact. In some embodiments, the virtual clinic space may include the virtual clinic space and virtual participants of the consultation. The virtual clinic space can be a virtually created environment, or it can be generated based on the real-time information of the environment (such as the consultation room) where the doctor or patient is located. Furthermore, the virtual participants of the consultation can be generated based on the real-time information of the participants of the consultation, and then the virtual clinic space can be generated based on the virtual clinic space and the virtual participants of the consultation. The consultation module 430 can present the first perspective of the virtual clinic space to the patient through the first XR device, present the second perspective of the virtual clinic space to the doctor through the second XR device, and present the third perspective of the virtual clinic space to the remote accompanying person through the third XR device of the remote accompanying person. The first perspective, the second perspective, and the third perspective of the virtual clinic space can be the perspectives of the virtual patient, the virtual doctor, and the virtual remote accompanying person in the virtual clinic space, respectively. In some embodiments, when the patient wears an XR device to move or rotate, the consultation module 430 can obtain the distance moved and the angle turned by the patient through the XR device, and change the corresponding perspective image of the patient based on the distance moved and the angle turned by the patient.

[0264] The processing device 210 can provide services related to the consultation link 530 by processing the sensory information and the physical examination data. For detailed description of the services related to the consultation link 530, please refer to Figure 5 and its related description.

[0265] In some implementations, the processing device 210 can provide services related to the consultation process through an agent corresponding to the medical consultation service / medical consultation process. For example, the consultation agent can process the perception information, physical examination data, and data from other sources (such as electronic health records) to provide services such as medical data display, consultation advice, and generation of diagnosis records.

[0266] In some embodiments, the hospital administrator can configure the medical agent, for example, through the management space application installed on the administrator's terminal. The configuration of the medical agent includes the configuration of general parameters, which are related to hardware configuration, concurrent request processing, cache strategy, security settings, model parameters, error handling and logging, performance monitoring and alarm, version management, etc. The configuration of the medical agent also includes the configuration of business parameters. Business parameters are related to specialist knowledge graphs, document templates such as medical records, custom vocabulary, etc.

[0267] Fig.12 It is a flowchart of an exemplary process of displaying medical data to a target user during a medical consultation process according to some embodiments of this specification. Fig.12 The process 1200 shown can be executed in the consultation session 530 .

[0268] Step 1210: presenting a first interface element to a target user through at least one terminal. The first interface element is related to medical data of a patient receiving medical consultation services, and the target users include at least patients and doctors.

[0269] The patient's medical data may include electronic health records, medical images, medical examination results and other types of data that can reflect the patient's health status. In some embodiments, the medical data may include multimodal data, which may include data in various forms such as text, pictures, graphics, and voice. In some embodiments, the medical data may include an electronic health record, and the first interface element is related to the patient's electronic health record. As mentioned above, the electronic health record is an electronic file that records various types of patient data. For example, the electronic health record includes a three-dimensional model of multiple organs and / or tissues of the patient.

[0270] The first interface element can be in various forms. For example, the first interface element can present at least a portion of the medical data. As just one example, the first interface element can present the homepage of the electronic health record. For another example, the first interface element can be an icon corresponding to the patient's medical data (e.g., an icon corresponding to the electronic health record).

[0271] In some embodiments, the medical consultation service is an on-site medical consultation service performed in a consulting room, that is, both the patient and the doctor are in the consulting room. In this case, the at least one terminal may include a first terminal in the consulting room (e.g., Fig.11 The first terminal 1111) can present the first interface element to the patient and the doctor. Alternatively, at least one terminal can include the doctor's second terminal (such as Fig.11 The second terminal 1112 shown) and the patient's third terminal (such as Fig.11 For example, the patient and the doctor can each wear an XR device during the consultation. The XR device worn by the patient can use AR / MR technology to overlay the first interface element in the patient's real-world field of view, and the XR device worn by the doctor can use AR / MR technology to overlay the first interface element in the doctor's real-world field of view. The patient and the doctor can view the first interface element together while being able to perceive the real environment of the consulting room.

[0272] In some embodiments, the medical consultation service is a remote medical consultation service, that is, both the patient and the doctor are outside the clinic, or only the patient is outside the clinic.

[0273] When both the patient and the doctor are outside the clinic, at least one terminal may include the doctor's second terminal (e.g. Fig.11 The second terminal 1112 shown) and the patient's third terminal (such as Fig.11 The third terminal 1113 shown in the figure). In some embodiments, the second terminal and the third terminal may include a second XR device and a first XR device for presenting a virtual consulting room space to the doctor and the patient, respectively. The first XR device worn by the patient may present the corresponding virtual consulting room space to the patient, and overlay a first interface element in the patient's virtual consulting room space. The second XR device worn by the doctor may present the corresponding virtual consulting room space to the doctor, and overlay a first interface element in the doctor's virtual consulting room space. The patient and the doctor may view the first interface element together while being able to perceive their respective corresponding virtual consulting room spaces.

[0274] When the patient is outside the clinic and the doctor is in the clinic, at least one terminal may include the doctor's second terminal (such as Fig.11 The second terminal 1112 shown) and the patient's third terminal (such as Fig.11 Alternatively, the at least one terminal may further include a first terminal in the consulting room (eg, Fig.11 The first terminal 1111) can present interface elements synchronized with the third terminal to the doctor.

[0275] In some embodiments, the target user further includes a remote patient's companion, and the at least one terminal further includes a fourth terminal of the remote patient's companion. For example, the fourth terminal may be a third XR device worn by the remote patient's companion, and the third XR device may present the virtual clinic space and the first interface element to the remote patient's companion. For more descriptions of the virtual clinic space, please refer to Fig.19 and its related description.

[0276] In some embodiments, the consultation module 430 may display the first interface element through at least one terminal based on a trigger condition. For example, the trigger condition may include detecting that the patient has started to see a doctor.

[0277] Step 1220, based on the perception information collected by one or more perception devices during the medical consultation process, obtain a control instruction issued by at least one of the target users.

[0278] The control instruction is an instruction for retrieving at least a portion of the medical data (such as an electronic health record) for display. For example, the control instruction is used to retrieve a three-dimensional model of an organ of interest of a patient in the electronic health record for display. In some embodiments, the control instruction can also be used to set display parameters (such as display angle, display size, display position). In some embodiments, the control instruction can also be used to mark key data on medical data (such as a three-dimensional model of an organ of interest).

[0279] For on-site medical consultation services, the sensing device may include one or more sensing devices located in the consulting room. For example, the sensing device may include an image sensor, a sound sensor, etc. in the consulting room. The sensing device may be installed at any location in the consulting room. For example, the image sensor may be installed on the wall or ceiling of the consulting room. In some embodiments, the sensing device in the consulting room may be integrated into a terminal in the consulting room. For example, the sound sensor may be integrated into a first terminal. For another example, the sound sensor may be integrated into a second terminal of the doctor and / or a third terminal of the patient.

[0280] For telemedicine consultation services, the sensing devices may include one or more sensing devices in the doctor's environment (e.g., consulting room, doctor's residence) and the patient's environment (e.g., patient's residence). The sensing device in the doctor's environment may be an independent sensing device or integrated in the second terminal. The sensing device in the patient's environment may be an independent sensing device or integrated in the third terminal.

[0281] In some embodiments, the perception information may include a voice signal collected by a sound sensor, and the control instruction may be obtained by semantically analyzing the voice signal. Semantic analysis refers to the analysis of the speech content contained in the voice signal. For example, when the consultation module 430 detects that the speech content contains keywords such as "display", "presentation", and "retrieval" related to the interface display, it can be determined that the control instruction has been received, and then the type of medical data the user wants to retrieve and / or the display parameters he wants to set are determined based on the speech content. In some embodiments, the target user can issue a control instruction by saying a pre-set wake-up word. When the consultation module 430 detects that the speech content contains a wake-up word, it can be determined that the control instruction has been received.

[0282] In some embodiments, the perception information may include an optical image of a target user (such as a patient and / or a doctor) captured by an image sensor, and the control instruction may be obtained by performing gesture recognition on the target user in the optical image. For example, the target user may issue a control instruction by performing a preset gesture (such as a click gesture, a slide gesture, a rotation gesture, using two fingers to make an expansion or contraction action, etc.). When the consultation module 430 detects a preset gesture in the optical image, it may determine that a control instruction has been received, and then determine the type of medical data that the user wants to retrieve and / or the display parameters that the user wants to set based on the type of gesture.

[0283] In some embodiments, the target user can use the control device to issue control instructions. For example, when seeing a doctor in a clinic, the doctor and / or patient can issue control instructions through a remote controller, smart control gloves, etc. to control the first terminal to display at least a portion of the medical data.

[0284] In some embodiments of the present specification, the target user can adjust the display content and / or display parameters through flexible methods such as voice and gestures, thereby optimizing the user experience and improving the efficiency of medical treatment.

[0285] Step 1230 , in response to the control instruction, retrieve at least a portion of the medical data and present at least a portion of the medical data through at least one terminal.

[0286] For example, the consultation module 430 can retrieve at least a portion of the medical data from a storage device and control the at least one terminal to present at least a portion of the medical data. When the control instruction includes a display parameter, the consultation module 430 can control at least one terminal to present at least a portion of the medical data based on the display parameter. In some embodiments, during the remote consultation process, based on the perception data collected by the perception device in the doctor's environment, the control instruction issued by the doctor can be detected, the three-dimensional model of the organ of interest can be retrieved for display, the display parameters can be adjusted, and key data can be annotated; in response to the control instruction, the display content on at least one terminal (such as the first and second XR devices) can be synchronously updated.

[0287] In some embodiments of the present description, multiple target users can browse medical data together through at least one terminal, and the presentation content and presentation method of the medical data on each terminal change synchronously, which helps to improve the communication efficiency between target users and enhance the interactivity of the medical treatment process.

[0288] Fig.13 1300 is a flowchart of an exemplary process of providing medical consultation services based on perception information according to some embodiments of the present specification. In some embodiments, process 1300 may include one or more of sub-processes 1310, 1320 and 1330.

[0289] Subprocess 1310 can be used to provide consultation advice based on the perception information. Subprocess 1310 can be executed in the consultation link 530. Fig.13 As shown, subprocess 1310 may include steps 1312 and 1314 .

[0290] Step 1312, generating a consultation recommendation based on the sensory information and the patient's patient data.

[0291] For a detailed description of sensory information, see Fig.11 and Fig.12 For more information about patient data, please refer to the description of the registration link 510. Consultation suggestions are suggestions to assist doctors in providing medical consultation services. Exemplary consultation suggestions may include additional inquiry suggestions, physical examination suggestions, prescription suggestions, treatment plan suggestions, etc.

[0292] In some embodiments, the consultation suggestion can be determined based on the knowledge database, consultation specifications, etc. corresponding to the registration department. For example, the consultation module 430 can determine the conversation content between the doctor and the patient based on the voice signal collected by the sound sensor, and search in the knowledge database, consultation specifications, etc. based on the conversation content and / or patient data to determine the consultation suggestion. As an example only, the consultation specifications can be searched based on the conversation content and / or patient data to determine which information in the consultation specifications has not been collected, and provide supplementary inquiry suggestions based on this information.

[0293] In some embodiments, the consultation recommendation can be generated based on the diagnosis model. Specifically, the consultation module 430 can determine the model input based on the perception information and the patient data, and input the model input into the diagnosis model, and the diagnosis model can output the corresponding consultation recommendation. For example, the model input can include one or more of the patient data, the conversation content determined based on the voice signal, the patient's status information determined based on the image data, etc.

[0294] In some embodiments, the diagnostic model can be acquired based on the seventh sample training set training. The seventh training sample set may include multiple seventh training samples and multiple seventh training labels corresponding thereto. The seventh training sample may include a sample model input similar to the model input described above, and the seventh training label may include a sample consultation recommendation. The seventh training sample set may be based on historical medical records and / or determined manually. The training process of the diagnostic model is similar to that of the first inquiry model and will not be repeated here.

[0295] In some embodiments, the medical consultation recommendations can be generated by an agent corresponding to the medical consultation service. The agent can learn the generation mechanism of medical consultation recommendations from various data such as historical consultation records, knowledge databases, consultation specifications, etc., and process the perception information and patient data based on the mechanism to provide medical consultation recommendations.

[0296] Step 1314, controlling at least a portion of at least one terminal to present medical consultation recommendations.

[0297] For example, when a patient receives medical consultation services on-site in a consulting room, the consultation module 430 may send consultation suggestions to the first terminal, and the first terminal may present the consultation suggestions to the patient and the doctor via the fourth interface element. For another example, the consultation module 430 may send consultation suggestions to the doctor's second terminal, and the second terminal may present the consultation suggestions to the doctor. For another example, when a patient receives telemedicine consultation services, the consultation module 430 may control the doctor's second terminal and the patient's third terminal to present the consultation suggestions respectively.

[0298] As the consultation process progresses, the perception information will be updated in real time, and the consultation recommendations will be updated accordingly. For example, before the doctor examines the patient, the consultation recommendations can include physical examination recommendations. When the doctor completes the physical examination of the patient, the consultation recommendations can be updated to prescription recommendations.

[0299] In some embodiments of the present specification, automatically generating consultation recommendations based on perception information can improve the accuracy of diagnosis and prescription, while improving the efficiency of doctors' consultations.

[0300] Subprocess 1320 can be used to generate a target diagnosis record based on the perception information. Subprocess 1320 can be executed after the consultation session 530 is completed. Fig.13 As shown, subprocess 1320 may include steps 1322 , 1324 , and 1326 .

[0301] Step 1322, generating an initial diagnosis record based on the perception information.

[0302] The initial diagnosis record is an automatically generated diagnosis record. In some embodiments, the initial diagnosis record may include the initial patient medical history, the initial diagnosis opinion, the initial diagnosis prescription (e.g., the initial treatment prescription and the initial examination prescription), the initial doctor's order, etc. For a detailed description of generating the initial diagnosis record, see Fig.15 and its related description.

[0303] In some embodiments, the initial diagnosis record can be generated by an agent corresponding to the medical consultation service. The agent can learn the generation mechanism of the diagnosis record from various data such as diagnosis record templates, knowledge dictionaries, knowledge databases, etc., and process the perception information and patient data based on the mechanism to generate the diagnosis record.

[0304] Step 1324, present the initial diagnosis record to the physician.

[0305] Exemplarily, the consultation module 430 may send the initial diagnosis record to the first terminal, and the first terminal may present the initial diagnosis record. For example, when the patient starts the consultation, the third interface element presents the pre-consultation record to the patient and the doctor; when the consultation is completed, the third interface element may be updated to present the initial diagnosis record.

[0306] For another example, the consultation module 430 can send the initial diagnosis record to the doctor's second terminal, and the second terminal can present the initial diagnosis record to the doctor. In some embodiments, the second terminal can present the initial diagnosis record to the doctor at a preset time (such as after the doctor finishes the consultation work for the day). Alternatively, the second terminal can present the initial diagnosis record to the doctor after the doctor enters a document review request. In some embodiments, a medical space application is installed on the second terminal, and the doctor can obtain the initial diagnosis record through the medical space application and give feedback information on the initial diagnosis record.

[0307] Step 1326, generating a target diagnosis record based on the initial diagnosis record and the feedback information input by the doctor on the initial diagnosis record.

[0308] The feedback information input by the doctor may include the doctor's modification and / or confirmation of the initial diagnosis record. The target diagnosis record is the diagnosis record after the doctor has modified and / or confirmed. In some embodiments, the target diagnosis record may include the target patient's medical history, target diagnosis opinion, target diagnosis prescription (e.g., target treatment prescription and target examination prescription), target doctor's order, etc. In some embodiments, the doctor may input feedback information via the first terminal or the second terminal in a variety of ways such as typing, voice, gestures, etc.

[0309] In some embodiments of the present specification, an initial diagnosis record can be generated based on the perception information, and then a target diagnosis record can be generated based on the doctor's feedback information on the initial diagnosis record. On the one hand, manual writing errors in the target diagnosis record can be reduced, and the efficiency of generating the target diagnosis record can be improved. On the other hand, the doctor's paperwork can be reduced, so that the doctor has more energy to care for the patient and improve the quality of medical treatment services.

[0310] Subprocess 1330 can be used to provide remote medical accompaniment services based on the perceived information. Patients can apply for remote medical accompaniment services before consultation. For details, see Fig.16 Subprocess 1330 can be executed in the consultation session 530. Fig.13 As shown, subprocess 1330 may include steps 1332 and 1334 .

[0311] Step 1332, determining whether the patient needs to communicate with the remote patient companion based on the perception information.

[0312] In some embodiments, the consultation module 430 can detect whether the patient has issued a request to communicate with the remote accompanying person based on the perception information (such as voice data and / or image data). For example, the patient can issue the request by speaking or making a specific gesture.

[0313] In some embodiments, the consultation module 430 can determine the patient's status information based on the perception information, and determine whether the patient needs to communicate with the remote accompanying person based on the patient's status information. For example, when the status information shows that the patient is in a state of high tension, fear, etc., it is determined that the patient needs to communicate with the remote accompanying person. For another example, when the status information shows that the patient cannot choose or determine the treatment plan by himself, it is determined that the patient needs to communicate with the remote accompanying person.

[0314] When it is determined that the patient needs to communicate with the remote patient companion, the consultation module 430 executes step 1334 .

[0315] Step 1334: Control at least a portion of at least one terminal to enlarge the second interface element.

[0316] When the patient receives on-site medical consultation services in the consulting room, the consultation module 430 can control the first terminal to enlarge the second interface element. When the patient receives remote medical consultation services, the consultation module 430 can control the patient's third terminal to enlarge the second interface element. Through the enlarged second interface element, the patient can view the real-time picture of the remote accompanying person and communicate better with the remote accompanying person.

[0317] In some embodiments, when a patient receives an on-site medical consultation service in a consulting room, when it is detected that the patient needs to communicate with a remote accompanying person, the consultation module 430 can remind the patient and the doctor to wear the first XR device and the second XR device respectively, and control the first and second XR devices to present the image data of the remote accompanying person. In other words, when the patient does not need to communicate with the remote accompanying person, he / she does not need to wear an XR device and can communicate directly with the doctor face to face. When the patient needs to communicate with the remote accompanying person, he / she and the doctor can be provided with an XR device to obtain a better communication experience.

[0318] In some embodiments of the present specification, the patient's communication needs can be detected based on the perception information and met in a timely manner, providing the patient with more humane care and a more realistic and immersive medical consultation experience.

[0319] Fig.15 is a flowchart of an exemplary process of generating an initial diagnosis record according to some embodiments of the present specification. Fig.15 The process 1500 shown may be used to implement step 1322 .

[0320] Step 1510, extracting key content from the perception information based on the diagnosis record template.

[0321] The diagnosis record template is used to define the format and content of the diagnosis record. For example, the diagnosis record template contains multiple template fields arranged in a specific format, and these template fields represent the content to be included in the diagnosis record. In some embodiments, the diagnosis record template may include template fields such as patient medical history (including basic patient information, description of patient symptoms, physical examination data, etc.), diagnosis opinion, diagnosis prescription (e.g., treatment prescription and examination prescription), and doctor's order.

[0322] The key content is the content related to the template field in the diagnosis record template. In some embodiments, the perception information includes a voice signal. Since the voice signal records the conversation between the patient and the doctor, the key content extracted from the voice signal includes content in natural language form. Specifically, the consultation module 430 can transcribe the voice signal into text, and extract the key content from the transcribed text based on multiple template fields in the diagnosis record template. For example, based on the template field "check prescription", the key content "take a CT scan of the leg" can be extracted from the transcribed text.

[0323] In some embodiments, the medical inquiry module 430 can extract key content from the medical examination data collected by the medical examination device during the medical consultation process. For example, when the diagnosis record template contains the template field "blood pressure", the patient's blood pressure value can be extracted from the data collected by the blood pressure meter as the key content.

[0324] Step 1520, converting the key content into professional content based on the knowledge dictionary.

[0325] The knowledge dictionary is a comparison dictionary of natural language description and professional language description. Specifically, the medical consultation module 430 can use the key content as an index to retrieve the corresponding professional language from the knowledge dictionary as professional content. For example, based on the knowledge dictionary, the key content "take a CT scan of the leg" can be converted into the professional content "CT scan of the leg".

[0326] In some embodiments, the diagnosis module 430 may convert the key content into professional content based on a term conversion model. For a detailed description of the term conversion model, please refer to the relevant description of step 830.

[0327] In some embodiments, different departments correspond to different knowledge dictionaries and / or terminology conversion models, and the knowledge dictionary and / or terminology conversion model corresponding to the patient's registration department can be used to generate professional content.

[0328] Step 1530, updating the diagnosis record template based on the professional content and knowledge database to generate an initial diagnosis record.

[0329] The knowledge database refers to the knowledge database of the registration department, which includes the department's medical treatment specifications (such as symptom description specifications, diagnosis specifications, prescription specifications, medical advice specifications, etc.). In some embodiments, the consultation module 430 can evaluate and / or adjust the professional content according to the knowledge database to make it conform to the department's medical treatment specifications. Further, the consultation module 430 can fill in the evaluated or adjusted professional content into the corresponding template field in the diagnosis record template, thereby generating an initial diagnosis record. In some embodiments, the consultation module 430 can further generate an initial diagnosis record based on the patient's electronic health record. For example, the consultation module 430 can search for content corresponding to the template field in the electronic health record, evaluate and / or adjust these contents according to the knowledge database, and fill in the evaluated or adjusted content into the corresponding template field in the diagnosis record template.

[0330] In some embodiments of the present specification, an initial diagnosis record is generated based on a knowledge database so that the initial diagnosis record complies with the consultation standards of the registration department, thereby improving the accuracy of the initial diagnosis record.

[0331] Fig.16 FIG. 1 is a schematic diagram of an exemplary process of providing remote medical escort services according to some embodiments of this specification. Fig.16 As shown, the process 1600 of providing remote medical consultation service includes:

[0332] Step 1610, receiving a request for remote medical consultation from a patient before the medical consultation.

[0333] The remote accompanying consultation request is used to request the remote accompanying person to participate in the consultation process and communicate remotely with the doctor and the patient. In some embodiments, the remote accompanying consultation request may include patient information (such as basic information, registration information, and consultation method), remote accompanying person information, and relevant information of the remote accompanying person's fourth terminal. In some embodiments, the patient can initiate a remote accompanying consultation request through the patient space application installed on the third terminal.

[0334] Step 1620: Send the request to the doctor's second terminal for approval.

[0335] As an example only, the consultation module 430 can send the request to the doctor's second terminal through the network. After receiving the remote accompanying consultation request, the second terminal can remind the doctor to approve it by vibration, ringing, etc. In some embodiments, before the doctor provides outpatient services, the second terminal can uniformly display the remote accompanying consultation requests issued by the patients who visited the doctor on the same day to the doctor for approval. In some embodiments, the preview of the outpatient clinic on the same day can include information on whether the patient has submitted a remote accompanying consultation request. The doctor can approve the patient's remote accompanying consultation request when checking the preview of the outpatient clinic on the same day. For more descriptions of the preview of the outpatient clinic on the same day, please refer to Figure 5.

[0336] Step 1630: In response to determining that the request is approved, control at least one terminal to display a second interface element related to the remote medical escort service.

[0337] When a patient receives on-site medical consultation services in a consulting room, at least one terminal may include a first terminal and / or a third terminal. The second interface element on the first terminal and / or the third terminal may be used to present a real-time picture of the remote accompanying person. At the same time, the sound output device on the first terminal and / or the third terminal may play the real-time sound of the remote accompanying person. At least one terminal also includes a fourth terminal for the remote accompanying person. The second interface element on the fourth terminal is used to present a real-time picture of the doctor and the patient in the consulting room. At the same time, the sound output device on the fourth terminal may play the real-time sound of the doctor and the patient. In some embodiments, the fourth terminal of the remote accompanying person may be an XR device. The consultation module 430 may generate a virtual consulting room space based on the real-time monitoring information of the consulting room, and then display the virtual consulting room space to the remote accompanying person through the remote accompanying person's XR device, so that the remote accompanying person has an immersive experience of being on site in the consulting room. For a detailed description of the virtual consulting room space, please refer to Fig.11 and its related description.

[0338] When a patient receives a telemedicine consultation service, at least one terminal may include a second terminal of the doctor, a third terminal of the patient, and a fourth terminal of the remote patient's companion. The second interface element on the second terminal and the third terminal may be used to present a real-time picture of the remote patient's companion. The second interface element on the fourth terminal may be used to present a real-time picture of the patient and the doctor. At the same time, the sound output device of each terminal may play the real-time sound of other participants in the consultation process.

[0339] In some embodiments, when the request is approved, the consultation module 430 can send the accompanying consultation entry link to the patient's third terminal and / or the remote accompanying person's fourth terminal. During the patient's consultation, the remote accompanying person can obtain the remote accompanying consultation service through the accompanying consultation entry link. In some embodiments, when it is detected that the patient starts to see a doctor, at least one terminal can display the second interface element.

[0340] In some embodiments of the present specification, the remote medical companion service can realize barrier-free real-time communication between patients, doctors and remote medical companions, thereby improving the medical experience.

[0341] Fig.17 is a schematic diagram of an exemplary process for providing health monitoring services according to some embodiments of the present specification. Fig.17 The process 1700 shown may be performed in the post-diagnosis phase 540 .

[0342] Step 1710, generating a health care plan based on the target diagnosis record.

[0343] The health monitoring plan is used to monitor the patient's health status after the patient has finished the consultation. In some embodiments, the health monitoring plan can monitor whether the patient complies with prescriptions and doctor's orders. In some embodiments, monitoring of health status may include one or more of medication monitoring, health habit monitoring, and physiological data monitoring. Among them, medication monitoring is used to monitor the patient's medication situation (such as medication time, medication type, medication method, and medication dosage). Health habit monitoring is used to monitor the patient's exercise, diet and other living habits. Physiological data monitoring is used to monitor the patient's physiological data. The health monitoring plan can define monitoring parameters, monitoring frequency, monitoring time, related thresholds, etc.

[0344] The health monitoring plan can be generated based on the target diagnosis prescription, target doctor's order, etc. in the target diagnosis record. For example, the post-diagnosis service module 440 can generate a medication monitoring plan based on the target diagnosis prescription, and generate a health habit monitoring plan and a physiological data monitoring plan based on the target doctor's order in the target diagnosis record. In some embodiments, the post-diagnosis service module 440 can further generate a health monitoring plan based on other data of the patient (such as other data in the electronic health record).

[0345] After the health monitoring plan is generated, one or more monitoring devices of the patient can be controlled based on the health monitoring plan to collect the patient's health monitoring information. The monitoring device can include any device that can collect the patient's health monitoring information, such as home monitoring devices (such as image sensors, sound sensors), smart medicine boxes, smart injection devices, smart watches, physical examination devices, etc. Specifically, steps 1720-1740 can be performed to collect health monitoring information.

[0346] Step 1720: Generate a health monitoring instruction based on the health monitoring plan. Step 1730: Send the health monitoring instruction to one or more monitoring devices. Step 1740: Obtain the patient's health monitoring information from the one or more monitoring devices.

[0347] Health monitoring instructions are instructions that instruct the monitoring device to perform health monitoring on the patient. In some embodiments, corresponding to the health monitoring plan, the health monitoring instructions may include one or more of medication instructions, health habit instructions, and physiological data monitoring instructions, and the health monitoring information may include one or more of the patient's medication information, health habit information (such as exercise information, diet information), physiological parameter information, etc. For example, the medication instruction may instruct the smart medicine box to remind the patient to take the medication at the medication time and collect the patient's medication information. For another example, the health habit instruction may instruct the smart watch to remind the patient to exercise at the exercise time and collect the patient's running information. For another example, the physiological data monitoring instruction may instruct the physical examination device to remind the patient to collect specific physiological parameters at the physical examination time and record the patient's physiological parameter information.

[0348] After receiving the health monitoring instruction, the monitoring device may perform specific operations based on the health monitoring instruction and send the collected health monitoring information to the processing device 210 .

[0349] Step 1750, updating the health care plan based on the health care information.

[0350] Specifically, the post-diagnosis service module 440 can evaluate the patient's health status based on the patient's health monitoring information, and update the health monitoring plan based on the evaluation results of the health status. For example, when it is determined that the patient's health status is poor based on the patient's health status information, the patient's medication monitoring plan (such as medication dosage), health habit monitoring plan (such as increasing exercise time), and physiological data monitoring instructions (reducing the time interval for blood pressure monitoring) can be updated and modified. In some embodiments, the post-diagnosis service module 440 can evaluate the patient's health risk level based on the patient's health monitoring information, and update the health monitoring plan based on the health risk level.

[0351] In some embodiments, when a change in the patient's health condition is detected (such as improvement or deterioration), the post-diagnosis service module 440 can send the health monitoring information to the doctor, and the doctor will determine whether the patient's health monitoring plan needs to be updated. In some embodiments, when an abnormality is detected in the patient's health condition, the post-diagnosis service module 440 can issue a medical reminder to the patient, such as issuing a medical reminder through the patient space application of the patient's third terminal. In some embodiments of this specification, a health monitoring plan can be generated based on the target diagnosis record, and the patient can be monitored according to the health monitoring plan, and then the health monitoring plan can be updated based on the health monitoring information. On the one hand, the patient can be reminded to comply with the doctor's orders and prescriptions in the target diagnosis record, and on the other hand, the patient's health condition can be deeply understood so that timely adjustments can be made to the patient's medication, living habits, etc.

[0352] Fig.181 is a schematic diagram of an exemplary medical consultation process 1800 according to some embodiments of this specification. The hospital consultation process 1800 is used to provide on-site medical consultation services, and the patient needs to go to the hospital to communicate with the doctor face to face in the consulting room. Fig.18 As described above, the medical consultation process 1800 includes a registration step 1810 , a waiting step 1820 , and a consultation step 1830 .

[0353] In the registration link 1810 , the patient is first questioned by controlling the patient's third terminal to determine the patient's registered doctor. Specifically, the registration link 1810 can be executed by the registration module 410 .

[0354] For a detailed description of the first inquiry, the third terminal, and determining the registered doctor, please refer to Figure 5-7 and its related description.

[0355] In the waiting stage 1820 , the patient is asked a second question by controlling the patient's third terminal to generate a pre-questioning record. Specifically, the waiting stage 1820 may be executed by the waiting module 420 .

[0356] For a description of the second inquiry and the generation of pre-inquiry records, see Figure 5 and Figure 8 and its related description.

[0357] In the consultation link 1830, in response to detecting that the patient has started to see a doctor, the first terminal is used to display a third interface element to the doctor and the patient. The third interface element is related to the pre-consultation record. For more descriptions of the third interface element and the pre-consultation record, see Figure 8 , Fig. 9 , Fig.11 and its related description.

[0358] In the consultation phase 1830, an initial diagnosis record may be further generated based on the sensory information collected by one or more sensory devices in the consulting room during the consultation process. For a detailed description of the initial diagnosis record, see Fig.13 and Fig.15 and its related description.

[0359] In some embodiments of this specification, a first inquiry is initiated to the patient through the patient's third device, and a registration service is provided to the patient, thereby improving the efficiency and accuracy of the patient's registration. A second inquiry is initiated to the patient through the patient's third terminal while the patient is waiting for the consultation, and a pre-consultation service is provided to the patient, and a pre-consultation record is generated. The pre-consultation record can be displayed to the patient and the doctor in the consulting room after the consultation begins, which can improve the doctor's consultation efficiency. By generating an initial diagnosis record based on the perception information, the doctor's paperwork can be reduced and the overall work efficiency of the doctor can be improved.

[0360] Fig.19 1 is a schematic diagram of an exemplary medical consultation process 1900 according to some embodiments of this specification. The hospital consultation process 1900 is used to provide telemedicine consultation services. Patients do not need to go to the hospital and can communicate with doctors online anywhere. Fig.19 As described above, the medical consultation process 1900 includes a registration step 1910 , a waiting step 1920 , and a consultation step 1930 .

[0361] Registration link 1910 and Fig.18 The registration process is similar to 1810. The waiting process is similar to Fig.18 The waiting phase in 1820 is similar.

[0362] In the consultation link 1930, in response to detecting that the patient has started to see a doctor, the doctor's second terminal and the patient's third terminal are controlled to present a third interface element related to the pre-consultation record. For more descriptions of the third interface element and the pre-consultation record, see Figure 8 , Fig. 9 , Fig.11 and its related description.

[0363] In some embodiments, the second terminal may include a second XR device, and the third terminal may include a first XR device. The first XR device and the second XR device may present a virtual clinic space to the patient and the doctor, respectively. For a detailed description of presenting a virtual clinic space, see Fig.11 When a remote patient attends the consultation, the virtual clinic space can be synchronously presented through the remote patient's third XR device. In some embodiments of the present description, by presenting the virtual clinic space, the participants (patients, doctors, remote patients, etc.) can achieve remote communication, reducing the distance and time restrictions on patients and remote patients.

[0364] In the consultation link 1930, an initial diagnosis record may be further generated based on the sensory information collected by the second terminal and the third terminal during the consultation process. In some embodiments, the consultation module 430 may collect the patient's physical examination data based on the doctor's second XR device and the patient's wearable device. The initial diagnosis record may be generated based on the sensory information and the physical examination data. For a detailed description of generating the initial diagnosis record, see Fig.11 , Fig.15 and its related description.

[0365] Specifically, the consultation module 430 can obtain or generate a three-dimensional patient model. The three-dimensional patient model can correspond to the patient or a part of the patient (such as the upper body). As an example only, the consultation module 430 can obtain an initial three-dimensional patient model from the patient's electronic health record, and update the initial three-dimensional patient model based on the patient's real-time dynamic data and physiological data to obtain a three-dimensional patient model. Further, the consultation module 430 can display the three-dimensional patient model to the doctor through the second XR device and obtain the doctor's physical examination instructions. Specifically, the consultation module 430 can display the three-dimensional patient model on the doctor's second field of view through the second XR device. Among them, the second field of view can be a real field of view within the doctor's sight range, or it can be a virtual background. The doctor's physical examination instructions may include physical examination parts, physical examination equipment, and physical examination operations. The doctor can input physical examination instructions in a variety of ways such as voice, gestures, and operation input devices (such as smart gloves, smart handles, etc.). For example, the second XR device can present virtual physical examination devices corresponding to a variety of physical examination devices. The doctor can select a virtual physical examination device through an input device and use the virtual physical examination device to perform a virtual physical examination operation on the three-dimensional patient simulation. The consultation module 430 can determine the physical examination part, physical examination equipment, physical examination operation, etc. based on the virtual physical examination operation performed by the doctor, thereby generating a physical examination instruction. In addition to the physical examination instruction, the doctor can also input other instructions to instruct the second XR device to rotate, zoom in, zoom out, etc. on the three-dimensional patient model.

[0366] After obtaining the doctor's physical examination instruction, the medical inquiry module 430 can control the patient's wearable device to collect the patient's physical examination data. For example, the medical inquiry module 430 can control the patient's smart watch to collect the patient's pulse within one minute based on the physical examination instruction.

[0367] In some embodiments of the present specification, doctors and patients can communicate remotely in a virtual consultation space through the second XR device and the first XR device, providing doctors and patients with barrier-free and immersive consultation experience. In addition, by presenting the doctor with a three-dimensional patient model of the patient, remote physical examination can be performed more accurately and conveniently, thereby improving the accuracy of diagnosis.

[0368] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0369] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0370] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0371] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0372] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0373] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0374] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A method for providing pre-consultation services, executed by at least one processor, characterized in that: include: Based on the department of the doctor registered by the patient, a virtual character is displayed on the patient terminal of the patient to conduct a pre-diagnosis inquiry to the patient; generating the pre-consultation record based on the data collected by the patient terminal in the pre-consultation inquiry; In response to detecting that the patient has started a consultation, interface elements related to the patient's pre-consultation record are presented via a clinic terminal in the clinic, wherein the processor is configured with a pre-consultation agent corresponding to the pre-consultation service of the department, the pre-consultation inquiry is executed using the pre-consultation agent, and the pre-consultation agent stores knowledge data corresponding to the department and can achieve self-evolution using artificial intelligence technology.

2. The method according to claim 1, characterized in that The pre-diagnosis inquiry includes multiple rounds of inquiry, and the pre-diagnosis inquiry of the patient includes: For each current round of queries except the first round of queries, Based on the inquiry content of historical inquiries, the patient's historical answers and the known information of the patient, using a second inquiry content determination model to determine the inquiry content corresponding to the current round of inquiries, where the second inquiry content determination model is a trained machine learning model; Based on the inquiry content corresponding to the current round of inquiry, the current round of inquiry is performed through the patient terminal.

3. The method according to claim 1, characterized in that The pre-inquiry is performed based on the pre-inquiry content, and the pre-inquiry content is determined by: Obtain the pre-consultation record template corresponding to the doctor's department and the known information of the patient; By comparing the pre-consultation record template with the known information of the patient, determining the missing information that has not been collected in the pre-consultation record template; The pre-diagnosis inquiry content is determined based on the missing information.

4. The method according to claim 1, characterized in that The pre-inquiry is conducted based on the pre-inquiry content, and the pre-inquiry content is determined by the following methods, including: Using the missing information determination model to process the department where the doctor is located and the known information of the patient to determine the missing information to be collected from the patient; The missing information is processed using a first inquiry content determination model to determine the pre-diagnosis inquiry content, wherein the missing information determination model and the first inquiry content determination model are trained machine learning models.

5. The method according to claim 3 or 4, characterized in that The pre-diagnosis inquiry includes multiple rounds of inquiry, and the pre-diagnosis inquiry content includes the inquiry content of each round of inquiry. The pre-diagnosis inquiry of the patient includes: For each current round of inquiry except the first round of inquiry, Determining semantic information and emotional information of the patient's historical answers based on data collected before the current round of questioning; Adjusting the inquiry content of the current round of inquiry based on the semantic information and the emotional information; Based on the adjusted inquiry content of the current round of inquiry, the current round of inquiry is performed through the patient terminal.

6. The method according to claim 5, characterized in that The step of performing the current round of inquiry based on the adjusted inquiry content of the current round of inquiry through the patient terminal includes: Determining the sound characteristics of the current round of inquiry based on the semantic information and the emotional information; Based on the sound feature and the adjusted inquiry content of the current round of inquiry, the patient terminal of the patient is controlled to perform the current round of inquiry on the patient terminal.

7. The method according to claim 5, characterized in that The adjusting the inquiry content of the current round of inquiry based on the semantic information and the emotional information includes: Acquiring physiological status information corresponding to the patient's inquiry in the current round, wherein the physiological status information is collected using a wearable device worn by the patient; Based on the semantic information, the emotional information and the physiological state information, the inquiry content of the current round of inquiry is adjusted.

8. The method according to claim 7, characterized in that The method further comprises: determining a feedback parameter based on at least a portion of the semantic information, the emotional information, and the physiological state information; The wearable device is controlled to apply feedback to the patient based on the feedback parameter, and the feedback includes at least one of force feedback and temperature feedback.

9. The method according to claim 1, characterized in that The patient terminal displays the virtual character through extended reality technology.

10. The method according to claim 1, characterized in that The appearance characteristics of the virtual character are determined based on the optical image of the doctor and / or the basic information of the patient.

11. The method according to claim 1, characterized in that The controlling the patient terminal of the patient to conduct a pre-diagnosis inquiry on the patient includes: After the patient completes the registration, the estimated waiting time of the patient is determined based on the doctor's consultation record of the day and the patient's registration record; In response to the estimated waiting time being greater than a first preset time threshold and less than a second preset time threshold, the pre-consultation inquiry is performed on the patient via the patient terminal, and the second preset time threshold is greater than the first preset time threshold.

Images